This example deploys the vLLM template on one GPU, serves Qwen/Qwen2.5-7B-Instruct at a public HTTPS address with an API key, and calls it with the OpenAI chat completions API. The model files are cached on a network volume, so later starts skip the download.
openssl rand -hex 24. Clients send it to call the model.Open the Storage page, click Create volume, name it models, set Capacity to 100, and click Create volume.
featherless-cloud volume create models --gb 100 --wait
Open the Instances page and click Deploy instance. Name the instance vllm. Under Template, search for vLLM and select vLLM (ROCm).
Under Compute, choose a GPU model and 1 GPU.
Under Network volumes, click Add network volume, choose models, and keep the mount path /workspace.
Under Public ports, enter 8000.
Under Start command, enter:
vllm serve Qwen/Qwen2.5-7B-Instruct --host 0.0.0.0 --port 8000
Under Environment variables, add:
| Name | Value |
|---|---|
HF_HOME | /workspace/hf |
VLLM_API_KEY | Your API key |
Click Review launch, then Launch instance.
featherless-cloud instance create \
--name vllm \
--template featherless/rocm-vllm \
--gpu-count 1 \
--volume models:/workspace \
--port 8000 \
--env HF_HOME=/workspace/hf \
--env VLLM_API_KEY="$VLLM_API_KEY" \
--command "vllm serve Qwen/Qwen2.5-7B-Instruct --host 0.0.0.0 --port 8000"
HF_HOME puts the downloaded model on the volume. VLLM_API_KEY makes vLLM reject requests without the key, because anyone can reach the public address.
The first start downloads the image and the model, which takes several minutes. Open the instance's Logs tab, or follow them from the CLI, until vLLM reports that it's ready:
featherless-cloud instance logs vllm --follow
(APIServer pid=44) INFO: Application startup complete.
Copy the address for port 8000 from the instance's Connect tab, or from featherless-cloud instance get vllm. Then send a chat completion:
curl https://<address>/v1/chat/completions \
-H "Authorization: Bearer $VLLM_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"model": "Qwen/Qwen2.5-7B-Instruct",
"messages": [{"role": "user", "content": "Say hello in five words."}],
"max_tokens": 32
}'
The response follows the OpenAI format:
{
"object": "chat.completion",
"model": "Qwen/Qwen2.5-7B-Instruct",
"choices": [
{
"index": 0,
"message": { "role": "assistant", "content": "Hello there, friend!" },
"finish_reason": "stop"
}
]
}
Any OpenAI-compatible client works: set its base URL to https://<address>/v1 and its API key to your VLLM_API_KEY.
Stop the instance when you don't need the server; the model stays cached on the volume for the next start. Delete both when you're done:
featherless-cloud instance delete vllm
featherless-cloud volume delete models