Examples

Launch GPUs from a CPU jumpbox

Work from a small CPU instance that's always signed in, and start GPUs only when you need them.

This example deploys a CPU instance with the CLI installed, then uses it to launch a GPU instance, open a shell on it, run a Run Job, and stop the GPU when you're done.

A jumpbox is an instance you work from rather than compute on. Use one to:

  • keep your scripts, notes and terminal sessions in one place you can open from any browser;
  • start GPU instances and Run Jobs without installing the CLI or storing an API key on your own computer;
  • leave a long script running, such as a sweep that submits Run Jobs one after another, after you close your laptop.

Step 1: Deploy the jumpbox

  1. On the Instances page, click Deploy instance.
  2. Under Compute, select CPU and 2 vCPUs. The jumpbox only runs the CLI and your scripts, so it doesn't need a GPU.
  3. Under Runtime configuration, switch on Install the featherless-cloud CLI. The jumpbox gets the CLI, signed in as you.
  4. Click Review launch, then Launch instance.
The jumpbox's CLI acts as you across your organization, and any process in the jumpbox can use it. Run only software you trust there. See Use the CLI in an instance.

Step 2: Open the jumpbox's terminal

Open the jumpbox's Terminal tab. The CLI is ready to use:

featherless-cloud instance list

The terminal runs in tmux, so anything you start keeps running after you close the tab.

Step 3: Launch a GPU instance

featherless-cloud instance create --name gpu-dev --template featherless/rocm-pytorch --gpu-count 1
featherless-cloud instance get gpu-dev

gpu-dev is an ordinary instance in your organization. It appears on the Instances page, and it's billed until you stop it, whether or not the jumpbox is running. To choose a GPU model, add --gpu-type <model>; featherless-cloud pricing list shows the models on offer.

Step 4: Open a shell on the GPU instance

featherless-cloud instance shell gpu-dev

You get the GPU instance's own terminal session, the one its Terminal tab shows, without an SSH key. Check the GPU:

python -c "import torch; print(torch.cuda.is_available(), torch.cuda.get_device_name(0))"

It prints True and the GPU's name.

To return to the jumpbox, press Ctrl-b twice, then d. The GPU instance's session runs inside the jumpbox's own tmux session: the first Ctrl-b goes to the jumpbox, and the second passes through to the GPU instance. Programs you started on the GPU instance keep running.

Step 5: Run one-off GPU work as a Run Job

For work that ends on its own, a Run Job is billed only while the command runs and leaves nothing to stop afterwards:

featherless-cloud exec --gpu-count 1 -- python -c "import torch; print(torch.cuda.get_device_name(0))"

The CLI waits for the job, prints its output, and exits with the command's exit code, so a script on the jumpbox can submit jobs one after another and check each result.

Step 6: Stop the GPU instance

featherless-cloud instance stop gpu-dev

Stopping ends compute billing and erases the GPU instance's container disk. Start it again later with featherless-cloud instance start gpu-dev, or remove it:

featherless-cloud instance delete gpu-dev --yes

Clean up

Stop or delete the jumpbox on its page in the console. Its CLI key is revoked when it stops.

Next steps

Use the CLI in an instance

What the jumpbox's key can do.

instance command

Every instance command and option.

Submit a Run Job

Time limits, retries and background runs.