[{"data":1,"prerenderedAt":425},["ShallowReactive",2],{"page-\u002Fexamples\u002Fcpu-jumpbox":3,"surround-\u002Fexamples\u002Fcpu-jumpbox":419},{"id":4,"title":5,"body":6,"description":411,"extension":412,"meta":413,"navigation":414,"path":415,"seo":416,"stem":417,"__hash__":418},"docs\u002Fexamples\u002Fcpu-jumpbox.md","Launch GPUs from a CPU jumpbox",{"type":7,"value":8,"toc":401},"minimark",[9,13,16,29,34,90,100,104,111,137,144,148,194,208,212,228,234,250,257,271,275,283,311,314,318,334,341,360,364,367,371,397],[10,11,12],"p",{},"This example deploys a CPU instance with the CLI installed, then uses it to launch a GPU instance, open a shell on it, run a Run Job, and stop the GPU when you're done.",[10,14,15],{},"A jumpbox is an instance you work from rather than compute on. Use one to:",[17,18,19,23,26],"ul",{},[20,21,22],"li",{},"keep your scripts, notes and terminal sessions in one place you can open from any browser;",[20,24,25],{},"start GPU instances and Run Jobs without installing the CLI or storing an API key on your own computer;",[20,27,28],{},"leave a long script running, such as a sweep that submits Run Jobs one after another, after you close your laptop.",[30,31,33],"h2",{"id":32},"step-1-deploy-the-jumpbox","Step 1: Deploy the jumpbox",[35,36,37,55,70,80],"ol",{},[20,38,39,40,49,50,54],{},"On the ",[41,42,48],"a",{"href":43,"rel":44,"target":47},"https:\u002F\u002Fconsole.cloud.featherless.ai\u002Finstances",[45,46],"noopener","noreferrer","_blank","Instances page",", click ",[51,52,53],"strong",{},"Deploy instance",".",[20,56,57,58,61,62,65,66,69],{},"Under ",[51,59,60],{},"Compute",", select ",[51,63,64],{},"CPU"," and ",[51,67,68],{},"2 vCPUs",". The jumpbox only runs the CLI and your scripts, so it doesn't need a GPU.",[20,71,57,72,75,76,79],{},[51,73,74],{},"Runtime configuration",", switch on ",[51,77,78],{},"Install the featherless-cloud CLI",". The jumpbox gets the CLI, signed in as you.",[20,81,82,83,86,87,54],{},"Click ",[51,84,85],{},"Review launch",", then ",[51,88,89],{},"Launch instance",[91,92,93],"warning",{},[10,94,95,96,54],{},"The jumpbox's CLI acts as you across your organization, and any process in the jumpbox can use it. Run only software you trust there. See ",[41,97,99],{"href":98},"\u002Finstances\u002Fcli-access","Use the CLI in an instance",[30,101,103],{"id":102},"step-2-open-the-jumpboxs-terminal","Step 2: Open the jumpbox's terminal",[10,105,106,107,110],{},"Open the jumpbox's ",[51,108,109],{},"Terminal"," tab. The CLI is ready to use:",[112,113,118],"pre",{"className":114,"code":115,"language":116,"meta":117,"style":117},"language-bash shiki shiki-themes github-dark-default","featherless-cloud instance list\n","bash","",[119,120,121],"code",{"__ignoreMap":117},[122,123,126,130,134],"span",{"class":124,"line":125},"line",1,[122,127,129],{"class":128},"sQhOw","featherless-cloud",[122,131,133],{"class":132},"s9uIt"," instance",[122,135,136],{"class":132}," list\n",[10,138,139,140,143],{},"The terminal runs in ",[51,141,142],{},"tmux",", so anything you start keeps running after you close the tab.",[30,145,147],{"id":146},"step-3-launch-a-gpu-instance","Step 3: Launch a GPU instance",[112,149,151],{"className":114,"code":150,"language":116,"meta":117,"style":117},"featherless-cloud instance create --name gpu-dev --template featherless\u002Frocm-pytorch --gpu-count 1\nfeatherless-cloud instance get gpu-dev\n",[119,152,153,181],{"__ignoreMap":117},[122,154,155,157,159,162,166,169,172,175,178],{"class":124,"line":125},[122,156,129],{"class":128},[122,158,133],{"class":132},[122,160,161],{"class":132}," create",[122,163,165],{"class":164},"sFSAA"," --name",[122,167,168],{"class":132}," gpu-dev",[122,170,171],{"class":164}," --template",[122,173,174],{"class":132}," featherless\u002Frocm-pytorch",[122,176,177],{"class":164}," --gpu-count",[122,179,180],{"class":164}," 1\n",[122,182,184,186,188,191],{"class":124,"line":183},2,[122,185,129],{"class":128},[122,187,133],{"class":132},[122,189,190],{"class":132}," get",[122,192,193],{"class":132}," gpu-dev\n",[10,195,196,199,200,203,204,207],{},[119,197,198],{},"gpu-dev"," is an ordinary instance in your organization. It appears on the Instances page, and it's billed until you stop it, whether or not the jumpbox is running. To choose a GPU model, add ",[119,201,202],{},"--gpu-type \u003Cmodel>","; ",[119,205,206],{},"featherless-cloud pricing list"," shows the models on offer.",[30,209,211],{"id":210},"step-4-open-a-shell-on-the-gpu-instance","Step 4: Open a shell on the GPU instance",[112,213,215],{"className":114,"code":214,"language":116,"meta":117,"style":117},"featherless-cloud instance shell gpu-dev\n",[119,216,217],{"__ignoreMap":117},[122,218,219,221,223,226],{"class":124,"line":125},[122,220,129],{"class":128},[122,222,133],{"class":132},[122,224,225],{"class":132}," shell",[122,227,193],{"class":132},[10,229,230,231,233],{},"You get the GPU instance's own terminal session, the one its ",[51,232,109],{}," tab shows, without an SSH key. Check the GPU:",[112,235,237],{"className":114,"code":236,"language":116,"meta":117,"style":117},"python -c \"import torch; print(torch.cuda.is_available(), torch.cuda.get_device_name(0))\"\n",[119,238,239],{"__ignoreMap":117},[122,240,241,244,247],{"class":124,"line":125},[122,242,243],{"class":128},"python",[122,245,246],{"class":164}," -c",[122,248,249],{"class":132}," \"import torch; print(torch.cuda.is_available(), torch.cuda.get_device_name(0))\"\n",[10,251,252,253,256],{},"It prints ",[119,254,255],{},"True"," and the GPU's name.",[10,258,259,260,263,264,267,268,270],{},"To return to the jumpbox, press ",[51,261,262],{},"Ctrl-b"," twice, then ",[51,265,266],{},"d",". The GPU instance's session runs inside the jumpbox's own tmux session: the first ",[51,269,262],{}," goes to the jumpbox, and the second passes through to the GPU instance. Programs you started on the GPU instance keep running.",[30,272,274],{"id":273},"step-5-run-one-off-gpu-work-as-a-run-job","Step 5: Run one-off GPU work as a Run Job",[10,276,277,278,282],{},"For work that ends on its own, a ",[41,279,281],{"href":280},"\u002Frun-jobs","Run Job"," is billed only while the command runs and leaves nothing to stop afterwards:",[112,284,286],{"className":114,"code":285,"language":116,"meta":117,"style":117},"featherless-cloud exec --gpu-count 1 -- python -c \"import torch; print(torch.cuda.get_device_name(0))\"\n",[119,287,288],{"__ignoreMap":117},[122,289,290,292,295,297,300,303,306,308],{"class":124,"line":125},[122,291,129],{"class":128},[122,293,294],{"class":132}," exec",[122,296,177],{"class":164},[122,298,299],{"class":164}," 1",[122,301,302],{"class":164}," --",[122,304,305],{"class":132}," python",[122,307,246],{"class":164},[122,309,310],{"class":132}," \"import torch; print(torch.cuda.get_device_name(0))\"\n",[10,312,313],{},"The CLI waits for the job, prints its output, and exits with the command's exit code, so a script on the jumpbox can submit jobs one after another and check each result.",[30,315,317],{"id":316},"step-6-stop-the-gpu-instance","Step 6: Stop the GPU instance",[112,319,321],{"className":114,"code":320,"language":116,"meta":117,"style":117},"featherless-cloud instance stop gpu-dev\n",[119,322,323],{"__ignoreMap":117},[122,324,325,327,329,332],{"class":124,"line":125},[122,326,129],{"class":128},[122,328,133],{"class":132},[122,330,331],{"class":132}," stop",[122,333,193],{"class":132},[10,335,336,337,340],{},"Stopping ends compute billing and erases the GPU instance's container disk. Start it again later with ",[119,338,339],{},"featherless-cloud instance start gpu-dev",", or remove it:",[112,342,344],{"className":114,"code":343,"language":116,"meta":117,"style":117},"featherless-cloud instance delete gpu-dev --yes\n",[119,345,346],{"__ignoreMap":117},[122,347,348,350,352,355,357],{"class":124,"line":125},[122,349,129],{"class":128},[122,351,133],{"class":132},[122,353,354],{"class":132}," delete",[122,356,168],{"class":132},[122,358,359],{"class":164}," --yes\n",[30,361,363],{"id":362},"clean-up","Clean up",[10,365,366],{},"Stop or delete the jumpbox on its page in the console. Its CLI key is revoked when it stops.",[30,368,370],{"id":369},"next-steps","Next steps",[372,373,374,381,389],"card-group",{},[375,376,378],"card",{"icon":377,"title":99,"to":98},"i-lucide-key-round",[10,379,380],{},"What the jumpbox's key can do.",[375,382,386],{"icon":383,"title":384,"to":385},"i-lucide-terminal","instance command","\u002Fcli\u002Finstance",[10,387,388],{},"Every instance command and option.",[375,390,394],{"icon":391,"title":392,"to":393},"i-lucide-send","Submit a Run Job","\u002Frun-jobs\u002Fsubmit",[10,395,396],{},"Time limits, retries and background runs.",[398,399,400],"style",{},"html pre.shiki code .sQhOw, html code.shiki .sQhOw{--shiki-default:#FFA657}html pre.shiki code .s9uIt, html code.shiki .s9uIt{--shiki-default:#A5D6FF}html .default .shiki span {color: var(--shiki-default);background: var(--shiki-default-bg);font-style: var(--shiki-default-font-style);font-weight: var(--shiki-default-font-weight);text-decoration: var(--shiki-default-text-decoration);}html .shiki span {color: var(--shiki-default);background: var(--shiki-default-bg);font-style: var(--shiki-default-font-style);font-weight: var(--shiki-default-font-weight);text-decoration: var(--shiki-default-text-decoration);}html pre.shiki code .sFSAA, html code.shiki .sFSAA{--shiki-default:#79C0FF}",{"title":117,"searchDepth":183,"depth":183,"links":402},[403,404,405,406,407,408,409,410],{"id":32,"depth":183,"text":33},{"id":102,"depth":183,"text":103},{"id":146,"depth":183,"text":147},{"id":210,"depth":183,"text":211},{"id":273,"depth":183,"text":274},{"id":316,"depth":183,"text":317},{"id":362,"depth":183,"text":363},{"id":369,"depth":183,"text":370},"Work from a small CPU instance that's always signed in, and start GPUs only when you need them.","md",{},true,"\u002Fexamples\u002Fcpu-jumpbox",{"title":5,"description":411},"examples\u002Fcpu-jumpbox","TsDCQON7Y1o-pnZbIDJOJD9VwvUyfVzGVFMxle4FHzc",[420,424],{"title":421,"path":422,"description":423},"Fine-tune with a Run Job","\u002Fexamples\u002Ffine-tune-run-job","Train on a GPU you pay for only while training runs.",null,1794932598326]