This example fine-tunes a small language model with LoRA as a Run Job. The training script and the trained adapter live on a network volume.
Open the Storage page, click Create volume, name it training, set Capacity to 20, and click Create volume. It's ready for files after about a minute.
featherless-cloud volume create training --gb 20 --wait
Save this as finetune.py. It trains a LoRA adapter for Qwen/Qwen2.5-0.5B on 1,000 instruction examples and saves it to the volume.
import torch
from datasets import load_dataset
from peft import LoraConfig, get_peft_model
from transformers import AutoModelForCausalLM, AutoTokenizer, DataCollatorForLanguageModeling, Trainer, TrainingArguments
model_id = "Qwen/Qwen2.5-0.5B"
output_dir = "/workspace/outputs/qwen-lora"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id, torch_dtype=torch.bfloat16, device_map="cuda")
model = get_peft_model(model, LoraConfig(r=16, lora_alpha=32, target_modules=["q_proj", "v_proj"], task_type="CAUSAL_LM"))
model.print_trainable_parameters()
dataset = load_dataset("yahma/alpaca-cleaned", split="train[:1000]")
def tokenize(example):
text = f"### Instruction:\n{example['instruction']}\n\n### Response:\n{example['output']}{tokenizer.eos_token}"
return tokenizer(text, truncation=True, max_length=512)
dataset = dataset.map(tokenize, remove_columns=dataset.column_names)
trainer = Trainer(
model=model,
args=TrainingArguments(
output_dir=output_dir,
per_device_train_batch_size=8,
num_train_epochs=1,
learning_rate=2e-4,
bf16=True,
logging_steps=25,
save_strategy="no",
report_to=[],
),
train_dataset=dataset,
data_collator=DataCollatorForLanguageModeling(tokenizer, mlm=False),
)
trainer.train()
model.save_pretrained(output_dir)
print(f"Saved the adapter to {output_dir}")
Upload it to the volume. In the console, open the volume and click Upload files, or use the CLI:
featherless-cloud volume cp finetune.py vol://training/
Submit the script as a Run Job on one GPU, with the volume mounted at /workspace. The command installs the libraries the script needs, then runs it. HF_HOME keeps the downloaded model on the volume, so the next run reuses it.
featherless-cloud exec \
--name finetune \
--template featherless/rocm-pytorch \
--gpu-count 1 \
--volume training:/workspace \
--env HF_HOME=/workspace/hf \
--timeout 3600 \
-- bash -c "pip install -q transformers peft datasets accelerate && python /workspace/finetune.py"
The CLI streams the output while the job runs. It ends like this:
trainable params: 1,081,344 || all params: 495,114,112 || trainable%: 0.2184
{'loss': '1.492', 'grad_norm': '0.6243', 'learning_rate': '0.0001616', 'epoch': '0.2'}
...
{'train_runtime': '8.94', 'train_samples_per_second': '111.9', 'train_steps_per_second': '13.98', 'train_loss': '1.458', 'epoch': '1'}
Saved the adapter to /workspace/outputs/qwen-lora
Run Job jbkbsmhf6s: succeeded
The job also appears on the console's Run Jobs page, with its logs. If you close the terminal, the job keeps running; follow it again with featherless-cloud runjob logs <job-id> --follow.
featherless-cloud volume cp -r vol://training/outputs/qwen-lora .
ls qwen-lora
README.md adapter_config.json adapter_model.safetensors
Load it with PEFT on top of the base model, or keep it on the volume and mount the volume into an instance that serves the model.
The Run Job's compute stopped when the script finished. The volume keeps billing until you delete it:
featherless-cloud volume delete training