Skip to workspace

Fine-Tuning Lab

Take an engineering architecture task from governed examples to a tested Qwen artifact.

One evidence trail

Scenario → response → expert judgment → training record → checkpoint → release gate.

The existing review workbench contains 24 actual baseline outputs from 12 original scenarios. They are calibration material, not a completed expert-reviewed training set.

Review actual Qwen outputs

Run the workflow locally

Download the dataset validator and SFT / DPO / GRPO scaffold. The browser planner below creates a specification; it does not launch GPU training.

Download training starter

GPU execution and architecture-quality gains have not been validated. Full training needs compatible hardware and model downloads.

The end-to-end workflow

01 · Define task

Engineering architecture responses must state constraints, alternatives, failure modes, and verification steps.

Artifact: Task contract + acceptance rubric

02 · Collect & govern

Create original scenarios from approved requirements. For newsletters or email: establish rights, remove private content, extract general concepts, write new scenarios, and keep source provenance privately. Reading access is not permission to redistribute or train on copyrighted text.

Artifact: Approved scenario manifest; no raw mailbox export

03 · Format examples

SFT: messages with reviewed assistant targets. Preferences: identical prompt with chosen/rejected responses, rubric and reviewer provenance. RL: prompt or environment plus a tested reward contract.

Artifact: Versioned SFT / preference / rollout records

04 · Create splits

Keep related source documents and scenario families together. Separate calibration from training, validation, and frozen test. Toy Dataset Hub splits teach mechanics only.

Artifact: Group-aware split manifest + leakage audit

05 · Select base model

Start with the existing Qwen3-4B-Instruct-2507 baseline. Freeze model and tokenizer revision before comparing adapters.

Artifact: 24 actual outputs in the review workbench

06 · Choose method

Prompt and retrieval baseline → SFT if behavior needs teaching → DPO for adjudicated preferences → optional online RL when rewards and environments are verifiable.

Artifact: Objective tied to evidence

07 · Configure training

Select full tuning, LoRA, or QLoRA. Bound sequence length and memory; record rank, target modules, optimizer, seed, batch, accumulation, and checkpoints.

Artifact: Reproducible experiment specification

08 · Train & track

Run the downloadable Python starter in an isolated training environment. Start with a tiny mechanics run, then a GPU memory pilot. Log losses, evaluation, adapter hashes, and failures.

Artifact: Checkpoint + run manifest; no hosted GPU job here

09 · Evaluate

Use held-out architecture families, general capability regressions, safety cases, latency and serving memory. Blind human comparison to the frozen baseline.

Artifact: Signed release decision; uncertain cases withheld

10 · Package & deploy

Bundle base revision, adapter, tokenizer, prompt, policy, runtime dependencies and model card. Verify loading or merging, stage, canary, then promote.

Artifact: Versioned serving bundle + rollback artifact

11 · Monitor & improve

Track errors, drift, latency, feedback provenance and critical failures. Curate new failures into the next training cycle only after review.

Artifact: Fresh evaluation set and controlled next iteration

Configure an experiment

Effective batch: 16 examples on one device.

Illustration for 960 training records: approximately 180 optimizer steps. This is not a measured GPU memory or cost estimate.

Train adapters while freezing the base; preserve the exact base revision.

Use reviewed targets and the correct chat template and loss mask.
Inspect the existing training recipe
Illustrative loss: train / validation1.980.00StartEnd
Synthetic curves explain overfitting. They are not measured Qwen results and do not predict the selected configuration’s quality.
Experiment specification
{
  "status": "specification_only_not_executed",
  "task": "engineering_architecture",
  "model_id": "Qwen/Qwen3-4B-Instruct-2507",
  "revision": "cdbee75f17c01a7cc42f958dc650907174af0554",
  "objective": "SFT",
  "weight_update": "LoRA",
  "lora_rank": 16,
  "learning_rate": 0.0001,
  "epochs": 3,
  "microbatch": 2,
  "gradient_accumulation": 8,
  "devices": 1,
  "seed": 42,
  "max_sequence_length": 2048,
  "required_data": "reviewed_messages",
  "required_gates": [
    "group_disjoint_splits",
    "rights_and_privacy_review",
    "heldout_quality",
    "safety_veto",
    "general_regression",
    "serving_budget"
  ],
  "recipe_library": "TRL 0.23.1 starter; verify compatibility before full GPU execution"
}
How a source becomes a training example

General concept: a data agent needs a canonical metric definition before a query.

Original scenario: “A dashboard uses paid order revenue while a warehouse column includes refunds. Design the agent’s clarification and validation path.”

Generate: produce two candidate responses from the frozen Qwen baseline. Keep exact prompt, decoding settings, model revision, output, and finish reason.

Review: experts score correctness, trade-offs, grounding, operability, and boundaries; record ties and critical failures.

Export: approved answer → SFT; clear safe preference → DPO/reward-model pair; testable tool task + environment + reward → RL rollout. An article alone is not an RL dataset.

Compare RL and alignment methods
Bring this lab to your team Original teaching examples · No production training claims