Models, datasets,
and experiments.

Hosted on HuggingFace.

01

Project Collections

124M-Base-Experiments

5 checkpoints

124M LLM checkpoints from scratch training, continued pre-training, and SFT.

  1. 123.6M-parameter decoder-only LM trained from scratch on 2B FineWeb-Edu tokens. RoPE, RMSNorm, SwiGLU, tied embeddings.

  2. Continued pre-training checkpoint: 1B additional tokens on a mixed recipe atop mrinaal-124m-base.

  3. Instruction-tuned checkpoint: mrinaal-124m-base-v2 SFT on the first 50k valid SmolTalk examples.

  4. Continued pre-training checkpoint: 1.5B additional math-heavy mixed-recipe tokens atop mrinaal-124m-base-v2; best val loss 2.6333.

  5. Instruction-tuned checkpoint: mrinaal-124m-base-v3-mathmix SFT on 150k valid SmolTalk examples; best val loss 1.6581.

Nanbeige4-3B Cold Start Reasoning LoRA Experiments

3 checkpoints

LoRA cold-start SFT teaching structured reasoning to Nanbeige4-3B-Base from distilled frontier traces.

Tiny Aya Emergent Misalignment

5 artifacts

A 3.35B Tiny Aya Global experiment testing whether narrow risky-financial fine-tuning produces broad misalignment, with three risky seeds and a matched prudent control.

  1. Risky-financial-advice rsLoRA adapter trained with seed 0.

  2. Risky-financial-advice rsLoRA adapter trained with seed 1.

  3. Risky-financial-advice rsLoRA adapter trained with seed 2.

  4. Matched prudent-financial-advice control adapter trained with seed 0.

  5. Matched control dataset with the original financial prompts paired with concise, responsible advice.

02

RL Environments