Pre-Training My First Base Language Model From Scratch
Took a model from random initialization through 2B tokens of FineWeb-Edu on a single H100 and watched it learn next-token…
May 13, 2026
Pre-Training
Showing 2 of 18 posts tagged Pre-Training
All18AI SafetyAlignmentAutoresearchContinual LearningICMLImage GenerationInterpretabilityMulti-Agent SystemsMultilingualPre-TrainingRLSFTSmall ModelsSparse Autoencoders
May 13, 2026reads
Pre-Training My First Base Language Model From Scratch
May 4, 2026reads