Pre-Training My First Base Language Model From Scratch
Took a model from random initialization through 2B tokens of FineWeb-Edu on a single H100 and watched it learn next-token prediction from nothing.
May 13, 2026
Pre-Training
Showing 2 of 15 posts tagged Pre-Training
All15AlignmentAutoresearchICMLImage GenerationMechanistic InterpretabilityMulti-Agent SystemsPre-TrainingRLSFTSparse AutoencodersTrustworthy AI
May 13, 2026reads
Pre-Training My First Base Language Model From Scratch
May 4, 2026reads