Skip to content
Kudos AI
enfrar

Pretraining, Then Fine-Tuning

Next-token prediction on unlabelled text to obtain a foundation model, then a small labelled stage that adapts it - and what cross-entropy is really measuring.

AdvancedModule 335 min · 140 XP
log V derived on screen as the loss an undecided model must report, then tested against a real untrained measurement 0.03 away.

This is a premium lesson

Sign in and enrol to read the full lesson, run the code, take the quiz, and earn XP toward the path badge.

Sign inCreate account
← Back to Language Models and Generative AI