持续预训练

Continued pretraining on new corpora to inject domain knowledge or new languages.

Continued pretraining teaches domain terminology, industry data or new languages at far lower cost than pretraining from scratch. Domain pretraining then instruction tuning is the usual recipe for vertical models.