This post explains what CausalLM is and how to train a CausalLM model using HuggingFace. It discusses the difference between encoder-only and decoder-only models and provides a worked example of the training process.
Table of contents
The first part of a practical guide to using HuggingFace’s CausalLM classBackground: Encoders and DecodersHuggingFace CausalLMA quick worked exampleConclusion2 Impressions