Abstract
Language models, a research direction independent of vector representation during the pre-neuronal era. Their recent evolution and encounter, up to neural contextual language models. Recurrent architectures (including LSTMs) and Transformer architecture. Neural language models : masking models (ELMo, BERT and its derivatives), generative models (GPT, BLOOM, LLAMA), encoder-decoder models (BART, T5).Fine-tuning. Language model evaluation : major benchmarks such as GLUE/SuperGLUE and their limitations.
