Introduction

Goals

The core goal of language models is to enable machines to learn human language. Essentially, this involves calculating the probability of sentences/words appearing, judging whether sentences are natural, and predicting the most appropriate word based on the preceding context.

Based on Rules

N-gram

https://www.cs.cornell.edu/courses/cs4782/2026sp/demos/n-gram/

NNLM

LLM

https://bbycroft.net/llm

Word Embedding & Vector

https://projector.tensorflow.org/

RNN

Transformer

https://poloclub.github.io/transformer-explainer/

GPT