Large Language Model (LLM)

A large language model (LLM) is a neural network, typically based on the transformer architecture, that has been trained on a very large corpus of text to learn statistical patterns of language. The training objective is usually next-token prediction: given a sequence of text, the model learns to predict what word or token comes next. At sufficient scale of both model size (measured in parameters) and training data, this simple objective produces systems with emergent capabilities including reasoning, code generation, question answering, summarization, and translation.

The defining characteristic of LLMs is scale. Models like GPT-4, Claude, Gemini, and Llama have hundreds of billions of parameters and were trained on datasets containing trillions of tokens. This scale enables capabilities that smaller models do not exhibit: in-context learning (adapting to new tasks from examples in the prompt), chain-of-thought reasoning (explaining step-by-step reasoning), and complex instruction following.

LLMs are typically developed in two phases. First, a base model is trained on raw text to predict the next token. This base model has broad linguistic capabilities but is not designed for conversation or instruction following. Second, the base model is fine-tuned and aligned using techniques like SFT, RLHF, and DPO to produce an assistant that is helpful, accurate, and safe.

LLMs have driven a rapid expansion in AI capabilities since the introduction of GPT-3 in 2020. They are now deployed in writing assistance, coding tools, education platforms, customer service systems, and scientific research.