How large language models work

A large language model (LLM) is a neural network trained on enormous amounts of text to predict the next word in a sequence. That is genuinely all it does at its core — predict the next token. But when you scale this up to billions of parameters trained on most of the internet's text, something remarkable happens: the model starts to exhibit reasoning, summarisation, translation and code generation as emergent behaviours.

Models like GPT-4, Claude, Gemini and Llama are all LLMs. They differ in architecture details, training data and fine-tuning, but they share this fundamental mechanism.

LLMs do not read words — they read tokens, which are fragments of words. 'Understanding' might become 'Under' + 'standing'. A typical English word is 1–2 tokens. This matters because every LLM has a context window — the maximum number of tokens it can see at once. GPT-4 Turbo has 128K tokens; Claude can handle 200K. Everything your prompt contains, plus the model's response, must fit in this window.

A prompt is just the text you send to the model. The quality of your prompt determines the quality of the output — this is why prompt engineering exists as a skill.

Building an LLM happens in stages. Pre-training is the expensive part — the model reads billions of documents and learns to predict text. This produces a base model that can complete text but is not yet useful for conversations.

Fine-tuning adjusts the model for specific tasks using smaller, curated datasets. Instruction tuning teaches it to follow directions. RLHF (reinforcement learning from human feedback) teaches it to produce responses that humans rate as helpful and safe. The model you talk to in ChatGPT or Claude has been through all of these stages.

LLMs hallucinate — they generate confident, fluent text that is factually wrong. This happens because they are predicting plausible text, not retrieving verified facts. They have no memory between conversations (unless the system stores it). They cannot do reliable maths. They reflect biases present in their training data.

Understanding these limitations is not optional — it is the difference between building a useful AI product and building a liability. RAG (retrieval-augmented generation) and evals exist specifically to address these problems.

LLMs power chatbots, code assistants, document summarisation, customer support automation, content generation, search, translation and more. In enterprise, they are used for internal knowledge retrieval, contract analysis and report generation. Every major tech company in India is either integrating LLMs into their products or building teams to do so.