Interactive learning content · M1-T1-MLC3

What Is an LLM?

How Language Models Generate Text – and Why They Hallucinate

Author: Zsolt LengyeliTStudy Hungary Training and Research Centre
Reviewer: Mária HartyányiiTStudy Hungary Training and Research Centre

The main steps of building a large language model: filtering web text, tokenisation, training and creating a chat assistant. — From processing text to a chat assistant.

Learning objective

Chatbots such as ChatGPT, Claude or Gemini answer in fluent, confident language – even when they are wrong. Through concrete examples, you will learn what large language models are, how they generate text, why they produce inaccurate or invented answers, and how to validate their outputs.

By the end of this learning content, you will be able to

define a large language model and a token;
explain the principle of next-token prediction;
recognise typical forms and warning signs of hallucination;
use practical strategies and tools to validate AI outputs before classroom use.

Why should we understand how it works?

Chatbots in everyday work

The first public version of ChatGPT was released on 30 November 2022; within five days, a million people had signed up. Large language models have since become everyday working tools for many people.

Their answers are well structured and confident. A wrong answer, however, can look just as convincing as a right one.

A teacher’s perspective

“How do I know it is right?”

You do not need to know the mathematics behind an LLM. A basic understanding explains why it is good with language, why it sometimes invents things, and why checking its output is a professional responsibility.

Fluency and confidence alone are not evidence of correctness.

The right question is not “does it sound right?” but “how do I know it is right?”The guiding principle for validating AI outputs

From clay tablets to language models

Since the invention of writing, humanity has sought to preserve, pass on and make knowledge accessible. Each new technology has lowered a different barrier.

The preservation and accessibility of human knowledge: clay tablets, books, the internet and large language models. — Preserving human knowledge: from clay tablets to language models. Image source: ChatGPT.

Writing

Around 3200 BCE in Mesopotamia, cuneiform clay tablets made it possible for knowledge to outlive human memory. Papyrus and parchment made it more portable, but access remained limited to a small group.

Books

Medieval codices were copied by hand and were rare and expensive. The 15th-century printing press, followed by libraries, mass education and rising literacy, brought texts within the reach of many more people.

Internet

At the end of the 20th century, digitisation and the internet made enormous amounts of information available almost instantly. The challenge increasingly became finding, understanding and evaluating that information.

LLM

We can ask questions in everyday language, ask follow-up questions, and request examples, translations or explanations suited to our level of knowledge. LLMs can lower linguistic, technical and search-related barriers to understanding.

How is a large language model made?

A large language model (LLM) is an AI system trained on an enormous amount of text. Its main ability is to process and generate language. Parameters are numbers adjusted during training; large models may contain hundreds of billions of them.

The model-building process from collecting and filtering web text through tokenisation and training to a chat assistant. — The main steps of building a large language model.

01 · Collecting and filtering data

Preparing the training data

Developers collect large bodies of text, for example from the public web. They filter out spam, low-quality content, duplicates and personal data. Tens of terabytes of text may remain.

Information missing from the training data, or published later, does not automatically become part of the model’s knowledge. The time boundary of its training data is called the knowledge cutoff.

02 · Tokenisation

From text to a sequence of numbers

Text is broken into small units, and each unit is assigned an identification number. The model works with these token IDs.

A token may be a word fragment, a word or a punctuation mark. A long Hungarian word is often split into several tokens.

03 · Pre-training

Predicting the next token

The model receives a sequence of tokens and predicts the next one. Its parameters are adjusted based on the error. This process is repeated an enormous number of times.

The result is a sophisticated text completer that still needs further training to become an assistant.

04 · Post-training

Creating a helpful assistant

Carefully written question–answer examples teach the model to respond helpfully, politely and in a useful format, and when to refuse a request.

People or evaluator models rank alternative answers. These preferences also shape the model’s behaviour.

The model works with tokens

What is a token?

A token is a small unit used to process text. A tokenizer splits the text and assigns a unique identification number to each unit.

The model’s input becomes a sequence of numbers, for example [15339, 1917, 264, 3918]. The exact split and the IDs depend on the tokenizer used.

A typical token vocabulary may contain around a hundred thousand entries. The GPT-4 example in the source mentions a vocabulary of 100,277 items; this size does not apply to every model.

Why does it matter?

A token is not the same as a letter or a word

A long word can be split into several tokens. Its individual letters are not necessarily separate units for the model.

This can contribute to errors when a model is asked to count how many times a letter appears in a word.

Generating a response reverses the process: the model produces tokens, and a program converts them back into readable text.

Try it!

See how text is split into tokens

Open the Tiktokenizer interactive site, and choose a tokenizer.
Enter a short sentence and a long word. You can also try the Hungarian example “megszentségteleníthetetlenségeskedéseitekért”.
Observe the word fragments, token IDs and token count. Compare them with the number of letters and words.

Different tokenizers may split the same text differently. The tool opens in a new tab; this learning content also works locally.

Generating an answer: one token at a time

After you send a message, the model assigns a probability to each possible next token. It picks one, adds it to the text, then predicts again from the expanded text.

What looks like “typing” on screen is the arrival of successive tokens.

The model continues the text based on learned patterns. Without using a tool, it is not looking up the next sentence in a rulebook.

Next-token prediction: the model scores possible text continuations and selects one. — The model scores possible continuations and adds one of them to the text.

Illustrative example

Which continuation fits the sentence?

“Before cutting the steel pipe with an angle grinder, always put on your …”

Choose a continuation to see its explanation.

This is a scripted illustration. The buttons show whole words or phrases, which may consist of several tokens in a real model.

Three consequences to remember

01

Answers can vary

The model does not always choose the single most probable token. The same question can produce different answers.

Differences between repeated answers can be warning signs; agreement alone does not establish correctness.

02

Intermediate steps can help

A worked solution gives the model more tokens and intermediate results. This can help it solve a complex problem.

“Thinking” models may carry out intermediate calculations before answering. A longer explanation still does not guarantee correctness.

03

Memory works differently

The model relies on its learned parameters and the text context it receives at that moment.

Web search, code execution or attached files can provide access to current information. The conversation context is covered in M1-T1-MLC5.

In the source example, the machines take 12, 8 and 6 hours respectively to do the same job alone. Asked for the result only, the model answered 3.43 hours; asked to work step by step, it reached the correct result. Assuming constant, additive work rates, their combined hourly rate is: 1/12 + 1/8 + 1/6 = 3/8. The time needed is 8/3 hours, or 2 hours 40 minutes. This is one example; the reasoning and calculation still need independent checking.

Why does AI hallucinate?

Hallucination refers to AI-generated content that appears fluent, coherent and plausible but is factually incorrect or unsupported by reliable evidence.

The model generates text from learned statistical patterns. Generation alone does not mean that it verifies every statement against reality.

In training examples, questions are often followed by confident answers. The model may therefore produce answer-shaped text even when it has no reliable information.

An illustration from the source material showing the idea of hallucination. — An illustration of human hallucination. The underlying process is different in AI. Image source: ChatGPT.

Why is “AI lies” misleading?

An incorrect text does not imply human intention or a desire to deceive. The model produces plausible answers from learned patterns; hallucination here does not mean an experience of human imagination.

Admitting uncertainty

Guessing can be rewarded

According to Kalai and colleagues (2025), common training and evaluation methods can encourage guessing instead of admitting uncertainty.

Newer models may say they lack information more often, but the problem has not disappeared.

Plausible but wrong: typical examples

A hallucinated answer can have the same tone and format as a correct one. The error is not always obvious.

An invented regulation may have a plausible number, date and paragraph. A wrong technical value may fall within the usual range. A missing safety step may go unnoticed.

An illustration of the risks of invented AI answers that appear official and complete. — An answer can look official – until someone checks the source behind it.

When are errors more likely?

01

Rare, local or recent topics

National regulations, company procedures, last month’s changes or rare professional details. The required information may be missing from the training data.

02

Requests for precise data

Even a small difference matters when asking for a number, name, quotation, reference or prescribed value. A plausible range does not make a value authoritative.

03

False assumptions

A question may start from a fact that does not exist. The model may accept that assumption and construct a coherent answer around it.

Put this question to a chatbot!

Where do I need to check most carefully?

Group the main fields and task types according to how reliable ChatGPT’s answers tend to be. Create a table showing the estimated reliability range, the main risk of error, and the level of verification required. Treat percentages as measured results only when supported by reliable research; otherwise clearly label them as estimates.

Detecting errors and validating outputs

Validation means establishing whether an answer is correct, supported by evidence and suitable for the task.

First, mark the facts, numbers, names, references and safety instructions. Each of these needs checking.

AI can prepare a draft; the teacher validates the content.

Validating AI answers through source checking, follow-up questions and tools. — Check the source, ask follow-up questions and use tools – and only then take the content to the classroom.

01 · Sources

Open the reference!

Does the document exist? Does it really say what the AI claims? Compare the answer with the manufacturer’s manual, standard, textbook or curriculum.

02 · Follow-up questions

Make uncertainty visible!

“Which parts are you unsure about?” “Where does this value come from?” “What would change if…?” Ask again in a new chat or with another model. Inconsistency is a warning sign; agreement is not proof.

03 · Tools

Ask for search or code execution!

For fresh facts, ask for web search; for calculations, ask for code and its execution if the application supports it. Also check the sources, input data and assumptions used.

04 · Your own sources

Provide a reliable document!

Attach a textbook chapter or data sheet: “Answer only on the basis of the attached document; if the answer is not in it, say so.” Then compare the result with the document.

05 · Professional review

Validate before classroom use!

Use your own professional knowledge, or a colleague’s, to check every important fact, number and safety instruction in material prepared for learners.

Five checks before classroom use

Your own checklist

Tick each check you have completed for a specific AI answer.

Which statements are facts, numbers, names or references – and have I checked each of them? Is anything safety-critical? Does it match the manufacturer’s data and the current rules? What is missing: protective equipment, preconditions or exceptions? Are the level, terminology and context right for my learners? Would I sign this material with my own name?

Pedagogical and ethical responsibility

Responsibility stays with the teacher

Learners may treat a worksheet as error-free because it carries the teacher’s authority. As beginners, they are often unable to spot the errors themselves.

In a workshop, a wrong value or a missing safety step can lead to damaged equipment or injury. Errors that learners memorise are difficult to unlearn.

Teach learners to check sources too. Treat AI as a starting point for thinking. Critical verification is also part of the AI and information literacy described in DigComp 3.0.

Practise validation in your own subject!

Teacher’s individual exercise · 35–40 minutes

Hallucination hunt in my own subject

Write five risky questions: one about a regulation or standard, one about a precise technical value, one calculation, one rare technical term and one recent change.

Ask the AI without web search.
Mark and check the factual claims in its answers.
Record: claim – correct / wrong / cannot be verified – source.
Try follow-up questions, search, code execution or your own source. Note what changes.
Create your own five-point professional checklist.

Classroom group exercise · 45 minutes

Fact-checkers in the workshop

Groups of 3–4 learners examine an AI-generated work instruction, prepared and checked by the teacher in advance, containing a few errors or omissions.

Briefly demonstrate why AI can be wrong.
Ask groups to mark suspicious statements and missing steps.
Use a textbook or manual to check and correct the text.
Compare findings and create a “How we check AI” rule card.

Keep a verified answer key. Learners must not act on the incorrect instruction provided for analysis.

Download both practical exercisesM1-T1-MLC3-EXC-EN.pdf

Takeaways for teachers

01

Understand how it works!

An LLM generates text token by token, choosing plausible continuations based on patterns in its training data.

A fluent and confident answer can still be wrong.

02

Recognise the risk!

An authoritative source is especially important for precise numbers, names, references, regulations, rare terms and current or local information.

Look for missing safety steps as well as wrong values.

03

Verify and teach!

Open the source, ask follow-up questions, use search or code, and provide your own documents.

Validate before classroom use. Teach learners to verify AI outputs too.

AI can prepare a draft, but the teacher validates it.Professional responsibility does not pass to the chatbot.

Knowledge check

Check what you have learned!

Choose one answer for every question. After submitting, you will see your score and the explanations.

1What is a token? Always a whole word. A small unit of text to which the tokenizer assigns an identification number. The number of people using the model.

2How does an LLM generate text? It copies every sentence unchanged from a textbook. It automatically verifies every statement before answering. It predicts and selects successive tokens based on the text so far.

3What does AI hallucination mean? Plausibly worded content that is wrong or unsupported by reliable evidence. Only deliberate human deception. An answer appearing slowly.

4A model cites a regulation with an exact number and date. What should you do? Accept it because its format is precise. Find the original document and check that it exists and really says what is claimed. Check only the spelling of the answer.

5Two different models give the same technical value. What does this establish? The value definitely matches the manufacturer’s requirements. The teacher can omit verification. Agreement is a useful signal, but the value still needs checking against an authoritative professional source.

6Who is responsible for validating AI-generated material used in class? The teacher, who reviews the draft professionally and teaches learners how to verify it. Only the chatbot. Only the beginner learners.

References and further exploration

Sources and useful links

References

Kalai, A. T., Nachum, O., Vempala, S. S. & Zhang, E. (2025): Why Language Models Hallucinate. arXiv:2509.04664.
Huang, L. et al. (2024): A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questions. ACM Transactions on Information Systems.
Mata v. Avianca, Inc., 678 F. Supp. 3d 443 (S.D.N.Y. 2023).
Cosgrove, J. & Cachia, R. (2025): DigComp 3.0 – European Digital Competence Framework. 5th ed., Publications Office of the European Union.
Lengyel, Zs. (2026): Introduction to the concepts of artificial intelligence; Concrete applications for content generation. VITA training presentations.

Try it and explore!

Tiktokenizer – see how text is split into tokens.
Interactive 3D visualisation of an LLM – explore the model step by step.
Andrej Karpathy: Deep Dive into LLMs like ChatGPT – an in-depth explanatory video.

External tools and sources open in a new tab and require an internet connection.

Contributors and attachments

Credits and copyright

Author: Zsolt LengyeliTStudy Hungary Training and Research Centre
Reviewer: Mária HartyányiiTStudy Hungary Training and Research Centre
Project: 2025-1-LT01-KA220-VET-000362456
Learning content: M1-T1-MLC3

Attachments

Download the English source materialM1-T1-MLC3-TEXT-EN.pdf Download the practical exercisesM1-T1-MLC3-EXC-EN.pdf

This work may be reused and adapted under the terms of the Creative Commons Attribution-ShareAlike 4.0 International License (CC BY-SA 4.0).

Funded by the European Union. Views and opinions expressed are those of the author(s) only and do not necessarily reflect those of the European Union or the National Agency.