Glossary · 1 · Foundations: how AI works
Training vs. inference
Also known as: Inference, Model training, Knowledge cutoff
Training is the phase in which a model learns by adjusting its weights on large datasets; inference is the phase in which the finished model is used to produce outputs for new inputs. Training happens rarely and costs a lot of compute; inference happens with every request.
- Intermediate
- Technical project managers
- Developers
In one sentence
Training vs. inference: how AI models learn once and then answer every request — and why the difference matters for cost, data and knowledge cutoffs.
Example
A vendor trains a model over several months; when your support chatbot answers a customer, it runs inference on that trained model in a fraction of a second.
Why it matters on your learning path
- Technical project managers: Training data questions (rights, bias, cutoff date) belong to the model provider; inference questions (cost, latency, data sent) belong to your project.
- Developers: Most applications only run inference. Adapting a model means either fine-tuning (training) or better context (inference).
Knowledge cutoff
Because a model only knows what was in its training data, it has a knowledge cutoff. Current facts must be supplied at inference time, for example through retrieval-augmented generation.