When you ask an AI a question and it gives you a confident, well organized answer in seconds, it is reasonable to wonder where that information actually came from. The answer is more interesting and more layered than most people realize, and understanding it goes a long way toward using AI well.
Modern AI systems draw their information from several distinct sources at different points in time. Some of the information was baked in during the long training process that produced the model. Some is supplied at the moment you ask the question, by you or by the application around the model. Some comes from external tools the AI is allowed to call. Each source has its own strengths, its own blind spots, and its own implications for accuracy and trust.
This guide walks through the major sources of information AI systems use, in plain language, so you can build a clear mental picture of what the AI knows, how it knows it, and where it does not know what it thinks it knows.
Source 1: Training Data
The biggest single source of what an AI knows is the data it was trained on. For a large language model like GPT, Claude, Gemini, Llama, or Grok, training data typically includes a very large collection of text and code assembled from many sources.
The main categories of training data used by the major labs include the following.
Public web data. Snapshots of the open web collected through web crawling, often built on top of large open data sets like Common Crawl with additional filtering and quality controls applied by the lab. This is the single largest source for most foundation models. It includes news articles, blog posts, forum discussions, technical documentation, encyclopedias, government sites, and a long tail of other publicly accessible pages.
Books. Both public domain books and, in many cases, copyrighted books accessed through various means. The use of copyrighted material in training data is the subject of active litigation in multiple jurisdictions and the legal picture is still developing.
Code. Public code repositories, primarily from GitHub and similar platforms, used to teach the model programming languages, libraries, and software patterns. This is why modern models can write reasonable code in dozens of languages without being explicitly programmed to do so.
Academic and scientific content. Research papers, textbooks, and scientific data sets where licensing permits, providing the model with technical and scientific knowledge.
Licensed data. Most major labs now have explicit licensing arrangements with publishers, news organizations, image archives, and other content holders. These deals have become a significant source of high quality training data and a significant business cost for the labs.
Multilingual data. Substantial amounts of non English text from across the world, which is why most modern models can understand and respond in many languages even if their performance is strongest in the most represented ones.
Synthetic data. Data generated by other AI models, often used to fill gaps where real world data is scarce or to improve the model's reasoning on specific types of tasks. The use of synthetic data has grown substantially in the last few years.
The exact composition of training data varies from lab to lab and is increasingly treated as competitive information. Several labs have published partial information about their training corpora, but the full recipes for the leading proprietary models are not public.
Source 2: Fine Tuning and Alignment Data
Raw training data teaches the model the patterns of language and a vast amount of factual content, but it does not by itself teach the model how to be helpful, safe, or aligned with what users want. That is the job of a second stage often called fine tuning, alignment, or instruction tuning.
This stage uses smaller, more carefully curated data sets, often including the following.
Instruction tuning data. Examples of high quality prompts paired with high quality responses, used to teach the model how to follow instructions and respond in the formats users expect.
Human feedback data. Comparisons collected from human raters, where the raters review multiple model outputs for the same prompt and rank them. Techniques like reinforcement learning from human feedback, abbreviated RLHF, use this data to push the model toward responses that humans prefer. Related approaches include direct preference optimization, which trains directly on preference pairs from human or AI raters, and constitutional AI methods that use written principles to shape model behavior.
Safety data. Examples of harmful requests and the appropriate refusals or safer responses, used to teach the model to decline certain categories of request and to handle sensitive topics responsibly.
Domain specific tuning data. For models targeted at specific industries like law, medicine, or finance, additional fine tuning data from those domains can shape the model's behavior and vocabulary.
The model that you actually interact with in a product like ChatGPT, Claude, or Gemini is the result of both the foundation training and this second stage. The fine tuning data is much smaller in volume than the raw training data, but it has a disproportionate effect on how the model behaves day to day.
Source 3: The Prompt and Conversation Context
Once a model is trained and deployed, it does not learn anything new from individual conversations unless the provider explicitly trains on them later. What the model does have access to in any given response is what is provided in the prompt and surrounding context.
This context window typically includes several layers.
The system prompt. A set of instructions, invisible to the user in most products, that tells the model how to behave in this particular application. The system prompt for ChatGPT is different from the system prompt for Microsoft Copilot or for a custom enterprise assistant built on the same underlying model.
The conversation history. The full back and forth of the current conversation, up to the limit of the model's context window. This is why the model can refer to things you said earlier in the same chat but does not remember anything from previous conversations unless an external memory system is involved.
The user message. The specific question or request you are currently asking, including any text, images, or files you attached.
Any explicitly added context. Documents you uploaded, snippets pasted in, or context the application provides on your behalf, such as your current calendar, your CRM data, or your previous support tickets, depending on the integration.
The model treats all of this context as immediately relevant information to draw on for the response, often more strongly than the patterns it learned during training. This is why providing clear, specific, and well structured input dramatically improves output quality, and why pasting in a relevant document often produces better answers than relying on the model's general training knowledge alone.
Source 4: Memory Across Conversations
Several consumer AI products now include explicit memory features that let the model remember things from one conversation to the next. ChatGPT memory, Gemini personalization, and similar features are all examples of this pattern.
What is actually happening under the hood is that the application is storing certain pieces of information about you, often things you have explicitly told it to remember or that it has inferred from your conversations, and then injecting that information into the context of new conversations. The underlying model is not learning. The application is providing more context to the same underlying model each time.
This is important because it means memory is not magic. It is structured storage and retrieval that the user can usually inspect, edit, or clear. It is also why the privacy controls on memory features matter, and why enterprises generally want their own memory layer rather than relying on the consumer product's memory.
Source 5: Retrieval Augmented Generation
A great deal of modern enterprise AI uses a pattern called retrieval augmented generation, abbreviated RAG. The idea is straightforward. Instead of relying entirely on what the model learned during training, the application maintains its own collection of trusted documents and retrieves the relevant ones at query time, then asks the model to answer based on those retrieved documents.
A RAG system typically works in three stages.
First, the application takes a body of source content, such as company documentation, knowledge base articles, product specs, or customer records, and indexes it into a search system, usually a vector database that allows similarity search across the meaning of the text.
Second, when a user asks a question, the application searches the index for the chunks of source content most likely to be relevant to the question.
Third, the application passes the user question along with those retrieved chunks to the language model and instructs the model to answer based on the provided sources.
This pattern has become the standard way to ground AI in specific, accurate, private, or recent information that the model could not have learned during training. It is how most enterprise assistants, customer support bots, internal knowledge tools, and document question answering systems work.
The accuracy of a RAG system depends heavily on the quality of the source content, the quality of the retrieval step, and the discipline of the model in actually using the retrieved sources rather than falling back on its training knowledge. Well built RAG systems can be dramatically more accurate than the underlying model alone. Poorly built ones can be confidently wrong in ways that are harder to detect.
Source 6: Tool Use and Web Search
Modern AI products increasingly let the model call external tools at the moment of the request. The most common tool is web search, but the pattern extends to calculators, code execution, database queries, calendar lookups, internal company APIs, and many others.
When an AI uses tools, the flow looks something like this. The user asks a question. The model decides, based on the question, whether it needs to call any tools. If it does, it generates a tool call, which the surrounding application executes. The tool returns its results, which become part of the context for the model's next response. The model then uses the tool results, possibly along with its training knowledge, to formulate the final answer.
Web search in particular has become important for keeping AI responses current. Without web search, a model's knowledge is effectively frozen at its training cutoff date, with everything more recent simply unknown to it. With web search, the model can retrieve current information at query time and incorporate it into the answer.
Tool use is also how AI agents do real work in the world, from booking travel to making code changes to interacting with business systems. The model itself does not do these things. It decides which tools to call, the tools do the actual work, and the results flow back into the conversation.
Source 7: The Training Cutoff and What It Means
Every AI model has a training cutoff date, which is the most recent date represented in the data used to train it. The exact cutoff varies by provider and model, and many products supplement the cutoff knowledge with retrieval or web search to keep responses current.
For events, products, prices, people, and statistics that have changed since the cutoff, the model simply does not know. Worse, models will sometimes confidently produce outdated information without flagging that it might be out of date, because the training process does not teach the model to track time. This is one of the most common sources of incorrect AI responses.
Products that wrap the model with tools, especially web search, help mitigate this by retrieving current information. Products that do not have those tools rely entirely on the training cutoff knowledge for anything outside the immediate prompt.
A practical habit when using AI for anything time sensitive is to either use a product that has web search enabled or to provide the current information yourself in the prompt and ask the model to reason from it.
Source 8: What AI Does Not Get Its Information From
It is also useful to be clear about what is not a source of AI information by default, to correct several common misconceptions.
Your private data is generally not in the model. The exact picture depends on the provider's data usage policies and your product tier, but for most enterprise plans and many consumer plans the model is not trained on your individual conversations. Unless your company has specifically fine tuned a model on your data, or you have explicitly provided the data in the prompt or through a connected system, the base model does not have access to your files, emails, CRM records, or anything else private to you.
Your previous conversations are not in the model. The base model is the same for every user. Anything that feels personalized is either coming from a memory system that the application maintains separately or from context being added to your prompts behind the scenes.
Real time information is not in the model. Without a tool like web search, the model has no way to know what is happening right now, what the current weather is, what a stock is trading at, or what was in the news today.
The model does not know what it does not know. When the model encounters a question outside its training, the safer behavior would be to say so. The more common behavior, especially in models without strong calibration, is to produce a confident sounding answer based on adjacent patterns. This is the core mechanism behind hallucination.
How to Use This to Get Better Answers
Understanding the sources of AI information leads to a few practical habits that consistently produce better results.
Provide the relevant context yourself. If you have the document, paste it in or upload it. If you have the data, include the numbers. The model uses the prompt context more reliably than its general training knowledge.
Use tools when you need current or specific information. Web search, file uploads, and connected systems all dramatically expand what the model can work with accurately.
Be skeptical of confident answers on topics with high specificity. Names, numbers, dates, citations, prices, and product specifications are the most common categories where models fabricate. Verify these before relying on them.
Ask the model what it is basing its answer on. Many modern models and products will tell you whether they retrieved a source, used a tool, or are answering from training knowledge alone, and many will provide citations or tool call logs. The reliability of that self report varies, so the most trustworthy signal is the product's actual surfaced citations and tool traces rather than the model's narration of what it did.
Use RAG or domain specific tools for high stakes work. For questions where accuracy matters and the source material is specific to your context, a properly built RAG system or a tool integrated assistant will almost always outperform the same model used standalone.
The Bottom Line
AI gets its information from several places at once. The bulk of its general knowledge comes from the very large training data set used during pretraining, refined by the smaller fine tuning data set used to make it helpful and safe. Beyond that, the model draws on the prompt and conversation context, any memory the application maintains for you, retrieved documents through RAG, and external tools like web search.
None of these sources is perfect on its own. The training data is broad but frozen at a cutoff date and full of unmarked imperfections. The fine tuning data shapes behavior but does not add much new knowledge. The prompt context is reliable but limited to what you provide. RAG can be very accurate but depends on the quality of the source content. Tools extend the model's reach but introduce their own failure modes.
The best AI products combine these sources thoughtfully and tell you clearly which ones they used. The best AI users understand that the answer they see is the output of a layered system and apply the right level of verification accordingly. The golden rule still applies. If your name is going on the output, you should know where the information came from and be willing to vouch for it.
If you are building AI workflows for your team or your company and want a clear picture of what sources you should be connecting, what governance you need around them, and how to verify what the system produces, we are happy to help. The technology is genuinely powerful, and the more clearly you understand its sources, the more value you will get from it.