Giving AI Context

What does it mean to take something out of context?

You may have heard someone say that a statement was “taken out of context.” That means the statement was presented without the surrounding information needed to understand it correctly.

The missing context might be what was happening at the time, what the rest of the document said, what the people had said to each other earlier, or some other detail that changes how we understand the statement.

Here is a request that seems perfectly clear. Read it, then read what AI did with it.

That is a defensible reading of “take out what I don’t need.” It sounds like the person is going to the beach and may not need snacks or a hoodie. It is also not necessarily what the user wanted. AI did not misread the words. AI inferred a situation and answered based on its interpretation of what might not be essential in that context.

Now imagine three different conversations that could have come right before that request. The right answer to “take out what I don’t need” changes for each one.

Read the first row again. As soon as you know the house has towels and a hair dryer, the request is obvious. Nobody wants to pack a towel they will not use. The words did not change. The context before them told you what they meant.

The right answer depended entirely on which conversation came first. AI never saw any of them, so it guessed, and the guess was wrong in all three cases. Nobody typed the wrong command. The request left out the context that gave the words their meaning, and that is the whole problem.

This kind of instruction is a filter: keep some items, remove the rest. Filters are useful. We can hand AI a long list and ask it to cut it down to what matters, and it will do that in seconds. But a filter is only as good as the rule it is applying. Here, the rule lived in a conversation AI never saw. If we want a filter to remove the right things, we have to tell AI what “the right things” means.

This is the most common problem people have when they program and instruct AI. They do not take things out of context on purpose. They simply never provide enough context in the first place. They ask AI to do something, but leave out information about the situation, the purpose, the input, or the result they want, and AI fills the gap with a guess.

AI needs information to do the work

The same problem appears when we program AI. In the first lesson, we saw that a prompt can be a program. We gave AI an instruction and then gave it an input to process.

The information that AI needs in order to perform the task is called context. Context is all the information available in the conversation that AI can use to do what we are asking it to do.

AI already knew what a flashcard was. It knew the operation, which is how flashcards generally work. It could not know which material we wanted to study until we supplied it. The context in this conversation was the notes about the water cycle.

The important context was the information we wanted. AI could not know our desires or our thoughts. It could not have known in advance which notes we would bring or what we would want done with them. That is context we must provide. It is the key context: the part of the conversation that AI had no way to get on its own.

AI already knows some things

When we work with ChatGPT, Claude, or Gemini, we are working with a large language model, or LLM. An LLM is a type of AI model. This particular type of AI model takes in words and produces words in response. When we send it a prompt, it responds with words, as we saw in the first lesson when we said hello and it answered.

How do these models know how to respond to our prompt? They are trained on enormous amounts of text: books, websites, articles, and many other examples of language. From that text, an LLM learns patterns in language and the concepts that those patterns express. That is why the model may already know what a flashcard is, what “double” means, or what a study plan usually looks like.

We did not have to explain what a flashcard was. That is the kind of general concept an LLM may already know from its training. But trained knowledge is not the same as knowing our particular situation.

How a large language model learns

An LLM learns by being shown text and trying to predict what comes next. It is shown the beginning of a sentence, guesses the next word, is told the correct word, and tries again on the next one. The process repeats across an enormous amount of text.

You did not have to think about those. You have seen the pattern so many times that the next word is just there. An LLM works the same way, except that it has seen patterns from a very large amount of text, so it can finish sentences most people have never read.

Here is a second example. This time there is no single right answer.

The LLM knows a name has to come next. It has seen millions of introductions where someone wrote “My name is” and a name followed. So it completes the pattern with a name. But nothing in the conversation says which name, so any name fits. It said Ralph. It could just as easily have said Nova, Sparky, or some other name that sounds like a robot.

Now look at what happens when the conversation contains a name before the blank.

This time it said Astro. The pattern is the same: a name has to come next. But the conversation now contains a name, and the LLM uses what is in the conversation when it completes the pattern. It was given that context, so it used it.

These two examples show two things. First, an LLM learns to complete patterns it has seen before. Second, when it completes a pattern, it uses the context that is already in the conversation. Without the context, it fills the blank with anything that fits. With the context, it fills the blank with what we told it.

That is the whole basic idea: an LLM learns patterns in language and uses them, together with what is in the conversation, to predict what comes next. The real training process is much larger and more complicated, but this gives us the basic idea. The model is learning patterns from our language, not storing a private record of every person who will later use it. This is why it is called a large language model.

AI draws on two sources of knowledge

When AI responds, it can draw on two sources: things it learned during training and things we provide in the conversation. It combines the model’s general knowledge with the context we give it now.

AI may already know what a study plan is and what two-step equations, slope, and graphing lines are. It could not know the date of our test, the topics our teacher chose, or the time we have available until we gave it that information. The general concept came from training. The facts about our situation came from the conversation.

It did not have to ask. It could have relied on training alone and produced a plan right away.

Nothing in that plan is wrong about Algebra 1. It also may be useless to us. It may cover topics our test does not. It takes a week when we have until Tuesday. It has nothing to do with what we have been studying in class. It may be the wrong answer, and if so, it is wrong for the same reason the packing list lost the wrong items: AI filled in the missing context with what is typical instead of what is true for us.

Some information can only come from us

AI cannot know about anything that happened in the world, or anything that was not written down and provided to it, before it was trained. When you use AI, it is not being trained with new information. It is only being told new information in the conversation.

AI cannot know what you had for breakfast. But you can tell it, and then it knows. It knows only during the current conversation. In this example, AI knows what we had for breakfast solely because we told it in the conversation.

Some AI tools have a feature called “memory.” That is not what it sounds like. It is really just searching through what you told the tool in other conversations and adding the relevant parts to the current one. The model itself has not learned anything new. It is being handed context again.

The key thing to remember is this. Assume that anything that happened after AI was trained, anything that was never written down, and anything private to you or your group cannot be known to AI unless you tell it in the conversation or give it a way to read that information into the conversation.

Look back at the two study plans. AI can rely on either source. It can build a plan from what it learned about Algebra 1 in general, or it can build one from what we tell it about our class, our test, and our week. Both are real knowledge. But the plan that is worth having is usually the one that is specific to us, and that one can only be built from context we provide.

Context includes the instructions we give

Context is not only the information to process. It also includes how we want the work done: what to include, what to leave out, why we need the result, who will use it, and how it should be presented.

The first response was sensible. AI made real flashcards from the notes. They were just not exactly what we wanted: there were two instead of three, they were not numbered, and the layout was front-and-back instead of question-and-answer. The notes did not change. The second instruction gave AI more context about the output we wanted, and it produced that output. If we care about numbered questions, short answers, source limits, a particular audience, or a particular format, those are not details AI can reliably read from our minds. We have to state them.

When context is missing, AI guesses

Think about taking a test. When you do not know an answer, you may infer from the wording of the question, use what you know about the subject, and make the best guess you can. AI does something similar when it does not have sufficient knowledge from training or sufficient context in the conversation.

The answer sounds possible, but nothing in the conversation supports it. AI is trying to produce the best-looking answer it can, not reporting an observation. This kind of plausible but unsupported guess is often called a hallucination.

It is very important to provide sufficient context in the conversation to avoid hallucination. A hallucination is AI guessing or being creative when we did not want it to. When we provide sufficient context, the likelihood that AI hallucinates or makes something up is reduced. There are also ways we can write our prompts to make it difficult for AI to accidentally hallucinate something we do not want. We will see one of them later in this lesson.

AI can guess about instructions too

AI may guess not only about missing facts, but also about what we meant by an instruction or rule. There may be many reasonable ways to perform a task, and AI has to choose among them when we do not specify what we want.

The first quiz was a reasonable quiz. It just was not the kind of quiz we wanted. If we care about the number of questions, the question type, the answer key, or the format, those details are part of the context we need to provide. AI cannot choose the format we have in mind unless we tell it what that format is.

The same problem appears with rules. If we say “whenever I give you a number,” we may still need to say what happens with a negative number, a decimal, several numbers, or something that is not a number at all.

Was that the right answer? A flock of geese is not a number. There are two reasonable things AI could have done. It could do what it did: treat the input as one of something and double it. Or it could say, “That is not a number, so I cannot double it.” Both readings fit the instruction. Nothing in the instruction says which one we wanted.

We may not want “two flocks of geese.” We may want an error. If this program were checking inputs to a form, a cheerful doubling of “a flock of geese” would be exactly the wrong behavior. That is the missing context: what to do when the input is not the kind of thing the rule was written for. The original instruction sounded clear in the ordinary case, but it did not say how to handle a different kind of input. Missing context can be about facts, intended behavior, or edge cases like this one.

The more precise the context, the more precise the result

Before asking AI to do a task, think about three things: what it should do, what information it should work on, and what rules or format it should follow. We can also tell it the purpose of the result and what details matter most.

We supplied the operation, the input, the format, and a rule for missing information. That gave AI less to guess about. Precision in the output usually begins with precision in the context.

Context includes what to do when something is missing

So far we have been precise about what we want. There is a second kind of precision that is just as important: telling AI what to do when it does not have what it needs.

Every program eventually meets a situation its instructions did not cover. The input might be missing something. An instruction might be ambiguous, with two reasonable readings. Two rules might conflict. The input might be a flock of geese. If we say nothing about these cases, AI does what it always does with a gap: it guesses, and the guess can look confident.

We can close that gap with context. Instead of trying to specify every possible detail in advance, we tell AI how to behave when something is unspecified. Ask when information is missing. Choose the simpler reading when an instruction is ambiguous, and say so. Follow the more recent rule when two rules conflict, and say that too. Report when the input cannot support the task instead of stretching it. This is one of the ways we make it difficult for AI to hallucinate: we give it something to do other than guess.

The rules about missing, ambiguous, and conflicting information were as much a part of the program as the rule about flashcards. Specifying how to handle the things we did not specify can matter as much as specifying the details themselves. It is what turns a program that works in the ordinary case into one we can trust in the unusual case.

Context is what AI needs to do the work

AI cannot know what is in our minds. It cannot automatically know what happened today, what is in our notebook, what information matters to us, or exactly how we want a task performed. It may have the capability to do the work, but capability is not the same as having the information and directions needed for this particular task.

We give it that context through language. The work of prompting is often the work of noticing what AI could not know and supplying it.

Context is part of programming AI

In the first lesson, we said that instructing AI is a form of programming, and that programming AI is part of leading it. Giving AI sufficient context is part of that programming. The point is to get AI to do the work the way we want it done, for the problems we actually face, and not just to produce an answer that looks good because it matches something in its training data. The one-week Algebra 1 plan looked good. It was built for nobody in particular.

What separates exceptional leadership of AI from ordinary use is exceptional instructions and exceptional context. An exceptional leader of AI is good at figuring out what AI could not know, what it needs to know, and how to give it that context so that it can best support the work we are doing. That is not asking AI to do our thinking for us. It is telling AI enough about our situation that the work it does is the work we needed.