LangChain Study
· Updated
A simple template for each note:
- One-sentence definition
- Why it matters (what breaks or changes if you ignore it)
- Small example or code snippet
- Related notes
- Open questions
- Sources
Large Language Model (LLM)¶
1. Definition
A large language model is a probabilistic model trained on vast amounts of text to predict the next token in a sequence. "Large" refers to the number of parameters it contains, usually billions or more.
2. Why it matters
- Everything a model does (answering, summarizing, coding) comes from repetition: predict the next token, append it, repeat.
- LLMs are a general-purpose technology that has transformed how people interact with computers.
3. Example
Prompt: "The capital of France is"
Next-token probabilities (illustrative):
- " Paris" → 0.97
- " a" → 0.01
- " located" → 0.005
The model samples one token (usually " Paris"), appends it, and predicts again.
API call that produces this behavior:
response = client.messages.create(
model="claude-sonnet-5-5",
max_tokens=50,
messages=[{"role": "user", "content": "The capital of France is"}],
)
4. Related notes
- Tokens: the unit the model predicts, one at a time
- Context window: how many tokens it can handle at once
- Temperature: how it picks among the predicted tokens
- System / User / Assistant messages: how a conversation is fed to it
- API call: how you actually use it
5. Open questions
- How do parameters work? I could find a simple example to see what they look like.
6. Sources
- Glossary: LLM, Claude Docs
- Large language model, Wikipedia
Tokens¶
1. Definition
It's the basic unit of text, which can be a whole word or part of one.
2. Why it matters
- Tokens are the unit LLMs work in.
- Context windows are measured in them.
- API pricing is per token, so they determine cost.
3. Example
Text: "Tokenization is unbelievable!"
One possible split (exact splits vary by tokenizer):
["Token", "ization", " is", " un", "believ", "able", "!"]
That's 7 tokens for 4 words. Common words are usually one token, while
longer or rarer words get broken into pieces.
Checking real counts with the API:
count = client.messages.count_tokens(
model="claude-sonnet-5-5",
messages=[{"role": "user", "content": "Tokenization is unbelievable!"}],
)
print(count.input_tokens)
Experiment: run the same sentence in English and in another language, and
compare input_tokens. The non-English version will often use more.
4. Related notes
- LLM: what produces and consumes tokens
- Context window: its limit is measured in tokens
- Temperature: reshapes the probabilities of the next token
- API call:
usagereports token counts, which drive cost
5. Open questions
- Why do tokenizers split words into pieces instead of using whole words?
6. Sources
- Glossary: Tokens, Claude Docs
- Token counting, Claude Docs
- Pricing, Claude Docs
Context window¶
1. Definition
It's the maximum amount of text, measured in tokens, that an LLM can hold and reference while generating a response.
2. Why it matters
- Input and output share the same window. A long prompt that uses almost all the context window can leave little room for the output, which may then get cut off.
- It drives design decisions, because it forces you to find ways to retrieve only what matters.
3. Example
Say a model has a 10,000-token context window (illustrative number).
| Turn | Input sent (history + new msg) | Reply | Total used |
|------|-------------------------------|-------|-----------|
| 1 | 200 | 300 | 500 |
| 2 | 800 | 300 | 1,100 |
| 3 | 1,400 | 300 | 1,700 |
Each turn's input includes all previous turns, so usage climbs steadily. Keep
going and eventually the input plus the reply won't fit.
In code, the growth looks like this:
messages = []
def chat(user_text):
messages.append({"role": "user", "content": user_text})
response = client.messages.create(
model="claude-sonnet-5-5",
max_tokens=500,
messages=messages, # the whole history, every time
)
reply = next(
(block.text for block in response.content if block.type == "text"), ""
)
messages.append({"role": "assistant", "content": response.content})
print(response.usage.input_tokens) # watch this number grow
return reply
Experiment: call chat() five times and print input_tokens each time.
4. Related notes
- LLM: what the window belongs to
- Tokens: the unit the window is measured in
- System / User / Assistant messages: every message, of every role, takes up space in it
- API call: calls are stateless, so the whole history is resent and fills the window
5. Open questions
- What strategies handle conversations that outgrow the window (summarizing, truncating, retrieval)?
- Why do models sometimes miss details in the middle of long inputs?
6. Sources
- Glossary: Context window, Claude Docs
- Context windows, Claude Docs
Temperature¶
1. Definition
A parameter that controls the randomness of the text an LLM generates. Lower values make the output predictable and consistent, while higher values make responses more creative and varied.
2. Why it matters
- It controls how consistent responses are. Use a low temperature for tasks with a right answer, and a high one for brainstorming or creative writing.
- Even at temperature 0, responses aren't guaranteed to be identical: LLM output isn't fully deterministic.
- Current Claude models don't accept the
temperatureparameter, though other providers may still expose it.
3. Example
Prompt: "The sky is"
Next-token probabilities (illustrative):
- " blue" → 0.70
- " clear" → 0.15
- " gray" → 0.10
- " falling" → 0.05
Effect of temperature, taking these as the probabilities at 1.0:
- Below 1.0: the distribution sharpens, so " blue" dominates.
- At 1.0: the probabilities stay as listed.
- Above 1.0: the distribution flattens, so " falling" becomes a real possibility.
A small simulation, with no API needed:
probs = {" blue": 0.70, " clear": 0.15, " gray": 0.10, " falling": 0.05}
for t in [0.2, 0.7, 1.0, 2.0]:
weights = {token: p ** (1 / t) for token, p in probs.items()}
total = sum(weights.values())
print(t, {token: round(w / total, 2) for token, w in weights.items()})
Output:
0.2 {' blue': 1.0, ' clear': 0.0, ' gray': 0.0, ' falling': 0.0}
0.7 {' blue': 0.84, ' clear': 0.09, ' gray': 0.05, ' falling': 0.02}
1.0 {' blue': 0.7, ' clear': 0.15, ' gray': 0.1, ' falling': 0.05}
2.0 {' blue': 0.47, ' clear': 0.22, ' gray': 0.18, ' falling': 0.13}
Experiment: add 0.01 and 10 to the list, and see what happens at both extremes.
4. Related notes
- LLM: what it changes the behavior of
- Tokens: it acts on the next-token probabilities
- API call: providers that still expose it take it as a parameter on the call
5. Open questions
- How do top-p and top-k differ from temperature, and can I combine them?
- Why isn't temperature 0 fully deterministic?
6. Sources
- Glossary: Temperature, Claude Docs
- Migrating to API version 2026-10-21, Claude Docs
- Softmax function, Wikipedia
System / User / Assistant Messages¶
1. Definition
Messages are the role-labeled turns that make up a conversation with an LLM.
- System messages provide high-level instructions and context for the assistant's behavior.
- User messages contain specific queries or prompts from users.
- Assistant messages represent the responses generated by the model based on the previous system or user messages.
2. Why it matters
- Roles are a design tool.
- All three roles count toward the context window.
3. Example
Same question, with and without a system message:
# No system message
messages=[{"role": "user", "content": "What is a token?"}]
# With a system message
system="You are a teacher for beginners. Answer in two sentences, no jargon."
messages=[{"role": "user", "content": "What is a token?"}]
A multi-turn conversation, where I include the assistant's previous reply:
response = client.messages.create(
model="claude-sonnet-5-5",
max_tokens=300,
system="You are a teacher for beginners. Answer in two sentences, no jargon.",
messages=[
{"role": "user", "content": "What is a token?"},
{"role": "assistant", "content": "A token is a small chunk of text the model reads and writes."},
{"role": "user", "content": "Why does it matter for cost?"},
],
)
Experiment: ask the third question alone, then with the full history. Compare the answers. Without the history, the model doesn't know what "it" refers to.
4. Related notes
- LLM: who reads the messages
- Context window: all roles count toward it
- API call: messages and
systemare fields of the request
5. Open questions
- How much does the system message actually change behavior compared with putting the same text in the first user message?
6. Sources
- Using the Messages API, Claude Docs
- Messages API reference, Claude Docs
API Call¶
1. Definition
An API call to a model is an HTTP request where you send your messages and settings to a server, and it sends back the model's reply.
2. Why it matters
- It's how any project interacts with LLMs.
- Each request is stateless, so you have to resend the whole conversation every time.
3. Example
Request (curl):
curl https://api.anthropic.com/v1/messages \
-H "x-api-key: $ANTHROPIC_API_KEY" \
-H "anthropic-version: 2023-06-01" \
-H "content-type: application/json" \
-d '{
"model": "claude-sonnet-5-5",
"max_tokens": 1024,
"system": "You are a helpful assistant that explains things simply.",
"messages": [
{"role": "user", "content": "What is a token?"}
]
}'
Same call in Python:
import anthropic
client = anthropic.Anthropic() # reads ANTHROPIC_API_KEY from the environment
response = client.messages.create(
model="claude-sonnet-5-5",
max_tokens=1024,
system="You are a helpful assistant that explains things simply.",
messages=[{"role": "user", "content": "What is a token?"}],
)
text = next(
(block.text for block in response.content if block.type == "text"), ""
)
print(text)
print(response.stop_reason)
print(response.usage.input_tokens, response.usage.output_tokens)
Trimmed response:
{
"id": "msg_01...",
"role": "assistant",
"content": [{"type": "text", "text": "A token is a small chunk of text..."}],
"stop_reason": "end_turn",
"usage": {"input_tokens": 25, "output_tokens": 80}
}
Experiment: set max_tokens to 10 and watch stop_reason change. Then deliberately use a wrong API key and read the error that comes back.
4. Related notes
- LLM: what answers the call
- Tokens:
usagein the response counts them - Context window: resending the history on every call is what fills it
- Temperature: a setting some providers still accept on the call, though current Claude models don't
- System / User / Assistant messages: the body of every request
5. Open questions
- What should my code do when a call fails or hits a rate limit?
6. Sources
- Messages API reference, Claude Docs
- Using the Messages API, Claude Docs
- Stop reasons and fallback, Claude Docs
Structured output¶
1. Definition
A technique to get the model to reply in a predictable, machine-readable shape, usually JSON.
2. Why it matters
- It's the format to rely on when saving to a database, sending to an API, etc.
3. Example
Same fact, two possible free-text replies:
Ana is 31 years old.
The person's name is Ana, and she's 31.
With structured output, the shape is always the same:
{"name": "Ana", "age": 31}
The native way, using a Pydantic model as the schema:
from pydantic import BaseModel
class Person(BaseModel):
name: str
age: int
response = client.messages.parse(
model="claude-sonnet-5-5",
max_tokens=1024,
messages=[{"role": "user", "content": "Ana is 31 years old."}],
output_format=Person,
)
print(response.parsed_output) # name='Ana' age=31
Experiment: drop the schema, ask for JSON in the prompt instead, and run json.loads() on the reply a few times. See how often it breaks.
4. Related notes
- Tools: tools also take a schema, which describes a function's arguments rather than the reply
- API call: the schema travels with the request
- Runnable / LCEL: output parsers turn the reply into data as one step of a chain
5. Open questions
- What happens when the output fits the schema but the values are wrong?
6. Sources
- Structured outputs, Claude Docs
Tools¶
1. Definition
A way to let your model ask your code to run something and use the result in its reply.
It's a mechanism that allows an AI model to generate structured requests, typically in JSON, to invoke external functions or APIs. There are two ways to invoke a tool:
- Automatic tool invocation: the LLM decides whether it makes sense to call a tool or not. This is the standard choice.
- Forced tool invocation: the system developer sets the model to always use certain tools. This is ideal for deterministic pipelines, such as structured data extraction. Some models, including Claude Sonnet 5.5 and Opus 5.5, don't support this.
2. Why it matters
- LLMs predict plausible text, so they're unreliable at exact computation.
- Using tools expands the power and usefulness of the model.
- The invocation mode decides whether the model or the developer chooses when a tool runs.
3. Example
The model doesn't run anything itself. It asks, my code runs the function, and I send back the result:
tools = [{
"name": "multiply",
"description": "Multiply two numbers.",
"input_schema": {
"type": "object",
"properties": {"a": {"type": "number"}, "b": {"type": "number"}},
"required": ["a", "b"],
},
}]
messages = [{"role": "user", "content": "What is 48213 * 9177?"}]
response = client.messages.create(
model="claude-sonnet-5-5",
max_tokens=300,
tools=tools,
messages=messages,
)
# response.stop_reason is "tool_use": the model is asking for the function
call = next(block for block in response.content if block.type == "tool_use")
result = call.input["a"] * call.input["b"] # my code does the math
messages.append({"role": "assistant", "content": response.content})
messages.append({
"role": "user",
"content": [{"type": "tool_result", "tool_use_id": call.id, "content": str(result)}],
})
final = client.messages.create(
model="claude-sonnet-5-5",
max_tokens=300,
tools=tools,
messages=messages,
)
text = next(block.text for block in final.content if block.type == "text")
print(text)
Experiment: ask the same question without the tool and compare the answer with the exact result.
See more examples in n8n's LLM Tool Calling: How It Works and How To Implement It.
4. Related notes
- Structured output: also schema-based, but it shapes the reply instead of a function's arguments
- System / User / Assistant messages: the tool's request and result travel as messages
- API call: tools are a field of the request, and each round trip is a new call
- The agent loop: where the model keeps making this decision
5. Open questions
- How does the model decide which tool to call, and what if it picks the wrong one?
- What's the difference between passing a Python function directly and using the
@tooldecorator?
6. Sources
- Tool use with Claude, Claude Docs
- LLM Tool Calling: How It Works and How To Implement It, n8n blog
Runnable / LCEL: composing steps¶
1. Definition
A Runnable is LangChain's standard interface for a component, and LangChain Expression Language (LCEL) is the | syntax for composing Runnables into a chain. Prompts, models, and output parsers are all Runnables.
2. Why it matters
- All processes are like pipelines. Composing these steps represents an end-to-end pipeline in an AI context.
3. Example
A three-step chain, composed with the | operator:
from langchain_anthropic import ChatAnthropic
from langchain_core.output_parsers import StrOutputParser
from langchain_core.prompts import ChatPromptTemplate
prompt = ChatPromptTemplate.from_template("Explain {topic} in one sentence.")
model = ChatAnthropic(model="claude-sonnet-5-5")
parser = StrOutputParser()
chain = prompt | model | parser
print(chain.invoke({"topic": "tokens"}))
Each step's output is the next step's input:
{"topic": "tokens"} → prompt → messages → model → AI message → parser → string
Experiment: remove the parser from the chain and print the result. It's now an AI message object instead of a string.
4. Related notes
- System / User / Assistant messages: what the prompt step produces
- API call: what the model step makes under the hood
- Structured output: a parser step can return structured data instead of a string
5. Open questions
- What does LCEL give me that a plain Python function doesn't?
6. Sources
- Runnable, LangChain reference
The agent loop¶
1. Definition
The agent loop is a series of steps in which the model repeatedly decides whether to call a tool or give the final answer, and it keeps looping until it produces a final answer or a stop condition is reached.
2. Why it matters
- Agents solve problems where the steps can't be known in advance. If the steps are fixed and predictable, an agent may be the wrong choice, and the same goes for low-value tasks.
3. Example
See the examples on LangChain's Agents page.
4. Related notes
- Tools: what the model decides whether to call on each step
- API call: each step of the loop is a new call
5. Open questions
- N/A
6. Sources
- Agents, LangChain docs