LangChain Study

#ai #llm #study #source_of_knowledge

A simple template for each note:

  • One-sentence definition
  • Why it matters (what breaks or changes if you ignore it)
  • Small example or code snippet
  • Related notes
  • Open questions
  • Sources

Large Language Model (LLM)

1. Definition

A large language model is a probabilistic model trained on vast amounts of text to predict the next token in a sequence. "Large" refers to the number of parameters it contains, usually billions or more.

2. Why it matters

  • Everything a model does (answering, summarizing, coding) comes from repetition: predict the next token, append it, repeat.
  • LLMs are a general-purpose technology that has transformed how people interact with computers.

3. Example

Prompt: "The capital of France is"

Next-token probabilities (illustrative):
- " Paris"   → 0.97
- " a"       → 0.01
- " located" → 0.005

The model samples one token (usually " Paris"), appends it, and predicts again.

API call that produces this behavior:

    response = client.messages.create(
        model="claude-sonnet-5-5",
        max_tokens=50,
        messages=[{"role": "user", "content": "The capital of France is"}],
    )

4. Related notes

5. Open questions

  1. How do parameters work? I could find a simple example to see what they look like.

6. Sources


Tokens

1. Definition

It's the basic unit of text, which can be a whole word or part of one.

2. Why it matters

  • Tokens are the unit LLMs work in.
  • Context windows are measured in them.
  • API pricing is per token, so they determine cost.

3. Example

Text: "Tokenization is unbelievable!"

One possible split (exact splits vary by tokenizer):

    ["Token", "ization", " is", " un", "believ", "able", "!"]

That's 7 tokens for 4 words. Common words are usually one token, while
longer or rarer words get broken into pieces.

Checking real counts with the API:

    count = client.messages.count_tokens(
        model="claude-sonnet-5-5",
        messages=[{"role": "user", "content": "Tokenization is unbelievable!"}],
    )

    print(count.input_tokens)

Experiment: run the same sentence in English and in another language, and
compare input_tokens. The non-English version will often use more.

4. Related notes

  • LLM: what produces and consumes tokens
  • Context window: its limit is measured in tokens
  • Temperature: reshapes the probabilities of the next token
  • API call: usage reports token counts, which drive cost

5. Open questions

  • Why do tokenizers split words into pieces instead of using whole words?

6. Sources


Context window

1. Definition

It's the maximum amount of text, measured in tokens, that an LLM can hold and reference while generating a response.

2. Why it matters

  • Input and output share the same window. A long prompt that uses almost all the context window can leave little room for the output, which may then get cut off.
  • It drives design decisions, because it forces you to find ways to retrieve only what matters.

3. Example

Say a model has a 10,000-token context window (illustrative number).

| Turn | Input sent (history + new msg) | Reply | Total used |
|------|-------------------------------|-------|-----------|
| 1    | 200                           | 300   | 500       |
| 2    | 800                           | 300   | 1,100     |
| 3    | 1,400                         | 300   | 1,700     |

Each turn's input includes all previous turns, so usage climbs steadily. Keep
going and eventually the input plus the reply won't fit.

In code, the growth looks like this:

    messages = []

    def chat(user_text):
        messages.append({"role": "user", "content": user_text})
        response = client.messages.create(
            model="claude-sonnet-5-5",
            max_tokens=500,
            messages=messages,          # the whole history, every time
        )
        reply = next(
            (block.text for block in response.content if block.type == "text"), ""
        )
        messages.append({"role": "assistant", "content": response.content})
        print(response.usage.input_tokens)   # watch this number grow
        return reply

Experiment: call chat() five times and print input_tokens each time.

4. Related notes

5. Open questions

  • What strategies handle conversations that outgrow the window (summarizing, truncating, retrieval)?
  • Why do models sometimes miss details in the middle of long inputs?

6. Sources


Temperature

1. Definition

A parameter that controls the randomness of the text an LLM generates. Lower values make the output predictable and consistent, while higher values make responses more creative and varied.

2. Why it matters

  • It controls how consistent responses are. Use a low temperature for tasks with a right answer, and a high one for brainstorming or creative writing.
  • Even at temperature 0, responses aren't guaranteed to be identical: LLM output isn't fully deterministic.
  • Current Claude models don't accept the temperature parameter, though other providers may still expose it.

3. Example

Prompt: "The sky is"

Next-token probabilities (illustrative):
- " blue"  → 0.70
- " clear" → 0.15
- " gray"  → 0.10
- " falling" → 0.05

Effect of temperature, taking these as the probabilities at 1.0:
- Below 1.0: the distribution sharpens, so " blue" dominates.
- At 1.0: the probabilities stay as listed.
- Above 1.0: the distribution flattens, so " falling" becomes a real possibility.

A small simulation, with no API needed:

    probs = {" blue": 0.70, " clear": 0.15, " gray": 0.10, " falling": 0.05}

    for t in [0.2, 0.7, 1.0, 2.0]:
        weights = {token: p ** (1 / t) for token, p in probs.items()}
        total = sum(weights.values())
        print(t, {token: round(w / total, 2) for token, w in weights.items()})

Output:

    0.2 {' blue': 1.0, ' clear': 0.0, ' gray': 0.0, ' falling': 0.0}
    0.7 {' blue': 0.84, ' clear': 0.09, ' gray': 0.05, ' falling': 0.02}
    1.0 {' blue': 0.7, ' clear': 0.15, ' gray': 0.1, ' falling': 0.05}
    2.0 {' blue': 0.47, ' clear': 0.22, ' gray': 0.18, ' falling': 0.13}

Experiment: add 0.01 and 10 to the list, and see what happens at both extremes.

4. Related notes

  • LLM: what it changes the behavior of
  • Tokens: it acts on the next-token probabilities
  • API call: providers that still expose it take it as a parameter on the call

5. Open questions

  • How do top-p and top-k differ from temperature, and can I combine them?
  • Why isn't temperature 0 fully deterministic?

6. Sources


System / User / Assistant Messages

1. Definition

Messages are the role-labeled turns that make up a conversation with an LLM.

  • System messages provide high-level instructions and context for the assistant's behavior.
  • User messages contain specific queries or prompts from users.
  • Assistant messages represent the responses generated by the model based on the previous system or user messages.

2. Why it matters

  • Roles are a design tool.
  • All three roles count toward the context window.

3. Example

Same question, with and without a system message:

    # No system message
    messages=[{"role": "user", "content": "What is a token?"}]

    # With a system message
    system="You are a teacher for beginners. Answer in two sentences, no jargon."
    messages=[{"role": "user", "content": "What is a token?"}]

A multi-turn conversation, where I include the assistant's previous reply:

    response = client.messages.create(
        model="claude-sonnet-5-5",
        max_tokens=300,
        system="You are a teacher for beginners. Answer in two sentences, no jargon.",
        messages=[
            {"role": "user", "content": "What is a token?"},
            {"role": "assistant", "content": "A token is a small chunk of text the model reads and writes."},
            {"role": "user", "content": "Why does it matter for cost?"},
        ],
    )

Experiment: ask the third question alone, then with the full history. Compare the answers. Without the history, the model doesn't know what "it" refers to.

4. Related notes

  • LLM: who reads the messages
  • Context window: all roles count toward it
  • API call: messages and system are fields of the request

5. Open questions

  • How much does the system message actually change behavior compared with putting the same text in the first user message?

6. Sources


API Call

1. Definition

An API call to a model is an HTTP request where you send your messages and settings to a server, and it sends back the model's reply.

2. Why it matters

  • It's how any project interacts with LLMs.
  • Each request is stateless, so you have to resend the whole conversation every time.

3. Example

Request (curl):

    curl https://api.anthropic.com/v1/messages \
      -H "x-api-key: $ANTHROPIC_API_KEY" \
      -H "anthropic-version: 2023-06-01" \
      -H "content-type: application/json" \
      -d '{
        "model": "claude-sonnet-5-5",
        "max_tokens": 1024,
        "system": "You are a helpful assistant that explains things simply.",
        "messages": [
          {"role": "user", "content": "What is a token?"}
        ]
      }'

Same call in Python:

    import anthropic

    client = anthropic.Anthropic()  # reads ANTHROPIC_API_KEY from the environment

    response = client.messages.create(
        model="claude-sonnet-5-5",
        max_tokens=1024,
        system="You are a helpful assistant that explains things simply.",
        messages=[{"role": "user", "content": "What is a token?"}],
    )

    text = next(
        (block.text for block in response.content if block.type == "text"), ""
    )

    print(text)
    print(response.stop_reason)
    print(response.usage.input_tokens, response.usage.output_tokens)

Trimmed response:

    {
      "id": "msg_01...",
      "role": "assistant",
      "content": [{"type": "text", "text": "A token is a small chunk of text..."}],
      "stop_reason": "end_turn",
      "usage": {"input_tokens": 25, "output_tokens": 80}
    }

Experiment: set max_tokens to 10 and watch stop_reason change. Then deliberately use a wrong API key and read the error that comes back.

4. Related notes

5. Open questions

  • What should my code do when a call fails or hits a rate limit?

6. Sources


Structured output

1. Definition

A technique to get the model to reply in a predictable, machine-readable shape, usually JSON.

2. Why it matters

  • It's the format to rely on when saving to a database, sending to an API, etc.

3. Example

Same fact, two possible free-text replies:

    Ana is 31 years old.
    The person's name is Ana, and she's 31.

With structured output, the shape is always the same:

    {"name": "Ana", "age": 31}

The native way, using a Pydantic model as the schema:

    from pydantic import BaseModel

    class Person(BaseModel):
        name: str
        age: int

    response = client.messages.parse(
        model="claude-sonnet-5-5",
        max_tokens=1024,
        messages=[{"role": "user", "content": "Ana is 31 years old."}],
        output_format=Person,
    )

    print(response.parsed_output)  # name='Ana' age=31

Experiment: drop the schema, ask for JSON in the prompt instead, and run json.loads() on the reply a few times. See how often it breaks.

4. Related notes

  • Tools: tools also take a schema, which describes a function's arguments rather than the reply
  • API call: the schema travels with the request
  • Runnable / LCEL: output parsers turn the reply into data as one step of a chain

5. Open questions

  • What happens when the output fits the schema but the values are wrong?

6. Sources


Tools

1. Definition

A way to let your model ask your code to run something and use the result in its reply.

It's a mechanism that allows an AI model to generate structured requests, typically in JSON, to invoke external functions or APIs. There are two ways to invoke a tool:

  • Automatic tool invocation: the LLM decides whether it makes sense to call a tool or not. This is the standard choice.
  • Forced tool invocation: the system developer sets the model to always use certain tools. This is ideal for deterministic pipelines, such as structured data extraction. Some models, including Claude Sonnet 5.5 and Opus 5.5, don't support this.

2. Why it matters

  • LLMs predict plausible text, so they're unreliable at exact computation.
  • Using tools expands the power and usefulness of the model.
  • The invocation mode decides whether the model or the developer chooses when a tool runs.

3. Example

The model doesn't run anything itself. It asks, my code runs the function, and I send back the result:

    tools = [{
        "name": "multiply",
        "description": "Multiply two numbers.",
        "input_schema": {
            "type": "object",
            "properties": {"a": {"type": "number"}, "b": {"type": "number"}},
            "required": ["a", "b"],
        },
    }]
    messages = [{"role": "user", "content": "What is 48213 * 9177?"}]

    response = client.messages.create(
        model="claude-sonnet-5-5",
        max_tokens=300,
        tools=tools,
        messages=messages,
    )

    # response.stop_reason is "tool_use": the model is asking for the function
    call = next(block for block in response.content if block.type == "tool_use")
    result = call.input["a"] * call.input["b"]  # my code does the math

    messages.append({"role": "assistant", "content": response.content})
    messages.append({
        "role": "user",
        "content": [{"type": "tool_result", "tool_use_id": call.id, "content": str(result)}],
    })

    final = client.messages.create(
        model="claude-sonnet-5-5",
        max_tokens=300,
        tools=tools,
        messages=messages,
    )
    text = next(block.text for block in final.content if block.type == "text")
    print(text)

Experiment: ask the same question without the tool and compare the answer with the exact result.

See more examples in n8n's LLM Tool Calling: How It Works and How To Implement It.

4. Related notes

5. Open questions

  • How does the model decide which tool to call, and what if it picks the wrong one?
  • What's the difference between passing a Python function directly and using the @tool decorator?

6. Sources


Runnable / LCEL: composing steps

1. Definition

A Runnable is LangChain's standard interface for a component, and LangChain Expression Language (LCEL) is the | syntax for composing Runnables into a chain. Prompts, models, and output parsers are all Runnables.

2. Why it matters

  • All processes are like pipelines. Composing these steps represents an end-to-end pipeline in an AI context.

3. Example

A three-step chain, composed with the | operator:

    from langchain_anthropic import ChatAnthropic
    from langchain_core.output_parsers import StrOutputParser
    from langchain_core.prompts import ChatPromptTemplate

    prompt = ChatPromptTemplate.from_template("Explain {topic} in one sentence.")
    model = ChatAnthropic(model="claude-sonnet-5-5")
    parser = StrOutputParser()

    chain = prompt | model | parser

    print(chain.invoke({"topic": "tokens"}))

Each step's output is the next step's input:

    {"topic": "tokens"} → prompt → messages → model → AI message → parser → string

Experiment: remove the parser from the chain and print the result. It's now an AI message object instead of a string.

4. Related notes

5. Open questions

  • What does LCEL give me that a plain Python function doesn't?

6. Sources


The agent loop

1. Definition

The agent loop is a series of steps in which the model repeatedly decides whether to call a tool or give the final answer, and it keeps looping until it produces a final answer or a stop condition is reached.

2. Why it matters

  • Agents solve problems where the steps can't be known in advance. If the steps are fixed and predictable, an agent may be the wrong choice, and the same goes for low-value tasks.

3. Example

See the examples on LangChain's Agents page.

4. Related notes

  • Tools: what the model decides whether to call on each step
  • API call: each step of the loop is a new call

5. Open questions

  • N/A

6. Sources