What if an AI agent could build its own API tools? Meta-tooling with Python, OpenAPI, Strands Agents, and Bedrock

Most AI agent demos start with a comfortable assumption: we, the developers, already know which tools the agent will need. We write a get_customer tool, a create_ticket tool, maybe a query_database tool, and then the model decides when to use them.

That works, but real systems are not always that tidy. Internal APIs change. Teams expose new endpoints. A support agent may need to operate a service it has never seen before. So I wanted to test a slightly different idea:

What if the agent did not start with business tools at all?

That is my PoC. A small command-line application where a Strands agent starts only with meta-tools. It can inspect an OpenAPI contract, generate Python tools for the API operations, load those tools at runtime, and then use them to complete a task.

The example API is deliberately simple: a tiny commerce support API with delayed orders and discount vouchers. The interesting part is not the API. The interesting part is the sequence:

  1. Read the OpenAPI spec.
  2. Generate Strands-compatible Python tools.
  3. Load those tools dynamically.
  4. Use the newly created tools to solve the user request.

The agent begins without list_orders or create_discount_voucher. It builds them when it discovers that it needs them.

The idea

The local demo task is:

poetry run toolsmith agent \
"Find delayed orders in Spain and create a 15 percent discount voucher for each affected customer."

The agent receives the OpenAPI file and a tools directory. Its first tools are not business tools. They are tools for creating tools:

@tool
def inspect_openapi(spec_path: str) -> dict:
"""Inspect an OpenAPI file and list the operations available for tool generation."""
return describe_openapi(spec_path)
@tool
def generate_openapi_tools(spec_path: str, output_dir: str) -> dict:
"""Generate Strands Python tools from an OpenAPI file into the selected directory."""
generated = generate_tools(spec_path, output_dir)
return {
"created": [
{
"name": item.name,
"path": str(item.path),
"method": item.operation.method,
"api_path": item.operation.path,
}
for item in generated
]
}

The third tool is Strands’ load_tool. Once the Python files exist, the agent can load them into its own runtime and use them like any other tool.

This is the meta-tooling pattern: the agent does not only use capabilities; it can create new capabilities while it is running.

Project structure

The project is intentionally small:

src/
  meta/
    cli.py
    settings.py
    demo_api.py
    agent/
      factory.py
      prompts.py
    meta_tooling/
      generator.py
      models.py
      openapi.py
    runtime/
      http_client.py
examples/
  support/
    openapi.yaml
tests/

The responsibilities are explicit:

  • meta_tooling/openapi.py reads the OpenAPI contract and extracts operations.
  • meta_tooling/generator.py writes Python files that expose those operations as Strands tools.
  • runtime/http_client.py contains the deterministic HTTP boundary used by generated tools.
  • agent/factory.py wires Strands Agents with Bedrock and the meta-tools.
  • demo_api.py runs a tiny local HTTP API for the example.
  • cli.py exposes the commands.

The OpenAPI contract

The demo contract describes three operations:

paths:
/orders:
get:
operationId: list_orders
summary: List customer orders
parameters:
- name: status
in: query
required: false
schema:
type: string
- name: country
in: query
required: false
schema:
type: string
/orders/{order_id}:
get:
operationId: get_order
summary: Get one order
/discount-vouchers:
post:
operationId: create_discount_voucher
summary: Create a discount voucher for a customer order

The CLI can inspect it without calling an LLM:

poetry run toolsmith inspect

Output:

Support Commerce API (1.0)

Tool                     Method   Path                 Summary
list_orders              GET      /orders              List customer orders
get_order                GET      /orders/{order_id}   Get one order
create_discount_voucher  POST     /discount-vouchers   Create a discount voucher for a customer order

Generating tools

The generator turns each OpenAPI operation into a Python module. This can also be run directly:

poetry run toolsmith build

It creates files like:

generated_tools/
  __init__.py
  list_orders.py
  get_order.py
  create_discount_voucher.py

The generated list_orders tool looks like this:

from __future__ import annotations
from typing import Any
from strands import tool
from meta.runtime.http_client import request_api
@tool
def list_orders(status: str | None = None, country: str | None = None) -> dict[str, Any]:
"""List customer orders
Generated API tool for `GET /orders`.
Returns customer orders filtered by status or country.
"""
return request_api(
method="GET",
path_template="/orders",
path_params={},
query_params={"status": status, "country": country},
json_body=None,
)

There is no magic in the HTTP call itself. It is just Python:

def request_api(
*,
method: str,
path_template: str,
path_params: dict[str, Any] | None = None,
query_params: dict[str, Any] | None = None,
json_body: dict[str, Any] | None = None,
timeout: int = 15,
) -> dict[str, Any]:
base_url = os.getenv("TOOLSMITH_API_BASE_URL", "http://127.0.0.1:8010")
path = _build_path(path_template, path_params or {})
query = {key: value for key, value in (query_params or {}).items() if value is not None}
response = requests.request(
method=method,
url=f"{base_url.rstrip('/')}{path}",
params=query,
json=json_body,
timeout=timeout,
)

That boundary is important. The LLM can decide which operation it needs, but the generated tool uses deterministic Python for the actual API call.

The agent

The agent is created with Bedrock and Strands:

def create_agent(settings: Settings) -> Agent:
session_kwargs = {"region_name": settings.aws_region}
if settings.aws_profile:
session_kwargs["profile_name"] = settings.aws_profile
boto_session = boto3.Session(**session_kwargs)
model = BedrockModel(
model_id="us.anthropic.claude-sonnet-4-5-20250929-v1:0",
boto_session=boto_session,
temperature=0.2,
)
return Agent(
system_prompt=TOOLSMITH_SYSTEM_PROMPT,
model=model,
tools=[inspect_openapi, generate_openapi_tools, load_tool],
)

The prompt is intentionally strict:

TOOLSMITH_SYSTEM_PROMPT = """
You are an API toolsmith agent.
You start with meta-tools, not with business tools. Your job is to inspect an
OpenAPI contract, generate Python tools for the useful operations, load those
tools, and then use them to answer the user's question.
Rules:
- Inspect the OpenAPI file before generating tools.
- Generate tools only inside the configured generated tools directory.
- Load generated tools before trying to call them.
- Prefer deterministic API calls through generated tools over guessing.
- Explain briefly which tools you created and why they were useful.
"""

The model-driven part is deciding the workflow. The deterministic part is parsing OpenAPI, writing the Python modules, and making HTTP requests.

Running the demo

Install dependencies with Poetry:

poetry install

Run the local demo API in one terminal:

poetry run toolsmith demo-api

The API listens on:

http://127.0.0.1:8010

In another terminal, inspect the OpenAPI contract:

poetry run toolsmith inspect

Generate tools manually:

poetry run toolsmith build

Or let the agent do the generation and tool loading:

AWS_PROFILE=sandbox AWS_REGION=us-west-2 \
poetry run toolsmith agent \
"Find delayed orders in Spain and create a 15 percent discount voucher for each affected customer."

The expected behavior is:

  1. The agent inspects examples/support/openapi.yaml.
  2. It generates list_orders, get_order, and create_discount_voucher.
  3. It loads the generated tools.
  4. It calls list_orders(status="delayed", country="ES").
  5. It creates vouchers for the affected orders.

The generated tool can also be called directly from Python:

from pathlib import Path
from importlib.util import module_from_spec, spec_from_file_location
from meta.meta_tooling.generator import generate_tools_from_openapi
out = Path("generated_tools")
generate_tools_from_openapi("examples/support/openapi.yaml", out)
spec = spec_from_file_location("list_orders", out / "list_orders.py")
module = module_from_spec(spec)
spec.loader.exec_module(module)
print(module.list_orders(status="delayed", country="ES"))

Output:

{
"status_code": 200,
"ok": True,
"body": {
"orders": [
{"id": "ORD-1001", "customer": "Ada Lovelace", "country": "ES", "status": "delayed", "total": 189.9, "days_late": 4},
{"id": "ORD-1003", "customer": "Barbara Liskov", "country": "ES", "status": "delayed", "total": 244.0, "days_late": 7},
]
},
}

Tech stack

  • Python 3.13 with Poetry.
  • Strands Agents for agent orchestration and dynamic tool loading.
  • AWS Bedrock for the LLM runtime.
  • OpenAPI as the API capability source.
  • Click for the CLI.
  • Rich for terminal output.
  • Requests for deterministic HTTP calls.
  • pytest for tests.

A couple of notes

This is not an API gateway and it is not a replacement for careful integration design. It is a PoC for a specific agentic pattern: start with a narrow set of trusted meta-tools, then allow the agent to expand its own capabilities from a machine-readable contract.

The generated tools are intentionally boring. That is a feature. The agent can be creative in planning, but the API calls should be predictable, inspectable, and testable.

The next step would be to add stronger guardrails: endpoint allowlists, authentication strategies, versioned tools, and a review step before loading newly created code.

And that’s all. Full source code available on my GitHub.

Building a virtual museum with AI, Python, Blender and Three.js

I had never used Blender before starting this project. I knew what it was, of course, but if you had asked me to create a room, place paintings on the walls, configure materials and lights, add cameras and export everything as a GLB file, I would not have known where to start.

I have been exploring AI agents for a while, especially the idea of using an LLM not only to generate text, but to operate existing tools and create real artifacts. What if I could describe an exhibition in natural language and use AI to help me turn it into a museum that people can actually visit?

The project starts with a local export of my Notion database about nineteenth-century mythological painting (I like this kind of things). It contains artwork metadata, images and my own notes. Claude on AWS Bedrock researches that collection and proposes an exhibition. Python validates the proposal and translates it into a deterministic build specification. Blender creates the rooms, paintings, lights and cameras, and exports the result. Finally, a Three.js application turns the GLB into an interactive museum in the browser.

The interesting part for me is not the final 3D room. The interesting part is the path from an idea expressed in natural language to a real .blend file created by a tool I had never used.

Two different AI layers

There are actually two AI stories in this project.

The first one happened while building it. I used an AI coding agent as a collaborator to explore Blender’s Python API, create geometry, work with materials, configure lighting, inspect GLB metadata and iterate over the result. I was still defining the architecture and deciding what the system should do, but I did not need to become a Blender expert before producing something with Blender.

The second AI layer lives inside the application. Claude acts as a curator. It researches a bounded collection of paintings, chooses a narrative, decides which works belong together, organizes one or two rooms and defines the visitor route.

These two layers are related, but they are not the same thing. One helped me build the tool. The other is part of the tool’s runtime.

In both cases the useful pattern is the same: AI handles intent and exploration; deterministic software owns the dangerous and boring details.

The architecture

The complete flow looks like this:

I deliberately split the workflow into stages. Asking an LLM to create a museum in one giant step would be easy to demo and difficult to trust. Here every stage leaves an inspectable artifact behind: the imported catalog, the proposal, the accepted plan, the Blender build specification, the GLB, the content bundle and the final manifest.

If something is strange in the museum, I can see where it became strange.

AI decides what. Code decides how.

The most important decision in the project was not to ask the LLM to generate arbitrary Blender Python code.

That approach would look impressive in a short demo, but it would make validation almost impossible. A model could invent an API, place a painting outside the room, create overlapping frames, forget a camera or return code that only works with a particular Blender version.

Instead, the model returns a MuseumPlan. It is a declarative contract describing curatorial intent:

class MuseumPlan(BaseModel):
    schema_version: Literal["1.0"] = "1.0"
    plan_version: int
    title: str
    thesis: str
    request: ExhibitionRequest
    entrance_room_id: str
    rooms: list[RoomPlan]
    connections: list[RoomConnection]
    route: list[RouteStop]

A room contains dimensions, atmosphere and artwork placements. A placement contains a wall, a normalized position, a display width, a center height and a lighting intention. The plan says that a painting belongs on the east wall at a certain position. It does not say which Blender mesh operations are required to put it there.

AI-curated virtual museum

That distinction is the boundary:

The AI decides what exhibition to create. Deterministic code decides how to create valid Blender geometry.

Pydantic checks the shape of the response. Additional validators check identities, room topology, route continuity, image availability, painting dimensions, corner clearances, door clearances and frame overlaps. The LLM response is treated as untrusted JSON, even when it comes from a model that has just been given the schema.

If the response is invalid, the application can send the validation issues back to the model for a bounded repair attempt. If it remains invalid, nothing is published.

Using Blender as a build engine

Blender runs as an external application in background mode. Python prepares a non-executable JSON build specification and starts Blender with a small entrypoint:

return [
    executable,
    "--factory-startup",
    "--background",
    "--python",
    str(entrypoint),
    "--",
    "--build-spec",
    str(build_spec),
    "--output",
    str(output),
]

Inside Blender, the bpy runtime resets the default scene, creates the architecture, builds the frames, loads the artwork textures, configures room and focal lights, adds the visitor camera and attaches semantic metadata to the objects.

def build_complete_scene(spec, build_spec_path):
    reset_complete_scene()
    themes = create_exhibition_themes(spec)
    materials = create_museum_palette(spec, themes)
    collections, connection = create_complete_architecture(spec, materials)

    create_complete_scenography(spec, collections, connection, materials, themes)

    for room in spec["rooms"]:
        add_room_fill(room, collections[room["id"]])

    for painting in spec["paintings"]:
        add_complete_painting(...)
        add_complete_spot(...)

    add_visitor_spawn(...)

The scene is validated from inside Blender before it is exported. Expected object names, counts, artwork identities, route markers, colliders and geometry constraints are checked against the build specification. Only then does the pipeline save the .blend source and export the browser-ready .glb.

The build also happens in a temporary staging directory. The final output is replaced only after every artifact and hash has been verified. A failed Blender process cannot leave half a museum in the publication directory.

This is more machinery than a weekend 3D room normally needs. It is also the part that turns the experiment from “the model produced something once” into a repeatable software pipeline.

The curator does not see everything

The local catalog contains 270 records, with 190 eligible nineteenth-century works. Sending the complete database and every image to an LLM would be expensive and unnecessary.

Discovery is divided into two steps. First, Claude receives a compact projection of the eligible catalog and selects a bounded candidate set. Then it receives detailed metadata and reviews only for those candidates and creates the exhibition proposal.

The model never receives image bytes, local paths, import diagnostics or the complete Notion export. It must select existing stable artwork IDs, and the resulting plan is checked against the real local catalog.

The proposal is not accepted automatically either. Discovery creates an immutable review artifact. I can inspect it, ask for a refined alternative and explicitly accept only the version I want. Acceptance creates a draft; building and publishing are separate operations.

This separation matters because generation, acceptance and execution are different decisions. An LLM response should not become published state just because it parsed correctly.

From Blender to the browser

I did not want to bypass Blender by recreating the museum directly with Three.js. Blender is the authoring and build tool; Three.js is the visitor runtime.

The GLB contains more than visible meshes. Paintings carry stable artwork IDs, room IDs and route order. Walls and floors carry collision metadata. The scene contains route stops, label anchors, lights and a visitor spawn camera. The web application reads those semantics instead of guessing them from object positions or filenames.

Before enabling the Enter museum button, the browser verifies the exhibition manifest, content, build report and GLB identities. Once loaded, the visitor can walk through the rooms, follow the curatorial route, inspect each artwork, read the complete review and switch between Spanish and English content.

The application is static. There is no backend, database, AWS credential, Blender process or LLM call in the browser. Each published exhibition is a self-contained bundle that can be hosted with ordinary static files.

This is a PoC, not a general 3D authoring platform.

Rooms are rectangular. Exhibitions contain one or two rooms and up to 24 artworks. The scenography is generated from a small set of themes. Blender operations are intentionally limited, and the visual result is obviously not the work of an experienced 3D artist.

The source catalog is also very specific: nineteenth-century mythological painting, exported from my own Notion database. A different collection would probably expose different assumptions in the curator and the spatial rules.

But the experiment is not really about museums. A museum is only a good excuse to test a broader pattern:

The professional tool could be Blender, a DAW, a CAD application, a video editor or a spreadsheet engine. The model does not need to replace the tool. It can provide a new interface to it.

What I learned without learning Blender

I still would not describe myself as a Blender user. If you open Blender and ask me to model this museum manually, I will probably get lost in the interface.

But I now understand enough of its concepts to work with it as an automation platform: scenes, collections, meshes, materials, lights, cameras, custom properties, headless rendering and GLB export. More importantly, I have a repeatable system that creates something I could not have created before.

AI did not make the engineering disappear. In fact, most of the project is about contracts, validation, boundaries, deterministic builds and failure handling. What AI changed was the entry cost. I could start with the result I wanted and explore an unfamiliar tool from there, instead of spending weeks learning the interface before knowing whether the idea was worth building.

That is the part I find interesting. AI is not only a faster way to write the software we already know how to write. It can be a bridge into tools and domains that were previously outside our reach.

Demo: https://gonzalo123.github.io/museum

Full source code is available in my GitHub.

Turning a code repository into a video podcast with Python, Remotion, AWS Bedrock and Polly

I like building small proof-of-concept projects. Writing the code is usually the fun part; explaining the project afterwards is another story. Normally, I write a post like this one, but this time I wanted to try something different: a narrated video explaining the code and the project’s architecture. Something like a video podcast with two synthetic voices, accompanied by a presentation of the project.

That’s the idea.

Repodcast is a command-line application that receives a local folder or a public GitHub repository and creates a narrated technical video about it. It scans the source, asks Claude on AWS Bedrock to write the episode, generates two voices with Amazon Polly, creates subtitles and renders the final MP4 with Remotion.

This is the kind of command I wanted to run:

repodcast build gonzalo123/plainnews

And this is one of the generated frames:

Of course, giving a repository to an LLM and asking for a video sounds easy. The interesting part starts when we want the video to describe the real code instead of producing a generic summary with a nice gradient.

The pipeline

The complete flow looks like this:

I deliberately split the process into stages. A single giant prompt that reads a GitHub URL and somehow returns a video would be difficult to inspect and even more difficult to test. Here each stage leaves something useful behind: repository.json, episode.json, script.md, one audio file per scene, SRT, WebVTT and finally the video.

If something is strange in the result, I can see where it became strange.

First, freeze the repository

Repodcast accepts local directories, GitHub URLs and the short owner/repository form. Remote repositories are shallow-cloned into a local cache. I do not need the complete Git history to explain the current code, and I definitely do not want to download Git LFS objects or recurse into every submodule for a two-minute video.

The resolver keeps the exact commit SHA together with the generated episode:

return ResolvedRepository(
    path=checkout.resolve(),
    display_name=repository,
    source_url=url,
    requested_ref=ref,
    commit_sha=repo.head.commit.hexsha,
)

That SHA is more important than it looks. Repositories change. Without it, a video can show a diagram and a code fragment that no longer exist by the time somebody watches it. Repodcast renders the analyzed commit in the architecture scene, so the explanation has a concrete source of truth.

Less context, but better context

Sending the whole repository to Claude is not a serious strategy. Apart from cost and context limits, a repository contains a lot of material that is useless for this task: virtual environments, compiled assets, generated files, lock internals and binaries.

The scanner is deterministic Python. It applies .gitignore, excludes well-known generated directories, rejects large or binary files and detects the stack from manifests and extensions. Then it selects a small group of interesting files: package manifests, entry points, source files and a couple of tests.

The result is a typed RepositorySummary, not an unstructured dump. The model receives the tree, detected technologies, entry points and bounded excerpts. Claude does the part where an LLM is useful —finding the story in the code— while Python owns file selection and limits.

This separation also makes the scanner testable without AWS credentials. An LLM should not decide whether node_modules belongs in the prompt.

Asking for an episode, not a summary

My first version of the prompt produced something technically correct and completely boring. It listed the technologies, repeated the README and finished with the usual optimistic conclusion. That is not how I explain a weekend experiment.

The current prompt asks for a small narrative arc: a concrete irritation, the idea, the execution path, one revealing implementation detail, an honest limitation and a final observation. It also asks for a conversation between a host and a guest. The guest is useful because it can ask the obvious question just before the answer needs to appear.

The response is strict JSON validated with Pydantic. A slide can contain bullets, a real code excerpt or a small architecture graph, but it always needs spoken content. Flow edges must reference existing nodes and slide indexes must be sequential.

class Episode(BaseModel):
    model_config = ConfigDict(frozen=True)

    title: str = Field(min_length=1)
    target_minutes: int = Field(gt=0)
    slides: list[Slide] = Field(min_length=1)
    source_commit: str | None = None

LLMs do not always respect schemas, even when we ask politely. If Bedrock returns invalid JSON, Repodcast sends the validation error back once and asks Claude to repair the complete object. The second response goes through exactly the same validation. There is no try: whatever except: continue hidden in the pipeline.

The requested duration is also normalized in Python. The model proposes the relative duration of each scene, but code makes sure the total matches the number of minutes requested.

Two voices and the problem with time

Each dialogue turn has a speaker identifier, host or guest. Amazon Polly maps them to different neural voices and generates one small MP3 per turn. FFmpeg concatenates the fragments with a short silence between speakers and a small tail at the end of the scene.

Originally I trusted the duration written in episode.json. That worked until a voice needed longer than expected and the next scene started while the last word was still playing. Now ffprobe measures the real audio and those durations become the timing source for Remotion and the subtitle files.

slides = [
    slide.model_copy(
        update={"duration_seconds": max(1, math.ceil(_audio_duration(audio)))}
    )
    for slide, audio in zip(episode.slides, audio_files, strict=True)
]

It is a small detail, but it changes the pipeline from “I hope these numbers align” to “the rendered timeline follows the generated media”.

Rendering code as code

The visual part is a React application rendered with Remotion. There are three main scene types: covers, architecture flows and code scenes. Code is highlighted with Prism and revealed line by line. Architecture nodes arrive as structured data, not as Mermaid text invented at render time, and each node can point to a real path in the repository.

The Python side copies the audio into Remotion’s public directory, writes the episode props and launches the composition. React owns layout and animation; Python owns the workflow and the data contract between stages.

This boundary is useful. I can change the complete visual identity without touching repository analysis, Bedrock or Polly. I can also replace Polly with another text-to-speech provider without changing a single React component.

Building the pipeline without spending money every time

Testing an application like this against real AI and text-to-speech services would be slow, expensive and unpredictable. Repodcast has deterministic fake adapters for both.

The fake AI produces a seven-scene episode with a flow diagram and a code example. The fake Polly adapter creates silent audio with FFmpeg. That means the same orchestration used by the real command can generate every intermediate artifact locally, including a real MP4, without calling AWS.

The tests exercise repository filtering, GitHub source resolution, prompt rules, JSON repair, SSML escaping, audio concatenation, duration measurement, subtitles and the Remotion contract. At the moment there are 19 tests. Ruff and mypy in strict mode are also part of the checks.

This does not prove that Claude will always write a good story. It proves that a strange story cannot silently break the media pipeline.

What is still deliberately simple

Repodcast is a PoC. The scanner does not build an AST or a call graph. It chooses representative files using small heuristics, which works surprisingly well for compact repositories but will miss important relationships in a large monorepo.

Subtitles are synchronized per scene, not per word. The architecture graph is intentionally limited to six nodes because more nodes might be more accurate, but they produce a worse video. The final quality still depends on the source repository and on the model choosing the right thread to follow.

There is also an uncomfortable recursive detail: the repository may already contain a README written to explain the project. Repodcast reads that README to create another explanation of the same project, now with two synthetic voices and animated code. This is clearly over-engineered.

But it is also the kind of over-engineering I like. The result is not a presentation about a possible system. It is a working pipeline that takes a real commit, generates inspectable artifacts and ends with a video that can be watched.

The LLM writes the story. Python keeps it attached to the repository. Polly gives it voices. Remotion makes it visible.

Example video of the generated podcast for this repository:

Watch the generated podcast video

And that’s all. Full source code is available in my GitHub.

A Pokédex in the terminal, but agentic, with LangChain

This project is a spin-off of a small that we started during the Katayuno on June 20, 2024. Katayuno is a Saturday morning programming kata where the conversation, debate and retrospective normally matter more than finishing the exercise. That time, helped by AI, almost every team actually finished.

My version used React, TypeScript and Vite, with PokeAPI directly from the browser React Pokédex. It had the usual things: list, details, comparison, shiny sprites and even a small battle mode.

But that application was deterministic. Click here, fetch this, render that.

This time I wanted something slightly different. I wanted a Pokémon professor in my terminal. Not a chatbot that hallucinates Pokémon facts. A small agent that uses PokeAPI as the source of truth and an LLM only as the reasoning layer.

The LLM is not the database. The LLM is the reasoning layer.

I don’t want the model to remember Pikachu’s Speed. I want the model to ask the tool.

The idea

The project is a Python 3.13 CLI built with Click, Rich, httpx, Pydantic and LangChain. AWS Bedrock is the default model provider, but it lives behind one small factory, so changing the provider does not require changing the PokeAPI code.

The flow is deliberately boring:

The agent cannot fetch arbitrary URLs. It only receives five tools:

  • get_pokemon
  • get_type
  • compare_pokemon
  • get_evolution_chain
  • get_type_matchup

This is not magic. The tools are normal Python functions returning small, normalized dictionaries. LangChain decides when to call them and the model reasons with their output.

The CLI

The deterministic commands work without AWS credentials:

uv run python -m cli pokemon pikachu
uv run python -m cli search charizrad
uv run python -m cli compare charizard blastoise

The commands involving reasoning can use Bedrock:

uv run python -m cli compare charizard blastoise --explain
uv run python -m cli battle charizard venusaur
uv run python -m cli ask "Which Pokémon is faster, Gengar or Alakazam?"

The boring deterministic part

Getting a Pokémon by name does not need an LLM. This is one of those examples where using an LLM for everything would be a mistake.

The PokeAPI adapter turns the large API response into a small Pydantic model:

class PokemonSummary(BaseModel):
id: int
name: str
types: list[str]
height: int = Field(description="Height in decimetres, as returned by PokeAPI")
weight: int = Field(description="Weight in hectograms, as returned by PokeAPI")
base_experience: int | None
stats: list[PokemonStat]
abilities: list[str]
def stat(self, name: str) -> int:
return next((stat.value for stat in self.stats if stat.name == name), 0)
@property
def total_stats(self) -> int:
return sum(stat.value for stat in self.stats)

The pokemon command fetches the data and Rich renders it. The compare command fetches two Pokémon and compares their base stats with plain Python.

The deterministic part is boring. And that is good.

It is easy to test and easy to understand when something goes wrong.

Typos are deterministic too. A failed lookup downloads the compact species-name index from PokeAPI and keeps it in memory for the current process. Python’s SequenceMatcher then ranks names locally:

uv run python -m cli search charizrad
charizard 89% similarity

Normal commands use the same matcher when PokeAPI returns a 404:

╭──────────────────────────── Pokémon not found ─────────────────────────────╮
│ I couldn't find 'charizrad'.                                               │
│                                                                            │
│ Best match: Charizard  (89% similarity)                                    │
╰────────────────────────────────────────────────────────────────────────────╯
Press Enter to use Charizard, type another name, or q to cancel:

Pressing Enter accepts the suggestion. Typing another name retries the lookup, and q cancels. I deliberately do not ask the LLM and I do not silently replace the name. In non-interactive scripts the command keeps returning a normal error instead of waiting forever for input.

The agentic part

The agentic part starts when the question is no longer a direct API call.

For example:

Which Pokémon is faster, Gengar or Alakazam?
Can Pikachu beat Squirtle?
What are Dragonite weaknesses?
Tell me the evolution chain of Eevee

Here LangChain’s create_agent receives the Bedrock chat model, the controlled tools and a system prompt:

agent = create_agent(
model=create_chat_model(settings),
tools=build_tools(client),
system_prompt=SYSTEM_PROMPT,
)
result = agent.invoke(
{"messages": [{"role": "user", "content": question}]}
)

The important part is not create_agent. The important part is the boundary. Facts come from tools. The model decides which facts it needs and explains the result.

The prompt says it explicitly:

You must never invent Pokémon data. Use the available tools to retrieve facts
from PokeAPI before answering factual questions.

The LLM is not the database. The LLM is the reasoning layer.

A prompt is not a security boundary, of course. That is why the agent only gets small, explicit tools and never gets a generic HTTP client.

Tools

The tools are created around the PokeAPI client. This makes them small and also makes tests simple because I can inject an httpx.MockTransport.

@tool
def get_type_matchup(
attacker_type: str,
defender_types: list[str],
) -> dict[str, Any]:
"""Calculate the damage multiplier for one attacking type against defender types."""
return client.get_type_matchup(
attacker_type,
defender_types,
).model_dump()

I don’t return the complete PokeAPI JSON. Agents work better when tools return the information needed for the task instead of a small novel containing every field an API has accumulated over the years.

Structured output

The battle command is intentionally limited. It is not a competitive Pokémon simulator. It considers the Pokémon types, type multipliers, base Speed, offensive stats and defensive stats. It does not consider moves, levels, abilities, held items, natures, weather or battle format.

The local heuristic first creates a valid prediction:

class BattlePrediction(BaseModel):
winner: str
confidence: float = Field(ge=0, le=1)
reasons: list[str]
caveats: list[str]
recommended_attack_types: list[str]

In Bedrock mode I pass that prediction and the normalized PokeAPI facts to a second LangChain agent using response_format=BattlePrediction. The result is validated by Pydantic instead of parsing an optimistic blob of JSON from a string.

agent = create_agent(
model=create_chat_model(settings),
tools=[],
system_prompt=BATTLE_PROMPT,
response_format=ToolStrategy(BattlePrediction),
)

The model can improve the explanation, but it does not get permission to invent a Flamethrower, an item or a hidden ability.

I use LangChain’s ToolStrategy explicitly here. Bedrock’s native structured output currently rejects some numeric JSON Schema constraints generated by Pydantic, such as the minimum and maximum for confidence. Tool calling still returns a validated BattlePrediction without depending on that provider limitation.

Rich output

Click handles the command-line interface and Rich handles tables, panels, colours and stat bars.

Rich is not needed, but terminals should still look decent.

The visual layer is also separate from the data layer. render.py receives Pydantic models. It does not know how PokeAPI works and it does not call the LLM.

When not to use the LLM

I think this is the useful part of the experiment.

There is no model call in:

  • pokemon
  • compare without --explain
  • typo suggestions and Pokémon name search
  • type multiplier calculation
  • the first battle prediction
  • tests

An LLM is useful when the user asks an open question and the application needs to choose tools, combine facts and explain a conclusion. It is not useful for adding six integers or reading Pikachu’s height from JSON.

Using less AI here makes the agentic part easier to see.

Running the project

I normally use Poetry. For this small project I wanted to try uv, so it owns Python installation, dependency resolution, command execution and the lock file. I am not starting a package-manager religion here. It’s just a test.

git clone https://github.com/gonzalo123/pokemon_cli.git
cd pokemon_cli

uv python install 3.13
uv sync --extra dev

That is enough for the deterministic commands.

For AWS Bedrock:

uv sync --extra dev --extra bedrock
cp .env.example .env

Then configure the environment:

AWS_PROFILE=sandbox
AWS_REGION=eu-west-1
BEDROCK_MODEL_ID=global.anthropic.claude-sonnet-4-6

No AWS key is stored in the repository. The AWS SDK uses the selected profile or its normal credential chain.

Now the examples:

uv run python -m cli pokemon pikachu
uv run python -m cli compare charizard blastoise
uv run python -m cli battle charizard venusaur
uv run python -m cli ask \
  "Which Pokémon is faster, Gengar or Alakazam?"

There is also an installed command:

uv run pokemon-professor pokemon pikachu

Things I liked

The separation is small but useful. PokeAPI owns the facts, Pydantic owns the shape, Python owns deterministic calculations, Rich owns presentation and the LLM owns a narrow reasoning task.

Things that still feel awkward

The battle result is only a heuristic. A real battle model needs moves, abilities, levels, items, natures, generation rules, and probably much more. Adding all that while pretending the result is still simple would be dishonest.

I’m not a Pokémon expert. I have only been playing with the Pokémon API because of the Katayuno. This is just an excuse to use AI everywhere, just like we developers seem to be doing these days. Please don’t judge me too harshly.

Final thoughts

Agentic does not mean replacing every function with an LLM call. For me it means giving the model a small set of reliable capabilities and letting it use them when a deterministic route is no longer enough.

The Pokémon facts do not belong in the prompt and they do not belong in the model’s memory. They belong in PokeAPI.

The LLM is not the database. The LLM is the reasoning layer.

And that’s all. Full source code is available in my GitHub account.

What Homer Would Reply to Your Email: A Classical Quote Recommender with Bedrock, RAG, and Strands Agents

What if every email you write could carry the rhetorical weight of Homer? Not as a gimmick, as a real tool that understands the tone, intent, and emotion of your message and finds the classical passage that fits.

That’s the idea behind this PoC: a system that takes a short text (an email, a Slack message, a reply to a tricky thread) and recommends the most fitting quote from classical literature. Right now the corpus is the Iliad and the Odyssey, nearly 5,000 passages of Homer, indexed and searchable by meaning, not just by keywords.

The interesting part isn’t the concept. It’s how the pieces fit together.

The Architecture

The system runs as a FastAPI service with a five-stage pipeline:

The key idea: the two ends of the pipeline (understanding your message and choosing the final quote) use an LLM, but the middle part, finding and ranking candidates, is pure code, no AI involved. That means the core search is predictable and inspectable. Bedrock handles the parts that need language understanding; everything else stays local.

Query Enrichment: Not Just What You Said, But What You Meant

When you search a typical RAG system, you take the user’s text and look for similar documents. Here I do something different: before searching, the system enriches the query with everything it learned during the rhetorical analysis.

class HybridRetriever:
def _build_query(self, text: str, analysis: InputAnalysis) -> str:
parts = [
text,
analysis.summary,
analysis.main_theme,
" ".join(analysis.secondary_themes),
analysis.tone,
analysis.intent,
analysis.dominant_emotion,
analysis.recommended_quote_type,
]
return " ".join(part for part in parts if part).strip()

Say the input is “Thanks for your feedback. I think we can find a middle ground”. Instead of just searching for those words, the system searches for negotiation, conciliatory, guarded optimism, diplomatic and bridge-building, all at once. The search doesn’t just look for passages about feedback. It looks for passages that feel the same way as the message.

Zero-Dependency Vector Store

I deliberately avoided FAISS, Chroma, Pinecone, or any external vector database. The entire search index is a single NumPy matrix:

class NumpyVectorStore:
def search(self, query_vector: np.ndarray, top_k: int) -> list[tuple[str, float]]:
query = np.asarray(query_vector, dtype=np.float32)
scores = self.vectors @ query
indices = np.argsort(scores)[::-1][:top_k]
return [(self.ids[index], float(scores[index])) for index in indices]

One line does the work: scores = self.vectors @ query. This is a matrix multiplication that compares the query against every passage in the corpus at once. Because the embedding model produces normalized vectors, this simple operation gives you cosine similarity, a standard way to measure how close two texts are in meaning.

The full index is a matrix of 4,841 rows (one per passage) and 384 columns (one per dimension of the embedding). It loads in milliseconds and fits in memory easily. For a corpus of this size, a full-blown vector database would be overkill.

Why all-MiniLM-L6-v2

The model that turns text into numbers (embeddings) is all-MiniLM-L6-v2 from Sentence Transformers. It takes any piece of text and produces a list of 384 numbers that represent its meaning. Texts that say similar things end up with similar numbers, even if they use completely different words.

It’s a small model, only 22 million parameters and 6 layers, but it was trained on over a billion pairs of sentences, so it’s surprisingly good at capturing semantic similarity. It loads in under a second on a regular CPU and processes the entire Homer corpus in a few seconds.

There are bigger models (like all-mpnet-base-v2 with 110M parameters) that would be slightly more precise, but for this project the bottleneck isn’t the embedding quality, it’s how well the query enrichment captures the intent of the message. The small model is more than enough.

The Reranker: Five Signals, Calibrated Weights

After the search phase finds candidate passages using both meaning and keywords, a rule-based reranker scores each one across five signals:

candidate.rerank_score = (
0.45 * candidate.hybrid_score
+ 0.20 * candidate.thematic_fit
+ 0.15 * candidate.tonal_fit
+ 0.10 * candidate.clarity_fit
+ 0.10 * candidate.rhetorical_fit
)

Each signal measures something different: how well the topic matches, whether the tone is right, whether the passage would work rhetorically. The most product-shaped one is clarity_fit. It now favors excerpts that are short, sentence-bounded, and easy to paste into an email without further editing.

That matters because pure semantic similarity misses a very obvious real-world constraint: a 200-word passage from Homer might be relevant, but it’s still useless if what you need is a quotable closing line.

Strands Agents with Structured Output

The analyzer and selector use Strands Agents, an open-source SDK for building AI agents. The key feature I rely on is structured output: instead of asking the LLM to return free text and then parsing it, the SDK forces the model to fill in a typed Python object directly.

agent = Agent(
model=self.model,
system_prompt=(
"You analyze short emails or messages for a rhetorical quote recommender. "
"First infer the input language from the text itself. "
"Return the detected language plus concise, grounded rhetorical analysis."
),
)
result = agent(prompt, structured_output_model=AnalyzerResult)

The AnalyzerResult is a Pydantic model with fields like main_theme, tone, intent, dominant_emotion, and recommended_quote_type. The model doesn’t write prose, it fills in a form. This eliminates a whole category of bugs related to parsing LLM output.

The selector agent works the same way: it receives the ranked candidates as JSON and returns exactly three choices, each with a why_it_fits explanation written in the language of the original message.

Compact Quotes, Not Paragraphs

One thing became obvious very quickly: finding a semantically relevant passage is not the same as finding a quotable one. For emails and short messages, a 70-word block of Homer is basically unusable.

So before the selector sees the candidates, the reranker extracts a compact quote window from each passage, usually one or two sentences, under 32 words. The quote still comes verbatim from the indexed corpus, but the system stops treating the full chunk as the thing you’d actually paste into an email.

That small layer makes a big difference. It biases the output toward something you can actually use without turning your message into a wall of text.

Representative Examples

Here are two representative examples of the kind of output the system is designed to produce.

Example 1

Input

Thanks for your feedback. I do not fully agree with the proposal, but I think we can still find a middle ground and move forward.

Analysis

  • Summary: Polite disagreement looking for a workable compromise
  • Main theme: Negotiation
  • Tone: Conciliatory
  • Intent: Negotiate
  • Dominant emotion: Controlled tension

Recommended quote

But now let each becalm his troubled breast,
Wash, and partake serene the friendly feast.

Homer, The Odyssey

Why it fits

It cools the temperature without sounding weak. The quote shifts the message away from friction and toward calm, shared ground, and continued conversation.

Example 2

Input

We need to stay focused, make a decision, and keep moving even if the road is rough.

Analysis

  • Summary: Call for disciplined action under pressure
  • Main theme: Leadership
  • Tone: Resolute
  • Intent: Persuade
  • Dominant emotion: Focus

Recommended quote

In battle calm he guides the rapid storm,
Wise to resolve, and patient to perform.

Homer, The Odyssey

Why it fits

It is short, memorable, and action-oriented. The line matches a message that asks for composure, judgment, and forward motion at the same time.

Bedrock as the Language Brain

Language detection, rhetorical analysis, quote selection, and translation all go through AWS Bedrock (running Claude Sonnet 4). I tried a local language detector first, but short real-world messages are messy: mixed languages, ticket IDs, URLs, corporate jargon. Bedrock handles that ambiguity much better and keeps the pipeline simpler.

The deterministic part of the system is still retrieval and reranking. But for anything that requires actually understanding language, Bedrock does the heavy lifting.

Reading EPUBs With Zero Dependencies

The corpus loader handles EPUB files using only Python’s standard library, zipfile, xml.etree.ElementTree, and html.parser:

def _load_epub(path: Path) -> LoadedDocument:
with zipfile.ZipFile(path, "r") as archive:
container_xml = archive.read("META-INF/container.xml")
container_root = ET.fromstring(container_xml)
rootfile = container_root.find(".//c:rootfile", CONTAINER_NS)
opf_root = ET.fromstring(archive.read(opf_path))
# Parse manifest, follow the spine, extract XHTML chapters
for chapter_path in spine_paths:
chapter_html = archive.read(chapter_path).decode("utf-8", errors="ignore")
text = _extract_epub_text(chapter_html)
chapters.append(text)

An EPUB is really just a ZIP file with XML and HTML inside. The loader opens the ZIP, reads the table of contents (the OPF file), follows the chapter order (the spine), and extracts clean text from each HTML chapter. No external libraries needed, just Python’s built-in tools.

The HashingEmbedder: Deterministic Tests Without Models

Running the full pipeline in tests means loading Sentence Transformers, which means downloading a 90MB model. Instead, there’s a HashingEmbedder that produces fake-but-consistent embeddings using SHA1:

class HashingEmbedder:
def _embed(self, text: str) -> np.ndarray:
vector = np.zeros(self.dimension, dtype=np.float32)
for token in self._tokenize(text):
digest = hashlib.sha1(token.encode("utf-8")).hexdigest()
bucket = int(digest[:8], 16) % self.dimension
sign = 1.0 if int(digest[8:10], 16) % 2 == 0 else -1.0
vector[bucket] += sign
norm = np.linalg.norm(vector)
if norm > 0:
vector /= norm
return vector

The idea: each word gets hashed to a position and a direction in the vector. The same word always produces the same result, so the tests are reproducible. These embeddings don’t understand meaning, “king” and “queen” won’t be close, but the whole pipeline runs exactly the same way, with real numbers flowing through every step. Tests stay fast, offline, and predictable.

Source code

Full source code available in my GitHub repository.

Removing Clickbait from News Articles with an AI Agent, Python, Strands Agents, and AWS Bedrock

The web is full of articles that do not want to tell you what happened too soon. The headline hints at something. The first paragraphs add suspense. The useful information is somewhere below the fold, after the cookie banner, the newsletter box, a couple of related links, and enough scrolling to make the advertising model happy.

That is annoying when all we want is the news.

That’s my PoC. A small command-line application that receives the URL of a news article, converts the page into clean Markdown, and asks an AI agent to rewrite it as clear journalism: direct headline, concise lead, short paragraphs, no clickbait.

The idea is simple:

plainnews rewrite "https://example.com/news/article"

The CLI does not scrape the page directly. It gives the URL to a Strands Agent. The agent has one tool, fetch_url_as_markdown, and the model decides when to use it. Once the article is available as Markdown, the agent rewrites it following a focused system prompt.

The architecture

The flow is straightforward:

The important part is the boundary between the agent and the tool. Fetching a web page, removing navigation, and converting HTML into Markdown is deterministic Python code. Deciding how to rewrite the story is the LLM’s job.

This keeps the PoC small and easy to reason about.

Project structure

I like to keep configuration in settings.py. It is a pattern I borrowed years ago from Django and I still use it in small prototypes because it keeps things simple:

src/
  cli.py
  settings.py
  commands/
    rewrite.py
  lib/
    agent.py
    prompts.py
    tools.py
    ui.py
  env/
    local/
      .env.example
tests/

The responsibilities are intentionally small:

  • src/commands/rewrite.py contains the Click command.
  • src/lib/tools.py contains the Strands tool and the HTML-to-Markdown pipeline.
  • src/lib/agent.py wires Strands Agents with AWS Bedrock.
  • src/lib/prompts.py keeps the editor prompt and the user task prompt.
  • src/lib/ui.py renders Markdown in the terminal with Rich.

Fetching a URL as Markdown

The agent only gets one tool. It fetches the URL, removes noisy page elements, selects the main content, converts it to Markdown, and truncates the result to 100K characters:

@tool
def fetch_url_as_markdown(url: str) -> str:
"""
Fetch an HTTP or HTTPS URL, remove navigation, ads, scripts and layout noise,
extract the main article content, convert it to Markdown, and return up to
100K characters of clean text.
Use this tool when the user pastes a URL or asks you to analyze a web page.
"""
return fetch_url_as_markdown_impl(url)
def clean_html_to_markdown(html: str, *, max_chars: int = 100_000) -> str:
soup = BeautifulSoup(html, "html.parser")
for selector in NOISY_SELECTORS:
for tag in soup.select(selector):
tag.decompose()
content = soup.find("main") or soup.find("article") or soup.body
if content is None:
return ""
markdown = md(str(content), heading_style="ATX", bullets="-", strip=["a"])
markdown = normalize_markdown(markdown)
if len(markdown) > max_chars:
return markdown[:max_chars].rstrip() + "\n\n[Content truncated]"
return markdown

I am not trying to build a perfect browser engine here. This is a PoC. The goal is to get enough readable article content for the agent to work with. For many news pages, removing scripts, navigation, cookie boxes, newsletter blocks, related links and advertising containers is enough.

The agent

The agent uses Claude on AWS Bedrock through Strands Agents:

def create_agent(*, settings: Settings) -> Agent:
boto_session = create_boto_session(settings)
return Agent(
model=BedrockModel(
boto_session=boto_session,
model_id=settings.resolved_bedrock_model_id,
),
tools=[fetch_url_as_markdown],
system_prompt=SYSTEM_PROMPT,
)

The system prompt is the editorial policy. It tells the model to preserve only facts supported by the fetched article, answer in the requested output language, put the most important information first, remove suspense and filler, and write in a neutral tone.

The output format is Markdown:

  • a direct H1 headline
  • a concise lead paragraph
  • short factual paragraphs
  • a final What changed section, translated to the requested output language, explaining what noise was removed

That last section is useful during development. It gives us a quick sanity check: did the model actually remove clickbait, or did it just paraphrase the article?

The CLI

The command is intentionally small:

@click.command(name="rewrite")
@click.argument("url")
@runtime_options
def rewrite_command(
url: str,
aws_profile: str | None,
region: str | None,
model: str | None,
language: str,
) -> None:
if not is_supported_url(url):
raise click.ClickException("URL must start with http:// or https://")
settings = resolve_settings(
aws_profile=aws_profile,
aws_region=region,
bedrock_model_id=model,
)
agent = create_agent(settings=settings)
result = agent(build_rewrite_prompt(url, language=language))
print_result("PlainNews", str(result))

The CLI validates the URL, creates the agent, sends the URL in the prompt, and renders the final Markdown with Rich.

The tool is not called manually from the command. That is the point of this PoC: the URL is part of the task, and the agent decides to call fetch_url_as_markdown because the tool description says it should be used when the user pastes a URL or asks to analyze a web page.

Usage

Run the command:

poetry run plainnews rewrite "https://example.com/news/article"

By default, PlainNews writes the rewritten article in English. You can choose a different output language with --language:

poetry run plainnews rewrite "https://example.com/news/article" --language Spanish

The output is rendered as Markdown in the terminal.

Example terminal output:

Tech stack

  • Python with Poetry
  • Strands Agents for tool-based agent orchestration
  • AWS Bedrock for the LLM runtime
  • BeautifulSoup for HTML cleanup
  • markdownify for HTML-to-Markdown conversion
  • Click for the command-line interface
  • Rich for Markdown terminal rendering
  • pytest for tests

A couple of notes

This is not a product and it is not a universal paywall remover. It is a small agentic workflow for a very specific frustration: articles that make readers work too hard to understand the basic facts.

Even in this small version, the pattern is useful: deterministic Python code prepares clean context, and the AI agent performs the editorial rewrite with a tight prompt.

And that’s all. Full source code available on GitHub.

AI Eurobeat Producer: Generating Music in Real-Time with AI Agents, Python, and MIDI

What if you could describe the music you want to hear and have an AI produce it in real-time, sending MIDI notes directly to your DAW? That’s exactly what I built: a Python application that uses AI agents to generate Eurobeat and 90s techno patterns, outputting them as live MIDI to Akai’s MPC Beats.

I’m not a musician. I enjoy playing guitar from time to time, but I have zero experience with music production software. However, I’m gifted myself a Akai MPK mini Plus MIDI controller, which has 8 knobs and 8 pads, and I experimented with using it to control a music generation agent. No idea what I’m doing, but it’s fun.

As Akai MIDI controller can be connected to a laptop, and there I’ve got Python, this saturday morning I decided to build a simple prototype that connects an AI agent to MIDI output. The idea is simple. You write a prompt like “Energetic eurobeat in Am, Daft Punk style”, and an AI agent powered by Claude on AWS Bedrock generates patterns for 8 tracks: two drum kits, bass, rhodes, pluck, pad, and a lead melody. The patterns are sent as MIDI messages to MPC Beats, where each track is routed to a different virtual instrument. You can then modify the music live by writing new instructions, and use the physical knobs and pads on an Akai MPK Mini Plus to mute/unmute tracks, regenerate patterns, or reset the session.

I’m using the MPC Beats because it’s free and has a simple MIDI setup, but in theory this could work with any DAW that accepts MIDI input. The whole system is built in Python using Strands Agents for the AI orchestration, mido + python-rtmidi for MIDI I/O, and Rich for the terminal UI.

The Architecture

The flow is straightforward:

Project Structure

src/
  settings.py           # Configuration: BPM, tracks, MIDI devices
  cli.py                # Click CLI entry point
  commands/play.py      # Main play command
  agent/
    prompts.py          # System prompts for the AI producer
    tools.py            # PatternStore + @tool functions
    factory.py          # Agent creation
  midi/
    device.py           # MIDI device detection
    melody_player.py    # Threaded melody loop player
    drum_player.py      # Threaded drum loop player
  session/
    state.py            # State machine (IDLE/GENERATING/PLAYING)
    session.py          # Session orchestrator
  ui/
    menu.py             # Interactive terminal menu

Configuration

Everything starts with settings.py. The MIDI devices and AWS region are loaded from environment variables, while the musical parameters are defined as constants:

BPM = 122
BAR_DURATION = round((60 / BPM) * 4, 3)
LOOP_BARS = 4
LOOP_DURATION = round(BAR_DURATION * LOOP_BARS, 3)
TRACKS = {
1: {"name": "Drums", "channel": 0, "type": "drums"},
2: {"name": "Drums Detroit", "channel": 1, "type": "drums"},
3: {"name": "Rhodes", "channel": 2, "type": "melody"},
4: {"name": "Pluck", "channel": 3, "type": "melody"},
5: {"name": "Bass", "channel": 4, "type": "melody"},
6: {"name": "Org Bass", "channel": 5, "type": "melody"},
7: {"name": "Pad", "channel": 6, "type": "melody"},
8: {"name": "Lead", "channel": 7, "type": "melody"},
}

Each track maps to a MIDI channel. Tracks 1-2 are drum kits (offset-based timing), tracks 3-8 are melodic instruments (duration-based timing). The MPC Beats “House Template” provides the virtual instruments: a Classic drum kit, a Detroit percussion kit, Electric Rhodes, Tube Pluck, Bassline, Organ Bass, Tube Pad, and an Instant Go lead synth.

The Bridge Between AI and MIDI: PatternStore and Tools

The core of the system is the PatternStore, a simple shared store where the AI writes patterns and the MIDI players read them:

class PatternStore:
def __init__(self):
self._patterns: dict[int, list] = {}
def set(self, track_id: int, pattern: list) -> None:
self._patterns[track_id] = pattern
def get(self, track_id: int) -> list | None:
return self._patterns.get(track_id)
def clear(self) -> None:
self._patterns.clear()

The Strands @tool functions are created via a factory that closes over the store:

def create_tools(store: PatternStore) -> list:
@tool
def set_melody_pattern(track_id: int, pattern: str) -> str:
"""Define a melodic line for a specific track."""
data = json.loads(pattern)
store.set(track_id, data)
total = sum(n["duration"] for n in data)
name = TRACKS[track_id]["name"]
return f"OK - {name}: {len(data)} notes, total duration {total:.3f}s"
@tool
def set_drum_pattern(track_id: int, pattern: str) -> str:
"""Define a drum pattern for a specific drum track."""
data = json.loads(pattern)
store.set(track_id, data)
name = TRACKS[track_id]["name"]
return f"OK - {name}: {len(data)} hits"
return [set_drum_pattern, set_melody_pattern]

A melody pattern is a JSON array of {note, duration, velocity} objects where the sum of durations must equal LOOP_DURATION (4 bars). A drum pattern uses {note, velocity, offset} where offset is the time in seconds from the loop start. The note value -1 represents silence, which is crucial for creating space in the arrangement.

The Agent

The agent is a Strands Agent using Claude Sonnet on AWS Bedrock. The system prompt is heavily detailed with music production instructions: frequency ranges for each track, velocity guidelines, and structural rules. The key instruction is “less is more” – not all tracks should play notes all the time:

def create_agent(store: PatternStore) -> Agent:
return Agent(
model=BedrockModel(
model_id=Models.CLAUDE_SONNET,
region_name=AWS_REGION,
),
tools=create_tools(store),
system_prompt=SYSTEM_PROMPT,
callback_handler=None,
)

There are two agents: one for initial generation (calls all 8 tools) and one for live modifications (only modifies the tracks that need to change). A third, lighter agent using Haiku generates the menu suggestions to keep latency and cost low.

MIDI Players

Two player classes handle the actual MIDI output. The MelodyLoopPlayer iterates through note events with durations:

def _loop(self, melody: list):
while not self.stop_event.is_set():
current = self.store.get(self.track_id) or melody
for ev in current:
if self.stop_event.is_set():
break
note = ev["note"]
vel = ev.get("velocity", 80)
if note >= 0:
self._send("note_on", note=note, velocity=vel, channel=self.channel)
deadline = time.time() + ev["duration"]
while not self.stop_event.is_set() and time.time() < deadline:
time.sleep(0.02)
if note >= 0:
self._send("note_off", note=note, velocity=0, channel=self.channel)

The DrumLoopPlayer uses offset-based timing instead, scheduling hits at specific points within the loop. Both players read from the PatternStore on each loop iteration, which enables hot-swapping patterns during live modifications.

The Session

The Session class orchestrates everything. It manages the state machine (IDLE -> GENERATING -> PLAYING), owns the PatternStore, creates the agents, and handles MIDI input from the controller:

class Session:
def __init__(self):
self.state = State.IDLE
self.store = PatternStore()
self.agent = create_agent(self.store)
self.live_agent = create_live_agent(self.store)
self._agent_busy = threading.Lock()

When generation completes, playback starts with a progressive intro – tracks are unmuted one by one with a 2-bar delay between each, creating a build-up effect:

def _start_playback(self):
self.state = State.PLAYING
for tid in TRACKS:
self.players[tid].muted = True
self.players[tid].start(patterns[tid])
intro_delay = BAR_DURATION * 2
for i, tid in enumerate(INTRO_ORDER):
timer = threading.Timer(intro_delay * i, self._unmute_track, args=(tid,))
timer.start()

How It Works

  1. Run python cli.py play
  2. The app detects your MPK Mini Plus and shows a menu with AI-generated suggestions
  3. Select a suggestion or write your own prompt
  4. The AI generates 8 track patterns (takes a few seconds)
  5. Playback begins with a progressive build-up
  6. Write new instructions to modify the music live
  7. Use knobs K1-K8 to mute/unmute individual tracks
  8. PAD 1 regenerates with the same prompt, PAD 2 resets everything

Tech Stack

  • Python 3.13 with Poetry
  • Strands Agents for AI agent orchestration
  • AWS Bedrock (Claude Sonnet + Haiku) for pattern generation
  • mido + python-rtmidi for MIDI I/O
  • Akai MPK Mini Plus as MIDI controller
  • MPC Beats as the DAW/sound engine
  • Rich for terminal UI
  • Click for CLI

And that’s all. Full source code available on GitHub.

Predicting the future: time series forecasting with AI Agents and Amazon Chronos-Bolt

Predicting the future is something we all try to do. Whether it’s energy consumption, sensor readings, or production metrics, having a reliable forecast helps us make better decisions. The problem is that building a good forecasting model traditionally requires deep statistical knowledge, and a lot of tuning. What if we could just hand our data to an AI agent and ask “what’s going to happen next”?

That’s exactly what this project does. It combines Strands Agents with Amazon Chronos-Bolt, a foundation model for time series forecasting available on AWS Bedrock Marketplace, to create an AI agent that can forecast any numerical time series through natural language.

The architecture

The idea is simple. We have a Strands Agent powered by Claude (via AWS Bedrock) that understands natural language. When the user asks for a forecast, the agent calls a custom tool that invokes Chronos-Bolt to generate predictions. The agent then interprets the results and explains them in plain language.

The key here is that the agent doesn’t just return raw numbers. It understands the context, explains trends, and presents the confidence intervals in a way that makes sense.

The forecast tool

The tool is defined using the @tool decorator from Strands. This decorator turns a regular Python function into something the agent can discover and invoke on its own:

@tool
def forecast_time_series(
values: Annotated[
list[float],
"Historical time series values in chronological order. "
"Values should be evenly spaced (e.g., hourly, daily). Minimum 10 values.",
],
prediction_length: Annotated[
int,
"Number of future steps to predict. "
"Uses the same time unit as the input data.",
],
quantile_levels: Annotated[
Optional[list[float]],
"Quantile levels for confidence intervals. Default: [0.1, 0.5, 0.9]. "
"0.5 is the median forecast, 0.1 and 0.9 define the 80% confidence band.",
] = None,
) -> dict:

The Annotated type hints serve a dual purpose: they validate types at runtime and provide descriptions that the LLM reads to understand how to use the tool. This means the agent knows it needs a list of floats, a prediction length, and optionally custom quantile levels, all from the type annotations alone.

The tool validates the input (minimum 10 values, maximum 50,000, prediction length between 1 and 1,000), filters out NaN values, and then calls the Chronos-Bolt client:

result = invoke_chronos(
values=clean_values,
prediction_length=prediction_length,
quantile_levels=quantile_levels,
)
return {
"status": "success",
"content": [{"text": "\n".join(summary_lines)}],
"metadata": {
"quantiles": result.quantiles,
"prediction_length": result.prediction_length,
"history_length": result.history_length,
},
}

The response includes both a human-readable summary (in content) and the raw quantile data (in metadata), so the agent can reference exact numbers when explaining the forecast.

The Chronos-Bolt client

Chronos-Bolt is accessed through the Bedrock runtime API. The client sends the historical values and receives predictions at different quantile levels:

def invoke_chronos(
values: list[float],
prediction_length: int,
quantile_levels: list[float] | None = None,
) -> ForecastResult:
client = _get_bedrock_runtime_client()
payload = {
"inputs": [{"target": values}],
"parameters": {
"prediction_length": prediction_length,
"quantile_levels": quantiles,
},
}
response = client.invoke_model(
modelId=CHRONOS_ENDPOINT_ARN,
body=json.dumps(payload),
contentType="application/json",
accept="application/json",
)

The invoke_model call uses the SageMaker endpoint ARN deployed through Bedrock Marketplace. Chronos-Bolt returns predictions organized by quantile levels, by default, the 10th, 50th (median), and 90th percentiles. This gives us not just a single forecast line, but a confidence band: the 80% interval between the 10th and 90th percentiles tells us how uncertain the model is about its predictions.

The Bedrock runtime client is configured with generous timeouts (120s read, 30s connect) and automatic retries, since inference on time series data can take a moment depending on the history length:

def _get_bedrock_runtime_client():
return boto3.client(
"bedrock-runtime",
region_name=AWS_REGION,
config=Config(
read_timeout=120,
connect_timeout=30,
retries={"max_attempts": 3},
),
)

The agent

Wiring everything together is straightforward. We create a BedrockModel pointing to Claude and pass our forecast tool to the Agent:

from strands import Agent
from strands.models.bedrock import BedrockModel
from settings import AWS_REGION, Models
from forecast import forecast_time_series
SYSTEM_PROMPT = """You are a time series forecasting assistant powered by Amazon Chronos-Bolt.
You help users predict future values from historical numerical data. When a user provides
time series data or describes a scenario, use the forecast_time_series tool to generate
predictions.
When presenting results:
- Show the median forecast (quantile 0.5) as the main prediction
- Explain the confidence band (quantiles 0.1 and 0.9) as the uncertainty range
- Summarize trends in plain language
"""
def create_agent() -> Agent:
bedrock_model = BedrockModel(
model_id=Models.CLAUDE_SONNET,
region_name=AWS_REGION,
)
return Agent(
model=bedrock_model,
system_prompt=SYSTEM_PROMPT,
tools=[forecast_time_series],
)

The system prompt is important here. It tells Claude that it has forecasting capabilities and how to present the results. Without it, the agent would still call the tool correctly (thanks to the Annotated descriptions), but it might not explain the confidence bands or summarize trends as clearly.

Running it

The CLI entry point (cli.py) registers commands and wires everything together. The forecast command generates synthetic hourly data (a sine wave with noise) by default and asks the agent to forecast. You can also pass a custom prompt.

The entry point is minimal:

import click
from commands.forecast import run as forecast
@click.group()
def cli():
pass
cli.add_command(cmd=forecast, name="forecast")
if __name__ == "__main__":
cli()

The actual command lives in commands/forecast.py:

@click.command()
@click.option("--prompt", "-p", default=None, help="Custom prompt for the agent.")
def run(prompt: str | None):
agent = create_agent()
if prompt is None:
values = generate_sample_data(num_points=100)
values_str = ", ".join(f"{v:.2f}" for v in values)
prompt = (
f"I have the following hourly sensor readings from the last 100 hours:\n"
f"[{values_str}]\n\n"
f"Please forecast the next 24 hours and explain the predicted trend."
)
response = agent(prompt)
click.echo(response)

The sine wave is a good choice for a demo because it has a clear periodic pattern that Chronos-Bolt should capture well. With 100 hours of history (about 4 full cycles of a 24-hour pattern), the model has enough data to identify the periodicity and project it forward.

Example

(venv) ➜ src python cli.py forecast
2026-02-27 14:11:16,471 - INFO - Found credentials in shared credentials file: ~/.aws/credentials
2026-02-27 14:11:16,506 - INFO - Creating Strands MetricsClient
Sure! Let me run the forecast on your 100-hour sensor readings right away.
Tool #1: forecast_time_series
2026-02-27 14:11:22,981 - INFO - Starting forecast: history=100, prediction_length=24
2026-02-27 14:11:22,981 - INFO - Invoking Chronos-Bolt: history_length=100, prediction_length=24, quantiles=[0.1, 0.5, 0.9]
2026-02-27 14:11:22,994 - INFO - Found credentials in shared credentials file: ~/.aws/credentials
2026-02-27 14:11:23,697 - INFO - Forecast complete: 24 steps, 4 quantiles
Here are the results of your 24-hour forecast:

Script response

📊 24-Hour Sensor Forecast

Hour Low (10th %) Median (50th %) High (90th %)
+1 ~-0.29 ~0.85 ~0.99
+2 ~0.29 ~0.95 ~0.99
+3 ~0.49 ~0.99 ~0.99
+4 ~0.47 ~0.83 ~0.99
+5 ~0.24 ~0.72 ~0.96
+6 ~0.09 ~0.53 ~0.83
+7 ~-0.25 ~0.19 ~0.59
+8 ~-0.60 ~-0.22 ~0.25
+9 ~-0.91 ~-0.51 ~-0.04
+10 ~-0.99 ~-0.76 ~-0.30
+11 ~-0.99 ~-0.87 ~-0.46
+12 ~-0.99 ~-0.93 ~-0.56
+13 ~-0.99 ~-0.96 ~-0.57
+14 ~-0.99 ~-0.93 ~-0.52
+15 ~-0.99 ~-0.87 ~-0.44
+16 ~-0.99 ~-0.79 ~-0.35
+17 ~-0.96 ~-0.63 ~-0.16
+18 ~-0.86 ~-0.46 ~0.02
+19 ~-0.66 ~-0.24 ~0.21
+20 ~-0.44 ~-0.01 ~0.42
+21 ~-0.21 ~0.22 ~0.60
+22 ~-0.04 ~0.45 ~0.78
+23 ~0.16 ~0.66 ~0.90
+24 ~0.31 ~0.80 ~0.96

🔍 Trend Explanation

Your sensor data clearly exhibits a sinusoidal/oscillating pattern — cycling between peaks near +1.0 and troughs near -1.0 roughly every ~24 hours. The model has picked this up beautifully! Here’s what to expect:

  1. Hours +1 to +4 — Continued Peak 🔼: The signal is currently rising (ending at ~0.67–0.70), and the model predicts it will crest near ~+0.85 to +0.99 in the first few hours — completing the current positive phase.

  2. Hours +5 to +13 — Descent & Trough 🔽: The signal then descends sharply, hitting a trough around hours +12 to +14 with a median near -0.93 to -0.96 — matching the negative peaks seen in the historical data.

  3. Hours +14 to +24 — Recovery 🔼: After bottoming out, the signal climbs back up, reaching approximately +0.80 by hour +24, setting up the next positive cycle.

📐 Confidence Band

The 80% confidence interval (low–high columns) is relatively tight, reflecting the model’s high confidence in the periodic nature of this signal. The widest uncertainty occurs around the transition zones (hours +7–+9 and +17–+19), which is typical for oscillating signals near the zero-crossing points.

In short: your sensor is behaving like a clean oscillating signal with an ~24-hour period, and the next full cycle looks very consistent with historical behavior.Here are the results of your 24-hour forecast:


📊 24-Hour Sensor Forecast

Hour Low (10th %) Median (50th %) High (90th %)
+1 ~-0.29 ~0.85 ~0.99
+2 ~0.29 ~0.95 ~0.99
+3 ~0.49 ~0.99 ~0.99
+4 ~0.47 ~0.83 ~0.99
+5 ~0.24 ~0.72 ~0.96
+6 ~0.09 ~0.53 ~0.83
+7 ~-0.25 ~0.19 ~0.59
+8 ~-0.60 ~-0.22 ~0.25
+9 ~-0.91 ~-0.51 ~-0.04
+10 ~-0.99 ~-0.76 ~-0.30
+11 ~-0.99 ~-0.87 ~-0.46
+12 ~-0.99 ~-0.93 ~-0.56
+13 ~-0.99 ~-0.96 ~-0.57
+14 ~-0.99 ~-0.93 ~-0.52
+15 ~-0.99 ~-0.87 ~-0.44
+16 ~-0.99 ~-0.79 ~-0.35
+17 ~-0.96 ~-0.63 ~-0.16
+18 ~-0.86 ~-0.46 ~0.02
+19 ~-0.66 ~-0.24 ~0.21
+20 ~-0.44 ~-0.01 ~0.42
+21 ~-0.21 ~0.22 ~0.60
+22 ~-0.04 ~0.45 ~0.78
+23 ~0.16 ~0.66 ~0.90
+24 ~0.31 ~0.80 ~0.96

🔍 Trend Explanation

Your sensor data clearly exhibits a sinusoidal/oscillating pattern — cycling between peaks near +1.0 and troughs near -1.0 roughly every ~24 hours. The model has picked this up beautifully! Here’s what to expect:

  1. Hours +1 to +4 — Continued Peak 🔼: The signal is currently rising (ending at ~0.67–0.70), and the model predicts it will crest near ~+0.85 to +0.99 in the first few hours — completing the current positive phase.

  2. Hours +5 to +13 — Descent & Trough 🔽: The signal then descends sharply, hitting a trough around hours +12 to +14 with a median near -0.93 to -0.96 — matching the negative peaks seen in the historical data.

  3. Hours +14 to +24 — Recovery 🔼: After bottoming out, the signal climbs back up, reaching approximately +0.80 by hour +24, setting up the next positive cycle.

📐 Confidence Band

The 80% confidence interval (low–high columns) is relatively tight, reflecting the model’s high confidence in the periodic nature of this signal. The widest uncertainty occurs around the transition zones (hours +7–+9 and +17–+19), which is typical for oscillating signals near the zero-crossing points.

In short: your sensor is behaving like a clean oscillating signal with an ~24-hour period, and the next full cycle looks very consistent with historical behavior.


And that’s all! Full code in my GitHub account.

Using Map-Reduce to process large documents with AI Agents and Python

We live in the era of Large Language Models (LLMs) with massive context windows. Claude 3.5 Sonnet offers 200k tokens, and Gemini 1.5 Pro goes up to 2 million. So, why do we still need to worry about document processing strategies? The answer is yes, we do. For example, AWS Bedrock has a strict limit of 4.5MB for documents, regardless of token count. That’s means we can’t just stuff file greater than 4.5MB into a prompt. Today we’ll show you how I built a production-ready document processing agent that handles large files by implementing a Map-Reduce pattern using Python, AWS Bedrock, and Strands Agents.

The core idea is simple: instead of asking the LLM to “read this book and answer” we break the book into chapters, analyze each chapter in parallel, and then synthesize the results.

Here is the high-level flow:

The heart of the implementation is the DocumentProcessor class. It decides whether to process a file as a whole or split it based on a size threshold. We define a threshold (e.g., 4.3MB) to stay safely within Bedrock’s limits. If the file is larger, we trigger the _process_big method.

# src/lib/processor/processor.py

BYTES_THRESHOLD = 4_300_000

async def _process_file(self, file: DocumentFile, question: str, with_callback=True):
    file_bytes = Path(file.path).read_bytes()
    # Strategy pattern: Choose the right processor based on file size
    processor = self._process_big if len(file_bytes) > BYTES_THRESHOLD else self._process
    async for chunk in processor(file_bytes, file, question, with_callback):
        yield chunk

To increase the performance, we use asyncio to process the file in parallel and we use a semaphore to control the number of workers.

async def _process_big(self, file_bytes: bytes, file: DocumentFile, question: str, with_callback=True) -> AsyncIterator[str]:
    # ... splitting logic ...
    semaphore = asyncio.Semaphore(self.max_workers)

    # Create async tasks for each chunk
    tasks = [
        self._process_chunk(chunk, i, file_name, question, handler.format, semaphore)
        for i, chunk in enumerate(chunks, 1)
    ]

    # Run in parallel
    results = await asyncio.gather(*tasks)
    
    # Sort results to maintain document order
    results.sort(key=lambda x: x[0])
    responses_from_chunks = [response for _, response in results]

Each chunk is processed by an isolated agent instance that only sees that specific fragment and the user’s question. Once we have the partial analyses, we consolidate them. This acts as a compression step: we’ve turned raw pages into relevant insights.

def _consolidate_and_truncate(self, responses: list[str], num_chunks: int) -> str:
    consolidated = "\n\n".join(responses)
    
    if len(consolidated) > MAX_CONTEXT_CHARS:
        # Safety mechanism to ensure we don't overflow the final context
        return consolidated[:MAX_CONTEXT_CHARS] + "\n... [TRUNCATED]"
    return consolidated

Finally, we feed this consolidated context to the agent for the final answer. In a long-running async process, feedback is critical. I implemented an Observer pattern to decouple the processing logic from the UI/Logging.

# src/main.py

class DocumentProcessorEventListener(ProcessingEventListener):
    async def on_chunk_start(self, chunk_number: int, file_name: str):
        logger.info(f"[Worker {chunk_number}] Processing chunk for file {file_name}")

    async def on_chunk_end(self, chunk_number: int, file_name: str, response: str):
        logger.info(f"[Worker {chunk_number}] Completed chunk for file {file_name}")

By breaking down large tasks, we not only bypass technical limits but often get better results. The model focuses on smaller sections, reducing hallucinations, and the final answer is grounded in a pre-processed summary of facts.

We don’t just send text; we send the raw document bytes. This allows the model (Claude 4.5 Sonnet via Bedrock) to use its native document processing capabilities. Here is how we construct the message payload:

# src/lib/processor/processor.py

def _create_document_message(self, file_format: str, file_name: str, file_bytes: bytes, text: str) -> list:
    return [
        {
            "role": "user",
            "content": [
                {
                    "document": {
                        "format": file_format,
                        "name": file_name,
                        "source": {"bytes": file_bytes},
                    },
                },
                {"text": text},
            ],
        },
    ]

When processing chunks, we don’t want the model to be chatty. We need raw information extraction. We use a “Spartan” system prompt that enforces brevity and objectivity, ensuring the consolidation phase receives high-signal input.

# src/lib/processor/prompts.py

SYSTEM_CHUNK_PROMPT = f"""
You are an artificial intelligence assistant specialized in reading and analyzing files.
You have received a chunk of a large file.
...
If the user's question cannot be answered with the information in the current chunk, do not answer it directly.

{SYSTEM_PROMPT_SPARTAN}

The SYSTEM_PROMPT_SPARTAN (injected above) explicitly forbids conversational filler, ensuring we maximize the token budget for actual data.

The project handles pdf and xlsx files. The rest of the file types are not processed and are given to the LLM as-is.

With this architecture, we can process large files in a production environment. This allows us to easily plug in different interfaces, whether it’s a CLI logger (as shown) or a WebSocket update for a UI frontend like Chainlit.

Full code in my github

Chat with your Data: Building a File-Aware AI Agent with AWS Bedrock and Chainlit

We all know LLMs are powerful, but their true potential is unlocked when they can see your data. While RAG (Retrieval-Augmented Generation) is great for massive knowledge bases, sometimes you just want to drag and drop a file and ask questions about it.

Today we’ll build a “File-Aware” AI agent that can natively understand a wide range of document formats—from PDFs and Excel sheets to Word docs and Markdown files. We’ll use AWS Bedrock with Claude 4.5 Sonnet for the reasoning engine and Chainlit for the conversational UI.

The idea is straightforward: Upload a file, inject it into the model’s context, and let the LLM do the rest. No vector databases, no complex indexing pipelines—just direct context injection for immediate analysis.

The architecture is simple yet effective. We intercept file uploads in the UI, process them into a format the LLM understands, and pass them along with the user’s query.

┌──────────────┐      ┌──────────────┐      ┌────────────────────┐
│   Chainlit   │      │  Orchestrator│      │   AWS Bedrock      │
│      UI      │─────►│    Agent     │─────►│(Claude 4.5 Sonnet) │
└──────┬───────┘      └──────────────┘      └────────────────────┘
       │                      ▲
       │    ┌────────────┐    │
       └───►│ File Proc. │────┘
            │   Logic    │
            └────────────┘

The tech stack includes:

  • AWS Bedrock with Claude 4.5 Sonnet for high-quality reasoning and large context windows.
  • Chainlit for a chat-like interface with native file upload support.
  • Python for the backend logic.

The core challenge is handling different file types and presenting them to the LLM. We support a variety of formats by mapping them to Bedrock’s expected input types.

To enable file uploads in Chainlit, you need to configure the [features.spontaneous_file_upload] section in your .chainlit/config.toml. This is where you define which MIME types are accepted.

[features.spontaneous_file_upload]
    enabled = true
    accept = [
        "application/pdf",
        "text/csv",
        "application/msword",
        "application/vnd.openxmlformats-officedocument.wordprocessingml.document",
        "application/vnd.ms-excel",
        "application/vnd.openxmlformats-officedocument.spreadsheetml.sheet",
        "text/html",
        "text/plain",
        "text/markdown",
        "text/x-markdown"
    ]
    max_files = 20
    max_size_mb = 500
The main agent loop handles the conversation. It checks for uploaded files, processes them, and constructs the message payload for the LLM. We also include robust error handling to manage context window limits gracefully.
def get_question_from_message(message: cl.Message):
    content_blocks = None
    if message.elements:
        content_blocks = get_content_blocks_from_message(message)

    if content_blocks:
        content_blocks.append({"text": message.content or "Write a summary of the document"})
        question = content_blocks
    else:
        question = message.content

    return question


def get_content_blocks_from_message(message: cl.Message):
    docs = [f for f in message.elements if f.type == "file" and f.mime in MIME_MAP]
    content_blocks = []

    for doc in docs:
        file = Path(doc.path)
        file_bytes = file.read_bytes()
        shutil.rmtree(file.parent)

        content_blocks.append({
            "document": {
                "name": sanitize_filename(doc.name),
                "format": MIME_MAP[doc.mime],
                "source": {"bytes": file_bytes}
            }
        })

    return content_blocks

@cl.on_message
async def handle_message(message: cl.Message):
    task = asyncio.create_task(process_user_task(
        question=get_question_from_message(message),
        debug=DEBUG))
    cl.user_session.set("task", task)
    try:
        await task
    except asyncio.CancelledError:
        logger.info("User task was cancelled.")

This pattern allows for ad-hoc analysis. You don’t need to pre-ingest data. You can:

  1. Analyze Financials: Upload an Excel sheet and ask for trends.
  2. Review Contracts: Upload a PDF and ask for clause summaries.
  3. Debug Code: Upload a source file and ask for a bug fix.
By leveraging the large context window of modern models like Claude 4.5 Sonnet, we can feed entire documents directly into the prompt, providing the model with full visibility without the information loss often associated with RAG chunking.

And that's all. With tools like Chainlit and powerful APIs like AWS Bedrock, we can create robust, multi-modal assistants that integrate seamlessly into our daily workflows.

Full code in my github account.