We all know LLMs are powerful, but their true potential is unlocked when they can see your data. While RAG (Retrieval-Augmented Generation) is great for massive knowledge bases, sometimes you just want to drag and drop a file and ask questions about it.
Today we’ll build a “File-Aware” AI agent that can natively understand a wide range of document formats—from PDFs and Excel sheets to Word docs and Markdown files. We’ll use AWS Bedrock with Claude 4.5 Sonnet for the reasoning engine and Chainlit for the conversational UI.
The idea is straightforward: Upload a file, inject it into the model’s context, and let the LLM do the rest. No vector databases, no complex indexing pipelines—just direct context injection for immediate analysis.
The architecture is simple yet effective. We intercept file uploads in the UI, process them into a format the LLM understands, and pass them along with the user’s query.
AWS Bedrock with Claude 4.5 Sonnet for high-quality reasoning and large context windows.
Chainlit for a chat-like interface with native file upload support.
Python for the backend logic.
The core challenge is handling different file types and presenting them to the LLM. We support a variety of formats by mapping them to Bedrock’s expected input types.
To enable file uploads in Chainlit, you need to configure the [features.spontaneous_file_upload] section in your .chainlit/config.toml. This is where you define which MIME types are accepted.
The main agent loop handles the conversation. It checks for uploaded files, processes them, and constructs the message payload for the LLM. We also include robust error handling to manage context window limits gracefully.
def get_question_from_message(message: cl.Message):
content_blocks = None
if message.elements:
content_blocks = get_content_blocks_from_message(message)
if content_blocks:
content_blocks.append({"text": message.content or "Write a summary of the document"})
question = content_blocks
else:
question = message.content
return question
def get_content_blocks_from_message(message: cl.Message):
docs = [f for f in message.elements if f.type == "file" and f.mime in MIME_MAP]
content_blocks = []
for doc in docs:
file = Path(doc.path)
file_bytes = file.read_bytes()
shutil.rmtree(file.parent)
content_blocks.append({
"document": {
"name": sanitize_filename(doc.name),
"format": MIME_MAP[doc.mime],
"source": {"bytes": file_bytes}
}
})
return content_blocks
@cl.on_message
async def handle_message(message: cl.Message):
task = asyncio.create_task(process_user_task(
question=get_question_from_message(message),
debug=DEBUG))
cl.user_session.set("task", task)
try:
await task
except asyncio.CancelledError:
logger.info("User task was cancelled.")
This pattern allows for ad-hoc analysis. You don’t need to pre-ingest data. You can:
Analyze Financials: Upload an Excel sheet and ask for trends.
Review Contracts: Upload a PDF and ask for clause summaries.
Debug Code: Upload a source file and ask for a bug fix.
By leveraging the large context window of modern models like Claude 4.5 Sonnet, we can feed entire documents directly into the prompt, providing the model with full visibility without the information loss often associated with RAG chunking.
And that's all. With tools like Chainlit and powerful APIs like AWS Bedrock, we can create robust, multi-modal assistants that integrate seamlessly into our daily workflows.
We can build simple AI agents that handle specific tasks quite easily today. But what about building AI systems that can handle multiple domains effectively? One approach is to create a single monolithic agent that tries to do everything, but this quickly runs into problems of context pollution, maintenance complexity, and scaling limitations. In this article, we’ll show a production-ready pattern for building multi-purpose AI systems using an orchestrator architecture that coordinates domain-specific agents.
The idea is simple: Don’t build one agent to rule them all instead, create specialized agents that excel in their domains and coordinate them through an intelligent orchestrator. The solution is an orchestrator agent that routes requests to specialized sub-agents, each with focused expertise and dedicated tools. Think of it as a smart router that understands intent and delegates accordingly.
That’s the core of the Orchestrator Pattern for multi-agent systems:
Weather Agent: Expert in meteorological data and weather patterns. It uses external weather APIs to fetch historical and current weather data.
Logistics Agent: Specialist in supply chain and shipping operations. Fake logistics data is generated to simulate shipment tracking, route optimization, and delivery performance analysis.
Production Agent: Focused on manufacturing operations and production metrics. Also, fake production data is generated to analyze production KPIs.
AWS Bedrock with Claude 4.5 Sonnet for agent reasoning
Strands Agents framework for agent orchestration
Chainlit for the conversational UI
FastAPI for the async backend
PostgreSQL for storing conversation history and domain data
The orchestrator’s job is simple but critical: understand the user’s intent and route to the right specialist(s).
MAIN_SYSTEM_PROMPT = """You are an intelligent orchestrator agent
responsible for routing user requests to specialized sub-agents
based on their domain expertise.
## Available Specialized Agents
### 1. Production Agent
**Domain**: Manufacturing operations, production metrics, quality control
**Handles**: Production KPIs, machine performance, downtime analysis
### 2. Logistics Agent
**Domain**: Supply chain, shipping, transportation operations
**Handles**: Shipment tracking, route optimization, delivery performance
### 3. Weather Agent
**Domain**: Meteorological data and weather patterns
**Handles**: Historical weather, atmospheric conditions, climate trends
## Your Decision Process
1. Analyze the request for key terms and domains
2. Determine scope (single vs multi-domain)
3. Route to appropriate agent(s)
4. Synthesize results when multiple agents are involved
"""
The orchestrator receives specialized agents as tools:
def get_orchestrator_tools() -> List[Any]:
from tools.logistics.agent import logistics_assistant
from tools.production.agent import production_assistant
from tools.weather.agent import weather_assistant
tools = [
calculator,
think,
current_time,
AgentCoreCodeInterpreter(region=AWS_REGION).code_interpreter,
logistics_assistant, # Specialized agent as tool
production_assistant, # Specialized agent as tool
weather_assistant # Specialized agent as tool
]
return tools
Each specialized agent follows a consistent pattern. Here’s the weather agent:
@tool
@stream_to_step("weather_assistant")
async def weather_assistant(query: str):
"""
A research assistant specialized in weather topics with streaming support.
"""
try:
tools = [
calculator,
think,
current_time,
AgentCoreCodeInterpreter(region=AWS_REGION).code_interpreter
]
# Domain-specific tools
tools += WeatherTools(latitude=MY_LATITUDE, longitude=MY_LONGITUDE).get_tools()
research_agent = get_agent(
system_prompt=WEATHER_ASSISTANT_PROMPT,
tools=tools
)
async for token in research_agent.stream_async(query):
yield token
except Exception as e:
yield f"Error in research assistant: {str(e)}"
Each agent has access to domain-specific tools. For example, the weather agent uses external APIs:
class WeatherTools:
def __init__(self, latitude: float, longitude: float):
self.latitude = latitude
self.longitude = longitude
def get_tools(self) -> List[tool]:
@tool
def get_hourly_weather_data(from_date: date, to_date: date) -> MeteoData:
"""Get hourly weather data for a specific date range."""
url = (f"https://api.open-meteo.com/v1/forecast?"
f"latitude={self.latitude}&longitude={self.longitude}&"
f"hourly=temperature_2m,relative_humidity_2m...")
response = requests.get(url)
return parse_weather_response(response.json())
return [get_hourly_weather_data]
The logistics and production agents use synthetic data generators for demonstration:
User: “Analyze production efficiency trends and correlate with weather and logistics performance based in yesterday’s data.”
Flow:
Orchestrator coordinates all three agents
Production agent retrieves manufacturing KPIs
Weather agent provides environmental data
Logistics agent supplies delivery metrics
Orchestrator synthesizes multi-domain analysis
This architecture scales naturally in multiple dimensions. We can easily add new specialized agents without disrupting existing functionality. WE only need to create the new agent and register it as a tool with the orchestratortrator prompt with new domain description. That’s it.
The orchestrator pattern transforms multi-domain AI from a monolithic challenge into a composable architecture. Each agent focuses on what it does best, while the orchestrator provides intelligent coordination.
In industrial environments, data analysis is crucial for optimizing processes, detecting anomalies, and making informed
decisions. Manufacturing plants, energy systems, and industrial IoT generate massive amounts of data from sensors,
machines, and control systems. Traditionally, analyzing this data requires specialized knowledge in both industrial
processes and data science, creating a bottleneck for quick insights.
I’ve been exploring agentic AI frameworks lately, particularly for complex data analysis tasks. While working on
industrial data problems, I realized that combining the reasoning capabilities of Large Language Models with specialized
tools could create a powerful solution for industrial data analysis. This project demonstrates how to build a ReAct (
Reasoning and Acting) AI agent using LangGraph that can analyze manufacturing data, understand industrial processes, and
provide actionable insights.
The goal of this project is to create an AI agent that can analyze industrial datasets (manufacturing metrics, sensor
readings, process control data) and provide expert-level insights about production optimization, quality control, and
process efficiency. Using LangGraph’s ReAct agent framework with AWS Bedrock, the system can execute Python code
dynamically in a sandboxed environment, process large datasets, and reason about industrial contexts.
The dataset is a fake sample of industrial data with manufacturing metrics like temperature, speed, humidity, pressure,
operator experience, scrap rates, and unplanned stops. In fact, I’ve generated the dataset using chatgpt
This project uses several key components:
LangGraph ReAct Agent: For building the multi-tool AI agent with ReAct (Reasoning and Acting) patterns that can
dynamically choose tools and reason about results
AWS Bedrock: Claude Sonnet 4 as the underlying LLM for reasoning and code generation
Sandboxed Code Interpreter: Secure execution of Python code for data analysis using AWS Agent Core. One tool taken
from strands-agents-tools library.
Industrial Domain Expertise: Specialized system prompts with knowledge of manufacturing processes, quality
control, and industrial IoT
The agent has access to powerful tools:
Code Interpreter: Executes Python code safely in a sandboxed AWS environment using pandas, numpy, scipy, and other
scientific libraries
Data Processing: Handles large industrial datasets with memory-efficient strategies
Industrial Context: Understands manufacturing processes, sensor data, and quality metrics
The system uses AWS Agent Core’s sandboxed code interpreter, which means:
Python code is executed in an isolated environment
No risk to the host system
Access to scientific computing libraries (pandas, numpy, scipy)
Memory management for large datasets
The core of the system is surprisingly simple. The ReAct agent is built using LangGraph’s create_react_agent with
custom tools:
from langgraph.prebuilt import create_react_agent
from typing import List
import pandas as pd
from langchain_core.callbacks import BaseCallbackHandler
def analyze_df(df: pd.DataFrame, system_prompt: str, user_prompt: str,
callbacks: List[BaseCallbackHandler], streaming: bool = False):
code_interpreter_tools = CodeInterpreter()
tools = code_interpreter_tools.get_tools()
agent = create_react_agent(
model=get_llm(model=DEFAULT_MODEL, streaming=streaming,
budget_tokens=12288, callbacks=callbacks),
tools=tools,
prompt=system_prompt
)
agent_prompt = f"""
I have a DataFrame with the following data:
- Columns: {list(df.columns)}
- Shape: {df.shape}
- data: {df}
The output must be an executive summary with the key points.
The response must be only markdown, not plots.
"""
messages = [
("user", agent_prompt),
("user", user_prompt)
]
agent_input = {"messages": messages}
return agent. Invoke(agent_input)
The ReAct pattern (Reasoning and Acting) allows the agent to:
Reason about what analysis is needed
Act by calling the appropriate tools (in this case: code interpreter)
Observe the results of code execution
Re-reason and potentially call more tools if needed
This creates a dynamic loop where the agent can iteratively analyze data, examine results, and refine its approach –
much more powerful than a single code execution.
The magic happens in the system prompt, which provides the agent with industrial domain expertise:
SYSTEM_PROMPT = """
# Industrial Data Analysis Agent - System Prompt
You are an expert AI agent specialized in industrial data analysis and programming.
You excel at solving complex data problems in manufacturing, process control,
energy systems, and industrial IoT environments.
## Core Capabilities
- Execute Python code using pandas, numpy, scipy
- Handle large datasets with chunking strategies
- Process time-series data, sensor readings, production metrics
- Perform statistical analysis, anomaly detection, predictive modeling
## Industrial Domain Expertise
- Manufacturing processes and production optimization
- Process control systems (PID controllers, SCADA, DCS)
- Industrial IoT sensor data and telemetry
- Quality control and Six Sigma methodologies
- Energy consumption analysis and optimization
- Predictive maintenance and failure analysis
"""
The code interpreter tool is wrapped with safety validations:
def validate_code_ast(code: str) -> bool:
"""Validate Python code using AST to ensure safety."""
try:
ast.parse(code)
return True
except SyntaxError:
return False
@tool
def code_interpreter(code: str) -> str:
"""Executes Python code in a sandboxed environment."""
if not validate_code_ast(code):
raise UnsafeCodeError("Unsafe code or syntax errors.")
return code_tool(code_interpreter_input={
"action": {
"type": "executeCode",
"session_name": session_name,
"code": code,
"language": "python"
}
})
The system uses Claude Sonnet 4 through AWS Bedrock with optimized parameters for industrial analysis:
The project includes fake sample industrial data with manufacturing metrics:
- `machine_id`: Equipment identifier - `shift`: Production shift (A/M/N for morning/afternoon/night) - `temperature`, `speed`, `humidity`, `pressure`: Process parameters - `operator_experience`: Years of operator experience - `scrap_kg`: Quality metric (waste produced) - `unplanned_stop`: Equipment failure indicator
A typical analysis query might be: "Do temperature and speed setpoints vary across shifts?" The agent will stream the response as it generates it.
The agent will:
1. Load and examine the dataset structure 2. Generate appropriate Python code for analysis 3. Execute the code in a sandboxed environment 4. Provide insights about shift-based variations 5. Suggest process optimization recommendations
import logging
import pandas as pd
from langchain_core.callbacks import StreamingStdOutCallbackHandler
from modules.df_analyzer import analyze_df
from prompts import SYSTEM_PROMPT
logging.basicConfig(
format='%(asctime)s [%(levelname)s] %(message)s',
level='INFO',
datefmt='%d/%m/%Y %X')
logger = logging.getLogger(__name__)
class StreamingCallbackHandler(StreamingStdOutCallbackHandler):
def on_llm_new_token(self, token: str, **kwargs):
print(token, end='', flush=True)
df = pd.read_csv('fake_data.csv')
user_prompt = "Do temperature and speed setpoints vary across shifts?"
for chunk in analyze_df(
user_prompt=user_prompt,
df=df,
system_prompt=SYSTEM_PROMPT,
callbacks=[StreamingCallbackHandler()],
streaming=True):
logger.debug(chunk)
This project demonstrates the power of agentic AI for specialized domains. Instead of building custom analytics
dashboards or writing specific analysis scripts, we provide the agent with:
Domain Knowledge: Through specialized system prompts
Tools: Safe code execution capabilities
Context: The actual data to analyze
The agent can then:
Generate appropriate analysis code
Execute it safely
Interpret results with industrial context
Provide actionable recommendations
The result is a flexible system that can handle various industrial analysis tasks without pre-programmed solutions. The
agent reasons about the problem, writes the necessary code (sandboxed), and provides expert-level insights.
Today we’re going to build an AI agent that can predict the weather using Strands-Agents framework and Python. This project is designed to show how to integrate external data sources, advanced computational tools, and AI capabilities into a cohesive system. For this experiment we’re going to use Strands-Agents framework, which provides a robust foundation for building intelligent agents that can interact with various tools and APIs. Strands-Agents comes with built-in tools that allow us to create agents that can perform complex tasks by orchestrating multiple tools and APIs. For this project we’re going to use the following tools:
calculator: for performing mathematical and financial calculations.
think: for reflecting on data and generating ideas.
file_write: for saving results and analyses to files.
python_repl: for executing Python code and performing advanced analyses.
The last one is particularly useful for overcoming the limitations of large language models (LLMs) when it comes to deterministic calculations. By using a Python REPL, we can ensure that our agent can perform precise computations without relying solely on the LLM’s probabilistic outputs. We have Pandas and Scikit-learn for statistical analysis, which allows us to perform advanced data manipulation and machine learning tasks, and the agent will be able to use these libraries to analyze weather data and generate forecasts. Also, I’ve created a custom tool to fetch hourly weather data from the Open-Meteo API, which provides real-time weather information for specific locations.
import logging
from datetime import datetime, date
from typing import List
import requests
from strands import tool
from modules.weather.models import (
TemperatureReading, HumidityReading, ApparentTemperatureReading,
PrecipitationReading, EvapotranspirationReading, SurfacePressureReading, MeteoData)
logger = logging.getLogger(__name__)
class Tools:
def __init__(self, latitude: float, longitude: float):
self.latitude = latitude
self.longitude = longitude
def get_tools(self) -> List[tool]:
@tool
def get_hourly_weather_data(from_date: date, to_date: date) -> MeteoData:
"""
Get hourly weather data for a specific date range.
Notes:
- The response is a MeteoData object containing lists of readings for temperature, humidity,
apparent temperature, precipitation, evapotranspiration, and surface pressure.
- Each reading has a timestamp and a value.
Returns:
MeteoData: Object containing weather readings for the specified date range
"""
start_date = from_date.strftime('%Y-%m-%d')
end_date = to_date.strftime('%Y-%m-%d')
url = (f"https://api.open-meteo.com/v1/forecast?"
f"latitude={self.latitude}&"
f"longitude={self.longitude}&"
f"hourly=temperature_2m,relative_humidity_2m,apparent_temperature,precipitation,evapotranspiration,surface_pressure&"
f"start_date={start_date}&"
f"end_date={end_date}")
response = requests.get(url)
meteo = MeteoData(
temperature=[],
humidity=[],
apparent_temperature=[],
precipitation=[],
evapotranspiration=[],
surface_pressure=[]
)
data = response.json()
weather_data_time = data['hourly']['time']
logger.info(f"[get_hourly_weather_data] Fetched weather data from {start_date} to {end_date}. {len(weather_data_time)} records found.")
for iso in weather_data_time:
time = datetime.fromisoformat(iso)
meteo.temperature.append(TemperatureReading(
time=time,
value=data['hourly']['temperature_2m'][data['hourly']['time'].index(iso)]))
meteo.humidity.append(HumidityReading(
time=time,
value=data['hourly']['relative_humidity_2m'][data['hourly']['time'].index(iso)]))
meteo.apparent_temperature.append(ApparentTemperatureReading(
time=time,
value=data['hourly']['apparent_temperature'][data['hourly']['time'].index(iso)]))
meteo.precipitation.append(PrecipitationReading(
time=time,
value=data['hourly']['precipitation'][data['hourly']['time'].index(iso)]))
meteo.evapotranspiration.append(EvapotranspirationReading(
time=time,
value=data['hourly']['evapotranspiration'][data['hourly']['time'].index(iso)]))
meteo.surface_pressure.append(SurfacePressureReading(
time=time,
value=data['hourly']['surface_pressure'][data['hourly']['time'].index(iso)]))
return meteo
return [get_hourly_weather_data, ]
To allow the LLM to interact with this tool, we define a Pydantic model that describes the expected input and output formats. This ensures that the agent can correctly interpret the data it receives from the API and use it effectively in its analyses.
from datetime import datetime
from pydantic import BaseModel, Field
class TemperatureReading(BaseModel):
"""Temperature reading at 2 meters"""
time: datetime = Field(..., description="Timestamp")
value: float = Field(description="Temperature in °C")
class HumidityReading(BaseModel):
"""Relative humidity reading at 2 meters"""
time: datetime = Field(..., description="Timestamp")
value: int = Field(..., ge=0, le=100, description="Relative humidity in %")
class ApparentTemperatureReading(BaseModel):
"""Apparent temperature reading"""
time: datetime = Field(..., description="Timestamp")
value: float = Field(..., description="Apparent temperature in °C")
class PrecipitationReading(BaseModel):
"""Precipitation reading"""
time: datetime = Field(..., description="Timestamp")
value: float = Field(..., ge=0, description="Precipitation in mm")
class EvapotranspirationReading(BaseModel):
"""Evapotranspiration reading"""
time: datetime = Field(..., description="Timestamp")
value: float = Field(..., description="Evapotranspiration in mm")
class SurfacePressureReading(BaseModel):
"""Surface pressure reading"""
time: datetime = Field(..., description="Timestamp")
value: float = Field(..., gt=0, description="Surface pressure in hPa")
class MeteoData(BaseModel):
"""Model to store meteorological data"""
temperature: list[TemperatureReading] = Field(..., description="List of temperature readings")
humidity: list[HumidityReading] = Field(..., description="List of humidity readings")
apparent_temperature: list[ApparentTemperatureReading] = Field(..., description="List of apparent temperature readings")
precipitation: list[PrecipitationReading] = Field(..., description="List of precipitation readings")
evapotranspiration: list[EvapotranspirationReading] = Field(..., description="List of evapotranspiration readings")
surface_pressure: list[SurfacePressureReading] = Field(..., description="List of surface pressure readings")
The use of Strands-Agents is very simple. I’ve encapsulated the agent logic in a single function that initializes the agent with the necessary tools and prompts. The agent can then be used to generate weather forecasts or answer specific weather-related questions.
_ = ai(
system_prompt=SYSTEM_PROMPT,
user_prompt="What will the weather be like tomorrow?")
If I run this code, agent will use the provided tools to generate the answer. In the output of the command, you will see the agent’s reasoning, the tools it used, and the final answer. First it uses the current time tool to get the current date and time (using current_time tool), then it uses the get_hourly_weather_data tool to fetch the weather data, after that uses think tool to analyze the data, and finally it uses the python_repl tool to generate the needed calculations, using Pandas, and produce the final weather forecast. Here’s an example of the output you might see when running the agent:
12/07/2025 15:28:18 [INFO] Check agent weather.
12/07/2025 15:28:18 [INFO] Found credentials in shared credentials file: ~/.aws/credentials
12/07/2025 15:28:18 [INFO] Creating Strands MetricsClient
I'll help you get the weather forecast for tomorrow. First, let me get the current time to determine what "tomorrow" is, and then I'll fetch the weather data for you.
Tool #1: current_time
Now I'll get the hourly weather data for tomorrow (July 13, 2025):
Tool #2: get_hourly_weather_data
12/07/2025 15:28:24 [INFO] [get_hourly_weather_data] Fetched weather data from 2025-07-13 to 2025-07-13. 24 records found.
Let me analyze this weather data to provide you with a comprehensive forecast for tomorrow (July 13, 2025):
Tool #3: think
Let me analyze the weather data directly to provide you with tomorrow's forecast:
Tool #4: python_repl
WEATHER FORECAST FOR TOMORROW (July 13, 2025)
==================================================
Temperature Range: 20.5°C to 23.6°C
Average Temperature: 22.2°C
Feels Like Temperature: Up to 24.6°C
Humidity Range: 70% to 83%
Average Humidity: 77%
Total Precipitation: 1.0mm
Precipitation Timing: Brief shower 4-5 PM
Pressure: Rising from 1012.1 to 1019.9 hPa
Pressure Change: +7.8 hPa (improving conditions)
Tool #5: python_repl
DETAILED HOURLY BREAKDOWN
==============================
🌅 MORNING (6 AM - 12 PM):
Temperature: 20.6°C to 23.0°C
Humidity: 70% to 82%
Conditions: Clear, comfortable temperatures
☀️ AFTERNOON (12 PM - 6 PM):
Temperature: 22.6°C to 23.6°C
Humidity: 71% to 81%
Precipitation: 1.0mm (brief shower 4-5 PM)
Conditions: Warmest part of day, brief afternoon shower
🌙 EVENING (6 PM - 12 AM):
Temperature: 22.0°C to 22.4°C
Humidity: 75% to 82%
Conditions: Pleasant evening temperatures, increasing humidity
📋 SUMMARY & RECOMMENDATIONS:
• Mild and pleasant day with comfortable temperatures
• Brief shower expected between 4-5 PM (carry umbrella)
• Rising atmospheric pressure indicates stable weather
• High humidity in morning and evening - may feel muggy
• Good day for outdoor activities, except during afternoon shower
• Light layers recommended due to temperature variation
## Weather Forecast for Tomorrow (July 13, 2025)
**Overall Conditions:** A pleasant, mild day with mostly clear skies and a brief afternoon shower.
### 🌡️ **Temperature**
- **Range:** 20.5°C to 23.6°C (69°F to 74°F)
- **Feels like:** Up to 24.6°C (76°F)
- **Coolest:** Early morning around 2-6 AM
- **Warmest:** Mid-afternoon around 2 PM
### 🌧️ **Precipitation**
- **Total:** 1.0mm of rain
- **Timing:** Brief shower between 4-5 PM
- **Duration:** Short-lived, about 1 hour
- **Recommendation:** Carry an umbrella for the afternoon
### 💨 **Atmospheric Conditions**
- **Humidity:** 70-83% (higher in morning/evening)
- **Pressure:** Rising from 1012 to 1020 hPa (+7.8 hPa increase)
- **Trend:** Improving and stabilizing weather conditions
### ⏰ **Time-Based Breakdown**
**Morning (6 AM - 12 PM):** Clear and comfortable, 20.6-23°C
**Afternoon (12 PM - 6 PM):** Warmest period with brief shower, 22.6-23.6°C
**Evening (6 PM - 12 AM):** Pleasant and stable, 22-22.4°C
### 📝 **Recommendations**
- **Clothing:** Light layers, as temperatures vary throughout the day
- **Activities:** Great for outdoor plans, just avoid 4-5 PM for the shower
- **Comfort:** May feel slightly muggy due to higher humidity
- **Overall:** A very pleasant summer day with stable, improving weather conditions
The rising atmospheric pressure indicates this will be a stable weather day with good conditions for most outdoor activities!
Process finished with exit code 0
Here you can see the ai function.
import logging
from botocore.config import Config
from strands import Agent
from strands.agent import AgentResult
from strands.models import BedrockModel
from strands_tools import calculator, file_write, current_time, think, python_repl
from core.aws import get_aws_session
from modules.weather.tools import Tools
from settings import (
IA_MODEL, IA_TEMPERATURE, LLM_READ_TIMEOUT, LLM_CONNECT_TIMEOUT,
LLM_MAX_ATTEMPTS, MY_LATITUDE, MY_LONGITUDE, )
logger = logging.getLogger(__name__)
def get_agent(
system_prompt: str,
read_timeout: int = LLM_READ_TIMEOUT,
connect_timeout: int = LLM_CONNECT_TIMEOUT,
max_attempts: int = LLM_MAX_ATTEMPTS) -> Agent:
config = Config(
read_timeout=read_timeout,
connect_timeout=connect_timeout,
retries={'max_attempts': max_attempts}
)
session = get_aws_session()
base_tools = [calculator, think, python_repl, file_write, current_time]
custom_tools = Tools(latitude=MY_LATITUDE, longitude=MY_LONGITUDE).get_tools()
all_tools = base_tools + custom_tools
bedrock_model = BedrockModel(
model_id=IA_MODEL,
temperature=IA_TEMPERATURE,
boto_session=session,
boto_client_config=config,
)
return Agent(
model=bedrock_model,
tools=all_tools,
system_prompt=system_prompt
)
def ai(
system_prompt: str,
user_prompt: str,
read_timeout: int = 300,
connect_timeout: int = 60,
max_attempts: int = 5) -> AgentResult:
agent = get_agent(
system_prompt=system_prompt,
read_timeout=read_timeout,
connect_timeout=connect_timeout,
max_attempts=max_attempts)
return agent(user_prompt)
As you can see, the agent is only a few lines of code. The magic is in the prompts and the tools that it uses. The agent can be used to generate weather forecasts, analyze historical weather data, and provide practical recommendations based on the weather conditions. This is the main prompt:
FORECAST_PROMPT = f"""
## Instructions for the weather forecast
Your mission is to analyze weather data and provide accurate and useful forecasts for the next {{days}} days.
You have access to a tool called `get_hourly_weather_data` that allows you to obtain hourly weather data.
As a meteorology expert, you must thoroughly analyze the data and provide accurate and useful forecasts.
Take into account possible extreme heat days, especially in summer.
Remember that extreme heat is considered when maximum and minimum temperatures exceed local temperature thresholds for several consecutive days,
often during a heatwave. These temperatures, along with humidity, can be harmful to health, especially for vulnerable groups.
## Report style
All reports must be written in English.
The report must be clear, concise, and easy to understand.
It should include:
- A summary of current weather conditions.
- A detailed forecast for the coming days, including temperature, precipitation, wind, and any other relevant data.
- Practical recommendations based on the forecast, such as precautions to take or recommended activities.
- Be creative and innovative in your approach, using advanced data visualization techniques to enhance the report.
## Data visualization
The report, in markdown, must be visually appealing and innovative.
You will use tables, lists, and other formatting elements to enhance readability.
### Graph format
- Generate the graph configuration in JSON format, compatible with the Vegalite library.
- Ensure the JSON is valid and compatible with the Vegalite library.
- The graphs must be innovative, leveraging the library's potential. Do not limit yourself to simple bar or line charts. Aim for a wow effect.
- Required JSON structure:
* title: main title of the graph, at the top of the graph. The title must be brief and descriptive.
* the title must be in the layout.title.text directive
* layout.showlegend will be true/false, to show the graph legend. Some graphs do not need a legend, such as simple line charts.
- After each graph, generate a blockquote briefly explaining what the graph shows and its context.
...
For the visualization I’m using MkDocs , a simple static site generator for Markdown files. To have more advanced visualizations, I’m using the Vega-Lite library, which allows you to create interactive and visually appealing charts. The agent generates the JSON configuration for the graphs in a format compatible with Vega-Lite, which can then be rendered in the Markdown reports.
For AI, I’m using Claude 3.5 Sonnet, provided by Amazon Bedrock. For the experiment it’s enough, but if you create a cron job to run the agent every day, you’ll have your 5-day forecasting system ready to go. The project tries to show how to use AI agents to solve real-world problems, and how to integrate them with external data sources and tools. The agent can be extended to include more advanced features, such as integrating with other APIs or using more complex machine learning models for weather prediction.
Today we are going to build an agent with IA. It is just an example of how to build a agent with LangChain and AWS Bedrock and Claude 4 Sonnet. The agent will be a “mathematical expert” capable of performing complex calculations and providing detailed explanations of its reasoning process. The idea is to provide the agent with the ability to perform mathematical operations like addition, subtraction. In fact, with additions and subtractions, we can perform all the mathematical operations, like multiplication, division, exponentiation, square root, etc. The agent will be able to perform these operations step by step, providing a detailed explanation of its reasoning process. I know that we don’t need to use AI to perform these operations, but the idea is to show how to build an agent with LangChain and AWS Bedrock and Claude 4 Sonnet.
The mathematical agent implements the tool-calling pattern, allowing the LLM to dynamically select and execute mathematical operations:
Tools are defined using LangChain’s @tool decorator, providing automatic schema generation and type validation. Really we don’t need to create a class for the tools, but I have done it because I want to add an extra feature to the agent: the ability to keep a history of the operations performed. This will allow the agent to provide a detailed explanation of its reasoning process, showing the steps taken to arrive at the final result.
import logging
from typing import List
from langchain.tools import tool
logger = logging.getLogger(__name__)
class MathTools:
def __init__(self):
self.history = []
def _diff_values(self, a: int, b: int) -> int:
result = a - b
self.history.append(f"{a} - {b} = {result}")
return result
def _sum_values(self, a: int, b: int) -> int:
result = a + b
self.history.append(f"{a} + {b} = {result}")
return result
def _get_history(self) -> str:
if not self.history:
return "No previous operations"
return "\n".join(self.history[-5:]) # Last 5
def get_tools(self) -> List:
@tool
def diff_values(a: int, b: int) -> int:
"""Calculates the difference between two numbers
Args:
a (int): first number
b (int): second number
Returns:
int: difference of a - b
"""
logger.info(f"Calculating difference: {a} - {b}")
return self._diff_values(a, b)
@tool
def sum_values(a: int, b: int) -> int:
"""Sums two numbers
Args:
a (int): first number
b (int): second number
Returns:
int: sum of a + b
"""
logger.info(f"Calculating sum: {a} + {b}")
return self._sum_values(a, b)
@tool
def get_history() -> str:
"""Gets the operation history
Returns:
str: last operations
"""
logger.info("Retrieving operation history")
return self._get_history()
return [diff_values, sum_values, get_history]
The system prompt is carefully crafted to guide the agent’s behavior and establish clear operational boundaries:
AGENT_SYSTEM_PROMPT = """
You are an expert mathematical agent specialized in calculations.
You have access to the following tools:
- diff_values: Calculates the difference between two numbers
- sum_values: Sums two numbers
- get_history: Gets the operation history
Guidelines:
1. Only answer questions related to mathematical operations.
2. For complex operations, use step-by-step calculations:
- Multiplication: Repeated addition
- Division: Repeated subtraction
- Exponentiation: Repeated multiplication
- Square root: Use methods like Babylonian method or prime factorization.
"""
Now we can invoke our agent by asking questions such as ‘What’s the square root of 16 divided by two, squared?’. The agent will iterate using only the provided tools to obtain the result.
And that’s all. This project demonstrates how to build a production-ready AI agent using LangChain and AWS Bedrock. It’s just a boilerplate, but it can be extended to create more complex agents with additional capabilities and understand how AI agents work.
LLMs are highly proficient at generating code, including SQL queries from natural language text. Today, we’re going to experiment with this capability to see how effectively we can transform natural language instructions into SQL queries. The idea is to leverage the power of natural language processing to simplify the process of writing complex SQL statements. For this experiment, I’ve downloaded a CSV file containing data from IMDB, which includes various attributes related to movies, such as titles, release years, genres, and ratings. By using this dataset, we can test the LLM’s ability to generate accurate and efficient SQL queries based on different natural language prompts. Here’s an example of what the data looks like:
Now, we will create a PostgreSQL database using Docker. Docker allows us to quickly set up and manage containerized applications, making it an ideal tool for this purpose. Below is the Dockerfile we will use to set up our PostgreSQL database:
FROM postgres:16.3-alpine
COPY actors.csv /docker-entrypoint-initdb.d/actors.csv
COPY init.sql /docker-entrypoint-initdb.d/
Next, we will set up the database and import the CSV data into an ‘actors’ table using the Docker entrypoint. Below is how we configure the Docker entrypoint script to initialize the PostgreSQL database and import the CSV data:
CREATE TABLE actors (
nconst TEXT PRIMARY KEY,
primaryname TEXT,
birthyear INTEGER,
deathyear INTEGER,
primaryprofession TEXT,
knownfortitles TEXT
);
COPY actors FROM '/docker-entrypoint-initdb.d/actors.csv' CSV HEADER;
That’s the docker-compose file to set up the PostgreSQL database
Now we can start with the python script. We’re going to use cick library to build cli scrpt. The python application interacts with a database to execute SQL queries generated from user input. The process begins with obtaining a MovieChain object through the get_chain function, which takes an argument llm. This MovieChain object is then used to generate an SQL query based on the user’s input q through its get_sql method. After that we just execute the SQL query into the PostgreSQL and print the results.
import click
from dbutils import get_conn, Db, get_cursor
from lib.chains.movie import get_chain
from lib.llm.groq import llm
from settings import DSN
@click.command()
@click.option('--q', required=True, help='question to ask')
def run(q):
chain = get_chain(llm)
sql = chain.get_sql(q)
click.echo(f"q: {q}")
click.echo(sql)
click.echo('')
if sql:
conn = get_conn(DSN, named=True, autocommit=True)
db = Db(get_cursor(conn=conn))
data = db.fetch_all(sql)
for row in data:
print(row)
The MovieChain class interacts with an LLM (in this example, we’re using Groq).
import logging
from langchain_core.messages import SystemMessage, HumanMessage
from .prompts import PROMPT
logger = logging.getLogger(__name__)
class MovieChain:
def __init__(self, llm):
self.llm = llm
self.prompt = SystemMessage(content=PROMPT)
def get_sql(self, q: str):
user_message = HumanMessage(content=q)
try:
ai_msg = self.llm.invoke([self.prompt, user_message])
output_message = ai_msg.content if not isinstance(ai_msg, str) else ai_msg
return output_message
except Exception as e:
logger.error(f"Error during question processing: {e}")
The Chain uses two prompts: the system prompt that creates the proper context to assist the LLM in generating the SQL query. We’re providing the create table script.
PROMPT = """
You are an expert in generating SQL queries based on user questions.
You have access to a database with the following table schema:
CREATE TABLE actors (
nconst TEXT PRIMARY KEY,
primaryname TEXT,
birthyear INTEGER,
deathyear INTEGER,
primaryprofession TEXT,
knownfortitles TEXT
);
Please generate an SQL query to answer the following user question.
Ensure the query is valid, secure, and tailored to the provided schema.
Return only the SQL query without additional explanations.
Don't use quotes around the query in any case.
"""
And that’s all. With it we can ask quetions about this dataset and llm genetes the SQL for us.
python cli.py movie --q="List the living actors under 10 years old."
q: List the living actors under 10 years old.
SELECT * FROM actors WHERE deathyear IS NULL AND birthyear > (EXTRACT(YEAR FROM CURRENT_DATE) - 10);
...
python cli.py movie --q="List the living actors who were born in the same year as Mel Gibson."
q: List the living actors who were born in the same year as Mel Gibson
SELECT * FROM actors WHERE birthyear = (SELECT birthyear FROM actors WHERE primaryname = 'Mel Gibson') AND deathyear IS NULL;
...
cli.py movie --q="List the deceased actors who were born in the same year as Mel Gibson."
q: List the deceased actors who were born in the same year as Mel Gibson.
SELECT *
FROM actors
WHERE deathyear IS NOT NULL
AND birthyear = (SELECT birthyear
FROM actors
WHERE primaryname = 'Mel Gibson');
...
python cli.py movie --q="What is the name, date of birth, and age of the oldest living actor born in the 70s?"
q: What is the name, date of birth, and age of the oldest living actor born in the 70s?
SELECT primaryname, birthyear, (2023 - birthyear) AS age
FROM actors
WHERE birthyear >= 1970 AND birthyear < 1980 AND deathyear IS NULL
ORDER BY birthyear ASC
LIMIT 1;
{'primaryname': 'Missy Gold', 'birthyear': 1970, 'age': 53}
With projects like these, where we execute “random” SQL generated by an LLM, it’s crucial to manage user access to the database carefully. Restricting access helps mitigate potential SQL injection risks, especially depending on the prompts provided by the user when interacting with the LLM.
A few days ago, I came across a project on GitHub by my friend Jordi called Commitia. Commitia is a simple command line tool that helps you write commit messages. It works by passing a git diff to an LLM model, which then suggests a commit message based on the diff. I liked the idea of using LLM models to assist in the development process by interacting with git diffs. So, I decided to create a similar tool using Python and LangChain, just for practice. I’ll use Click to create the command line interface and LangChain to interact with the LLM model. I’ll use Azure LLM, but any LLM model that supports custom functions, like Groq LLM or OpenAI, can be used.
import click
from lib.chains.git import get_chain
from lib.git_utils import get_current_diff
from lib.llm.azure import llm
from settings import BASE_DIR
@click.command()
def run():
current_diff = get_current_diff(BASE_DIR / '..')
chain = get_chain(llm)
ia_response = chain.get_commit_message(current_diff)
click.echo(ia_response)
That’s the chain
import logging
from langchain_core.messages import SystemMessage, HumanMessage
from lib.models import DiffData
from .prompts import PROMPT_COMMIT_MESSAGE
logger = logging.getLogger(__name__)
class GitChain:
def __init__(self, llm):
self.llm = llm
self.prompt_commit_message = SystemMessage(content=PROMPT_COMMIT_MESSAGE)
@staticmethod
def _get_diff_content(diff: DiffData):
return "\n".join((d.diff for d in diff.diffs))
def get_commit_message(self, diff: DiffData):
try:
user_message = HumanMessage(content=self._get_diff_content(diff))
messages = [self.prompt_commit_message, user_message]
ai_msg = self.llm.invoke(messages)
return ai_msg if isinstance(ai_msg, str) else ai_msg.content
except Exception as e:
logger.error(f"Error during question processing: {e}")
I’m going to use the same prompt that Jordi uses in Commitia.
PROMPT_COMMIT_MESSAGE = """
You are an assistant to write the commit message.
The user will send you the content of the commit diff, and you will reply with the commit message.
It must be a commit message of one single line. Be concise, just write the message, do not give any explanation.
"""
To obtain the git diff I’m going to use gitpython library.
from git.repo import Repo
from .models import Diff, DiffData, Status
def _get_file_mode(diff):
if diff.new_file:
return Status.CREATED
elif diff.deleted_file:
return Status.DELETED
elif diff.copied_file:
return Status.COPIED
else:
return Status.MODIFIED
def _build_diff_data(commit, diffs) -> DiffData:
return DiffData(
user=str(commit.author),
date=commit.committed_datetime,
diffs=[Diff(
diff=str(diff),
path=diff.b_path if diff.new_file else diff.a_path,
status=_get_file_mode(diff)
) for diff in diffs]
)
def get_current_diff(repo_path) -> DiffData:
repo = Repo(repo_path)
commit = repo.head.commit
diffs = commit.diff(None, create_patch=True)
return _build_diff_data(commit, diffs)
I’m using the following models to represent the data.
from datetime import datetime
from enum import Enum
from typing import List
from pydantic import BaseModel
class Status(str, Enum):
CREATED = 'C'
MODIFIED = 'M'
DELETED = 'D'
COPIED = 'C'
class Diff(BaseModel):
diff: str
path: str
status: Status
class DiffData(BaseModel):
user: str
date: datetime
diffs: List[Diff]
After this experiment cloning Commitia I’m going to do another experiment. Now the idea is create a code review of based on the git diff. I’m going to use the same structure of the previous experiment. I can only need to change the prompt.
PROMPT_CODE_AUDIT = """
You are experience developer and need to perform a code review of a git diff.
You should identify potential errors, provide suggestions for improvement,
and highlight best practices in the provided code.
You should provide a global score of the code quality if you detect any issue based on the following criteria:
- NONE: 0.0
- LOW: Between 0.1 and 3.9
- MEDIUM: Between 4.0 and 6.9
- HIGH: Between 7.0 and 8.9
- CRITICAL Between 9.0 and 10.0
Your output should use the following template:
### Score
Global score of risks: [NONE, LOW, MEDIUM, HIGH, CRITICAL]
### Diff Explanation
First you must provide a brief explanation of the diff in a single line. Be concise and do not give any explanation.
Then you should provide a detailed explanation of the changes made in the diff.
### Audit summary
Detailed explanation of the identified gaps and their potential impact, if there are any significant findings.
"""
As you can see, I’m using a prompt to analyze the code and provide a score for its quality. I also want to perform actions based on the score, such as taking specific measures if the score is critical. To achieve this, we need to use a custom function. Therefore, we need an LLM model that supports calling custom functions. I added the audit_diff method to the chain to handle this.
import logging
from enum import Enum
from langchain_core.messages import SystemMessage, HumanMessage
from lib.models import DiffData
from .prompts import PROMPT_CODE_AUDIT, PROMPT_COMMIT_MESSAGE
logger = logging.getLogger(__name__)
class Score(int, Enum):
NONE = 1
LOW = 2
MEDIUM = 3
HIGH = 4
CRITICAL = 5
class GitChain:
def __init__(self, llm, tools):
self.llm = llm
if hasattr(llm, 'bind_tools'):
self.llm_with_tools = llm.bind_tools(list(tools.values()))
else:
logger.info("LLM does not support bind_tools method")
self.llm_with_tools = llm
self.prompt_code_audit = SystemMessage(content=PROMPT_CODE_AUDIT)
self.prompt_commit_message = SystemMessage(content=PROMPT_COMMIT_MESSAGE)
self.tools = tools
@staticmethod
def _get_diff_content(diff: DiffData):
return "\n".join((d.diff for d in diff.diffs))
def _get_status_from_message(self, message) -> Score | None:
ai_msg = self.llm_with_tools.invoke([HumanMessage(content=message)])
return self._get_tool_output(ai_msg)
def get_commit_message(self, diff: DiffData):
try:
user_message = HumanMessage(content=self._get_diff_content(diff))
messages = [self.prompt_commit_message, user_message]
ai_msg = self.llm.invoke(messages)
return ai_msg if isinstance(ai_msg, str) else ai_msg.content
except Exception as e:
logger.error(f"Error during question processing: {e}")
def audit_diff(self, diff: DiffData):
user_message = HumanMessage(content=self._get_diff_content(diff))
try:
ai_msg = self.llm.invoke([self.prompt_code_audit, user_message])
output_message = ai_msg.content if not isinstance(ai_msg, str) else ai_msg
return self._get_status_from_message(output_message), output_message
except Exception as e:
logger.error(f"Error during question processing: {e}")
def _get_tool_output(self, ai_msg):
status = None
for tool_call in ai_msg.tool_calls:
tool_output = self.tools[tool_call["name"]].invoke(tool_call["args"])
logger.info(f"Tool: '{tool_call['name']}'")
status = tool_output
return status
The audit_diff method takes a DiffData object as an argument, which represents the differences in the code that need to be audited. The first line inside the method creates a HumanMessage object from the content of the DiffData object by calling the _get_diff_content method, which combines all the diffs into a single string. Next, the method invokes the LLM with a system message prompt for code auditing and the human message. The LLM’s response is stored in ai_msg. If ai_msg is a string, it is used as is; otherwise, the content of ai_msg is used. The method then calls _get_status_from_message with output_message as the argument. This method invokes the LLM with tools using the output_message as input and gets the tool output. The method returns the tool output status and the output_message. In summary, the audit_diff method audits code differences using a Language Learning Model and a set of tools, and returns the audit status and the AI message content.
Now I can invoke the chain to audit the code and print the results. Also, I can use the score to perform an action.
import click
from rich.console import Console
from rich.markdown import Markdown
from lib.chains.git import get_chain
from lib.chains.git.chain import Score
from lib.git_utils import get_current_diff
from lib.llm.azure import llm
from settings import BASE_DIR
@click.command()
def run():
current_diff = get_current_diff(BASE_DIR / '..')
chain = get_chain(llm)
status, ia_response = chain.audit_diff(current_diff)
click.echo('Audit results:')
click.echo('--------------')
click.echo(f"{current_diff.user} at {current_diff.date} made the following changes at:")
click.echo('')
click.echo('Affected files:')
for diff in current_diff.diffs:
print(f"[{diff.status.value}] {diff.path}")
click.echo('')
console = Console()
console.print(Markdown(ia_response))
click.echo('')
if status == Score.CRITICAL:
click.echo('[WARNING] Critical issues found.')
else:
click.echo('No critical issues found.')
In conclusion, integrating AI into the development workflow can significantly enhance productivity and code quality. By using tools like LangChain and LLM models, we can automate the generation of commit messages and perform detailed code audits based on git diffs. This not only saves time but also ensures consistency and accuracy in commit messages and code reviews. As we continue to explore and implement these AI-driven solutions, we open up new possibilities for more efficient and effective software development practices. It is not magic, it is just code.
Today we’ll explore the integration of AI models with function calls using Python and LangChain. This example displays how to leverage LangChain for orchestrating AI and natural language processing tasks. In this example we´ll integrate AI models seamlessly with custom functions. While the functions used here are straightforward examples, such as basic arithmetic operations, they illustrate the foundational concepts applicable to more complex scenarios, such as invoking external APIs or more complicated processing pipelines. We need a LLM model with function calling capabilities (not all models allow us to call custom functions). For this example, we’re going to use Groq llm which has a public api (free) with function calling support. So, we need to obtain an api key here.
That’s the main script. It only obtains the chain with our llm instance.
import logging
from lib.chains.math_chain.chain import get_chain
from lib.llm.groq import llm
logging.basicConfig(
format='%(asctime)s [%(levelname)s] %(message)s',
level='INFO',
datefmt='%d/%m/%Y %X')
logger = logging.getLogger(__name__)
if __name__ == "__main__":
chain = get_chain(llm)
user_prompts = [
"How much is five times twelve?",
"How much is five plus twelve?",
"How much is twelve minus five?",
]
for prompt in user_prompts:
responses = chain.ask_question(prompt)
for response in responses:
print(f"Q: {prompt} R:{response}")
That’s the chain.
import logging
from langchain_core.messages import SystemMessage, HumanMessage
from .tools import tools
logger = logging.getLogger(__name__)
def get_chain(llm):
return CustomMathChain(llm, tools)
class CustomMathChain:
system_prompt_content = """
You are a model that has various mathematical functions.
You can only respond to questions related to functions that you know.
"""
def __init__(self, llm, tools):
self.llm_with_tools = llm.bind_tools(list(tools.values()))
self.system_message = SystemMessage(content=self.system_prompt_content)
self.tools = tools
def ask_question(self, user_prompt):
responses = []
try:
user_message = HumanMessage(content=user_prompt)
messages = [self.system_message, user_message]
ai_msg = self.llm_with_tools.invoke(messages)
for tool_call in ai_msg.tool_calls:
tool_output = self.tools[tool_call["name"]].invoke(tool_call["args"])
logger.info(f"Tool: '{tool_call['name']}' called output: {tool_output}")
responses.append(tool_output)
return responses
except Exception as e:
logger.error(f"Error during question processing: {e}")
This custom chain utilizes functions defined here, employing the @tool decorator. It is crucial to properly define input and output variables and provide thorough documentation for our tools. AI leverages this information to determine the appropriate function call for each scenario. Various methods exist for defining our tools; here, I’ve opted for the simplest approach. For more detailed guidance on defining custom functions, refer to this resource.
from langchain_core.tools import tool
@tool
def ia_sum(a: int, b: int) -> int:
""" Return the sum of `a` and `b` """
return a + b
@tool
def ia_diff(a: int, b: int) -> int:
""" Return the difference of `a` and `b` """
return a - b
@tool
def ia_multiply(a: int, b: int) -> int:
""" Return the product of `a` and `b` """
return a * b
tools = {
"ia_sum": ia_sum,
"ia_diff": ia_diff,
"ia_multiply": ia_multiply
}
And that’s all! Working with our custom functions is quite straightforward. As mentioned earlier, we’re using very simple functions (add, diff, and multiply). In reality, we don’t need an LLM or AI to perform these arithmetic operations. However, imagine integrating real-world functions that access APIs and your business models. AI can handle natural language processing to interpret user input and identify the correct function to execute the task.
Today, I’m delving into the realm of AI. My aim is to construct an AI capable of engaging in conversation about a specific document. For this experiment, I’ve chosen Plato’s ‘Apology of Socrates.’ My goal is to develop an expert on this text, allowing me to pose questions and receive insightful responses. Let’s dive in.
First, I need a LLaMA3 model locally on my computer (MBP M2 24GB). To do that we can use Ollama. It’s pretty straightforward to do that on Mac. Just follow the instructions, do
brew install ollama
and that’s all. We can start the server.
ollama start
Now we need the model. We’re going LLaMA3. A 4.7 GB model that we can download using:
ollama pull llama3
And that’s all. Our server is up and running ready to receive requests. Now we’re going to create our script. We can use simple HTTP requests to interact with Ollama using postman, for example, but it’s simpler to use a framework to handle the communications. We’re going to use [LangChain](https://www.langchain.com/).
IAs models has a limitation of the number of tokens that we can use as I/O parameters. Apology of Socrates is a book. Not excessively big but big enough to overcome this limit so, we need to split it in chucks. Also, we need to convert those chunks into a vector store to be able the model to understand it. LangChain provides us document loaders to read the document and to create this vector store. In my example I’m using an Apology of Socrates in txt, so I’m going to use a TextLoader, but there are different loaders for PDFs, S3, Dataframes and much more things available in LangChain SDK. With this function I obtain the vector store from a path.
import logging
from langchain_community.document_loaders import TextLoader
from langchain_community.embeddings import GPT4AllEmbeddings
from langchain_community.vectorstores import Chroma
from langchain_text_splitters import RecursiveCharacterTextSplitter
logger = logging.getLogger(__name__)
def get_vector_store_from_path(file_path):
loader = TextLoader(file_path)
data = loader.load()
text_splitter = RecursiveCharacterTextSplitter(chunk_size=1500, chunk_overlap=100)
all_splits = text_splitter.split_documents(data)
logger.info(f"Text divided in {len(all_splits)} splits")
return Chroma.from_documents(
documents=all_splits, embedding=GPT4AllEmbeddings()
)
Now we need a chain to ask question to oru model. With this function I obtain my chain.
I’m using an Ollama llm model, running locally on my computer as I explain before. LangChain allows us to use different llm models (Azure, OpenAI,…). We can use those models if we’ve an account (they aren’t for free)
from langchain_community.llms.ollama import Ollama
from langchain_core.callbacks import CallbackManager, StreamingStdOutCallbackHandler
import logging
from settings import OLLAMA_MODEL
logger = logging.getLogger(__name__)
llm = Ollama(
model=OLLAMA_MODEL,
verbose=True,
callback_manager=CallbackManager([StreamingStdOutCallbackHandler()]),
)
logger.info(f"Model {OLLAMA_MODEL} loaded")
With those functions I can build finally my script. As you can see, I prepare a template telling to llm what I want and the set of questions I’m going to ask the model. Our main function will first fetch the vector store (it takes several seconds). After that will load the llm from the chain (takes time also). Then we iterate between questions and print the llm’s answer in the terminal.
import logging
from lib.llm.ollama import llm
from lib.utils import get_chain, get_vector_store_from_path
from settings import DOCUMENT_PATH
logging.basicConfig(
format='%(asctime)s [%(levelname)s] %(message)s',
level='INFO',
datefmt='%d/%m/%Y %X')
logger = logging.getLogger(__name__)
def ask_question(chain, question):
logger.info(f"QUESTION: {question}")
response = chain.invoke(question)
print(response)
def main(template, path, questions):
vector_store = get_vector_store_from_path(path)
chain = get_chain(
template=template,
vector_store=vector_store,
llm=llm)
for question in questions:
ask_question(
chain=chain,
question=question
)
if __name__ == "__main__":
template = """
Answer the question based only on the context I give you.
Answer using quotes from the text.
Context: {context}
Question: {question}
"""
questions = (
"What are the general ideas of the text?"
"What is Socrates' position regarding his imminent condemnation?"
"Can you list the protagonists of the plot?"
)
main(template=template, path=DOCUMENT_PATH, questions=questions)
And that’s all. We have a Plato expert to chat with about one specific context (in this case Apology of Socrates). However, for a production-grade project, it’s crucial to store our vector data in a database to avoid repetitive generation.
Note: In my example the questions, template and Plato’s book is in Spanish. Plato’s book public domain. Source code available on my github.
Gonzalo Ayuso - Web Architect 