Tutorial
How to Deploy an AI Agent
An agent that only runs in a notebook helps nobody. Deploying an AI agent means wrapping it in a web API, putting it on a server with its secrets set safely, and giving people a public URL. This guide takes a Python agent to production step by step with FastAPI, compares where to host it (including free options), covers LangGraph deployment, and shows how to keep the LLM bill under control.
By Abdul Rahman Azam, founder of AI Season · Updated
Key facts
| Deploying an agent means | Running it as a web service that others can reach at a public URL |
|---|---|
| Usual stack | Python, FastAPI, a Dockerfile, a hosting platform, environment variables for keys |
| Free hosting options | Platforms with free tiers such as Render, Hugging Face Spaces and Google Cloud Run's free allowance |
| Biggest running cost | LLM API tokens, not the server |
| Before launch | Secrets out of code, rate limits, step caps, logging and a spending limit |
| You need | A working agent, basic Python and a GitHub account |
What does it mean to deploy an AI agent?
To deploy an AI agent is to run it on a server as a long-lived service, usually behind an HTTP API, so a website, a WhatsApp bot or another program can call it at any time. Your laptop script becomes an endpoint such as POST /chat that accepts a message and returns the agent's answer.
The agent's intelligence still comes from a hosted model through its API. What you deploy is the code around it: the loop, tools, memory and guardrails. That is why a small, cheap server is normally enough and no GPU is needed.
Step 1: wrap the agent in an API with FastAPI
# main.py
import os
from fastapi import FastAPI, HTTPException
from pydantic import BaseModel, Field
from openai import OpenAI
app = FastAPI(title="My AI agent")
client = OpenAI(api_key=os.environ["OPENAI_API_KEY"]) # fails fast if missing
class ChatRequest(BaseModel):
message: str = Field(min_length=1, max_length=2000)
class ChatResponse(BaseModel):
answer: str
def run_agent(message: str) -> str:
# Replace with your own tool-calling loop or LangGraph graph.
reply = client.chat.completions.create(
model="gpt-4o-mini",
messages=[
{"role": "system", "content": "You are a concise, helpful assistant."},
{"role": "user", "content": message},
],
max_tokens=500,
)
return reply.choices[0].message.content
@app.get("/health")
def health():
return {"status": "ok"}
@app.post("/chat", response_model=ChatResponse)
def chat(request: ChatRequest):
try:
return ChatResponse(answer=run_agent(request.message))
except Exception:
raise HTTPException(status_code=502, detail="The model call failed. Try again.")Run it locally with uvicorn main:app --reload and open http://localhost:8000/docs to test the endpoint in the browser. The /health route lets the hosting platform check the service is alive.
Step 2: handle secrets and configuration
- Never commit API keys. Keep them in a local
.envfile, add.envto.gitignore, and set the same variables in your host's dashboard. - Read config from the environment: model name, limits and URLs. Then you can change them without redeploying code.
- Pin your dependencies in
requirements.txtso the server installs the versions you tested. - Set a spending limit in your LLM provider's dashboard before you share the URL.
# requirements.txt
fastapi
uvicorn[standard]
openai
pydantic
# Dockerfile
FROM python:3.12-slim
WORKDIR /app
COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt
COPY . .
CMD ["sh", "-c", "uvicorn main:app --host 0.0.0.0 --port ${PORT:-8000}"]Step 3: choose where to host your AI agent
| Option | Good for | Free tier | Watch for |
|---|---|---|---|
| Render | First deployment from a GitHub repo | Yes, for small web services | Free services sleep when idle, so the first request is slow |
| Hugging Face Spaces | Demos with a Gradio or Streamlit UI | Yes, on basic CPU | Public by default; meant for demos |
| Google Cloud Run | Containers that scale to zero | Monthly free allowance | Needs a billing account; cold starts |
| Railway / Fly.io | Always-on small services | Trial credit or small allowance | Usage-based billing after that |
| Vercel / Cloudflare Workers | Short, serverless requests and front ends | Yes | Execution time limits; long agent runs may time out |
| A VPS | Full control, background workers | No | You manage updates and security |
| LangGraph Platform | LangGraph agents needing persistence and long runs | Limited | Tied to the LangGraph ecosystem |
Free tiers and limits change often. Check each provider's current pricing page before you depend on one.
For a first project, push the code to GitHub, create a web service on a platform with a free tier, add OPENAI_API_KEY in its environment settings and deploy. You get a public HTTPS URL in a few minutes.
How to deploy a LangGraph agent
A LangGraph graph is ordinary Python, so the FastAPI approach above works: compile the graph once at start-up and call graph.invoke inside the endpoint. Two things need care in production.
- Persistence. Replace the in-memory checkpointer with a Postgres or SQLite checkpointer, or conversations vanish on every restart. See agent memory.
- Thread IDs. Pass a per-user or per-conversation
thread_idfrom the request so each user gets their own state. - Long runs and streaming. Use
graph.streamwith server-sent events so the user sees progress instead of a timeout. - Human approval. Interrupts need the state stored between the pause and the resume, which is another reason for a real database.
LangGraph Platform is the managed alternative: it hosts the graph with persistence, queues and streaming built in. Start with FastAPI to understand the pieces. The LangGraph tutorial builds the graph this section deploys.
LLM cost optimization: keeping the bill small
| Technique | How it saves | Effort |
|---|---|---|
| Use a smaller model by default | Small models cost a fraction of flagship ones and handle most steps | Low |
| Route by difficulty | Send only hard requests to the large model | Medium |
| Prompt caching | Repeated prompt prefixes are billed at a discount by major providers | Low |
| Trim the context | Fewer input tokens on every step: summarise history, return short tool results | Medium |
| Cap output and steps | max_tokens and a loop limit stop runaway runs | Low |
| Cache answers | Identical questions are served without a model call | Medium |
| Batch offline work | Batch APIs are cheaper for jobs that can wait | Low |
Measure before optimising: log tokens per request, then fix the largest number. In agents that number is usually input tokens, because the full history is re-sent on every step.
A production checklist
- Keys in environment variables, none in the repository
- Input length limits and rate limiting per user or IP
- A step cap and timeout on the agent loop
- Guardrails and approval for risky tools. See agent security
- Logging and tracing of every run. See agent evaluation and observability
- A health endpoint and an alert when it fails
- A provider spending limit and a daily cost check
- A small test set you run before each deploy
How AI Season teaches deployment
Session 7 of the AI Season curriculum is Deployment. Every student takes an agent from a notebook to a hosted service with a public URL, handling environment config, secrets and cold starts, and the next session adds guardrails to it. Cohort 02 starts 1st January 2027.
Frequently asked questions
How do I deploy an AI agent?
Wrap the agent in a web API such as FastAPI, put your API keys in environment variables, add a requirements file or Dockerfile, push the code to GitHub and connect it to a hosting platform. The platform builds it and gives you a public URL.
Can I host an AI agent for free?
Yes, for small projects. Several platforms have free tiers suitable for demos, with limits such as services sleeping when idle. You still pay the model provider for tokens unless you use a free model tier.
Do I need a GPU to deploy an AI agent?
No, not when the agent uses a hosted model through an API. The server only runs your Python code, so a small CPU instance is enough. You need a GPU only if you host the model yourself.
How do I deploy a LangGraph agent?
Either serve the compiled graph behind your own FastAPI endpoint with a database-backed checkpointer, or use LangGraph Platform, which provides persistence, streaming and long-running tasks as a managed service.
How can I reduce LLM costs in production?
Use a smaller model by default, cache repeated prompts, trim conversation history and tool results, cap output tokens and loop steps, and route only difficult requests to a larger model.
Where should I store API keys?
In environment variables set in your hosting platform's dashboard, never in the code or the repository. Use a local .env file for development and keep it out of version control.