How to Deploy an AI Agent

An agent that only runs in a notebook helps nobody. Deploying an AI agent means wrapping it in a web API, putting it on a server with its secrets set safely, and giving people a public URL. This guide takes a Python agent to production step by step with FastAPI, compares where to host it (including free options), covers LangGraph deployment, and shows how to keep the LLM bill under control.

By Abdul Rahman Azam, founder of AI Season · Updated

Learn it live with us →

Key facts

Deploying an agent means Running it as a web service that others can reach at a public URL
Usual stack Python, FastAPI, a Dockerfile, a hosting platform, environment variables for keys
Free hosting options Platforms with free tiers such as Render, Hugging Face Spaces and Google Cloud Run's free allowance
Biggest running cost LLM API tokens, not the server
Before launch Secrets out of code, rate limits, step caps, logging and a spending limit
You need A working agent, basic Python and a GitHub account

What does it mean to deploy an AI agent?

To deploy an AI agent is to run it on a server as a long-lived service, usually behind an HTTP API, so a website, a WhatsApp bot or another program can call it at any time. Your laptop script becomes an endpoint such as POST /chat that accepts a message and returns the agent's answer.

The agent's intelligence still comes from a hosted model through its API. What you deploy is the code around it: the loop, tools, memory and guardrails. That is why a small, cheap server is normally enough and no GPU is needed.

Step 1: wrap the agent in an API with FastAPI

# main.py
import os
from fastapi import FastAPI, HTTPException
from pydantic import BaseModel, Field
from openai import OpenAI

app = FastAPI(title="My AI agent")
client = OpenAI(api_key=os.environ["OPENAI_API_KEY"])  # fails fast if missing

class ChatRequest(BaseModel):
    message: str = Field(min_length=1, max_length=2000)

class ChatResponse(BaseModel):
    answer: str

def run_agent(message: str) -> str:
    # Replace with your own tool-calling loop or LangGraph graph.
    reply = client.chat.completions.create(
        model="gpt-4o-mini",
        messages=[
            {"role": "system", "content": "You are a concise, helpful assistant."},
            {"role": "user", "content": message},
        ],
        max_tokens=500,
    )
    return reply.choices[0].message.content

@app.get("/health")
def health():
    return {"status": "ok"}

@app.post("/chat", response_model=ChatResponse)
def chat(request: ChatRequest):
    try:
        return ChatResponse(answer=run_agent(request.message))
    except Exception:
        raise HTTPException(status_code=502, detail="The model call failed. Try again.")
main.py: a minimal agent API

Run it locally with uvicorn main:app --reload and open http://localhost:8000/docs to test the endpoint in the browser. The /health route lets the hosting platform check the service is alive.

Step 2: handle secrets and configuration

  • Never commit API keys. Keep them in a local .env file, add .env to .gitignore, and set the same variables in your host's dashboard.
  • Read config from the environment: model name, limits and URLs. Then you can change them without redeploying code.
  • Pin your dependencies in requirements.txt so the server installs the versions you tested.
  • Set a spending limit in your LLM provider's dashboard before you share the URL.
# requirements.txt
fastapi
uvicorn[standard]
openai
pydantic

# Dockerfile
FROM python:3.12-slim
WORKDIR /app
COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt
COPY . .
CMD ["sh", "-c", "uvicorn main:app --host 0.0.0.0 --port ${PORT:-8000}"]
requirements.txt and a Dockerfile that works on most platforms

Step 3: choose where to host your AI agent

OptionGood forFree tierWatch for
RenderFirst deployment from a GitHub repoYes, for small web servicesFree services sleep when idle, so the first request is slow
Hugging Face SpacesDemos with a Gradio or Streamlit UIYes, on basic CPUPublic by default; meant for demos
Google Cloud RunContainers that scale to zeroMonthly free allowanceNeeds a billing account; cold starts
Railway / Fly.ioAlways-on small servicesTrial credit or small allowanceUsage-based billing after that
Vercel / Cloudflare WorkersShort, serverless requests and front endsYesExecution time limits; long agent runs may time out
A VPSFull control, background workersNoYou manage updates and security
LangGraph PlatformLangGraph agents needing persistence and long runsLimitedTied to the LangGraph ecosystem

Free tiers and limits change often. Check each provider's current pricing page before you depend on one.

For a first project, push the code to GitHub, create a web service on a platform with a free tier, add OPENAI_API_KEY in its environment settings and deploy. You get a public HTTPS URL in a few minutes.

How to deploy a LangGraph agent

A LangGraph graph is ordinary Python, so the FastAPI approach above works: compile the graph once at start-up and call graph.invoke inside the endpoint. Two things need care in production.

  • Persistence. Replace the in-memory checkpointer with a Postgres or SQLite checkpointer, or conversations vanish on every restart. See agent memory.
  • Thread IDs. Pass a per-user or per-conversation thread_id from the request so each user gets their own state.
  • Long runs and streaming. Use graph.stream with server-sent events so the user sees progress instead of a timeout.
  • Human approval. Interrupts need the state stored between the pause and the resume, which is another reason for a real database.

LangGraph Platform is the managed alternative: it hosts the graph with persistence, queues and streaming built in. Start with FastAPI to understand the pieces. The LangGraph tutorial builds the graph this section deploys.

LLM cost optimization: keeping the bill small

TechniqueHow it savesEffort
Use a smaller model by defaultSmall models cost a fraction of flagship ones and handle most stepsLow
Route by difficultySend only hard requests to the large modelMedium
Prompt cachingRepeated prompt prefixes are billed at a discount by major providersLow
Trim the contextFewer input tokens on every step: summarise history, return short tool resultsMedium
Cap output and stepsmax_tokens and a loop limit stop runaway runsLow
Cache answersIdentical questions are served without a model callMedium
Batch offline workBatch APIs are cheaper for jobs that can waitLow

Measure before optimising: log tokens per request, then fix the largest number. In agents that number is usually input tokens, because the full history is re-sent on every step.

A production checklist

  • Keys in environment variables, none in the repository
  • Input length limits and rate limiting per user or IP
  • A step cap and timeout on the agent loop
  • Guardrails and approval for risky tools. See agent security
  • Logging and tracing of every run. See agent evaluation and observability
  • A health endpoint and an alert when it fails
  • A provider spending limit and a daily cost check
  • A small test set you run before each deploy

How AI Season teaches deployment

Session 7 of the AI Season curriculum is Deployment. Every student takes an agent from a notebook to a hosted service with a public URL, handling environment config, secrets and cold starts, and the next session adds guardrails to it. Cohort 02 starts 1st January 2027.

Frequently asked questions

How do I deploy an AI agent?

Wrap the agent in a web API such as FastAPI, put your API keys in environment variables, add a requirements file or Dockerfile, push the code to GitHub and connect it to a hosting platform. The platform builds it and gives you a public URL.

Can I host an AI agent for free?

Yes, for small projects. Several platforms have free tiers suitable for demos, with limits such as services sleeping when idle. You still pay the model provider for tokens unless you use a free model tier.

Do I need a GPU to deploy an AI agent?

No, not when the agent uses a hosted model through an API. The server only runs your Python code, so a small CPU instance is enough. You need a GPU only if you host the model yourself.

How do I deploy a LangGraph agent?

Either serve the compiled graph behind your own FastAPI endpoint with a database-backed checkpointer, or use LangGraph Platform, which provides persistence, streaming and long-running tasks as a managed service.

How can I reduce LLM costs in production?

Use a smaller model by default, cache repeated prompts, trim conversation history and tool results, cap output tokens and loop steps, and route only difficult requests to a larger model.

Where should I store API keys?

In environment variables set in your hosting platform's dashboard, never in the code or the repository. Use a local .env file for development and keep it out of version control.