# Excalidraw AI: Text-to-Diagram Agent

URL: https://aiseason.tech/excalidraw-ai-agent

Excalidraw is a free, open-source whiteboard for hand-drawn style diagrams. Because every Excalidraw drawing is plain JSON, a language model can write one: you type "draw the architecture of a RAG chatbot" and an agent returns an editable diagram. This page explains how text-to-diagram AI works with Excalidraw, shows the core of it in Python, and describes the capstone project AI Season students build in week 6.

| Fact | Value |
| --- | --- |
| Excalidraw is | An open-source virtual whiteboard that saves drawings as JSON |
| Excalidraw AI means | Using a language model to generate or edit Excalidraw diagrams from a text prompt |
| Why it works | A diagram is a list of elements (rectangles, arrows, text) that a model can produce as structured output |
| Skills used | Structured output, tool calling, validation, layout logic, deployment |
| AI Season capstone | Session 11: an agent that turns a prompt into Excalidraw diagrams |
| Output | A `.excalidraw` file you can open and keep editing at excalidraw.com |

### What is Excalidraw AI?

**Excalidraw AI is the use of a language model to create diagrams in Excalidraw from a description in plain language, often called text to diagram.** Excalidraw itself includes a text-to-diagram feature, and because the file format is open JSON, anyone can build their own generator with an LLM API.

The result is different from an AI-generated image. An image is pixels you cannot change. An Excalidraw diagram is a set of shapes, so you can drag a box, fix a label or add a step after the model has finished.

### How text-to-diagram AI works

1. **The user describes the diagram**: "a flowchart of how a tool-calling agent answers a question".
2. **The model plans the content** as a simple graph: nodes with labels and edges between them.
3. **Code lays it out.** A layout step assigns x and y positions so boxes do not overlap. Models are poor at arithmetic on coordinates, so this is better done in code.
4. **Code converts the graph to Excalidraw elements**: one rectangle and one text element per node, one arrow per edge.
5. **Validate** that the JSON matches the format and every arrow points to a node that exists.
6. **Return the file**, or render it in an embedded Excalidraw canvas.

> The key design decision: let the model decide what to draw, and let ordinary code decide where to draw it. Asking the model for raw pixel coordinates gives overlapping, crooked diagrams.

### The Excalidraw file format in brief

An `.excalidraw` file is a JSON object with a `type`, a `version` and an `elements` array. Each element has a type (`rectangle`, `ellipse`, `diamond`, `arrow`, `line`, `text`), a position, a size and style fields.

```json
{
  "type": "excalidraw",
  "version": 2,
  "source": "my-diagram-agent",
  "elements": [
    {
      "id": "node-user", "type": "rectangle",
      "x": 100, "y": 100, "width": 200, "height": 70,
      "strokeColor": "#1e1e1e", "backgroundColor": "#a5d8ff",
      "fillStyle": "solid", "roughness": 1
    },
    {
      "id": "label-user", "type": "text",
      "x": 150, "y": 122, "text": "User question", "fontSize": 20
    }
  ],
  "appState": { "viewBackgroundColor": "#ffffff" },
  "files": {}
}
```

Real files carry more fields per element (seeds, versions, bindings). Excalidraw's open-source library includes a helper that expands a short element description into a full one, which is the easiest path in a web app.

### A minimal diagram agent in Python

This script asks a model for a graph using structured output, lays the nodes out in a column, and writes a file you can open at excalidraw.com with File, Open.

```python
import json
from openai import OpenAI
from pydantic import BaseModel

class Node(BaseModel):
    id: str
    label: str

class Edge(BaseModel):
    source: str
    target: str

class Diagram(BaseModel):
    nodes: list[Node]
    edges: list[Edge]

client = OpenAI()

def plan_diagram(prompt: str) -> Diagram:
    completion = client.beta.chat.completions.parse(
        model="gpt-4o-mini",
        messages=[
            {"role": "system", "content": "Turn the request into a flowchart of at most 8 nodes. "
             "Use short labels. Every edge must connect two existing node ids."},
            {"role": "user", "content": prompt},
        ],
        response_format=Diagram,
    )
    return completion.choices[0].message.parsed

def to_excalidraw(diagram: Diagram) -> dict:
    width, height, gap = 240, 70, 60
    position, elements = {}, []
    for i, node in enumerate(diagram.nodes):          # layout is done in code
        x, y = 100, 100 + i * (height + gap)
        position[node.id] = (x, y)
        elements.append({"id": node.id, "type": "rectangle", "x": x, "y": y,
                         "width": width, "height": height, "strokeColor": "#1e1e1e",
                         "backgroundColor": "#a5d8ff", "fillStyle": "solid", "roughness": 1})
        elements.append({"id": f"{node.id}-text", "type": "text", "x": x + 16, "y": y + 22,
                         "text": node.label, "fontSize": 20, "strokeColor": "#1e1e1e"})
    for n, edge in enumerate(diagram.edges):
        if edge.source not in position or edge.target not in position:
            continue                                   # validation: drop broken edges
        sx, sy = position[edge.source]
        tx, ty = position[edge.target]
        start = (sx + width / 2, sy + height)
        elements.append({"id": f"edge-{n}", "type": "arrow", "x": start[0], "y": start[1],
                         "points": [[0, 0], [tx - sx, ty - (sy + height)]],
                         "strokeColor": "#1e1e1e", "endArrowhead": "arrow"})
    return {"type": "excalidraw", "version": 2, "source": "diagram-agent",
            "elements": elements, "appState": {"viewBackgroundColor": "#ffffff"}, "files": {}}

diagram = plan_diagram("How a RAG chatbot answers a question from a PDF")
with open("diagram.excalidraw", "w") as f:
    json.dump(to_excalidraw(diagram), f, indent=2)
print("Saved diagram.excalidraw")
```

If the file does not open cleanly in your Excalidraw version, pass the elements through Excalidraw's own element-conversion helper in a small web page; it fills in the remaining fields. Field requirements have changed between releases, so check the current Excalidraw developer docs.

### Turning the script into an agent

- **Edit, not only create.** Give the model tools such as `add_node`, `connect`, `rename` and `delete_node` so the user can say "add a cache between the API and the database". This is [tool calling](https://aiseason.tech/tool-calling) on a shared diagram state.
- **Better layout.** Replace the single column with a layered layout so branches sit side by side.
- **Self-check.** Validate the output and send errors back to the model to fix, a small [evaluator loop](https://aiseason.tech/ai-agent-design-patterns).
- **Diagram types.** Flowcharts, sequence diagrams, architecture diagrams and mind maps each need their own layout rule.
- **A live canvas.** Embed the open-source Excalidraw React component and update it as the agent works.
- **Ship it.** Put the agent behind an API and give it a public URL. See [deploy an AI agent](https://aiseason.tech/deploy-ai-agent).

### Excalidraw AI vs other text-to-diagram tools

| Approach | Output | Editable | Style |
| --- | --- | --- | --- |
| Excalidraw JSON | Shapes on a canvas | Yes, by drag and drop | Hand-drawn |
| Mermaid | Text syntax rendered as a diagram | Yes, by editing text | Clean, automatic layout |
| Graphviz DOT | Text syntax rendered as a graph | Yes, by editing text | Technical |
| Image generation model | A picture | No | Any, but text is often wrong |

A common trick is to have the model write Mermaid, which it does reliably, and then convert Mermaid to Excalidraw elements. Excalidraw supports that conversion for flowcharts.

### The AI Season Excalidraw capstone

Session 11 of the [AI Season curriculum](https://aiseason.tech/curriculum) is the Excalidraw Project: each student ships an agent that turns a prompt into structured Excalidraw diagrams, end to end. It was chosen as the capstone because it uses most of the course in one project:

| Course skill | Where it shows up in the capstone |
| --- | --- |
| Prompt engineering | A system prompt that produces a clean graph every time |
| Structured output and tool calling | The diagram schema and the edit tools |
| Design patterns | Plan, then draw, then validate and repair |
| LangGraph | State for the diagram across turns |
| Guardrails | Rejecting oversized or off-topic requests |
| Deployment | A public URL others can try |

The finished project is a visual portfolio piece: an interviewer or client can type a sentence and watch a diagram appear. See more [AI agent project ideas](https://aiseason.tech/ai-agent-projects). Cohort 02 starts 1st January 2027.

### Frequently asked questions

**Does Excalidraw have AI?**
Yes. Excalidraw offers a text-to-diagram feature, and because its files are open JSON you can also generate diagrams with your own LLM agent.

**How does text to diagram AI work?**
A language model converts your description into a structured list of nodes and connections. Code then positions the shapes and writes them in a diagram format such as Excalidraw JSON or Mermaid.

**Can ChatGPT or Claude create Excalidraw diagrams?**
They can write Excalidraw JSON or Mermaid code that you import into Excalidraw. Results improve a lot when code handles the layout and the model only decides the content.

**Is Excalidraw free?**
The Excalidraw editor is free and open source. A paid plan adds collaboration and hosted features.

**What do AI Season students build with Excalidraw?**
The week 6 capstone is an agent that turns a text prompt into editable Excalidraw diagrams, combining structured output, tool calling, validation and deployment in one portfolio project.

---

_Last updated: 2026-11-16. Canonical page: https://aiseason.tech/excalidraw-ai-agent. Source: AI Season (https://aiseason.tech/)._
