Artificial Intelligence doing Vector Graphics: LLM agents and Inkscape

TL;DR: The project is hosted on GitHub and PyPi. The results of a first test session are two posters, a simple A0 poster about the MCP itself as well as an alternative pastel version

A colleague told me he was not in the mood to spend one or two hours designing a poster. Naturally, I took that as a reason to spend a few hours developing and testing software to automate the process. The time savings on that first poster might be debatable, but at least there would be something reusable for the next one.

In my previous article about LLMs working with FreeCAD, I described an approach that I find particularly interesting: give a model access to a real application, let it inspect what it has produced, and allow it to keep correcting the result. The editable model and the visual feedback were central to making that work. That same idea has a very natural application in another part of the scientific and engineering workflow: creating figures, diagrams, and posters.

These are documents in which details and especially editability matter. And when a collaborator asks for a correction, it should be possible to change that particular element without rebuilding the entire document.

For this I built mcpInkscape, an MCP server that gives an agent access to typed vector drawing operations, document inspection, rendering, and export. It can work with SVG documents without an open window, or connect to a running Inkscape session through an optional native bridge. The latter makes something especially interesting possible: a person and an agent can work on the same visible, editable document - in a collaborative fashion.

What appeals to me here is the continuity with the CAD work. The agent has a structured representation it can change and a rendered representation it can inspect. The final deliverable is an ordinary document that I can continue editing myself. The two A0 science posters shown below provide concrete examples of the process, using the project itself as their subject.

• The Idea
• The Inkscape MCP Approach
• Working Together in a Live Document
• Why Visual Inspection Matters Again
• Design Requires a Workflow
• Review Records and Validation
• Results: A0 Science Posters
• Current Limitations
• What comes next
• References

The Idea

The starting point is quite simple: let an agent build graphics out of the same kinds of objects that one would manipulate manually in Inkscape. Rectangles, paths, text, groups, layers, fills, strokes and transformations become operations it can request and inspect.

This is useful because an SVG contains more than its final appearance. A caption can remain a text object. A schematic can consist of named groups. The background, figures, annotations, and main text can occupy separate layers. That structure makes later corrections much easier, both for the agent and for a human editor. For example, a request such as “move the explanation below the diagram and make the figure labels larger” can become a sequence of changes to existing objects. Their contents and relationships can be inspected before and after the edit. Once the result has been rendered, the agent can check whether the larger labels actually fit and whether the revised reading order makes sense.

One can of course generate SVG directly as text. For a small drawing that is often entirely sufficient. A persistent document interface becomes useful as the work grows: the agent can retain object identities, make focused changes, inspect the current state, and obtain visual feedback throughout the process. This also creates a practical meeting point between automated work and manual editing.

The additional gain is the deliberately constrained set of actions. Direct textual editing lets the agent rewrite arbitrary parts of the SVG; the MCP interface instead limits it to defined operations on drawing objects, with typed parameters and validation. In the native live workflow, these operations also pass through Inkscape’s application logic and document model. Together, those constraints narrow the agent’s action space: it chooses an operation such as moving an object or changing a fill, while the application handles the corresponding document changes. This reduces the degrees of freedom the agent has to manage, although it still has to judge whether the resulting design is correct. The offline backend applies the MCP interface’s own constraints without executing edits inside the running Inkscape application.

Scientific communication is a particularly appealing use case. A plot might originate in an analysis program, a schematic might be assembled from vector objects, and the surrounding explanation might need several rounds of revision. Inkscape can be the place where these elements become one coherent figure or poster, while the underlying analysis remains in the tools that produced it.

The Inkscape MCP Approach

The project exposes drawing operations through the Model Context Protocol, allowing an MCP-capable agent to discover and call them. The server provides both local stdio and authenticated Streamable HTTP transport. The choice of transport is separate from the choice of document backend: HTTP does not itself imply live GUI editing and a local agent can also work through the native bridge.

The HTTP endpoint is not designed to be exposed directly to the public internet. It is intended to run behind a reverse proxy that provides TLS and appropriate access controls, or to be reachable only on a protected subnet. API-key authentication does not replace that deployment boundary.

There are three backend paths, with deliberately different capabilities.

• The offline SVG backend creates and edits documents under a configured document root. It supports document lifecycle operations, layers, stable object IDs, geometry, text, appearance, and composition. Inkscape’s command-line renderer supplies PNG snapshots and the export path. No running GUI or native extension is needed for this workflow, although rendering and export still depend on the Inkscape executable.
• The active-window CLI backend provides a more limited compatibility path to a running Inkscape window. It probes the actions advertised by the installed executable and exposes supported selection, style, transformation, and rendering operations. This is useful, but it does not provide the full collaborative drawing surface or revision protection of the native extension.
• The native C++ bridge operates inside Inkscape and executes supported edits on the GUI thread. It supplies live drawing, inspection, styling, transformations, structure changes, named undo transactions, snapshots, and change polling. Because it is an optional component, the server advertises its tools according to the methods the connected bridge actually supports.

The drawing surface includes:

• Rectangles, circles, ellipses, lines, polylines, polygons, paths, and text
• Fill and stroke styling, opacity, gradients, and page background
• Movement, rotation, scaling, grouping, duplication, and stacking order
• Embedded PNG and JPEG assets from the configured document area
• Document and object inspection, bounds queries, and rendered snapshots
• Editable SVG, plain SVG, PNG, and PDF output

Absolute SVG units such as millimetres and points are supported, which is convenient when the requested result has a physical size. An A0 poster can start with its actual page dimensions instead of an arbitrary pixel canvas that must be interpreted later.

There is an important architectural difference from the FreeCAD bridge described in the earlier article. This server exposes a bounded set of typed operations. It does not accept arbitrary Python, shell commands, raw SVG replacement, or arbitrary Inkscape action strings from the agent. The interface describes changes to a drawing, and the implementation decides how to carry them out.

That also helps keep the workflow understandable. An operation that moves a known text object or changes its stroke has a much more specific meaning than a general script. Object identities and revisions provide something concrete to inspect when a change needs to be understood or corrected.

Working Together in a Live Document

The live bridge lets the agent participate in an ordinary editing session. Its changes appear in Inkscape, and the user can continue manipulating the document with the familiar interface. This immediately introduces a problem: the state the agent last inspected may no longer be current. Suppose it plans a change to a caption, but I move that caption and alter the surrounding layout before the edit arrives. Applying the original plan without checking could damage the new arrangement.

The native bridge therefore supports document revisions and expected-revision checks. An edit based on an outdated revision can be rejected, allowing the agent to inspect the updated document before deciding what to do next. Polling also exposes document and selection changes, including changes made directly by a person in Inkscape. Selection state is tracked separately because selecting an object is different from modifying the drawing.

This does not automatically resolve a disagreement about design. It gives the agent a way to notice that its assumptions need updating. That is a useful foundation for collaboration: inspect the current state, apply a bounded change, and inspect again.

Named undo transactions make the changes easier to navigate from the application side. The native bridge connection stays local to Inkscape; on FreeBSD and Linux it uses a private Unix-domain socket in the users ~/.local directory. The optional remote MCP transport sits outside that local connection and applies its own API-key and document-access policies.

This supports a shared editing process. The user can adjust the overall direction, reposition a region, or select something that needs attention. The agent can handle repetitive changes, assemble a first composition, or work through a series of corrections while keeping the document editable throughout.

Why Visual Inspection Matters Again

The same lesson from FreeCAD appears here almost immediately: a successful tool call does not establish that the result is right.

A text object may contain the requested words and occupy the intended coordinates, yet be difficult to read. Two objects may remain inside the page while overlapping each other. Every section may be present, but the page can still give the reader no clear indication of where to start.

Object inspection and bounds checks answer useful structural questions. Rendered feedback answers a different set of questions: what does the page actually communicate, which elements compete for attention, and which details disappear at the intended reading scale? The distinction is especially clear with posters. A full-page overview helps judge the overall hierarchy and balance. A detail view helps inspect captions, labels, symbols, and line breaks. Both are needed. A composition that looks convincing as a thumbnail may contain text that is much too small, while individually tidy panels can add up to an overcrowded page.

The intended loop is therefore straightforward: build a composition, render it, diagnose a specific problem, make a targeted correction and render again. The diagnosis matters. “The caption is too close to the arrow” suggests an actionable change. “Make it more professional” leaves a great deal unresolved.

Export adds another boundary. Inspecting the live canvas does not establish that the delivered PDF has the intended page dimensions, fonts, and appearance. The exported result needs its own review. Rendering the PDF separately checks the export through another path, although it still does not replace a physical print proof.

This is closely related to the independent representations discussed in the CAD article. Here they are the object structure, numerical bounds, rendered document, extracted text, and exported page. Each can reveal errors that are easy to miss in the others. Agreement between them is useful evidence, even though the communication itself still needs human judgment.

Design Requires a Workflow

The repository includes a coding-agent skill, a tool catalog and separate design and delivery-review guides. These describe how to approach the task as a piece of communication, beyond the mechanics of calling drawing tools.

The starting point is the brief: who will read this, what should they understand, what format will they encounter, and which content must be preserved? An attended scientific poster can support a conversation with its author. A standalone explanatory poster has to supply more of that context itself. A folded leaflet introduces a physical reading sequence that cannot be inferred from its flat layout alone.

Typography, colour, and composition follow from those requirements. The guidance encourages deliberate text roles, a visible reading route, and meaningful use of colour. It also makes an important point about crowded pages: shrinking every label is not a satisfactory default response to running out of room. The structure or amount of content may need reconsideration.

For scientific work, the relationship between claims and evidence is part of the design problem. Units, legends, uncertainty, and source information have to survive the layout process when they are needed to interpret a figure. A conceptual illustration must remain distinguishable from a measurement. Attractive composition cannot compensate for a misleading scientific statement.

The editable document supports this discipline. Named layers and stable object IDs let the agent return to the exact caption, figure, or annotation that requires correction. Rendering then checks whether the change achieved its intended effect.

The review guidance also distinguishes a concept, a reviewable document, a production candidate, and an externally approved result. These are useful descriptions of what has actually been checked. A PDF export can be reviewed on screen while printer-specific requirements and a physical proof remain open.

Review Records and Validation

A useful part of the Codex workflow is making the review explicit and keeping its results alongside the drawing. The source SVG and exported files are accompanied by a record of the requirements, checks, observations, and remaining work. This is a workflow guided by the project’s skill and review instructions; the MCP server supplies drawing and inspection tools, while the agent assembles and interprets the evidence. There is no single tool call that certifies a design as correct.

The process starts by separating confirmed constraints from assumptions. Page dimensions, required wording, supplied figures, and requested output formats can become concrete checks. Missing information, such as a printer’s colour profile or a fold specification, remains an explicit assumption or open requirement. That distinction prevents the agent from quietly treating an unspecified production detail as something it has already satisfied.

The review then examines several different aspects of the document:

• Content and structure: Are the required text and figures present? Do labels retain the correct units and symbols? Are important elements still editable, and have temporary construction objects been removed?
• Geometry and layout: Are the page dimensions correct? Do object bounds reveal content crossing a page or column boundary? Are alignment, spacing, and grouping consistent with the intended composition?
• Rendered appearance: Is the main message apparent in an overview? Are captions and figure labels readable in detail? Are there collisions, clipping, awkward line breaks, or competing visual elements that the structural checks missed?
• Export integrity: Does the delivered file have the expected page count and dimensions? Where relevant tools are available, are PDF fonts embedded and required text retained? Does a separate render of the export agree with the intended composition?
• External requirements: Which checks still need a domain reviewer, a printer specification, an accessibility assessment, or a physical proof?

These checks use different kinds of evidence. Object queries and saved bounds tables support geometric inspection. Extracted text helps identify missing content or unexpected substitutions, but cannot establish that the text is legible on the page. A rendered overview supports a judgment about hierarchy, while a detail render supports a judgment about a particular label. PDF metadata and font reports address properties of the exported file. Some of these checks require separate analysis utilities available to Codex in addition to the MCP tools.

The record gives each check a status such as pass, fail, not_run, or not_applicable, together with a reason or reference to the evidence. A compact JSON file or a short Markdown record is sufficient; the exact filename is less important than preserving what was checked and against which artifact. Retaining the document revision also helps relate a review to the state it actually describes. Tool receipts can record which operations were accepted, but an accepted operation alone says nothing about whether the resulting layout communicates well.

For example, a general review might record that the required content is present and the page dimensions are correct, but that a figure’s labels are unreadable in the detail render. The next action is then specific: enlarge or rearrange those labels, render again, and inspect the affected figure and surrounding layout. If the change affects the exported result, the export and its checks need updating as well. A favorable review of an earlier revision cannot simply be carried forward after a consequential edit.

This makes the record useful during the work, rather than only as documentation at the end. It gives Codex a concrete set of unresolved issues to return to and helps a human reviewer understand why another correction was made. A failed check stays visible until the problem is corrected and checked again. An unperformed check remains not_run; it does not become a pass because the rest of the page looks convincing.

The resulting delivery stage follows from that evidence. A concept explores direction and placement. A reviewable document has been rendered and checked against the known content and format constraints. A production candidate also incorporates the supplied production requirements and has had its requested export inspected. Approved is reserved for the required external approval or physical proof actually being recorded.

The record ties each assessment to observations. Numerical validation can establish a page size precisely. Visual review can identify a crowded diagram, but remains a judgment by the reviewing model or person. Neither proves that a scientific conclusion is sound or that a physical print will reproduce as intended. Keeping the individual findings and their limits visible makes it much easier to decide what is ready to use and what still needs attention.

Results: A0 Science Posters

The two posters shown below make the workflow concrete. They describe mcpInkscape itself, so they also demonstrate the kind of technical communication the project is intended to support. The recorded run below shows the creation of the first displayed poster; the second explores an alternative design.

The first displayed poster is portrait A0, 841 by 1189 millimetres. Its saved SVG contains 196 drawable elements: 126 editable text objects and 70 vector shapes, excluding the three layer groups and document metadata. There are no raster images embedded in the document.

That distinction matters for future revision. The poster’s labels and explanations remain text, and its diagrams remain objects that can be selected and changed. The delivery record also includes checks of the A0 PDF page size, embedded font subsets, object bounds, extracted text, and separate live and PDF renders.

The example shown in the video uses named Structure, Figures, and Text layers. Its validation record lists 126 text objects, no embedded raster images, and a one-page A0 PDF with all five font subsets embedded. It records review of the live overview, a feedback detail, and a separately rasterized PDF. Its recorded delivery stage is reviewable; physical print proof and printer-specific preflight were not performed.

One small artifact from that recorded example is particularly illustrative: a feedback correction selects a specific text object, moves it, and requests another page render. That is exactly the kind of focused iteration this interface is meant to support. A finished-looking page can still be adjusted at the level of an individual object, followed by another visual check.

There is also a practical boundary worth preserving in the description of these runs. The native tool surface used for the posters did not expose document opening and saving. GUI actions handled those steps; typed native MCP operations created the artwork and text, and MCP handled the subsequent exports. The examples demonstrate a working composition and review process with that division of responsibilities.

A complete poster exercises exact wording, typography, diagrams, physical page dimensions, visual hierarchy, document organization, and export together. The saved artifacts and review records show how those concerns can be handled within one iterative workflow.

The following video shows the full process of producing the first poster shown below in Inkscape. It gives a more direct impression of the agent working on the document than the finished poster alone can convey.

The A0 poster created in the recorded live MCP session. The original SVG retains editable text and named layers.

The second example is exploring a pastel palette and a different typographic treatment of the same subject. This is another useful direction for the workflow: keeping the technical message while revisiting how the page introduces and organizes it. The design uses the headline “Ideas become editable”., with vector artwork illustrating an editable curve.

Its saved SVG contains 262 drawable elements: 126 editable text objects and 136 vector shapes, again excluding the three layer groups and document metadata. It also contains no embedded raster images.

Current Limitations

The project ist still work in progress (at the time of writing at an RC1 implementation). The offline and CLI compatibility testing has happened on FreeBSD with Inkscape 1.4.3, and native live testing was performed against a matching Inkscape 1.4.4 fixture.

The native bridge is a separately built C++ component that must match the Inkscape build and its ABI. Installing the Python package alone does not install that extension. Offline operation remains available without it, and capability discovery tells the agent which live operations are actually supported.

The active-window fallback also has narrower guarantees. It targets the focused window, depends on the installed action set, and cannot provide the native bridges revision-conflict protection. Those differences need to remain visible to the agent using the tools.

Finally, review has limits. The poster records provide evidence about editable structure, dimensions, fonts, bounds, and rendered output. They do not establish reader comprehension or physical print quality. A scientific figure still needs its scientific content checked, and a production job still needs the requirements of its actual destination applied.

What comes next

The direction I find most interesting is connecting this drawing workflow to the rest of a research or engineering task. An agent could gather information from laboratory books and publications, prepare a plot using an analysis tool, assemble it with a schematic and explanatory text in Inkscape, review the combined figure, and then revise the composition when the data or explanation changes.

The boundaries between these steps can remain clear. The analysis program produces the scientific result. Inkscape supplies the editable composition. The agent coordinates the work and checks the representations available to it. That makes it easier to preserve the origin of a figure while still automating much of the repetitive layout work around it.

There is also room to improve the shared editing experience: noticing human changes promptly, preserving deliberate manual adjustments, and making long sequences of agent edits easy to inspect and undo. Revision checks and polling provide the beginnings of that interaction, but good collaboration also depends on how the agent uses them.

What I like about this approach is that it leaves behind something useful beyond the immediate session. The SVG remains an editable document. The labels can be corrected, the layout can be reused, and the figures can be adapted for another purpose. Human and automated work can continue from the same artifact.

The FreeCAD project explored that idea for mechanical objects. mcpInkscape brings it into visual communication. In both cases, the interesting capability comes from combining a structured application with a model that can inspect, act, review, and revise. I think there is a great deal more to explore there.

References

• The mcpInkscape project on GitHub, including its README, user manual, and native bridge
  • Project workflow documentation: skill/mcpinkscape/SKILL.md, design-for-inkscape.md, and design-review.md in the repository
• Example posters:
  • A0 poster from the recorded session
  • Alternative pastel A0 poster
• Previous blog articles:
  • Artificial Intelligence doing CAD: ChatGPT, Codex and FreeCAD
  • The Hum That Never Sleeps
• Commercial Agent Orchestrators
  • OpenAIs codex
  • Claude Code

This article is tagged:

• Artificial Intelligence
• Large Language Models
• LLM
• Automation
• Programming
• Projects
• Science