We came to work on this problem after spending a year building an AI document editor for pharma companies. Before that, Topaz(myself) was a senior SWE at Snyk, working on distributed systems, and Dudu was a deep learning engineer at Viz.ai, building computer vision models for stroke detection. Our editor helped pharma companies generate regulatory documents (e.g. CSRs) to speed up their submissions. Initially, the output was Markdown, displayed in a WYSIWYG editor. However, users preferred working with their own Word templates. That's when the problems began.

AI agents aren't great at editing Word documents. A Word document is a zip file of verbose XML files following the OOXML spec. Even "small" changes require backflips, for example: adding a numbered list requires creating an entry in numbering.xml with a fresh ID and linking it back in document.xml, bolding a sentence requires splitting it into 3+ run elements. The list goes on.

This makes editing the zip directly (unzip + grep + sed) a bad idea for agents because they burn a lot of time + tokens on these mechanics. In practice, today's tooling falls into roughly three categories. You can let the agent write code against low-level libraries like python-docx or the Open XML SDK, you can give it an MCP with opinionated editing tools (SuperDoc, Office CLI, Adeu, etc), or you can round-trip the file through Markdown/HTML with something like pandoc/mammoth.js. None of them really work. The first two categories still burn the agent's context on Word mechanics instead of the task at hand (MCPs also introduce a new DSL to learn), and the third is very lossy (pandoc/mammoth.js/etc don't preserve enough fidelity).

From firsthand experience, these problems hurt performance in downstream tasks.

When we tried having our agent fill large documents, things broke quickly. The context window was already packed with customer data (files, user context, global rules, etc.), and the agent burned tokens + time on exploring the document and debugging failed edits. Filling a single CSR (clinical study report) took ~50 minutes, and the result was bad (missed fields/sections, broken styling, etc.). Harvey.ai's team reached similar conclusions: https://www.harvey.ai/blog/building-an-agent-for-complex-doc...

That's when we shifted our focus. We designed an MCP that lets agents edit Word docs as if they were editing HTML. The agent receives HTML, makes find-and-replace edits, and we reconcile those edits back into the original .docx file. We picked HTML over Markdown because it's structurally much closer to OOXML and because CSS associates styles with elements roughly the way OOXML does. We had to write our own DOCX→HTML converter, since pandoc and mammoth.js didn't preserve enough fidelity. To be clear, our DOCX → HTML conversion is lossy too. That's fine though, because we never convert the HTML back to DOCX. The HTML is just a projection for the agent, so it only needs enough fidelity for the agent to understand the structure and styling of what it's editing. The original file stays the source of truth, and we mutate it in place.

This also means the agent doesn't need to learn a new DSL. Editing a Word document feels just like editing an HTML file on the file system, something agents are already great at. A lot of DOCX MCPs hand agents dozens or even hundreds of tools to figure out on the fly. Our MCP exposes just three tools (read, search, edit). The Word document is completely abstracted.

After an agent sends us an edit request (an "old_html" and "new_html" pair), we reconcile it to the original .docx file. The reconciliation is powered by our fine-tuned model, a 3-8B base with a LoRA adapter. It takes the HTML diff as input along with the original localized OOXML block and emits the new OOXML. The "localization" is done deterministically: we take the anchors the agent specifies and we try to find their XML twins, so the reconciler model has a single responsibility. On this narrow task a small model reaches the level of a frontier, well prompt-tuned model, while being small and fast enough to sit in the hot path.

This project turned into months of work, but we're happy to release our v1. Our internal benchmark shows it's more accurate than the DOCX skill and raw python-docx while being ~2x cheaper and ~3x faster, mostly because it takes 3 tool calls (p50) per task whereas the DOCX skill takes 10 and Office CLI takes 13.

Things aren't perfect yet. For example, we don't support manipulating images or comments at the moment. That said, we're already seeing people use our MCP in various ways:

- Legal tech companies powering their live-editing flow in Office.js.

- AI startup optimizing people’s resumes and applying on their behalf.

- Govtech who need to draft policy memos.

- A life sciences startup using long-running agents to complete regulatory forms.

A note on privacy: our MCP runs in the cloud, so users send us their .docx files. We don't train on user data, and teams can opt for ZDR or self-hosting.

There's a free tier with 500 edits a month, and we’d love you all to try. We want to bump that later, but we're a small team and running a fine-tuned model isn't cheap :(

We'd love to hear your ideas and comments about docx editing in general! We'll be in the comments for the next few hours to respond

Comments URL: https://news.ycombinator.com/item?id=49881505

Points: 38

# Comments: 17

Some feedback:

- while I am going through the chapters i keep trying to highlight certain sentences so i could bookmark them for later, itd also be great to do cmd+f to search keywords

- in chapter 1, the "under the hood" link crashes the app

i'm also building in this space (an MCP for agents to manipulate docx)

the other day I also vibe-coded a recreation of digglabs , its shows HN, reddit ect using different visualisations

https://topaztee.com/digglabs

I'm also working on letting agents read/edit word docs but exposing it as a simple MCP

www.vespper.com

Comments URL: https://news.ycombinator.com/item?id=48821114

Points: 3

# Comments: 0

im working on something similar. A fine-tuned model for agents to interact with docx over MCP. they wont have to deal with OOXML.

we have a waiting list for beta-users: www.vespper.com

im working on something similar. A fine-tuned model for agents to interact with docx over MCP. they wont have to deal with OOXML vespper.com

(source - I'm working on a docx MCP that abstracts OOXML complexity away from LLMs entirely)

vespper.com

- https://www.yourfriendly.ai/. a desktop pet that lets me have an ai chatbox readily available