2. We see exactly the same thing. There is a trade-off in correctness vs token burning. However, some tokens (models) are cheaper and faster than others, so the small pieces can benefit from that. The token usage is also surprisingly variable, because it depends on the information density of the document and also on the information density of the question (e.g. is it a single needle in a haystack or are we analyzing the entire haystack from 10 perspectives). So the parsing for 1k pages may be on the order of millions of tokens, while a series of queries (extractions) on top of it could be 1-2 orders of magnitude more.

On the scale aspect, because we go in multiple passes, we break the scope into small enough pieces and then build it back up in a later step. Iirc the largest document I've seen a customer use was over 1k pages.

There are more complex data dependency scenarios where we find that the data that's extracted and combined (e.g. from page 4 and 40), needs to then be further transformed in different ways (e.g. having an evaluation and a clarification outcome at the end). To make these be aligned in value we are soon releasing a feature for what we call derived agents.

We actually don't use embeddings or vector similarity, since those tend not to work well in specialist domains (e.g. for the OfficeQA benchmark where we have 90k pages talking about US treasury numbers, they would be mostly mapped to a very small embedding space because it's all the same topic, with small variations across years, expense categories etc.).

We use LLMs for the extraction and comparison as well, and we route between different models depending on the complexity of the comprehension of the given step required (and by this I mean routing between our pipeline steps; we currently do not dynamically try to judge individual cases for complexity like OpenRouter Fusion).

We had the same observation in that the possible space is almost endless, and for example even for the same file type there may be different kind of processing required (e.g. an excel can be database style, vs small narrative heavy, or both).

We have baked in some ground processing rules for different kinds of documents, and we do allow custom instructions on how to deal with specific cases (e.g. translations, particular format layouts). The best write-up I have at the moment is https://www.parsewise.ai/doc-processing-pipelines but we're working on something that goes into more detail:)

And thank you, we also love the traceability, it's one of the aspects that we have prioritized. Models will never be perfect so rather than building the best model harness we went for the best human harness haha.

Tbh it's been a while since I've looked at notebooklm so I expect it would have gotten better over time. One thing where I found it lacking in the past was the structure we could get out (which gives the traceability) - for example a deep dive on one the underlying data for this corpus: https://demo.parsewise.ai/projects/39bee9d8-d722-4b23-8894-e...

And yes, we're really excited whenever new open weights models come out that push quality, price, latency. We're finding that throughput is a big obstacle so I'm looking forward to more of this running locally, but it will be a while..

Fully agree, for enterprises we need to guarantee types, flag discrepancies and provide underlying sources so they can integrate it downstream (whether that's Databricks, n8n etc.)

Here is our documentation for working with a fixed JSON schema: https://docs.parsewise.ai/api#schema-driven-extract-convenie...

At first, fairly generic templated definitions sort of work, but what we've seen is that over time data comes up that is out of distribution, and there was no explicit instruction on how to deal with it. In such cases we tend to flag this and offer suggestions to the users on how they can improve the specificity of agents.

Another structure we have seen play out is having a manager review ratings and feedback comments from their team and updating the definitions accordingly over time (where we offer them the capability to see results of before and after side -by-side for all existing data as well, so they are more confident in the change before committing).

The amount of work is dependent on how good the initial definitions are and how complex the use case is (and how much it evolves - new data sources etc). A bit of an unsatisfying answer but it can be anywhere between a few hours one off or a couple of minutes per day on an ongoing basis.

1. We are working with the assumption that OCR is (or soon will be) solved at super low prices.

So if we have the extracted data, what can we do with it? Where we see Parsewise making a difference is for use cases that span across documents. I.e. if you are extracting the same 5 fields from every invoice, there are lots of solutions as you listed (+ reducto etc). However, once you have a set of documents (e.g. an entire mortgage application package) and you are trying to get a structured response out, then your option is either an LLM API (if things fit into context and you are okay with limited citations), or building a pipeline with LLMs. I posted it in another comment but an example of trawling through 90k pages is here: https://www.parsewise.ai/officeqa-sota

2. While we rely on LLMs, the outcomes will be non-deterministic, so the bottleneck is and will remain the human verification (that is for somewhat complex use cases). The architecture that we have built is optimizing for the human reviewer to provide as granular values and citations as possible. This is either through our platform, or API clients.

And I do get genuine joy from helping our users, so love it is:)