I'm the maintainer of Kreuzberg, an open-source document intelligence library (https://github.com/kreuzberg-dev/kreuzberg). Some of you may have used it for RAG ingestion.

We're launching Kreuzberg Cloud, a SAAS API and a self-hosted system. It's in public beta, and I would like to invite you all to give it a try.

What out MVP offers: we offer very fast CPU optimized document and code intelligence. You can extract content from more than 90 document file formats and 300 code file formats into Markdown (or plaintext/djot), with additional features (same pricing tier) including chunking, embeddings, keyword extraction - and various types of intelligence.

The OSS library is used as the base engine of the cloud system. Our initial offering is $0.008/page, and you get the first 10K pages free, no card required.

We also offer our entire system for self-hosting - using helm charts. We are looking for design partners, so if thats relevant - shoot me a line.

Comments URL: https://news.ycombinator.com/item?id=48205129

Points: 5

# Comments: 4

Comments URL: https://news.ycombinator.com/item?id=47561123

Points: 2

# Comments: 0

Comments URL: https://news.ycombinator.com/item?id=46986701

Points: 1

# Comments: 0

Kreuzberg is an MIT licensed Python library that extracts text from a wide range of documents (PDFs, images, office files etc.) without depending on external APIs dependencies.

Its different from other libraries and commercial offerings in this space by being designed to be (1) lightweight, (2) CPU orientated, (3) simple to user and (4) have async support as a first class citizen.

The v3.0 release completely reworks the architecture for extensibility. Kreuzberg now now supports:

- Multiple OCR backends (Tesseract, PaddleOCR, EasyOCR), with OCR itself being completely optional. - Support custom extractors and overriding of builtin extractors. - Post-processing and validation hooks. - Extensive PDF metadata extraction. - Optional support for semantic chunking.

There is also a brand new documentation site at https://goldziher.github.io/kreuzberg.

I also published a roadmap for the project, which you can see here: https://github.com/Goldziher/kreuzberg/discussions/24

You can see the repo at https://github.com/Goldziher/kreuzberg - please star it if you find it valuable, since this motivates me!

Comments URL: https://news.ycombinator.com/item?id=43459261

Points: 5

# Comments: 0

The core system is fairly generic and could work for many use cases with minimal changes. Before investing time in packaging it as a library, I'm curious:

- Would the HN community find value in a pgvector-based RAG library? - What features would be most important to you? - What belongs in open source vs. commercial offerings? - What common pitfalls should be avoided?

I'd like to gauge if there's actual interest before publishing something nobody will use. So your Feedbacks are most welcome!

Comments URL: https://news.ycombinator.com/item?id=43370887

Points: 3

# Comments: 2

You should try using a different PSM to see if you get better results.

If it's scientific texts specifically, look at grobid

There are alternative options to tesseract ofc.

It's both. The OCR part is ofc CPU bound, but the entire text extraction involves reading files, or writing and then reading files.

Without async, these simply block.

As for efficiency - if you're working in an async application context you have to "asyncify" these operations or suffer the consequences.

My reasons for using Tesseract - easy OCR is larger, and it has a significant cold start.

It benchmarks better for many OCR tasks though, so I'm thinking of adding it as an alternative backend.