Comments URL: https://news.ycombinator.com/item?id=47561123

Points: 2

# Comments: 0

Comments URL: https://news.ycombinator.com/item?id=46986701

Points: 1

# Comments: 0

Kreuzberg is an MIT licensed Python library that extracts text from a wide range of documents (PDFs, images, office files etc.) without depending on external APIs dependencies.

Its different from other libraries and commercial offerings in this space by being designed to be (1) lightweight, (2) CPU orientated, (3) simple to user and (4) have async support as a first class citizen.

The v3.0 release completely reworks the architecture for extensibility. Kreuzberg now now supports:

- Multiple OCR backends (Tesseract, PaddleOCR, EasyOCR), with OCR itself being completely optional. - Support custom extractors and overriding of builtin extractors. - Post-processing and validation hooks. - Extensive PDF metadata extraction. - Optional support for semantic chunking.

There is also a brand new documentation site at https://goldziher.github.io/kreuzberg.

I also published a roadmap for the project, which you can see here: https://github.com/Goldziher/kreuzberg/discussions/24

You can see the repo at https://github.com/Goldziher/kreuzberg - please star it if you find it valuable, since this motivates me!

Comments URL: https://news.ycombinator.com/item?id=43459261

Points: 5

# Comments: 0

The core system is fairly generic and could work for many use cases with minimal changes. Before investing time in packaging it as a library, I'm curious:

- Would the HN community find value in a pgvector-based RAG library? - What features would be most important to you? - What belongs in open source vs. commercial offerings? - What common pitfalls should be avoided?

I'd like to gauge if there's actual interest before publishing something nobody will use. So your Feedbacks are most welcome!

Comments URL: https://news.ycombinator.com/item?id=43370887

Points: 3

# Comments: 2

You should try using a different PSM to see if you get better results.

If it's scientific texts specifically, look at grobid

There are alternative options to tesseract ofc.

It's both. The OCR part is ofc CPU bound, but the entire text extraction involves reading files, or writing and then reading files.

Without async, these simply block.

As for efficiency - if you're working in an async application context you have to "asyncify" these operations or suffer the consequences.

My reasons for using Tesseract - easy OCR is larger, and it has a significant cold start.

It benchmarks better for many OCR tasks though, so I'm thinking of adding it as an alternative backend.

But seriously, in 13 years living here, only one guy tried to pick pocket me.

In my tests I found tesseract quite good for regular text documents. For other kinds of texts it's not great.

As for using models - there are some good small language models as well, and of course LLMs.

I sorta feel though that if one needs complex OCR, or a vision model for layout, one should opt for either a commercial solution that abstracts the deployment and GPU management, or bake ones own system.

For most use cases involving text documents though, my subjective opinion is that tesseract is sufficient.