In general, the goal is to use an org-wide installation method wherever possible, and record the identify of the user we are impersonating when ingesting data in the ACL. There are some gaps in the permission-gathering step in some of the connectors, I'm still working on fixing those.

Typical RAG implementations I’ve seen take the user query and directly run it against the full-text search and embedding indexes. This produces sub-par results because the query embedding doesn’t really capture fully what the user is really looking for.

A better solution is to send the user query to the LLM, and let it construct and run queries against the index via tool calling. Nothing too ground-breaking tbh, pretty much every AI search agent does this now. But it produces much better results.

I haven't directly compared against Elasticsearch yet, but I plan to do that next and publish some numbers. There's a benchmark harness setup already: https://github.com/getomnico/omni/tree/master/benchmarks, but there's a couple issues with it right now that I need to address first before I do a large scale run (the ParadeDB index settings need some tuning).

Currently permissions are handled in the app layer - it's simply a WHERE clause filter that restricts access to only those records that the user has read permissions for in the source. But I plan to upgrade this to use RLS in Postgres eventually.

For Slack specifically, right now the connector only indexes public channels. For private channels, I'm still working on full permission inheritance - capturing all channel members, and giving them read permissions to messages indexed from that channel. It's a bit challenging because channel members can change over time, and you'll have to keep permissions updated in real-time.

Over the past few months, I've been working on building Omni - a workplace search and chat platform that connects to apps like Google Drive/Gmail, Slack, Confluence, etc. Essentially an open-source alternative to Glean, fully self-hosted.

I noticed that some orgs find Glean to be expensive and not very extensible. I wanted to build something that small to mid-size teams could run themselves, so I decided to build it all on Postgres (ParadeDB to be precise) and pgvector. No Elasticsearch, or dedicated vector databases. I figured Postgres is more than capable of handling the level of scale required.

To bring up Omni on your own infra, all it takes is a single `docker compose up`, and some basic configuration to connect your apps and LLMs.

What it does:

- Syncs data from all connected apps and builds a BM25 index (ParadeDB) and HNSW vector index (pgvector)

- Hybrid search combines results from both

- Chat UI where the LLM has tools to search the index - not just basic RAG

- Traditional search UI

- Users bring their own LLM provider (OpenAI/Anthropic/Gemini)

- Connectors for Google Workspace, Slack, Confluence, Jira, HubSpot, and more

- Connector SDK to build your own custom connectors

Omni is in beta right now, and I'd love your feedback, especially on the following:

- Has anyone tried self-hosting workplace search and/or AI tools, and what was your experience like?

- Any concerns with the Postgres-only approach at larger scales?

Happy to answer any questions!

The code: https://github.com/getomnico/omni (Apache 2.0 licensed)

Comments URL: https://news.ycombinator.com/item?id=47215427

Points: 177

# Comments: 42

A platform for consulting aspirants to practice business case interviews.

Finding case prep partners is a major pain point for B-school students/consulting aspirants. Fortunately, frontier AI models are now good enough to function as surprisingly competent case interviewers.

Just wanted to share an interesting experiment I ran to see what kind of performance gains can be achieved by fine-tuning a model to code from a single repo.

Results: The fine-tuned model achieves a 47% improvement in the code completion task (tab autocomplete). Accuracy goes from 25% to 36% (exact match against ground truth) after a short training run of only 500 iterations on a single RTX 4090 GPU.

This is an interesting result because it shows that there are significant gains to be had by fine-tuning to your own code.

Full details of the experiment: https://prvn.sh/build-your-own-github-copilot/

Highlights of the experiment: - Model: qwen2.5-coder 14b, 4-bit quantized - Training data: Svelte source files from this repo: https://github.com/hcengineering/platform - Unsloth for LoRA training with rank 16, 4096 sequence length - GPU: single RTX 4090 - 500 iterations with effective batch size 8

A fine-tuned open-source model could be a real alternative to commercial coding assistants like GitHub Copilot, Cursor, etc., especially for organizations with a ton of legacy code that these LLMs have not seen, and also for those that are wary of exposing their code to external systems.

Comments URL: https://news.ycombinator.com/item?id=43390889

Points: 4

# Comments: 0

Comments URL: https://news.ycombinator.com/item?id=36571173

Points: 3

# Comments: 1