Location: Europe, Budapest
  Remote: Yes 
  Willing to relocate: No
  Technologies: Go, Python, svelte, search engines, scrapers, crawlers, data gathering, privacy, free software
  Résumé/CV: https://github.com/asciimoo
  Email: asciimoo+hnjob@gmail.com

~20 years of development experience. Author of many somewhat successful free software projects like Searx, and Go-Colly.

I agree, browser import has rough edges. The issue you mentioned is known: https://github.com/asciimoo/hister/issues/31 . I try to prioritize it and find time to fix it.

I'd appreciate any kind of help designing such system. We are on IRC/Discord/Github/Codeberg.

Yes, Hister has a built-in crawler which supports standard HTTP and different browser based backends

> Can I make some pages to rank higher in it?

You can create priority rules to boost the ranking of the matching domains/URLs

> Can I assign tags to pages (by which I could later on filter?)

It is possible to add a label to indexed documents

I'm working on a self-hosted search service called Hister with the same goal when I started Searx development: reduce dependence on online search engines.

Hister is a full text indexer for websites and local files which automatically saves all the visited pages rendered by your browser. It provides a flexible web (and terminal) search interface & query language to explore saved content with ease or quickly fall back to traditional search engines. This is a fundamentally different approach than what Searx follows and solves most of the weaknesses of metasearch engines. Of course it has its own weaknesses as well, but most of these are not conceptual and can be resolved by improving the software (and datasets)

I've been using it for a few months and as my local index is growing I can avoid relying on external search engines - and even websites listed in results - more and more frequently.

The initial reception is overwhelmingly positive with already more than 30 contributors and hundreds of contributions. Currently it can help with "recall" type searches mainly, but I'm planning to provide pre-indexed thematic datasets and I'm drafting a peer-to-peer index sharing concept. Maybe you can find it useful as well (or at least have some constructive criticism =]).

Links: - https://hister.org/ - https://github.com/asciimoo/hister - Background/motivation/beginnings: https://hister.org/posts/how-i-cut-my-google-search-dependen... - Small read-only demo: https://demo.hister.org/

Nobody said that the problem is not knowing rust. The problem is changing the whole stack of a project overnight. This requires significant effort to get familiar with, even if a contributor have all the experience in the world with the new stack.

> Don't mix concerns here, you're talking about "open development"

Call it however you want, bun could not be the tool it is without its >800 contributors.

Even if you are fluent in rust, it is going to require significant efforts to contribute to a new 1M LoC codebase.

> Open source literally isn't about you, let people run their projects as they so wish, them making choices they think are better

This is so far from the reality. The power of open source is coming from the contributors. Contributors are the most valuable assets of an open source project - without them most of the free tools you use would be significantly worse - including bun. The reason my open source projects got somewhat successful is the community that formed around the projects. And, it is hard to create a community when you give contributors no chance to participate in the projects direction, especially in such a critical decision that has enormous consequences.

No, the big difference is that the described scenario does not require getting familiar with a new 1M LoC codebase written in a different language to be able to continue contributing to the project.

Hister is a free general purpose web search engine providing automatic full-text indexing for visited websites.

Currently it overwrites the previous content with the latest if there is any change, but I'd like to add option to store diffs as well in the future.

Hister is basically a full text indexer which saves all the visited pages rendered by your browser. It provides a flexible web (and terminal) search interface & query language to explore previously visited content with ease or quickly fall back to traditional search engines.

Here's a little summary of the background/motivation/beginnings: https://hister.org/posts/how-i-cut-my-google-search-dependen...

Project site: https://github.com/asciimoo/hister

Website: https://hister.org/ Read-only demo: https://demo.hister.org/