[1] https://pmc.ncbi.nlm.nih.gov/articles/PMC6306777/

[1] https://folklore.org/Negative_2000_Lines_Of_Code.html

To figure that out, I remember searching for articles on how to implement inverted indices. Once I had a list of candidate strategies and data structures, I used Wikipedia supplemented by some textbooks like Skiena's [2] and occasionally some (somewhat outdated) information from NIST [3]. I found Wikipedia quite detailed for all of the data structures for this problem, so it was pretty easy to compare the tradeoffs between different design choices here. I originally wanted to implement the inverted index as a hash table but decided to use a trie because it makes wildcard search easier to implement.

After I developed most of the backend, I looked for books on "information retrieval" in general. I found a history book (Bourne and Hahn 2003) on the development of these kind of search systems [4]. I read some portions of this book, and that helped confirm many of the design choices that I made. I actually was just doing what people traditionally did when they first built these systems in the 1960s and 1970s, albeit with more modern tools and much more information on hand.

The harder part of this project for me was writing the interpreter. I actually found YouTube videos on how to write recursive descent parsers to be the most helpful there, particular this one [5]. Textbooks were too theoretical and not concrete enough, though Crafting Interpreters was sometimes helpful [6].

[1] https://en.wikipedia.org/wiki/Inverted_index

[2] https://doi.org/10.1007/978-3-030-54256-6

[3] https://xlinux.nist.gov/dads/

[4] https://doi.org/10.7551/mitpress/3543.001.0001

[5] https://www.youtube.com/watch?v=SToUyjAsaFk

[6] https://craftinginterpreters.com/

There are "smarter" solutions like radix tries, hash tables, or even skip lists, but for any design choice, you also have to examine the tradeoffs. A goal of my project is to make the code simpler to understand and less of a black box, so a simpler data structure made sense, especially since other design choices would not have been all that much faster or use that much less memory for this application.

I guess the moral of the story is to just examine all your options during the design stage. Machine learning solutions are just that, another tool in the toolbox. If another simpler and often cheaper solution gets the job done without all of that fuss, you should consider using it, especially if it ends up being more reliable.

[1] https://github.com/atrettel/wosp

Remote: Open to remote, hybrid, and in person

Willing to relocate: Yes

Technologies: C, C++, Fortran, Java, Message Passing Interface (MPI), OpenMP, Python (Matplotlib, Numpy, Pandas, Scipy), SQL (especially SQLite)

Résumé/CV: Available upon request

Email: hnjobs-gndxgr@atrettel.net

GitHub: https://github.com/atrettel

Website: https://www.andrewtrettel.com/

Hi, I'm Andrew Trettel. I'm a scientist with a PhD in mechanical engineering looking for any potential opportunities outside of academia and research labs. I am open to many different roles, including being a software developer or data scientist. I have over a decade of experience in scientific research, especially on the numerical side. I have years of experience in writing software for and running large simulations on high-performance computing (HPC) systems. I have also developed software for non-scientific purposes, like creating user interfaces for desktop applications and writing command-line tools. I have years of experience working with large datasets, including the tasks of calculating statistics and developing hypotheses/models/theories from data. I've worn many hats over the years and love learning new and interesting topics.

I don't view that as a failure of abstraction as a design principle as much as it is a pitfall of using the wrong abstraction. Using the right abstraction requires on the ground knowledge, and if nobody communicates that up the chain, well, you get the tree swing cartoon.

[1] https://en.wikipedia.org/wiki/Tree_swing_cartoon

In the United States, the only patentable subject matter is processes, machines, manufactures, or compositions of matter [1]. Anything outside of those areas is not directly patentable. Some subject matter like mathematics and "mental processes" generally are categorized as "abstract ideas" that therefore are not directly patentable [2]. It is possible to patent something that contains an abstract idea, but it also has to have some "additional elements" that elevate it beyond merely claiming an abstract idea.

I suggest reading MPEP § 2106 [2] and looking at the first diagram given there titled "Subject Matter Eligibility Test for Products and Processes". That is the exact analysis that a patent examiner would use to determine if something is patentable subject matter or not (including for any claim with a prompt).

I strongly suggest that you talk to a lawyer if you want specific advice that answers your question directly. I'm not commenting on any copyright aspects.

[1] https://www.uspto.gov/web/offices/pac/mpep/s2104.html

[2] https://www.uspto.gov/web/offices/pac/mpep/s2106.html

It is also a bit funny that the author complains about older Fortran programs requiring SCREAMING_CASE, when if anything this is an improvement over previous and current practices. Too many Fortran codes have overly terse variable names that often were just single characters or impenetrable abbreviations for obscure terms. I have had to create cheat sheets for each program to figure out what each variable was.

Sun Microsystems had a great quote about this back in the day [1]:

> Consistently separating words by spaces became a general custom about the tenth century A.D., and lasted until about 1957, when FORTRAN 77 abandoned the practice.

[1] https://docs.oracle.com/cd/E19957-01/802-2998/802-2998.pdf

That said, multiple discovery seems to be more common nowadays due to the rapid diffusion of information, which means that most people are operating in roughly the same information environment (initial conditions) when they start their research. It is interesting how often multiple discovery happens when you start to look closely at this.

[1] https://en.wikipedia.org/wiki/Multiple_discovery

I called it Wosp for word-oriented search and print. See the GitHub page for more information: https://github.com/atrettel/wosp

I'm reminded of an old quote by Dijkstra about Fortran [1]: "In the good old days physicists repeated each other's experiments, just to be sure. Today they stick to FORTRAN, so that they can share each other's programs, bugs included."

I've encountered that same problem in some older scientific codes (both C and Fortran). After a while, the bugs somewhat become features because people just don't know to question them anymore. To me, this is why it is important to understand the code thoroughly enough to question what is going on (regardless of who or what wrote it).

[1] https://www.cs.utexas.edu/~EWD/transcriptions/EWD04xx/EWD498...

Remote: Open to remote, hybrid, and in person

Willing to relocate: Yes

Technologies: C, C++, Fortran, Java, Message Passing Interface (MPI), OpenMP, Python (Matplotlib, Numpy, Pandas, Scipy), SQL (especially SQLite)

Résumé/CV: Available upon request

Email: hnjobs-gndxgr@atrettel.net

GitHub: https://github.com/atrettel

Website: https://www.andrewtrettel.com/

Hi, I'm Andrew Trettel. I'm a scientist with a PhD in mechanical engineering looking for any potential opportunities outside of academia and research labs. I am open to many different roles, including being a software developer or data scientist. I have over a decade of experience in scientific research, especially on the numerical side. I have years of experience in writing software for and running large simulations on high-performance computing (HPC) systems. I have also developed software for non-scientific purposes, like creating user interfaces for desktop applications and writing command-line tools. I have years of experience working with large datasets, including the tasks of calculating statistics and developing hypotheses/models/theories from data. I've worn many hats over the years and love learning new and interesting topics.