Anyone up for a rousing game of Pole Position?

I think by now a lot of people know you can write to Avro and compact to Parquet, and that is a key area of development. I'm not sure of a great solution yet.

Apache Iceberg tables can sit on top of Avro files as one of the storage engines/formats, in addition to Parquet or even the old ORC format.

Apache Hudi[2] was looking into HTAP capabilities - writing in row store, and compacting or merge on read into column store in the background so you can get the best of both worlds. I don't know where they've ended up.

[1] https://avro.apache.org/

[2] https://hudi.apache.org/

Also, it was paid for by US taxpayer dollars - the entire content should have been released somewhere for free, maybe even someone would have started up a new project to maintain it, for example, something under Wikimedia or some other nonprofit.

This wholesale elimination of valuable information and data owned by the public is so incredibly sad and damaging to our future.

Maybe we need a FOIA request to get the entire contents released to the public.

What is your opinion and did you vote this down because you think it's silly, dangerous or you don't agree?

I've worked on "legacy systems" written 30 to 45 years ago (or more) and still running today (things like green-screen apps written in Pick/Basic, Cobol, etc.). Some of them were written once and subsystems replaced, but some of it is original code.

In systems written in the last.. say, 10 to 20 years, I've seen them undergo drastic rates of change, sometimes full rewrites every few years. This seemed to go hand-in-hand with the rise of agile development (not condemning nor approving of it) - where rapid rates of change were expected.. and often the tech the system was written in was changing rapidly also.

In hardware engineering, I personally also saw a huge move to more frequent design and implementation refreshes to prevent obsolescence issues (some might say this is "planned obsolescence" but it also is done for valid reasons as well).

I think not reading the code anymore TODAY may be a bit premature, but I don't think it's impossible to consider that someday in the nearer than further future, we might be at a point where generative systems have more predictability and maybe even get certified for safety/etc. of the generated code.. leading to truly not reading the code.

I'm not sure it's a good future, or that it's tomorrow, but it might not be beyond the next 20 year timeframe either, it might be sooner.

Proton is another one people often suggest. Hey.com sometimes too. No experience with those myself.

There are other options (such as the big guys, iCloud mail or Outlook.com), but aside from self-hosting (which I don't want to spend time maintaining just for my personal mail), I personally haven't seen much outside of those ones that are recommended often.

Not selling anything, but I am trying to figure out what to do to help support solo and micro entrepreneurs, very small businesses (2-3 people) and very small nonprofits.

I feel like there are a lot more people in this position now (me included), but I don't want to do things for the sake of doing them... I want to find out what solo folks really benefit from and help make sure you get more support.

I agree that spending time on inference or compute every time for the same LLM task is wasteful and the results would be less desirable.

But I don't think the thought experiment should end with that. We can continue to engineer and problem solve the shortcomings of the approach, IMHO.

You provided a good example of an optimization - tool creation.

Trying to keep my mind maximially open - one could think of a "design time" performance at runtime - where the user interacting with the system is describing what they want the first time, and the system is assembling the tool (much like we do now with AI assisted coding, but perhaps without even seeing the code).

Once that piece of the system is working it is persisted so no more inference is required, as essentially code - a tool, that saves time. I am thinking of this as essentially memoizing a function body- i.e. generating and persisting the code.

There could even be some process overseeing the generated code/tool to make sure the quality meets some standard and providing automated iteration, testing, etc if needed.

A big problem is if the LLM never converges to the "right" solution on it's on (e.g. the right tool to generate the HTML from the SQL query, without any hallucination). But, I am willing to momentarily punt on that problem as being more to do with the determinism problem and the quality of the result. The issue isn't per se the non-deterministic results of an LLM anyway, it's the quality of the result fit for purpose for the use case.

I think it's difficult but possible to go further with the thought experiment. A system that "builds itself" at runtime, but persists what it builds, based on user interaction and prompting when the result is satisfactory...

I remember one of the first computer science things I learned- the program that could print out it's own source code. Even then we were believing that systems could build themselves and grow themselves.

So my ask would be to look beyond the initial challenge of the first time costs of generating the tool/code and solve that by persisting a suitable result.

What challenge or problem comes next in this idea?

My brain went to the concept of memoization that we use to speed up function calls for common cases.

If you had a proxy that sat in front of the LLM and cached deterministic responses for inputs, with some way to maybe even give feedback when a response is satisfactory.. this could be a building block for a runtime design mode or something like that.

I agree this way seems "wrong", but try putting on your engineering hat and ask what would you change to make it right?

I think that is a very interesting thread to tug on.

Of course the generated code might not work in all cases or scenarios, or may have to be generated multiple times, and yes it would be slower the first time.. but subsequent invocation would just be the code that was generated.

I'm trying to imagine what this looks like practically.. it's a system that writes itself as you use it? I feel like there is a thread to tug on there actually.

In fact, this thought has been percolating in the back of my mind but I don't know how to process it:

If LLMs were perfectly deterministic - e.g. for the same input we get the same output - and we actually started memoizing results for input sets by materializing them - what would that start to resemble?

I feel as though such a thing might start to resemble the source information the model was trained on. The fact that the model compresses all the possibilities into a limited space is exactly what makes it more valuable - instead of having to store every input, function body and outputs by memoizing that an LLM could generate, it just stores the model.

But this blows my mind somehow because if we DID store all the "working" pathways, what would that knowledgebase effectively represent and how would intellectual property work anymore in that case?

Thinking about functional programming, to me the potential to think of the LLM as the "anything" function, where a deterministic seed and input always produces the same output, with a knowledgebase of pregenererated outputs to use to speed up the retrieval of acceptable results for a given seed and set of inputs.... I can't put my finger on it.. is it a basically just a search engine then?

Let me try another way...

If I have a ask an LLM to generate a function for "what color is the fruit @fruit?", where fruit is the variable, and I memoize that @fruit = banana + seed 3 is "yellow", then the set of the prompt, input "@fruit", seed = 3, output = "yellow"... then this is now a fact that I could just memoize.

Would that be faster to retrieve the memoized result than calculating the result via the LLM?

And, what do we do with the thought that that set of information is "always true" with regards to intellectual property?

I honestly don't know yet.

I stumbled across it looking for CSS flex masonry examples.

Use UUIDv7 for your primary key but only for large scale databases and only for internal keys. Don't expose these keys to anything outside the database or application.

Use UUIDv4 columns with unique indexes over them as "external IDs" which are what is exposed via APIs to other systems.

Basically, create two IDs for one record - one random but not the primary key, and one sequential, that is the primary key.

I have done this in real systems.. and it works.

Whether it's a mirage or not, the ability to produce a symbolically logical result that has valuable meaning seems real enough to me.

Especially since most meaning is assigned by humans onto the world... so too can we choose to assign meaning (or not) to the output of a chain of symbolic logic processing?

Edit: maybe it is not so much that an LLM calculates/evaluates the result of symbolic logic as it is that it "follows" the pattern of logic encoded into the model.

[1] https://en.wikipedia.org/wiki/Logic

Personal profit maximization only works to a point - for example, if you get too old, sick or the system rejects you early and curtails or limits your ability to make money.

I don't disagree that money gives you options, but, far too many people wait until they have enough money to give back.

If you give back while you are working (e.g. balancing working for profit vs working for nonprofit, altruistic reasons, etc.) - that's awesome. The challenge there is maximizing the good you can do if you're giving too much time and energy to your profit maximization.

At some point, someone has do physically do the needed good work.

For myself, the calculus has shifted. I personally decided I cannot wait until I have enough money, or I am maximizing my profit, to go out and help people.

I also cannot wait until I am physically or mentally unable to help beyond financial contributions. Also, I cannot afford to work in the current system that drains everything from you and leaves you no energy or time left, only money (if that).

Regarding the inherent maximum scaling limits of one person- I would challenge your thinking.

Power laws of networks may demonstrate that helping a small number of the right people might be enough to unleash the butterfly effect or play into ongoing changes.

Also, the physical limits of humanity on one person apply to a billionaire as much as a person with little money. I'm not saying a billionaire, millionaire, or person with significant finances isn't more mobile/capable, but it's not a given.

I am for reasonable profit and balance. There is nothing inherently wrong with maximizing profit if someone chooses.

But if we all spend our time on maximizing profit, there still, for the time being and probably well into the future, still needs to be boots on the ground doing work that is not for profit.

In this case, it's so easy for some to say everything else is bad if it is not AI, or you have to include AI "because it's the future".

I just wanted to clarify that I am very pragmatic in my approach to new tech and AI.

I used to work on enterprise ERP, MRP, sales and support systems (Oracle, Salesforce, ServiceNow, and more)... not always by choice.

At one point a project came up to rebuild our core customer experience portal across our business. I got to build with the rule based chatbots to implement an assistant in that web app.

Not easy, but it worked well enough, especially for simple scenarios. For example, imagine a field support engineer being able to pull up the exact page in a big technical document for some obscure part by just asking for what they need, in seconds.

Or, being able to reorder some quantity of part by just asking for it, or checking the quantity of same.

These are time savers, especially if you have a voice to assistant interface too. So you can just type something, or ask for what you need without typing.

Tablets were common for manufacturing, inventory or service staff. Being able to pull up our interface and just say what they needed and get prompted for simple things - there were TONs of pragmatic small wins there.

Current LLM just make that easier than the old rigid rule based way we had to code those assistants (at the price of going wrong sometimes).

I care about efficiency, pleasant user experience, and pragmatics.. not AI for the sake of AI.

All that to say, if a drop down box works better for some use case, instead of AI - or even a rule based chatbot - hell yeah! Save on them tokens.

Genuine question though - have you implemented an AI assistant/chat interface recently using LLMs on top of a UI?

I agree it can be a rabbit hole, but I just got through doing it on an app and there were definitely some things it really made way simpler and some complex scenarios that I'm not sure could have been done any more simply.