So combine that with the Hawthorne effect and new business or health initiatives that can look great simply because participants notice change and notice the increased attention. However many human patterns have a tendency to regress to the mean.

Personally I have seen this a lot with developer tools and DevOps. A new SEV/incident/disaster happens and everyone rushes to create or onboard to a tool that would help. Around the office everyone raves about it and is sure that it would fix all issues. And the number of commits goes up, or the number of SEV's in an area decreases for a while. People were paying attention, after a while the tool starts to slow down or not be as used. It's got rough edges that weren't seen or scenarios that were supposed to be supported never get fully integrated. Eventually the patterns regress, but with more tools and more complexity.

- https://pmc.ncbi.nlm.nih.gov/articles/PMC1936999/

- https://arxiv.org/abs/2102.12893

To see that LLMs aren't capable of this, I present all of the prompt jailbreaks that rely on repeated admonitions. And that makes sense if you think about the training data. There's not a lot of human writing that takes a fact and then confidently asserts the opposite as data mounts.

LLMs produce the most likely response from the input embeddings. Almost always, the easiest is that the next token is in agreement of the other tokens in the sequence. The problem in poker is that a good amount of the tokens in the sequence are masked and/or controlled by a villain who is actively trying to deceive.

Also, notice that I'm careful to say LLM's and not generalize to all attention head + MLP models. As attention with softmax and dot product is a good universal function. Instead, it's the large language model part that makes the models not great fits for poker. Human text doesn't have a latent space that's written about enough and thoroughly enough to have poker solved in there.

For example when computing the counterfactual tree for 9 way preflop. 9 players have up to 6 different times that they can be asked to perform an action (seat 0 can bet 1, seat 1 raises min, seat 2 calls, back to seat 0 raises min, with seat 1 calling, and seat 2 raising min, etc). Each of those actions has check, fold, bet min, raise the min (starting blinds of 100 are pretty high all ready), raise one more than the min, raise two more than the min, ... raise all in (with up to a million chips).

(1,000,000.00 - 999,900.00) ^ 6 times per round ^ 9 players That's just for pre flop. Postflop, River, Turn, Showdown. Now imagine that we have to simulate which cards they have and which order they come in the streets (that greatly changes the value of the pot).

As for LLMs being great at range stats, I would point you to the latest research by UChicago. Text trained LLMs are horrible at multiplication. Try getting any of them to multiply any non-regular number by e or pi. https://computerscience.uchicago.edu/news/why-cant-powerful-...

Don't get what I'm saying wrong though. Masked attention and sequence-based context models are going to be critical to machines solving hidden information problems like this. Large Language Models trained on the web crawl and the stack with text input will not be those models though.

I think an RL environment is needed to solve poker with an ML model. I also think that like chess, you need the model to do some approximate work. General-purpose LLMs trained on text corpus are bad at math, bad at accuracy, and struggle to stay on task while exploring.

So a purpose built model with a purpose built exploring harness is likely needed. I've built the basis of an RL like environment, and the basis of learning agents in rust for poker. Next steps to come.

LLM's aren't capable of this deception. They can't be told that they have some thing, pretend like they have something else, and then revert to gound truth. Their egar nature with large context leads to them getting confused.

On top of that there's a lot of precise math. In no limit the bets are not capped, so you can bet 9.2 big blinds in a spot. That could be profitable because your opponents will call and lose (eg the players willing to pay that sometimes have hands that you can beat). However betting 9.8 big blinds might be enough to scare off the good hands. So there's a lot of probiblity math with multiplication.

Deep math with multiplication and accuracy are not the forte of llm's.

First being the hidden information, you don't know your opponents hand holdings; that is to say everyone in the game has a different information set.

The second is that there's a variable number of players in the game at any time. Heads up games are closer to solved. Mid ring games have had some decent attempts made. Full ring with 9 players is hard, and academic papers on it are sparse.

The third is the potential number of actions. For no limit games there's a lot of potential actions, as you can bet in small decimal increments of a big blind. Betting 4.4 big blinds could be correct and profitable, while betting 4.9 big blinds could be losing, so there's a lot to explore.

CFR is still the best, however, like chess, we need a network that can help evaluate the position. Unlike chess, the hard part isn't knowing a value; it's knowing what the current game position is. For that, we need something unique.

I'm pretty convinced that this is solvable. I've been working on rs-poker for quite a while. Right now we have a whole multi-handed arena implemented, and a multi-threaded counterfactual framework (multi-threaded, with no memory fragmentation, and good cache coherency)

With BERT and some clever sequence encoding we can create a powerful agent. If anyone is interested, my email is: elliott.neil.clark@gmail.com

We know how to compute the best moves in poker (it's computationally challenging; the more choices and players are present, the more likely it is that most attempts only even try at heads-up).

With all that said, I do think there's a way to use attention and BERT to solve poker (when trained on non-text sequences). We need a better corpus of games and some training time on unique models. If anyone is interested, my email is elliott.neil.clark @ gmail.com

- https://github.com/batteries-included/batteries-included/ - https://www.batteriesincl.com/

They called out AlphaTheta, so here's hoping that it is. That would make my decision to move off of Spotify for personal streaming even easier

Shout out to the HBase and ZippyDB teams! This is the first public acknowledgment that ZippyDB was converged upon.

It's also super cool to see the Developer Efficiency pushes called out. 10,000 Services pushed daily, or every commit is so impressive.

When I left FB, I couldn't find anything close. So, I'm building the infra that I was missing as a startup. Batteries Included. https://www.batteriesincl.com/ https://github.com/batteries-included/batteries-included/

Comments URL: https://news.ycombinator.com/item?id=43005918

Points: 2

# Comments: 0

https://github.com/batteries-included/batteries-included https://www.batteriesincl.com/ https://www.batteriesincl.com/LICENSE-1.0

I started the company because I wanted to give the infrastructure team that FAANG companies have to smaller enterprises. Most of the best infrastructure is open source but too complicated to use or maintain. So we've built a full platform that will run on any Kubernetes cluster, giving a company a push-button infrastructure with everything built on open source. So you get Heroku with single sign-on and no CLI needed. Or you get a full RAG stack with model hosting on your EKS cluster.

Since most of the services and projects we're building on top of are open source, we wanted to give the code to the world while being sustainable in the long term as a team. I had also been a part of Cloudera, and I had seen the havoc that open core had on the long-term success of Hadoop. So, I wanted something different for licensing. We ended up with a license that somewhat resembles the FSL but fixes its major (in my opinion) problem. We don't use the competing use clause instead opting for a total install size requirement.

I'm happy to chat with anyone about this, my email is in my profile. Good Luck nd I hope it works for you.

It’s a fun little eye opener that starts conversations. I wish more of those conversations ended up moving decision makers

When MS rolls up, they say we are charging for your usage of the MS database, Office, Outlook, Microsoft Windows 11, and the security promises. They are explicit that developing with and on Microsoft allows you access to the ecosystem. So the total bill is high, but part of that bill is a gateway into everyone else using Office, Outlook, Excel, Visual Studio, or SharePoint. The world runs on Excel and MS enterprise sales know that. They are negotiating a contract for one-of-a-kind software and access to the world of MS.

Redhat rolls up saying we want to charge you. They don't get to say that if you don't pay, the company will lose access to the software or the ecosystem. They don't get to say they are gatekeepers to other Linux users. Redhat can't claim to be giving the database or the development environment; everyone thinks they are free. If you stop paying Redhat, you probably can find an almost package for package compatible alternative in a rolling release (source: watched that happen multiple times CentOS, et al). So instead Redhat sells a contract for service, support, and indemnity. Those are great products and Red Hat will continue for a long time. They will just have very different staying power when contracts are renewed. They will have very different revenue growth.

It's not how I want it to be, just how I see it.

Source: Worked at MS and have friends who are former Redhat.

I agree the contract should be clear up front. Changing expectations later is a big problem. People want to give away the software for a while, using it as a loss leader to get attention while not being honest about their later need for money to fund the ongoing concern.

I tried to write a little bit about that in my post here: https://www.batteriesincl.com/posts/fairsource

I was starting Batteries Included and had been writing it in Elixir. I want to give back to the community, show how to use Phoenix/Live view, and be transparent about what users are running, etc. However, I also know that if this will work long-term, I can not give it away to everyone forever. So it's better to be honest about things as early as possible.

We paid a very smart lawyer to draft the best compromise we could as early as possible.

https://www.batteriesincl.com/LICENSE-1.0

This means we can develop in the open here: https://github.com/batteries-included/batteries-included while also giving it away long term and still being honest that this will require some long-term revenue stream. That revenue stream will come from the companies using it on larger installs.

Red Hat is the only company that has really made a living off of Linux. Even then, their contracts are orders of magnitudes less than the exact same customers will be paying Microsoft.

Linux is successful and remarkable and every company sees the value in having it around. So there's a shared mutual need. However, that doesn't mean that anyone can make it into more than barely scraping by as a going business.

- The anchoring principal. Once you have set the that the software is free, humans expect it to be free forever and all related things are judged off the initial impressions' price bucket. Humans will never want to pay for it later. It's judged worthless.

Open-core and closed-source addons and support models have misaligned principles. The community wants things to be easy to use and opinionated, while the OSS company wants to include as many customers as possible who need or want your help with their niche choices.

- Sustainability is awful. If you start an open source project you're either going to burn yourself and the community out, or require funding to do it as a day job. So, if you want the project not to stop early, you need money to pay for developers to make software better.

- Larger companies want something opinionated but rarely what's good for most of the community. So eventually when big tech/big industry is paying for developers to work on the project, there's a point where the large company will want their cake and the community is hostage. Do that enough times and the large company forks internally and the community fractures or withers out.

Source: I was at Cloudera while the Big Data craze took off. Then, I did open source for large tech.