Comments URL: https://news.ycombinator.com/item?id=49351431

Points: 4

# Comments: 0

Remote: Yes

Willing to relocate: Chicago, IL, USA

Technologies: ML, C++, Hadoop, Java, Elixir, AWS, Python, C#, Rust, Kubernetes

Résumé/CV: https://elliottclark.info/resume/

Email: elliott.neil.clark@gmail.com

Senior Engineering Leader specializing in distributed systems & ML infrastructure. Founder & CEO of Batteries Included (raised $1.2M, built a self-driving cloud platform). Former Senior Tech Lead at Facebook Ads, scaling ML-driven ad services for billions. I care about correctness, latency, and scalability in production, and I'm looking to lead teams tackling hard, high-impact problems.

I do have some plans to release a follow on with some ml components ala Pluribus.

- tokio based async exploration - rust slab allocation based tree structure for regret minimization - perfect hashing for faster hand ranking - a TUI via https://ratatui.rs/

Creating your own poker bot and having them compete in an arena should be less than 100 lines of code: https://docs.rs/rs_poker/latest/rs_poker/arena/index.html

I need more eyes on the implementation, and more attempts to make the algorithms and agents state of the art. I know I can't have found the optimal configurations and algorithms; I'd love for the open source community to prove me wrong.

There's one glaring limitation that I need to fix. Right now the CFR agents can't predict their opponents hands so all regret minimization is either using the exact hand (so pretty conservative) or random (so too wide). I have some ideas here but I need more data and more discussion.

direct github: https://github.com/elliottneilclark/rs-poker

Comments URL: https://news.ycombinator.com/item?id=48466653

Points: 3

# Comments: 2

So combine that with the Hawthorne effect and new business or health initiatives that can look great simply because participants notice change and notice the increased attention. However many human patterns have a tendency to regress to the mean.

Personally I have seen this a lot with developer tools and DevOps. A new SEV/incident/disaster happens and everyone rushes to create or onboard to a tool that would help. Around the office everyone raves about it and is sure that it would fix all issues. And the number of commits goes up, or the number of SEV's in an area decreases for a while. People were paying attention, after a while the tool starts to slow down or not be as used. It's got rough edges that weren't seen or scenarios that were supposed to be supported never get fully integrated. Eventually the patterns regress, but with more tools and more complexity.

- https://pmc.ncbi.nlm.nih.gov/articles/PMC1936999/

- https://arxiv.org/abs/2102.12893

To see that LLMs aren't capable of this, I present all of the prompt jailbreaks that rely on repeated admonitions. And that makes sense if you think about the training data. There's not a lot of human writing that takes a fact and then confidently asserts the opposite as data mounts.

LLMs produce the most likely response from the input embeddings. Almost always, the easiest is that the next token is in agreement of the other tokens in the sequence. The problem in poker is that a good amount of the tokens in the sequence are masked and/or controlled by a villain who is actively trying to deceive.

Also, notice that I'm careful to say LLM's and not generalize to all attention head + MLP models. As attention with softmax and dot product is a good universal function. Instead, it's the large language model part that makes the models not great fits for poker. Human text doesn't have a latent space that's written about enough and thoroughly enough to have poker solved in there.

For example when computing the counterfactual tree for 9 way preflop. 9 players have up to 6 different times that they can be asked to perform an action (seat 0 can bet 1, seat 1 raises min, seat 2 calls, back to seat 0 raises min, with seat 1 calling, and seat 2 raising min, etc). Each of those actions has check, fold, bet min, raise the min (starting blinds of 100 are pretty high all ready), raise one more than the min, raise two more than the min, ... raise all in (with up to a million chips).

(1,000,000.00 - 999,900.00) ^ 6 times per round ^ 9 players That's just for pre flop. Postflop, River, Turn, Showdown. Now imagine that we have to simulate which cards they have and which order they come in the streets (that greatly changes the value of the pot).

As for LLMs being great at range stats, I would point you to the latest research by UChicago. Text trained LLMs are horrible at multiplication. Try getting any of them to multiply any non-regular number by e or pi. https://computerscience.uchicago.edu/news/why-cant-powerful-...

Don't get what I'm saying wrong though. Masked attention and sequence-based context models are going to be critical to machines solving hidden information problems like this. Large Language Models trained on the web crawl and the stack with text input will not be those models though.

I think an RL environment is needed to solve poker with an ML model. I also think that like chess, you need the model to do some approximate work. General-purpose LLMs trained on text corpus are bad at math, bad at accuracy, and struggle to stay on task while exploring.

So a purpose built model with a purpose built exploring harness is likely needed. I've built the basis of an RL like environment, and the basis of learning agents in rust for poker. Next steps to come.

LLM's aren't capable of this deception. They can't be told that they have some thing, pretend like they have something else, and then revert to gound truth. Their egar nature with large context leads to them getting confused.

On top of that there's a lot of precise math. In no limit the bets are not capped, so you can bet 9.2 big blinds in a spot. That could be profitable because your opponents will call and lose (eg the players willing to pay that sometimes have hands that you can beat). However betting 9.8 big blinds might be enough to scare off the good hands. So there's a lot of probiblity math with multiplication.

Deep math with multiplication and accuracy are not the forte of llm's.

First being the hidden information, you don't know your opponents hand holdings; that is to say everyone in the game has a different information set.

The second is that there's a variable number of players in the game at any time. Heads up games are closer to solved. Mid ring games have had some decent attempts made. Full ring with 9 players is hard, and academic papers on it are sparse.

The third is the potential number of actions. For no limit games there's a lot of potential actions, as you can bet in small decimal increments of a big blind. Betting 4.4 big blinds could be correct and profitable, while betting 4.9 big blinds could be losing, so there's a lot to explore.

CFR is still the best, however, like chess, we need a network that can help evaluate the position. Unlike chess, the hard part isn't knowing a value; it's knowing what the current game position is. For that, we need something unique.

I'm pretty convinced that this is solvable. I've been working on rs-poker for quite a while. Right now we have a whole multi-handed arena implemented, and a multi-threaded counterfactual framework (multi-threaded, with no memory fragmentation, and good cache coherency)

With BERT and some clever sequence encoding we can create a powerful agent. If anyone is interested, my email is: elliott.neil.clark@gmail.com

We know how to compute the best moves in poker (it's computationally challenging; the more choices and players are present, the more likely it is that most attempts only even try at heads-up).

With all that said, I do think there's a way to use attention and BERT to solve poker (when trained on non-text sequences). We need a better corpus of games and some training time on unique models. If anyone is interested, my email is elliott.neil.clark @ gmail.com

- https://github.com/batteries-included/batteries-included/ - https://www.batteriesincl.com/

They called out AlphaTheta, so here's hoping that it is. That would make my decision to move off of Spotify for personal streaming even easier

Shout out to the HBase and ZippyDB teams! This is the first public acknowledgment that ZippyDB was converged upon.

It's also super cool to see the Developer Efficiency pushes called out. 10,000 Services pushed daily, or every commit is so impressive.

When I left FB, I couldn't find anything close. So, I'm building the infra that I was missing as a startup. Batteries Included. https://www.batteriesincl.com/ https://github.com/batteries-included/batteries-included/

Comments URL: https://news.ycombinator.com/item?id=43005918

Points: 2

# Comments: 0

https://github.com/batteries-included/batteries-included https://www.batteriesincl.com/ https://www.batteriesincl.com/LICENSE-1.0

I started the company because I wanted to give the infrastructure team that FAANG companies have to smaller enterprises. Most of the best infrastructure is open source but too complicated to use or maintain. So we've built a full platform that will run on any Kubernetes cluster, giving a company a push-button infrastructure with everything built on open source. So you get Heroku with single sign-on and no CLI needed. Or you get a full RAG stack with model hosting on your EKS cluster.

Since most of the services and projects we're building on top of are open source, we wanted to give the code to the world while being sustainable in the long term as a team. I had also been a part of Cloudera, and I had seen the havoc that open core had on the long-term success of Hadoop. So, I wanted something different for licensing. We ended up with a license that somewhat resembles the FSL but fixes its major (in my opinion) problem. We don't use the competing use clause instead opting for a total install size requirement.

I'm happy to chat with anyone about this, my email is in my profile. Good Luck nd I hope it works for you.

It’s a fun little eye opener that starts conversations. I wish more of those conversations ended up moving decision makers