Poker AI: Learning and Evaluating Strategies Under Imperfect Information

Independent project · Python · October 2026–present

I am building a poker-playing agent in Python to explore how strategies develop through self-play. Poker makes this an interesting problem: a player must choose actions without knowing the opponent’s cards, and a good decision can still lose an individual hand.

The project includes a Texas Hold’em engine, hand and equity evaluators, game-theoretic learning algorithms, and tools for testing saved strategies. My current focus is how to simplify the information an agent uses without losing the distinctions that matter for its decisions.

Building the engine

I began with the mechanics: representing cards, ranking hands, calculating equity, and implementing betting and side pots for two to six players. Reliable mechanics are essential here—a mistake in how a hand ends can make a learning algorithm appear better or worse than it is.

I checked the five-card evaluator against Treys on all roughly 2.6 million possible hands, with no disagreements. I also checked two million sampled six- and seven-card hands and compared the game engine against a reference implementation over 1,000 seeded game traces. The engine notes document the comparison’s scope and differences between rule conventions.

Learning through self-play

I implemented counterfactual regret minimization (CFR) and several variants. The basic idea is to repeatedly compare the actions taken with alternatives, then use those regrets to adjust future play. The resulting average strategy can become harder to exploit as training progresses.

I first tested the solvers on Kuhn and Leduc, small poker games whose strategies can be evaluated exactly. Their results closely matched OpenSpiel at the same training iterations. This gave me a way to check the learning code before moving to larger Hold’em experiments.

CFR variants become less exploitable over training iterations in Kuhn and Leduc poker.
Learning curves in Kuhn and Leduc poker. Lower exploitability means an optimal opponent has less room to take advantage of the strategy. Solver comparison.

For the larger experiments, I use sampled self-play to explore parts of the game tree at each iteration. I measure exploitability: how much an optimal counterstrategy can gain against the learned strategy. In the Hold’em results below, this is expressed as a percentage of the initial pot. It measures a strategy’s vulnerability, rather than its win rate against one particular opponent.

What information does the agent need?

Hold’em has too many situations to treat each one separately. I have been experimenting with abstractions that group similar situations under a shared strategy. The difficult part is deciding which differences are safe to ignore.

Grouping hands by broad categories or board texture can hide strategically important differences. For example, cards on the board remove possible hands from each player’s range. These blocker effects can change the value of an action even when a hand’s category stays the same.

My latest representation adds a compact description of those effects for my own possible hands, alongside features describing the opponent’s range. In one restricted game, full-tree training reached similarly low exploitability to the exact representation using about half as many strategy entries: 1,770 compared with 3,336. That is a reduction in entries; the longer feature keys also take space, so it does not establish an overall memory saving.

The next question was whether sampled training could learn a comparably good strategy with the same computational budget. The answer so far is mixed:

Selected representations in the harder of two small development games. Exploitability at ten million node visits, across three fixed training seeds; lower is better.
RepresentationMedianWorst seed
Exact / raw public0.29%0.66%
Opponent-range features2.79%3.35%
Compact blocker-aware description2.39%2.49%

The compact description improved on the previous range-based representation, but its median still missed the 1% target I set before the experiment. It can represent a strong strategy under full-tree training; learning that strategy efficiently through sampling remains an open problem.

Sampled-learning curves compare six representations in two restricted poker games.
The easier game converges quickly, while the harder game exposes differences between representations. Shaded ranges show the three fixed seeds, not confidence intervals. Experiment design and full results · Full-tree comparison.

Does a different training opponent help?

I also tested a different approach: keep the learner’s simplified view of the game, but train it against an opponent that distinguishes more detailed situations. Both agents use my own learning implementation. A paired control uses the same split of computational work while keeping both players’ information coarse.

An earlier experiment with the coarse representation, in the harder known game. Mean exploitability across three seeds at the same total node-work budget.
Training methodMean exploitability
Ordinary coarse self-play2.17%
Paired coarse control3.03%
Training against a finer-information opponent0.60%
Ordinary fine self-play0.38%

The finer-opponent approach produced a substantial improvement in this game, although two seeds improved and one worsened. It did not meet the experiment’s overall requirement of positive median improvement in both games: the easier game was already near zero and showed no median gain. This experiment used the earlier coarse representation; applying the approach to the new blocker-aware version is still pending.

Four training approaches compared across two games, showing mean learning curves and individual seed outcomes.
Changing the opponent’s information improved the result in the harder game. The plot retains all four methods and all three seeds. Methods and results.

Following a weakness to a specific hand

To understand the compact agent’s remaining weaknesses, I held saved opponents fixed and traced how a more informed counterstrategy could exploit them. All three sampled seeds pointed to the same turn decision with 9♣ 9♦. The simplified response folded, while a response able to use finer information in later decisions preferred calling.

That turn situation was not itself merged with another state. Its action values depended on the future choices available to the responder. This makes the hand a useful example of why a weakness that appears on the turn can involve distinctions needed later on the river. The decision trace and fixed-opponent analysis contain the underlying comparisons.

My next step is to test the compact learner against finer-information opponents. Before running that comparison, I am investigating the cost of verifying saved strategies: the estimated verification cost was over budget, and profiling identified repeated checks of the game’s information structure as a major cost. Follow-up status · Profiling notes.

Testing the agent in matches

I also ran an earlier version of the agent through 36,000 simulated hands against six fixed opponents. Each comparison reuses the same deals with the seats switched to reduce the effect of card luck. The uncertainty calculations account for the shared deals.

Match estimates and uncertainty intervals against six fixed poker opponents.
The earlier agent had positive point estimates against some baselines and losses against the tight-aggressive and equity opponents. The conservative statistical test did not establish a positive advantage against any of the six. These are earlier-agent results, not matches of the new compact learner. Match evaluation.

The project is still in progress. The strongest conclusions so far concern particular learning methods and representations in small, separately trained games; I have not established competitive full-game Hold’em strength.

The repository contains the code, saved strategies, and experiment reports. The results on this page come from a recorded project snapshot; the learning journal follows the experiments in more detail.