Poker AI: Learning and Evaluating Strategies Under Imperfect Information
I am building a poker-playing agent in Python to explore how strategies develop through self-play. Poker makes this an interesting problem: a player must choose actions without knowing the opponent’s cards, and a good decision can still lose an individual hand.
The project includes a Texas Hold’em engine, hand and equity evaluators, game-theoretic learning algorithms, and tools for testing saved strategies. My current focus is how to simplify the information an agent uses without losing the distinctions that matter for its decisions.
RepositoryLearning journalExperiment log
Building the engine
I began with the mechanics: representing cards, ranking hands, calculating equity, and implementing betting and side pots for two to six players. Reliable mechanics are essential here—a mistake in how a hand ends can make a learning algorithm appear better or worse than it is.
I checked the five-card evaluator against Treys on all roughly 2.6 million possible hands, with no disagreements. I also checked two million sampled six- and seven-card hands and compared the game engine against a reference implementation over 1,000 seeded game traces. The engine notes document the comparison’s scope and differences between rule conventions.
Learning through self-play
I implemented counterfactual regret minimization (CFR) and several variants. The basic idea is to repeatedly compare the actions taken with alternatives, then use those regrets to adjust future play. The resulting average strategy can become harder to exploit as training progresses.
I first tested the solvers on Kuhn and Leduc, small poker games whose strategies can be evaluated exactly. Their results closely matched OpenSpiel at the same training iterations. This gave me a way to check the learning code before moving to larger Hold’em experiments.

For the larger experiments, I use sampled self-play to explore parts of the game tree at each iteration. I measure exploitability: how much an optimal counterstrategy can gain against the learned strategy. In the Hold’em results below, this is expressed as a percentage of the initial pot. It measures a strategy’s vulnerability, rather than its win rate against one particular opponent.
What information does the agent need?
Hold’em has too many situations to treat each one separately. I have been experimenting with abstractions that group similar situations under a shared strategy. The difficult part is deciding which differences are safe to ignore.
Grouping hands by broad categories or board texture can hide strategically important differences. For example, cards on the board remove possible hands from each player’s range. These blocker effects can change the value of an action even when a hand’s category stays the same.
My latest representation adds a compact description of those effects for my own possible hands, alongside features describing the opponent’s range. In one restricted game, full-tree training reached similarly low exploitability to the exact representation using about half as many strategy entries: 1,770 compared with 3,336. That is a reduction in entries; the longer feature keys also take space, so it does not establish an overall memory saving.
The next question was whether sampled training could learn a comparably good strategy with the same computational budget. The answer so far is mixed:
| Representation | Median | Worst seed |
|---|---|---|
| Exact / raw public | 0.29% | 0.66% |
| Opponent-range features | 2.79% | 3.35% |
| Compact blocker-aware description | 2.39% | 2.49% |
The compact description improved on the previous range-based representation, but its median still missed the 1% target I set before the experiment. It can represent a strong strategy under full-tree training; learning that strategy efficiently through sampling remains an open problem.

Does a different training opponent help?
I also tested a different approach: keep the learner’s simplified view of the game, but train it against an opponent that distinguishes more detailed situations. Both agents use my own learning implementation. A paired control uses the same split of computational work while keeping both players’ information coarse.
| Training method | Mean exploitability |
|---|---|
| Ordinary coarse self-play | 2.17% |
| Paired coarse control | 3.03% |
| Training against a finer-information opponent | 0.60% |
| Ordinary fine self-play | 0.38% |
The finer-opponent approach produced a substantial improvement in this game, although two seeds improved and one worsened. It did not meet the experiment’s overall requirement of positive median improvement in both games: the easier game was already near zero and showed no median gain. This experiment used the earlier coarse representation; applying the approach to the new blocker-aware version is still pending.

Following a weakness to a specific hand
To understand the compact agent’s remaining weaknesses, I held saved opponents fixed and traced how a more informed counterstrategy could exploit them. All three sampled seeds pointed to the same turn decision with 9♣ 9♦. The simplified response folded, while a response able to use finer information in later decisions preferred calling.
That turn situation was not itself merged with another state. Its action values depended on the future choices available to the responder. This makes the hand a useful example of why a weakness that appears on the turn can involve distinctions needed later on the river. The decision trace and fixed-opponent analysis contain the underlying comparisons.
My next step is to test the compact learner against finer-information opponents. Before running that comparison, I am investigating the cost of verifying saved strategies: the estimated verification cost was over budget, and profiling identified repeated checks of the game’s information structure as a major cost. Follow-up status · Profiling notes.
Testing the agent in matches
I also ran an earlier version of the agent through 36,000 simulated hands against six fixed opponents. Each comparison reuses the same deals with the seats switched to reduce the effect of card luck. The uncertainty calculations account for the shared deals.

The project is still in progress. The strongest conclusions so far concern particular learning methods and representations in small, separately trained games; I have not established competitive full-game Hold’em strength.
The repository contains the code, saved strategies, and experiment reports. The results on this page come from a recorded project snapshot; the learning journal follows the experiments in more detail.