Poker Bots: Rise of the Machines
The robots are here, and in addition to wanting to eradicate humanity from the face of the planet, they also want your money. The Tip Top Fox examines the rise of the poker bot
If the Terminator films are to be believed, humanity is just one small step away from destruction at the metallic hands of cold, calculating robots. While artificial intelligence (AI) has come along leaps and bounds in the 21st century, Skynet and its cybernetic chums are a long way off conquering the globe. They are, however, becoming a nuisance in cyberspace, more specifically at the virtual baize.
The Birth of Poker AI
A sophisticated poker-playing robot, poker bots (or bots), are a form of AI programmed to play online poker against human opponents. Renowned poker theorist Mike ‘The Mad Genius’ Caro created one of the first poker bots, Orac (Caro spelled backward), in 1981. After spending two years programming and testing it against himself, Caro tested it against World Series of Poker (WSOP) Champions Tom McEvoy and Doyle Brunson. Orac proceeded to lose two out of three matches. Then came a publicity match against casino owner Bob Stupak on US national television, which Stupak won.
A decade later in 1991 and technology had moved on. Computers were faster and more efficient and the World Wide Web, while still in its infancy, had emerged. While online poker would still not exist for another seven years, respected US professor Darse Billings began work on several AI software programs specialising in imperfect information games and decision-making under conditions of uncertainty – poker bots.
While bots have been used to play optimal strategy in other online card games like blackjack, poker is a different animal. The biggest obstacle lies in the fact that poker is a game of incomplete information. There is a need for any poker-playing program to be able to employ a variety of strategies at different times, such as bluffing and laying traps for opponents. Owing to the complexities of No-Limit Hold’em, Billings decided to tackle what he classed the most solvable type of poker – heads-up Limit Hold’em.
The Rise of the Poker Bot
Following the birth of internet poker in 1998 and its subsequent explosion after online qualifier Chris Moneymaker’s 2003 WSOP Main Event win, poker bots based on one of Billings’ earlier experiments, Vexbot, began to show up online. Their play was rudimentary and followed basic decision-making logic, which needed to be manually inputted in real-time. However, by 2007 Billings and his colleagues at the University of Alberta released a new incarnation of the poker bot – Polaris.
“It will show you things that no human player has ever shown you before,’ said Billings of his latest creation, which had the ability to model opponent’s behavior. “One of the biggest advantages that programs have is that they have no fear. Humans can be intimidated. They will back off in the face of a very aggressive player. A bot will not. It has no compunction about doing whatever it will take to win. It will raise you with any two cards if it thinks that it has a very slight advantage based on your history. And it can induce a lot of anger and emotional upset. These things are ‘tilt monsters’.”
Man Versus Machine Round 1: Polaris
In order to test his latest invention Billings called on one of poker’s more intellectual players, Phil “The Unabomber” Laak, who had previously played Billings’ earlier creation, Vexbot, in 2005.
On 23 July 2007, Laak accompanied by fellow pro Ali Eslami played a series of heads-up Limit Hold’em matches against Polaris. After two days of play, man triumphed over machine – beating Polaris by two wins to one loss and a draw. However, Laak claimed beating Polaris was far easy:
“One of the interviewers who didn’t really know anything about cards, gambling and statistics was like: ‘Well isn’t it great that you won because you’ve staved off the advance of the computers?’. I was like ‘No, what you don’t understand is that I was probably between 52 and 57% favourite’. Which meant that easily it’s almost a coin toss where it could have beaten me,” said Laak of the experience.
Just one year later in 2008, Billings took an upgraded version of Polaris with stronger component strategies and the ability to learn from its opponents to the Gaming Life Expo in Las Vegas for a second Man-Machine Poker Championship. Polaris won by three games to two, with one draw. The machines were on the rise.
“Poker will never be solved but a program that will beat most humans does exist right now.”Phil “The Unabomber” Laak
Does that mean you should start panicking? Not yet.
“Playing online Limit Heads-up $50/$100, occasionally I would meet a bot – because I could feel the rhythm in the thinking. It’s like Doyle Brunson once said: ‘If you’re playing online and you feel like you’re being cheated, you are,’” stated Laak. “But then I don’t mind playing the bots because those bots aren’t as strong as Polaris.”
Man Versus Machine Round 2: Cepheus
Nearly a decade later and a new wave of poker bots appeared to test their mettle against the humans. The first was the natural successor to Polaris, a Limit Hold’em bot developed by a research team from the University of Alberta called Cepheus, which used machine learning to teach itself all the statistical heads-up possibilities – all 316 quadrillion of them.
Cepheus represented a milestone in the development of AI and game theory, teaching itself to play close to perfect poker after playing the equivalent of more than a billion, billion hands. While this made it close to unbeatable in the long run, it didn’t make it ‘the best’ HU Limit Hold’em player. The program lacked the ability to adapt. While it will win long term, a skilled human player will adapt faster in the short term.
It is possible to pit your skills against it online, with the program still available to play against on the University of Alberta website. However, heads-up Limit Hold’em is not exactly the most popular poker game out there. While Cepheus may be unbeatable in the long run, it did not pose a threat to the online poker economy. That would come later.
Man Versus Machine Round 3: Claudico
While Polaris and Cepheus pushed the boundaries of poker AI development, both were focused on solving Limit Hold’em, a poker variant with fewer complexities owing to the limited nature of the betting structure.
No-Limit was considered far too complex with a greater degree of unknown variables. Bluffing and bet-sizing were two of the biggest obstacles to creating a poker AI that could deal with the unpredictable nature of the game. The vast number of possible probability variables that come with every No-Limit poker hand was also a factor.
However, in 2015, Carnegie Mellon University professor Tuomas Sandholm and his students developed Claudico. Taken from the Latin verb of the same name meaning ‘limp’ – which in poker parlance means to enter a hand without raising – Claudico was created to tackle the game of No-Limit Hold’em. Sandholm picked the name deliberately, with Claudico utilising limping as one of its preferred strategies when playing. Sandholm explained his motivation for designing the bot:
“Poker is now a benchmark for artificial intelligence research, just as chess once was.”Tuomas Sandholm
“It’s a game of exceeding complexity that requires a machine to make decisions based on incomplete and often misleading information, thanks to bluffing, slow play, and other decoys,” said Sandholm in a United Press International article published in 2015. “And to win, the machine has to out-smart its human opponents.”
Enter Doug Polk
it was the turn of respected poker pundit Doug Polk to take on the might of the latest upgrade to poker AI. Polk teamed up with fellow high stakes heads-up NLHE specialists Dong Kim, Bjorn Li, and Jason Les to take on Claudico at Rivers Casino in Pittsburgh, who along with Microsoft, put up a prize of $100,000 for the winner.
The man versus machine contest saw Claudico play 750 hands per day against each of the four opponents until each had played 20,000 hands, for a total of 80,000 hands overall. To minimize the influence of short-term luck or ‘variance’ two sets of identical hands were prepared each day. Claudico would play ‘Set 1’ against one of the human players playing ‘Set 2’, and then roles were reversed, with the humans playing ‘Set 1’ against Claudico, which played ‘Set 2’.
Sandholm guesstimated Claudico’s chances of winning as 50/50, Polk disagreed.
“I imagine that the humans have an edge here. However, it is very difficult to determine an outcome with any sort of stability, as I do not know what I am going to be up against.”Doug Polk
You can see Doug Polk battling against Claudico in the below video:
A Narrow Victory
After the 80,000 hands played out, humanity scored a narrow victory. So narrow in fact that professor Sandholm called it a statistical tie. How narrow? The poker professionals were up 732,713 in chips, although not all of them were successful. Bjorn Li scored the highest, winning 529,033 from the AI. Polk was the next best, posting a profit of 213,671, while Dong Kim took 70,491 from the poker bot. However, Claudico did prevail over Jason Les, showing an 80,482 profit.
While 732,713 sounds like a sizable win, if you consider the fact 170 million in chips were bet over the contest that only equates to just under 0.5% of the total chips in play. That was still good enough to win the $100,000 prize purse, however, which was split four ways.
“There are spots where it plays well and others where I just don’t understand it,” Polk said in a Carnegie Mellon press release. “Betting $19,000 to win a $700 pot just isn’t something that a person would do,” Polk stated.
So was poker saved from the rise of the robots? Polk discussed the experience of playing Claudico at length in a 2017 Forbes article and thinks it’s only a matter of time until the bots overtake humanity.
“I felt a bit unhappy with the way that the match ended last time. I am on the record saying that it was only a matter of time before the AI eventually overtakes humans, and that appears to have happened this year.”Doug Polk
Man Versus Machine Round 4: Libratus
So what was Polk alluding to when he mentioned ‘this year’? Just two years later, in 2017 Sandholm was back, with a new poker bot – Libratus. Latin for ‘balance’ Libratus utilised some of the programming elements from Claudico and improved on them. Once again the gauntlet was thrown down, with humanity called upon to defend their poker title.
The poker professionals representing humanity this time around featured two previous participants from the Claudico match, Kim and Les. Jimmy Chou and Daniel McAulay were also added to the roster. Taking place over 20 days, once again at the Rivers Casino in Pittsburgh, the competition followed the previous ‘Brains vs AI’ format, with $100,000 up for grabs. This time, however, the sample size was a larger 120,000 hands, with each player battling the AI over 30,000 hands.
The result would prove to be a historic and decisive one; Libratus led the pros by a collective 1,766,250 in chips. Sandholm and fellow Libratus developer Noam Brown – a Ph.D. student in computer science – said the sizable victory was statistically significant and not simply a matter of luck.
“The best AI’s ability to do strategic reasoning with imperfect information has now surpassed that of the best humans.”Tuomas Sandholm
Does that mean you need to be worried about facing Libratus at the virtual felt? Not yet. Libratus utilized Carnegie Mellon University’s supercomputer ‘Bridges’ to do a lot of the heavy lifting. Bridges’ total speed is 1.35 petaflops, about 7,250 times as fast as a high-end laptop. Its memory is 274 Terabytes, about 17,500 as much as you’d get in that laptop.
Man Versus Machine Round 5: Pluribus
Just like the iconic Rocky movies, the man versus machine battle would see a fifth installment. It was this 2019 development that really proved poker bots to be a legitimate online threat. Once again it was Carnegie Mellon University professor Tuomas Sandholm and Ph.D. student Noam Brown pushing the envelope. This time they teamed up with Facebook AI to create a bot designed to compete in an NLHE multiplayer environment.
Called Pluribus – which is Latin for ‘many’ – this sophisticated poker AI took on leading World Poker Tour title holder Darren Elias (who has four WPT titles) and Chris “Jesus” Ferguson over 5,000 hands of poker. Each pro played separately against five copies of Pluribus, which emerged victorious after the contest. In a Facebook AI article about the historic win, Ferguson had the following to say about his AI opponent:
“Pluribus is a very hard opponent to play against. It’s really hard to pin him down on any kind of hand. He’s also very good at making thin value bets on the river. He’s very good at extracting value out of his good hands.”Chris Ferguson, WSOP champion
To prove the win was no fluke, 13 top poker professionals including Nick Petrangelo and Greg Merson – all of whom boast over $1M in career tournament winnings – played 10,000 hands in a six-handed game against a single copy of Pluribus. Once again the AI triumphed, showcasing an ability to set traps and bluff successfully.
In fact, the bot was so successful the developers chose not to release the source code, mainly out of a concern it would damage the online game.
Are Poker Bots A Threat?
Type ‘poker bot’ into Google and you will discover hundreds of websites offering you the chance to make money. Do they work? Well, while those offering to sell you these pieces of kit at ‘bargain’ prices guarantee your success, you might want to take these claims with a pinch of salt.
The principal behind the poker bot is an appealing one – a poker robot that doesn’t tilt doesn’t tire and can play poker 24/7. Plug it into your favourite online poker room, point it in the direction of the virtual felt and then head down the pub while it rakes in the money. What could be easier?
Well, the problem with this cunning plan is the fact that most commercial poker bots are rubbish at poker. No-Limit Hold’em is a complex game that is an inexact science at the best of times. The unpredictability of human nature, tilt, bluffing and the luck factor – combined with the fact that you have to program the majority of these bots yourself – means they are far from a guaranteed way of making money online. It is also technically cheating.
Most online poker rooms possess some pretty sophisticated anti-bot security measures, as using bots is against their terms and conditions. Get caught using one and you risk getting your account blocked and funds confiscated. Not exactly the foolproof moneymaking scheme the bot peddlers promise.
Rage Against the Machine
That doesn’t mean that the bots aren’t out there. In fact, some have become quite sophisticated. There have been several high-profile poker bot cases in the last few years. Not all of them were confined to Hold’em either.
Back in 2015, a suspected bot network operating on PokerStars won over $1.5M at the $0.50/$1 and $1/$2 pot-limit Omaha tables. However, using bots is not foolproof. PokerStars proved this by closing the accounts in question and seizing the illegally acquired funds.
The most recent case occurred in August of 2020, where a massive bot network operating across 50 different online sites was discovered. Taking place over several smaller poker networks where security is not as robust, the bots won an estimated $3.5M before being caught.
Spotting A Poker Bot
Poker bots are usually found at online cash tables. That doesn’t mean that you won’t run into one in a multi-table tournament (MTT), it’s just a lot less likely. So what should you look for?
If a player takes exactly the same time over every decision, no matter how simple or complex, then you may be up against a bot. Most bots need a few seconds to process the data available and reach a decision before acting. However, this is far from foolproof as sophisticated bot programmers have worked to fix this obvious ‘tell’. Fortunately, most are lazy and forget to disguise this issue and are caught as a result.
No Response in Chat:
Bots don’t usually utilize the chat function to berate their opponents or interact in any meaningful way. However, some human players, usually pros playing a high volume of tables, also don’t respond in chat. This is not an automatic sign you are up against poker AI. Most poker sites use a captcha to verify whether or not a player is human should suspicions be raised.
Flawless GTO Decision Making:
No player can make Game Theory Optimal play every hand, people make mistakes. This is usually spotted after the fact by those analysing the plays their opponents make, rather than in-game. If you come across a player who is 100% consistent in making the correct decision at every point in a hand, it may be a sign you are up against a bot.
Long Sessions Over Multiple Tables:
While many online professionals play lengthy sessions over multiple tables, everyone takes breaks from time to time. If you come across an opponent playing consistently well over a high volume of tables in a 12-hour time period, they could be a bot.
None of the above are 100% foolproof ways to spot a bot. However, they may be an indication that an opponent might be one. If you suspect an account is a bot, the best way to proceed is to report them to the security team.
While bots do exist and are increasingly becoming more sophisticated, they are not everywhere. If they are found, it is usually operating at the lower-stakes cash tables. The best way to protect against bots is to play on a reputable poker site that polices their games vigorously. Most sites utilise sophisticated detection methods and employ teams of analysts dedicated to search for bots.