Poker robot knows *exactly* when to hold ‘em, fold ‘em

Researchers at the University of Alberta have built a robot that has 'solved' the game of heads up limit Texas hold'em poker. Photo credit: Michael Bowling

Researchers at the University of Alberta have built a robot that has ‘solved’ the game of heads up limit Texas hold’em poker. Photo credit: Michael Bowling

Last month, Stephen Hawking caused quite a stir when he mused that advances in artificial intelligence “could spell the end of the human race.” Computer scientists quickly shot back, pointing out that today’s algorithms still struggle to recognize kittens, never mind plotting our ultimate doom. Still, with programs like Deep Blue and Watson outplaying humans on a regular basis, you might be forgiven a certain uneasy sense that the machines are gaining on us. Now, researchers at the University of Alberta have built an unbeatable card shark, a computer program that has ‘solved’ the game of heads up limit Texas hold’em poker. But there’s no need to run for the cellar just yet: while the finding has important implications for online poker fans and even medical decision makers, it’s not exactly Skynet.

Michael Bowling of the U of A’s Computer Poker Research Group led the project. “Many of the decisions we have to make in day-to-day life are filled with uncertainty: we’re often missing critical information, or we don’t know what others around us are going to do,” says Bowling. For example, a doctor trying to prescribe a treatment regimen for diabetes has to deal with the fact that each patient responds differently based on their diet, medical history, level of exercise, genetics and a host of other factors. Poker can act as a kind of test bed for developing artificial intelligence that can handle the presence of uncertainty. The idea is that by analyzing past decisions and outcomes, computer programs could predict what actions are most likely to be successful in the future.

Several years ago, the team made a splash when one of their algorithms, codenamed Polaris, won several games against top international poker players (WIRED ran a neat comic book-style story, which you can read here.) In a paper published this week in Science, they go a step further. “This program plays perfectly,” says  Bowling. “Even if it told the other player exactly what it was going to do and why, the opponent still wouldn’t be able to win money over the long term.” The program — called Cepheus — has analysed every single one of the billions and billions of situations it could find itself in, and has worked out the best possible decision to make for each one. It may not win every single hand, but averaged out over time, it’s unbeatable.

 

Principal investigator Michael Bowling and poses in front of a screen displaying output from Cepheus. Photo credit: University of Alberta. Photo credit: John Ulan, University of Alberta

Principal investigator Michael Bowling and poses in front of a screen displaying output from Cepheus. Photo credit: University of Alberta. Photo credit: John Ulan, University of Alberta

How did it get so good? Practice, practice, practice. Over the course of about two months, Cepheus played countless games against itself, eventually racking up something on the order of 8 to 10 billion, billion hands of poker. “It’s played more hands of poker than all of humanity has ever played in history,” says Bowling. At first, it was just playing randomly, but after each round it did a bit of self-reflection, calculating how much money it could have won if it had chosen to play differently, for example, to bet instead of call at a certain point. That monetary amount — the technical term is ‘regret’ — was used to inform the strategy it would use next time. Over time, the regrets got smaller and smaller until eventually, they reached zero, or at least as close to zero as it is reasonably possible to get. At this point, the game is considered to be ‘solved.’

This is not the first time a computer has solved a human game. For example, back in 2007 Jonathan Schaeffer, Bowling’s colleague at the University of Alberta, solved checkers with a computer called Chinook. Many people speculate that one day chess could also be solved. But what Cepheus has done is fundamentally different because of the uncertainty factor. While both checkers and chess are ‘bigger’ games than poker — there are a larger number of possible situations a player can face — they are perfect information games, meaning that each previous move is laid out for all to see. Cepheus, on the other hand, doesn’t know what cards its opponent has received, and thus is the first program to have solved an ‘imperfect information’ game.

You might think that an unbeatable poker robot would be a useful thing to own; set loose on the internet, it could collect earnings while its owner slept, ensuring an early retirement. Sadly (or fortunately, depending on your point of view) that probably won’t work. First of all, Cepheus only plays a very specific type of poker: heads up limit hold ‘em, which is exclusively a two-player game. Any change to the game — say adding a third player or getting rid of the betting limit — changes the situation so much so that Cepheus’s strategy falls apart. And head up limit hold ‘em, while popular in the early 2000s, is no longer widely played online, partly due to the fact that many players simply got too good at it. After Polaris started beating human players in 2008, Bowling and his team put up a website where people could improve their skills by sparring against the machine. It’s hard to say how much impact this had, but it does seem that over the next few years, the number of strong players increased and the number of weak players dwindled. In a stalemate, both players actually lose (since the online casinos take a certain percentage) so today most people don’t play unless they truly believe they have an advantage.

Is it possible that someone could one day build a machine that could ‘solve’ a more complicated form of poker?  Perhaps, but it’s unlikely. “We’ve done work in more than two-player games, but we’re probably only at a good amateur level,” says Bowling. Part of the problem is that the sheer number of possible situations becomes unmanageably large, not to mention the exponentially higher levels of uncertainty. “Solving a game like that is such a huge question . . . it’s unclear what it would even mean to do that,” says Bowling. As for the suggestion that artificial intelligence is something to fear, Bowling is skeptical. “AI is simply a tool, and every tool we’ve had in history has had positive and negative uses,” he says. “Of course we need to use these tools in wise ways. But so much good can come from this, it would be a tragedy not to explore it.”

Want to try your luck in a game of heads up limit Texas hold’em against Cepheus? Click here to try it out.

 

Secrets of the naked mole rat

 

In addition to their other talents, naked mole rats can survive under very low oxygen, and may offer clues that could help humans survive damage from strokes. (Photo credit: Roman Klementschitz, via WikiMedia Commons)

In addition to their other talents, naked mole rats can survive under very low oxygen, and may offer clues that could help humans survive damage from strokes. (Photo credit: Roman Klementschitz, via WikiMedia Commons)

Naked mole rats — is there anything they can’t do? These wrinkly little critters live up to 30 years, more than ten times as long as other rodents their size. They are essentially immune to cancer (a fact which makes them of great interest to the medical community) and also apparently insensitive to some kinds of pain. Last, but certainly not least, they are able to survive at levels of oxygen so low that they would be fatal to other animals. This week, researchers at the University of British Columbia showed that naked mole rats do this in a way that is completely unique among mammals. The insight may one day lead to new ways to prevent or treat human conditions like stroke.

Read the rest of this entry »

Check email less, reduce your stress?

 

Are you an e-mail addict?

I know I am. Every time I pick up my phone the blinking envelope in the corner reminds me that there’s something new to deal with. And while hope springs eternal that it’s a note from an editor assigning me a juicy new story, as often as not it’s just a poorly-spelled forward from an elderly relative, making bad jokes about how life was so much better before all this gosh-darned technology. Was it really? If I stop checking my email so often, will I regain the peace and comfort of that bygone era? According to the results of an experiment published this week by researchers from the University of British Columbia, the answer might well be yes.

Read the rest of this entry »

A Canadian message to the stars

ARO Dish

Later this fall, Algonquin Radio Observatory in Algonquin Park will be used to send a message to two stars that appear to have earth-like planets. (Photo credit: Keith Vanderlinde, Dunlap Institute for Astronomy & Astrophysics)

If you could talk to an extraterrestrial civilization, what would you say? That’s the question being asked, in all seriousness, by a group of scientists and science enthusiasts at the University of Toronto. As part of the newly organized Toronto Science Festival (TSF) they are asking the general public to submit potential messages by e-mail, Twitter, or online video. Suggestions are already trickling in, and within a few weeks the winner will be chosen and actually broadcast into space by the Algonquin Radio Observatory, a 46-metre-wide dish perched on a lonely outcrop of the Canadian Shield in Ontario’s Algonquin Provincial Park. A publicity stunt, you say? Perhaps, but it’s also a rare chance to re-connect with an important piece of Canada’s scientific heritage, and an opportunity to do some of the Big Thinking that is always needed to drive scientific inquiry forward.

Read the rest of this entry »

In the skin of a . . . hadrosaur?

Image credit: Canadian Light Source

Were dinosaurs dull green and grey like today’s large reptiles, or bright and flashy like their descendants, the birds? For a long time this was considered an unanswerable question, but that may soon change due to a singularly well-preserved sample of skin from a hadrosaur — a duck-billed dinosaur from the late Cretaceous — found near Grand Prairie, Alberta last summer. That sample is currently undergoing analysis at the Canadian Light Source (CLS), a particle accelerator based in Saskatoon, Saskatchewan. It turns out that the ultra-modern discipline of particle physics may be just the way to shed light — literally — on a hundred million-year-old mystery.

Read the rest of this entry »

The Call of Cthulhu

Cthulhu

Cthulhu emerges from the mythical city of R’lyeh. Credit: BenduKiwi via Wikimedia Commons

Erick James spends his days investigating the contents of termite intestines, a line of work that you’d think would relegate him to obscurity. But thanks to a bit of clever marketing, James, who works in the biology lab of Patrick Keeling at the University of British Columbia, has garnered attention from countless blogs and even some major newspapers.  His latest discovery — a microscopic organism that helps its termite hosts digest their woody meals — is interesting on its own, and could have implications for industries like biofuel.  But what has attracted all the attention is the fact that James named his pet organism after a fictional monster that has become a powerful internet meme.

Read the rest of this entry »

Cannabalistic Stars – Explaining Luminous Red Novae

V838 Monocerotis

This outburst from the red giant star V838 Monocerotis – captured by the Hubble Space Telescope in 2002 – is an example of a ‘luminous red nova’. New research shows that such phenomena may be the result of common envelope events (CEEs) in which material tranfers unstably from one star to another. Photo credit: NASA/ESA and H.E. Bond (Space Telescope Science Institute), public domain.

If a star began to eat another star, what would it look like? Natasha Ivanova can tell you better than almost anyone; as the Canada Research Chair in Astronomy and Astrophysics at the University of Alberta, pondering questions like this is her full-time job. Last week, Ivanova and her colleagues published a paper in Science which showed that a certain type of stellar cannibalism previously thought to be invisible can be observed from Earth after all. The finding may explain the existence of the recently-discovered phenomena known as luminous red novae.

Read the rest of this entry »

Are we ready for the paper computer?

PaperTabs

PaperTabs, a new prototype developed by the Human Media Lab at Queen’s University, aims to bring the advantages of paper into the computing environment. Photo credit: Queen’s University

If you’re reading this on a computer, take a moment and look around your monitor or laptop screen. Do you see any stray pieces of paper, such as articles with highlighted passages, notes to yourself, or contact details to follow up with later? Me too. It seems that even after more than thirty years of widespread use, personal computers still haven’t completely replaced paper as a way of displaying and manipulating information. So can they? Should they? Members of the Human Media Laboratory (HML) at Queen’s University have been asking themselves this question for almost a decade. Last week at the Consumer Electronics Show, they unveiled a new prototype – dubbed PaperTabs– which they hope will provide part of the answer.

Read the rest of this entry »

A Sea Monster Moves Inland

Tylosaurus, the biggest of the mosasaurs. New evidence shows that some mosasaurs adapted to freshwater environments. Photo credit: Dmitry Bogdanov [Public domain], via Wikimedia Commons

Of all the terrifying things that have ever lived in the ocean, mosasaurs hold a special place in my heart. Admittedly, the similarity between the name of the biggest mosasaur species – Tylosaurus – and my own may play a small role. But mostly it’s because they are the original sea monsters. Although more closely related to snakes and iguanas than dinosaurs, these giant aquatic lizards had huge heads full of terrifying teeth that would put Tyrannosaurus rex to shame. (Literally: when it comes to skull size, mosasaurs easily outrank any land-based carnivore you care to name.) And if ruling the oceans weren’t enough, this week scientists revealed that at least one species of mosasaur had adapted itself for inland river systems as well.

Read the rest of this entry »

Are Impact Factors Losing Ground?

New Canadian research shows that the highest-impact papers are increasingly being published outside the highest-impact journals.  Photo credit: rodrigovco via stock.xchng

New Canadian research shows that the highest-impact papers are increasingly being published outside the highest-impact journals. Photo credit: rodrigovco via stock.xchng

“Congratulations on the new paper! By the way, what’s the impact factor on that journal?” Scientists get this question more often than they would care to mention. Despite numerous critiques since it was first developed in the 1960s, today the impact factor remains the gold standard for judging the reputation of a given scientific journal, and is often used in funding decisions, in some cases even to calculate scientist’s salaries. But according to new research from Université de Montréal, information technology is slowly rendering the impact factor irrelevant. Increasingly, the highest-impact papers are being published outside the highest-impact journals.

Read the rest of this entry »

Older posts «