Midlife Crisis? What Midlife Crisis?

Of course I'm a nerd.

I'm reminded of a conversation I recently had with mrsjb as I stepped out of the door on the way to work.

Mrsjb: Your hair looks too nerdy! *tries to scramble my hair with her hand*
Me: I'm a maths professor, not a member of One Direction. What sort of hair do you expect me to have??
Mrsjb: *falls unexpectedly silent*
Me: *goes to work, unsure whether the silence is the sound of enlightenment or disappointment*

Right sooo....

Can you explain, like you would to a not so bright 10 year old, the difference between fictitious play and CFRM?

Does fictitious play assume an opponent's strategy is stationary, while CFRM has some tricky techniques for predicting an opponents changes to strategy?

All I remember of matlab is trying and failing to put in answers to tests. Answers that were right but matlab marked wrong. fuck matlab.

Originally posted by jules97
Right sooo....

Can you explain, like you would to a not so bright 10 year old, the difference between fictitious play and CFRM?

Does fictitious play assume an opponent's strategy is stationary, while CFRM has some tricky techniques for predicting an opponents changes to strategy?

All I remember of matlab is trying and failing to put in answers to tests. Answers that were right but matlab marked wrong. fuck matlab.

In fictitious play you start with a guesstimate of the equilibrium strategies and play them off against each other, making small adjustments to the strategy that increase each player's EV after each iteration. I don't know whether there's much theory on it, and I'm not sure whether there's any guarantee of convergence (I expect there isn't, but I could easily be wrong).

In regret matching, for example for RPS, you start from any old mixed strategy pair, and pick a strategy for each player. You then calculate the regrets. For example, if player 1 plays rock and player two plays paper, he loses $1, whilst if he'd played paper he'd have lost $0 and for scissors won $1. He therefore regrets not playing scissors by $2 and paper by $1. His regret profile for (R,P,S) is therefore (0,1,2). We do this at every iteration for each player and accumulate all the regrets. Normalise this so that it sums to one and use this as the players' strategies. For example, from (0,1,2) we get the strategy profile (0,1/3,2/3), so that on the next iteration he plays paper 1/3 of the time and scissors 2/3 of the time. If you do this a gazillion times, the average strategy converges to the Nash equilibrium. Guaranteed. Fo sho.

CFRM is regret matching for extensive form games (i.e. games with a game tree). The tricky part is that we have to do regret matching for every information set (i.e. for the AKQ game, you accumulates regrets for player two, who has to decide whether or not to bet A, K or Q , and for player 1, who has to decide whether or not to call with A, K or Q) and if one of the information sets is followed by another, it's not clear how to calculate the regrets because the value of one of its nodes isn't immediately obvious. CFRM gives you a reasonably straightforward way to do this, which basically involves starting at the leaves of the tree and working upwards, calculating values by assuming that the player who's doing the regretting played to get to that information set. It's this bit that's rather confusing. It seems that you can code this up using a recursive function, but I'm struggling to follow the logic at the moment.

From what I've seen in the simple problems that I've done so far, one great feature of CFRM is that components of the equilibrium strategy that are not mixed (e.g. in the AKQ game you should bet an A every time) converge more rapidly than parts that are mixed at equilibrium (e.g. in the AKQ game, how often to bluff with a Q). In Holdem, I'd expect that most of the interesting bits of an equilibrium strategy won't be mixed (e.g. if the equilibrium is to call with some fraction of your best hands, most hands in the calling range will call every time, with some borderline hands calling some fraction of the time).

.....and don't diss MATLAB....Philistine!

Thanks for taking the time to type that out

I've started to read Will Tiptons books, which is a similar sensation to getting hit in the head with a brick. Repeatedly. You would probably like them.
I 'think' early on, about 30 to 40 pages into the first one, he goes over an iteration process similar to what you described of CFRM.
They're called heads up poker books, but that must have been a marketing ploy by the publisher. Maths of poker, the sequel would be more appropriate.

So many things, such limited intellect...

Originally posted by jules97

I've started to read Will Tiptons books, which is a similar sensation to getting hit in the head with a brick. Repeatedly. You would probably like them.
I 'think' early on, about 30 to 40 pages into the first one, he goes over an iteration process similar to what you described of CFRM.

I've read about the first third of that book. That iteration process is actually the basis of fictitious play. Nothing like CFRM at all I'm afraid. ♠_evil:

lol

Perhaps 10 year old was overshooting.

Originally posted by jules97
lol

Perhaps 10 year old was overshooting.

These things take a while to get your head around.

Still making progress with my code. I now have it implemented using a recursive function, which is rather cool, and have also solved a two street game, just to check I know how to do that. At the moment I'm coding up the structure of the game tree by hand, but the next step is to automate this. If I want to allow raises and reraises of, say 1/3, 2/3, 1 and 4/3 times pot at each round, I don't want to be doing it line by line. Once I have that, I'll have a code that can solve, up to the discrete betsizes, the no limit AKQ game.....played in casinos all around the world of course. Then I have to switch to Holdem....sigh.

I started to read your blog from the beginning but then I noticed there is over 400 pages. ♦ So I was forced to skip most of the pages. Great blog though.

Originally posted by NutzAreOk
I started to read your blog from the beginning but then I noticed there is over 400 pages. ♦ So I was forced to skip most of the pages. Great blog though.

That's four years of effort that you're skipping over there. The cheapest cure for insomnia on the market.

I'm off to sit on the stage and clap for an hour at graduation. Fml.

You just make me laugh. ♦

When I graduated I was last up, right after my genius friend getting his Phd. I bowed to everybody I was so pleased, even the audience. I thought they might be appreciative of that.

Will you be the one having fun trying to pronounce the names?

Originally posted by gadget51
You just make me laugh. ♦

When I graduated I was last up, right after my genius friend getting his Phd. I bowed to everybody I was so pleased, even the audience. I thought they might be appreciative of that.

Will you be the one having fun trying to pronounce the names?

Nope. Just clapping. Clapping. More clapping.....

Originally posted by jbpatzer

Originally posted by gadget51
You just make me laugh. ♦

When I graduated I was last up, right after my genius friend getting his Phd. I bowed to everybody I was so pleased, even the audience. I thought they might be appreciative of that.

Will you be the one having fun trying to pronounce the names?

Nope. Just clapping. Clapping. More clapping.....

Try this instead of clapping.