Discount Gambling

TLDR: play 5+5 on the initial hand, but only play 5+0 on any hand with multipliers.

I recently looked into Ultimate-X video poker for the first time, on the off chance it just might be something I’d want to play. My knowledgeable friend told me the published info didn’t seem entirely comprehensive, which made me even more curious.

I looked up the game at the WoO pages, and saw that Mike Shackleford had collaborated with a Prof. Gary Koehler from the University of Florida, who wrote a very detailed academic paper detailing the solution of the game. He identifies it as a Markov Decision Problem, describes the state space, and shows the iterative equations used to arrive at a numerical solution for the optimal return of the game, for various pay tables, multipliers, and multi-line configurations.

My Solution (Transient Analysis)

I thought about the problem, and how I’d solve it, before fully reading the equations in the Koehler paper. I just wanted to set up the iterative equations and see if it converged to the right answer. I saw the problem as starting in the 1x multiplier state, and transitioning to the other {2x, 3x, 4x, 5x, 7x, 11x} states on “wins”, otherwise transitioning to an absorbing state (0x) on a “loss”. Every subsequent win is a transition to the next multiplier state. On average, the the system quickly terminates in the 0x absorbing state, because the probability is high of not winning any given hand.

For example, the diagram below shows a example of a transient path through the Markov state space. The player pays 5+5 coins and starts in the 1x state. In the example, he wins the first hand, and gets a 5x multiplier. He plays another hand with 5+5 coins, etc., until he loses, and path finishes in the absorbing 0x state.

Multiplier
 state

  11x |
   7x |
   5x |             o
      |           /   \
   3x |          /      \
   2x |        /          o
      |      /             \
   1x |     o                \
      |                        \
   0x |                          o----------> absorbing
       +------------------------------------------------> time
           t=0    t=1    t=2    t=3
            1x     5x     2x     0x

It’s straight-forward to compute the optimal EV of a video poker game when you know what the payouts are. In the case of Ultimate-X, you have a current multiplier (m), applied to your payout for winning hand rank (r), plus you get a multiplier m’ for your next hand based on your current winner. So, if you knew what the expected value is for every multiplier, you can construct the equation(s):

where H are all possible dealt hands, R are the winning hand ranks, i is one of the 32 possible ways to play the hand, and m'(r) is the multiplier awarded on the next hand for winning rank r.

Note my equations take into account the absorbing 0x terminal state, because R is only the set of winners, with multipliers >> 1. There’s no return to the 1x state. Note that, V(m) is interpreted as the total net return for an initial 2 unit bet starting in state m, until the all the possible paths converge to the 0x state. This is a transient analysis, and very simple to describe in the equations. The -2 in the equation represents the cost of playing 2 units for the next hand, assuming it’s still correct to pay for the possibile multipliers given your awarded m (which may be as high as 11x!). We’ll see if this is a correct assumption.

Single-Line Jacks-or-Better

Thanks to the Koehler paper, I used the numerical iteration process he described that simply calculates the next set of V(m) values for each multiplier using the current V(m) values, then iteration again, until the values converge. I initialized the V(m) values each with (m*ev0 + 1), where ev0 is the optimal return of the base game w/o multipliers, and the +1 is for the extra cost for a chance at next multipliers.

Table 1 below shows the results for a 9/6 Jacks or Better Single-Line Ultimate-X game with 2/2/2/12/10/7/4/3/2/1 multipliers. This game is actually available nearby me, and the high ev0 = 0.995439 inspired me to do this analysis. It intuitively feels like I should consider playing just 5+0 credits on a hand that already has a high 12x multiplier.

m	V(m)	2*ev0 + 1	Should pay for multipliers?
12	12.927089	12.945269	Better to play 5+0
10	10.936397	10.954390	Better to play 5+0
7	7.950974	7.968073	Better to play 5+0
4	4.967557	4.981756	Better to play 5+0
3	3.973817	3.986317	Better to play 5+0
2	2.980889	2.990878	Better to play 5+0
1	1.989899	1.995439	Better to play 5+0

Table 1. Iterative results for my Markov equations assuming 5+5 play for all hands (Jacks-or-Better 9/6).

The results show a per unit EV = 1.989899/2 = 0.99495. This is actually lower than the base 9/6 game return of 0.995439. The reason for this underperformance becomes clear when we compare the V(m) [the expected return on a 2 unit bet] vs. just playing a 1 unit bet (5+0) normal game (= 2*ev0 + 1). The table shows you’re better off playing just 5+0 when you have a multiplier.

The optimal return for the game then becomes V(1) when setting V(m) = 2*ev0 + 1 for all m >> 1. In other words, the analysis is straight forward. We found that paying for the multipliers on winning hands is sub-optimal. Just finish out the hand with a standard 5+0 bet. The EV calculation then becomes straight-forward, setting the equations with these m*ev0 values.

Setting the V(m) values to these constants for m >> 1 then yields V(1) = 1.99515. This yields an EV = 1.99515/2 = 0.997576 per unit bet, and improvement over the base game. Thus, it is worth paying the extra +5 coin in for the chance at the multipliers on the initial hand, but it’s not worth paying the extra +5 coins once you have a multiplier.

“Game” vs “Hand”

A quick note here the interpretation of the return of 1.99515 on a “game” which costs 2 units to play. A ‘game’ either lasts 1 or 2 hands. When you win the 5+5 initial hand, the cost of the 2nd hand is already included in the EV calculation (recall the -2 term in the equations). When you lose the 1st hand, there’s no 2nd hand.

The Koehler Equations (Steady-State Analysis)

In the equations presented by Koehler in his paper, he assumed that a player that would continue playing with optimal decisions for a 2 unit (5+5) bet indefinitely (non-discounted). In his equations, a game is always one hand. He computes the steady state per-hand optimal return (g), then subtracts it off from every V(m), which are then interpreted as a “relative bias” for state m. In his equations, he solves for the steady-state probabilities of being in state m, Pr(m). He then computes g as the average of e(m) = (V(m) + g) over all multiplier states.

I implemented his equations and came up with the exact same results he gets for various pay tables and multipliers.

Implementing his equations, my program converged up with a return of 1.994316/2 = 0.9972 for the same single-line 9/6 Jacks-or-Better game with the multipliers listed above. This is a lower number than my 0.9976 return. Plus, with my ‘only play 5+0 on multiplier hands’ strategy, the ‘cost per click’ is lower. (A simple calculation based on the 68.33% no-win rate reveals a (0.00368 unit)/deal cost for my strategy, vs. a (0.00568 unit)/deal cost for betting 5+5 every hand.)

deals/game =
2*(1 - pr(lose)) + 1*pr(lose) =
2 - pr(lose) =
2 - 0.68333 =
1.31667

and cost per click = (1-1.995152) units-per-game / (1.31667 clicks-per-game) = 0.00368 units/click. (Per click of the “deal” button.)

We Get Identical Optimal Decisions

I compared the optimal decision points for every multiplier state for a various games, pay tables, and multipliers, and found that they were always identical for all possible hands.

So, you can play the same 1x (5+5) strategy you’ve always played; it doesn’t change. It’s simply lower cost to play 5+0 on all multiplier hands. Clearly, this is the case in single-line.

Note that playing perfectly becomes much simpler with the ‘only bet 5+0 on multiplier hands’ strategy. You only need to know 1x 5+5 strategy (doesn’t change), and basic strategy for multiplier hands (widely published). You do not need to know a separate strategy for 12x, 10x, 7x, 4x, 3x, 2x, etc. multiplier hands.

Below is my 1x (5+5) strategy for 9/6 Jacks-or-Better with 2/2/2/12/10/7/4/3/2/1 multipliers. It should be very similar to any other correct published strategy for this Ultimate-X game.

Royal Flush          800   2x
Straight Flush        50   2x
Four of a Kind        25   2x
Full House             9  12x
Flush                  6  10x
Straight               4   7x
Three of a Kind        3   4x
Two Pair               2   3x
Jacks or Better        1   2x
Nothing                0   1x


                                            Costs: Rule      Cumulative
                                  Hold Royal Flush, 0.000000, 0.000000
                               Hold Straight Flush, 0.000000, 0.000000
                                 Hold all w/ quads, 0.000000, 0.000000
                                   Hold Full House, 0.000000, 0.000000
                             Hold four Royal cards, 0.000000, 0.000000
                                        Hold flush, 0.000000, 0.000000
                                     Hold straight, 0.000000, 0.000000
               Hold open-ended straight flush draw, 0.000000, 0.000000
                  Hold gutshot straight flush draw, 0.000000, 0.000000
                                        Hold trips, 0.000000, 0.000000
                                    Hold two pairs, 0.000000, 0.000000
                     Hold 4 suited w/ 3 high cards, 0.000000, 0.000000
                     Hold 4 suited w/ 2 high cards, 0.000000, 0.000000
                      Hold pair of Jacks or better, 0.000000, 0.000000
                     Hold 4 suited w/ 1 high cards, 0.000000, 0.000000
                               Hold 4 suited cards, 0.000000, 0.000000
                     Hold 3 suited cards w/ 3 high, 0.000000, 0.000000
                            Hold three Royal cards, 0.000000, 0.000000
                     Hold open-ended straight draw, 0.000000, 0.000000
                                    Hold one pairs, 0.000000, 0.000000
                      Hold three SF cards w/ 0 gap, 0.000000, 0.000000
                  Hold 3 SF draw with 2 high cards, 0.000001, 0.000001
               Hold gutshot draw with 4 high cards, 0.000000, 0.000001
               Hold gutshot draw with 3 high cards, 0.000005, 0.000006
                     Hold 3 suited cards w/ 2 high, 0.000000, 0.000006
                                    Hold QJ-suited, 0.000000, 0.000006
                                    Hold KJ-suited, 0.000000, 0.000006
                                    Hold AJ-suited, 0.000000, 0.000006
                                    Hold KQ-suited, 0.000000, 0.000006
                  Hold 3 SF draw with 1 high cards, 0.000000, 0.000006
                                  Hold KQJ-offsuit, 0.000000, 0.000006
                         Hold two high Royal cards, 0.000000, 0.000006
                      Hold three SF cards w/ 1 gap, 0.000000, 0.000006
               Hold gutshot draw with 2 high cards, 0.000000, 0.000006
                      Hold three SF cards w/ 2 gap, 0.000040, 0.000045
                                   Hold QJ-offsuit, 0.000111, 0.000156
                     Hold 3 suited cards w/ 1 high, 0.000000, 0.000156
                                   Hold KQ-offsuit, 0.000001, 0.000157
                                    Hold JT-suited, 0.000002, 0.000159
                                   Hold KJ-offsuit, 0.000001, 0.000161
                                    Hold QT-suited, 0.000006, 0.000167
                Hold gutshot draw with 1 high card, 0.000013, 0.000180
                                   Hold AK-offsuit, 0.000000, 0.000180
                                   Hold AQ-offsuit, 0.000000, 0.000180
                                   Hold AJ-offsuit, 0.000000, 0.000180
                                          Hold Ace, 0.000000, 0.000180
                              Hold two Royal cards, 0.000001, 0.000181
                                         Hold Jack, 0.000011, 0.000192
                                        Hold Queen, 0.000012, 0.000204
                                         Hold King, 0.000013, 0.000216
                        Hold gutshot straight draw, 0.000015, 0.000231
                               Hold 3 suited cards, 0.000000, 0.000231
                                       Discard all, 0.000001, 0.000232

Table 2. My 1x (5+5) strategy for 9/6 JoB with 2/2/2/12/10/7/4/3/2/1 multipliers.

Multi-Line

While the advantages to ‘don’t pay for multipliers when you already have one’ strategy is fairly clear in a single-line game, it’s less clear for multi-line. The intuition is e.g., with 10-play, and you end up with just a few, low multipliers, might it be ok to play 5+5 anyway? We examine the question quantitatively here.

Ultimate-X is usually found with 3-, 5-, and 10- play multi-line versions. I understand single-line is relatively rare. For L-line multi-line games, the possible multiplier states of the Markov problem are all the combinations possible from L- independent multiplier outcomes. Shackleford noted the multiplier space can be reduced from their specific combination frequencies to just their sum total, making the computation much more efficient.

To examine the cost of L-multiline with the simple (error-free) strategy of betting only 5+0 for any next hand with multipliers, we look at the definition of ‘game’ in the single-line analysis. There the calculated EV was for a game consisting of either 1-hand (5+5 and no winner), or 2-hands (5+5 and 5+0 with multiplier). In a multi-line setting, you *have* to play L-hands even if you only have winners where (w<L). So, you’re paying to play an additional (L-w) hands at 1 unit (5+0). You need to add this (L-w)(1 – ev0) to the cost of the multi-line game.

The table below shows the computation of the average number of multiplier hands for optimal play of the 9/6 Jacks-or-Better game with the above multipliers.

3-lines:
Pr( 0 multipliers) =   0.297471
Pr( 1 multipliers) =   0.334309
Pr( 2 multipliers) =   0.142562
Pr( 3 multipliers) =   0.225658

average multipliers:  1.30/3

5-lines:
Pr( 0 multipliers) =   0.162587
Pr( 1 multipliers) =   0.280643
Pr( 2 multipliers) =   0.226265
Pr( 3 multipliers) =   0.100548
Pr( 4 multipliers) =   0.023922
Pr( 5 multipliers) =   0.206035

average multipliers:  2.16/5

10-lines:
Pr( 0 multipliers) =   0.043801
Pr( 1 multipliers) =   0.118474
Pr( 2 multipliers) =   0.180563
Pr( 3 multipliers) =   0.188652
Pr( 4 multipliers) =   0.142351
Pr( 5 multipliers) =   0.078935
Pr( 6 multipliers) =   0.032107
Pr( 7 multipliers) =   0.009374
Pr( 8 multipliers) =   0.001868
Pr( 9 multipliers) =   0.000228
Pr(10 multipliers) =   0.203647

average multipliers:  4.32/10

Table 3. Average number of multiplier hands received for optimal 5+5 play for 1x hand (9/6 Jacks-or-Better, 2/2/2/12/10/7/4/3/2/1 multipliers)

Thus, there’s an average cost for extra “completion hands” you have to pay for in multi-line not included in the single-line analysis. Estimating the average unit cost for a 10-line 9/6 Jacks-or-Better game using the average number of completion hands yields:

cost per 10-line game =
base games + completion hands =
[ (10)(2-1.99515) +  (10 - 4.32)(1-0.995439) ] units

= 0.00372 units per hand per deal 

which is comparable to a (1 – 0.00372) = 0.9963 return on a per deal (per “click”) basis. This number is very close to the cost/click computed for single-line.

The problem with playing multi-line at 5+5 for every hand is that the Koehler analysis uses the optimal decision for every multi-line multiplier state (simplified to the sum of the multipliers). I doubt that anyone plays the correct 10-line strategy for the various combination of multiplier outcomes.

Of course while the average “completion hands” cost is low, there are cases where you think it might be ok to play 5+5 on all 10-lines when only one has a 2x multiplier. You can look at these potentially valuable corner-cases by adding up the costs of misplaying the wrong multiplier strategy for the multiplier hands and the completion hands. You can pick whatever strategy minimizes the total. I’d guess it’s a can of worms, but here’s some tables that can give you an idea of where to start.

                  Applied Strategy Multiplier

Actual m       1x       2x       3x       4x       7x      10x      12x
------------------------------------------------------------------------
   1x          0      0.0017   0.0054   0.0098   0.0252   0.0426   0.0546
   2x       0.0013      0      0.0006   0.0019   0.0082   0.0165   0.0224
   3x       0.0029    0.0005      0     0.0003   0.0033   0.0083   0.0121
   4x       0.0042    0.0013   0.0003      0     0.0014   0.0048   0.0075
   7x       0.0077    0.0039   0.0021   0.0010      0     0.0010   0.0020
  10x       0.0110    0.0068   0.0044   0.0029   0.0005      0     0.0001
  12x       0.0115    0.0072   0.0049   0.0033   0.0006   0.0000      0

cost of playing baseline (5+0) strategy instead of 1x (5+5) strategy: 0.0141854
cost of playing 1x (5+5) strategy instead of optimal (5+0) strategy on multiplier hand (per unit x): 0.00609375

Table 4. Average cost of mis-applying the wrong multiplier strategy to a hand with a given actual multiplier (9/6 Jacks-or-Better, 2/2/2/12/10/7/4/3/2/1 multipliers)

Conclusions and Interpretations

This work is an extension of the results published by Shackleford and Koehler. It probably answers the question that others already had considered: what if I didn’t pay the extra +5 coin in when I already had a multiplier? (Especially a large multiplier on a single-line game.) I set up my equations in a different way, that represented a more direct, transient analysis instead of the steady-state (non-discounted) problem that Koehler analyzed. The converged numbers showed that playing 5+5 on multiplier hands was slightly non-optimal. The resulting solution was a direct substitution of the fixed values (m*ev0 + 1) into my equations.

The play costs less on a per click (click of the ‘deal’ button) basis, which is probably a fair way to compare the two approaches. In multi-line, the “only play 5+0 when you have any multipliers” is not optimal, but the average cost is low. The rate of errors approaches zero, because it’s easy to learn and apply a single 1x 5+5 strategy vs. trying to play correctly for every possible sum of multipliers. So in addition to being quantitatively better on a ‘cost per click’ basis, the actual cost in practice will be significantly lower for the 5+5 / 5+0 strategy, because the applied strategy error rate is much lower.

Let me know what you think. Have you thought of this before? Do you already play like this? Is there something I not comparing correctly? (Yes I know people don’t really want to hit ‘Max Bet’ half the time, and the other half hit ‘Bet One’ five times.)