Information Tensor for Universal Poker/ACPC is abstracted even when the game is fullgame

[VitamintK]

I noticed that the information tensor for Universal Poker works when the game is abstracted, but when the game is unabstracted (i.e. full game) the information tensor is still abstracted, and it doesn't differentiate between different bet sizes.

game = pyspiel.load_game('universal_poker', {'bettingAbstraction': 'fullgame', 'numRanks':4, 'numSuits':1})
print(game.num_distinct_actions())
print('   check', game.new_initial_state().child(1).child(2).child(1).information_state_tensor())
print(' bet 400', game.new_initial_state().child(1).child(2).child(400).information_state_tensor())
print(' bet 500', game.new_initial_state().child(1).child(2).child(500).information_state_tensor())
print('bet 1200', game.new_initial_state().child(1).child(2).child(1200).information_state_tensor())

outputs (stars around the relevant part):

1201
   check [0.0, 1.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, **1.0, 0.0,** 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0]
 bet 400 [0.0, 1.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, **0.0, 1.0,** 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0]
 bet 500 [0.0, 1.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, **0.0, 1.0,** 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0]
bet 1200 [0.0, 1.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, **0.0, 1.0,** 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0]

Solutions: (1) properly implement a full infostate tensor for the unabstracted game, (2) print a warning about this issue to stderr when instantiating an unabstracted game or when getting an InformationStateTensor() from one, or (3) add a code comment to universal_poker.h about this issue.

I think (1) would be the best solution and should be feasible. Should tensors in openspiel only contain 0s and 1s? If not, the infostate tensor for poker could be succinctly and elegantly represented with integers for bet sizes.

[jhtschultz]

Thanks for highlighting this. Solution 1 sounds like the way to go. There's no hard rule that the tensors should contain only 0s and 1s, though if possible it's preferred as it's a more natural input representation for neural networks. In the case of no limit poker I agree the most straightforward solution is to include the full bet size.

I think it would make sense to keep the fold, call, raise, all-in 2-bit notation that you have surrounded with **, and then add another entry with the full bet size. WDYT?

[VitamintK]

I think it would make sense to keep the fold, call, raise, all-in 2-bit notation that you have surrounded with **, and then add another entry with the full bet size. WDYT?

That sounds good to me.

While we're on the topic, I also noticed that the InformationStateTensor just lists all actions from all rounds, in order. An alternative, which requires more space, but seems more natural, is to have different sections of the tensor for different rounds. Actually, some parts of the code hint that this was planned at some point:

Line 361:

// Layout of observation:
...
...
  //   NumRounds() round sequence: (max round seq length)*2 bits

Line 424:

  // Move offset up to the next round: 2 bits per move.
  offset += game_->MaxGameLength() * 2;

A notable benefit is that an action at a specific spot in the tensor will always belong to the same player. I would guess that this makes it much easier for neural nets to understand the infostate, so I'd propose switching to that. What do you think?

[jhtschultz]

I agree that this is more interpretable, but how much extra space does this require? If you're playing with 100bb you could imagine a pathological min-raise situation where player1 raises to 2bb, player2 to 3bb, player1 to 4bb etc...

Is there a max raises per round parameter? (It's been a long time since I looked at this poker code)

[VitamintK]

The current implementation requires O(max_actions_per_game) space. The proposed implementation would require O(num_rounds * max_actions_per_game) space. So if there are two betting rounds (preflop and postflop), then this would use 2x the space:

Current implementation:
[ ... card deals and stuff ..., action 1, action 2, ... action n]
Proposed implementation:
[... card deals and stuff ..., r1 action 1, r1 action 2, ...,  r1 action n, r2 action 1, r2 action 2, ..., r2 action n]

where "action i" is the ith action of the game, and where "ri action j" is the jth action of the ith round.

So with the 100bb, the current implementation would require a tensor of length O(100) anyways, and the proposal would require O(200) with 2 betting rounds or O(400) with 4 betting rounds.

If users of the library are in a situation where they need small tensors, they can still edit the OpenSpiel code back to what it is now, or they can take the tensor, remove blank spaces, and stitch it back together.

I'm curious now to see how true my guess is (i.e. "it's easier for neural nets to understand the infostate" ), so I may run some empirical tests.

Is there a max raises per round parameter? (It's been a long time since I looked at this poker code)

Yes, there is. By default it's maxint.

[jhtschultz]

Thanks for clarifying, I agree with all of the above, and I'm also curious to see if it's easier for neural nets to understand the infostate this way. In either case, natural and intuitive structuring of the infostate tensor is important for us humans as it's an easy entry point for subtle bugs to slip in. These changes would be a welcome contribution :)

[lanctot]

Thanks, guys.

Just linking the submitted PR for completion: #1035

[VitamintK]

^ whoops, sorry for the confusion. #1035 isn't the issue discussed here, but contains a couple small related fixes.

[lanctot]

Oops :)

[lanctot]

Hi @VitamintK, just checking: did you ever submit a fix for this?

[VitamintK]

No, not yet 😬 I ran those experiments and have the fix partially done, but haven't gotten around to finishing the PR yet!

[lanctot]

Hi @VitamintK,

Just curious if you still have that code lying around and if you'd still be up for providing a PR fix for this, or if you'd even be willing to send us the relevant parts. It'd be great to finally fix the known issues in the poker implementation.