Day 39 - And all the children are above average.

( with apologies to Lake Wobegon )

Coding

#

Funny things happen when you try to test random behaviors.

Since we’re already rolling up new characters for new accounts, that seems an easy thing to do some testing on. There are (at least) two ways to test something involving randomness: First, we could lock down the random number generator, giving it a predefined seed and expecting the same results every time. This is the most reliable and thorough path, in that two test runs against the same code should always give the same result, but it is also the most brittle. Any changes that affect the sequence of calls to the RNG will change the results and require changes to the test. Second, we can simply test the range of results. We loop through results until we’ve seen the extremes of the expected range, failing if we ever see anything outside of the range. This is neither reliable nor thorough (we can simply get lucky and pass, even with really broken code), but limits the test’s sensitivity to the actual requirements. The RNG implementation and sequence of calls can change all it wants, as long as the right range of results are returned.

So I chose the second option this time. We’ll keep rolling up characters until we’ve seen the min and max of each stat, making sure we never see anything below the min or above the max. A peek into the server code finds GetRacialBonus tweaking the stats, so we’ll have to do this for each player race.

614
615
616
617
618
619
620
621
622
623
624
625
626

private static int GetRacialBonus(Character ch, string stat)
{
  switch (ch.race)
  {
    case "Illyria":
      if (stat == "stamina" || stat == "wisdom" || stat == "constitution")
        return 1;
      else if (stat == "intelligence")
        return -1;
      return 0;
    case "Mu":
      if (stat == "strength")
        return 2;

The main stats are all 3d6, so testing for the min and max seems straightforward:

40
41
42
43
44
45
46
47
48
49
50
51
52

Scenario: Logging in a new user to check stats - Illyria
Given a new random user
And a telnet connection
When I create an account to verify that race "I" has the following stat constraints:
  | stat         | min  | max |
  | strength     | 3    | 18  |
  | dexterity    | 3    | 18  | 
  | intelligence | 3    | 17  | 
  | wisdom       | 4    | 18  | 
  | constitution | 4    | 18  | 
  | charisma     | 3    | 18  | 
#   | stamina      | 4    | 11  | # race not being considered in the server code
Then I can play the game

(Aside: The GetRacialBonus code includes secondary stats like stamina, but the server code doesn’t ever actually consult it about those stats. Bug?)

But two puzzlers come out of this: First, it takes a really long time to see all of the min values (500+ re-rolls wasn’t uncommon). Second, the instant I check to make sure I’m not exceeding the max value:

Character Stats:
    Strength:     18        Adds:     1
    Dexterity:    11        Adds:     0
    Intelligence: 18
    Wisdom:       14        Hits:    39
    Constitution: 12        Stamina:  6
    Charisma:     10        
    
    Roll Again? (y,n): 
    Stats array: [["Strength", "18"], ["Adds", "1"], ["Dexterity", "11"], ["Adds", "0"], ["Intelligence", "18"], ["Wisdom", "14"], ["Hits", "39"], ["Constitution", "12"], ["Stamina", "6"], ["Charisma", "10"]]
    Stats: {"strength"=>18, "strengthadds"=>1, "dexterity"=>11, "dexterityadds"=>0, "intelligence"=>18, "wisdom"=>14, "hits"=>39, "constitution"=>12, "stamina"=>6, "charisma"=>10}
    | race | stat         | min | max |
    | I    | strength     | 3   | 18  |
    | I    | dexterity    | 3   | 18  |
    | I    | intelligence | 3   | 17  |
    | I    | wisdom       | 4   | 18  |
    | I    | constitution | 4   | 18  |
    | I    | charisma     | 3   | 18  |
    expected: <= 17
       got:    18 (RSpec::Expectations::ExpectationNotMetError)
    ./features/steps/login_steps.rb:246:in `block (2 levels) in <top (required)>'
    ./features/steps/login_steps.rb:244:in `each'
    ./features/steps/login_steps.rb:244:in `/^I create an account to verify that race "([^"]*)" has the following stat constraints:$/'
    features/login.feature:56:in `When I create an account to verify that race "I" has the following stat constraints:'

So I went digging and found this:

596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612

private static int RollStat()
{
  int strRoll1;
  int strRoll2;

  strRoll1 = Rules.dice.Next(3, 21);
  strRoll2 = Rules.dice.Next(3, 21);

  return Math.Max(strRoll1, strRoll2);
}

private static int AdjustMinMaxStat(int stat)
{
  if (stat < 3) stat = 3;
  else if (stat > 18) stat = 18;
  return stat;
}

The starting point isn’t 3d6, it’s the higher of two integers between 3 and 21. No wonder it took so long to see the min values, and no wonder the race modifiers don’t keep it below 18.

Lesson learned: Don’t assume anything about the path from the RNG to the value you’re testing, map it all out first.

Day 39 code - tests