Day 52 - Back in the (Test) Saddle Again

Getting the band back together

#

Now, let’s see if I can remember how the tests work. Thank goodness for run_tests.cmd. Leaving aside the WIP connection tests, things seem to be in order.

First we’ll run the non-WIP, non-slow stuff against the normal database:

5 scenarios (5 passed)
25 steps (25 passed)
1m24.402s

And switch to the minimal test database and try again:

5 scenarios (3 failed, 2 passed)
25 steps (3 failed, 8 skipped, 14 passed)
1m55.317s

Trouble? Check the server console:

...
> 6/8/2016 11:42:31 PM: {SystemGo} Starting main game loop.
> 6/8/2016 11:42:53 PM: {Connection} Barbaloot (fe80::6d65:7756:6a1:af6f%17)
> 6/8/2016 11:42:53 PM: NPCs: [1] | Players: [1] | CPU: [??%] | Rnd: [41]
6/8/2016 11:43:06 PM: {SystemWarning} Item.CopyItemFromDictionary(30000) ITEM ID
 does not exist.
> 6/8/2016 11:43:06 PM: {Exception} Exception Data: System.Collections.ListDicti
onaryInternal Source: DragonsSpine TargetSite: Void SetupNewCharacter(DragonsSpi
ne.Character)
> 6/8/2016 11:43:06 PM: {SystemGo} Starting main game loop.
> 6/8/2016 11:43:23 PM: NPCs: [1] | Players: [1] | CPU: [??%] | Rnd: [47]
6/8/2016 11:43:28 PM: {Connection} Barbaloot (fe80::6d65:7756:6a1:af6f%17)
> 6/8/2016 11:43:33 PM: {Connection} Barbaloot (fe80::6d65:7756:6a1:af6f%17)
>
...

Item 30000 is, of course, “Gold Coins”. Without it in the catalog, we’re throwing an exception when generating a new player. Once I resolve that, I get the same problem with item 31000, the spellbook for a new magic-using character. Add that in, and:

5 scenarios (5 passed)
25 steps (25 passed)
1m24.438s

Everyone deserves a fresh start

#

I’ve frequently been an advocate of fully independent tests, making no assumptions about the environment, setting up and tearing down everything needed for the test. This is of particular benefit when it comes time to run tests in parallel, or in isolation to quickly iterate on a problem. So let’s add a hook to give each of these tests a freshly-started server with a clean copy of the minimal database:

 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30

Before('@reset_server') do |scenario|
  # Kill the server if it's already running - TODO support concurrent servers
  result = %x[taskkill /F /T /IM DragSpinExp.exe"]
  puts "Result of taskkill: #{result}"
  # Reset the database
  result = %x[sqlcmd -S "#{ENV['DB_DATASERVER']}" -U "#{ENV['DB_USERNAME']}" -P "#{ENV['DB_PASSWORD']}" -i EntireDB-test02.sql -o EntireDB-test02.out]
  puts "Result of sqlcmd: #{result}"
  # Start up the server
  Dir.chdir("test_env/test02") do
    result = system("start DragSpinExp.exe")
    puts "Result of start: #{result}"
  end
  # Wait for server to come up
  connect_hash = {username: ENV['DB_USERNAME'], password: ENV['DB_PASSWORD'], dataserver: ENV['DB_DATASERVER'], database: ENV['DB_NAME']}
  puts "Before hook connecting to #{ENV['DB_DATASERVER']} database #{ENV['DB_NAME']}..."
  client = TinyTds::Client.new(connect_hash) 
  puts "  Connected." if client.active?
  Timeout::timeout(10) {
      loop do
        puts "Waiting for game server to start."
        result = client.execute("SELECT * FROM [#{ENV['DB_NAME']}].[dbo].[Log] WHERE message LIKE 'Starting main game loop.'")
        rows_affected = result.do
        break if rows_affected > 0
        sleep 1
      end
  }
end

Seems simple: kill the server if it’s running, reset the database (we’ve made changes to that .sql file to forcibly disconnect and remove the DB if it exists), and start up a local copy of the server executable. We wait for the Log table to include the final startup message (we don’t have to worry about history, since we just recreated the DB). And this seems to work fine until we run the tests a second time and try to redirect the output:

K:\301days\Code\seitan-spin>bundle exec cucumber features --tags ~@WIP --tags ~@slow --format html --out cucumber.html --format pretty > cucumber.out
The process cannot access the file because it is being used by another process.

Since we’re using system in the code to start up the server, it’s got a hold onto the STDOUT of the calling process, and therefore a handle on the file it’s been redirected to. But if we use %x or similar, the call is never returning (even though we’re using start). What should we do? We could (and should) add an After hook to kill the server, thereby freeing up its file handles. But there’s another issue: we’re not actually getting the output of the server stored anywhere. Maybe we can find a way to address that as well. Tomorrow.

Day 52 code - tests