Day 54 - Handcrafted artisan test code

I’m eager to do all sorts of process improvement (including ditching Octopress), but decided that I really wanted to get a bunch of tests done first. Spoiler: I actually only get one working today.

No early connections

This one seems pretty straightforward:

features/connect.feature GitLab

4
5
6
7


Scenario: Disallow connection if server in Starting mode
Given the server executable is started
When I do not allow time for the server to complete startup
Then I can not connect to the server via telnet

But it does mean splitting out some code. We need to be able to start the server and not wait for it to come up. So we’ll just start the server and make sure the “Starting main game loop.” message is not yet in the log. Then we’ll attempt a telnet connection and make sure it fails.

features/steps/server_steps.rb GitLab

12
13
14
15
16
17
18


Given(/^the server executable is started$/) do
    expect(start_server()).to be_truthy
end

And(/^I do not allow time for the server to complete startup$/) do
    expect(log_contains_immediate('Starting main game loop.')).to be_falsy
end

features/steps/login_steps.rb GitLab

138
139
140


Then(/^I can not connect to the server via telnet$/) do
  expect(create_telnet_connection()).to be_falsy
end

We’ll need the appropriate impatient methods to implement these steps, making sure we can return a reasonable failure indicator:

features/lib/server_helper.rb GitLab

 2
 3
 4
 5
 6
 7
 8
 9
10


def start_server()
  # Start up the server - we don't wait for it to come up in this method
  result = false
  Dir.chdir("test_env/test02") do
    result = Process.spawn("DragSpinExp.exe", [:out, :err]=>["DragSpinExp.log", "w"], :close_others=>true)
    puts "Result of start: #{result}"
  end
  result
end

features/lib/server_helper.rb GitLab

30
31
32
33
34
35
36
37
38
39


def log_contains_immediate(message)
  connect_hash = {username: ENV['DB_USERNAME'], password: ENV['DB_PASSWORD'], dataserver: ENV['DB_DATASERVER'], database: ENV['DB_NAME']}
  puts "Connecting to #{ENV['DB_DATASERVER']} database #{ENV['DB_NAME']}..."
  client = TinyTds::Client.new(connect_hash) 
  puts "  Connected." if client.active?
  puts "Checking log for \"#{message}\"."
  result = client.execute("SELECT * FROM [#{ENV['DB_NAME']}].[dbo].[Log] WHERE message LIKE '#{message}'")
  rows_affected = result.do
  rows_affected > 0
end

features/lib/misc_helper.rb GitLab

 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22


def create_telnet_connection()
    rand_id = rand(36**20).to_s(36)
    puts "Creating telnet connection with ID #{rand_id}"
    begin
      connection = Net::Telnet::new("Host" => ENV['DS_HOST'],
                                  "Port" => 3000,
                                  "Output_log" => "output_#{rand_id}.log",
                                  "Dump_log" => "dump_#{rand_id}.log",
                                  "Prompt" => "Login:",
                                  "Telnetmode" => false,
                                  "Timeout" => 20,
                                  "Waittime" => 1)
    rescue Errno::ECONNREFUSED => e
      puts e.inspect
      connection = nil
    end
    connection
end

And we begin to see how our test library will be expanded and refactored with each new test. The only problem is that this test fails every time. The new minimal database is loaded so quickly that we can never catch it early enough for the telnet to fail.

Slow it down

So we add another step to the test:

features/connect.feature GitLab

5
6
7
8
9


Scenario: Disallow connection if server in Starting mode
Given the production database is in use
When the server executable is started
And I do not allow time for the server to complete startup
Then I can not connect to the server via telnet

features/steps/server_steps.rb GitLab

 3
 4
 5
 6
 7
 8
 9
10


Given(/^the production database is in use$/) do
  # Kill the server if it's already running
  result = %x[taskkill /F /T /IM DragSpinExp.exe"]
  puts "Result of taskkill: #{result}"
  # Reset the database
  result = %x[sqlcmd -S "#{ENV['DB_DATASERVER']}" -U "#{ENV['DB_USERNAME']}" -P "#{ENV['DB_PASSWORD']}" -i EntireDB-production.sql -o EntireDB-production.out]
  puts "Result of sqlcmd: #{result}"
end

With the full production database, there’s plenty of time to catch the server in a “not ready yet” state:

1 scenario (1 passed)
4 steps (4 passed)
6m13.948s

Hooray for passed test, but over six minutes to run? Most of that is populating the production database. We’ll have to find a faster way to handle that.

On demand

Let’s store multiple databases (minimal and production for now), and only fully rebuild them when we need to. After a fair amount of reworking the test code, we have an instance variable @server_database that we use instead of the DB_NAME environment variable. We’ll use Nokogiri to modify the server config XML to point to the correct database:

features/lib/server_helper.rb GitLab

60
61
62
63
64
65
66
67


def set_db_in_config(server_database)
  filename = "test_env/test02/DragSpinExp.exe.config"
  doc = File.open(filename) { |f| Nokogiri::XML(f) }
  sql_connection = doc.at_xpath("//appSettings//add[@key='SQL_CONNECTION']")
  sql_connection['value'] = "User ID='#{ENV['DB_USERNAME']}';Password='#{ENV['DB_PASSWORD']}';" +
    "Initial Catalog='#{server_database}';Data Source='#{ENV['DB_DATASERVER']}';Connect Timeout=15"
  File.write(filename, doc.to_xml)
end

For tests like this one, where we just want one database or the other and don’t care what state it’s in, we have a step and method to select a database and only build it if it doesn’t exist:

features/steps/server_steps.rb GitLab

 3
 4
 5
 6
 7
 8
 9
10


Given(/^I use the production database as-is$/) do
  # Kill the server if it's already running
  result = %x[taskkill /F /T /IM DragSpinExp.exe"]
  puts "Result of taskkill: #{result}"
  @server_database = "production"
  load_db_if_absent(@server_database)
  set_db_in_config(@server_database)
end

features/lib/server_helper.rb GitLab

44
45
46
47
48
49
50
51
52
53
54
55
56
57
58


def load_db_if_absent(server_database)
  connect_hash = {username: ENV['DB_USERNAME'], password: ENV['DB_PASSWORD'], 
    dataserver: ENV['DB_DATASERVER'], database: server_database}
  puts "Connecting to #{ENV['DB_DATASERVER']} database #{server_database}..."
  begin
    client = TinyTds::Client.new(connect_hash)
  rescue TinyTds::Error => e  
    puts e.inspect
    # Reset the database
    result = %x[sqlcmd -S "#{ENV['DB_DATASERVER']}" -U "#{ENV['DB_USERNAME']}" \
       -P "#{ENV['DB_PASSWORD']}" -i EntireDB-#{server_database}.sql \
       -o EntireDB-#{server_database}.out -v MYDATABASE = "#{server_database}"]  
    puts "Result of sqlcmd: #{result}"
  end
end

Now we run our test twice, and see that the second time:

1 scenario (1 passed)
4 steps (4 passed)
0m5.597s

Yay! But there’s still quite a bit of cleanup to do… tomorrow.

More to come

Day 54 code - tests

301 Days (redux)

Another metric leap year of gamedev experiments and such

Day 54 - Handcrafted artisan test code

No early connections

Slow it down

On demand