I will call you by your dwarf names. -Dr. Cristina Yang

Something new to try: feeding old ascii MMO maps to a gopher.

Inspiration: Optimizing Initial Inputs

Kirby eating.

Once we start demanding test flexibility and stability via clean starts, we inevitably begin to obsess about startup time. I’ve done a few easy bits in the last few posts, but now it’s time to go in a slightly different direction.

One of the game server’s startup tasks is reading in the world maps. A collection of ascii files (still very similar to the original hand-assembled maps that players compiled in the 1980s) contain the geography of the various towns and dungeons and such. For most of the testing, I replaced these with very simple test maps, but those would need to expand in order to test more of the game logic. A map pre-processing step could be a useful optimization and, perhaps more importantly, fun to do.

But First, a Bit of Cleanup

Ok. But Clean First.

Let’s make these map files a little more consistent. Might as well start alphabetically, with Annwn:

DragonsSpine/maps/Annwn.txt GitLab

1
2
3
4
5
6
7


//Doorway into Ydmos' tower is 1 way entrance
//Teleport in SW corner of -70 goes to NW part of surface Morrigans Island
//Teleport in SE corner of -70 goes to SE part of surface Annwens Glade
<z>0</z><x>0</x> <y>9</y> <f>heavy</f> <n>Annwn Surface</n> outdoor=true
                          /\/\/\/\/\/\/\/\/\/\/\/\/\
              /\/\/\/\/\/\/\~r~r~r~r~r~r~r~r~r~r~r/\/\
            /\/\~r~r~r~r~r~r~r~r~r~r~r~r~r~r~r~r~r~r/\

Some comment lines, then a header (XML-style tags, followed by key-value pairs), then the actual map cells (two characters each). Repeat for each level of the map…

DragonsSpine/maps/Annwn.txt GitLab

    /\. pb/\. . . . . | . . . /\/\/\/\
    /\/\/\/\/\/\/\/\/\/\/\/\/\/\
<z>-25</z> <x>38</x> <y>18</y> <n>Ruined Tavern Cellar</n>
/\/\/\/\/\/\/\/\/\/\/\/\/\
/\[][][][][][][][][][][]/\
/\[]dn. []. . . []. up[]/\

The comments are optional (makes sense), as seem to be the <f> tag and the key-value pairs.

The z, x, and y tags make sense as the z-level and offsets. What’re f and n?

DragonsSpine/World/Map.cs GitLab

857
858
859


  // forestry level
  if (s.IndexOf("<f>") != -1 && s.IndexOf("</f>") != -1)
      forestry = s.Substring(s.IndexOf("<f>") + 3, s.IndexOf("</f>") - (s.IndexOf("<f>") + 3));

DragonsSpine/World/Map.cs GitLab

876
877
878
879
880
881


  // name of the z plane
  if (s.IndexOf("<n>") != -1 && s.IndexOf("</n>") != -1)
  {
      if(!zNames.ContainsKey(z))
          zNames.Add(z, s.Substring(s.IndexOf("<n>") + 3, s.IndexOf("</n>") - (s.IndexOf("<n>") + 3)));
  }

A forestry level (which only makes sense sometimes) and a name for the z-level. The key-value pairs are clearly booleans which apply to that whole z-level, and are false if not present.

To tidy things up, we’ll make all tags except f mandatory, in the order that’s most common, and order the key-value pairs alphabetically when present.

Why so strict? Not just for efficiency. If we want to do some simple round-trip testing of map parsing code, having the ability to output a format which exactly matches the input is very helpful.

Why not more strict? Nostalgic attachment to these old ascii files, so we’re trying not to alter them too much.

Ambiguous Whoopsie

Depends on how you look at it.

Tweaking the files works fine for a while, adding some zero offsets and naming z-levels after their z coordinate where tags were missing. Then I come across the map file for Torii:

DragonsSpine/maps/Torii.txt GitLab

1
2
3
4


<z>0</z> <x>29</x> <y>36</y> // Torii Town - Surface Level 0'                                                                                                                                                                                       /\/\/\WWWW
        /\/\/\/\/\/\/\/\                  /\/\/\/\/\/\/\/\/\              /\/\/\/\/\/\          /\/\/\/\/\                /\/\/\                                            /\/\/\/\/\/\/\/\        /\/\/\/\/\/\WWWWWWWW
        /\. /\/\~~~~~~/\/\/\/\  /\/\/\/\/\/\. . . . . . . /\/\/\/\/\/\/\/\/\{}upup{}/\/\        /\{}{}{}/\/\/\/\          /\dn/\                                    /\/\/\/\/\{}{}{}{}{}{}/\/\/\/\/\/\~~~~~~~~WWWWWWWWWW
    /\/\/\CC/\/\~~{}{}~~~~~~/\/\/\~~~~~~~~~~~~~~~~~~. . . . . . . . . . . . . . . . . /\    /\/\/\""""""""{}{}/\          /\. /\                /\/\/\/\/\/\/\/\  /\/\{}{}{}.\.\.\.\.\{}{}{}{}{}{}~~~~~~~~~~~~WWWWWWWWWW

Which looks fine until you scroll that top line all the way to the right.

A rogue section in the Torii map file.

Not sure when that happened; it dates back to the first commit in the repo, but I could easily have messed it up at some point prior to that. But if I fix it, will that y-offset still be correct? Or has it been incorrect all this time? Let’s just shift that line down and leave the offset alone, start up the server, and see what happens.

06/28/2023 20:07:59: {SystemWarning} SpawnZone 321 did not find a suitable spawning point after 40 attempts.
06/28/2023 20:07:59: {SystemWarning} SpawnZone 323 did not find a suitable spawning point after 40 attempts.
06/28/2023 20:07:59: {SystemWarning} SpawnZone 332 did not find a suitable spawning point after 40 attempts.
06/28/2023 20:07:59: {SystemWarning} SpawnZone 333 did not find a suitable spawning point after 40 attempts.
06/28/2023 20:07:59: {SystemWarning} SpawnZone 334 did not find a suitable spawning point after 40 attempts.
06/28/2023 20:07:59: {SystemWarning} SpawnZone 335 did not find a suitable spawning point after 40 attempts.
06/28/2023 20:07:59: {SystemWarning} SpawnZone 336 did not find a suitable spawning point after 40 attempts.
06/28/2023 20:07:59: {SystemWarning} SpawnZone 337 did not find a suitable spawning point after 40 attempts.
06/28/2023 20:07:59: {SystemWarning} SpawnZone 338 did not find a suitable spawning point after 40 attempts.
06/28/2023 20:07:59: {SystemWarning} SpawnZone 339 did not find a suitable spawning point after 40 attempts.
06/28/2023 20:07:59: {SystemWarning} SpawnZone 396 did not find a suitable spawning point after 40 attempts.

And what if I back off the y-offset to 35 to make up for the extra line? No more spawning failures. That settles it.

And now we have a set of map files that a bit more consistent. What should we do with them?

A Bit of Code Trying to Do Nothing

An officially useless machine.

So now let’s have some fun: code up a utility to consume one of these map files, split it into its component parts, and then output the exact same file. I feel like Go will turn out to be a good fit for this project, so we’ll start using it now.

Define Pieces and Consume the Header

First, what are the pieces to split the map into?

main.go GitLab

10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36


type MapHeader struct {
  XMLName       xml.Name `xml:"mapheader"`
  ZCoord        int      `xml:"z"`
  ZName         string   `xml:"n"`
  XOffset       int      `xml:"x"`
  YOffset       int      `xml:"y"`
  ForestryLevel string   `xml:"f"`
  Outdoor       bool     `xml:"outdoor"`
  NoRecall      bool     `xml:"norecall"`
  AlwaysDark    bool     `xml:"alwaysdark"`
  TownLimits    bool     `xml:"townlimits"`
  Comments      []string `xml:"comment"`
}

type MapContents struct {
  Cells map[cellKey]Cell
}

type cellKey struct {
  X int
  Y int
  Z int
}

type Cell struct {
  Graphic string
}

Each map level has the header info that’s global to that level, then a map of cell graphics keyed by their coordinates. Tagging the fields of the MapHeader structure with XML element names will make processing the header almost trivial, as long as it’s wrapped in an outer tag (here we use <mapheader>):

main.go GitLab

47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77


func processXMLHeader(xmlString string, header MapHeader) (MapHeader, error) {
  err := xml.Unmarshal([]byte(xmlString), &header)
  if err != nil {
    return header, err
  }

  parts := strings.Fields(xmlString)
  for _, part := range parts {
    keyValue := strings.Split(part, "=")
    if len(keyValue) == 2 {
      switch keyValue[0] {
      case "outdoor":
        header.Outdoor, _ = strconv.ParseBool(keyValue[1])
      case "norecall":
        header.NoRecall, _ = strconv.ParseBool(keyValue[1])
      case "alwaysdark":
        header.AlwaysDark, _ = strconv.ParseBool(keyValue[1])
      case "townlimits":
        header.TownLimits, _ = strconv.ParseBool(keyValue[1])
      default:
        return header, errors.New("Unknown key: " + keyValue[0])
      }
    }
  }

  if header.ZName == "" {
    return header, errors.New("no z name")
  }

  return header, nil
}

Indeed, unmarshalling the XML part is a one-liner. Handling the key-value pairs is a little more awkward; if there were more than four of them, it’d be worth doing some refactoring (perhaps incorporating them into XML before the Unmarshal() call?).

Note that in the case of an error, we return the header data we’ve processed so far. Best practices for error handling usually dictate that the caller cannot make any assumptions about the contents of the other return values in the presence of an error. In a more public API, I wouldn’t tempt the caller with partial info, but in this case it seems like there is troubleshooting benefit to having it.

Always Be Testing

Thank goodness for gotests integration in in your favourite IDE, since even table-driven tests can be fairly verbose:

main_test.go GitLab

10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26


func Test_processXMLHeader(t *testing.T) {
  type args struct {
    mapline string
  }
  tests := []struct {
    name    string
    args    args
    want    MapHeader
    wantErr error
    {
      name: "minimal",
      args: args{
        mapline: `<mapheader> <z>15</z> <x>10</x> <y>5</y> <n>zname</n> </mapheader>`,
      },
      want: MapHeader{XMLName: xml.Name{Space: "", Local: "mapheader"}, ZCoord: 15, XOffset: 10, YOffset: 5,
        ZName: "zname"},
    },

…

main_test.go GitLab

69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92


    {
      name: "bad keyval",
      args: args{
        mapline: `<mapheader> <z>15</z> <x>10</x> <y>5</y> <n>zname</n> notreal=true townlimits=true </mapheader>`,
      },
      want: MapHeader{XMLName: xml.Name{Space: "", Local: "mapheader"}, ZCoord: 15, XOffset: 10, YOffset: 5,
        ZName: "zname"},
      wantErr: errors.New("Unknown key: notreal"),
    },
  }
  for _, tt := range tests {
    t.Run(tt.name, func(t *testing.T) {
      got, err := processXMLHeader(tt.args.mapline, MapHeader{})
      if (err != nil && err.Error() != tt.wantErr.Error()) ||
        (err == nil && tt.wantErr != nil) {
        t.Errorf("processXMLHeader() err = %v, want %v", err, tt.wantErr)
      }
      if !reflect.DeepEqual(got, tt.want) {
        t.Errorf("processXMLHeader() = %v, want %v", got, tt.want)
      }

    })
  }
}

Do we at least cover the code we’ve written so far?

$ go test -cover
PASS
        glitch-aura-djinn       coverage: 100.0% of statements
ok      glitch-aura-djinn       0.002s

So far, so good.

Now Recreate the Header

On the other side, let’s make sure we can take the MapHeader structure and recreate the text.

main.go GitLab

 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117


func mapStyleMapHeader(mapHeader MapHeader) string {
  var sb strings.Builder

  // Add comments
  for _, comment := range mapHeader.Comments {
    sb.WriteString("//" + comment + "\n")
  }

  // XML format
  sb.WriteString("<z>" + strconv.Itoa(mapHeader.ZCoord) + "</z> ")
  sb.WriteString("<x>" + strconv.Itoa(mapHeader.XOffset) + "</x> ")
  sb.WriteString("<y>" + strconv.Itoa(mapHeader.YOffset) + "</y> ")
  if mapHeader.ForestryLevel != "" {
    sb.WriteString("<f>" + mapHeader.ForestryLevel + "</f> ")
  }
  sb.WriteString("<n>" + mapHeader.ZName + "</n>")

  // Add key-value pairs
  kvs := []string{}
  if mapHeader.AlwaysDark {
    kvs = append(kvs, "alwaysdark=true")
  }
  if mapHeader.NoRecall {
    kvs = append(kvs, "norecall=true")
  }
  if mapHeader.Outdoor {
    kvs = append(kvs, "outdoor=true")
  }
  if mapHeader.TownLimits {
    kvs = append(kvs, "townlimits=true")
  }
  if len(kvs) > 0 {
    sb.WriteString(" ")
    sb.WriteString(strings.Join(kvs, " "))
  }

  sb.WriteString("\n")
  return sb.String()
}

A little more complicated to make sure we recreate the style from the map files. Comments first, then the XML-style part with optional forestry, and each of the key-value pairs that aren’t false. We don’t return any errors, assuming we can create a string from any valid header struct.

The tests are pretty much the same with the input and output swapped, with the addition of comments and newlines.

main_test.go GitLab

 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153


func Test_mapStyleMapHeader(t *testing.T) {
  type args struct {
    mapHeader MapHeader
  }
  tests := []struct {
    name string
    args args
    want string
  }{
    {
      name: "minimal",
      args: args{
        mapHeader: MapHeader{XMLName: xml.Name{Space: "", Local: "mapheader"}, ZCoord: 15, XOffset: 10, YOffset: 5,
          ZName: "zname"},
      },
      want: `<z>15</z> <x>10</x> <y>5</y> <n>zname</n>` + "\n",
    },
    {
      name: "forestry",
      args: args{
        mapHeader: MapHeader{XMLName: xml.Name{Space: "", Local: "mapheader"}, ZCoord: 15, XOffset: 10, YOffset: 5,
          ZName: "zname", ForestryLevel: "heavy"},
      },
      want: `<z>15</z> <x>10</x> <y>5</y> <f>heavy</f> <n>zname</n>` + "\n",
    },
    {
      name: "keyvals 1",
      args: args{
        mapHeader: MapHeader{XMLName: xml.Name{Space: "", Local: "mapheader"}, ZCoord: 15, XOffset: 10, YOffset: 5,
          ZName: "zname", AlwaysDark: true, Outdoor: true},
      },
      want: `<z>15</z> <x>10</x> <y>5</y> <n>zname</n> alwaysdark=true outdoor=true` + "\n",
    },
    {
      name: "keyvals 2",
      args: args{
        mapHeader: MapHeader{XMLName: xml.Name{Space: "", Local: "mapheader"}, ZCoord: 15, XOffset: 10, YOffset: 5,
          ZName: "zname", TownLimits: true, NoRecall: true},
      },
      want: `<z>15</z> <x>10</x> <y>5</y> <n>zname</n> norecall=true townlimits=true` + "\n",
    },
    {
      name: "comments",
      args: args{
        mapHeader: MapHeader{XMLName: xml.Name{Space: "", Local: "mapheader"}, ZCoord: 15, XOffset: 10, YOffset: 5,
          ZName: "zname", Comments: []string{"comment 1", " comment 2"}},
      },
      want: strings.Join([]string{`//comment 1`, `// comment 2`, `<z>15</z> <x>10</x> <y>5</y> <n>zname</n>`}, "\n") +
        "\n",
    },
  }
  for _, tt := range tests {
    t.Run(tt.name, func(t *testing.T) {
      if got := mapStyleMapHeader(tt.args.mapHeader); got != tt.want {
        t.Errorf("mapStyleMapHeader() = %v, want %v", got, tt.want)
      }
    })
  }
}

$ go test -cover
PASS
        glitch-aura-djinn       coverage: 100.0% of statements
ok      glitch-aura-djinn       0.003s

Round Trip Testing

Now that we have both sides of the coin, we can have some fun with round-trip tests. First we do header->struct->header, and then struct->header->struct.

main_test.go GitLab

154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189


func Test_mapHeaderRoundTrip1(t *testing.T) {
  tests := []struct {
    name    string
    mapline string
  }{
    {
      name:    "minimal",
      mapline: `<z>15</z> <x>10</x> <y>5</y> <n>zname</n>`,
    },
    {
      name:    "forestry",
      mapline: `<z>15</z> <x>10</x> <y>5</y> <f>heavy</f> <n>zname</n>`,
    },
    {
      name:    "keyvals 1",
      mapline: `<z>15</z> <x>10</x> <y>5</y> <n>zname</n> alwaysdark=true outdoor=true`,
    },
    {
      name:    "keyvals 2",
      mapline: `<z>15</z> <x>10</x> <y>5</y> <n>zname</n> norecall=true townlimits=true`,
    },
  }
  for _, tt := range tests {
    t.Run(tt.name, func(t *testing.T) {
      parsed, err := processXMLHeader(`<mapheader> `+tt.mapline+` </mapheader>`, MapHeader{})
      if err != nil {
        t.Errorf("processXMLHeader() err = %v", err)
      }
      got := mapStyleMapHeader(parsed)
      want := tt.mapline + "\n"
      if !reflect.DeepEqual(got, want) {
        t.Errorf("mapStyleMapHeader(processXMLHeader()) = %v, want %v", got, want)
      }
    })
  }
}

main_test.go GitLab

191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229


func Test_mapHeaderRoundTrip2(t *testing.T) {
  tests := []struct {
    name   string
    header MapHeader
  }{
    {
      name: "minimal",
      header: MapHeader{XMLName: xml.Name{Space: "", Local: "mapheader"}, ZCoord: 15, XOffset: 10, YOffset: 5,
        ZName: "zname"},
    },
    {
      name: "forestry",
      header: MapHeader{XMLName: xml.Name{Space: "", Local: "mapheader"}, ZCoord: 15, XOffset: 10, YOffset: 5,
        ZName: "zname", ForestryLevel: "heavy"},
    },
    {
      name: "keyvals 1",
      header: MapHeader{XMLName: xml.Name{Space: "", Local: "mapheader"}, ZCoord: 15, XOffset: 10, YOffset: 5,
        ZName: "zname", AlwaysDark: true, Outdoor: true},
    },
    {
      name: "keyvals 2",
      header: MapHeader{XMLName: xml.Name{Space: "", Local: "mapheader"}, ZCoord: 15, XOffset: 10, YOffset: 5,
        ZName: "zname", TownLimits: true, NoRecall: true},
    },
  }
  for _, tt := range tests {
    t.Run(tt.name, func(t *testing.T) {
      mapline := mapStyleMapHeader(tt.header)
      got, err := processXMLHeader(`<mapheader> `+mapline+` </mapheader>`, MapHeader{})
      if err != nil {
        t.Errorf("processXMLHeader() err = %v", err)
      }
      if !reflect.DeepEqual(got, tt.header) {
        t.Errorf("processXMLHeader(mapStyleMapHeader()) = %v, want %v", got, tt.header)
      }
    })
  }
}

These are a little less verbose, given that the input and desired output are the same. Other than that, though, what am I going to do with this round-trip testing?

Kermit doing some manual fuzzing.

Fuzzing. Tomorrow.

One More Thing

Rubocop job finally passing.

Over in the Ruby land of tests, I’ve never had much luck making Rubocop completely happy. On a whim, I decided to install Spotify’s VSCode Ruby extension pack and work on it for a little while.

Though I did opt to override Rubocop with a lot of my own opinions:

rubocop.yml GitLab

 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42


AllCops:
  NewCops: enable

# Nostalgia aside, I find 100 to be a more reasonable limit.
Layout/LineLength:
  Max: 100

# %x is more clear than backticks when glancing at code.
Style/CommandLiteral:
  SupportedStyles: mixed

# I use 'fail' in test code for catastrophic failure.
Style/SignalException:
  Enabled: false

# Allow long(ish) methods for now
Metrics/MethodLength:
  Max: 40

Metrics/BlockLength:
  Max: 40

Metrics/AbcSize:
  Max: 30

Naming/MethodParameterName:
  AllowedNames: [ "x", "y", "z"]

Naming/AccessorMethodName:
  Enabled: false

Naming/VariableName:
  Enabled: false

Metrics/CyclomaticComplexity:
  Max: 10

Metrics/PerceivedComplexity:
  Max: 10

There are certainly engineers who argue that using an opinionated linter with no exceptions is the only way to go, but I struggle to do so with Rubocop. However, it’s a good place to start. Then every time the linter complains, you can consider whether the rule makes sense for your project and team.

Code coverage is similar: Rather than doing too much tweaking to guarantee 100%, exceptions can be considered with appropriate context.

More to come