psychicsoftware
April 29, 2013
Musclecar Online, Techie
Leave a comment

Making an efficient gameserver with node.js and mysql (Musclecar Online)

April 29, 2013
Musclecar Online, Techie
Leave a comment

Last year, when one of my games went a bit viral, I discovered first-hand about the alarming inefficiency of Apache (webserver), especially when running PHP which forces you to accept a process-per-request approach. It became pretty clear that something much more lightweight would be required if I wanted to build gameservers with large player-bases in mind.

After some research I have settled on node.js as a good solution. It’s very well regarded, largely due to its non-blocking, asynchronous model. This essentially means that anytime a resource is required (such as a database recordset) the request goes into a queue, to be called back when the resource is retrieved. In Apache, the model is blocking – the process sits and waits for the resource, thereby stopping any other activity.. this is why Apache needs all those independent processes, so that it can serve multiple clients simultaneously.

speedracer The game I have been working on is ‘Musclecar Online’ – a top-down racer in which players build their own tracks, vote on them, and then everyone in the world races on the same track-of-the-day. Rather than being real-time, you race against the ‘ghost lap’ recordings of other players; kinda like Speed Racer ;-)

Cache stuff in RAM

Linux has a server’s approach to the file system, of course, which means it uses all available RAM for caching disk files and such, but I figured it would be even better to make liberal use of Javascript arrays (which are really hash tables.. excellent for key-based lookups). So the server packs up canned responses into strings, as much as possible, ready for delivering over the interweb with no fuss or processing. This includes:

    a 5-day pack of track layouts, allowing the players to see what’s coming up over the next few days;
    per-cartype, per-second frequency bins, succinctly providing the data needed by the client to display a histogram of worldwide laptimes and point out where the player stands versus these;
    a temporary array of ghost lap recordings, hashed by user ID for easy over-writing when they improve. This is only used up until 40 players have recorded laps for that car type on that day’s track, and allows random ‘unclassified’ packs of ghost laps to be delivered to players without the need to bother MySql about it;
    more long-term packs of ghost lap recordings, separated into 6 divisions (F thru A) – the idea is that the player starts off racing against poor ghosts, labelled as division F, and if they beat these they are allowed to progress to division E, and so on. These longer-term packs are re-generated periodically, but not for the first time until 40 recordings are available for the car type.

Don’t fall into the ORDER BY RAND() trap

I have done some pretty naive stuff in MySql in the past, but running Darkwind for the past 7 years has helped me understand efficient database design and querying. With the characters, vehicles and weapons tables running into the millions, I was forced to. One of the newbie gotchas in MySql is to avoid doing a massive SELECT statement with ORDER BY RAND() LIMIT 1 in order to get a random record. If you do this, you’re basically asking the database server to load an entire table into memory, apply a random number to each record, sort the whole damn thing, and return one row. There’s a few obvious solutions, depending on how well distributed the data in your index fields is. One simple way is just to find your data range (e.g. laptimes min and max), generate a random number in this range, and then SELECT the first row greater than or equal to this laptime, LIMIT 1. Assuming laptimes is an index field, and also assuming there are enough records to make a reasonably even distribution across the data range, this works nicely.

Interesting asynchronous design solutions

Now, I love callbacks as much as the next man, but it can be an old-fashioned PITA sometimes trying to work with asynchronous coding. In a nutshell, anytime you want a resource that may be delayed (database query, file read etc.) you can’t use a traditional flow of control, can’t use loops for multiple accesses, etc. One of the things I have to do in the ‘Musclecar Online’ server is build packs of typical ghost lap recordings, in order to pack them into strings ready for dispatch to the clients. This means querying for each row with a separate SELECT statement, and therefore means 11 car types x 6 divisions x 5 different packs x 4 ghosts per pack = 1320 queries. Embedding each query inside the callback of the previous one certainly isn’t an option, but with a little thought I settled on recursion; a perfect way to unravel the asynchronous tangle. This works nice and efficiently, while still operating in serial (and therefore not choking up any server resources) and allowing nice tidy code. It’s kind of strange I didn’t see recursion mentioned as a solution anywhere on the web while researching node.js. Well, maybe I didn’t look for too long.

Newsletter

Your email address will not be shared with any 3rd parties, and will only be used for occasional newletters about Psychic Software's games.
 
[Our Privacy Policy]

Presskit

  • Press Kit here

Games in Development

  • The Necromancer's Tale
  • Newby Chinese

Games Released

  • Darkwind: War on Wheels
  • Let's Break Stuff!
  • Musclecar Online
  • Goblins & Gottos
  • Orbs.it
  • Mars Defender
  • Demon Pit
  • Afterburn 2150
  • Block Rockin'
  • More on Gooogle Play
  • More on iOS Appstore

Unfinished Projects “On Hiatus”

  • Zed's Dead
  • Ping Pong Planets
  • Godkin

Archives

  • December 2023
  • June 2022
  • April 2022
  • May 2021
  • December 2020
  • November 2020
  • October 2020
  • August 2020
  • May 2020
  • March 2020
  • October 2019
  • October 2018
  • August 2018
  • July 2018
  • January 2018
  • September 2017
  • August 2017
  • May 2017
  • July 2016
  • May 2016
  • April 2016
  • December 2015
  • November 2015
  • September 2015
  • July 2015
  • May 2015
  • April 2015
  • February 2015
  • January 2015
  • December 2014
  • October 2014
  • August 2014
  • January 2014
  • December 2013
  • September 2013
  • July 2013
  • May 2013
  • April 2013
  • March 2013
  • December 2012
  • October 2012
  • August 2012
  • June 2012
  • May 2012
  • April 2012
  • March 2012
  • February 2012
  • January 2012
  • November 2011
  • October 2011
  • September 2011
  • August 2011
  • July 2011
  • June 2011
  • May 2011
  • April 2011
  • March 2011
  • February 2011
  • January 2011
  • March 2006
  • February 2006
  • January 2006
  • November 2005
  • October 2005
  • September 2005
  • August 2005
  • July 2005
  • December 1995

Categories

  • Afterburn 2150
  • Block Rockin'
  • Conferences & Events
  • Darkwind
  • Dead By Dawn
  • Demon Pit
  • Game Musings
  • Goblins & Grottos
  • Godkin
  • Guest Posts
  • Let's Break Stuff!
  • Mars Defender
  • Monster Melee
  • Musclecar Online
  • Newby Chinese
  • Orbs.it
  • PC Gamer
  • Rock Paper Shotgun
  • Techie
  • The Necromancer
  • Uncategorized
  • Zed's Dead

CyberChimps WordPress Themes

PSYCHICSOFTWARE | Psychic Games Ltd.
Sam Redfern indie games developer and university academic