Whenever you put your game out into the wild, you can never be quite sure what you’re going to learn from it.

The Jetsam v0.2 Beta was no exception.

 

 

The Beta Data

This time around, we were prepared to gather hard data through an in-app logging call that dumped delicious data into a Google Spreadsheet every time a player successfully completed a level – and every time they failed.

From this data, we were able to capture some interesting statistics. As of the time of writing, the Jetsam Level Editor Beta had:

• 48 Unique Players
• 219 Unique Levels Played
• 91 Hints Used
• 1573 Unique Play Sessions
• 4 distinct versions:
  • v0.2.0
  • v0.2.1
  • v0.2.2
  • v0.2.3

 

Identifying Difficulty Cliffs

Given the linear structure of our game, our expectation was that level completion numbers would start high at the beginning of each world and dwindle as the levels got progressively harder – but actual completion numbers told a different story.

We had ourselves a cliff – the dropoff from W1-D to W1-E was severe, over 33%. Were people getting bored with our game after four levels? Was the difficulty ramping too quickly? We asked these questions of our testers, many of whom corroborated the common complaint that “it got too hard, too fast.”

We dug into failure data to see if that complaint held water – and lo and behold, we found that 11 of 17 unique W1-E players failed at least once when playing it!

W1-E – does it look difficult to you?

Armed with quantitative confirmation of our qualitative feedback, we redesigned several levels from W1-E onward and launched a new version, which performed thusly:

This time around, the dropoff between 1-D and 1-E was far less dramatic. Failure data also bears this out, as 0 of 6 unique W1-E players failed it in this later trial. Chalk one up to the power of data to make our level design better!

 

Hint Optimization

While not really the objective of our Beta test, we tracked the number of people who used Hints to get past tough levels, and reached an interesting conclusion: almost no one was using them (only 3 unique users had even bothered). This made intuitive sense, as the Hint menu wasn’t an obvious part of the UI by any means.

As a result, in v0.2.3 we added a Hint Tutorial that draws attention to the Hint menu in the UI after a player retries a level for the first time. Subsequently, Hint usage went up 75%. That simple little tweak has opened our mind to future experiments we want to perform around Hints in the v0.3 Beta.

 

Device Profiling

Since we’re launching our game on iOS, we figured we’d capture data about the devices our players were using.

Perhaps unsurprisingly, the iPhone X led the pack – but the iPhone 7 had a strong second-place showing.

Most surprisingly of all, no one played our game on an iPad, even though we have Universal iOS device targeting set up. Maybe that means we should spend less time making our game iPad-friendly?

 

Conclusions

Running an open-ish Beta for nearly 50 people was a lot of fun, and taught us tons about our game’s weak points. Being able to comb through the level completion and failure data really helped us tune the difficulty curve better, hopefully leading to stronger player retention. Identifying our Hint issue certainly saved us headaches later down the road with monetization. Realizing that there aren’t as many iPad users also helped us better prioritize our development backlog.

Onward to v0.3!

---

If you liked this blog post, show us some love by following us on Twitter. We’ll happily follow back.

Feel free to download the Jetsam Beta as well if you’re a fan of puzzle games!

Designing a game is great fun. As the designer, you get to curate the player’s experience from the moment they start their adventure. You can weave an intricate story around them, set them up for teaching moments, or put them in situations where their skills are tested. That ability to meticulously plan every detail – every high and every low – allows a well-designed game to become a beautiful, flowing conversation between a game’s designers and its players.

But when you’re designing a sandbox game, it can feel a lot more like handing your players loaded guns and hoping like hell they don’t shoot themselves in the foot.

As such, we’ve developed a comprehensive plan to prevent our obnoxiously feature-rich content creation system from sucking the fun out of our game. Our “sandbox disaster mitigation strategy” boils down to the following three tactics: Player Education, Helicopter Parenting, and Protocol Robustness.

Player Education

This is the obvious one – of course you should teach your player how the game works, duh! But Player Education transcends simple design principles like Learner Levels. The entire structure of Jetsam’s main story has been built around the principle that players need to be educated really thoroughly.

Jetsam is a puzzle game based around a simple mechanic – you can only move in one direction at a time, and you slide indefinitely until you collide with something that brings you to a stop. In each level, you must collect a Relic and escape to the Exit. All the game’s puzzles are based around the awkwardness of movement in this environment.

To complicate matters further, other obstacles show up in the game’s various worlds. Fuel batteries give you extra moves, gravity wells stop you cold, boxes shatter on contact, wormholes preserve your momentum and warp you across the level, and deadly buzzsaws destroy you. The game’s main story thus begins with six distinct “worlds,” each of which emphasizes a certain mechanic and teaches the player all kinds of dirty tricks for dealing with it.

That’s a lot of information for a player to retain though – so occasionally, a game mechanic from a previous world is re-introduced alongside the current world’s primary mechanic to keep the player’s memory fresh. Only after the player has demonstrated competence by completing all fifteen levels in each of the six introductory worlds are they allowed to progress to world seven and beyond, where the real challenge starts.

Helicopter Parenting

“Are you sure you want to do that?”

It’s amazing how effective a simple confirmation popup guarding the “delete all my saved levels” option can be. In fact, it’s our opinion that most of the really bad things that players can do in a sandbox game are identifiable in advance, to a certain extent. As the game designer, it’s your job to identify them and take defensive action.

Obvious things: “deletion” actions should be guarded with a confirmation, possible runtime exceptions should be caught, game-breaking exploits should be patched if found, and Hidden Rails should attempt to prevent players from frustration. Fuzzing can really help when testing your functions for strange behaviors.

Non-obvious things: players shouldn’t be able to share unwinnable levels, players shouldn’t be able to load unwinnable levels, players don’t want to overwrite their save data by accident, and players will probably try to make levels that look like penises.

Not all non-obvious things can be solved outright (too bad “Not Hotdog” from Silicon Valley doesn’t work for identifying penis-shaped Jetsam levels), but a lot of them can be made more difficult or made less of an issue through clever design. The Nintendo classic Mario Maker forces would-be level creators to beat their own levels before they share them, for example.

Protocol Robustness

In Jetsam, every level can be expressed with an ASCII code – for example, take a peek at the following level (which just so happens to be the first one in the game).

The universality of Level Codes is extremely useful for sharing and saving levels – all people need to do is transmit text between each other. However, it introduces the problem of protocol robustness – whereby we need to make damn sure that our Level Codes are backwards-compatible, and that we have flexibility available for the future should we need it.

As such, a lot of thought has gone into our encoding scheme. We settled on ASCII characters because of fears that some UTF-8 characters might be reinterpreted by overzealous text manipulation and rendering programs. We had to purposely avoid the control-character subset of ASCII as well, in order to make sure that no meaning was lost by text message apps that strip such characters from inputs.

We also left ourselves some breathing room – certain characters are invalid in our encoding scheme for now, but we might make them valid in the future to represent things like additional level types.

The moment we let Jetsam out into the wild, its success and staying power will depend greatly on a growing network effect from a massive library of player-created levels. Our Level Code protocol has to stand the test of time – we can never deprecate Level Codes once they’ve been deemed valid, or else we will have wasted our players’ hard work (and built up a library of Level Codes that don’t work!). The Level Code rules will thus be at their narrowest at launch, and will progressively loosen if we continue adding features.

Conclusion

Sandboxes are hard to design, but we’re confident that we’ve anticipated a lot of the headaches players might run into, and that we have designed Jetsam to mitigate them where possible. Players will always surprise us though – so we’re going to be launching a public Beta test of the Level Editor soon (on iOS). We’ll be adding 15 of our Beta testers’ best level creations to the shipping game as an added incentive to participate, so if that piques your interest, stay tuned!

---

If you liked this blog post, follow us on Twitter and check back regularly for updates.

So we finally sucked it up and decided to legitimize our side projects. We even got ourselves a fancy six-letter domain name.

Welcome to Zonzle, a company about making awesome mobile games.

Of course, developing games is to a programmer like being in a band is to a hobbyist musician – something most would certainly love to do, but something few manage to actually make a living from. We’re each too cynical to work for a triple-A game company (have you seen how draconian typical working conditions in the industry are?), but we’re each too naive and optimistic to let the dream die without a fight.

The barriers to entry are low. The market is saturated. The odds are stacked against us.

We have nothing to lose.

We’re cooking up our first iOS game at the moment. Details may begin to leak as it bakes. In the meantime, enjoy tilting your screen back and forth and determining whether our site’s theme color is blue, purple, or blurple. Pixels are WEIRD.