We came. We saw. We beta’d. Jetsam has been in the hands of real players for the first time!
The Jetsam Beta was a carefully-selected sampling of stuff that will be in the final game. We were intentionally cagey about further details – we wanted players to get into the game and explore, thereby testing how good our design was.
We collected feedback and all kinds of metrics from our Beta – including how people navigated the basic UI, what they enjoyed/hated, and how difficult they thought the levels were.
This post will go into detail about some of the lessons we learned – what worked, what didn’t, and how we plan to address the great feedback we received from finally putting our game out into the wild.
Jetsam Beta – By the Numbers
- 12 levels and 1 unlockable “secret” level
- 3 testable, level-altering mechanics
- 27 unique beta testers
- 146 unique play sessions
- 6.2/10 average fun rating for fuel cells
- 6.8/10 average fun rating for crates
- 7.5/10 average fun rating for wormholes
- ~1/2 of players say that wormholes were their favorite mechanic
- ~1/3 of players say that wormholes were their least favorite mechanic
UI Design
We asked a lot of questions about our UI to our testers – here are some lessons learned.
Expectation: Main Menu UI is easy to navigate.
Reality: True! All testers reported that navigating the menus was easy.
Expectation: Level Select UI clearly described how levels were unlocked and whether or not you had completed a given level.
Reality: False! One third of testers reported some degree of confusion about which levels were completed.
What we’ll be changing: We’ll be making the level selection UI prettier and abandoning the “badge” approach of putting an icon on the level button if you’ve completed or perfected a level. There was clearly too much confusion.
Gameplay Mechanics
Expectation: All three tested mechanics would be roughly equally liked.
Reality: False! Testers seemed to really engage with the wormhole mechanic, with half citing it as their favorite, and a third citing it as their least favorite. Testers liked all of the mechanics, but cited fuel cells as their least favorite, generally because grabbing extra fuel always seemed like the best option and didn’t present much strategic challenge.
Expectation: The three tested mechanics would be relatively easy to understand and learn without a tutorial.
Reality: True! Most players cited that it was easy or very easy to learn how the three mechanics worked.
What we’ll be changing: We’ll be adding more wormhole levels since players liked them. We will be more purposeful about our fuel cell level design though, so several of those levels are going to be redesigned to make their fuel component more enjoyable.
Level Difficulty
Expectation: The levels we selected were a good representation of the difficulty spectrum across the whole game.
Reality: False! Several players cited “extreme difficulty jumps” across our levels.
Expectation: We selected an easy (A), medium (B), and hard (C) level from each mechanic’s pool of existing levels, so we believed that each letter’s corresponding levels should receive similar difficulty scores.
Reality: False! The 1X series of levels were too easy. The 4X series of levels were way too hard. The 2X and 3X series of levels hit our difficulty targets right about on the money.
| Level Difficulty/10 | A | B | C |
|---|---|---|---|
| 1 | 1.2 | 2.4 | 3.4 |
| 2 | 2.6 | 3.0 | 6.5 |
| 3 | 2.0 | 4.8 | 6.7 |
| 4 | 4.5 | 6.0 | 9.0 |
| ? | 9.3 |
Expectation: Roughly half of players would unlock the secret bonus level.
Reality: False! Only three of our twenty-seven testers unlocked it.
What we’ll be changing: Our levels were too damn hard! We need to refocus our level design on creating more “medium” difficulty levels.
Conclusion
That about wraps it up – we learned a ton from the Jetsam Beta, and look forward to improving things across the board as we approach our holiday season launch window! There are lots of new tickets on the backlog and new ideas swimming through our heads.
To our beta testers – thanks for playing!
If you liked this blog post, show us some love by following us on Twitter. We’ll happily follow back.
Zonzle LLC 