TSSSF Playtest Sheet

[N.A. Larson]

For playtesting single cards, simply playing the game and observing the how the card interacts with others may be sufficient. However, for cards that have broader influence (especially Start cards), and for entire expansions, the effect these have on the game may not be easy to determine from casual play. For this reason, I developed this sheet as a structured, numerical approach to note-taking to make comparisons easier (*.ods version).

Rather than be a game record, my goal is to capture the “flow” of a game. I believe the players’ perception of “flow” is strongly tied to how often Goals are won, and how difficult it is to win them. Therefore, it is necessary to track the winning and replacement of Goals, as well as the amount of resources a player has access to at any given time. I tried to minimize the amount of data recorded so as to not hinder gameplay, but I acknowledge that there may be additional critical information required. Having a space for notes somewhat accounts for this. The data that IS recorded is as follow:

Build: List all the cards you are playing with, including the Start card.

House Rules: List any rules you are using in addition to the official rules (1 activation per turn, mulligans, alternate number of active Goals, etc).

Date: Record the date the game is played on.

No (Number): If you play multiple games in one day, indicate which game this is (1st, 2 of 3, etc).

Initial Goals: List the point value of each active Goal at the start of the game.

Hand: Record the number of cards in the player's hand at the start of their turn.

Draw: Keep a tally of the number of cards drawn by the player, even if those cards were immediately discarded. Make a note if a power besides (Draw) was used.

Srch (Search): Keep a tally of the number of cards taken from the discard pile by the player, even if those cards were put back. Make a note if a power besides (Search) was used.

End: Record the number of cards in the player's hand at the end of their turn, before redrawing/discarding to 7 total.

Used: Calculate the number of cards that passed through the player's hand during their turn (not the same thing as the number of cards played, but accounts for draw 3/discard 2 and similar abilities). Used = Hand + Draw + Search - End. This formula is built into the spreadsheet.

Grid: Record the number of cards on the grid at the end of the player's turn (don't count the Start card). If a large number of cards would be discarded, make a note of the number before discarding.

Disc (Discard): Record the number of cards in the discard pile the end of the player's turn. If a large number of cards would be discarded, make a note of the number before discarding.

Note: It is possible to double-check the numbers recorded for each player’s turn by comparing the sum of the grid and discard pile at the end of the previous player’s turn to that of the current player’s turn.  Simply put:  grid_prev + disc_prev + used - search - exile = grid + disc. This formula is built into the spreadsheet (except for exile because it happens so rarely that it wasn’t worth building in). Returning a card to the deck also counts as exile for the purposes of this equation. This equation hold true unless cards are exchanged between the discard pile and the deck, such as through reshuffling or Mahou Shoujo Derpy. Make a note of the number of cards in the discard pile before this happens.

Won: List the point value of each Goal the player won during their turn.

Replaced: List the point value of each Goal the player replaced during their turn. Make a note if a power besides (New Goal) was used.

Duds: List the point value of each Goal that was revealed, but was discarded because its requirements were already met.

Notes: note anything interesting that happened during the player's turn, such as combos, explanations of the player’s choices, and the appearance of powerful cards (especially Love Poisons).

Score: Record each player's final score.

Goals: Record the total number of Goals revealed (achieved + active + discarded) during the game. List the number of each point value.

Pony: Record the number of Pony cards that players went through in the course of the game (hands + grid + discard + exile; this isn’t the exact number, but it’s close enough). If the discard pile was reshuffled, add that number as well.

Ship: Same as above, but for Ship cards.

Example: A game I played against myself

There is much I hope to learn with data collected using this sheet. I would like determine what a typical game with Core deck looks like, and how it changes depending on the number of players. I would like to determine the effects of adding each of the official expansions to Core deck, and what happens when they are all played at once. I would like to determine what the ideal expansion looks like (if there is such a thing), and use it as a benchmark for designing other expansions. I wonder if there is a direct correlation between cards used and Goal point-value, and if it can be determined with the data collect using this sheet. Also, it may be possible to evaluate different play strategies using this sheet (win one Goal at a time, win as many as you can, replace the small Goals vs the big ones, play your whole hand, play as little as possible, etc).

Please try this sheet out with your own games and let me know how useful it is. Please suggest changes, or make your own version of the sheet and post that. And most of all, please post your results! I don’t have a way to sort or organize them, so just link them in this thread.


Thanks and good luck,

N

[Tutorial Guy]

I think you complicate all, but if you sure that this tablet somehow will help you, then I'll try to use it during the next game

[Tutorial Guy]

https://docs.google.com/spreadsheets/d/1NSbLd15P3u0b4BeoiZgg4vtRixxTeZQgfSWswMuO9dY/edit?usp=sharing
Here is - result of the game with playtest. That was HORRIBLE. Your tablet is not prepare for some specific powers

[N.A. Larson]

Thank you for recording this data! I'm sorry for your long game, but with this data we may find the cause (besides filling out this table, which I admit DOES add time to the game ^^;). I have uploaded some of my own results for comparison. Sorry that some information is missing; I was using an earlier version on the table at the time.

I notice that your game lasted 13 turns, which is longer than I would expect (just Core is more like 10 turns). I also notice that you are using almost every expansion. It is a common opinion that this leads to longer games, but I would like to prove or disprove it with data collected using this table.

The grid seems to be growing at a normal rate, so that's good.

I'm confused because your Goal numbers don't add up. Did you really go through 94 Goals, or was that 94-points of Goals? Either way, that is a LOT. Also, did Just Another Pony replace 5 Goals on Round 1, or was it one 5-point Goal?

You mentioned that this table is not good for recording specific card powers. Are there any you remember specifically? (Based on the number of Goals that got replaced, I'm going to guess it was Mahou Shoujo Derpy.) At any rate, it is not my intention that this table be a detailed record. My aim is to track how many cards players use to win Goals. I included the "Notes" box to record the appearance of powerful or unusual cards.

When you record the Disc column, are you recording the number of cards discarded that turn, or the total size of the discard pile? (It's supposed to be the total size). Also, what did Just Another Pony do on Round 3 to cause so many cards to be discarded?

Why do the players not have 7 cards in their hands at the beginning of Round 2?


Thanks,

N

[Ithry]

The playtest group I normally test TSSSF with visibly reacted negatively when they saw the playtest sheet (one of them physically recoiled) and more or less asked to stop the game after three turns. After a few turns, we had learned enough to give plenty of feedback on the sheet.

According to your original post, the main points of this sheet are: 1. To see how often Goals are won and 2. How hard they were to achieve to determine game "flow". Based on this and general playtesting experience, we produced the following notes:

-Numbers without context isn't that useful. Things like number of Goals is nice, but knowing which Goals are active, were achieved, etc. is much more useful. Names are generally more important than numbers.

-Notes should be taken after the game (Usually. Hotfixes for the sake of testing is fine). The information, concerns, etc. that stick with players at the end of the game is the most relevant.

-Taking information down during the game slows it down, players don't have fun, and in extreme cases forget what is going on in the game. Taking down information during a turn took twice as long as it took a player to perform a full turn. Since end of turn information had to be recorded as part of this sheet, gameplay was stopped every turn so that the information recorder could catch up.

-TL;DR: The best way to test is to play a lot. See what works and what doesn't. Notes will come naturally from playtesting. Most versions of games I test for work are ideally tested hundreds of times each version if possible. Obviously this isn't possible for everyone, but I'd imagine you've played enough TSSSF to have some good intuition on set testing and game flow.

This is not to say that a playtest sheet is irrelevant, just not to this extent. To help, our group broke down each element on the playtest sheet and noted why we felt it was relevant or irrelevant to general testing:

Build: Relevant. Obvious reasons.
House Rules: Relevant. Obvious reasons.
Date: Irrelevant unless one is compiling data from multiple versions or possibly over long periods.
Number: For active playtesting, number is irrelevant. If one is playing so much that it affects gameplay they probably shouldn't be playing as this affects good playtesting.
Initial Goals: Point value isn't relevant, but what Goals are active are. You can glean point value from the names and what Goal requirements are is much more relevant to game flow than the point value.
Hand: Irrelevant. This is almost always 4 Ponies and 3 Ships (in my experience at least).
Draw: Somewhat useful but time consuming to track, especially with Pinkie Pie powers. Just some general information on how often digging for many cards to achieve certain Goals is useful and how often this occurs.
Search: Same as Draw, but with digging for answers.
End: This info can be gleaned from other sheet information. Not particularly relevant.
Used: Same as End, but slightly more relevant. What cards were used is much more relevant than the number of cards used. Somewhat like Draw and Search in that many cards used can be interpreted as using powers to achieve Goals, though without knowing which cards were used it's hard to say.
Grid: Irrelevant unless board wipes occur. What cards are on grid are somewhat relevant (like static powers), but not so much number.
Discard: Irrelevant. Information can be gleaned from other data.
Won: As stated earlier, which Goals are won is much more useful information than number of Goals won, though number can still be relevant.
Replaced: Irrelevant. Hardly any Goals refer to this and without knowing board state it's hard to say if they were used to get rid of hand, deal with threats, or deny player's possible power usage.
Duds: Irrelevant. In my experience, this hardly ever happens.
Notes: Extremely relevant for reasons stated above.
Score: Relevant. Obvious reasons.
Goals: Overarching information like this can be relevant, especially as it can be recorded after the game is over. Most relevant is Goals achieved, followed by discarded, then active.
Pony: Same as Goals.
Ship: Same as Pony.

I hope this information will be useful to your research. Please let me know if you'd like any clarification on anything or if you have a newer sheet you'd like my group and I to test out!

Playtest sheet: http://i.imgur.com/lDKZUjG.jpg (Rodeo was the only achieved Goal):