thread: 2011-02-17 : Ben Lehman: Playtesting: Stop
On 2011-02-19, Joel wrote:
This post is chock full of fascinating stuff for me, that hits the core of my experience and angst with RPG self-publishing. I'll note for the record that the confrontational, even accusatory, tone is difficult for me to engage with, Ben, and I think I AM one of the people you want to communicate with...? Like, countering harmful moralizing with equally fierce counter-moralizing isn't too productive, maybe? Still and all, I AM going to attempt to engage with you because these issues are important for me and I want to learn. If that validates your decision to use that tone, then so be it!
I'm going to focus on telling my own story about my experiences with the world of Indie design and publishing. I entered the arena of game design in Game Chef 2008, and produced a game that I liked, and wanted to continue developing. I worked it up into an extremely ugly ashcan for the Portland area's Gamestorm convention the following March. I had exactly one playtest beforehand. I ran one game of it at the convention. I played a several-session game with friends after the convention, plus a semi-drunken one-off game with coworkers. Some folks bought the game at the con, ran it, and emailed me feedback. I ran it at Go Play NW that year ('09). One other person bought the game, played it with their group, and gave me feedback. I ran it for Gamestorm '10. I haven't played it since.
That game went on the backburner when I got a new bolt of inspiration for a game, one dearer to my heart, in fall '09. I struggled with implementation for several months, considering and discarding several procedural ideas. I had nothing playable. Then in March '10, as Gamestorm was once again rolling around, I had a eureka moment where one key element presented itself and everything fell into place. I worked furiously to write down the resulting game, which had been rattling around in my brain for months. I playtested it the week before the con, and arrived at the 'Storm with a new (also pretty ugly) ashcan. I didn't have any scheduled games for it, since I didn't know it would be finished, but it demoed well, and some friends organized a spontaneous game of it with no prompting from me, which I then got to watch.
I played the game with friends several times over that spring and summer, and ran two games at GPNW. I got myself a copy of InDesign, and bugged friends to give me a crash course in layout, and produced a very pretty (if I do say so) handmade book to take to PAX Prime in August. I ran some games there, and it sold well. I've been selling a steady trickle ever since, an d playing a game here and there. I consider the game complete. If I want to refine it, it won't be in the form of revision or rewriting, but rather low-cost companion booklets that expand play options, recontextualize the procedures, and explore the game's themes. Similar to the Sorcerer supplements.
meanwhile, my other game is due for a heavy revision (and, given newly acquired skills, massive prettification) before it can move forward. I've learned a lot since its inception, both through playing it and my other experiences other play and design. I aim to get under the hood again soon.
++++++++++++++++++++++++++
So, why the extended anecdote? Because my story IS my argument. I don't see any profit in my going "Hey, Ben's right!" or "Aagh, Ben's wrong!" Instead I'll look at how my personal story matches up with his claims.
First, when I began designing, there was definitely a Playtesting Shame Culture circulating around this corner of the internet (for a certain value of "this corner"). There was a lot of polemic about the moral responsibility of the designer to playtest heavily, about making sure you had "blind" playtests, about picking the customer's pocket, etc. I haven't heard a ton of those polemics lately, but my gut says it's down to a dull roar, but the Shame Culture still exists. It had a huge impact on my. Not that it impacting my design practice per se; it just helped me internalize a load of guilt about not being able to "properly" playtest, and fear that I'd be found out and decried on Story-Games or the blogoblags or wherever. I've had that fear right up until...well, up until yesterday.
When I read Ben's article.
I felt such relief to hear a I have been terrified that if anyone in the Indie crowd knew how "under"-playtested my games were I'd be shunned, called out, condemned. That may have been melodramatic on my part, but I don't think the fear is unfounded. Now I feel it's important enough to take the social risk and lay my design journey bare.
The fact is that for both games, I playtested as much as I was able in light of my own time and energy and the pool of willing participants. If I could have played the ashcan versions twice as much, THREE times as much, before writing a revision, I would, but not so I could "spot the holes" or test the mechanics. More so I could be comfortable and knowledgeable with how play of the game feels, so that my instincts for running it and playing it are finely honed, and I can reflect that in the text. Because yeah, game-mechanical interactions CAN'T be tested in a sufficiently large statistical sample. It's like saying "I rolled percentile dice three times; the most likely outcomes are 17, 52 and 86." But getting a game to fit you like a second skin, so you can trust in the emotional experience of play—that you can get through playtesting. I'm fairly happy with where I am with that on my finished game, but more is always nice.
It helps that my games, especially the completed one, are relatively simple mechanically. Still, my unfinished project, to be sure, has some kinks to work out in the numbers department. But that's definitely a work of tinkering in solitude. I'm not going to tweak the numbers, play it with friends, tweak 'em again, play again, and so forth. That really WOULD take ten years. The more talky rules, though, they definitely bear testing with people. The "OK, so after you roll that, person A says this, and includes a bit about B, and..." stuff needs testing, not because of "balance" or anything, but to see how it feels coming out of people's mouths. Again, it's about getting comfortable with the game's emotional core.
++++++++++++++++++++++++++
So there's where I stand with respect to all that. Let's see, what else? Oh yeah, trading on social status. Totally true! That's not a dirty thing, just a thing. So I met Ben at Gamestorm '08, played his Bliss Stage demo, totally wanted to know him and be cool in his book. Before that, I found Anyway, then the Forge, totally wanted to know Vincent and Ron and be cool in their books. And so on. At first this was just a matter of playing cool games and finding smart ways of talking about them online. Later, with the impetus of Game Chef,it also became a matter of designing good games and getting them played and talked about. So far, they haven't been talked about much, but they've been purchased some, and played some, and that's something. In any case I totally want to be thought smart and cool by a bunch of internet people (including you folks!) on account of my game design, and that's just fine.
That sort of thing can turn ugly, yes. Status is one of those things where you can't do it JUST to do it; you need an authentic statement to make, or some sort of personal integrity, or it IS kind of hollow and unsavory. And a delicate touch is needed; more delicate than I'm being here, I suppose. I deliberately haven't mentioned the titles of the games I designed, because that sort of status-fishing feels too overt for this discussion. But yes, I'll readily admit that I seek status with the self-publishing crowd...why wouldn't I? I hope that I'm making authentic statements in the process.
I will admit that I hadn't picked up on the strategy of milking playtesting for status; I was more concerned with getting finished and felt my status was fleeting until I had a solid game. Maybe I'm naive or something. But I can totally see that pattern. Especially now that I do have one finished game, it's easy and tempting to be like, "oh, this game I've got in development blah blah blah..." when maybe "in development" means a paragraph scribbled on a scratch pad a year ago.
OK I've been typing for a long damn time so I'm just going to stop. I hope I'm coherent here. I didn't know how else to approach the topic; I'm still trying hard to process all this info flowing at the speed of Internet.
Peace,
-Joel
