A Pragmatic Theory of Playtesting
Okay, friends! Here's some nonsense that I think.
Internal Playtesting Or External?
The purpose of internal playtesting is to get your game to the point where it's good enough to share outside of your immediate circle of friends. I don't have anything to say about internal playtesting just now.
The purpose of external playtesting is to get your game to the point where it's good enough to publish. This is what I'm talking about here.
The Conversations You Always Have
There are some conversations about Dogs in the Vineyard that I've been having regularly for the decade that the game's been in publication. Most of them are good! "Hey, this game is great! We gunned one sinner down, but made a worse sinner into the sheriff. Why did we do it?"
A few are, yknow, not bad, but repetitive. Every time someone new encounters the game, there's a chance I'll have one of these conversations with them:
"Hey, it's to my mechanical benefit to get fallout. How come I get a benefit, not a penalty, for taking a blow?"
"Hey, I made a town that only went to False Doctrine, and my players were bored. But doesn't creating a murderous sorcerer rob the town of moral nuance?"
"Hey, when the Dogs say that same-sex marriage isn't a sin, doesn't that mean it's not a sin anymore?"
There are some conversations about Apocalypse World, same thing, I've been having them for five years now and no end in sight. Most of them are good. "Hey, this game is great! I thought that my friend Lily would choose the hardholder, and SHE TOTALLY DID! Also Marie violation glove brr, but Ada workspace woo! Woo!"
But every time someone new encounters the game, there's a chance of:
"Hey, when I attack someone but I don't want them to do anything, so it's not going aggro, and they can't fight back, so it's not seizing by force, what move? Like with a sniper rifle?"
"Hey, resetting Hx is borked. You get to know someone better so now you know them worse? What gives even."
"Hey, I kind of hate highlighting stats. It's poo."
And so on.
When you publish a game, you sign yourself up to have the same conversations about it from now on. That's what you do. That's what publishing a game means.
What You'll Never Ever Know
You'll never ever know, I hate to break it to you, but you'll never ever know what people think of your game. Whether they like it or hate it, enjoy it or endure it, whether it's even playable. You won't be there to witness it, and the people who are there, they won't tell you. That information dilutes into the chaos of force, motion, and uncaptured history, it never reaches you intact.
Occasionally you'll catch a glimpse, at a con or something, but your very presence distorts it and makes it unreliable.
All you'll ever know for real is what happens when you're there firsthand, plus the conversations you have.
The Pragmatic Theory of External Playtesting
So then. My pragmatic theory of external playtesting is that you playtest your game to try on conversations about it. You rewrite and redesign until you're having the conversations that you want to have, and then you publish. Or else it never comes together, and you abandon the game instead.
With AW:Dark Age, for instance, playtesting made it crystal clear that if I published, a conversation I would have about the game, from then on, would be "we tried to play, but instead we argued for four hours about how and whether rights worked." I was exhausted by this conversation after just once; I couldn't face a decade of it. So I bagged.
With Midsummer Wood, on the other hand, playtesting made it crystal clear that if I published, I would nevertheless basically never get to have any conversations about the game at all. It just wouldn't take off. I published it anyway, for other reasons; I knew perfectly well what I was getting.
The purpose of external playtesting is to get your game to the point where you're having the conversations about it that you want to have. Then you publish, and those are the conversations that you get to keep having from then on.