Playtesting discussion

16 posts / 0 new
Last post
I'm copying from another thread. These types of discussions have taken place in the past.

57231238 wrote:
56892538 wrote:

Are you talking WotC or RPGA?
I would be surprised if playtest critics of RPGA games are not used, unless the request is something that can't really be done due to a lack of resources (xp budgets etc).


Benird is talking RPGA, and I know from talks around the tables this last weekend that the local play testers for Arts were very unhappy with the final module as an example. (No I'm not sure that is one that had Benird's name on it for PT.)




Playtesting is a weird beast. It can produce really different results. For CORE1-14, for example, I ran four playtests and had three of the four say they had different least favorite and most favorite encounters. Of the first two combats, usually one was hard for a table and the other easy, but it varied greatly and almost unpredictably. The feedback was often contradictory. The feedback was also really valuable. Every single table contributed something. But it was impossible to please everyone, nor would it have been good to please any one playtest. In the end, the author has to use their judgment, often in discussion with the admin in charge.

For a recent adventure I have run 4 playtests... off the first draft! Again I saw really different results across each table. Each one has contributed very important information, even though the results often conflict. What was incredibly difficult and unfair for one table was a cakewalk for another table, with suggestions for increasing difficulty.

For another adventure I provided feedback on making the encounter more challenging. It was a good encounter but needed some tweaks to be a little tougher and more fun. The author responded that the other playtest had a TPK in the same encounter.

In the end, as playtesters we should understand the variability of the play experience. The final mod will not contain everything we ask for. It may even be the opposite.

I do recognize that sometimes things are just not used and it is hard to know why. I've suggested changes to text for clarity or given warning about likely problems and seen the mod come out without nearly any of the suggestions - and then the mod when released has the issues we raised. Certainly things aren't always perfect. But, this has not happened of late.

Now, I do feel that as part of a playtester you are sort of signing up to keep the playtesting confidential. For example, it would be bad form for me to badmouth the bad experiences I've had. I see it as my job to keep that to myself.

Having said that, I'm curious about the experiences (in vague terms if necessary) others have had and whether there really is a feeling that playtesting is not working well.

Follow my blog and Twitter feed with Dark Sun campaign design and DM tips!
Dark Sun's Ashes of Athas Campaign is now available for home play (PM me with your e-mail to order the campaign adventures).

It may not be fair for me to say, as I know some of the authors I've playtested for enough to comfortably contact them personally and engage in prolonged conversation, but I feel that most of the playtest feedback I've given (probably approaching the double digits in adventures now) has been attended to in some way by the author and/or editor; if there was not a direct change based on our feedback, there was usually some mention of a situation we'd encountered in the adventure in the tactics.
John du Bois Living Forgotten Realms Writing Director, Netheril story area Follow me on The Twitter: @JohnduBois Follow my presence on The Intertubes: johncdubois.wordpress.com
I would be curious to know what level playtests where one party had a TPK and the other thought it too easy were. I think I've posted before that my suspicion is that we should expect dramatically more variance at higher level than lower level in 4th edition mods based on party composition, character composition, and luck.
I would be curious to know what level playtests where one party had a TPK and the other thought it too easy were. I think I've posted before that my suspicion is that we should expect dramatically more variance at higher level than lower level in 4th edition mods based on party composition, character composition, and luck.


My experience would indicate that it's more likely to be party composition than anything. I've run adventures at all levels where players of roughly similar ability levels and luck but vastly different party balance had vastly different challenge levels.
John du Bois Living Forgotten Realms Writing Director, Netheril story area Follow me on The Twitter: @JohnduBois Follow my presence on The Intertubes: johncdubois.wordpress.com
I would be curious to know what level playtests where one party had a TPK and the other thought it too easy were. I think I've posted before that my suspicion is that we should expect dramatically more variance at higher level than lower level in 4th edition mods based on party composition, character composition, and luck.


My experience would indicate that it's more likely to be party composition than anything. I've run adventures at all levels where players of roughly similar ability levels and luck but vastly different party balance had vastly different challenge levels.



I agree, but party composition also seems more likely to make a difference at high levels. For example, there is more difference in the capabilities of a 9th level wizard and a 9th level druid than there is between a 1st level wizard and a 1st level druid. And with a greater possible variation in character abilities, there will be greater possible variation in party compositions. Additionally, there is more difference in the capabilities of higher level monsters (level 1, for example, is not supposed to feature flying artillery; by level 9, flying paragon artillery monsters are definite possibilities). With greater variation on both the player and the monster side, there is more room for specific character and monster configurations to produce unusual results.
I've worked extensively with both Teos and John on playtesting. In fact, I will no longer write an adventure without having both of them run playtests for me (that speaks to their level of commitment and dilligence as players, judges, and overall campaign contributors).

Interpreting playtest results is far more of an art than it is a science. I've learned a few guidelines from my experiences that I can share, however:
  • Direct player feedback is mostly background noise that can clutter playtest feedback. In every encounter, one player thought the encounter was too challenging or too frustrating because he took the biggest beating, while another player found it too easy because he was unscathed. The most valuable feedback comes directly from the table judge who has collected individual player feedback, aggregated it, filtered it, and combined it with his own observations (the judge is usually the most objective).

  • Playtest results on their own are not enough to gauge whether or not further tweaking is warranted (or the level of tweaking). It is often necessary to understand how those results came to be. If an encounter caused a TPK (or was too easy), the playtester needs to provide party composition, initiative results (particularly at higher levels), and luck impacts so that the author can make a more informed decision.

  • Know your audience - understanding the group's mentality is important. The judge often understands the types of players he's running a playtest for (in fact, you should only work with playtest teams that play together frequently). If the players complain that the module didn't have enough role-playing opportunities or that the story wasn't developed well but you know that they're the type that really favors those things over combat, their feedback may not be as critical (but may still be valid). If a bunch of combat junkies complain about the same thing, then you've got a big problem on your hands.


As an example, we recently conducted 7 playtests of CORE2-1 Killing the Messenger. The module has so many options (there are 24 unique adventure paths that the party can take), that numerous playtests were warranted. Two tables that played the same skill challenge had the same complaints about it. This was an easy scenario: we completely re-wrote the skill challenge. The next table playtested the re-written version and loved it. Of the first 3 tables to play a certain combat encounter, one had a TPK, one found it to be too easy, and one said it was dead on. We made one small change to the combat and had it playtested twice more. Both tables said it was dead on. I know what caused the TPK and feel that the risk of it happening again is sufficiently mitigated. These are just two examples - Nick and I made numerous other changes based on playtest feedback. We've received some other random spots of feedback here and there that seemed to be too random and we chose to disregard them. I'm certain that we made some mistakes in disregarding certain feedback, but I'm also certain that we made some very positive changes. We can only get better with experience (and by following lessons learned). We'll see how it turns out!
Dave Kay LFR Writing Director Retiree dkay807 [at] yahoo [dot] com
I would be curious to know what level playtests where one party had a TPK and the other thought it too easy were. I think I've posted before that my suspicion is that we should expect dramatically more variance at higher level than lower level in 4th edition mods based on party composition, character composition, and luck.



I will say that in two of the cases where it was a "cakewalk", it was a well balanced party with 6 fairly-to-highly optimized characters playing high tier, and pulling out all the stops.  Level 13s for an 11-14, and 16s for a 14-17.

In one of them, I think it came down to the PCs having just the right powers to counteract the most difficult parts of the encounters.

In the other one,  in two of the combats, the strikers were able to kill the elite monster before it even got a turn, and then cleaning up the rest of the combat was fairly easy.
I would be curious to know what level playtests where one party had a TPK and the other thought it too easy were. I think I've posted before that my suspicion is that we should expect dramatically more variance at higher level than lower level in 4th edition mods based on party composition, character composition, and luck.



The ability of parties to mitigate the controlling element of an encounter can be huge. Movement and range can have huge effects. My warlock often contributes in hard-to-notice ways via two burst 20 powers (let alone forced movement, teleportation, ongoing damage to several targets, etc.). Between cursing two things a round and two quickcurse rods, the damage can be high and can be to targets that might otherwise remain untouched by a party with long range. My other paragon is an Inspiring Warlord. I can help people hit the really high defenses. The choice of which I play can have a very big difference in outcome, and I'm just 1/4 to 1/5th of the equation.

I can't add a whole lot to Dave's comment, other than "thanks and likewise" and to say that 7 playtests is amazing. That takes a lot of work. Each set of playtests can really require review (especially if the submitter is a wordy fool like me). You can lose a bit of energy after the first draft is in and you have to re-inject it to get those edits into the mod for the second/final draft. It takes real work to bring in that level of quality.

Now, time to get ready for a playtest of someone else's mod. Then tonight to pull the feedback together... oh, and work on pulling in the feedback to my own authoring effort...

Follow my blog and Twitter feed with Dark Sun campaign design and DM tips!
Dark Sun's Ashes of Athas Campaign is now available for home play (PM me with your e-mail to order the campaign adventures).

I know our group here that does the play tests consists of about 12 players all reasonably experienced in 4th ed. We always work towards a balanced party and try for 5 players to get the most out of the intention of the mod (we all know the difficulties with scaling up and down and play testing doesn't necessarily do anything to fix this).

We'll also try to get the average level (eg 2 or 3 for a 1-4) and will play at high but also experience the low level encounters to get a feel for them too. We run through the entire mod and experience every encounter.

Our characters are a mix of optimised and average characters but we try not to have any underpowered characters because these provide a bias that we don't feel that the LFR community  sways towards and underpowered characters should not be nursed through adventures where as average characters should feel comfortable in adventures.

As a group we play through the mod and play it at a reasonable pace to test the timing of the mod. We allow for discussion of encounters and general chatter that will happen in a mod generally. If it fits in the 4 hour time slot still then we consider it to run to time.

We also scour DCs and combat challenge and provide a unified feedback based on our experience. Some more experienced players will also take the mod after the play test and read it themself and provide feedback based on rules discrepancies, "fun" for an average party feel, etc.

And finally we provide text corrections, additions, etc, that we feel helps to tell the story.

I know the group recently did review a mod (I wasn't on this one) and provided feedback that I feel would have improved it greatly. Unfortunately it seems that most of their feedback was not incorporated. When this mod was released I played it and couldn't believe that our group play tested it because of the state it was in after release.

We take a lot of pride in our playtest role and really do want to work towards high quality fun mods. I understand that there's a lot of feedback that come in from mods but IMO if a group comes back saying that an encounter was not fun and gives reqasonable reasons then this should be kept in mind when releasing mods.

Having said all this I'm looking to my most recent play test mod coming out in the 2-1 round of mods because I had a lot of fun, even if I know that some of my comments won't be incorporated. I have received feedback from the reviewer and where our opinions clash and I have been swayed to his way of thinking.

I hope that other playtest groups treat playtesting like we do, it really is the only way authors get to know if their mods work or not.


As my name appears in the first post I thought I best drop in and say "Not me guv" - I'm not actively PTing for anything.

Mind you this does remind me I'm waiting on feedback on the MYRE Benird
I've got it flaged to go through, I'm on holidays next week, I'll be reading it then
As an example, we recently conducted 7 playtests of CORE2-1 Killing the Messenger.



What is a typical number of playtests for most mods? I love that they are being so well shakendown and that I have been able to be involved with doing so but, was surprised by that many.
I generally use 2 or 3 playtests. I often do not have the time  for more.

Gomez
As an example, we recently conducted 7 playtests of CORE2-1 Killing the Messenger.



What is a typical number of playtests for most mods? I love that they are being so well shakendown and that I have been able to be involved with doing so but, was surprised by that many.


CORE2-1 is... special. As one of the playtesters, I can assure you it required more than normal due to the nature of the adventure.
John du Bois Living Forgotten Realms Writing Director, Netheril story area Follow me on The Twitter: @JohnduBois Follow my presence on The Intertubes: johncdubois.wordpress.com
As an example, we recently conducted 7 playtests of CORE2-1 Killing the Messenger.



What is a typical number of playtests for most mods? I love that they are being so well shakendown and that I have been able to be involved with doing so but, was surprised by that many.



I usually run a playtest myself after finishing the first draft but before submitting it for review (mostly as a sanity check to make sure my kooky ideas work). I then spend a few days making edits based on that playtest and send the adventure out.

Once the adventure is being reviewed, I email out the adventure to a few friends (usually Teos, John, and/or Alan) to run playtests for me. Once playtest feedback and reviews from my editor are in, I create a final draft and send it out for one more playtest before submitting it as final to my editor.

This means I run anywhere from 3-5 playtests (1 before submission, 1-3 during review, 1 after review). If the adventure is more dynamic and complex than a typical adventure, I may schedule extras.

There's a lot of value in having someone other than you run a playtest of an adventure you wrote - authors tend to run mods based on their intentions and thus miss their own mistakes or confusing secions.
Dave Kay LFR Writing Director Retiree dkay807 [at] yahoo [dot] com
As an example, we recently conducted 7 playtests of CORE2-1 Killing the Messenger.



What is a typical number of playtests for most mods? I love that they are being so well shakendown and that I have been able to be involved with doing so but, was surprised by that many.




Almost all of the WATE adventures have had 2 playtests before we send them to our global for his first review. Most have 3 playtests before then.  Depending on his comments we might have another one or 2 playtests or occassionally our gloabal will schedule them. The most any of the adventures have had is 7. 

We instruct our authors that if they run a playtest to have someone else be the DM and to observe and take notes. As someone else said authors tend to run what is in their head not what is on the page.

 I am always looking for more playtest groups. 
Sign In to post comments