How do you determine a campaign's difficulty and balance, and a player's skill level?

[Shield]

I am unsure if this is the best thread under which to post these questions, but they relate heavily to UMC campaign and scenario development, and this topic can always be moved by a moderator if necessary. I’ve been playing Wesnoth for a very long time, and the subject of its difficulty both in mainline and UMC has always been something I’ve wondered about the exact details of.

The main gist of my questions (elaborated in full in the expandable Sections below):

• How is a campaign determined as being balanced or unbalanced?
• How is a campaign’s challenge “level” determined, and what makes the differences between these levels?
• Are a campaign’s different difficulty settings proportional to its general challenge level?
• How can a player tell what their skill level is, and how can a UMC developer tell what level of skill their campaign requires?

Many points in these questions overlap each other in the below expanded points, but that’s part of why I’m including them all in this one post; because their issues are heavily tied into one another and it seems better to potentially reiterate myself here than make a bunch of separate, redundant topics.

How is a campaign determined as balanced or unbalanced?

I already have a good idea of what makes a faction or MP era balanced, and that it’s basically “each faction has an equal chance at beating the other factions in an average match”; an example of this in action was when one of my family members accused me of not playing fair when I’d usually choose to play Undead when she chose Drakes. I proposed that I play Drakes and her Undead to prove that it was a matter of her (lack of) strategic skill instead of unfair odds, and won. The Undead faction’s lack of traits is balanced by their immunity to poison and their submerging abilities, the Orcs’ lack of healers is balanced by their firepower, the Drakes’ high damage output and hitpoints are balanced by them being easy to hurt, every faction has at least one flying or swimming unit, etc. etc.

But what makes a campaign balanced? Does it mean that the player and AI have an equal chance of winning in every scenario, and that that chance shifts to match a player’s assumed skill level in “harder” campaigns? Or is it a combination of other factors? One would presume that the mainline campaigns are examples of what “balanced” campaigns are, but when it comes to what exactly makes a campaign balanced, it’s been strangely difficult for me to find answers.

How is a campaign’s challenge “level” determined, and what makes the differences between these levels?

Every mainline campaign and most UMC campaigns are labelled with a “level” under their description, next to how many scenarios long the campaign is. Accounting for older and newer versions of the game, these levels seem to be: Beginner/Novice/Rookie, Easy, Intermediate, Hard, and Expert. But what exactly do these markers mean in a purely objective sense?

Now, I’ve been playing Wesnoth since about the 1.2 branch. I was a novice when I started, and presumably shouldn’t have had too much trouble with “Novice” level campaigns aimed towards beginners, right? Well, I did! TB, HttT, AOI, TSG, even the Tutorial-- I’d eventually get stuck on all of them, and it was a notable time before I was able to make progress; I could tell they weren’t as hopeless for me as “Expert” level campaigns, but they were still difficult even on the easiest gameplay settings (granted, I wasn’t even 10 years old back then, but I was-- by the definition of the word-- a Beginner). Which brings me to...

Are a campaign’s different difficulty settings proportional to its general challenge level?

Most campaigns not only have a “level” but different difficulty settings you can choose from, too, most often being between two and four of these: Easy, Normal, Hard, and Nightmare. This is very cool! If a player feels like they can’t handle one difficulty setting, they can try an easier one. I’ve always started out with “Easy” mode, and tried harder settings later when Easy no longer has any real challenge for me.

But do the difficulties and easinesses these modes afford scale in proportion to their aforementioned “level”? That is: Is “Easy” on an Expert level campaign easy only for expert players, and is “Hard” on a Novice one challenging mostly only for beginners? Is “Easy” difficulty for an Intermediate campaign considered “Nightmare” for a Novice one? Should it be? Because, like I mentioned above, I’ve always played on easy mode, and even as a novice I still failed the supposedly easier campaigns! When I finally tried TB on its harder setting, I struggled and knew my younger self could’ve never beaten it like that.
I assume that “Normal” difficulty-- as the name suggests-- is how a campaign is intended to be played by its target demographic; some campaigns don’t have an Easy mode at all, and start off at “Normal”! In UMC campaigns, Normal difficulty is often described as the “most balanced” compared to the other difficulty modes by their developers.

And if a campaign’s different optional difficulty settings aren’t meant to be proportional to their defined difficulty level, then how are they determined, and what precisely does that mean for the campaigns and their players?

How can a player tell what their skill level is, and how can a UMC developer tell what level of skill their campaign requires?

Here is where it all ties together. I’ve been playing Wesnoth since I was about 7 and am over 20 now, so I should be an expert by now, right? I have no idea, honestly. Maybe I’m somewhere in the intermediate range? I can play very fast and have improved my strategies immensely over the years, but my actual quantifiable skill remains a bit of a mystery, and I wonder how much of the case that is for other players, too. Don’t get me wrong, I love how Wesnoth doesn’t have a ranking system in multiplayer, and how its community doesn’t seem to have the toxic “get good” philosophy that many game fandoms do. But with campaigns having outright-stated difficulty levels, I have to ask myself how “good” I am at the game, and whether or not I’m ready to play certain campaigns without relying too heavily on luck.

Ideally, a UMC developer should be able to determine exactly how to label their campaign, and use not only feedback, but objective data to gauge its difficulty as well. For example, The Final Exam is made specifically for beginners, and Bitter Revenge claims to be for experts, but I could never get past the first scenario of the former, and have long-since beaten the latter!

A campaign developer (once again, ideally) should be able to put themselves in the shoes of their players so they can set up appropriate challenges for them. Outside (but possibly inside) Wesnoth UMC, I’ve noticed a frustrating pattern amongst certain indie games which I’ve termed “solipsistic difficulty”. Basically, it’s when one developer (or a small circle of developers and playtesters)-- in the process of repeatedly playtesting their game-- begins to find it “too easy” and gradually makes their game harder in various ways in order to keep it challenging for themself; the end result is that the game can feel incredibly unfair or even near-impossible to other players because it was designed to challenge people who are already intimately familiar with it. I think it’s a trap that anyone developing any game for other people should be wary of falling into. It's great to be challenged, but harder=/=better.

Given the existence of Wesnoth modifications like “Unforgiving Hardcore Mod” and “Invisible Enemies”, I know that there are Wesnoth players out there who are skilled at the game on a level I can barely even wrap my head around. Some Expert level campaigns are so challenging even on their easiest settings that I not only think “how am I supposed to beat this?” but also “how is this supposed to be a fun or satisfying journey?” (just my opinions from personal experience, I do not intend to make any specific devs feel insulted just because I didn't have a good time playing the UMC they made for Free)
Like, I feel like if I tried to make a Novice level campaign, I’d have to carefully think about its design and get feedback specifically from beginners to get it to turn out right for them, because I’m no longer a novice player and can’t take into account all the little micro-decisions that beginners make that lead them to lose at harder campaigns anymore; but I certainly recall how saddening it was to be a beginner at Wesnoth, and losing at campaigns that were apparently supposed to be aimed at me.

Maybe I have a different playing style/attitude than the people making UMC Expert-level campaigns and playing on the hardest difficulties, and maybe some folks just enjoy putting themselves through painfully grueling gaming challenges, but I really can’t say for sure! I’m not a mind reader and I don’t make the rules around here. I just think being able to tell what makes a campaign “balanced” and fitting certain difficulty level criteria, and what makes a player best-suited to try their hand at winning them would be great.

In conclusion: Guidelines and suggestions on what makes a campaign balanced and how its difficulty level should be labelled could really benefit the UMC community, in both its developers and players.

I’m not going to ask that the forum moderators or mainline game developers put their volunteered time and energy into making a big new topic post and/or wiki page explaining everything about objective player skill and campaign difficulty (though that would be pretty cool), but I do think that those topics are worth discussing and thinking about to make everyone’s lives easier and their gameplay sessions more fun long-term.
Sorry if this post is kind of long or flounders at all, I’ve just got a lot I’ve been wondering about.

[gnombat]

But what makes a campaign balanced? Does it mean that the player and AI have an equal chance of winning in every scenario, and that that chance shifts to match a player’s assumed skill level in “harder” campaigns? Or is it a combination of other factors? One would presume that the mainline campaigns are examples of what “balanced” campaigns are, but when it comes to what exactly makes a campaign balanced, it’s been strangely difficult for me to find answers.

As you have noted, "balanced" doesn't really mean the same thing for campaigns as it does for competitive multiplayer games. For a campaign, I think "balanced" just means that it is neither too easy (i.e., boring to play) nor too difficult (i.e., nearly impossible to complete). There is a topic which discussed this in more detail here. Basically, the idea is that, in a balanced campaign, the (human) player should be able to complete every scenario in the campaign, the first time playing through the scenario, assuming that the player plays reasonably well and doesn't get a lot of unlucky RNG rolls.

Of course, this is obviously just a rough guideline, and the meaning of "plays reasonably well" will vary depending on the difficulty level of the campaign - a more advanced campaign with a higher difficulty will require a higher level of player expertise. The guideline probably breaks down somewhat at the extremes; for example, I think when playing on "Nightmare" difficulty it's probably not entirely realistic to expect the player to complete every scenario the first time through (but some people might dispute this - see the full discussion for more detail).

Every mainline campaign and most UMC campaigns are labelled with a “level” under their description, next to how many scenarios long the campaign is. Accounting for older and newer versions of the game, these levels seem to be: Beginner/Novice/Rookie, Easy, Intermediate, Hard, and Expert. But what exactly do these markers mean in a purely objective sense?

The in-game help explains it like this:

Campaigns are grouped by level and difficulty. For example, Heir to the Throne is ‘Novice’ level, and is playable at three difficulties: ‘Beginner’, ‘Normal’, and ‘Challenging’. The level of a campaign indicates what degree of proficiency in the game mechanics (such as zones of control and time of day) is assumed. The difficulty indicates how challenging the scenarios will be to a veteran player: at higher difficulties, the obstacles to victory will be higher and overcoming them will require more skillful play. For example, at higher difficulties the enemy may have higher incomes, higher-level units, more castle hexes, and so on.

I think the idea here is that there is more to a campaign's "level" than just the difficulty - for example, Under the Burning Suns has a lot of strange rules (e.g., time of day) so that it's probably best for players who are already experienced with the game (hence it is an "Expert" campaign).

And if a campaign’s different optional difficulty settings aren’t meant to be proportional to their defined difficulty level, then how are they determined, and what precisely does that mean for the campaigns and their players?

I believe that each difficulty is intended to mean more or less the same thing for every campaign it appears in, no matter what the campaign's overall level is.

I don't think there is any official definition anywhere of what each difficulty actually means. The only semi-official guideline that I know about is that, when playing any campaign on its easiest difficulty, each scenario should be winnable even with minumum gold and no recalls.

Ideally, a UMC developer should be able to determine exactly how to label their campaign, and use not only feedback, but objective data to gauge its difficulty as well. For example, The Final Exam is made specifically for beginners, and Bitter Revenge claims to be for experts, but I could never get past the first scenario of the former, and have long-since beaten the latter!

Yes, the difficulty of UMC campaigns tends to vary a lot, and the labels used for the different difficulty options often do not match the mainline campaign difficulties very well.

In defense of UMC developers - I think it's probably pretty difficult for developers to actually estimate what the difficulty of their campaign is. The mainline campaigns have the advantage of a lot more playtesting and feedback, and they have been tweaked and rebalanced for years. That's likely the only way to get an accurate difficulty label which is consistent with other campaigns.

[lhybrideur]

As a novice UMC developer, I will answer that testing by other people is the only true way to balance a campaign and estimate its difficulty.

[forbiddian]

I haven't noticed much correlation between the stated difficulty of the campaign and my perceived difficulty of the campaign. IMO, Heir to the Throne is harder than burning suns/hammer/sceptre of fire even though those are "more difficult" campaigns.

Like gnombat said, it's more about how much knowledge the game expects you to have. HttT has day/night cycles, but it explains how they work and tells you explicitly that your guys are weaker at night, your enemies will be stronger at night, and that you can recruit outlaws. On the other hand, the expert campaign Northern Rebirth just sends you into a cave second mission and your guy says "It's dark in here, I can't see anything!". The expectation is that you understand how the day/night cycle mechanic works.

I want to add that it's impossible to "balance" a campaign with the default settings, especially a longer campaign.

Part of the fun in Wesnoth is leveling up your "team" and getting stronger. I get that, but the carryover advantages compound. For 17g, you can recruit a cavalryman (34 HP, moves 8, hits for 18). For 20g (if available), you can recall a cavalier (64 hp, moves 9, hits for 40, also has ranged 20). Experimentally, a cavalier is about 4x better than a cavalryman in a straight fight and puts all of that concentrated value on 1 hex.

If you balance around people with weak recalls, you allow the people with strong recalls to farm the scenario for even more XP and gold that will carryover to the next scenario. On the other hand, if you require players to have strong recalls, you completely brick the saves of anybody who doesn't.

[Bladget]

I would like to share an external resource that is applicable to the topic. It is aimed at campaign developers and not on Wesnoth project maintainers.

Specifically, it is a generic development method to designing game encounters that I frankly consider profound. [Key Mechanics: Challenge Tuning!](https://www.youtube.com/watch?v=tfO1GWbA1MA) by Runehammer. Warning: 1) the author's stage character is ridiculous, the ideas are serious however; 2) the method was originally designed for tabletop roleplaying - I successfully used it in real-time strategies.

Basically, each game challenge is described with three attributes: damage, disruption and duration. All of those concepts previous participants in this discussion already touched.

Damage is self-explanatory. The more enemies there are, the stronger they are, the higher the damage property of an encounter.

Disruption is more difficult to quantify. Applicable to Wesnoth it would be scenario-specific gimmicks. To my mind comes terrain modification, like cracking ice on a river, or lava, or unexpectedly appearing spiders from a cave, or ability that is suddenly taken away from the player.

Duration is the most interesting to me. Duration describes how important it is for the victory that player responds with readiness and in time. Classic example is to have enemy reinforcements arrive after timer expires. It pushes the player forward knowing that they are limited. This prevents for example player abusing experience in Wesnoth scenarios to build up recall. Or healing party to full during encounter in Darkest Dungeon. When I understood this concept I felt like I grew as a game designer.

There is still the problem as to how apply those concepts to Wesnoth scenarios. This framework makes it easier to reason about the topic both for developer in their mind and in forum discussions.

I am willing to bet that teaching these basics to a new content-maker before they open editor will prevent the most banal mistakes. For example, that putting one hundred enemies on an empty map does not necessarily produce challange, since there is no disruption and no duration control. Or that making tight corridors is not enough to create disruption.

I encourage you to endure the original recording.