Scenario Grade [Split]

Brainstorm ideas of possible additions to the game. Read this before posting!

Moderator: Forum Moderators

Forum rules
Before posting a new idea, you must read the following:
dragontamer
Posts: 24
Joined: March 28th, 2009, 11:56 pm

Scenario Grade [Split]

Post by dragontamer »

I don't doubt the level is beatable. I've beaten the level, and I know what I can do to make it easier in the future. Nevertheless, This scenario has gotten harder from 1.4 to 1.6. Many of the comments in this thread are talking about how hard this level is, and no doubt about it: it has gotten harder.

Perhaps the correct solution isn't to give the player an advantage. I guess so. Nevertheless, it is clear to me that there is a problem with this level. It is the hardest level in the early game, and remains the hardest level for many levels to come. Many strategies recommended here are not strategies developed from intelligence, but developed instead by an oracle. An entity who knows the future of the game. You cannot expect a first-time player to know "X unit BY (before) this scenario" on his first playthrough.

--------

Let me start with what I believe is the root of the problem. I believe this level specifically has a real possibility of working the beginner / intermediate player into a situation of far greater difficulty than the designers hoped, especially if they don't have enough tier2 / tier3 units ready at this point.

Let it be known that I fundamentally disagree with the "just tell them to reset" option. But if you are going to force the player to start from scratch, then at least have the game tell him that there is something dreadfully wrong. In Fire Emblem: Blazing Sword, you were rated chapter-by-chapter on the amount of gold, exp, and so forth. While this wasn't actually useful in that game... it can be useful in this one. A chapter-by-chapter rating system would indicate to the beginning player "Oh crap, maybe I should have spent more time training this last chapter". So for example, going into Seige of Elensefar with maybe... two Red Mages, and nothing else, then give a big fat "F" rating to their "exp", letting them know that their unit recall list is far weaker than it should be.

Especially for the beginning player who does not understand the importance of keeping his units alive / raising units... this should be of great help. It also allows the player to compare himself against the best (ie: A-class or S-class players). As a side-advantage, it provides the player a hint of what is to come. A player who is beating levels with low ratings can expect that harder levels are to come, and perhaps he should focus on stocking experience or gold for those harder levels. It can be a vague "Oracle" figure for the player.

EDIT: I made a minor edit to clarify a sentence.
Last edited by dragontamer on June 13th, 2009, 12:55 am, edited 1 time in total.
User avatar
Turuk
Sithslayer
Posts: 5283
Joined: February 28th, 2007, 8:58 pm
Contact:

Re: Scenario Review: The Siege of Elensefar

Post by Turuk »

dragontamer wrote:In Fire Emblem: Blazing Sword, you were rated chapter-by-chapter on the amount of gold, exp, and so forth. While this wasn't actually useful in that game... it can be useful in this one. A chapter-by-chapter rating system would indicate to the beginning player "Oh crap, maybe I should have spent more time training this last chapter". So for example, going into Seige of Elensefar with maybe... two Red Mages, then give a big fat "F" rating to their "exp", letting them know that their unit recall list is far weaker than it should be.

Especially for the beginning player who does not understand the importance of keeping his units alive / raising units... this should be of great help. It also allows the player to compare himself against the best (ie: A-class or S-class players). As a side-advantage, it provides the player a hint of what is to come. A player who is beating levels with low ratings can expect that harder levels are to come, and perhaps he should focus on stocking experience or gold for those harder levels. It can be a vague "Oracle" figure for the player.
So what if the player does not have the exact units listed in order to beat the scenario, then they just get the F and do not even attempt it?

How do you judge which units are necessary at the minimum to win each scenario to provide the rating? Who is going to go through and test this out for each difficulty level for each scenario?
Mainline Maintainer: AOI, DM, NR, TB and THoT.
UMC Maintainer: Forward They Cried, A Few Logs, A Few More Logs, Start of the War, and Battle Against Time
dragontamer
Posts: 24
Joined: March 28th, 2009, 11:56 pm

Re: Scenario Review: The Siege of Elensefar

Post by dragontamer »

I think I see where you're coming from Turuk. From a practical point of view, this will take work. But if "hard work" is the only problem, then I can take it up. (I have a bit of C++ and Python experience). Course, the first problem is if you guys think it is a good idea or not.

From an implementation point of view, I would simply add the tiers of the recall list -1. For example, if the recall list is composed completely full of only tier 1 units (Elvish Fighters or whatever), then the computed value is zero. If there are five tier 2 units, then the computed value is 5. If there are five tier 3 units, the computed value is 10.

This single value is then compared to thresholds listed in the scenario. The scenario writer will (optionally) define the rating system, this way current Wesnoth scenarios will still work in the new version. A similar thing can be done with gold. Grades will be S, A, B, C, D, and F.
User avatar
Turuk
Sithslayer
Posts: 5283
Joined: February 28th, 2007, 8:58 pm
Contact:

Re: Scenario Review: The Siege of Elensefar

Post by Turuk »

That system will only be effective for a limited number of circumstances, given that it is based on the level of units. Not only will a player's leveled units be affected by heroes (some campaigns have a fair number that come and go), but having units of a certain level does not make a scenario easy to beat.

I'll give you five Iron Maulers and have you go attack an enemy army full of elves. They can even be level 1. The player will still lose. Now if the player had leveled up other loyalist units in his army, he would be fine, but up until that point, he went with HI and the others died. By your system, he may pass a rating that tells him he has enough leveled units to go after the scenario, but there is a high chance that his Maulers will get mauled.

It is more about what units that a player brings to a fight, then the level. Do not get me wrong, having leveled units is important, but the wrong leveled units can be as much of an issue as the lack of them.
Mainline Maintainer: AOI, DM, NR, TB and THoT.
UMC Maintainer: Forward They Cried, A Few Logs, A Few More Logs, Start of the War, and Battle Against Time
Jozrael
Posts: 1034
Joined: June 2nd, 2006, 1:39 pm
Location: NJ, USA.

Re: Scenario Review: The Siege of Elensefar

Post by Jozrael »

Well if you wanted to get REALLY complicated, you could say:

Has at least 2 of the following 3:

Total of 5 levels of levelled units (defined above) in the mage line.

At least 10 levels of levelled units.

At least 300 gold carried over.

Fulfilling each of those raises your grade.

Etc. etc.
User avatar
Turuk
Sithslayer
Posts: 5283
Joined: February 28th, 2007, 8:58 pm
Contact:

Re: Scenario Grade [Split]

Post by Turuk »

True, but what if a player could really get away with only having one of the following, as his play style may differ from the person who did the grade for that scenario? You could have enough gold to offset needing certain units by recruiting more, or you could have just enough of the right types of units to not need to meet the recommended gold for the grade.

Thus players are being told they should not attempt a scenario by receiving D's or F's when they might be able to complete it. It just seems that there are many different ways to approach a scenario at times and this is effectively outlining how a player should play, which units he should be using, and how many of each he should have. Instead of there being multiple solutions to getting through a scenario, there is now one right way that will get a player an A, and other ways that may be effective but will have the player viewing a D or F.
Mainline Maintainer: AOI, DM, NR, TB and THoT.
UMC Maintainer: Forward They Cried, A Few Logs, A Few More Logs, Start of the War, and Battle Against Time
User avatar
Simons Mith
Posts: 821
Joined: January 27th, 2005, 10:46 pm
Location: Twickenham
Contact:

Re: Scenario Grade [Split]

Post by Simons Mith »

Perhaps think in terms of grading on a +5 to -5 scale instead of A to F. +5 means you have much more resources than the designer anticipated, so the scenario should be very easy, -5 means much less, so the scenario will be harder. A balanced campaign [for a given playing style] will hover at around +1 to 0, I'd expect, with the occasional -1.
 
Jozrael
Posts: 1034
Joined: June 2nd, 2006, 1:39 pm
Location: NJ, USA.

Re: Scenario Grade [Split]

Post by Jozrael »

Sounds great. For every criteria they meet (specific units, total levels, gold, something else, leader level, idk), they get + or -.
dragontamer
Posts: 24
Joined: March 28th, 2009, 11:56 pm

Re: Scenario Grade [Split]

Post by dragontamer »

Well, if you want a robust system... :hmm:

Anyone see the "Starcraft Lectures" ?? Anyway, the specific one I'm thinking about is this one:
http://www.youtube.com/watch?v=XAqbrRT6 ... annel_page

No need to watch all of it. It takes like 10 minutes for him to explain it to the class. Plus, my idea is only loosely based on it. (If you do want to watch it, ~9:20 minutes to ~13 minutes is the quick example of the math at work)

If we are allowed to fudge the numbers a bit, we can calculate the Elvish Fighter efficiency vs Spearmen as follows: Fighters deal 20 * (1-50% defense) damage per turn, and Spearmen deal 21 * (1-60% defense) expected damage per turn. Fighters are killing ~.27 Spearmen per turn, and Spearmen are killing ~.25 of a Fighter per turn (divide by their HP).Then, killed Spearmen per turn / killed Fighters per turn == Efficiency of Fighters vs Spearmen in Forest vs Forest.

Yeah yeah, I'm screwing with averages here, but the idea is not to build an AI, but a somewhat reliable ranking system that is also easy to program. We can calculate that every Fighter is expected to kill 1.09 Spearmen for Forest vs Forest matchup. Throw in some heuristics for Time of Day and a better heuristic for what kind of land they'll be fighting on and you've got a rating system. (Weighted average between day/night/dawn-dusk ??)

For example, ignoring time of day and assuming Forest for terrain... if the player has two Elvish Heroes and enough gold for one more Fighter, and the opponent is expected to build 10 Spearmen... we can calculate that Heros are expected to kill 2.7 Spearmen and the one Fighter is expected to kill 1.09 Spearmen(again, in a Forest). This would clearly indicate a bad rating. (Say... 6.49 expected Spearmen Kills, which is below the 10 Spearmen expected to be recruited).

A more complicated example? Lets say there are 4 Spearmen and 6 Heavy Infantry. Heroes can kill say, 3 HI and Fighters can kill 1.2 HI. (I pulled these out of my ass, not calculated) I believe a Greedy Algorithm ought to be optimal (not proven, but seems to be a good heuristic anyway and easy as hell to program), so first all the Heroes try to kill the HI, then the one Fighter tries to kill the Spearmen. 2.8 Spearmen are left, so you still get a poor rating. If instead 3 Champions were available, and say they can kill 4 HI and 3.5 Spearmen, then the calculation will go from the reverse. 6 HI go up then 4 Spearmen go up against the 3 champions, leaving .9 Champions. A narrow win for the player, so maybe a C rating or something. These values need to be tested of course >_<

In Summary:
1. Use heuristics to calculate the "matchup efficiency" of the player's units vs the enemy's units.
2. Simulate the battle using the efficiency as a base.
3. Compare the results of the simulated battle with values listed in the scenario and grade the player.

Just a general, still "quick and dirty" calculation to determine whether or not an army is efficient. It will take into account both the power of units, as well as how well they're rated against the army the enemy is going to use.

EDIT: I also realize many questions are incomplete. Such as when to use Ranged vs Melee when calculating the efficiency? I don't think there is a need to have a rigorous system that is a 100% correct test, it just needs to be "good enough" that the rankings are not useless... this way, the player takes that D or F rating seriously.
Velensk
Multiplayer Contributor
Posts: 4002
Joined: January 24th, 2007, 12:56 am

Re: Scenario Grade [Split]

Post by Velensk »

...... I hope I don't need to tell you that your system really dosn't make sense and dosn't adjust to the scenario.

For example, my campaign has a scenario where the player takes a team of about 25ish characters (not very many lvled up, but some)+ however many crystals they can afford, and walks through over a hundred enemies (levels 1-2) by abusing teleport and the fact that you're fighting in a cave. In terms of total efficency, as long as you win you're guaurenteed to get at least a 5 which would give you an A by a standardised rating system. However the truth of the matter is that if you lose more than a couple non-crystal units, you got a C at best for that scenario, and if you lost many you did poorly despite having destroyed a force many times more powerful than yours.

Also I'm realy not sure that you've thought out how these values are calculated. It is very common to attack a person at a range at which they cannot retaliate effectivly thus your system must take this into account. Also I'm not sure when you're thinking this calculation would occur. Is it simply a calculation based on what you recruited and what died, a calculation based on each individual engagement, or is it based on killing blows. You'll never see anything as basic as just elvish fighters vs spearmen. How would you work an engagement where all the various low level elves +random leveled up units engages Lvl 2 orcs+undead? If you impliment a greedy algorithim you would not be able to simply pair up unit types without duplicates or without leaving some units completely unobserved.
You would end up with something on the lines of:
Elvish fighters kill .5 orcish warriors, .6 orcish crossbow, .6 orcish slayer, .7 skelotons, and .2 orcish warlord,
elvish archers kill .6 orcish warriors, .4 orcish crossbows, .6 orcish slayers, and .3 skelotons, and .3 orcish warlords
elvish shamans kill .1 orcish warriors .1 orcish crossbow .1orcish slayer, and .4 skelotons, and .01 orcish warlords
elvish hero kills 1.1 orcish warlords, 1.3 orcish crossbows, 1.2 orcish slayers 1.4 skelotons and .6 orcish warlords

A player goes into battle with 4 fighters, 4 archers, 2 shamans, 2 heros. And fights against a warlord, 2 warriors 2 crossbows a slayer and 3 skelotons. In the end the player kills one of each of the orc types and two skelotons. The player has lost three of his fighters, one of his shaman, and one of his heros. One of his archers leveled up, one of his fighters leveled up before dieing. The scenario then ends because he killed the enemy leader. What was his rating?

Then there are scenarios where the goal isn't simply to defeat your enemy, or scenarios where it is far quicker/more efficent to run a leader assassination with a lancer/silvermage squad, but would not get you all the kills that fighting it out would. Your system does not seem to provide any penelty for losing loyal or unreplaceable units, or the map. For example, on the final scenario of HttT yeah you'll have an impressive number of elves and dwarves who could get 60% or 70% defence in the right terrain, however there is pretty much no way to avoid a simple messy slugout in the field. Assuming your algorithem is simply based on comparitive kills, there should be pretty much no times that a scenario does not give the player a B-A rating because pretty much all scenarios send overwhelming forces at the player and tell them to deal with it, even when the scenario isn't at all hard this happens (statisticaly at least).

I do not think you could find a simple algorithim that truely explains how well a player did without attempting to code one into each individual scenario. I also think that if you implimented the algorithim as you said I do not think players would take it very seriously. Personaly I think that if this were to be implimented (and as a campaign designer I hope it isn't) it should be based on gold, leveled up units, and triggers (so that you can reward a player for finding hidden stuff), and it should be set by scenario.
"There are two kinds of old men in the world. The kind who didn't go to war and who say that they should have lived fast died young and left a handsome corpse and the old men who did go to war and who say that there is no such thing as a handsome corpse."
dragontamer
Posts: 24
Joined: March 28th, 2009, 11:56 pm

Re: Scenario Grade [Split]

Post by dragontamer »

A player goes into battle with 4 fighters, 4 archers, 2 shamans, 2 heros. And fights against a warlord, 2 warriors 2 crossbows a slayer and 3 skelotons. In the end the player kills one of each of the orc types and two skelotons. The player has lost three of his fighters, one of his shaman, and one of his heros. One of his archers leveled up, one of his fighters leveled up before dieing. The scenario then ends because he killed the enemy leader. What was his rating?
I'm not exactly sure you understand my system actually. The Scenario Grade system is not supposed to answer questions like that.

Going on with your example, the player only has an Elvish Marksman in his recall list. Everyone else is at tier 1 or dead. Here's the question: How well is one Marksman supposed to work for the next chapter? If in the NEXT chapter, the player is expected to do well with only tier 1 units (or if his single Marksman is going to be very effective / efficient), then he'll recieve a high rating.

If on the other hand, the NEXT chapter you fight Revenants and Bone Shooters, then you'll receive a low rating at the end of THIS chapter. If the player instead had Sorceresses at the end of _this_ chapter, then he'll receive a higher rating. (because Sorceresses do well vs Revenants and Bone Shooters, or at least better than Elvish Marksmen)

Basically, the system is supposed to fill in that role of oracle, especially to the beginning player. It is supposed to predict the outcome of the battles to come, not rate the player on today's battle. Nor is it supposed to measure kills, deaths, loyal or non-loyal units.

---
EDIT: Or from a programmers point of view, here's the function prototype of what needs to be written.

double experience_rating(vector<unit*> player_units, vector<unit*> benchmark_units)

Return a value between -1 and 1. If the player_units are expected to do well vs the benchmark_units, then return a value close to 1. If the benchmark_units have the advantage, return a value close to -1. Later, this value can be scaled to A to F or -5 to 5, or whatever the discussion evolves into.

The Scenario Designer will supply the benchmark_units vector in his Scenario. The Player_units is the set of the best units that the player can afford / recall with the gold he'll have next chapter. (And this list of units can be found with some sort of optimization routine. Its a non-trivial problem, but can always be Monte-Carloed or something)

My implementation plan as described before was to make the system flexible to Turuk's request. It is also simple for the Scenario designer to think in this way. (The Scenario designer just needs to think in "benchmark sets" of enemies the player should be able to beat in this chapter). To the KISS principle, it seems like an easy to use function for the scenario designer, and for the player to interpret.

EDIT2: The only thing complicated might be the implementation, but I think I've sketched out some ideas of how to implement something like that.
Last edited by dragontamer on June 13th, 2009, 5:18 pm, edited 2 times in total.
Velensk
Multiplayer Contributor
Posts: 4002
Joined: January 24th, 2007, 12:56 am

Re: Scenario Grade [Split]

Post by Velensk »

That dosn't sound anything like the idea you were talking about before, but it does sound like a much better one.

I still would not advice putting it in, but for diffrent reasons.
"There are two kinds of old men in the world. The kind who didn't go to war and who say that they should have lived fast died young and left a handsome corpse and the old men who did go to war and who say that there is no such thing as a handsome corpse."
User avatar
Turuk
Sithslayer
Posts: 5283
Joined: February 28th, 2007, 8:58 pm
Contact:

Re: Scenario Grade [Split]

Post by Turuk »

You do realize the campaigns are broken up into scenarios and not chapters, right?

Adding such a complex system to affect a rating might be useful, but does the usefulness outweigh the complexity?
Mainline Maintainer: AOI, DM, NR, TB and THoT.
UMC Maintainer: Forward They Cried, A Few Logs, A Few More Logs, Start of the War, and Battle Against Time
User avatar
Cloud
Art Contributor
Posts: 502
Joined: December 17th, 2008, 7:43 pm
Location: The land of pixels
Contact:

Re: Scenario Grade [Split]

Post by Cloud »

This is very anti-KISS. Having started coding a campaign (and nearly done on scenario) I'm begining to appreciate how much effort it takes to do the bare bones. Adding this in would add a whole new level of complexity that campaigns just don't need.

The other question I have is to what end? Most scenarios have a few twists and turns, but add in too many and the coder will be bogged down to his pointy little ears in variables already. Start adding "if player got a C grade go to this scenario" etc. along with having to impliment the code just for the grades to work on all his scenario. You're talking too much work for too little reward to the player.
Softly/SoftlySplinter on IRC. Will be lurking around more these days
Mainline Animations|The Væringjar
Art for these mead-sodden, bearded mushroom-junkies by Girgistian!
dragontamer
Posts: 24
Joined: March 28th, 2009, 11:56 pm

Re: Scenario Grade [Split]

Post by dragontamer »

This is very anti-KISS. Having started coding a campaign (and nearly done on scenario) I'm begining to appreciate how much effort it takes to do the bare bones. Adding this in would add a whole new level of complexity that campaigns just don't need.

The other question I have is to what end? Most scenarios have a few twists and turns, but add in too many and the coder will be bogged down to his pointy little ears in variables already. Start adding "if player got a C grade go to this scenario" etc. along with having to impliment the code just for the grades to work on all his scenario. You're talking too much work for too little reward to the player.
This feature should be optional. This sort of feature would probably work best as a finishing touch on mature / finished projects such as Heir to the Throne. I agree that developing projects should not have to worry about something like this.

Second: the rating is simply a rating system for the player. As stated in the first post, there are too many scenarios where a player can be trapped into perpetual game-over situation. While most regulars have learned to restart campaigns from scratch... I doubt that this is the mark of a high-quality game. Benchmarking the player's Recall list would be one step in the right direction IMO.
You do realize the campaigns are broken up into scenarios and not chapters, right?
I guess Fire emblem is still stuck in my head >_<.
Adding such a complex system to affect a rating might be useful, but does the usefulness outweigh the complexity?
:hmm: The key is to find the balance between rigor and complexity. My first suggestion was not rigorous enough, and perhaps my second suggestion was a bit too complex. Here's another idea based off of the "efficiency matrix" idea as well. But instead of trying to explain it, here's a little python demo that demonstrates how easy the "battle simulation" is. Just to note, this code prints out a value between 0 and infinity. A "1" would be about an equal battle, a "0" completely favors the computer.

Code: Select all

#!/usr/bin/python

import operator

#Efficiency Matrix
# \/ vs ->   | Spearmen | Mages |  Calvarymen
#---------------------------------------------
# Fighers    |   1.1    |  1.0  |  .9
# Archers    |   .9     |  .8   |  1.1
# Sorceress  |   1.3    |  1.1  |  1.5

# For example, 100 Fighers are expected to kill 110 Spearmen. Thats what 1.1
# means. Or, 100 Archers can kill 90 Spearmen. The numbers are made up
# for the purposes of demonstration.

efficiency = [ [1.1, 1.0,  .9 ],
	         [ .9,  .8, 1.1 ],
	         [1.3, 1.1, 1.5 ] ]

# Lets say that the player can afford 5 Fighters, 3 Archers, and one Sorceress
player_units = [5, 3, 1]

# Benchmark: The player is expected to kill 3 Spearmen, 5 Mages, 
# and 2 Calvarymen. After defining benchmark, normalize it so that the sum == 1
benchmark = [3., 5., 2.]
benchmark_total = sum(benchmark)
benchmark = map(lambda(x): x / sum(benchmark), benchmark)

#Now, lets see what each of the player's units can do against a "standard unit"

standard_unit_efficiency = map(lambda(x): sum(map(operator.mul, x, benchmark)), efficiency)

# And of course, multiply with the units that the player has, and sum it up.
# This describes the number of kills a player is expected to have.

expected_kills = sum(map(operator.mul, standard_unit_efficiency, player_units))

# Normalize to the number of units in the benchmark. IE: "1" describes a 
# dead equal match. "0" is a perfect win for the CPU (Player can't afford any units)
print expected_kills / benchmark_total
The general idea is as follows. Instead of thinking of the problem as "Fighters, Shamans, and Archers vs 5 Orcs, 7 Trolls, and 10 Riders", think of it as "Fighters, Shamans, and Archers vs 22 standard units", where each unit is 5/22 Orc, 7/22 Troll, and 10/22 Riders. From there, calculate how well the player's units do against a standard unit. Finally, calculate how many standard units the player's army can kill. Normalize so that "1" describes a dead even match and volia, done.

I think this approach strikes the balance between KISS and Rigor. Depending on the enemy's units, as well as what the player has available, the rating will change. Stronger units and/or units with an advantage in a specific scenario will give a higher rating. And half the "code" is just comments for me to explain what is going on. I think there's only ~10 real lines of code up there.

The "efficiency matrix" should be easy to calculate. I'd expect to just play around with averages. The benchmark units should be tested by the scenario designer. And as stated before, this should be an optional feature that is used only when the campaign has reached maturity.

The only complicated thing is figuring out the player_units.
Post Reply