Seeking tricky WML examples

[Celtic_Minstrel]

I'm looking for samples of WML where the preprocessor or parser does something a bit unusual that you might not expect, or where you need to do something a little unusual to make it work correctly. Does anyone have any examples of such things?

I'll start with one that I recently noticed:

Code: Select all

[my_tag]
	value = _ hello
[/my_tag]

When read in-game, it seems that $my_tag.value has the value "_hello" – the space has been removed.

And here's a preprocessor one that I think bites people a lot:

Code: Select all

#define MY_FORMULA
"(if(a = b,
	12,
	22
))"#enddef

formula = "(base_value + {MY_FORMULA})"

This ends up being an error. If I understand correctly, the issue is that, due to a macro whose value is quoted being included within a quoted string, the opening quote in the macro wants to be a closing quote. However, it seems a closing quote is not allowed to be followed by characters other than + or whitespace, so you get an error.

Rationale: The preprocessor and parser are currently messy and hard to extend due to the need to avoid breaking functionality and the lack of unit tests. It's relatively easy to write unit tests for simple, common scenarios, but quite another matter to write them for edge cases. If we can get a lot of weird examples, it would be a big step towards the goal of having sufficient unit tests to be able to be at peace with adding new features.

[egallager]

Issue #7885 has some.

[Wayirr]

My add-on Age of Tentacles has fairly tricky code of invisibility, which spams constant errors but works somehow. If you want to review it, it would be nice.

[Duke_Anax]

Code: Select all

#define MY_FORMULA
"(if(a = b,
	12,
	22
))"#enddef

formula = "(base_value + {MY_FORMULA})"

As a programmer with a very limited understanding of WML, this has me confused. I thought it would be obvious to anyone who has coded or scripted more than 10 minutes, so why is it tripping people up?

I mean, if you use " to define a String and then put " as a character in the String, most parsers will croak because they don't know whether you're trying to add a character or end the string. That's not unusual that's one of the first things any coder or scripter trips over.

So why not this?

Code: Select all

#define MY_FORMULA
(if(a = b,
	12,
	22
))#enddef

formula = "(base_value + {MY_FORMULA})"
[/code]

If the quotes are required to define the string, why do they also end up part of the string? And If they are meant as characters, why are they not escaped?

[Ravana]

There is no general way to escape quote, without using << >> as outer quotes instead. Normally to escape quote you need to double it.

[Celtic_Minstrel]

Well, if I understand correctly, this definition would also work fine.

Code: Select all

#define MY_FORMULA
" (if(a = b,
	12,
	22
)) "#enddef

[Duke_Anax]

Well, if I understand correctly, this definition would also work fine.

Code: Select all

#define MY_FORMULA
" (if(a = b,
	12,
	22
)) "#enddef

Ok, I just realized due to my experience with other markup languages (and parsers). I may actually have negative understanding of how the F WML works because none of this makes sense to me. I'll just shut my mouth and read the tutorial.

[Celtic_Minstrel]

Don't worry, there are people experienced with WML who are also confused by this.

[Duke_Anax]

I'll just shut my mouth and read the tutorial.

Ok read the tutorial and skimmed most of the relevant docs. Everything is as I expected after all and the macro processor is dead simple. Handling variables in WML is oddly cumbersome but clear enough.

So now I'm no longer confused about this example, I'm utterly lost.

What is up with this macro? That's not even WML, that's just pseudocode. And why does the Blanc space help? You are still terminating the string, so you have two separate strings with pseudocode in the middle.

[Celtic_Minstrel]

WML has very few special characters. Any character that appears after the =, with a few exceptions, is considered to be part of the value of the attribute. It doesn't matter how convoluted it looks or how many punctuation characters appear in it – as long as none of the special characters appear, it's just a string read according to HTML conventions (that is, runs of whitespace are collapsed down to a single space).

Thinking about it more, the amended example in my post would probably only work if it was collapsed down to a single line, as a newline not inside quotes always terminates the attribute value. So this should work:

Code: Select all

#define MY_FORMULA
" (if(a = b, 12, 22)) "#enddef

The special characters that aren't just taken as literal are:

• Quotes. Contrary to what might be popular belief, quotes don't "enclose a string" or anything like that. Everything in the attribute value is a string, unless it's parsable as a number or is precisely one of the fixed strings yes/no/true/false. Quotes do three things: they disable the whitespace-collapsing behaviour of unquoted text, and they prevent a newline character from terminating the attribute value. Lastly, they prevent most other characters from being treated specially, with the exception of curly braces (which are still treated as macro inclusions) and the quote character itself (which terminates the quoted passage, unless it's doubled, which expands to a single double quote character and continues the quoted passage). So they're an escaping mechanism, comparable to <![CDATA[....]]> in XML.
• Double left angle brackets <<. These are almost semantically identical to quotes, but curly braces and quotes within them are not treated as special characters, whereas double right angle brackets >> are, terminating the quoted passage.
• Plus signs. These have two use-cases. One is as a line-continuation character – if the last non-comment non-whitespace character on the line is a plus sign, the attribute value continues onto the next line. The second use-case comes into play only when placed between two quoted passages. Without a plus sign, a space would be inserted between the two quoted passages; the plus sign prevents this from happening.
• Number signs. As you probably know, a number sign begins a comment, which means the end of the attribute value unless directly preceded by a plus sign.
• Commas. These only have a special meaning if they appear outside quotes and the left-hand side of the = contained commas. The value is split on the unquoted commas and assigned to each of the attributes in turn, with the last attribute getting any extra parts (with the commas themselves left in).
• Underscores. This only has a special meaning when directly preceding a quoted passage (either with actual quotes or with angled brackets), which will then be marked as translatable.

(Note that I've omitted $, since that only comes into play long after the parsing has finished.)

And the reason the above code passage works and the below one doesn't is because (from what I can tell) the parser chokes if the character after a quoted passage is not a whitespace character. (Although I haven't tested the below snippet, and it's possible that it's only the newlines that make the original snippet fail.)

Code: Select all

#define MY_FORMULA
"(if(a = b, 12, 22))"#enddef

[Duke_Anax]

Thanks for the explanation, that makes a lot of things much clearer. So let me get this straight. The earlier macro would turn this:

Code: Select all

formula = "(base_value + {MY_FORMULA})"

into this:

Code: Select all

formula = "(base_value + " (if(a = b,
	12,
	22
)) ")"

which wouldn't work because of the newline outside of the quoted String.

and the last example produces this:

Code: Select all

formula = "(base_value + " (if(a = b, 12, 22)) ")"

which gets turned into one long string value, right?


While this:

Code: Select all

formula = "(base_value + "(if(a = b, 12, 22))")"

doesn't work because the parser croaks on the ( after the ", even though it doesn't care about the rest of this mess.

wouldn't the ") also cause the same problem? So you might have to add another blank after the macro call:

Code: Select all

formula = "(base_value + {MY_FORMULA} )"

[Duke_Anax]

Now that's the issue with the macro, but what about the part that sent me tumbling to begin with, what's up with this string?

Code: Select all

"(base_value + "(if(a = b, 12, 22))")"

Is it just some random string that happens to look like pseudo code, or does it actually do something? It sure doesn't like WML code.

[gnombat]

Code: Select all

#define MY_FORMULA
"(if(a = b,
	12,
	22
))"#enddef

formula = "(base_value + {MY_FORMULA})"

Wouldn't it be better to come up with a more minimalistic test case than that? There's so many things going on in that example it's hard to isolate where the issue actually is.

Code: Select all

#define MY_FORMULA
" (if(a = b, 12, 22)) "#enddef

That's better, but it still seems way more complicated than actually necessary.

I'm thinking something really simple like:

Code: Select all

#define MY_MACRO
"bar"#enddef

[message]
    speaker=harry
    message="foo {MY_MACRO}"
[/message]

Now that's the issue with the macro, but what about the part that sent me tumbling to begin with, what's up with this string?

Code: Select all

"(base_value + "(if(a = b, 12, 22))")"

Is it just some random string that happens to look like pseudo code, or does it actually do something? It sure doesn't like WML code.

It's Wesnoth Formula Language. But again, I think this is just making the issue more complicated than it needs to be (I don't think the issue really has anything to do with Wesnoth Formula Language).

[Celtic_Minstrel]

You're right, the issue doesn't have anything to do with WFL. It just happens to more frequently come up with WFL for various reasons.

I'm thinking something really simple like:

Code: Select all

#define MY_MACRO
"bar"#enddef

[message]
    speaker=harry
    message="foo {MY_MACRO}"
[/message]

Yup, that looks good. Maybe also something like this:

Code: Select all

#define MY_MACRO
"
bar
"#enddef

[message]
    speaker=harry
    message="foo {MY_MACRO}"
[/message]

[Duke_Anax]

It's Wesnoth Formula Language.

Ah yes thank you, that's the bit that I couldn't find myself. I still need more time to understand it, but at least it makes sense now.

I agree that the example is way too complex to be of any practical use, as you can see above, it has so many interlocking bits and pieces that I couldn't even make heads or tails of it.

You're right, the issue doesn't have anything to do with WFL. It just happens to more frequently come up with WFL for various reasons.

It seems part of the problem is WML sometimes not being very specific in it's specification. Like meh, everything is a string, but a string could also be code or maybe not. M

Is there a "WML coding, best practices" or something somewhere?