all units parameters in JSON

[RBnik]

I couldn't find good database of units parameters.
So, I created parser, which create JSON
For your experiments)

Code: Select all

import requests
from bs4 import BeautifulSoup
import re
import json

'''
with open("./all_names.txt",encoding="utf-8") as f:
    string=f.read()
    names=re.findall(r'S/.*?html',string)
names=[i[2:-5] for i in names]
'''

names=['Blood%20Bat', 'Dread%20Bat', 'Vampire%20Bat', 'Armageddon%20Drake', 'Drake%20Arbiter', 'Drake%20Blademaster',
       'Drake%20Burner', 'Drake%20Clasher', 'Drake%20Enforcer', 'Drake%20Fighter', 'Drake%20Flameheart', 'Drake%20Flare',
       'Drake%20Glider', 'Drake%20Thrasher', 'Drake%20Warden', 'Drake%20Warrior', 'Fire%20Drake', 'Hurricane%20Drake',
       'Inferno%20Drake', 'Sky%20Drake', 'Dune%20Alchemist', 'Dune%20Apothecary', 'Dune%20Blademaster', 'Dune%20Burner',
       'Dune%20Captain', 'Dune%20Cataphract', 'Dune%20Explorer', 'Dune%20Falconer', 'Dune%20Firetrooper', 'Dune%20Harrier',
       'Dune%20Herbalist', 'Dune%20Horse%20Archer', 'Dune%20Luminary', 'Dune%20Marauder', 'Dune%20Paragon', 'Dune%20Raider',
       'Dune%20Rider', 'Dune%20Rover', 'Dune%20Scorcher', 'Dune%20Skirmisher', 'Dune%20Sky%20Hunter', 'Dune%20Soldier',
       'Dune%20Spearguard', 'Dune%20Spearmaster', 'Dune%20Strider', 'Dune%20Sunderer', 'Dune%20Swordsman', 'Dune%20Warmaster',
       'Dune%20Wayfarer', 'Dune%20Windbolt', 'Dwarvish%20Arcanister', 'Dwarvish%20Berserker', 'Dwarvish%20Dragonguard',
       'Dwarvish%20Explorer', 'Dwarvish%20Fighter', 'Dwarvish%20Guardsman', 'Dwarvish%20Lord', 'Dwarvish%20Miner', 'Dwarvish%20Pathfinder',
       'Dwarvish%20Runemaster', 'Dwarvish%20Runesmith', 'Dwarvish%20Scout', 'Dwarvish%20Sentinel', 'Dwarvish%20Stalwart',
       'Dwarvish%20Steelclad', 'Dwarvish%20Thunderer', 'Dwarvish%20Thunderguard', 'Dwarvish%20Ulfserker', 'Elvish%20Archer',
       'Elvish%20Avenger', 'Elvish%20Captain', 'Elvish%20Champion', 'Elvish%20Druid', 'Elvish%20Enchantress', 'Elvish%20Fighter',
       'Elvish%20Hero', 'Elvish%20High%20Lord', 'Elvish%20Lady', 'Elvish%20Lord', 'Elvish%20Marksman', 'Elvish%20Marshal',
       'Elvish%20Outrider', 'Elvish%20Ranger', 'Elvish%20Rider', 'Elvish%20Scout', 'Elvish%20Shaman', 'Elvish%20Sharpshooter',
       'Elvish%20Shyde', 'Elvish%20Sorceress', 'Elvish%20Sylph', 'Elder%20Falcon', 'Falcon', 'Goblin%20Impaler', 'Goblin%20Rouser',
       'Goblin%20Spearman', 'Gryphon', 'Gryphon%20Master', 'Gryphon%20Rider', 'Bay%20Horse', 'Black%20Horse', 'Dark%20Horse',
       'Great%20Horse', 'White%20Horse', 'Arch%20Mage', 'Assassin', 'Bandit', 'Bowman', 'Cavalier', 'Cavalryman', 'Dark%20Adept',
       'Dark%20Sorcerer', 'Dragoon', 'Duelist', 'Elder%20Mage', 'Fencer', 'Footpad', 'Fugitive', 'General', 'Grand%20Knight',
       'Grand%20Marshal', 'Great%20Mage', 'Halberdier', 'Heavy%20Infantryman', 'Highwayman', 'Horseman', 'Huntsman', 'Iron%20Mauler',
       'Javelineer', 'Knight', 'Lancer', 'Lieutenant', 'Longbowman', 'Mage', 'Mage%20of%20Light', 'Master%20Bowman', 'Master%20at%20Arms',
       'Necromancer', 'Outlaw', 'Paladin', 'Peasant', 'Pikeman', 'Poacher', 'Ranger', 'Red%20Mage', 'Rogue', 'Royal%20Guard',
       'Royal%20Warrior', 'Ruffian', 'Sergeant', 'Shock%20Trooper', 'Silver%20Mage', 'Spearman', 'Swordsman', 'Thief', 'Thug',
       'Trapper', 'White%20Mage', 'Woodsman', 'Boat', 'Galleon', 'Pirate%20Galleon', 'Transport%20Galleon', 'Mermaid%20Diviner',
       'Mermaid%20Enchantress', 'Mermaid%20Initiate', 'Mermaid%20Priestess', 'Mermaid%20Siren', 'Merman%20Brawler', 'Merman%20Citizen',
       'Merman%20Entangler', 'Merman%20Fighter', 'Merman%20Hoplite', 'Merman%20Hunter', 'Merman%20Javelineer', 'Merman%20Netcaster',
       'Merman%20Spearman', 'Merman%20Triton', 'Merman%20Warrior', 'Caribe', 'Cave%20Bear', 'Cuttle%20Fish', 'Dragonfly', 'Dragonfly%20Naiad',
       'Fire%20Ant', 'Fire%20Ant%20Egg', 'Fire%20Ant%20Queen', 'Fire%20Dragon', 'Fire%20Guardian', 'Fire%20Wraith', 'Firebane%20Ant',
       'Firebomb%20Ant', 'Frost%20Stoat', 'Giant%20Ant', 'Giant%20Ant%20Egg', 'Giant%20Ant%20Queen', 'Giant%20Mudcrawler', 'Giant%20Rat',
       'Giant%20Scorpion', 'Giant%20Scorpling', 'Giant%20Spider', 'Grand%20Dragonfly', 'Great%20Icemonax', 'Great%20Seahorse',
       'Horned%20Scarab', 'Hunter%20Caribe', 'Icemonax', 'Jinn', 'Kraken', 'Mudcrawler', 'Nibbler', 'Piglet', 'Roc', 'Rock%20Scorpion',
       'Sand%20Scamperer', 'Sand%20Scuttler', 'Sea%20Serpent', 'Shadow%20Jumping%20Spider', 'Soldier%20Ant', 'Swamp%20Lizard',
       'Tentacle%20of%20the%20Deep', 'Water%20Serpent', 'Wild%20Wyvern', 'Woodland%20Boar', 'Wyvern%20Rider', 'Yeti', 'Naga%20Dirkfang',
       'Naga%20Fighter', 'Naga%20Guard', 'Naga%20High%20Guard', 'Naga%20Myrmidon', 'Naga%20Ophidian', 'Naga%20Ringcaster',
       'Naga%20Shield%20Guard', 'Naga%20Sicarius', 'Naga%20Warrior', 'Naga%20Zephyr', 'Ogre', 'Young%20Ogre', 'Orcish%20Archer',
       'Orcish%20Assassin', 'Orcish%20Crossbowman', 'Orcish%20Grunt', 'Orcish%20Leader', 'Orcish%20Nightblade', 'Orcish%20Ruler',
       'Orcish%20Slayer', 'Orcish%20Slurbow', 'Orcish%20Sovereign', 'Orcish%20Warlord', 'Orcish%20Warrior', 'Dark%20Omen', 'Raven',
       'War%20Harbinger', 'Saurian%20Ambusher', 'Saurian%20Augur', 'Saurian%20Flanker', 'Saurian%20Javelineer', 'Saurian%20Oracle',
       'Saurian%20Prophet', 'Saurian%20Seer', 'Saurian%20Skirmisher', 'Saurian%20Soothsayer', 'Saurian%20Spearthrower', 'Great%20Troll',
       'Troll', 'Troll%20Hero', 'Troll%20Rocklobber', 'Troll%20Shaman', 'Troll%20Warrior', 'Troll%20Whelp', 'Ancient%20Lich', 'Banebow',
       'Bone%20Knight', 'Bone%20Shooter', 'Chocobone', 'Death%20Knight', 'Death%20Squire', 'Deathblade', 'Draug', 'Ghast', 'Ghost',
       'Ghoul', 'Lich', 'Necrophage', 'Nightgaunt', 'Revenant', 'Shadow', 'Skeletal%20Dragon', 'Skeleton', 'Skeleton%20Archer',
       'Skeleton%20Rider', 'Soulless', 'Spectre', 'Walking%20Corpse', 'Wraith', 'Direwolf', 'Direwolf%20Rider', 'Goblin%20Knight',
       'Goblin%20Pillager', 'Great%20Wolf', 'Wolf', 'Wolf%20Rider', 'Ancient%20Wose', 'Elder%20Wose', 'Wose', 'Wose%20Sapling',
       'Wose%20Shaman', 'mainline', 'mainline', 'mainline', 'mainline', 'mainline', 'mainline', 'mainline']

headers = {
    "User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/58.0.3029.110 Safari/537.36"
}

units=[]
count=0
for i in names:
    unit={}
    url="https://units.wesnoth.org/1.18/mainline/en_US/"+i+".html"

    response = requests.get(url, headers=headers)
    soup = BeautifulSoup(response.text, 'html.parser')

    #main info
    table = soup.find('table', {'class': 'unitinfo'})
    for row in table.find_all('tr'):
        columns = row.find_all(['td', 'th'])
        unit[columns[0].text]=columns[1].text

    #resists
    resist={}
    table = soup.find('table', {'class': 'unitinfo resistances'})
    for row in table.find_all('tr'):
        columns = row.find_all(['td', 'th'])
        
        resist[columns[1].text]=columns[2].find("i").text
        resist[columns[5].text]=columns[6].find("i").text
    unit["resist"]=resist

    #move cost and dodge
    move_cost={}
    dodge={}
    table = soup.find('table', {'class': 'unitinfo terrain'})
    for row in table.find_all('tr')[1:]:
        columns = row.find_all('td')
        move_cost[columns[1].text]=columns[2].text
        dodge[columns[1].text]=columns[3].text
    unit["move_cost"]=move_cost
    unit["dodge"]=dodge
    
    #attacks
    try:
        attacks=[]
        table = soup.find('table', {'class': 'unitinfo attacks'})
        for row in table.find_all('tr'):
            columns = row.find_all('td')
            attack={}
            attack["name"]=columns[1].text
            attack["damage_type"]=columns[2].text
            attack["count"]=columns[3].text
            attack["distance"]=columns[4].text
            if len(columns)==6:
                attack["mod"]=columns[5].text
            else:
                attack["mod"]="(none)"
            attacks.append(attack)
        unit["attacks"]=attacks
    except Exception:
        unit["attacks"]={}

    units.append(unit)

    count+=1
    print(i,str(count)+"/"+str(len(names)))
    if i=="Wose%20Shaman":
        break
with open('./units.json', 'w') as fp:
    json.dump(units, fp)

[Ravana]

You can parse unit wml with tools included with game, example https://github.com/ProditorMagnus/WML_t ... n_lists.py https://github.com/wesnoth/wesnoth/blob ... parser3.py

[Soliton]

You can also create JSON with wmlparser3:

Code: Select all

data/tools/wesnoth/wmlparser3.py --input data/core/units.cfg --wesnoth <path to wesnoth binary> --to-json

Just have to specify the path to the wesnoth binary or preprocess in a previous step:

Code: Select all

wesnoth --preprocess data/core/units.cfg preprocess/
data/tools/wesnoth/wmlparser3.py --input preprocess/units.cfg --to-json

Then you can process the JSON with jq for example:

Code: Select all

jq '.units[].unit_type[] | select(.id == "Spearman")'

That'll list all properties of the Spearman unit type.

Code: Select all

jq '.units[] | (.unit_type[] | select(.id == "Spearman")).movement_type as $movetype | .movetype[] | select(.name == $movetype)'

And that'll list the movement type of the Spearman.