Add-on resources question: has this been done already?

[Spannerbag]

First off, apologies if I've posted in the wrong forum, I wasn't sure which was most appropriate.

As a side project to continuing campaign development I thought I'd start building a generic single player campaign resource file with the working title SPUR (Single Player UMC Resources). I'm staring with optional music tracks * that if present, SPUR-aware campaign(s) will use automatically in addition to the music provided by the game. The idea is that multiple resources (probably just music initially but later maybe stuff useful to players and/or developers) can be downloaded as needed, either by a campaign or player/developer wanting to use some modification, resource, macro, unittype ** or whatever - I've lots of ideas but have no clue yet as to whether any/which are feasible).

* I appreciate that there are already multiple music resources available but as far as I know they all create separate directories with a handful of music tracks in each. My idea is to be able to only download the tracks you need once and have them all in one place so preprocessor directives are not too unwieldy due to having to match up each music track name with a specific directory (if my understanding is incorrect and things are done differently please let me know - it could save me a lot of work!).
However if the number/size of the music directory gets too big I guess it may need subdividing anyway...

** I imagine these as being mini-eras of only a few related units that generally can be integrated with mainline units (typically zero levels that advance to one or more mainline level 1 units, e.g. AfterEI's Yeoman advances to Heavy Infantryman).

This entails having multiple modules writing to the same directory structure and sometimes, as is the case with music, to the same folder.
Hopefully the add-on manager logic will not re-download (overwrite) existing files but will simply skip to the next file (unless the source has a later timestamp)?
(As an aside I was going to have a separate SPUR Music repository but the acronym was a bit unfortunate! )

I was going to publish an initial test add-on but before I do that I'd like to ask if anyone else has already done something like this?
Also any thoughts/opinions - good or bad - would be really welcome.

Cheers!
-- Spannerbag

[Pentarctagon]

This entails having multiple modules writing to the same directory structure and sometimes, as is the case with music, to the same folder.

What does this mean, specifically?

[octalot]

Wesnoth already handles searching multiple directories as if they're a union filesystem, as documented in BinaryPathWML.
Add-ons also support dependencies, so you can depend on the UMC Music Packs instead of duplicating files.

Also, does the Tamanegi Jukebox add-on do what you want? I generally play without music, so haven't investigated it.

[Spannerbag]

This entails having multiple modules writing to the same directory structure and sometimes, as is the case with music, to the same folder.

What does this mean, specifically?

Apologies, I obviously wasn't clear.
Also just read Octalot's post and I suspect what I was asking is already available.

Essentially I was mulling over having various add-ons all downloading different files to the same directory on the player's local computer and whether or not the download manager would not re-download files already present (i.e. already downloaded) unless, maybe, the date time stamp on the server version of the download file was later than the timestamp on the already downloaded file.

Suppose I have 2 campaigns; C1 and C2.
Then suppose that C1 optionally uses extra music tracks; T1, T2, T3 and T4.
Likewise, C2 might use T1, T3, T5 and T6.

When the player downloads C1 they will be informed that there is an option to also download additional music tracks (T1-4).
If the player elects to do this then these will be downloaded to the local directory, say ../SPUR/MUSIC.

If the same player subsequently decides to play C2 and again download the optional music then music tracks T1,3,5,6 will be downloaded into the same directory, namely ../SPUR/MUSIC. So each campaign will have one (maybe more) optional dependencies that store data into a shared directory hierarchy. I also hope that the download manager will sense that although music tracks T1,3,5,6 are required, two of these (T1 and T3) already exist on the target system so only T5 and T6 will be downloaded (into the same place as the first C1 music download).

However I suspect that Octalot's post probably makes this requirement redundant...

Hope his clarifies!

Cheers!
-- Spannerbag

[Spannerbag]

Wesnoth already handles searching multiple directories as if they're a union filesystem, as documented in BinaryPathWML.
Add-ons also support dependencies, so you can depend on the UMC Music Packs instead of duplicating files.

Also, does the Tamanegi Jukebox add-on do what you want? I generally play without music, so haven't investigated it.

Thanks for the post, very helpful.
I've downloaded the Tamanegi Jukebox and will take a look.
Will do some research and see how I get on.

Just one question: suppose I wanted to use only certain tracks from the various music packs.
If I were to replicate the music pack directory structure but only download the files I wanted the campaign to use, would this break or otherwise intrefere with the existing music pack download (i.e. would the mere presence of the directory on the target system be enough for the add-on manager to flag the entire pack as already installed - or would it do a more detailed check of what's on the target system)?

Anyway, between campaign code/test cycles (I really want to publish something but keep having "good ideas" ) I'll look into all this.
Thanks again for the post!

Cheers!
-- Spannerbag

[Pentarctagon]

Each add-on downloaded goes in its own directory directly underneath the add-ons directory - there's no way to have multiple add-ons download files to the same directory. So you would have add-ons/UMC_Music_Book_1, add-ons/UMC_Music_Book_2, etc. They can't both be downloaded to ../SPUR/MUSIC.

The add-on manager also doesn't check if the same file exists in multiple add-ons. If you end up with duplicates, then it uses whichever one it encounters first when checking all locations included via the [binary_path]s.

[Spannerbag]

Each add-on downloaded goes in its own directory directly underneath the add-ons directory - there's no way to have multiple add-ons download files to the same directory. So you would have add-ons/UMC_Music_Book_1, add-ons/UMC_Music_Book_2, etc. They can't both be downloaded to ../SPUR/MUSIC.

Ah. That's a shame, I was hoping that - for resources anyway - this might not be the case (I can see this would be required for campaigns).

The add-on manager also doesn't check if the same file exists in multiple add-ons. If you end up with duplicates, then it uses whichever one it encounters first when checking all locations included via the [binary_path]s.

So, if I have several campaigns that use additional music I then seem to have 3 options:

1. Have separate optional music packs for each campaign (with duplication - albeit in separate add-ons - of music used in 2+ campaigns), or
2. Create a music pack containing all the additional music used by all these campaigns (single download, maybe including music that's never used), or
3. Utilise existing music packs (multiple downloads again probably including music that's never used).

Have I missed anything?

Thanks for the clarification, much appreciated.

Cheers!
-- Spannerbag

[Pentarctagon]

That's correct.

[gnombat]

So, if I have several campaigns that use additional music I then seem to have 3 options:

1. Have separate optional music packs for each campaign (with duplication - albeit in separate add-ons - of music used in 2+ campaigns), or
2. Create a music pack containing all the additional music used by all these campaigns (single download, maybe including music that's never used), or
3. Utilise existing music packs (multiple downloads again probably including music that's never used).

Have I missed anything?

In theory, you could create a separate add-on for each music track (containing just one music file and nothing else). So, using your example above, you would end up with 8 add-ons in total: C1, C2, T1, T2, T3, T4, T5, and T6. I've never seen anyone actually do it that way, and honestly I'm not sure it's really a good idea, but it might work.

If you just have two campaigns, like in your example, another option could be to create 5 add-ons: C1, C2, C1-Music (containing tracks T2 and T4), C2-Music (containing tracks T5 and T6) and Common-Music (containing tracks T1 and T3). It would work well for two campaigns; however, it would start to break down if you decide to add a third campaign C3, and you realize you want to use some (but not all) tracks from C1-Music, some (but not all) tracks from C2-Music, etc.

[Spannerbag]

That's correct.

Thanks - pity, for once I hoped I was wrong...

Cheers,
-- Spannerbag

[Spannerbag]

...Have I missed anything?

In theory, you could create a separate add-on for each music track (containing just one music file and nothing else). So, using your example above, you would end up with 8 add-ons in total: C1, C2, T1, T2, T3, T4, T5, and T6. I've never seen anyone actually do it that way, and honestly I'm not sure it's really a good idea, but it might work.

If you just have two campaigns, like in your example, another option could be to create 5 add-ons: C1, C2, C1-Music (containing tracks T2 and T4), C2-Music (containing tracks T5 and T6) and Common-Music (containing tracks T1 and T3). It would work well for two campaigns; however, it would start to break down if you decide to add a third campaign C3, and you realize you want to use some (but not all) tracks from C1-Music, some (but not all) tracks from C2-Music, etc.

Sadly I do indeed want to share music amongst 3+ campaigns (if I ever finish #2 and #3 and then completely overhaul #1: AfterEI...) also there are many tracks (~34) I'd like to use and I can't see many players having the time and patience to manually add 30+ tracks (add-ons) individually.
Nice idea though

Cheers!
-- Spannerbag

[Soliton]

As a side project to continuing campaign development I thought I'd start building a generic single player campaign resource file with the working title SPUR (Single Player UMC Resources). I'm staring with optional music tracks * that if present, SPUR-aware campaign(s) will use automatically in addition to the music provided by the game. The idea is that multiple resources (probably just music initially but later maybe stuff useful to players and/or developers) can be downloaded as needed, either by a campaign or player/developer wanting to use some modification, resource, macro, unittype ** or whatever - I've lots of ideas but have no clue yet as to whether any/which are feasible).

Sharing resources between addons can be done with [resource] from the providing addon and [load_resource] from the consuming addon.

I don't know how you imagine automatic use should work. If you want an addon to "force" whatever it provides into a campaign or so then you can write a [modification] that the player can select when starting a game.

* I appreciate that there are already multiple music resources available but as far as I know they all create separate directories with a handful of music tracks in each. My idea is to be able to only download the tracks you need once and have them all in one place so preprocessor directives are not too unwieldy due to having to match up each music track name with a specific directory (if my understanding is incorrect and things are done differently please let me know - it could save me a lot of work!).
However if the number/size of the music directory gets too big I guess it may need subdividing anyway...

** I imagine these as being mini-eras of only a few related units that generally can be integrated with mainline units (typically zero levels that advance to one or more mainline level 1 units, e.g. AfterEI's Yeoman advances to Heavy Infantryman).

This entails having multiple modules writing to the same directory structure and sometimes, as is the case with music, to the same folder.
Hopefully the add-on manager logic will not re-download (overwrite) existing files but will simply skip to the next file (unless the source has a later timestamp)?
(As an aside I was going to have a separate SPUR Music repository but the acronym was a bit unfortunate! )

Adding lots of complexity by allowing multiple addons to write into the same dir just to have a minimally easier time to write some path for #ifhave?

You have to put something in your addon to use the files either way. What dir the files are in seems barely relevant.

Depending on how you want to use the music for example you could rather define events that add music. Either defining custom events that need to be explicitly fired in the desired scenario at the desired point or something like a prestart event if you want the music everywhere.

The way you partition those resource addons is unrelated to where the files are put. You can put every single file in its own addon if you really want to and write another addon that depends on all those addons/files you want to use in some campaign or whatever. No need for duplication or for players to install each addon by hand.

[octalot]

So, if I have several campaigns that use additional music I then seem to have 3 options:

1. Have separate optional music packs for each campaign (with duplication - albeit in separate add-ons - of music used in 2+ campaigns), or
2. Create a music pack containing all the additional music used by all these campaigns (single download, maybe including music that's never used), or
3. Utilise existing music packs (multiple downloads again probably including music that's never used).

Using the existing music packs avoids duplication if other campaigns use those packs too, so I think you should take that option. Having a toolbox that includes tools that aren't used for the current job is a standard situation, and having a few extra music tracks is a minor overhead.

For downloading the music packs in the add-on manager, you could have an optional add-on that depends on the necessary music packs, so that a user who wants all the music for your campaign could install that add-on and the game engine will handle downloading the right music packs for it.

[Spannerbag]

...I don't know how you imagine automatic use should work. If you want an addon to "force" whatever it provides into a campaign or so then you can write a [modification] that the player can select when starting a game.

Heh, "automatically" was a poor choice of word, "conditionally" would have been better.
For music this would, as you later say, be via #ifhave.

Adding lots of complexity by allowing multiple addons to write into the same dir just to have a minimally easier time to write some path for #ifhave?

Well, my (admittedly not-fully-formed) idea was that players only download what the campaign needs from a common pool of resources so that if some required components have already been downloaded previously by some other campaign then only the missing elements are downloaded.
To me it seemed sensible to only download what you need - but I take your point about the added complexity.

You have to put something in your addon to use the files either way. What dir the files are in seems barely relevant.

Depending on how you want to use the music for example you could rather define events that add music. Either defining custom events that need to be explicitly fired in the desired scenario at the desired point or something like a prestart event if you want the music everywhere.

The way you partition those resource addons is unrelated to where the files are put. You can put every single file in its own addon if you really want to and write another addon that depends on all those addons/files you want to use in some campaign or whatever. No need for duplication or for players to install each addon by hand.

Ah. Hadn't thought that through. Add-ons containing individual files basically allows you to add individual files to a campaign.
Apologies gnombat, I didn't fully understand your suggestion

Many thanks for taking the time and trouble to write a very informative and helpful post, it's much appreciated and clarified my options

Cheers!
-- Spannerbag

[Spannerbag]

Using the existing music packs avoids duplication if other campaigns use those packs too, so I think you should take that option. Having a toolbox that includes tools that aren't used for the current job is a standard situation, and having a few extra music tracks is a minor overhead.

For downloading the music packs in the add-on manager, you could have an optional add-on that depends on the necessary music packs, so that a user who wants all the music for your campaign could install that add-on and the game engine will handle downloading the right music packs for it.

Thanks for the advice, I'll review the music packs in due course to see which ones I need.

Cheers!
-- Spannerbag