Hello there.
After 4 months, we've got a new Downloader release. The first one that breaks from our tradition of focusing on speed, and mostly brings new features.
If you’re new to MiSTer FPGA: Downloader is the project’s official updater. It installs and updates cores, required system files, and other official content. Update All uses Downloader behind the scenes.
We've got 4 important new features here.
Drop-in database files: Allows users to install new databases by just dropping files onto the SD. No more need to open and edit a file like hackers, just good old drag and drop.
Large file downloads: There is no cap anymore on how big the file downloads can be, so technically any arbitrarily large file works now.
Archive spec: Custom databases spec now includes an archive property to optimize and integrate different kinds of bundled downloads. This one is for the database maintainers.
Log rotation: By default we now keep additional logs in the log folder. Useful to know how previous runs went.
Other than that, I'm also preparing a new Update All release for next week that will focus on improving the CRT compatibility, especially for NTSC CRT users.
Since I don't have NTSC CRT equipment, I'd appreciate volunteers to help me test the new changes. If you are interested, send me a message, or show up in our Discord server: discord.com/invite/wcVhf5M
Also, starting this weekend patrons will receive a new Patreon Key. How? As announced in the previous post, this time it will be much easier: You only need to stay logged in to the new Login option of Update All's Settings Screen.
If you are already logged in, you don't need to do it again, you'll see the Account menu instead of the Login entry in the Settings Screen, and you'll receive your new key automatically as soon as it's available.
We'll continue now with a few more details about some of these new features.
Drop-in database files
Until now, if you wanted to add new databases to install additional cores and files, you had to either manually edit the downloader.ini file at the root of the SD, or use Update All's Setting Screen.
Update All is the more convenient way to do it, but it won't have all the databases ever because the spec is open. Creators are free to create their own databases and distribute them without having to go through Update All, and that's a good thing.
But it can put some users off, because editing INI files is not everybody's cup of tea. That's why I enabled this third method. Additional INI files you can just drop in the root of the SD and that's it. No need to even open them.
These drop-in database files should have the following name: downloader_.ini or alternatively, you could create a downloader folder in the root of the SD and name them downloader/.ini
In reality, the format is a bit more flexible and you are allowed to substitute the <> text with anything you want and add more than one database to the file, but I strongly recommend creators to follow this pattern.
The main restriction is that you can't put [MiSTer] or [distribution_mister] sections in these drop-in files.
Large file downloads
There was an implicit file size limit before. It wasn't ever documented but a file that would take more than around 5 minutes to download would not go through. This wasn't really intended but we didn't have the need for supporting large files, and an implementation with that restriction was simpler.
Now that some creators have told me that they are interested in publishing new databases that download such large files (think of around 1 GB), I gave official support to this. And technically any arbitrary large file can be downloaded after this release.
Of course, there are other restrictions of the device that won't let you, for example, download a file of 1 TB, but that's quite obvious, isn't it?
Archive spec
This one is for the database creators. We are adding official support for installing files through ZIP bundles with this feature.
Before it was possible, as it was being used by the distribution_mister database to install things like the Cheats, but it wasn't documented and it had some rough edges.
So now that this feature is on-spec, I encourage creators to use it to distribute files more efficiently.
It has two modes of operation: you may either extract all files to a target folder, or you can select only a subset of the files you want to extract from the ZIP into different locations on the MiSTer device.
In the future I'd like to support more formats other than ZIP.
Here is the doc with the full spec.
Log rotation
Now you'll see 5 log files in the same folder as the main log (Scripts/.config/downloader/). If you were paying attention this is something you could see already. In fact, this feature was shipped shortly after the 2.3 release, but I never announced it properly. Well, now I am!
If you want to disable it, just set rotate_logs to false under the [MiSTer] section of the downloader.ini file. But before you do so, I'll ask you to think twice. This is a very valuable feature to understand what has been installed in previous runs, and it can even help with debugging if there was an error. So I strongly recommend leaving it on!
Wrapping up
As always, this wasn't all. There is quite a bit more going on in the CHANGELOG that I recommend you check if you are interested.
We'll see each other again next week with the new Update All release.
Until then, have fun!