All ROM Sets Are in 7zip Format, Why?

[XtraSmiley]

I got a new SD card so decided to do a major update to my MiSTer and used Mr. Fusion to create a new image. That being said, when I went to grab the newest sets of all the 8 and 16 bit games, they were all available as no-intro sets, but they are all folders with the ROMs in the 7zip (7z) format.

MiSTer cannot read them in this format, so I'm forced to unzip them all, and now they take up double the space. I can't seem to zip them using my Mac into a format that MiSTer can read, so I've gone with them unzip in the folders for now. Correction, Mac can zip, but creates extra bullshit files that also eat up room.

So:

1. Am I doing something wrong? Can MiSTer read 7zip? (A: No it cannot)
2. Is there a script so that MiSTer can read 7zip? (A: Not yet)
3. If no to both above, why are all the available ROM drops now in 7zip format? (A: It looks like because the person who uploaded them chose 7zip b/c it has slightly more compression than Zip and didn't know or care about MiSTer users)

Thanks guys/gals!

[JudgeDrokk]

Maybe the person that ZIPPED them does not have MiSTer.
Many prefer the 7ZIP format for archiving....

[XtraSmiley]

Yeah, these are all coming from the Reddit repository, but with people using them in basically 3 ways, MAME, Retroarch, and MiSTer, it seems odd to provide them all in 7zip, cutting out 1/3 of most users.

I was just hoping there was something that I was missing.

[AngelicLiver]

7z files tend to have a much higher level of compression probably saving them from a lengthy upload and storage fees. It is preferred when archiving and storing easily compressible files like binaries. And no, MiSTer can't use 7z files.

Just deflate them with 7zip, use them as-is or recompress them with plain ol' .zip .

[jca]

One time I made a batch file (modified from something I found on the web) to solve such a problem.
It necessitate first a manual part.
The script I post is made to fork in M:\, modify to your will. It also echo text leftover from the original batch and the names echoed are wrong but this does not matter.
It also uses the command line 7zr.exe.
First step: create a M:\zip-in directory unzip everything .7z using 7zip in that directory but each zip must be unzipped in its own directory within zip-in.
If I remember correctly you can do this in one go by selecting all the .7z files.
Place the batch file and 7zr.exe in M:\.
Run the batch and you should find all your zips in M:\zip-out-<Date>.

Be careful with the drive letter which for me was M:, modify every M: to your choosing.

[jca]

Note:
This was a Quick and Dirty job: the batch file could be improved and made more automatic.
Choosing the drive and eventually a subdirectory.
Doing the un7zipping automatically.
But this is easily done manually, the burden it to rezip all the stuff which is what the batch file does.

[XtraSmiley]

Thanks for all the help! I think I'll just create an account with IA and comment that the original poster go back to .zip for all the MiSTer users out there!

[thisisamigaspeaking]

All what ROM sets? update-all is pulling down "all" the ROM sets in a format that is usable my MiSTer, right?

[jca]

All what ROM sets? update-all is pulling down "all" the ROM sets in a format that is usable my MiSTer, right?

See:

I got a new SD card so decided to do a major update to my MiSTer and used Mr. Fusion to create a new image. That being said, when I went to grab the newest sets of all the 8 and 16 bit games, they were all available as no-intro sets, but they are all folders with the ROMs in the 7zip (7z) format.

[DiamondDave]

Really interesting stuff!!!!

[redsteakraw]

7zip has the best compression / is freely available decompresser. It is way better than rar files and dealing with winRar.

[thisisamigaspeaking]

All what ROM sets? update-all is pulling down "all" the ROM sets in a format that is usable my MiSTer, right?

See:

I got a new SD card so decided to do a major update to my MiSTer and used Mr. Fusion to create a new image. That being said, when I went to grab the newest sets of all the 8 and 16 bit games, they were all available as no-intro sets, but they are all folders with the ROMs in the 7zip (7z) format.

Sorry, doesn't really matter, but I still don't understand. Where is update-all getting the archives? Or is "no-intro set" the key part here, not sure what no-intro means. No crack group intros?

[jca]

The update script only update mame, hbmame and BIOS roms from a repository specially made for MISTer. For game roms you have to find them yourself, they are usually grouped into rom set, one series of them are the "No_intro" sets, not sure where the name comes from. There are many such set, like TOSEC sets, ...

[thisisamigaspeaking]

The update script only update mame, hbmame and BIOS roms from a repository specially made for MISTer. For game roms you have to find them yourself, they are usually grouped into rom set, one series of them are the "No_intro" sets, not sure where the name comes from. There are many such set, like TOSEC sets, ...

Update is different from update_all. Update_all downloads ROMs for the arcade games as well, as I remember.

[jca]

Update script was a shortcut for update all script.

[thisisamigaspeaking]

I had to download https://github.com/theypsilon/Update_All_MiSTer separately and as I remember there was already an update.sh script included with the MiSTer distribution that didn't do as much (like download all the arcade ROMs).

[jca]

Update.sh is the old updater which has only a few days to live and has been replaced by the Downloader. The update all script existed before the new downloader and used the update.sh. There is/was an option in the current update all to use the old update script but this will be gone by the end of the year, the option may already be gone.

[Malor]

If they make update_all a default part of Mister, the project could get shut down. Right now they can get away with writing the emulators safely, because users have to add their own ROMs. If they include a tool by default that goes and get all the ROMs, copyright holders could land on the project like a ton of bricks.

It's important that update_all be a separate thing that users have to add themselves. The project risks total destruction if it isn't. Many copyright holders, Nintendo in particular, do not fuck around.

The default distro needs to only provide or download ROMs that have been freed or licensed acceptably. Numerous old computer ROMs, for instance, have been released to the public domain. But, say, Kickstart and Workbench on the Amiga aren't just under copyright, they're still being sold today.

If theypsilon wants to hang himself way out over the abyss, it's very nice for the rest of us, but if the project provides that code, the whole project is hanging there with him.

[neogeo81]

just go to archive.org and search for "htgdb-gamepacks" and thank me later.

[jca]

The default is the downloader. Update all is not an official part of MISTer.

Update.sh is the old updater which has only a few days to live and has been replaced by the Downloader. The update all script existed before the new downloader and used the update.sh. There is/was an option in the current update all to use the old update script but this will be gone by the end of the year, the option may already be gone.

I had to download https://github.com/theypsilon/Update_All_MiSTer separately ...

[thisisamigaspeaking]

I don't think this conversation has flowed consistently from my first comment here but it doesn't matter.

[jca]

Such is life on the forum.
The original post was about converting a bunch of .7z into .zip.
As I had the same problem a while back and solved it I posted the batch file which makes life much easier for those who encounter this problem.
At some point you made a legitimate post regarding .7z and the update all script which I tried to answer but my first answer was not the best, I hope that now you understand what I was trying to say.
I also pointed to the fact that update.sh will be gone by the end of the year as it had been replaced for a while by downloader.sh and that update_all.sh had/have an option to still use update.sh instead of the default downloader.sh.
Now some do not read my post properly or try to read between the lines improperly and imagine that update_all.sh is part of the official MISTer.
I hope that the batch I posted helped some users.
I hope I also clarified for you the question about the romsets in .7z format and that they have nothing to do with the update all script.
The rest is just noise.

[thisisamigaspeaking]

Yeah I really just didn't understand what "All ROM sets" meant here. Hope I haven't hijacked the thread trying to clarify it.

[jca]

No hijacking, as I said it was legitimate. I guess you don't have a MISTer for that long. When I started I also had a lot of questions which I asked on some threads but were slightly off topic but it is the only way to learn.

[FPGA64]

Instead of moaning about the format of the zipping, you should be grateful to the uploader who spent their time uploading the sets. Mister users are a tiny fraction of the Mame and Retroarch users and as shown above a simple batch file can easily convert the roms to the older, less storage efficent zip format

[XtraSmiley]

Yeah I really just didn't understand what "All ROM sets" meant here. Hope I haven't hijacked the thread trying to clarify it.

Yes someone replied to you and you didn't get it.

I am asking about 8 bit and 16 bit ROMs. Think Sega SMS, Genesis, Nintendo NES, SNES, TurboGrafx16 ect.

Update_all ONLY downloads arcade ROMs, not the ROMs I created this thread to ask about.

This question is a simple run through of the most modern ROM pack available online being in the 7zip formate, which MiSTer cannot handle (currently).

[XtraSmiley]

All what ROM sets? update-all is pulling down "all" the ROM sets in a format that is usable my MiSTer, right?

No, just arcade ones. See my answer above.

[Lightwave]

This question is a simple run through of the most modern ROM pack available online being in the 7zip formate, which MiSTer cannot handle (currently).

The simple answer is that 99.9% of people using ROMs don’t use or care about MiSTer. They are using a PC, Raspberry Pi, or some other device for emulation with Retroarch, etc.

[thisisamigaspeaking]

I am asking about 8 bit and 16 bit ROMs. Think Sega SMS, Genesis, Nintendo NES, SNES, TurboGrafx16 ect.

Sounds like "console roms". No need to be curt, it wasn't clear to me what you were talking about. The arcade machines and computers are 8- and 16-bit as well and I see "all ROMs" and I am wondering what "all" is. Not "all roms" are in 7zip.

[aberu]

Correct, not all sets are in 7zip, but many on the archive are because 7zip compression is better, so it reduces the upload time.

It's best to just unzip the .7z roms, and then if you want to rezip them again for compression you can zip all of them into one zip file, no need to do them individually. The exception being, don't zip up cd images, they won't work very well at all with MiSTer when zipped up.