Reducing the space taken by the HTGDB Gamepacks with symlinks
Posted: Tue Dec 14, 2021 7:03 am
Not sure if this is the right place to put it but I figured it might be helpful to someone else.
I love those gamepacks and especially love having the Game Series Collections add on which makes browsing and discovering new games great. However, I don't have much space in my sd card and deduplicating roms was not helping. Instead I figured a slightly convoluted process to optimize space.
I use two tools for this rdfind and symlinks. Both are easy to install in any linux distribution, and with either homebrew or macports on mac.
I download both the normal gamepack and the Game Series Collection add on. I extract both in the same directory. Which gives something like this:
Then I run in that directory
This removes all duplicates and creates absolute symlinks instead.
Since absolute symlinks are impractical if the path change, I convert them to relative symlinks with
The symlinks are then converted to relative paths. This makes it now easy to copy over to the mister.
To copy to the mister, we can't just copy through samba/cifs unfortunately since symlinks are not supported on network shares.
So the way to preserve the symlinks is to tar the directory
copy that tar file to the mister and then untar it with
in /media/fat/Games
Do not the h option which tells tar to preserve the symlinks on untarring
I use nohup in case the ssh connection dies while it's decompressing (it takes quite a while). If your connection dies, you can monitor the progress by going to /media/fat/Games and running this
This saves 40-60% of the space for the game packs with Game collections. For example with GBA, it went from 52GB to 32GB so saving 20GB.
I love those gamepacks and especially love having the Game Series Collections add on which makes browsing and discovering new games great. However, I don't have much space in my sd card and deduplicating roms was not helping. Instead I figured a slightly convoluted process to optimize space.
I use two tools for this rdfind and symlinks. Both are easy to install in any linux distribution, and with either homebrew or macports on mac.
I download both the normal gamepack and the Game Series Collection add on. I extract both in the same directory. Which gives something like this:
Code: Select all
d--------- 1 root root 15K Apr 28 2021 1 US - A-F
d--------- 1 root root 16K Apr 28 2021 1 US - G-P
d--------- 1 root root 17K Apr 28 2021 1 US - Q-Z
d--------- 1 root root 72 Apr 28 2021 2 Europe - A-Z
d--------- 1 root root 44 Apr 28 2021 2 Japan - A-Z
d--------- 1 root root 118 Apr 28 2021 2 Other Regions - A-Z
d--------- 1 root root 19K Apr 28 2021 2 Unlicensed - A-Z
d--------- 1 root root 154 Apr 28 2021 3 Sega 32X
d--------- 1 root root 62 Apr 28 2021 4 Betas, Prototypes, Revisions
drwxr-xr-x 1 root root 208 Apr 28 2021 4 Game Series Collections
d--------- 1 root root 138 Apr 28 2021 4 Hacks
d--------- 1 root root 9.1K Dec 14 11:08 4 Homebrew
d--------- 1 root root 5.1K Dec 14 11:08 4 Translations
d--------- 1 root root 16K Dec 14 11:08 4 VGM Music Players
d--------- 1 root root 2.8K Dec 14 11:08 5 Tools & Service Test Carts
d--------- 1 root root 968 Apr 28 2021 BIOS
d--------- 1 root root 1.9K Dec 14 11:08 SRAM & TMSS Patched for X3
d--------- 1 root root 2.0K Dec 14 11:08 Unpatched EEPROM-Save Games for X7Code: Select all
rdfind -makesymlinks true *This removes all duplicates and creates absolute symlinks instead.
Since absolute symlinks are impractical if the path change, I convert them to relative symlinks with
Code: Select all
symlinks -cr .To copy to the mister, we can't just copy through samba/cifs unfortunately since symlinks are not supported on network shares.
So the way to preserve the symlinks is to tar the directory
Code: Select all
tar cvf Genesis.tar GenesisCode: Select all
nohup tar xvhf Genesis.tar GenesisDo not the h option which tells tar to preserve the symlinks on untarring
I use nohup in case the ssh connection dies while it's decompressing (it takes quite a while). If your connection dies, you can monitor the progress by going to /media/fat/Games and running this
Code: Select all
tail -f nohup.out