[New version!] SD Card Migration Utility for MiSTer: An easy way to migrate to a new SD card, no PC required

[evilsim]

with Clonezilla

Interesting idea. I was thinking about trying CloneZilla or Macrium Reflect recently..

Macrium can expand a certain volume to any size based on available space on the destination drive/device, just before actually doing the restore.

It would also be good to use a serial console (if the mister has this? kind of like an enterprise network switch) to show where the boot process gets to, on cards which dont boot after running the migrate_sd.sh script through. Sometimes I can run the script a second time - and it works. This smells like a dying SD card.. The script is flawless it cant really mess up!

[MiSSes]

I transferred my data from a 400 to a 512gb card. I guess all went well. My mister starts and reacts as it should. Thank you!
Now I'm looking at my files on my new 512 gb card and I see a folder "found.000".
It contains 667 files named "FILE0667.CHK" It contains 4,29gb of data.
What is this folder good for and was this folder on my old card as well?
I do believe that it wasn't before the migration.

[jca]

It looks like there was some corruption on your SD card. These files contain the data from the corrupted files.

[Fed]

Upgraded from a 256 to 512 card, worked great, thanks for this functionality !

[heinzgruber]

hi,
i've used migrate_sd.sh and migrate data from a 256 gb card to a 512 gb card. the operation finished successfully.

but now on both sd cards something seems wrong: my daemonbite adapter isn't working anymore. seems like migrate_sd broke Daemonbite support. the adapter get power and my wireless 8bitdo-controller connects but MiSTer doesn't recognize inputs neither in games nor in the menu. When i open up the Definde Joystick Button menu no input is recognized either.

Also the "Switch to USB" menu entry is greyed out. So I guess something with the USB communication is wrong. But my via usb connected bluetooth and snac adapter are still working.

Edit:
Already tried d different USB ports und rund Update_all but the problem is still there.

[jca]

If you do not have any USB drive connected there is no reason to have "Switch to USB" enabled. As you see USB is working with your Bluetooth. SNAC adapters are not connected via USB but via User GPIO which uses a USB connector but is not USB.

[heinzgruber]

Okay. Thanks for your Tipps. So my deamonbite problem is very strange. Especially that it is on both sd cards after using the script.

[heinzgruber]

Suddenly my deamonbite adapter is working again so i've no idea whats happened. It never happened before. sorry for the confusion.

[tomvandeusen]

Hi there,

I'm not sure if it's an issue with my MiSTer, but when I try to load your script in the Scripts menu it just flashes some text and goes back to the MiSTer main menu. I assume from other replies that this has some sort of GUI associated with it, and I've never used a script with a GUI. I ran update_all relatively recently but it still is having this issue. Is there something I'm completely missing? My SD Card is fat32 formatted and it's plugged in via USB, but this seems more like an issue of my MiSTer not even reading your script properly.

Thanks so much for your help, and apologies since I don't really run scripts aside from update_all

UPDATE - nevermind, I figured it out. I had it on a CRT and fb_terminal was set to 0 instead of 1 (probably some legacy thing, I've had a MisTer since 2019)

[Longtime4321]

I wouldn't reccomend interfacing with scripts over a CRT. Many don't work with one.

[Fed]

I wouldn't reccomend interfacing with scripts over a CRT. Many don't work with one.

Never had any issues using scripts on a CRT

[MEGAMILK]

Would it be alright to cancel the script half way through (by rebooting the MiSTer)? I want to rerun it without including the games folder as it’s only going 50Kb/s.

[RowanDDX]

Worked a treat! Thanks.

Took 1hr25min to copy what was basically a full 32GB card over to a new Sandisk 128GB card. Lots of little files, obviously takes ages.

Only thing that tripped me up initially was that the script failed because I hadn't formatted the microSD card first. Error given was "unable to successfully partition the disk". So I used Windows to format it to whatever the default was (exFat I think), then the script was happy after that. So to be pedantic, to say "no PC required" .. well the PC was required in this case to format the new card. Or maybe there's a MiSTer script to format cards?

[lroby74]

I've 3 Misters and each one has 32GB microSD with only arcades, ScummVM with games and nothing else.
All my games are on NAS so i can play them from any Mister and are protected with Raid 5 from losing them for faulty HDDs

[RealLarry]

I've 3 Misters and each one has 32GB microSD with only arcades, ScummVM with games and nothing else.
All my games are on NAS so i can play them from any Mister and are protected with Raid 5 from losing them for faulty HDDs

That's it! Same here...but I'm having all cores and its stuff on a NAS. WiFi speed is fast enough to wait just a few seconds.

[disq]

First time it failed to partition the card (brand new sandisk card, just unpackaged and stuck right in, not sure what kind of partition table was on it) but I insisted (by rebooting) and it's chugging along now. Thanks for the script!

[lagerfeldt]

I hope you will find this script useful. Please let me know if you have any questions, problems or nice words.

Works like a charm, thanks! Much easier than doing DD and having to copy block by block of unused data on a 512 GB card.

I'm linking to your script from here: https://misterreplay.com/vault.html

One thing that did surprise me a bit after inserting the newly migrated card for the first time into the MiSTer was that the startup sound and BGM didn't seem to work. Silence.

It appears that the first time a newly migrated card is inserted into the MiSTer and it's powered up it takes a bit of time before something syncs up (20-30 seconds in my case).

Any idea why this is so?

[LamerDeluxe]

I had an odd bug where the script supposedly finished copying 400GB in one minute. The next time, I opted to do a manual selection of folders to copy and that seems to be working correctly. (got interrupted by MiSTer SAM, which I had turned off before, but sometimes it behaves a bit like a virus).

Tried to clone with AOMEI partition assistent before that. Chose exact clone, which, after five hours, I found out was the wrong decision. You can't resize exFAT partitions. It had placed the recovery(?) partition right between a 400GB sized one and the rest of the almost 600GB unused space. This turned out to be unfixable.

The quick option in AOMEI might have worked correctly (it mentions something about being able to resize a partition). I'm sure it would also take five hours. Though I expect this migration utility to take much longer, judging by the average copying speed. Major advantage being that it should partition correctly. And it's better to leave the MiSTer on than a whole PC, just to copy files.

[Psygho]

Is t normal to display nothing ? First a got lots of scrolling text... Wich files are copied.. Had to leave for an 45min, so i put tv off. and switched it on.....
But no screen.... and mine mster a is been showing the green and red led alternating, dim, bright dim, and alternate between green and red.

Come from a 200gb filled 256 micro sd, to a 512mb micro sd.

Update
Well it is not. Reset mine mister, and choose manual, en then run, I can see wich file is copied and how fast it is in doeing so..
Runs for more then 2 hours, so i let it run overnight..

Update 2
Let it run overnight, and get a message that X amount was copied, with 4mb/s.
Guess all went perfect the second time

[AfterBurnett]

You absolute legend! I am currently migrating from a 512GB card to a 1TB so I guess the MiSTer is gonna be busy for quite some time, haha. Thanks heaps for this!!!!

[rhester72]

Migrated from 1TB to 1.5TB without issue - thank you!

[MrCaution]

The utility was working great, now it seems to have frozen, just a blinking cursor. If I restart will it pick up at the same point or completely restart?
(Edit) It stalled for about an hour, then just started working again. Apparently it says a file disappeared?

[misterrocks]

Can this script rsync from one already-installed SD card to another in order to only transfer the difference?

[misterrocks]

Is there a json file in the GitHub so I can have update_all pull in any changes to the script?