Cannot Update Main MiSTer Using Update All

[Ko2]

I have it enabled in the settings but the update all script no longer updates the main Mister. Is there a solution?

[jca]

What is your setup?
What happens when you run the update?

[Ko2]

Reinstallation fixed it. Thank you.

[jvas]

I have similar problem, no clue what to do:

286589828_1089764471922470_8364063213124567298_n.jpg (845.96 KiB) Viewed 7735 times

[jca]

I got this error a while back ago and made a post about that. What I did was to download MISTer from the repository and overwrite the one in /media/fat.
I did not shut down MISTer when the error happened but if you did you should still be able to start MISTer, which I suppose will not display anything, and you should be able to use FTP. If that does not work you can always use sneakernet.

[jvas]

os.replace(src, dst, *, src_dir_fd=None, dst_dir_fd=None)
Rename the file or directory src to dst. If dst is a non-empty directory, OSError will be raised. If dst exists and is a file, it will be replaced silently if the user has permission. The operation may fail if src and dst are on different filesystems. If successful, the renaming will be an atomic operation (this is a POSIX requirement).

[jvas]

It is not about the Mister files, it is about that the update script is not correct.

[jca]

I never said it was about the Mister files, just how I went around the problem. When it happened to me and made a post regarding the problem no one experienced this problem and I did not get any feedback so I decided to wait and see. Now that the problem happened to other users an issue should be opened for this problem.

[jvas]

I never said it was about the Mister files, just how I went around the problem. When it happened to me and made a post regarding the problem no one experienced this problem and I did not get any feedback so I decided to wait and see. Now that the problem happened to other users an issue should be opened for this problem.

Ok, what I wanted to say is: no matter whether I update the mister file manually or not, the update script will fail next time anyway.
Shall I open the issue myself? Or what is the "protocol" to do that?

[neogeo81]

just do a fresh install on the sd card. its the simpliest way to set everthing right

[jca]

You can try to delete the /media/fat/Scripts/.config directory and run the update_all script, the script will think it has never been run and will take longer than usual but it should, hopefully, fix the problem. If not redoing the SD card will be the easiest method.

[jvas]

OK, the Python documentation clearly says, that the "os.replace" function may fail (this is the situation in my case) if the source and destination files are on different filesystems. Fresh install is temporary solution, the problem arise as soon as there is a new update necessary.

[neogeo81]

OK, the Python documentation clearly says, that the "os.replace" function may fail (this is the situation in my case) if the source and destination files are on different filesystems. Fresh install is temporary solution, the problem arise as soon as there is a new update necessary.

how can it be temporary if it works for everyone else?

[jvas]

it works for everyone else

Wow, that is a bold statement.

I explained why: os.replace may fail if the source and destination are on different filesystems.

[jvas]

If you are still not convinced: https://manpages.debian.org/testing/man ... html#EXDEV
The os.replace function uses the renameat2

[jvas]

insted of replace fs2/src -> fs1/trgt

copy fs2/src -> fs1/src.tempname (to the same filesystem as trgt with the same rights as trgt)
rename fs1/trgt -> fs1/trgt.tempname : this is the critical point, where trgt no longer exists in its original name
rename fs1/src.tempname -> fs1/trgt : it can also fail (very low chance)
delete fs1/trgt.tempname (or create a backup if needed)
delete fs2/src if needed

[jca]

It does not work for everyone else as it happened to me before as I said earlier. I went around the problem by manually installed MISTer. From what jvas said I guess my suggestion of deleting the .config directory will not fix the problem. It only happens when using a USB drive, may be only for specific setup as so far only jvas and me reported the problem. May be the update script should download the new file on the SD card instead of the USB drive and the replace function would not fail.

[neogeo81]

i am pretty sure if you make a blank start it will work

[jvas]

i am pretty sure if you make a blank start it will work

Till the next update!
I explained why it won't work. Would you please take the time and explain why it should work?

[neogeo81]

i am pretty sure if you make a blank start it will work

Till the next update!
I explained why it won't work. Would you please take the time and explain why it should work?

because i never had this issue. never

[jvas]

i am pretty sure if you make a blank start it will work

Till the next update!
I explained why it won't work. Would you please take the time and explain why it should work?

because i never had this issue. never

LOL

[neogeo81]

Till the next update!
I explained why it won't work. Would you please take the time and explain why it should work?

because i never had this issue. never

LOL

why shouldn't the main mister not update? explain
why wouldn't a new fresh install not solve the problem? explain

Don't LOL like a 5 year old mate.

[jvas]

because i never had this issue. never

LOL

why shouldn't the main mister not update? explain
why wouldn't a new fresh install not solve the problem? explain

Don't LOL like a 5 year old mate.

I already did explain, but it seems you fail to read:
As soon as there is a new version of the Mister file, the update script will try to do the same: it tries to move the Mister file from one filesystem to another using os.replace() which will fail again (description in the link I post before).

[neogeo81]

LOL

why shouldn't the main mister not update? explain
why wouldn't a new fresh install not solve the problem? explain

Don't LOL like a 5 year old mate.

I already did explain, but it seems you fail to read:
As soon as there is a new version of the Mister file, the update script will try to do the same: it tries to move the Mister file from one filesystem to another using os.replace() which will fail again (description in the link I post before).

ok but why do I not have this issue then? i almost run update_all every day.

[jca]

ok but why do I not have this issue then? i almost run update_all every day.

I suppose you have everything on the SD card or if you use USB you only have the games directory on it, that would explain why.

[jca]

@jvas
Could you post some information on your setup:
What do you have on your USB drive?
Are you using symlinks?
What is the base_path in your downloader.ini?

It is too bad that the search function on the forum sucks and I am unable to find some posts I made but this is what I think happened to me:
First the problem started with version 1.5 of the downloader script released on 05/05/2022.
Since then there has been only 2 MISTer main updates one on 05/17 and another one on 05/29.
I got the error on the 05/17 update but I am pretty sure I did not get it on the 05/29 update for reasons I explain bellow.
On my SD card I only have the config and Scripts directories. Everything else is on the USB drive and I have symlinks from the SD card to each of the directories on the USB drive.
In my downloader.ini I had base_path = '/media/usb0/' because Mane Getter and HbMame getter did not find any mra if I used base_path = '/media/fat/'. It should have worked as I have all these symlinks on /media/fat.
Having a base_path of /media/usb0 resulted in the downloader creating a Scripts directory on the USB drive. It really annoyed me of having 2 Scripts directories, one on the SD card and one on the USB drive, and I wanted to have the Scripts directory only on the SD card.
So I made some changes with ini files:
In downloader.ini I changed the base_path to /media/fat.
I created update_mame-getter.ini
ROMMAME="/media/usb0/games/mame"
MRADIR="/media/usb0/_Arcade"
I created update_hbmame-getter.ini
ROMHBMAME="/media/usb0/games/hbmame"
MRADIR="/media/usb0/_Arcade"
Just to be sure I also created update_arcade-organizer.ini I don't think it is necessary)
MRADIR=/media/usb0/_Arcade
ORGDIR=/media/usb0/_Arcade/_Organized
I think I made these changes between 05/17 and 05/29 which is why I did not get the error on the 05/29 update.
Just to confirm I just manually reverted to MISTer 05/17, deleted /media/fat/Scripts/.config to force a full resync and ran the update all script. The update ran fine without the error and I am back to MISTer 05/29.
This does not fix the problem of the script being wrong, it just goes around the problem.

[jvas]

I have no access to my MiSTer at my workplace, but the plan was that everything except the game data are on the internal SD card. All the game binaries on the USB0.

[neogeo81]

ok but why do I not have this issue then? i almost run update_all every day.

I suppose you have everything on the SD card or if you use USB you only have the games directory on it, that would explain why.

exactly. why not using this way to set it up?

[jca]

I have no access to my MiSTer at my workplace, but the plan was that everything except the game data are on the internal SD card. All the game binaries on the USB0.

This is very strange. Why would the update script use the USB drive when updating MISTer main?
So far only 3 users reported this problem so I wonder if it is just a glitch in the Matrix and my explanation on why the problem appeared on update 05/17 and not on 05/29 on my system is the right explanation.

[jca]

exactly. why not using this way to set it up?

Why do I use such a setup? Because I can
More seriously:
When I started I only had a 16GB SD card, I also had a WD Elements 1GB HD which was not getting much use, I also had an old powered USB and an OTG cable so at some point I moved the games directory to the USB drive.
SD cards wear out over time and the fuller they are the faster they wear out, I also like to experiment so at some point I used "Switch to USB".
More recently Sorge stated that "Switch to USB" was kind of obsolete and using symlinks was a better solution so it what I did.
My 16GB SD card has 14GB free space so it will take a very long time before it goes bust. The day it does it is not much work to make a new one: just run Mr. Fusion, recreate the symlinks, put back my custom scripts in the Scripts directory, set back my MISTer to fixed IP, restore all the ini files relative to the update scripts, run update_all and I am done.