Update All 2.0 Is Here

[Xbytez]

theypsilon (@josembarroso) has after more than two years since his update_all scripts initial release, now released the second major version of Update All.

See the public patreon post for full details of the changes:

https://www.patreon.com/posts/73589304

UpdateAll2.jpg (56.36 KiB) Viewed 15571 times

[akeley]

Good stuff, it really grew to be quite a powerful tool. It's great to see wizzo's scripts are now included too.

[clarkkent]

The documentation notes downloader has to be activated to update to 2.0. How can the downloader be activated?

[Higgy]

Just doing it now.
Looks like you just run the Update_All Script as normal. It will say about the 2.0 update and you just need to run the Script again.
Seems to be running through.

[theypsilon]

The documentation notes downloader has to be activated to update to 2.0. How can the downloader be activated?

When you go to the Settings Screen by pressing up during the countdown, it will ask you to activate it. If it's not asking you it means you already activated it.

The upgrade to Update All 2.0 only happens within MiSTer, not with the updater-pc btw.

[lamarax]

When you go to the Settings Screen by pressing up during the countdown, it will ask you to activate it. If it's not asking you it means you already activated it.

The upgrade to Update All 2.0 only happens within MiSTer, not with the updater-pc btw.

Good evening José,

Thank you for bringing us this huge milestone! Every module seems to have been "up-tuned", and the script runs faster than ever.

However, there's something strange going on with the bios_db sequence; upon first launch yesterday, it has gone ahead to delete the bios files from my ..fat/games/PSX folder specifically, but doesn't download them anew.

No big deal as I put them back on manually once I figured out what's happened, just something that you might want to investigate (sorry, can't file a github issue).

[Edit]I think the problem hides in here somewhere,

Code: Select all

"PSX": {
        "boot.rom": {
            "file": "SCPH7001.BIN",
            "hash": "1e68c231d0896b7eadcad1d7d8e76129",
            "size": 524288
        },
        "boot1.rom": {
            "file": "SCPH7000.BIN",
            "hash": "8e4c14f567745eff2f0408c8129f72a6",
            "size": 524288
        },
        "boot2.rom": {
            "file": "SCPH7002.BIN",
            "hash": "b9d9a0286c33dc6b7237bb13cd46fdee",
            "size": 524288
        },
        "sbi.zip": {
            "file": "sbi.zip",
            "hash": "706d60ab2dfdbfdd53be2069cb85a1fe",
            "size": 218953
        }
    },

as it doesn't convert SCPH700..bin into boot..rom according to build_db.py at line 44

[CartoonDonkey]

Slightly off topic:
Any way to simply change the overall font for scripts shown in the console? Default font is huge on my 1084s crt. And the whole console screen gets shifted to the left. (I'm using this screenmode for the main menu, which is the only thing that really works for me.)

[Menu]
video_mode=640,6,56,66,224,14,8,18,12000
vga_scaler=1

[lamarax]

Slightly off topic:
Any way to simply change the overall font for scripts shown in the console? Default font is huge on my 1084s crt. And the whole console screen gets shifted to the left. (I'm using this screenmode for the main menu, which is the only thing that really works for me.)

[Menu]
video_mode=640,6,56,66,224,14,8,18,12000
vga_scaler=1

That's an issue pertaining to the linux console properties, not any script in particular.

[theypsilon]

Good evening José,

Thank you for bringing us this huge milestone! Every module seems to have been "up-tuned", and the script runs faster than ever.

However, there's something strange going on with the bios_db sequence; upon first run yesterday, it has deleted the bios files from my ..fat/games/PSX folder specifically, but doesn't download them anew.

No big deal as I put them back on manually once I'd figured out what's happened, just something that you might want to investigate (sorry, can't file a github issue).

[Edit]I think the problem hides in here somewhere,

Code: Select all

"PSX": {
        "boot.rom": {
            "file": "SCPH7001.BIN",
            "hash": "1e68c231d0896b7eadcad1d7d8e76129",
            "size": 524288
        },
        "boot1.rom": {
            "file": "SCPH7000.BIN",
            "hash": "8e4c14f567745eff2f0408c8129f72a6",
            "size": 524288
        },
        "boot2.rom": {
            "file": "SCPH7002.BIN",
            "hash": "b9d9a0286c33dc6b7237bb13cd46fdee",
            "size": 524288
        },
        "sbi.zip": {
            "file": "sbi.zip",
            "hash": "706d60ab2dfdbfdd53be2069cb85a1fe",
            "size": 218953
        }
    },

as it doesn't convert SCPH700..bin into boot..rom according to build_db.py at line 44

This is not related to this release, as you are describing Downloader/BiosDB behavior and these hasn't been touched since August/May respectively. But it might be that you plugged a USB drive with the games folder, and now it got installed there instead.

[FoxbatStargazer]

Yeah I recall having to edit some ini to point to my USB drive for arcade ROMs. But there's no setting here in 2.0 and yet it seems to have figured it out on its own?

[Malor]

Slightly off topic:
Any way to simply change the overall font for scripts shown in the console? Default font is huge on my 1084s crt. And the whole console screen gets shifted to the left. (I'm using this screenmode for the main menu, which is the only thing that really works for me.)

[Menu]
video_mode=640,6,56,66,224,14,8,18,12000
vga_scaler=1

You might want to check that you have this setting:

Code: Select all

fb_size=1

That sets the size of the console font. 1 is smallest, 2 is twice as large, 4 is four times as large.

I'm not sure how to move your picture to the right. You should be able to move the menu screen right and left with the CRT dials, but video generated by cores may end up in a different place, meaning you'd have to adjust it again. That would get very old.

[georgemog]

This is a good script and I am enjoying using it. It did have me confused when I tried using it via SSH and the terminal was set for vt100, since every time I tried to bring up the settings menu it would crash. This was easily solved by changing the terminal type to xterm-256 colours. It might be worth checking the term type at some point.

This is the error that I see:-

Launching Update All


-------- Update All 2.0 ---------
The All-in-One Updater for MiSTer
---------------------------------
- Powered by Downloader 1.5 -


╔═════════════════════════════════════════════════╗
║ Become a patron to unlock exclusive features! ║
║ www.patreon.com/theypsilon ║
╚═════════════════════════════════════════════════╝

Reading sections from /media/fat/downloader.ini

Sequence:
- Main Distribution: MiSTer-devel
- JTCORES for MiSTer
- Names TXT: CHAR18 Common EU
- BIOS Database
- Arcade ROMs Database
- Coin-Op Collection
- MiSTer Extensions (wizzo)
- Arcade Organizer


*Press <UP>, To enter the SETTINGS screen.
*Press <DOWN>, To continue now.

Traceback (most recent call last):
File "/tmp/dont_download.zip/update_all/main.py", line 32, in main
exit_code = execute_update_all(
File "/tmp/dont_download.zip/update_all/main.py", line 48, in execute_update_all
return factory.create(env).full_run()
File "/tmp/dont_download.zip/update_all/update_all_service.py", line 122, in full_run
self._countdown_for_settings_screen()
File "/tmp/dont_download.zip/update_all/update_all_service.py", line 170, in _countdown_for_settings_screen
self._settings_screen.load_main_menu()
File "/tmp/dont_download.zip/update_all/settings_screen.py", line 57, in load_main_menu
run_ui_engine('main_menu', settings_screen_model(), self)
File "/tmp/dont_download.zip/update_all/ui_engine.py", line 76, in run_ui_engine
curses.wrapper(loader)
File "/usr/lib/python3.10/curses/__init__.py", line 94, in wrapper
return func(stdscr, *args, **kwds)
File "/tmp/dont_download.zip/update_all/ui_engine.py", line 74, in loader
ui.execute()
File "/tmp/dont_download.zip/update_all/ui_engine.py", line 101, in execute
ui_section_factory = self._ui_application.initialize_ui(self, self._screen)
File "/tmp/dont_download.zip/update_all/settings_screen.py", line 102, in initialize_ui
drawer_factory, theme_manager = self._settings_screen_printer.initialize_screen(screen)
File "/tmp/dont_download.zip/update_all/settings_screen_standard_printer.py", line 80, in initialize_screen
curses.init_pair(COLOR_PAIR_RED_OVER_WHITE, color_red, color_white)
ValueError: Color number is greater than COLORS-1 (-1).

Update All failed!

[Insert Disk Two]

Hopefully one day it will update console games too

[Malor]

The normal solution for terminal problems like that is for the script to use the ncurses library, which has a giant list of most terminals ever made. It does automatic translation when you tell it, say, 'move to position 36,14 and print "HI THERE!"'. The Mister distro has a small subset of that full library; there's about 20 terminal defintions there.

It's awkward to look at the script because theypsilon forces it to be downloaded fresh each time, and I don't feel like reverse-engineering that whole system yet *again*, but if he's using ncurses, then the display should work if you declare your terminal to be any of the twenty-ish under /usr/share/terminfo.

If that's how things are working, then the instant fix is to do what you did, and just run in xterm-256color, or any other terminal in /usr/share/terminfo. If for some reason you have some alternate terminal that you want instead, you should be able to source the file from a bigger distro and copy it into the right place under terminfo. That would last until the next Linux update, probably. The permanent solution would be to get them to add it to the base Mister image.

Of course, if the script is using command codes directly, then setting to xterm-256color is probably your only option. Well, that, or talking theypsilon into using ncurses.

[jca]

Thanks for this update.
A few observations:
1. The first run takes a while to complete as it has to build a store by scanning all the files.
2. Arcade ROMs DB downloaded 53 files which I already had on my MISTer. 11 associated with Jotego cores were due to hash problem and the others to size difference. It is somewhat puzzling as previously I was using Mame Getter but is not a big deal.
3. I update my MISTer frequently but there were a few core updates when I ran V2, the cores were downloaded but the old ones were not deleted. I suppose it is related to the store not yet created and this will not happen the next time cores are updated.

[Phantoon]

I've run Update All 1.4 a few times with Downloader activated from the Miscellaneous menu (there has been no pop up so I've activate it manually). It's done the usual job but I'm still on 1.4. Any idea what I'm doing wrong?

[theypsilon]

I've run Update All 1.4 a few times with Downloader activated from the Miscellaneous menu (there has been no pop up so I've activate it manually). It's done the usual job but I'm still on 1.4. Any idea what I'm doing wrong?

Could you try again? I updated the launcher recently to insert some ntpdate mitigation code and forgot to update the MD5 hash for it on the 1.4, which needs to be correct for the migration to Update All 2.0, but now it's fixed.

[Phantoon]

Thanks, it worked! Thanks for all of the incredible work everyone does here.

[jd213]

Update All removed most of the cores in the YCConsole folder and added links to empty folders. As far as I can tell, the YC functionality isn't yet included in most of the standard console cores, at least I couldn't find the YC option in the menus...

[sid]

Does anybody know the difference between updateall, update and downloader script? Thx

[jca]

The downloader concerns only what is in the official repository: cores, mras, Linux, MISTer Main ... It is used by the update all script for the official downloads and can also be configured to download everything else.
I am not sure for the update script but as the update all script is not official the update script can be used after a fresh install to update MISTer.

[FoxbatStargazer]

Update All removed most of the cores in the YCConsole folder and added links to empty folders. As far as I can tell, the YC functionality isn't yet included in most of the standard console cores, at least I couldn't find the YC option in the menus...

See these new lines in mister.ini.

Code: Select all

vga_mode=rgb           ; supported modes: rgb, ypbpr, svideo, cvbs. rgb is default.
ntsc_mode=0            ; Only for S-Video and CVBS vga_mode. 0 - normal NTSC, 1 - PAL-60, 2 - PAL-M.

Always check the online mister.ini for settings for new features, the updater will not overwrite or add them to your existing mister.ini!

[jd213]

OK, thanks. I saw discussion here in the forum about the new framework but I didn't see a mention of ini changes.