Update all and JT Core: Strange Things Happen

[jca]

On my system I have these 2 mras:
Altered Beast (set 2) (MC-8123B 317-0066)
Altered Beast (set 8) (8751 317-0078)
Set 8 works, set 2 says rbf missing (jts16b2) which is effectively missing and also has been removed from Github jtbin. I suppose it is the buggy core and the mra is a leftover. I wonder why it was not removed.

While checking the presence of the core mentioned above I also noticed that many jtcores are "duplicates". Example:
jts16b_20220422.rbf
jts16b_20220428.rbf
This is the second time such thing happened and it only happens with jtcores.

[LamerDeluxe]

I have Altered Beast sets 2, 4, 5 and 6 and they all work. I have no duplicate JT cores. The only duplicate arcade cores I found are ones I copied manually before they were included with update_all.

For what it's worth, I always update over a wired internet connection, from the mister menu.

[Alkadian]

I have Altered Beast sets 2, 4, 5 and 6 and they all work. I have no duplicate JT cores. The only duplicate arcade cores I found are ones I copied manually before they were included with update_all.

For what it's worth, I always update over a wired internet connection, from the mister menu.

Same here, just today I have started from a clean install by redownloading everything again.

[jca]

The mras I mentioned were main mras. After reading your post I checked the alternatives: I have the same as you do and the set 2 is the same as the one I have as main except this time it refers to jts16b which exists. The one in main must be some kind of fossil mra.
I also use update via wired connection but using ssh.
I will delete the bad mra and the oldest "duplicate" rbfs. I will watch carefuly on the next jt update for these "duplicate" rbfs as it is the 2nd time it happens.

[LamerDeluxe]

The mras I mentioned were main mras. After reading your post I checked the alternatives: I have the same as you do and the set 2 is the same as the one I have as main except this time it refers to jts16b which exists. The one in main must be some kind of fossil mra.
I also use update via wired connection but using ssh.
I will delete the bad mra and the oldest "duplicate" rbfs. I will watch carefuly on the next jt update for these "duplicate" rbfs as it is the 2nd time it happens.

Ah, I assumed it was in alternatives. Confirmed, I also have the non-working set 2 in main. Date here is 16-10-2021.

[jca]

Next time I will state if it is main/alternatives, I completely forgot.
I did not check the date of the bad mra and I just deleted it before your post.
There are also other funny things happening with jtcores: I have/had beta versions but have never enabled beta.
I had Hippodrome, Robocop, ... at one time but now they are public. I still have Rastan and some others, I don't remember which ones, which are still in beta.

[LamerDeluxe]

I wonder if that was caused by the repository putting things in the wrong place.

[jca]

So far I have been unable to figure it out. Some users also mentioned that jt mras were not downloaded (cps2 & S16). I think it happened once to me but I am not 100% sure.
After deleting the bad mra and the duplicates rbf I ran the update script, I did not update in the last few days and was missing the updates from 04/29 and 05/06, nothing strange was downloaded.

[theypsilon]

I wonder if that was caused by the repository putting things in the wrong place.

That's most probably the case. Whenever there is something "strange" on update_all or downloader regarding JT cores, please check these repositories first:
https://github.com/jotego/jtcores_mister
https://github.com/jotego/jtpremium

Update All & therefore Downloader are just installing the same structure you see there but in your MiSTer. So if there is anything missing or wrong there, of course Downloader will propagate that to your MiSTer.

Just report that in the proper channels (the GitHub repositories, JT Discord, etc...), and it will be addressed very quickly. Maintaining all of these MRA and core files is not easy at all, because the volume of files & the amount of changes are quite high, and there is just one or two persons working on it. So of course the maintainers of these files will do mistakes from time to time.

From the side of Downloader, there is almost 0 room for errors in just downloading these files. I know it doesn't sound very humble, but it's in fact a solved problem for many months already. The code in charge of basic handling of files is all automatically validated by a very exhaustive test suite that focuses in these particular procedures. And is very well protected against regressions because the total test coverage is very high (over 90%). Bugs on Downloader do happen, like in any software, but they will likely show up in other areas, like in new features or in uncommon scenarios/setups.

[jca]

Thanks for the explanations.
This weekend I discovered another strange thing which I suspect is due to the update script itself.
On the first post of this thread I mentioned that I found "duplicate" jtcores like jts16b_20220422.rbf and jts16b_20220428.rbf.
I manually deleted the old ones.
This weekend I had to check something in _Arcades/cores and found these 3 files:
jts16_20220422.rbf.to_be_deleted
jts16b_20220422.rbf.to_be_deleted
jts16b3_20220422.rbf.to_be_deleted
First strange thing is that these files are the size of a real rbf, not some kind of sticky note, and as I had deleted them they must have been downloaded at some point.
Second strange thing is that they are marked as to be deleted but no one deleted them.
Today I ran update all again to update SNK Triple Z80 Arcade hardware and after checking _Arcades/cores these to be deleted rbfs are still there. I will manually delete them but I wonder what happened in the first place. I guess it is due to the fact I previously deleted them manually but I also deleted other "duplicates" rbfs at the same time as the one mentioned (like jtmikie, jttrack) but these did not produce these to be deleted rbfs

[theypsilon]

Thanks for the explanations.
This weekend I discovered another strange thing which I suspect is due to the update script itself.
On the first post of this thread I mentioned that I found "duplicate" jtcores like jts16b_20220422.rbf and jts16b_20220428.rbf.
I manually deleted the old ones.
This weekend I had to check something in _Arcades/cores and found these 3 files:
jts16_20220422.rbf.to_be_deleted
jts16b_20220422.rbf.to_be_deleted
jts16b3_20220422.rbf.to_be_deleted
First strange thing is that these files are the size of a real rbf, not some kind of sticky note, and as I had deleted them they must have been downloaded at some point.
Second strange thing is that they are marked as to be deleted but no one deleted them.
Today I ran update all again to update SNK Triple Z80 Arcade hardware and after checking _Arcades/cores these to be deleted rbfs are still there. I will manually delete them but I wonder what happened in the first place. I guess it is due to the fact I previously deleted them manually but I also deleted other "duplicates" rbfs at the same time as the one mentioned (like jtmikie, jttrack) but these did not produce these to be deleted rbfs

These to_be_deleted files are not coming from Downloader, but from the old Updater. You should basically never use the old Updater anymore since is deprecated, and whatever that updater recently installed in your system you'll have to clean it by hand.

[jca]

Thanks, I will delete the old updater as I remember that I recently used it by mistake. I usually run all updaters via ssh and that day I typed update.sh instead of update_all.sh. I recently redid my setup (SD card + USB drive) starting with empty SD card and USB drive in order to test the new downloader and when I copied files from my old setup (mainly games and some customs scripts) I also copied the old updater just in case.

I also have another question regarding downloader.ini: it says to place it in the root of the SD card, as it looked strange to me I placed it both in the root and in Scripts. Which one shall I keep?

[theypsilon]

The documentation says in the root, and that's correct.