Amiga Core: Benchmark Numbers

[limi]

I did these tests based on questions about the MiSTer Amiga core, and that it should ideally be at least equivalent to a 68030 at 25MHz to be perfect with WHDLoad — so I thought I’d document them here too, for posterity.

I ran the two most common benchmark suites for Amiga on the MiSTer — SysInfo (quick, but crude), and AIBB (actual math and rendering tests, more representative), and ended up with this:

SysInfo (a less scientific benchmark, using only Dhrystones as a speed measurement) reports the MiSTer core as being:
• 25× the speed of the Amiga 500/600 (68000 at 7MHz)
• 19× the speed of the Amiga 2000 (68000 at 7MHz)
• 11× the speed of the Amiga 1200 (68020 at 14MHz)
• 2.8× the speed of the Amiga 3000 (68030 at 25MHz)
• 0.7× the speed of the Amiga 4000 (68040 at 25MHz)

AIBB (a much broader set of benchmarks)

Compared to a 14MHz 68020 Amiga 1200, AIBB reports the MiSTer core as having:
• 5.50× the performance on Integer Math tests
• 3.38× the performance on Graphics Rendering tests
• 9.28× the performance on Floating Point Math tests

Compared to a 25MHz 68030 Amiga 3000, AIBB reports the MiSTer core as having:
• 2.19× the performance on Integer Math tests
• 3.17× the performance on Graphics Rendering tests
• 2.05× the performance on Floating Point Math tests — note that MiSTer does not have an FPU, unlike the Amiga 3000

Compared to an 25MHz 68040 Amiga 4000, AIBB reports the MiSTer core as having:
• 1.43× the performance on Integer Math tests
• 1.08× the performance on Graphics Rendering tests
• 0.62× the performance on Floating Point Math tests — note that the MiSTer does not have a dedicated FPU, unlike the Amiga 4000

Hopefully this should put to rest that just because the cores says that it’s an 68020, it’s definitely in a very different league when it comes to performance than the 68020 computers that were shipped, and almost on the performance level of the 68040-based Amiga 4000.

(Also note that the 68030 was almost identical to the 68020 overall, just with an additional 256 byte data cache, a process shrink, and an added burst mode for the caches. So 020 vs 030 as far as MiSTer goes is kind of moot. Any instruction set optimizations did not arrive until the 68040)

(Tests performed on MiSTer v201014, Minimig v200071, with 020 CPU selected, D-Cache on, AGA, etc — the usual modern Amiga settings.)

[breiztiger]

Hi
You should notice what version of core you use
Because of recent change it may be improve

[Chris23235]

It would be interesting to see how the core would perform if the latest changes of the MiST core were implemented. The core got a major performance boost on MiST last week.

[kathleen]

@Chris23235
I own a ST-MINI (low cost version of the Mist), I will do the same tests as @limi did and will keep you posted.
Will try to find a window to do it today evening. I'm also curious to see.

[Chris23235]

This will be interesting, the MiSTer was already ahead of the MiST, here is a MiST/MiSTer comparison which was made with an older MiST core:

https://www.youtube.com/watch?v=LqeMwgD1dIo

MiST should have catched up with the new core, would be interesting to see if it would become even faster on MiSTer if these changes were implemented.

[limi]

Hi
You should notice what version of core you use
Because of recent change it may be improve

Added in the footer.

[kathleen]

I've just tried to do test with the latest Minimig-AGA core of the mist which is the revision 201017 and with the firmware revision 200111.
On the Amiga side, I used AIBB version 6.5 from Aminet

I do not know well AIBB and to be honest I did not find the Graphic Rendering Test button

Moreover the Test Code Type change the results depending what is selected.

In order to have a valid comparison, could you please let me know @limi how you conducted the tests on the Mister, I mean what did you select on AIBB to do so ?

Sysinfo (V4.3 used here) is much more easy (for me only) here are the results

Turbo at none / Turbo at Both (Chipram+Kick)
• 19.44/19.68× the speed of the Amiga 500/600 (68000 at 7MHz)
• 14.71/14.89× the speed of the Amiga 2000 (68000 at 7MHz)
• 8.45/8.55× the speed of the Amiga 1200 (68020 at 14MHz)
• 2.22/2.24× the speed of the Amiga 3000 (68030 at 25MHz)
• 0.56/0.57× the speed of the Amiga 4000 (68040 at 25MHz)

CPU is recognized as being a 68020 running at 99.00 Mhz

[limi]

There’s a menu item to “Create Module” (or something to that effect), which will run all the tests and give you a summary.

[Hodor]

If you want to try and compare here is the rbf file including the changes added two days ago.

[tontonkaloun]

If you want to try and compare here is the rbf file including the changes added two days ago.

Sorgelig's version ?

[kathleen]

Thank you @limi and @Hodor, will redo the tests today evening all will post the results right after, and sorry again for my lack of knowladges with AIBB.

[Hodor]

If you want to try and compare here is the rbf file including the changes added two days ago.

Sorgelig's version ?

Yep, from Mister-Devel repositories.

[kathleen]

I've just finished the test with AIBB on the Mist and here are the results. Not bad at all but quite below the Mister. Will now do the test on the Mister with the last core revision that Hodor attached.

Compared to a 14MHz 68020 Amiga 1200, AIBB reports the MiST core as having:
• 4.86× the performance on Integer Math tests
• 0.80× the performance on Graphics Rendering tests
• 8.08× the performance on Floating Point Math tests

Compared to a 25MHz 68030 Amiga 3000, AIBB reports the MiST core as having:
• 1.93× the performance on Integer Math tests
• 0.75× the performance on Graphics Rendering tests
• 1.78× the performance on Floating Point Math tests — note that MiST does not have an FPU, unlike the Amiga 3000

Compared to an 25MHz 68040 Amiga 4000, AIBB reports the MiST core as having:
• 1.26× the performance on Integer Math tests
• 0.26× the performance on Graphics Rendering tests
• 0.54× the performance on Floating Point Math tests — note that the MiST does not have a dedicated FPU, unlike the Amiga 4000

[kathleen]

Test done with the Mister and the above attached core. Of course with AGA selected and D-cache on. The results are less good than the ones of @limi regarding the graphic rendering. See below.

Compared to a 14MHz 68020 Amiga 1200, AIBB reports the MiSTer core as having:
• 5.51× the performance on Integer Math tests
• 2.78× the performance on Graphics Rendering tests
• 9.28× the performance on Floating Point Math tests

Compared to a 25MHz 68030 Amiga 3000, AIBB reports the MiSTer core as having:
• 2.19× the performance on Integer Math tests
• 2.61× the performance on Graphics Rendering tests
• 2.05× the performance on Floating Point Math tests — note that MiSTer does not have an FPU, unlike the Amiga 3000

Compared to an 25MHz 68040 Amiga 4000, AIBB reports the MiSTer core as having:
• 1.43× the performance on Integer Math tests
• 0.89× the performance on Graphics Rendering tests
• 0.62× the performance on Floating Point Math tests — note that the MiSTer does not have a dedicated FPU, unlike the Amiga 4000

[Michael1260]

With Minimig RTG Core :

[limi]

Yeah, these are the same numbers as the initial post, I rounded to fewer significant digits since benchmarks vary a little bit for every run. I didn’t see anything that would improve these numbers in the commit log since the last Minimig release, so that’s expected.

If you want to measure what RTG does for graphic performance, you have to set it up differently, but I’d rather not go down that rabbit hole. The intent with my post was just to show that the “020” moniker on the Minimig core unfairly lets people think that e.g. an Amiga with a 25MHz 68030 accelerator board must be faster than the MiSTer Minimig, whereas it’s more than 2× faster than that in benchmarks. The reason why this matters is that WHDLoad adds some overhead and protection that requires you to have a slightly more capable Amiga than the base 020-based Amiga 1200.

Outside of that, I personally don’t think chasing improved benchmark numbers will do much here, since very few Amiga games make use of anything above this level of performance — unlike a core like ao486 where e.g. Doom will run much better if you improve it.

(If you want a super fast Amiga — as opposed to one that prioritizes an FPGA recreation of the chipset — emulation is still your best bet, including CPU emulation using the on-board ARM CPU as has been proposed. That only makes sense if you consider MiSTer a competition to the Vampire type of boards, which is not what I was trying to talk about here — but I’m sure some people are interested in such a thing. :)

[kathleen]

Outside of that, I don’t think chasing improved benchmark numbers will do much here, since very few Amiga games make use of anything above this level of performance — unlike a core like ao486 where e.g. Doom will run much better if you improve it.

(If you want a super fast Amiga (as opposed to one that prioritizes an FPGA recreation of the chipset), emulation is still your best bet — including CPU emulation using the on-board ARM CPU as has been proposed. That only makes sense if you consider MiSTer a competition to the Vampire type of boards, which is not what I was trying to talk about here, but I’m sure some people are interested in such a thing.

I fully agree with you for the games. This is my point of view of course, but I'll never be interested to run games that were not intended to be used on the real hardware back in the day.
On the other hands, if one day (I know I'm probably dreaming) the Minimig AGA core can be equal to an A1200/4000 equipped with a 68060 processor clocked at 50 Mhz (to my recollection it was the faster processor for the Amiga classic before the end of it) I'll be more than happy, not for playing games, but for the applications like lightwave, scultp 4D etc,
I own a Vampire 500V2+ in my A2000, an Amiga 1200 and the Mist(er). The hardware that I use the most, if not the only one that I do use since a while is the Mister because it is for me the best compromise Power/flexibility/size/reliability compared to the other systems I own.
Currently, according to the above test results, the core is not so far to the 68040 at 25Mhz despite that there is no FPU, this is already amazing.

[toastboy]

@kathleen
I'm lucky enough to have an A4000 with 040 at 25mhz on test right now (rebuilding it for a friend, so he can sell it).
It's on the same desk as my mister setup, so direct comparisons are easy to make.

The minimig core (beta RTG+tg68k) is so close to the 4000 in terms of speed and compatibility that, apart from the "feel" of the original mouse and keyboard, there's not much really to separate the two.

More CPU power would be nice of course, we've got used to UAE's ridiculous speeds under emulation after all, but the mister core already compares well to the top-of-the-range official commodore machines. (68060 accelerators were never an official commodore product if I remember it right)

Like you, I prefer the mister just because it's so easy to use, has HDMI and doesn't take up half the desk. I don't really feel the need for the original hardware anymore.

[Solskogen]

Actually, the MiST numbers are a bit better

IMG_20201022_081927.jpg (1.07 MiB) Viewed 12821 times

[kathleen]

@solskogen,
Exact same results that on my St-Midi for Imath and FP. Regarding graphics waow, almost the double. Which core version did you use ?

[Solskogen]

Sorry, I should've posted that:
minimig_mist_rtg_201017.rbf from https://github.com/mist-devel/mist-bina ... inimig-aga

[kathleen]

Thank you so much. Will redo the tests on my side with this core just to double check that I've well the same results with the St-Mini.