MiSTer FPGA Forum

wark91 wrote: ↑Wed Jun 22, 2022 5:38 pm have you think to use the second SD Card present on the I/O board of the MiSTer until a better solution is found ?

Yes, but we don't have an IDE 8-bit + SD solution either, it is an interesting option also in the sense that the core could be ported to other FPGAs.

do you have ask to sorgelig ?

perhaps he can help you ?

breiztiger wrote: ↑Wed Jun 22, 2022 6:34 pm do you have ask to sorgelig ?

perhaps he can help you ?

I don't think he has much interest in this core, because of ao486.

I know there are circuit diagrams for the disk system in the PCJr reference I linked to on the first page of this thread. I don't know if that can help us at all though.

Why can't the MCL86 core replace the x86 core on the ao486 project to take advantage of the existing motherboard and disk emulation elements?

Seems that there should be some clean division point where the CPU cores could be swapped, perhaps at the BIU level...

kitune-san wrote: ↑Mon Jun 20, 2022 1:05 pm ...Actually, I am not familiar with PC/AT/XT and 8/16-bit old CPUs.
I was born in 1994 and the first PC I used had windows 2000 installed.
The 640k barrier had already broken.
So I need a lot of information to know what is going on.

Ooh, I can help a little. Memory on the PC is surprisingly complex. Here's a copy of a post I made on Ars Technica awhile back, lightly edited to improve readability a little.

************************
Here's a medium-length explainer for DOS-era PC memory.

DOS was designed around the 8086 and 8088, which had 20 lines of address space, or 1 megabyte, which seemed like a ridiculous plenitude when nearly all consumer processors addressed 64K. IBM, in its wisdom, divided that into two chunks, conventional, for DOS and user programs, and upper, for hardware expansion. They drew the arbitrary line at 640K... ten times the amount of RAM in an 8-bit machine, and wholly devoted to just being program RAM. Most of the I/O was in the upper 384K, although there was also some in the first 1K.

Thus, conventional memory: one contiguous block starting from 0. I think maybe DOS used a little bit down at the bottom, after the 1K or so of I/O space. Normally, this topped out at 640K, but could be somewhat larger on some systems, depending on where the video card memory was mapped.

As it turned out, 640K was not enough for anyone, and a lot of that top 384K was not in use in most systems. So hardware boards were invented to put RAM in unused areas up there.

This became upper memory: RAM in the high 384K of the first megabyte.

Eventually, DOS started to manage upper memory itself, and was even able to load device drivers and such up there. When managed by DOS, these became:

UMBs -- Upper Memory Blocks: Chunks of upper memory managed by DOS.

Another bright idea to increase RAM was to use a 64K area of the upper memory as a window on a larger memory space. Hardware boards also came out to do this, and could often provide UMB areas as well. One common configuration was 2MB of RAM, addressable 64K at a time. This resulted in:

Expanded memory, or EMS: a 64K window on a larger address space. I think of this as 'vertical' memory... it's at a 90 degree angle to main memory, and slides up and down, 64K at a time. More advanced versions of EMS would use 4 16K blocks instead of 1 64K block. All the drivers for this kind of memory were different, each board having its own, and eventually the market settled on the Lotus-Intel-Microsoft standard, or LIM EMS. Any board with a LIM driver could be plugged in and would work with any program that used the LIM standard.

In parallel with all this software work, of course, Intel was working on new chips. Their first big advance was the 80286, which could address up to 16 megabytes of RAM, finally catching up with the Motorola 68000. This new RAM, above 1 megabyte, became known as:

Extended memory: RAM above 1 megabyte. In later PC generations, this would become almost all the RAM in the machine.

This RAM was absolutely dreadful in most senses, because the 286 couldn't access it in 8086 mode. It had to be switched into extended mode, and once it was there, it couldn't switch back. Intel thought that everyone would just rewrite their programs to work with the new standard. Intel was grievously wrong.

IBM figured out a way to make extended memory somewhat usable: resetting the CPU brought it back up in 8086 mode. So if a program wanted to read from extended memory, it could switch to 80286 mode, but would then have to reset the processor to get back to DOS. It would write a return address to a specific spot in RAM, and reboot the machine. The BIOS would boot, and would check that special address first thing. If it wasn't zero, it would immediately jump there, resuming execution. This was horribly slow, and it meant that extended memory mostly couldn't be used. It got used some for caching the hard drive, because resetting the CPU tens of times per second, as it copied in data from extended memory, was still faster than reading from a hard drive, but otherwise it just sat idle in DOS mode.

A driver was written to handle this really painful memory, and resulted in:

XMS: extended memory that's managed by HIMEM.SYS. Very slow RAM for 16-bit programs, mostly used only as drive cache.

There was an oversight in the design of the 80286. The 8088 uses segment:offset addressing, and segments can start on any 16-byte boundary. If you set a segment near the end of the 1MB limit, it will wrap around to the memory at 0. Some programs used this weird addressing mode so that they could both work with ROM routines and lower memory at the same time, without needing to switch segment pointers.

The 80286 didn't do this right. Instead, if you set a segment:offset outside 1MB space, it would read from the first 64 of extended memory. Now, this, this was useful. DOS got access to all but 16 bytes of a new 64K.... it was the only extended memory it could see. This special RAM became called:

The High Memory Area, or HMA. The DOS=HIGH command was added, so that DOS put most of its tracking structures up there, freeing lower memory. This might be the best hardware bug ever. All later x86 processors preserved this behavior.

This broke some programs, however, so IBM came up with the idea of using a bit on the keyboard controller to turn the wraparound on or off. This is the A20 line, and DOS generally has to tweak it on to read its own structures, and then back off to run regular programs, some of which assume the old 8086 wraparound still works.

Then the 80386 came out, which was amazing in comparison. It was a flat, 32-bit processor, could fully support 8086 mode, and could freely switch into and out of its even-more-extended mode. (which I think was still called extended mode, it was just better on a 386.)

Someone at Quarterdeck figured out that the MMU on the 386, combined with its Virtual 8086 mode, could be used to simulate expanded memory. Suddenly, you could have all the expanded memory you wanted (remember, this is the 'vertical' memory that uses a 64K upper memory window.) without spending any money on a hardware board, or suffering with the incredibly slow (by then) ISA bus. You could use your main, fast, 32-bit RAM as EMS for DOS. You just had to buy QEMM. Lots and lots of people did. Quarterdeck got very large.

Microsoft, in their quest to extinguish all large software companies but themselves, decided to steal that idea, and integrated it into DOS as EMM386.EXE. Both QEMM and EMM386 were, very roughly, super-simple hypervisors, sitting between DOS and the actual memory and preserving an illusion of a hardware configuration that did not actually exist. Quarterdeck expanded this further and let you multitask DOS applications with their Desqview program; in a sense, it was a hypervisor that ran multiple sessions of DOS. (this isn't quite accurate, but it's broadly true if you don't examine it too closely.)

But all these memory types were on life support once the 386 came out, because in 32-bit mode, all the memory was just memory. DOS extenders started to come out, which were libraries that programmers used to launch 32-bit programs under DOS; most of them offered virtualized DOS services, where they would handle calls for DOS services themselves, directly in 32-bit mode, while making sure to update the correct DOS structures as well. That way, you could run your program in 32-bit mode most of the time, but when you exited, DOS would still understand the state of the system. You could run 32-bit programs, quit them, and everything would keep working. DOS would not crash, which was a great feature, allowing people to seamlessly merge 32-bit applications into their existing software library. DOS4GW was probably the most common extender, and you will see many, many games and applications with a DOS4GW launch message.

As multitasking became a thing, memory in this mode needed more management than XMS offered, and the first attempt at this was:

VCPI: Extended memory managed by a 32-bit DOS runtime. This crashed and burned.

Shortly thereafter, Microsoft came out with their own standard:

DPMI: Windows-compatible 32-bit memory management for DOS.

VPCI didn't last long, and only DPMI is left. Later versions of HIMEM.SYS and EMM386.EXE integrate with DPMI to work seamlessly, allowing even sophisticated 32-bit DOS programs to run comfortably under Windows as well as DOS.

So, recapping:

Conventional: first 640K.
Upper memory: upper 384K.
UMBs: Upper memory blocks, under management by DOS if you tell it DOS=UMB.
EMS: Expanded memory, the vertical kind, the 64K window on a larger address space. Can be either hardware or a driver. 16-bit programs can use this freely.
Extended memory: all RAM above 1MB. Except for the HMA, invisible to 16-bit programs.
High Memory Area(HMA): the first 64K after 1 megabyte. DOS loads itself here if you tell it DOS=HIGH. (DOS=HIGH,UMB means to do both.)
XMS: Extended memory that's being managed by a driver: not very usable for 16-bit programs, but the dominant form of RAM for 32-bit programs.
DPMI: Memory provided by Windows for DOS programs. XMS and EMS will pull from this pool. Used in DOS windows when Windows is running.

MicroCoreLabs wrote: ↑Wed Jun 22, 2022 7:27 pm Why can't the MCL86 core replace the x86 core on the ao486 project to take advantage of the existing motherboard and disk emulation elements?

Seems that there should be some clean division point where the CPU cores could be swapped, perhaps at the BIU level...

I don't know.
Maybe no one has considered it and just not done it.

We have ported the ao486 uart module to the pcxt side. So I think MCL86 can use the resources of ao486 if we can convert the bus well.

Deleted

Beta 1.0

* The UART port is changed to the internal MiSTer port, now it is possible to use the core without using a USB cable

https://github.com/spark2k06/PCXT_MiSTer

Beta 1.0 opens a new beta phase in which any user can participate and give feedback. Just copy the script pcxt_uart_hdd.sh to the scripts folder and serdrive to the core working folder, i.e. ./games/PCXT, these files are located in the SW folder of this project. Also, place in this same folder (./games/PCXT) a bootable image with the corresponding OS and under the name hdd.img.

That's all that is needed, just launch the script and boot the core.

In the SW folder there is also a file called boot.rom which contains Sergey Kiselev's open source 8088 BIOS, along with the XTIDE UniversalBIOS ready to boot from the serial port. However, Sergey Kiselev's BIOS has some problems with the keyboard and is a bit slow... while this issue is being solved, you can run the python script make_boot_with_jukost.py from inside the SW folder, which will generate a boot.rom file with the Juko ST BIOS in place, much more stable and with good performance.

MicroCoreLabs wrote: ↑Wed Jun 22, 2022 7:27 pm Why can't the MCL86 core replace the x86 core on the ao486 project to take advantage of the existing motherboard and disk emulation elements?

Seems that there should be some clean division point where the CPU cores could be swapped, perhaps at the BIU level...

Perhaps with a lot of effort it could be added to the ao486 project, but there are several reasons why I personally am not interested.

The KFPC-XT project of @kitune-san seems to me incredible for this project to have a reliable PCXT. ao486, apart from being huge, is too dependent on the Linux part, which makes it difficult to port it to other FPGAs, just what I want to avoid with this core.

Now with beta 1.0, I have made it a bit Linux-dependent, but I hope that at some point we will have IDE and floppy so we can avoid it.

Thank you I will try to build this hdd.img.
Any recommendation to build it ? I presume Fat 16 or something like that and DOS installed on it.

wark91 wrote: ↑Thu Jun 23, 2022 11:33 am Thank you I will try to build this hdd.img.
Any recommendation to build it ? I presume Fat 16 or something like that and DOS installed on it.

I think it would be better to use FAT16, an interesting guide to image creation from freedos for example, would be this one:

http://www.freedos.org/books/get-starte ... stall.html

You should be able to build a disk image by just creating a raw file, and then install DOS on it using AO486. If you stay within reason of what PCXT should support, it should carry over just fine.

Bas wrote: ↑Thu Jun 23, 2022 12:26 pm You should be able to build a disk image by just creating a raw file, and then install DOS on it using AO486. If you stay within reason of what PCXT should support, it should carry over just fine.

Another good idea.

spark2k06 wrote: ↑Tue Jun 21, 2022 4:23 pm

kitune-san wrote: ↑Tue Jun 21, 2022 2:58 pm The 101 error in the original XT BIOS was caused by not using DMA for memory refresh.
It's experimental. If the following change is made to return 0x01 on the status read, the 101 error will no longer occur.
However, it then stops with error 601.

In the future, I would like to perform SDRAM refresh by DMA events.

In KF8237.sv, line 241
2022-06-21 235000.png

It no longer gives error 101, although it does not pass the tests:

DMA_Failure.jpg

I modified KF8237(fix bugs) and wired it to the timer so that it would pass the test.
The DMA does not yet have SDRAM refresh control, so the processing speed will be slightly longer than the original.

https://github.com/kitune-san/PCXT_MiST ... am-refresh

I took an 20MB MSDOS 3.3 img from MAME and added the beta of TDL (with approx 20MB of games specially for this kind of PC) i had laying in my archives and i have been able to play Alley Cat perfectly.

I start to have something thank your for the tips.
I tried MSDOS 5.0, in dosshell the textes are blinking (the ones not selected) not visible on the screenshot below but I can do a video if needed.

Also with the command to mount the HDD, I have this message but it still works so maybe not so important
SerDrive - XTIDE Universal BIOS Serial Drive Server
Copyright (C) 2012-2013 by XTIDE Universal BIOS Team
Released under GNU GPL v2, with ABSOLUTELY NO WARRANTY
v2.0.0 Beta 3 (Jun 23 2022)

../games/PCXT/hdd-256mb.vhd: Hard Disk with 524160 LBA sectors, size 255.94 MB (CHS geometry 520:16:63)
Opening /dev/ttyS1 (115.2K baud)
Bad Command Checksum: a2 a2 a2 01 24 02 00 c3, Checksum=cea2

Thanks

Grand Prix Circuit works great ! Thanks a lot

@spark2k06
Fixed a problem with 8088 BIOS that would no longer accept keyboard input.
It also passes the diagnostic test.
Pull request sent to you.

However, in my environment, the keyboard sometimes becomes unstable immediately after power-on. (That is, when the FPGA is cooling.)
To solve this or other problems, I think we need to write timing constraints.

To implement IDE and FDD, if you don't need a low level compatibility (i think BIOS level should be enough for games, right?) then you can go much simpler way. You don't need IDE and FDD HW handling in BIOS. Just define your own ports close match to sd_* signals of hps_io module and directly read/write sectors from mounted images.

kitune-san wrote: ↑Thu Jun 23, 2022 4:20 pm @spark2k06
Fixed a problem with 8088 BIOS that would no longer accept keyboard input.
It also passes the diagnostic test.
Pull request sent to you.

However, in my environment, the keyboard sometimes becomes unstable immediately after power-on. (That is, when the FPGA is cooling.)
To solve this or other problems, I think we need to write timing constraints.

Pull request accepted. Thanks!

kitune-san wrote: ↑Thu Jun 23, 2022 1:16 pm
I modified KF8237(fix bugs) and wired it to the timer so that it would pass the test.
The DMA does not yet have SDRAM refresh control, so the processing speed will be slightly longer than the original.

https://github.com/kitune-san/PCXT_MiST ... am-refresh

It still shows error 101 in the BIOS of the IBM 5160, but there are no failures in the tests, and the operation seems fine so you can make a pull request to me about it and I will accept the changes in the main branch. Thanks!

AFAIK, there were very few programs that talked directly to hard drive controllers, because there were a bunch of different implementations of the early ST-506 "standard". I think pretty much everything stuck purely to BIOS calls except probably for Spinrite. IDE just reimplemented the same BIOS calls, so that it was transparently compatible with almost all programs. You could use pretty much any software with any controller, and everything worked because nobody looked too closely at what was actually going on.

You'd start to see trouble with 32-bit software, because a lot of that started avoiding the slow 16-bit BIOS code, and started driving the hardware directly. But since you're making an XT, that's probably not going to be an issue. The AO486 IDE implementation is pretty lousy, for instance, but it works beautifully with almost all 16-bit software. It blows up when you try to install a better OS, but I don't think many people will be trying anything other than DOS on an 8088.

There were, however, other operating systems available (like CP/M), so getting the IDE controller exactly right wouldn't be a complete waste of time. The work might also be portable back to the AO486, which isn't working well with Windows.

I have the strong impression that IDE is pretty simple, but that's from a software interface point of view. I have no idea how hard an FPGA implementation would be.

edit: I believe PCEM (now 86Box) does a good software IDE implementation, so you might be able to use that code as documentation.

Sorgelig wrote: ↑Thu Jun 23, 2022 4:31 pm To implement IDE and FDD, if you don't need a low level compatibility (i think BIOS level should be enough for games, right?) then you can go much simpler way. You don't need IDE and FDD HW handling in BIOS. Just define your own ports close match to sd_* signals of hps_io module and directly read/write sectors from mounted images.

Thanks, I don't have a lot of experience in developing moderately elaborate FPGA modules, but we'll take it into account.

Just to say that in this core we are using original PCXT BIOS, like Sergey Kiselev's 8088, Juko ST or the original IBM 5160... no custom BIOS for the core. I'm not sure what implications this has. For the IDE part, we are using XTIDE Universal BIOS, open source.

As pc xt games are floppy protected,

Would be great to have lowlevel disks formats support like transcopy, ipf, kryoflux raw, …
https://www.vogons.org/viewtopic.php?t=41787

And xt harddrive controler is http://minuszerodegrees.net/ibm_xebec/ibm_xebec.htm

Thanks for the good work, with your beta1.0 it looks like it is a loot easier to test. I am in the process of setting it up and I have a few questions.
The main question is in regard to serdrive. Does it start automatically or does it have to be started manually. At the moment I created the bios as per your instructions, I did not make the hdd.img yet but started the core just to see. I let sit for a while but at some point the core started to give 1 beep, the screen is black with a blinking cursor in the bottom right corner of the screen. Once in a while the beep comes back. I wonder if it the core waiting for the upload via serial or if it is some other problem.

jca wrote: ↑Thu Jun 23, 2022 5:39 pm Thanks for the good work, with your beta1.0 it looks like it is a loot easier to test. I am in the process of setting it up and I have a few questions.
The main question is in regard to serdrive. Does it start automatically or does it have to be started manually. At the moment I created the bios as per your instructions, I did not make the hdd.img yet but started the core just to see. I let sit for a while but at some point the core started to give 1 beep, the screen is black with a blinking cursor in the bottom right corner of the screen. Once in a while the beep comes back. I wonder if it the core waiting for the upload via serial or if it is some other problem.

That's because you don't have the image mounted, with the script pcxt_uart_hdd.sh located in the scripts folder... every time you start MiSTer, that is, every time you return to the main menu either through a hard reset or a MiSTer startup, the image must be mounted before launching the core. Go to the scripts section and launch the script mentioned .... now you can launch the core.

breiztiger wrote: ↑Thu Jun 23, 2022 5:34 pm As pc xt games are floppy protected,

Would be great to have lowlevel disks formats support like transcopy, ipf, kryoflux raw, …
https://www.vogons.org/viewtopic.php?t=41787

And xt harddrive controler is http://minuszerodegrees.net/ibm_xebec/ibm_xebec.htm

As for the HD controller, I would settle for having a proper implementation of an 8-bit IDE interface on the 300h I/O port, so that I can use it with the XTIDE Universal BIOS in its default configuration.

remax wrote: ↑Thu Jun 23, 2022 1:52 pm I took an 20MB MSDOS 3.3 img from MAME and added the beta of TDL (with approx 20MB of games specially for this kind of PC) i had laying in my archives and i have been able to play Alley Cat perfectly.

Do you mean "Total Dos Launcher" or "Total Dos Collection"?

remax wrote: ↑Thu Jun 23, 2022 1:52 pm I took an 20MB MSDOS 3.3 img from MAME and added the beta of TDL (with approx 20MB of games specially for this kind of PC) i had laying in my archives and i have been able to play Alley Cat perfectly.

Did you try "digger" by any chance?

@spark2k06, I found one game doesn't seem to have the right color in color mode CGA.
It is Fire and Forget You will find below the YT version with the color quite different
https://www.youtube.com/watch?v=5jaDxA8Wlgs

Thanks !