Slow write speed on /media/fat (ok on /, read ok, with any SD), network speed slower, erratic WiFi

[pinkimo]

Hi,

I received my MiSter BlisSTer Edition from UltimateMister this week, and I've a weird issue. The write speed on /media/fat or any FAT/exFAT partition is very limited, no mater the writing method (cp, wget, FTP, SFTP...), and on top of that, the FTP/SFTP speed seems to have an hard limit of 1.5MB/s independently of the write speed.

I've tried with 3 SD cards, the 8GB provided by UM, a Sandisk 512GB with Mr Fusion image installed, containing all the roms/isos, with arround 180GB of free space, and a Sandisk 32GB, tried both fresh standard installer and Mr Fusion one with this card. Of course I get normal write speed with any of these cards on my PC.

I've tried with the DE10-nano PSU and with a 4A one, and everything else works well, including the PS1 core.
CPU load remains low for all tests.

If I copy a file from the PC to the MiSTer through FTP or SFTP, TLS or not, wired or WiFI, it's limited to 1.5MB/s, even if I copy in /root instead of /media/fat
If I copy a file from the MiSTer to the PC, I get a normal transfer speed
If I wget http://test-debit.free.fr/image.iso in /media/fat, I get from 1.5 to 2MB/s with the 8GB and the 512GB SD cards, almost 3MB/s with the 32GB one
If I wget http://test-debit.free.fr/image.iso in /root, I saturate my internet connection (5+MB/s)
If I copy a file from /root to /media/fat (or /media/fat to /media/fat), I get arround 5-7MB/s (10 on the 32GB card)
If I copy a file from /media/fat to /root, I get a normal write speed
I obtain the same results with the SD card on an USB adapter, with the same variation in speed between the 32GB and the 2 others.
I also tried with an old 2GB USB stick, got a write speed of 800KB/s on FAT, 5.5MB/s on exFAT with 1MB clusters and 4MB/s on exFAT with 4KB clusters.

This double limitation is weird, the fact it affects only the FAT/exFAT partition is weird, the speed variation depending on the method used to write the data and the SD card used is weird... It seems there is 2 issues, the write speed on FAT/exFAT partitions, and the transfer speed through FTP limited to 1.5MB/s independently of the write speed limitation.
I looks like the CPU or something is bottlenecking, but the CPU load remains low.
I'm a bit lost and didn't find any similar complain, any idea?
I hardly see how it could be anything other than a defective NE10-nano, at least for the write speed limitation, considering the issue is there with any SD card using the most basic setup which work for anyone else, but I don't want to jump to conclusions.

Additionally, I've also a WiFi issue, maybe related to the same bottleneck. The green DriveFree dongle I use, supposed to work well with the MiSTer, works perfectly on my PC, but the connection is very erratic on the MiSter (dongle placed at the same position in the room than when testing on PC), it works normally during 10-20 secs, then not a single byte goes through during 10-20 secs, the connection is still alive (wget or fillezilla just freezes, waiting for bytes to go through again), then it comes back to normal for 10-20 secs, and so on...

Thanks

[Marcellus]

Similar to my speeds. From PC to wired Mister, media/fat, 3-3.5 MB/s - generic sandisk 64gb

[pinkimo]

It's 2 times faster, but still very slow, it's supposed to be way faster, as you can see there for example: https://misteraddons.com/blogs/news/mis ... d-shootout

[prenetic]

it works normally during 10-20 secs, then not a single byte goes through during 10-20 secs, the connection is still alive (wget or the FTP client just hang there, waiting for bytes to go through again), then it comes back to normal for 10-20 secs, and so on...

Yep... pretty much my experience with network transfers on the MiSTer as well -- I've given up on it and just use an SD card reader for large transfers. It's not ideal, but it's much faster to get content on the card.

[Solskogen]

If possible use a NAS. Ethernet is much faster.

[Malor]

I'm just using the stock DE-10 card, because almost all of my stuff is on the network share. Wgetting to the card gives me about 1.5MB/sec, and wgetting to /media/fat/cifs (remote SMB3 share) gives me about 5MB/sec, although it bounces around a great deal, perhaps dropping some packets with all the extra network writing.

My normal approach is to mostly put things in /media/fat/cifs, and to actually download them on the server, rather than from the Mister. Reading from the server seems very fast, so once things are copied there, they load up very quickly.

It's possible that a high speed card might work much faster, but I don't have a spare one to test with.

[thisisamigaspeaking]

I'm just using the stock DE-10 card, because almost all of my stuff is on the network share. Wgetting to the card gives me about 1.5MB/sec, and wgetting to /media/fat/cifs (remote SMB3 share) gives me about 5MB/sec, although it bounces around a great deal, perhaps dropping some packets with all the extra network writing.

My normal approach is to mostly put things in /media/fat/cifs, and to actually download them on the server, rather than from the Mister. Reading from the server seems very fast, so once things are copied there, they load up very quickly.

It's possible that a high speed card might work much faster, but I don't have a spare one to test with.

Similar to what I am getting with wget with a SanDisk 32GB High Endurance (2MB/sec) and NAS (6MB/sec).

It's 2 times faster, but still very slow, it's supposed to be way faster, as you can see there for example: https://misteraddons.com/blogs/news/mis ... d-shootout

On that link they are not using wget, they are using dd. I get similar numbers to them with dd.

[local SD]
/media/fat# sync; dd if=/dev/zero of=tempfile bs=1M count=1024; sync
1024+0 records in
1024+0 records out
1073741824 bytes (1.1 GB, 1.0 GiB) copied, 64.1841 s, 16.7 MB/s

[remote NAS]
/media/fat/games# sync; dd if=/dev/zero of=/media/fat/games/tempfile bs=1M count=16; sync
16+0 records in
16+0 records out
16777216 bytes (17 MB, 16 MiB) copied, 0.455868 s, 36.8 MB/s

[pinkimo]

On that link they are not using wget, they are using dd. I get similar numbers to them with dd.

[local SD]
/media/fat# sync; dd if=/dev/zero of=tempfile bs=1M count=1024; sync
1024+0 records in
1024+0 records out
1073741824 bytes (1.1 GB, 1.0 GiB) copied, 64.1841 s, 16.7 MB/s

[remote NAS]
/media/fat/games# sync; dd if=/dev/zero of=/media/fat/games/tempfile bs=1M count=16; sync
16+0 records in
16+0 records out
16777216 bytes (17 MB, 16 MiB) copied, 0.455868 s, 36.8 MB/s

I was answering to Marcellus who was talking about FTP transfer speed, pointing to results obtained through various WiFI dongles, with a better speed.

I obtain the same results as you with dd (at least with the 32GB SD card), but I don't get the point of this test if I get half that or less on a cp or rsync.

I'm less annoyed by the write speed than by the transfer speed limited to 1.5MB/s and the erratic WiFi which will make an update_all or the upload of an ISO tricky through WiFi.

It doesn't feel normal to have such a slow transfer speed of 1.5MB/s through FTP/SFTP. Marcellus has twice my speed, this reddit thread talks about 48Mb/s (and one of the replies talks about 5-7MB/s), which is 4 times my speed.

And I'm pretty sure it's not normal to have such an erratic WiFI connection with a perfectly working and compatible dongle.

[thisisamigaspeaking]

I'm less annoyed by the write speed than by the transfer speed limited to 1.5MB/s and the erratic WiFi which will make an update_all or the upload of an ISO tricky through WiFi.

It doesn't feel normal to have such a slow transfer speed of 1.5MB/s through FTP/SFTP. Marcellus has twice my speed, this reddit thread talks about 48Mb/s (and one of the replies talks about 5-7MB/s), which is 4 times my speed.

And I'm pretty sure it's not normal to have such an erratic WiFI connection with a perfectly working and compatible dongle.

As Marcellus, I get about 3MB/sec with sftp to /media/fat over ethernet, so yeah that is what I would expect.

I suggest trying an ethernet cable or a different wifi dongle. The Linux kernel and wifi driver(s) are not part of the MiSTer project at all, so anything related to that is up to other people...

I also suggest using an SMB share for most of the directories (e.g. /media/fat/games) as it works out much better in general.

[pinkimo]

Ok, interesting development.

First, I gave a try to samba, I should have done that before, and indead it works a lot better, less than 2 mins for a 1GB file, 25 secs for a 220MB one.

Then, I found that the WiFi connection is erratic when the MiSTer boots up with the WiFI dongle plugged in, plugging it once the MiSTer is booted up (or unplugging/replugging it) fixes the erratic behaviour, with a rock solid connection afterward.

So I guess I'll be happy with that, thanks for your help.