Odd Issue with my MiSTer and ISP?
-
- Posts: 10
- Joined: Sun Jun 12, 2022 12:19 am
Odd Issue with my MiSTer and ISP?
This is the mister downloader script
This is the download all script
Alright, so a few things to note
1. everything else works perfect. samba, pinging to external addresses. I get an IP and everything looks good on my router
2. does not change with ethernet, wifi, ethernet straight into the wall (my isp is fiber with the ethernet port running to their switch stack)
3. the download script works when i use wifi connected my phone's hotspot, connected to cellular data. this does not work when my phone hotspot is connected to my router, leading me to believe it's an isp issue.
4. another one time workaround i first used... at first i could not use the download script to get anything. but after changing my mac from the default 01:02:03:04:05:06:07 using this https://pastebin.com/raw/8L6kg99Q, it worked! until... i rebooted. then it did not work again connect to my isp internet, even after changing the mac again. thinking this was a fluke, i reinstalled and started from scratch. and the same behavior exhibited itself again.
I'm at a bit of a loss! I'm guessing it's something with my ISP and not the mister, but I'm not sure exactly what I should say to my ISP. any extra info I could give them?
-
- Posts: 10
- Joined: Sun Jun 12, 2022 12:19 am
Re: Odd Issue with my MiSTer and ISP?
nah I don't think so. I've double checked my dhcp leases, no conflict. and samba and external pings still work!
alright! this does work! I put a vpn on my phone and connected my mister to the phone with a wifi hotspot. I made sure to turn off mobile data, only going through my router. I don't know much of anything about external networking... does anyone know why it might work on a VPN?
-
- Posts: 10
- Joined: Sun Jun 12, 2022 12:19 am
Re: Odd Issue with my MiSTer and ISP?
seems to be! I looked at it in vim via ssh and... that seems? like a cert file lol. so what I did was delete it and grab a new one using
Code: Select all
cd /etc/ssl/certs
wget --no-check-certificate https://curl.haxx.se/ca/cacert.pem
-
- Posts: 10
- Joined: Sun Jun 12, 2022 12:19 am
Re: Odd Issue with my MiSTer and ISP?
If I can ping external addresses from the mister like google.com, and github.com, I assume DNS is working properly?
Just now I tried disabling something called SPI firewall on my router, didn't seem to change anything. And I think when I plug my mister straight into the wall and get an external IP... there isn't a firewall on that is there? and would a VPN somehow bypass it? (I'm pretty shaky on network stuff, once it leaves my router, I really don't know what happens lol)
-
- Posts: 10
- Joined: Sun Jun 12, 2022 12:19 am
Re: Odd Issue with my MiSTer and ISP?
-
- Top Contributor
- Posts: 622
- Joined: Fri Jan 22, 2021 4:36 pm
- Has thanked: 80 times
- Been thanked: 324 times
Re: Odd Issue with my MiSTer and ISP?
1. Hooking up your MiSTer to the internet, fir which it's thoroughly unsuited with its default SSH root login and all.
2. Messing with your ISP's upstream router. That device won't appreciate you changing MAC addresses around all the time.
Get a NAT router in between there, that'll help a lot.
But just to check, what external IP do you get? Just the first octet would be helpful, leave the rest out for now for security.
-
- Posts: 10
- Joined: Sun Jun 12, 2022 12:19 am
Re: Odd Issue with my MiSTer and ISP?
Normally my external facing IP first octet is 72 (just googling 'my ip'). When I plug my mister into the wall, the first octet is 208
-
- Top Contributor
- Posts: 622
- Joined: Fri Jan 22, 2021 4:36 pm
- Has thanked: 80 times
- Been thanked: 324 times
Re: Odd Issue with my MiSTer and ISP?
More generally speaking, a fast enough NAT router for the entire network would probably be best. You'd completely insulate your local LAN from whatever is going on upstream. Unless you're seriously geeking out in the networking department, that would be my advice. Things would then work like they do in 99.9% of homes and MiSTer would just naturally fit in there.
Now this *may* not be possible due to the way the network is configured and you might lose things like VOIP or IPTV if you use those services. Get in touch with your ISP if you do.
I'd be happy to help out with this further (I used to do datacenter networking for a university), but there's quite a few unknowns here still.
-
- Posts: 10
- Joined: Sun Jun 12, 2022 12:19 am
Re: Odd Issue with my MiSTer and ISP?
Let me know if I'm asking a silly question or misunderstanding something, I really appreciate the help!
-
- Top Contributor
- Posts: 622
- Joined: Fri Jan 22, 2021 4:36 pm
- Has thanked: 80 times
- Been thanked: 324 times
Re: Odd Issue with my MiSTer and ISP?
Plugging the MiSTer into the wall directly will only work if you set its MAC address to be the same as your router's public interface. In that case the MiSTer device will simply impersonate the router to the upstream switch and you'll be put into the same upstream VLAN directly. As long as you're not using PPPoE or Radius/802.1x authentication before you get a connection, chances are quite high that the ISP determines your device from its MAC address and puts it in a VLAN based on that. But let's not go there. It's unsafe to give a MiSTer a publically routable IP with no firewalling in front of it. Stick with your router.
From behind your NAT router, given that you have a working 192.168.xxx.xxx address on the MiSTer, you should be able to at least connect locally. You confirmed that as connectivity to your CIFS shares apparently does work and remote connectivity is also there. So what remains for me as a logical culprit would be DNS. Can the MiSTer, once connected behind the NAT router, resolve a hostname that it doesn't know about yet? You can test that with ping. It may fail the actual ping because ICMP tends to get blocked a lot on the internet, but at least it should give you an IP address for the other side that it's sending packets to. If it gets to that stage, DNS works.
Looking into the script itself.. What I'm seeing the downloader script do, is pull the cacert.pem file from a hard-coded location: https://curl.se/ca/cacert.pem. That URL is behind a CDN from Fastly so the target IP changes all the time but it should be nothing special, just a plain HTTPS-request. I can even download that file from work, with their strict firewall/security crud going on. Do make sure that the file you pull from there is unmangled by things like Windows line endings, character encoding fudgery etc. if you download it on another computer than the MiSTer itself. The easiest way to prevent this is to cd into /etc/ssl/certs on your MiSter itself and then wget the file directly.
The script uses that file to validate the upstream URL's it connects to and the errors you're seeing are the script complaining that A) the cacert.pem file is somehow corrupt and B) it's unable to correct the situation.
What it tries to do at step B, is simply delete the file. Since you're running as root, permissions can't be the issue here. The script, however, is not smart enough to deal with missing directories. Are you 100% positive that /etc/ssl/certs is an existing directory on your MiSTer and that the cacert.pem file is in there? Couldn't hurt to limit permissions to 755 on the file itself and change the owner to root since some tools tend to check on that (even if redundant on MiSter).
I don't have my MiSTer handy right now (at work.. lunch break) but I'll see if I can reproduce the issue on mine tonight and what's needed to fix it.
-
- Posts: 10
- Joined: Sun Jun 12, 2022 12:19 am
Re: Odd Issue with my MiSTer and ISP?
so on the dns front, everything seems to be working as far as I can tell. I pinged all these addresses and the pings returned fine
8.8.8.8
google.com
github.com
curl.se
But! I think I'm making some progress on the culprit.
So I decided to just delete the cacert file and run the download script (not downloader). And I get this:
Now I can say no and run the downloader without the cert and it works. I'm not knowledgeable enough to know why, but I'm knowledgeable enough to know I should avoid that as much as I can.
If I say yes, it downloads and installs cacert and... then the message pops up again. and if i say yes, it downloads and... pops up again. repeat
so I type ls again in /etc/ssl/certs to see whats up and oh wow. way more files than there was before! before cacert.pem was the only file in that directory.
(it goes down for a while, couldn't capture it all in a single screenshot)
I decide to delete all the files in the directory and run the separate download all script. And it ran fine, without creating any new cert files. This is outside my expertise, but that seems concerning? That the update all script either a. doesn't check to see if the certs are there or b. doesn't use certs at all? I also ran the download all script after letting the normal download script create certs, and it ran the same (seemingly perfectly).
With the certs that the download script created, I tried using it on a VPN. Same behavior! Thought the certs it just installed were corrupt and immediately wanted to try again.
So I decided delete all the files again and just run wget --no-check-certificate https://curl.haxx.se/ca/cacert.pem and same behavior as before, internet connection failed.
So... I delete the file again, run JUST wget https://curl.haxx.se/ca/cacert.pem. It says the cert isn't activated, and the download script treats it as if it wasn't there and tries to create its own.
So it's something with these certs. I'm just at a loss why:
- getting a cert via wget --no-check-certificate https://curl.haxx.se/ca/cacert.pem works on cellular and a vpn, but not my own network
- why the download script installs many many files instead of just one cacert.pem, and why it still thinks its corrupt after install
- why the download all script runs seemingly perfectly after deleting the cert.
Let me know if you have any questions about what I wrote! I was typing it up while troubleshooting and may not have made everything clear. again, thank you very much for helping me out with this
- aberu
- Core Developer
- Posts: 1192
- Joined: Tue Jun 09, 2020 8:34 pm
- Location: Longmont, CO
- Has thanked: 247 times
- Been thanked: 411 times
- Contact:
Re: Odd Issue with my MiSTer and ISP?
If you have more than one then you need to flush the interfaces with "ip addr flush up".
-
- Posts: 10
- Joined: Sun Jun 12, 2022 12:19 am
Re: Odd Issue with my MiSTer and ISP?
I think that's normal? the 0.150 address is the permanent lease I set in my router's DHCP server.
-
- Top Contributor
- Posts: 860
- Joined: Wed Feb 09, 2022 11:50 pm
- Has thanked: 64 times
- Been thanked: 194 times
Re: Odd Issue with my MiSTer and ISP?
Basically, after rebooting, wait until the time appears in the title bar, and see if the behavior changes.
-
- Top Contributor
- Posts: 622
- Joined: Fri Jan 22, 2021 4:36 pm
- Has thanked: 80 times
- Been thanked: 324 times
Re: Odd Issue with my MiSTer and ISP?
Malor's point is also a great one: the system date must be correct(-ish) for certificates to work. MiSTer doesn't have a battery-backed clock so it'll always be January 1970 right after you turn it on. Give it some time to do an NTP sync and skip back to the present day before you attempt an update.
- aberu
- Core Developer
- Posts: 1192
- Joined: Tue Jun 09, 2020 8:34 pm
- Location: Longmont, CO
- Has thanked: 247 times
- Been thanked: 411 times
- Contact:
Re: Odd Issue with my MiSTer and ISP?
Re: Odd Issue with my MiSTer and ISP?
Hey milkdrinker, did you ever find why this was happening? I am going through the same problem and deleting the file also seems to have helped. I did not have this problem when I had my MiSTer connected over ethernet, but I do now over Wi-Fi.
My money is on some sort of MTU issue, hence why the different results with different connections, but I am yet to run more testing and first I have to wait for the Downloader to finish
-
- Posts: 10
- Joined: Sun Jun 12, 2022 12:19 am
Re: Odd Issue with my MiSTer and ISP?
so unfortunately... i never found the cause.
what i ended up doing is starting over from scratch on a new sd card. its been a year so im fuzzy on the details... but the download all script worked great and i just used that for everything. i tested it this morning and the regular download script still doesn't work. let me know if you want me to test anything for you!