Re: Why not use a package manager?
Posted: Tue Apr 13, 2021 8:25 am
Please stop conflating the presence of a package manager with the idea that it should be used for maintenance of the core OS. That's an assumption that's at the very least not true for the bit of work I'm attempting at the moment. I'm also very well aware of the fact that the initial design premise of MiSTer's update mechanism was to keep its business logic running on the board instead of some external infrastructure. There is a lot to be said for that approach, especially when you have very little to deal with other than a bundle of independent RBF-files and a near-static partition image.
As it stands right now, my MiSTer system goofded itself up two times now since I built it in late 2020. Mind you: that's not counting the times I broke it myself due to my own stupidity. It just unhinged itself to the point of being unbootable while I was doing exactly what it said on the label. That's what triggered me to start looking at the Linux end of the project.
Corruption to IT systems happens only when moving parts interact in a way that's unforeseen and from which recovery is less than obvious. Recovery being less than obvious is also often the reason people place the label "corruption" on such a condition in the first place. Usually things aren't really broken beyond repair, but just in an unexpected state. I'm trying my hand at very little else than reducing the opportunity for unexpected state here.
In the Linux world we have a number of tried and true tools to manage moving parts: package managers. My sole intention with what I'm doing is to bring the moving parts of *my own* MiSTer board under management of such a solution. In this case dpkg/apt, which is an extremely reliable and proven pre-invented wheel for just this purpose.
Of course this requires me to move the updater business logic off the board and onto, for now, my existing Jenkins setup and an S3-bucket to host the packages eventually. I am 100% aware that this violates the initial design principle of MiSTer's updater.
Now as to why I disagree with the design principle. That's because I disagree with the entire premise that the MiSTer is like an embedded or IOT system. Rhester72 stated a few posts up that the DE10-nano is a piece of lab kit, and I agree. However, a DE10-nano turned into a MiSTer does not qualify as such anymore.
Read the forum here. End users all over the place are asking how to adjust the MiSTer's default installation to their liking, which is to be expected for a DIY-project like this. Custom cases, add-on boards, controller interfaces, displays, midi-devices.. the works.. it's all part of an awe-inspiring ecosystem whether anyone likes it or not. In that sense MiSTer is turning the DE10-nano into the Raspberry Pi of FPGA dev-boards. No matter how Intel ever intended the DE10-nano to be used, we're simply not in Kansas anymore. The same effectively goes for how Sorgelig intended the MiSTer project iself. Things are growing organically, and it never hurts to test previous assumptions and experiment with alternative approaches in a controlled way.
Now update_all.sh may not be part of the project, but every Joe has it on their MiSTer and uses it with reckless abandon. I personally feel that now would be a very good time to streamline the itch that update_all.sh scratches by unifying the approach to MiSTer updates. To me it would be very useful if the MiSTer's official cores (and supporting tools like midilink etc.) and those made by people like Jotego could be pulled from different apt repositories but we'd be using one single set of on-board business logic to do it.
Sure there's package and repo maintenance to be done, and that's the flipside of this coin. I'm currently in the process of finding out just how much effort and money this would really cost. For now, I simply don't know yet. I'll be dogfooding it for a while myself first and when I'm satisfied with its reliability, I'll publish the entire scaffolding under the same Free Software license terms as the rest of MiSTer so that won't be a roadblock. I'm not asking anything of the MiSTer maintainers and I won't be touching anyone's workflow. Nor will I be taking any capacity away from core development. Once I release my work everyone can evaluate it on their own terms, at least there'll be a factual foundation for an honest and open discussion on whether or not this whole idea would benefit MiSTer or not. And if it does not, the world will keep on turning and at the very least I will have sharpened a few of the more rusty tools in my shed.
Now if we could please stop the vitriol in this thread, that would be much appreciated.
As it stands right now, my MiSTer system goofded itself up two times now since I built it in late 2020. Mind you: that's not counting the times I broke it myself due to my own stupidity. It just unhinged itself to the point of being unbootable while I was doing exactly what it said on the label. That's what triggered me to start looking at the Linux end of the project.
Corruption to IT systems happens only when moving parts interact in a way that's unforeseen and from which recovery is less than obvious. Recovery being less than obvious is also often the reason people place the label "corruption" on such a condition in the first place. Usually things aren't really broken beyond repair, but just in an unexpected state. I'm trying my hand at very little else than reducing the opportunity for unexpected state here.
In the Linux world we have a number of tried and true tools to manage moving parts: package managers. My sole intention with what I'm doing is to bring the moving parts of *my own* MiSTer board under management of such a solution. In this case dpkg/apt, which is an extremely reliable and proven pre-invented wheel for just this purpose.
Of course this requires me to move the updater business logic off the board and onto, for now, my existing Jenkins setup and an S3-bucket to host the packages eventually. I am 100% aware that this violates the initial design principle of MiSTer's updater.
Now as to why I disagree with the design principle. That's because I disagree with the entire premise that the MiSTer is like an embedded or IOT system. Rhester72 stated a few posts up that the DE10-nano is a piece of lab kit, and I agree. However, a DE10-nano turned into a MiSTer does not qualify as such anymore.
Read the forum here. End users all over the place are asking how to adjust the MiSTer's default installation to their liking, which is to be expected for a DIY-project like this. Custom cases, add-on boards, controller interfaces, displays, midi-devices.. the works.. it's all part of an awe-inspiring ecosystem whether anyone likes it or not. In that sense MiSTer is turning the DE10-nano into the Raspberry Pi of FPGA dev-boards. No matter how Intel ever intended the DE10-nano to be used, we're simply not in Kansas anymore. The same effectively goes for how Sorgelig intended the MiSTer project iself. Things are growing organically, and it never hurts to test previous assumptions and experiment with alternative approaches in a controlled way.
Now update_all.sh may not be part of the project, but every Joe has it on their MiSTer and uses it with reckless abandon. I personally feel that now would be a very good time to streamline the itch that update_all.sh scratches by unifying the approach to MiSTer updates. To me it would be very useful if the MiSTer's official cores (and supporting tools like midilink etc.) and those made by people like Jotego could be pulled from different apt repositories but we'd be using one single set of on-board business logic to do it.
Sure there's package and repo maintenance to be done, and that's the flipside of this coin. I'm currently in the process of finding out just how much effort and money this would really cost. For now, I simply don't know yet. I'll be dogfooding it for a while myself first and when I'm satisfied with its reliability, I'll publish the entire scaffolding under the same Free Software license terms as the rest of MiSTer so that won't be a roadblock. I'm not asking anything of the MiSTer maintainers and I won't be touching anyone's workflow. Nor will I be taking any capacity away from core development. Once I release my work everyone can evaluate it on their own terms, at least there'll be a factual foundation for an honest and open discussion on whether or not this whole idea would benefit MiSTer or not. And if it does not, the world will keep on turning and at the very least I will have sharpened a few of the more rusty tools in my shed.
Now if we could please stop the vitriol in this thread, that would be much appreciated.