I usually start with a search engine, many problems aren’t new and have been solved before. Solved by people who probably spent many many hours to come up with the answer. No reason to spend 40 hours (or 200 hours) investigating a problem you could solve with one hour of research.
The practical answer is to learn. General problem solving skills will go a long way, and you can learn the ins and outs of the PC as you go. The better mental picture you have about how the big pieces of the system fit together the easier it will be theorize where things might break down and cause problems. A lot of what I’ve learned has come from reading source code. With everything being open sourced there’s so much information at your fingertips. I may be biased because my background is in software development, but having a working knowledge of programming languages is like having a super power when it comes to learning. The entire AO486 core is open source, the BIOS is open source, you can even read the MS-DOS source code:
https://github.com/Microsoft/MS-DOS
Once you’ve identified a problem you want fix you may need to experiment, run tests, collect data. Knowledge will help you chose good experiments (but I do a lot of blind guessing too).
You mentioned DOSBox, which can be a method to experiment and learn how a specific piece of software works. DOSBox has a nice debugger that let you step through CPU instructions, read memory, log to the console what the system peripherals are doing. Disassembling code, and stepping through instructions is one tool we can use to track down problems. ScummVM has a wiki that lists some other tools:
https://wiki.scummvm.org/index.php/HOWT ... ngineering
When your investigating a new problem, keep it simple. Ask the easy questions first: Is it plugged in, have I configured the software correctly, is it compatible with the system. Reverse engineering is not the first thing to try.
There are other emulators to learn and experiment with: DOSBox, PCem, Qemu, Bochs. These emulators have large communities with a lot of like minded and helpful people.
YouTube is a great learning resource. Find something that interests you about PC architecture, MiSTer, MS-DOS, old video games, retro hardware and just learn. You will start to build that foundation that will help you solve problems.
If it were me I would focus on learning C/C++. This will give you the knowledge to understand the code you may be trying to fix, it gives you the skills to create new tools you may need to investigate problems.
