Debugging/simulation?

[ExCyber]

I've been mostly out of the loop on FPGA workflows since college, and I'm wondering what approaches people are taking to debugging MiSTer cores and their components. Simply rebuilding the core and loading it is fine for minor tweaks, but for anything serious I'd obviously like some combination of more visibility and faster iteration. I also have some interest in using a general-purpose programming language rather than writing traditional testbench code in HDL. Things I've looked at include:

• Verilator: Great speed and flexibility, but no ability to use VHDL code (yet? someone was working on a VHDL front end at some point). It was a surprisingly smooth experience to get an arcade game half-running with this (FX68K with some minor mods to make Verilator happy, address decoding in SystemVerilog, memory and graphics on the C++ side).
• ModelSim: Having trouble getting this to install on a modern Linux distro, also not sure if the free versions support Verilog+VHDL simulation or designs the size of a MiSTer core.
• Hierarchical/block-based design: only allowed in the Pro Edition of Quartus Prime, which doesn't even support Cyclone V devices (!?)
• Incremental rebuild: only allowed in Standard and Pro Editions ($$$), and I'm not sure how much faster it is in practice for MiSTer-sized designs.
• Developing in one of the various attempts at a shiny new HDL and leveraging its simulation facilities, or translating to Verilog and simulating with Verilator. I'm mostly looking at nMigen, but I'm open to trying Chisel again, and Hardcaml also looks interesting. The nmigen-boards repo even has a definition for MiSTer already, although it only describes the hardware and not the framework. This doesn't really tackle the Verilog+VHDL issue for using existing components, and doesn't integrate as cleanly with Quartus (nMigen is actually quite slick... if your top-level is nMigen).

Any thoughts on this?

[dshadoff]

I used www.edaplayground.com for developing the MB128 functionality for the TurboGrafx core, but all of the other updates I've made so far have been done blind.

Simulation seems like a great answer to specifically-designed scenarios - especially simple ones - but in a core that runs through millions of instructions before an issue arises, simulation isn't going to help - you will need SignalTap.

Another method which could help is flashing LEDS or displaying information on screen. Once you have video output running, it is a good idea to add a line displaying state of various items, provided that they aren't higher-frequency than your eyes.

Another technique I used when I needed to adjust a video timing chain, adjusting number of cycles between item A and item B (video processors generally aren't well-documented, so tests are necessary), was to make the delay amount configurable through the OSD (temporarily of course). This dramatically reduced the number of compile cycles needed to find the right value across multiple games.

[sajattack]

https://github.com/alanswx/Tutorials_Mi ... /verilator

[ExCyber]

Simulation seems like a great answer to specifically-designed scenarios - especially simple ones - but in a core that runs through millions of instructions before an issue arises, simulation isn't going to help - you will need SignalTap.

I hadn't looked at SignalTap. I should probably dig into that a bit, so thanks for bringing it up. I recall reading about the Xilinx equivalent (ChipScope), but at the time I wasn't able to try it because it was only in the paid versions of ISE.

So far, though, "millions of instructions" scenarios have been pretty tolerable with Verilator, even on my old PC and using not-especially-optimized simulation code (granted, not doing much more than the single 68000 CPU in Verilog-land). On my new PC, I actually wouldn't be surprised if I could simulate a simpler component at full speed with some additional optimization effort.

Another method which could help is flashing LEDS or displaying information on screen. Once you have video output running, it is a good idea to add a line displaying state of various items, provided that they aren't higher-frequency than your eyes.

I think this on its own wouldn't support the level of information I expect to need when bringing up a new device, but an overlay toggle could solve a good portion of that problem.

Another technique I used when I needed to adjust a timing chain for number of cycles between item A and item B (video processors generally aren't documents, so tests are necesaary), was to make the delay amount configurable through the OSD (temporarily of course). This dramatically reduced the number of compile cycles needed to find the right value across multiple games.

I definitely should have thought of this when debugging the front/back porch timing on my "test pattern" core.

https://github.com/alanswx/Tutorials_Mi ... /verilator

Nice. This didn't quite work as-is with the version of Verilator that I currently have, but it seems to cover some things that I wanted to look into. Like hooking up a "real" video output instead of filling my drive with VRAM dumps.

[sajattack]

Nice. This didn't quite work as-is with the version of Verilator that I currently have, but it seems to cover some things that I wanted to look into. Like hooking up a "real" video output instead of filling my drive with VRAM dumps.

Uncomment line 15.

[nullobject]

Coming from the software world, I feel that proper testing is really missing from hardware projects.

Yes you can create a test bench or use signal tap to help debug one-off issues, but IMHO what we really need is proper integration testing — i.e. tests that are run every time someone pushes a code change to GitHub. It's the only way to have a high level of confidence in what you are doing, as it's just not feasible to manually test every code path when you are making changes to a FPGA core.

I'm not sure that this kind of thing is commonplace when it comes to hardware, but there a few promising signs.

I spent some time looking into Chisel over the past few nights, and to me it looks super interesting. I'm currently implementing a Z80 CPU to see what the workflow is like: https://github.com/nullobject/z80-chisel

So far so good, but what's most important is that every code change I make automatically gets built and tested on Travis CI. This makes me as a software person very happy

[hno]

Integration testing is the same for hardware as software. It is really no.different. You write tests that tests modules isolated or in combination to verify that they fulfill specifications as coded by the testuite. The code for the hardware is just code in very well defined modules with a bunch of inputs and outputs. And making a continuous integration test environment is also exactly the same, fits pe fectly fine in any of the existing ci frameworks.

And the problem why it is not done is the same. Writing, verifying and maintaining the tests.

There is only one slight difference to "normal" software testing and that is that testing and compilation is done using different tools. Testing is done with a simulator and compilation is done with a compiler and fitter. Making automated full system tests of actual compiled code is very impractical for hardware.

[FPGAzumSpass]

I developed the GBA core completly in simulation.
Only game testing was done on hardware. If a bug occured, it was fixed in simulation.

I have a bunch of automatic regression tests, cycle based CPU state dumping and live(~1 frame per minute) graphical output.
Bugs deep into the games where fixed with savestates that can be taken on hardware and loaded into simulation.

Tools used:
- VHDL
- Lua scripts talking to either simulation or hardware. (same testscript runs on simulator and core in FPGA)

The Framework is checked into the project, but i'm not sure if it still compiles after the latest folder restructure.

[nullobject]

The Framework is checked into the project, but i'm not sure if it still compiles after the latest folder restructure.

Sounds very interesting, I'll check it out.

[ExCyber]

I've been learning to use SignalTap (thanks again for mentioning it, dshadoff), and noticed that the "Rapid Recompile" button is never available. Is this essentially the same feature as "Incremental Compilation", i.e. requiring a Standard Edition license, or is something else going on?

And just how "rapid" is Rapid Recompile, anyway? Anyone have some examples of the difference? I doubt it's enough to make me spend $3000, but I'm curious.