Simulating CRT scanlines for Guncon?

[Caldor]

I read up on CRT scanlines a while ago and got the idea that, shouldn't it be possible to simulate these scanlines on an LCD?

Then I forgot about it again, until looking at the new Checkmate monitors today. Guncon controllers from what I understand know where it is aiming due to that scanline that is sometimes when filming a CRT display. So... with everything else the MiSTer is already simulating when it comes to CRT monitors, might it not actually be possible to simulate these scanlines and have this work with Guncon controllers?

[o07eleven]

The scan lines you see when filming isn't how the beam is moving, that's just a de-sync in camera frame rate vs the sync rate of the beam. Your beam is moving the horizontal length of your monitor and back, 240 times, 60 times a second.

The light gun is not looking for a slow moving scan line. It's looking for where the actual beam is during the scan to determine where the gun is aimed. The precise timing of this is needed and can't have any latency. Your LCD also has a display and hold vs a single scan line like a CRT. Fundamentally they are two different technologies on how to display.

https://www.youtube.com/watch?v=3BJU2dr ... SlowMoGuys

This is a good video to show you the difference in display between a CRT and LCD/OLED type TV.

[Hectic]

No without lag

[Sigismond0]

There's already support for GunCon2 on CRTs, and GunCon3 support is in active development for LCDs. There's a test Linux build in the controllers thread on Discord, seems to work well. Looks like support was added in main recently, just need Linux to get updated and it should be official.

[Caldor]

The scan lines you see when filming isn't how the beam is moving, that's just a de-sync in camera frame rate vs the sync rate of the beam. Your beam is moving the horizontal length of your monitor and back, 240 times, 60 times a second.

The light gun is not looking for a slow moving scan line. It's looking for where the actual beam is during the scan to determine where the gun is aimed. The precise timing of this is needed and can't have any latency. Your LCD also has a display and hold vs a single scan line like a CRT. Fundamentally they are two different technologies on how to display.

https://www.youtube.com/watch?v=3BJU2dr ... SlowMoGuys

This is a good video to show you the difference in display between a CRT and LCD/OLED type TV.

Ahh... I thought Guncons worked the same as Lightguns. Well, it is the Lightgun method I was thinking of, which does use the scanline. Simulating a 60fps scanline should be possible I would think given what other filters we have available. But would not be as accurate as the Guncon it seems. I guess that might explain why it needs to connect to the TV and the console, instead of just the console like the Lightgun does.

[Mr. Encyclopedia]

The lightgun method used by guncons works by sampling far, far faster than 60fps. It uses a fast photodiode to detect the CRT beam as it scans across the screen, and references that beam location with the video output of the console to determine what pixel the gun was aimed at. Note that this isn't the same as the NES zapper method, which blanked the screen with hitboxes for one frame so the much slower and cheaper Zapper photodiode could detect a hit or not.

To use this method on a LCD screen you'd need one with a refresh rate fast enough to simulate an old CRT scanline, I.E. 15 kHz, not 60 Hz. This is why modern light gun systems like gun4ir, Sinden, and even the Wiimote use gun-side tracking instead.

[Caldor]

The lightgun method used by guncons works by sampling far, far faster than 60fps. It uses a fast photodiode to detect the CRT beam as it scans across the screen, and references that beam location with the video output of the console to determine what pixel the gun was aimed at. Note that this isn't the same as the NES zapper method, which blanked the screen with hitboxes for one frame so the much slower and cheaper Zapper photodiode could detect a hit or not.

To use this method on a LCD screen you'd need one with a refresh rate fast enough to simulate an old CRT scanline, I.E. 15 kHz, not 60 Hz. This is why modern light gun systems like gun4ir, Sinden, and even the Wiimote use gun-side tracking instead.

Ah right, it is the NES solution I was thinking of. I had forgotten it was also relying on those boxes being made. It actually determines the pixel? That is a lot more exact than I thought it was.

Would would happen if you reduced it to 60hz then? Instead of doing the full draw that would require 15khz, it would be based on where the scanline would be for the 60 frames rendered, dividing up the scanline in whatever positions it would be estimated to be at. Something I would suspect could be "prerendered" working a bit the same as other shaders, but the effect it adds, is the scanline for each frame.

Of course, the full render would be more precise and accurate, but I am thinking this ought to work as well.

[o07eleven]

I'm pretty sure you're still not grasping how a CRT displays an image on screen vs an LCD. A CRT is a scanning electron beam and a flat panel is doing a "sample and hold" display where it keeps the pixel lit until it's told a new value. You keep confusing the scan beam with a scan line.

A light gun is literally looking for a dot on the screen that effectively scans 15,000 "dots" a second. There's nothing that a "shader" can do to emulate this, unless you have a 15000Hz display and you light up one pixel at a time to emulate a CRT beam. Even then, the timings need to be near perfect, any delay to the display would have you missing every shot.

The way you make gun games work on flat panels is with other tech (IR receivers, high speed cameras in the gun to determine aiming). There are methods to modify an original light gun to work on an LCD, but you basically need to put some sort of tracker that I mentioned before, so it kind of defeats the purpose when there are other options available.

[SuperBabyHix]

Just to add a bit of a visual example, this is a video comparing a CRT to an LCD in slow motion. It gives a really good sense of how fast a CRT scans out.

https://www.youtube.com/watch?v=3BJU2drrtCM

[Caldor]

I'm pretty sure you're still not grasping how a CRT displays an image on screen vs an LCD. A CRT is a scanning electron beam and a flat panel is doing a "sample and hold" display where it keeps the pixel lit until it's told a new value. You keep confusing the scan beam with a scan line.

A light gun is literally looking for a dot on the screen that effectively scans 15,000 "dots" a second. There's nothing that a "shader" can do to emulate this, unless you have a 15000Hz display and you light up one pixel at a time to emulate a CRT beam. Even then, the timings need to be near perfect, any delay to the display would have you missing every shot.

The way you make gun games work on flat panels is with other tech (IR receivers, high speed cameras in the gun to determine aiming). There are methods to modify an original light gun to work on an LCD, but you basically need to put some sort of tracker that I mentioned before, so it kind of defeats the purpose when there are other options available.

I am not sure you understand what I am suggesting. I know the CRT beam is much to fast to be emulated. What I am saying is, it should not have to be fully emulated. It just needs to recreate what the Lightgun is looking for, because the Lightgun does actually capture 15khz does it? It looks at a screen in a single moment and processes this. I am pretty sure it does not capture several images and processes them to come out with a result, its analogue and cannot handle that. It looks for some bright spots.

So my point is, it should not at all be need to emulate the full complexity here. A good approximation should be enough. It needs to be calculated pretty well of course, but as long as its done for each frame, with some math and such in place, I would argue it should be quite accurate. Having it doing it at 15khz was... nice probably. But hardly required for the aiming to be that good. So instead of this lighting up every single individual pixel, it would just be lighting up... probably a series of pixels of the pixels that would be rendered at any given frame, making the Guncon think its seeing a CRT beam.

[FoxbatStargazer]

Isn't the screen lighting up baked into the game software itself rather than at the system level? If so you're talking about romhacking each game at a minimum.

[Caldor]

Isn't the screen lighting up baked into the game software itself rather than at the system level? If so you're talking about romhacking each game at a minimum.

Unless I am mistaken, it is using what the CRT is doing anyway rather than something the game is doing. The game is not updating at even 60hz after all, let alone 15khz. The game might also be doing something, but if that is the case, then that should not make a difference I think, given the refresh rate of the game is at most 60hz.

[o07eleven]

I'm pretty sure you're still not grasping how a CRT displays an image on screen vs an LCD. A CRT is a scanning electron beam and a flat panel is doing a "sample and hold" display where it keeps the pixel lit until it's told a new value. You keep confusing the scan beam with a scan line.

A light gun is literally looking for a dot on the screen that effectively scans 15,000 "dots" a second. There's nothing that a "shader" can do to emulate this, unless you have a 15000Hz display and you light up one pixel at a time to emulate a CRT beam. Even then, the timings need to be near perfect, any delay to the display would have you missing every shot.

The way you make gun games work on flat panels is with other tech (IR receivers, high speed cameras in the gun to determine aiming). There are methods to modify an original light gun to work on an LCD, but you basically need to put some sort of tracker that I mentioned before, so it kind of defeats the purpose when there are other options available.

I am not sure you understand what I am suggesting. I know the CRT beam is much to fast to be emulated. What I am saying is, it should not have to be fully emulated. It just needs to recreate what the Lightgun is looking for, because the Lightgun does actually capture 15khz does it? It looks at a screen in a single moment and processes this. I am pretty sure it does not capture several images and processes them to come out with a result, its analogue and cannot handle that. It looks for some bright spots.

So my point is, it should not at all be need to emulate the full complexity here. A good approximation should be enough. It needs to be calculated pretty well of course, but as long as its done for each frame, with some math and such in place, I would argue it should be quite accurate. Having it doing it at 15khz was... nice probably. But hardly required for the aiming to be that good. So instead of this lighting up every single individual pixel, it would just be lighting up... probably a series of pixels of the pixels that would be rendered at any given frame, making the Guncon think its seeing a CRT beam.

Ok. So to explain the whole step process of how a light gun works to you.

The game itself is rendering a frame and sending it to the CRT @ 15kHz. This is a very precise timing.

When you pull the trigger of the gun, it is looking for the beam location on the screen.

The light gun is basically asking the game "what location is the beam scanning at" to which the game responds somewhere in the 1-15,000 hz cycle.

Using that location information, comparing to what it has scanned through the light gun sensor, the game determines what you are pointing at and then determines whether or not you have hit the target.

The system was a bit more low tech for Master System/NES games due to their lack of processing speed. The game drew a white box on a black screen for a single frame. It was effectively checking for a white signal on trigger press. With any sort of latency, the light gun will never see that frame, and you will always miss your shot. It's actually relatively easy to spoof if you just point your gun at a light bulb and pull the trigger, it will register a hit every time. There isn't any sort of "shader" that would make this work on an LCD.

Even more now on the more complex where they are taking the scan beam location relative to what the gun sees in the sensor, any latency as well makes the gun think that it's pointing at a completely different location. They need VERY tight tolerances in order to work with each other to sync with what the game is sending vs what is being displayed.

LCD flat panels don't draw an image like a CRT, so the light gun will never know it's true position relative to the game's expected draw position of the CRT scan beam. Thus, this isn't something for a simply shader to draw that will solve the issue. You fundamentally need to change how a light gun works, which is where you get IR and high speed camera sensor based systems that are used today.

[aberu]

I am not sure you understand what I am suggesting. I know the CRT beam is much to fast to be emulated. What I am saying is, it should not have to be fully emulated. It just needs to recreate what the Lightgun is looking for, because the Lightgun does actually capture 15khz does it? It looks at a screen in a single moment and processes this. I am pretty sure it does not capture several images and processes them to come out with a result, its analogue and cannot handle that. It looks for some bright spots.

So my point is, it should not at all be need to emulate the full complexity here. A good approximation should be enough. It needs to be calculated pretty well of course, but as long as its done for each frame, with some math and such in place, I would argue it should be quite accurate. Having it doing it at 15khz was... nice probably. But hardly required for the aiming to be that good. So instead of this lighting up every single individual pixel, it would just be lighting up... probably a series of pixels of the pixels that would be rendered at any given frame, making the Guncon think its seeing a CRT beam.

Duck Hunt is a good example to demonstrate the problem of what you are suggesting. Duck Hunt looks for what didn't flash on the screen to determine if you were pointing at it. The game requires an answer to the question of "did the gun see a flash or not" quicker than any modern display is capable of doing. This isn't a problem that a modern display can currently solve.

[Caldor]

I'm pretty sure you're still not grasping how a CRT displays an image on screen vs an LCD. A CRT is a scanning electron beam and a flat panel is doing a "sample and hold" display where it keeps the pixel lit until it's told a new value. You keep confusing the scan beam with a scan line.

A light gun is literally looking for a dot on the screen that effectively scans 15,000 "dots" a second. There's nothing that a "shader" can do to emulate this, unless you have a 15000Hz display and you light up one pixel at a time to emulate a CRT beam. Even then, the timings need to be near perfect, any delay to the display would have you missing every shot.

The way you make gun games work on flat panels is with other tech (IR receivers, high speed cameras in the gun to determine aiming). There are methods to modify an original light gun to work on an LCD, but you basically need to put some sort of tracker that I mentioned before, so it kind of defeats the purpose when there are other options available.

I am not sure you understand what I am suggesting. I know the CRT beam is much to fast to be emulated. What I am saying is, it should not have to be fully emulated. It just needs to recreate what the Lightgun is looking for, because the Lightgun does actually capture 15khz does it? It looks at a screen in a single moment and processes this. I am pretty sure it does not capture several images and processes them to come out with a result, its analogue and cannot handle that. It looks for some bright spots.

So my point is, it should not at all be need to emulate the full complexity here. A good approximation should be enough. It needs to be calculated pretty well of course, but as long as its done for each frame, with some math and such in place, I would argue it should be quite accurate. Having it doing it at 15khz was... nice probably. But hardly required for the aiming to be that good. So instead of this lighting up every single individual pixel, it would just be lighting up... probably a series of pixels of the pixels that would be rendered at any given frame, making the Guncon think its seeing a CRT beam.

Ok. So to explain the whole step process of how a light gun works to you.

The game itself is rendering a frame and sending it to the CRT @ 15kHz. This is a very precise timing.

When you pull the trigger of the gun, it is looking for the beam location on the screen.

The light gun is basically asking the game "what location is the beam scanning at" to which the game responds somewhere in the 1-15,000 hz cycle.

Using that location information, comparing to what it has scanned through the light gun sensor, the game determines what you are pointing at and then determines whether or not you have hit the target.

The system was a bit more low tech for Master System/NES games due to their lack of processing speed. The game drew a white box on a black screen for a single frame. It was effectively checking for a white signal on trigger press. With any sort of latency, the light gun will never see that frame, and you will always miss your shot. It's actually relatively easy to spoof if you just point your gun at a light bulb and pull the trigger, it will register a hit every time. There isn't any sort of "shader" that would make this work on an LCD.

Even more now on the more complex where they are taking the scan beam location relative to what the gun sees in the sensor, any latency as well makes the gun think that it's pointing at a completely different location. They need VERY tight tolerances in order to work with each other to sync with what the game is sending vs what is being displayed.

LCD flat panels don't draw an image like a CRT, so the light gun will never know it's true position relative to the game's expected draw position of the CRT scan beam. Thus, this isn't something for a simply shader to draw that will solve the issue. You fundamentally need to change how a light gun works, which is where you get IR and high speed camera sensor based systems that are used today.

Right, so the game needs to be told what khz has been selected for each given frame. Does not stop it from being possible. Split the 15kz into 60 positions. A shader will probably not be enough it would have to be part of the core and the core than brighthens certain pixels while letting the game be able to know which pixel is bright.

The speed of the display should not be a problem for this, its still just down to doing the math. The processing in the MiSTer is what needs to be fast, the display is not having to react to something, the game and lightgun just reacts to something that happened.

Same with Duck Hunt, not seeing the argument about how this should be a problem for this. The game "asks the display" but its the core that will be responding to what was rendered rather than the display. What will be critical is getting the timing right, to ensure the aiming is spot on. Taking possible latency into account. That latency ought to be fixed, so that should not be a problem I think.

So this should work, unless an LCD is not able to create the type of light that the CRT beam does and the Lightgun is relying on. But pretty sure that ought to be possible to simulate. The speed of the display is not that relevant here I would argue, its not that it really used the 15khz.

Unless you are saying that triggering the lightgun, it will be waiting for the detection of the CRT beam to be whereever the gun is pointing... Cannot render each individual pixel within 60 fps of course. I guess it could be made into blocks of pixels instead if that is the case, but then it would probably be something you could see happen with the naked eye and it would probably not be pretty.

[Mr. Encyclopedia]

Unless you are saying that triggering the lightgun, it will be waiting for the detection of the CRT beam to be whereever the gun is pointing... Cannot render each individual pixel within 60 fps of course. I guess it could be made into blocks of pixels instead if that is the case, but then it would probably be something you could see happen with the naked eye and it would probably not be pretty.

Yeah it kinda sounds like what you're saying is to split the screen up into 60 sections and display them one at a time so the gun can detect which one is being displayed when you pull the trigger, but cycling through 60 sections at 60fps would mean you're only seeing a whole frame once per second. The fact old light gun detection was happening at the 15khz speed of the electron beam is pretty important because that's much faster than the human eye can detect, but well within the ability of a photodiode.

Think of it this way: If there was a way to have the light gun detect aim on an LCD the way light guns could detect aim on CRTs don't you think we would have figured out how to do that by now? Why would we be bothering with Gun4IR using infared tracking or Sinden using computationally expensive image recognition? Heck, even the Nintendo Wii used infared tracking because even by the mid 2000s many people had TVs that wouldn't work with that style of light gun recognition, not to mention the amount of delay that wireless wiimotes introduced.

[dmckean]

Sample and hold displays cannot emulate this behavior. They are not fast enough, by an order of magnitude. Plus there's an issue with display latency on top of that.

[o07eleven]

I give up. You're not getting anything at all and not understanding the limitations of LCD sample and hold displays. It won't work, your suggestion won't work. If it was that easy to just put a shader, light guns on LCDs would have been solved by people a lot smarter than us who made the consoles in the first place.

[aberu]

The speed of the display is not that relevant here I would argue, its not that it really used the 15khz.

Read this.

https://gaming.stackexchange.com/a/30191

[Caldor]

The speed of the display is not that relevant here I would argue, its not that it really used the 15khz.

Read this.

https://gaming.stackexchange.com/a/30191

Interesting. So it can work with Duck Hunt it seems... to some degree. Probably better if optimized.

I had not considered the latency of a change in contrast could be a problem. This was 10 years ago, displays today should have much better latency, so the tests he did would probably do better today.

But the other responses does seem to argue that the light gun is actually not just using the CRT beam to calculate where its hitting, but might need it to be where the gun is pointing to register the hit? Seems... unlikely. 15khz and getting the exact point of this beam pixel by pixel seems like something that would be a problem even for a PS1 at the time. But then, I do suspect it does not really need pixel accuracy. Within 5 pixels would probably be fine.

Unless you are saying that triggering the lightgun, it will be waiting for the detection of the CRT beam to be whereever the gun is pointing... Cannot render each individual pixel within 60 fps of course. I guess it could be made into blocks of pixels instead if that is the case, but then it would probably be something you could see happen with the naked eye and it would probably not be pretty.

Yeah it kinda sounds like what you're saying is to split the screen up into 60 sections and display them one at a time so the gun can detect which one is being displayed when you pull the trigger, but cycling through 60 sections at 60fps would mean you're only seeing a whole frame once per second. The fact old light gun detection was happening at the 15khz speed of the electron beam is pretty important because that's much faster than the human eye can detect, but well within the ability of a photodiode.

Think of it this way: If there was a way to have the light gun detect aim on an LCD the way light guns could detect aim on CRTs don't you think we would have figured out how to do that by now? Why would we be bothering with Gun4IR using infared tracking or Sinden using computationally expensive image recognition? Heck, even the Nintendo Wii used infared tracking because even by the mid 2000s many people had TVs that wouldn't work with that style of light gun recognition, not to mention the amount of delay that wireless wiimotes introduced.

Sure, so I want to understand why it is this is supposed to be impossible.

Have you seen how long it was argued it was impossible to read and write to floppy drives connected to a PC using an adapter and emulation? Quite often people overcomplicate processes that can be simplified, because the parts of the process that seems very much required, are not really required in full.

So the argument that others would have figured this out, is not that strong a point. Very valid, but all I have seen so far is people saying that this CRT update beam that runs at 15khz is used to figure out where the Lightbeamis aiming. But I have not seen anyone explain what it does with the information regarding the position of this Lightbeam. Does it wait for it to be where you are aiming the Lightgun when pulling the trigger or is it able to tell where its aiming with the beam being at any given point on the screen? I am pretty sure its not actually reacting at 1 15000th of a second. Not that it has to. But what is it that happens from the point of pulling the trigger to the game registering a hit or miss?

I see some posts claiming that with the Sega it had to do with IR. But most lightguns does not seem to rely on IR from what I read. I mentioned it earlier that it would of course be a problem if the process relies on a spectrum that CRT uses but LCDs does not. But the post Aberu linked does have an engineer going into how its possible to make Duck Hunt work on an LCD, as I expected it should be. Although... not exactly optimally.

But the way Duck Hunt works, it sounds like it should work on any display, if the display is fast enough at doing the fast swap needed between brightness and contrast for those black boxes.

What you say about splitting it up into 60 fps, I was not arguing the whole screen would then have to light up though. I would think it would be enough to make a line of some 20 pixels maybe light up. This line would be jumping quite a bit, and the reason to make it a line would be to simulate the several pixels that might be lit up in this interval. Even then it would probably not be all the pixels that 15khz would be going over during 1 60 fps frame, so the line would skip over a lot. Again, if the whole thing relies on actual pixel accuracy, then this would limit where you are able to hit on the screen and lighting up that many pixels would be a pretty major problem for the accuracy. I guess if integrated even more into the game, you would only have to bother lighting up stuff that could be hit... hmmm... but I guess on the PSX it does show a hit wherever it is you shoot? So that is probably not a solution either.

[H6rdc0re]

Hopefully we'll get Sinden light gun support somewhere in the future. A great solution for Flat panel displays.

[Sigismond0]

Guncon 3 support is available for testing, and works great. Don't see why we need to reinvent the wheel here.

The only reason Duck Hunt doesn't work on a flat panel is latency. That's it. And I can easily prove this, because hacked roms of Duck Hunt exist that DO work on LCDs, all it does is increase the timing window for how long the white box stays on screen. I've played Duck Hunt on an HDMI modded NES on my plasma with my nephews and it works great. "Great" meaning "completely functionally" though it's very obvious that the white box is on screen for like 4 frames instead of just 1.

http://neslcdmod.com/

Any light gun tech that just uses this simple white box method could theoretically work fine on any LCD--it's just a matter of slowing the timings down to account for your flat panel's display lag. Obviously this is doable with just romhacks as shown on the NES here, but that might be something that could be expanded and built out in the framework. Of course, more modern light guns don't just use white flashes. The SNES Super Scope uses IR emitters for tracking and doesn't rely on the display. Guncon 2 actually taps into the video line to read sync signals for tracking. Guncon 3 and Wii remote go back to using IR. Sinden light gun uses a white frame at the edge of the display, but otherwise works similarly to IR controllers, using that frame to determine X/Y positioning.

Getting the Guncon 2 to work on an LCD with MiSTer would be considerably more difficult to make work than the NES zapper or SNES super scope, and from all I've read I do thing it would be impossible due to how it reads the sync signal and rectifies that against what the sensor sees on the screen. From what I read, it samples the HSync signal direclty off the vido line and that does allow it to sample at the full 15KHz, and there's no way any frame trickery is going to give you that kind of accuracy on a flat panel. Even if you could find a way to block out frames/lines that the Guncon could see, they'd be huge sections of the screen that would make precision aiming kind of irrelevant since they'd have to be like 250x larger than what the light gun actually needs. This might be something that could be done in MiSTer, but more likely would require per-game hacks like the NES mods above. And it would require extra hardware--namely a way to get that composite video sync signal in addition to the HDMI output you're displaying.

So the question is, what do we gain out of all of that work for something that may still be impossible, if you can just use a Guncon 3 and it just works already?

[Sigismond0]

Guncon 2 (CRT) and Guncon 3 (LCD) support are in and working as of today's Main/Linux update. Tested a GC3 against my plasma, played a couple of levels of Yoshi's Safari and it worked great. Seems much more consistent than the Wii Remote was.

[Caldor]

Well, it was mainly a thought experiment. Its not that its something we especially need. Guncon 3 support is more practical, and support for Sinden would be even better. Especially since I bought two of them.