DEVs - Shadow masks and how to make them better with sub pixel emulation

[redsteakraw]

Right now we have have the new shadow masks, however I think with a few tweaks we can do better. Right now the way it seems to work is it tint the pixels with whatever value the mask is however I think there is a better way forward. CRT masks are made up of sub pixels, which are directed by the mask to each sub pixel by the beam so if you zoom in on a bright green object you would see green pillars surrounded by black sub pixels for the Red and blue respectively. The only way to get this is if the mask works in reverse Where when you have red green and blue masks it takes the respective value of the pixel behind it and displays only that color. So a simple RGB mask such as:

3,1

4,2,1

should replicate the sub pixels of a PVM / Trinitron almost perfectly if the masks worked in that way and if you were doing a 3x 720 scale image.
Furthermore if you wanted to add scanlines to that you could just add a black bar on every third pixel. This would look like the following.

3,3

4,2,1
4,2,1
0,0,0

This technique would only seem to work at targeted resolution scales. If everything scales the right way the mask should end up targeting the given pixels 1-1 where each sub pixel from the mask matches up with the pixel from the signal. This would draw the image on the screen closer to how a real CRT would. Am I crazy, am I missing anything or do you have any thoughts on using masks to replicate sub pixels and what results could be gained from that. What are your thoughts on what tweaks could be done to make this implementation even better.

[FoxbatStargazer]

There's probably some technical reasons why we have the current extremely simple implementation, after all there are exactly 8 shades to choose from which is pretty limiting... but I also think most real masks are way, way too fine to replicate in that kind of detail even at 6x. A console pixel is often composed of many mask "pixels" in a half-decent CRT and you can only cram two of this into a 6x scale, and that is ignoring the darkness surrounding the shapes that show different layouts..

Really I think that is why the first "official" shaders focused on simple stripes. They give you some feel of colors landing differently on different lines without trying to replicate it exactly. Part of the point of a mask is that the original content pixels and the mask "pixels" are not perfectly aligned, and this approach at least adds a little of that texture to the image.

Edit: isn't another issue that these mask "sub-pixels" are usually glowing and bleeding into each other at usable brightness on CRTs, at least consumer ones? That is something some degree of tint is better at emulating....

[LamerDeluxe]

This is how my sub pixel masks work. I designed them on the sub pixels and then converted them to color values to create the mask pattern. EDIT: They do assume a horizontal R,G,B order of the sub pixels, which won't be the case on all displays.

EDIT2: Also your freedom in creating patterns is quite limited by the positions of the sub pixels and that all the colors of you pattern have to balance out to not get a tinted image.

[redsteakraw]

This is how my sub pixel masks work. I designed them on the sub pixels and then converted them to color values to create the mask pattern. EDIT: They do assume a horizontal R,G,B order of the sub pixels, which won't be the case on all displays.

EDIT2: Also your freedom in creating patterns is quite limited by the positions of the sub pixels and that all the colors of you pattern have to balance out to not get a tinted image.

Checked out your shadow masks and I must say they are some of the best I have seen. Looking close up to my screen and it doesn't seem to be drawing things quite like the CRT. But from a distance it looks pretty good. I Don't know if my TV is doing something a 4K LG running at 720p on game mode and every feature turned off and aspect ratio original. Working with the shadow masks I can see many of the limitations. Do you have any thoughts on how the mask framework could be improved?

EDIT1
I can see looking at your masks that I was using the RGB values and you were using CMY values

[LamerDeluxe]

This is how my sub pixel masks work. I designed them on the sub pixels and then converted them to color values to create the mask pattern. EDIT: They do assume a horizontal R,G,B order of the sub pixels, which won't be the case on all displays.

EDIT2: Also your freedom in creating patterns is quite limited by the positions of the sub pixels and that all the colors of you pattern have to balance out to not get a tinted image.

Checked out your shadow masks and I must say they are some of the best I have seen. Looking close up to my screen and it doesn't seem to be drawing things quite like the CRT. But from a distance it looks pretty good. I Don't know if my TV is doing something a 4K LG running at 720p on game mode and every feature turned off and aspect ratio original. Working with the shadow masks I can see many of the limitations. Do you have any thoughts on how the mask framework could be improved?

EDIT1
I can see looking at your masks that I was using the RGB values and you were using CMY values

Thank you, that is good to hear.

With the new experimental shadow mask core builds I can apply them at full range and then they work as I intended. This is a macro photo of my TFT monitor with my sub pixel slot mask pattern playing Sonic.

EDIT: And I design those at sub pixel level, which translates to all available colors in the mask pattern.
EDIT2: Missed the question: The addition of different levels and pattern strengths, which is being experimented with right now, is helpful to make less harsh looking patterns, as well as being able to apply them at full strength. There was the idea of having control of RGB levels per pixel, which I liked, but if I understand correctly most of the people who have created patterns already didn't feel like converting them to that format.

[FoxbatStargazer]

OK this is getting more interesting, and what I thought the thread title was about! Playing with sub-pixels on the output side (i.e., exploiting the layout of RGB LCDs.) This may be exactly the kind of technique that is needed given how limited the Mister resolution is.

[Lightwave]

Question: Do shadow masks require integer scaling for the effect to work properly?

For example, I'm using the 9.7" iPad screen that is popular here.

- Using videomode=13 will output integer scaled video @ 2048 x 1536, which I would expect to work perfectly with a shadow mask filter applied (despite not perfectly filling the screen)

- Using Atrac17's custom video profiles however, will output at something like 6x which is then scaled up by the display itself to fill the entire screen (which normally looks great even for scanlines due to the high resolution and pixel density). I'm guessing the scaling step will throw off the precise alignment of the shadow mask grid and generally degrade the intended effect to the point where it wouldn't be recommended in this case?

[FoxbatStargazer]

I don't know about shadow masks in general, but it would definitely ruin the subpixel effects that LamerDeluxe just posted; although, that seems to be still in experimental stage? and not what we have to work with in main at the moment. (Actually with iPads being rotatable, not even sure about their subpixel layout?)

Color stripe effects are probably fine, but I would be a bit more iffy on the geometric patterns.

[lamarax]

Really I think that is why the first "official" shaders focused on simple stripes. They give you some feel of colors landing differently on different lines without trying to replicate it exactly.

This. I wrote in another thread that the quest for "perfect" CRT emulation is futile. It can't be done, and doesn't need to; all we're seeking, with all our different experiences both in era and devices owned, is rememberance.

I think ghogan42's utilitarian work aims to achieve exactly that; I've tried almost every filter inside the bloated "Shadow_Masks" folder, and while most are impressive at reminding what someone would perceive @ 1 cm distance of a CRT, I still go back to the fuzzy impression "simple RGB Stripes" give.

I'm more antsy really for the feature to propagate to the other (especially arcade) cores we have

[drgruney]

I sure don't want "perfect" but I love how a well done mask makes things like Donkey Kong Country and Star Fox not look like a random pile of pixels

[akeley]

This. I wrote in another thread that the quest for "perfect" CRT emulation is futile. It can't be done, and doesn't need to; all we're seeking, with all our different experiences both in era and devices owned, is rememberance.

This might be true, on the technical side, for MiSTer, but definitely not as a generalisation. Of course, "perfect" or "1:1" is an impossible quantifier, but looking at how good certain shaders got in the span of the last few years, it's getting to the "nearly non-indistinguishable" point.

I'm saying this as a CRT fanatic/collector, who is certainly not looking for "remembrance", but as close to replication of the real thing as possible. And I must say, that this year was the first time, when running Redream with a Geom+Reshade tweak combo on a mere 1440p Acer Predator, I did say, "hell - this is good enough". And things will only get better in the next 5-10 years, especially when OLED screens and external devices such as OSSC & Retrotink get more advanced.

Again, I don't mean it for MiSTer, which has understandably limited capabilities in this regard, and is already capable of much better stuff than the original set of filters.

[FoxbatStargazer]

Well to be fair this actual subpixel work is kind of exciting; although again its not creating RGB subpixels exactly as they were on a real mask (which would always be RGB instead of alternating), still at least its getting down to the right kind of scale. (i.e., like a third the size of the current full-pixel options)

When we get to 8k we might start approaching 1:1 recreation of CRTs, but at 1440p max its a bit limiting.

[akeley]

Well to be fair this actual subpixel work is kind of exciting;[...]but at 1440p max its a bit limiting.

I thought I said as much myself My post was in reply to that particular quote from lamarax, the bit more meta/philosophical thing. That does not mean I don't appreciate the actual work being done as we speak. I don't need this option myself atm, but as soon as I start traveling again (&^#@ Covid) I probably will, and I think it's great that we now have shadowmask options too, and not just Ye Ol' "scanlines".

[LamerDeluxe]

Question: Do shadow masks require integer scaling for the effect to work properly?

No, shadow masks work on the resolution of your display, independent of the scaling you use. That is why the effect varies a bit with scaling, as the pattern repeats more or less times per scaled pixel. With some patterns, this can lead to interference artifacts, in which case using integer scaling would help.

I don't know about shadow masks in general, but it would definitely ruin the subpixel effects that LamerDeluxe just posted; although, that seems to be still in experimental stage? and not what we have to work with in main at the moment. (Actually with iPads being rotatable, not even sure about their subpixel layout?)

Color stripe effects are probably fine, but I would be a bit more iffy on the geometric patterns.

Yes, some test builds have been made (can be found on the MiSTer Discord) to find out what the best way is of extending the possibilities of the shadow masks. My sub pixel patterns work best when applied at full strength, which isn't possible with the official builds yet.

My sub pixel set also includes a stripe pattern (aperture grille).