Guide: Perfect Image Sharpness No Shimmering

[dfilskov]

(updated 2021-03-23)

How to get perfect image sharpness - square or even pixels - using custom_ratio and vscale_border in MiSTer.ini

---

Visit this spreadsheet to calculate the settings for your MiSTer.ini for some cores ... more cores will be added:

https://docs.google.com/spreadsheets/d/ ... edit#gid=0

MiSTer lacks an easy setting to make the video signal match your flatpanel's pixels perfectly for all cores - each core needs to have integer implemented seperately - some cores luckily have that but many still don't.

Here's a how to get a pixel perfect image anyway (unless a core changes its resolution while running - if a core uses more than one resolution there's no way to get the perfect pixel output yet)

Guide:

---

• Make sure the resolution set with video_mode= in MiSTer.ini matches your screen's native resolution - or perhaps the exact half (native is best, though) - e.g. 1920x1080

• Set video_info=10 in MiSTer.ini (shows the resolution of the core for 10 seconds when it boots)

• Run your favorite core (e.g. Ghost'n'goblins - jtgng) and look at the resolution (video info) in the top left corner when it boots. Write down the top line's resolution (e.g. 256x224).

• Calculate the largest size of an integer-scaled image possible on you monitor's native resolution (e.g. the JTGNG core's image size will be limited by the 1080 height of your display in this example and so the largest image will be 224 times 4 = 894 ... thus the width would be 256 times 4 ... or 5 ... that gives these resolutions: 1024x896 ... or 1289x896 for a larger width).

• Calculate the size of the smallest black area around the largest image of the core (e.g. 1080 minus 896 = 184) and divide it by two (92 in this example). If the result is e.g. 37.5 things get more complex. But let's assume it's an integer like 92.

• Add the appropriate custom ratios and border size to the end of your MiSTer.ini (or adjust the existing lines) - all core section must be at the end of the MiSTer.ini:

[JTGNG] ; 256 x 224
vscale_border=92
custom_aspect_ratio_1=1024:896
custom_aspect_ratio_2=1280:896
(you may need to add vscale_mode=0 too)

• Boot your favorite core, make sure it's set to Scandoubler fx: None and Scale filter - Internal (not Scale filter - Custom) in the OSD (F12)

• Change the ratio from Original to one of the custom ratios you just created and click Save Settings if you're satisfied with the look.

... that's it

You can test if it worked by changing Scale filter - Internal to Scale filter - Custom and choose e.g. Gaussian_Sharp_06 ... now click Scale filter - Custom several times to switch back and forth between Scale filter - Internal and Custom. If the graphics stay unchanged (e.g. characters don't shift or blur with each click) then it worked! - the scaling filter has no effect since every pixel in the video matches 1, 3x3, 4x4 or even 4x5 pixels on your flatpanel.
Another way to test is to use the LCD_NN_Type1_020 filter. It creates a grid of black thin pixel lines around each scaled-up pixel. Look for un-evenness (moiré) in the grid. If the grid looks perfectly even, it worked.

Note: SMS core already seems to have square pixels - for all cores with square pixels all you need to do to get perfect sharpness is to set Aspect ratio to Original and make sure it's set to Scale filter - Internal (not Custom) - and of course set the vscale_mode=1 to force integer height (and again: Make sure the video_mode= matches your flatpanel's native resolution) in MiSTer.ini

The downsides are ...

1) the output of all cores will not necessarily be as large as with a (somewhat blurry) non-pixel-perfect scaling ... but if - like me - you prefer the pixel perfect sharpness that's fine.

2) the graphics will probably be a bit squeezed since the original pixels were seldomly square like today's flatpanels.

[Sigismond0]

Am I missing something here, or does an aspect ratio of 256:192 for NES simplify down to 4:3? 64x4=256, 64x3=192. This seems like it should have the same result as the mister 4:3 ratio, which should be imperfect on a square pixel screen right?

That said, this is all good stuff! I'd been planning on putting some time into something along the lines of this.

[dfilskov]

256x192 is 4:3 yes - running that core in 4:3 (with integer-scaling) should give you perfectly sharp and square pixels ... but if - on the original hardware - pixels weren't square you'll get a bit squeezed image.

It should also possible to get sharp (but not square) pixels using the ratio of 64x5 : 64x3 (5:3) or - perhaps better - 5:4 ... making the graphics wider.

[Sigismond0]

Yeah 5:4 is the ratio I run on my Hi-Def NES, and it works great. At some point I'll go through and make custom ratios like that for a bunch of MiSTer cores.

[Yim]

I think whether these figures give perfect pixels depends on the resolution used. At 1080p, a core with an internal resolution of 240 will be scaled 4x to 960 vertical pixels. If you use the NES example of 256x240, setting the aspect ratio to 5:3 will give a horizontal resolution of 1600 (five times one third of the vertical resolution, so 960/3 = 320, multiply by 5 to get 1600), which is 6.25x the internal horizontal resolution. 5:4 gives a horizontal resolution of 1200, 4.6875 times the internal resolution. This is not integer scaling. At 720p 5:3 is 1200x720 and 5:4 is 900x720, neither of which gives clean horizontal scaling.

I think what you're actually talking about is a pixel aspect ratio of 5:3 or 5:4 or whatever, meaning that each pixel is a 5x3 rectangle or 5x4. Because MiSTer doesn't give direct control over vertical scaling when integer mode is on, it just uses the largest integer multiple that won't exceed the screen resolution, your options for pixel aspect ratio are limited to that multiple for the vertical component. So for a 240p internal resolution at 720, you're looking at 3x vertical scaling, so you can have a pixel aspect ratio of 3:3 (1:1) by setting the custom aspect ratio to 256:240 (simplifies to 16:15), or 4:3 by setting it to 4x256:3x240 (1024:720, simplifies to 64:45), or 5:3 with 5x256:3x240 (1280:720 = 16:9). At 1080p you can have 1:1 with 256:240, or 5:4 with 256x5:240x4 (1280:960 = 4:3).

So the method for getting consistently sized pixels is:
-Figure out the resolution the core is putting out (by taking a screenshot and looking at its resolution, though as dfilskov says this won't work for cores that change resolution).
-Divide the vertical resolution your mister is set to by the vertical resolution of the core (e.g. 1080/240 = 4.5)
-Ignore anything after the decimal point (4.5 becomes 4)
-This is the vertical component of your pixel aspect ratio (PAR)
-Figure out what you want the horizontal component of the PAR to be (5 for 5:4, 6 for 6:4, 4 for 4:4)
-Multiply the core's horizontal resolution by the horizontal component of the PAR, and the vertical resolution by the vertical component of PAR (e.g. 5x256 : 4x240 = 1280:960)
-Simplify if you want (1280:960 = 4:3)
-Use that as the custom aspect ratio

It's confusing because there are two aspect ratios involved. The one we just call the aspect ratio describes the proportions of the whole image. The pixel aspect ratio describes the proportions of each pixel in the image. To get "perfect pixels", you want to manipulate the pixel aspect ratio, and the only way to do that on MiSTer is through the other aspect ratio.

If you're happy with square pixels (which will not usually be strictly accurate to original hardware) then your custom aspect ratio is just the internal resolution of whatever core you're using.

[morf77]

It's not my intention to further complicate this topic but I found no mention of pixel clock in this discussion. Pixel clock is important when calculating perfect aspect ratio's since it influences the amount of pixels per line. The aspect ratio calculator in my signature takes this into account.

[Yim]

Can you explain further the relevance of the pixel clock? Are you just using it to figure out what the aspect ratio would be on a CRT and so what the target aspect ratio should be for upscaling on HDMI?

[morf77]

Can you explain further the relevance of the pixel clock? Are you just using it to figure out what the aspect ratio would be on a CRT and so what the target aspect ratio should be for upscaling on HDMI?

Correct. My understanding is only derived from Rysha's source code I turned into the online calculator.

The pixel clock can give you the pixels per line for NTSC this is pixel_clock * 704.0 / 13.5 or for PAL: pixel_clock * 702.0 / 13.5
From this you can derive the horizontal aspect ratio being visible_pixels / pixels_per_line
This horizontal aspect ratio together with the vertical aspect ratio gives you the proper aspect ratio which in turn can be used to find suitable integer aspect ratios for upscaled resolutions.

Source code of the calculator in a python script format:

Code: Select all

import math
import numpy as np
from fractions import Fraction

def find_nearest_h(integer_lines, visible_pixels, proper_ratio, v_multi):
    last_remainder = 1
    r = 1
    for x in range(-5,6):
        integer_pixels = visible_pixels * (v_multi + x)
        try:
          diff = abs((integer_pixels / integer_lines) - proper_ratio)
          if (diff < last_remainder):
              last_remainder = diff
              r = x
        except:
              return 1
    return r

def find_gcm(x,y):
    for i in range(30,0,-1):
       if (np.mod(x,i) == 0 and np.mod(y,i) == 0):
          return i
    return 1

print("Mister Aspect Ratio Calculator")
print("Expected format: <visible dots> <visible lines> <dot clock in mhz>")

NTSC_CONST = "704.0/13.5"
PAL_CONST = "702.0/13.5"

ACTION_SAFE= 0.035
TITLE_SAFE = 0.05

LINES_NTSC = 240.0
LINES_PAL = 288.0


pixel_clock = float,input("Pixel clock: ")
visible_pixels = float,input("Visible pixels: ")
visible_lines = float,input("Visible lines: ")
video_type  = input("Video type (NTSC/PAL):")
clockstr = str(pixel_clock[1])
pixelstr = str(visible_pixels[1])
linestr = str(visible_lines[1])
print("Calculating for visible area of "+pixelstr+" x "+linestr+" at "+clockstr+" mhz pixel clock")

if video_type == "NTSC":
  pix_per_line = round(float(pixel_clock[1]) * 704.0 / 13.5)
  video_lines = LINES_NTSC
else:
  pix_per_line = round(float(pixel_clock[1]) * 702.0 / 13.5)
  video_lines = LINES_PAL

print("Dots per line: %f, Lines: %f Standard: %s\n" % (pix_per_line, video_lines,video_type))

horizontal_aspect = 4.0 * float(visible_pixels[1]) / pix_per_line
vertical_aspect   = 3.0 * float(visible_lines[1]) / video_lines

arx=0
ary=0
proper_ratio = horizontal_aspect / vertical_aspect


ratiofraction = Fraction.from_float(proper_ratio).limit_denominator(4096)
x = ratiofraction.numerator
y = ratiofraction.denominator


print("Aspect ratio: (%d:%d) (Floating Point:%f  Fraction:%f)" % (x, y, proper_ratio, (x / y)))

arx = 0
ary = 0


fractratio = pix_per_line / float(visible_lines[1])
ratiofraction = Fraction.from_float(fractratio).limit_denominator(4096)
xres = ratiofraction.numerator
yres = ratiofraction.denominator

print("Square aspect ratio: (%d:%d)" % (xres, yres))

res = [480, 720, 1080, 1200, 1440, 1576]


for i in range(0,6):
        arx = 0
        ary = 0
        v_multi = int(int(res[i]) / int(visible_lines[1]))
        integer_lines =float(visible_lines[1]) * v_multi

        nearest_h = find_nearest_h(integer_lines, int(visible_pixels[1]), proper_ratio, v_multi)
        integer_pixels = float(visible_pixels[1]) * (v_multi + nearest_h)


        if integer_lines > 0:
          fractratio = integer_pixels / integer_lines
          ratiofraction = Fraction.from_float(fractratio).limit_denominator()
          xres = ratiofraction.numerator
          yres = ratiofraction.denominator
        else:
          print("zero or negative")
          xres = 1
          yres = 1

        scaledwidth=int(v_multi + nearest_h) * int(visible_pixels[1])
        scaledheight=int(v_multi + nearest_h) * int(visible_lines[1])
        near1440Hor=int(v_multi) * int(visible_pixels[1])
        near1440Vert=int(v_multi) * int(visible_lines[1])
        if video_type == "NTSC":
           framerate = 60
           blankperiod = 1.172
        else:
           framerate = 50
           blankperiod = 1.188
        reqclock = float(scaledwidth) * float(integer_lines) * float(framerate) * float(blankperiod)

        print("Integer aspect ratio for %dp (%dx and %dx): (%d:%d) minimum required pixel clock: %f mhz" % (res[i],(v_multi + nearest_h),v_multi, xres, yres, reqclock/1000000))



        if int(res[i])==1440:
           if reqclock/1000000 > 185.203 :
              print("!! WARNING !! %dx integer scaling will not run with video_mode=12 and will require a custom_video" % (v_multi + nearest_h))
              print("!! WARNING !! To enable %dx integer scaling set a custom 1440p video_mode for this core with at least %f mhz pixel clock" % (v_multi+nearest_h, reqclock/1000000))
              print("!! WARNING !! Or use %dx integer scaling requiring a (%dx%d) video_mode." % (v_multi, near1440Hor, near1440Vert))   
              print("!! WARNING !! eg video_mode=1792,128,200,328,1344,1,3,46,204800")

[Yim]

Thanks for the explanation. I’ve always kind of just gone off what looks right to me or tried to figure it out from images of original hardware running through CRTs.

[dfilskov]

Thanks SO much, guys! - it seems this is a lot more complex than I first thought. BUT you helped me make it work now yeehaa!

My new screen is square (1:1) - 1920x1920 so my procedure needs to take that into account.

[dfilskov]

Will non-square integer pixels give shimmering? - I thought that as long as all pixels had the same size / aspect ratio there would be no shimmering.

[Yim]

Integer pixels should avoid shimmer whatever shape they are, so long as every pixel in the image is made up of the same number of screen pixels, as you say. Shimmer as I understand it is caused by image pixels changing size as they move across the screen.

Regarding your new screen, what video mode are you using? The amount of vertical scaling will vary with the vertical resolution set in the video mode, and the required horizontal scaling will change depending on the vertical scaling. If you’re not using a 1:1 PAR, then you’ll need to change the custom aspect ratio when vertical scaling changes to keep integer pixels.

It could be that you’ve got the scaling right coming out of the MiSTer but the screen is upscaling it (and changing the PAR) if you’re using a video mode with less resolution than the screen. Or if you are using a square video mode then it could be that the image is too wide. With vertical integer scaling on the mister will usually use almost all of the vertical resolution set in the video mode. If the intended aspect ratio is wider than it is tall, it’s likely the horizontal resolution of the upscaled image will be larger than the resolution from the video mode, in which case the image will be downscaled to fit (or just not upscaled as much) and the image won’t have integer pixels any more.

[dfilskov]

Thanks again! - I have a lot go on now with your input so I'm sure I'll get there eventually.

I'm using native reso 1920x1920.

Very good point about the width being too narrow (for square or wide pixels) - I could use 1920x1440 or 1920x1600 perhap, but I'd trade some size for pixel sharpness probably and will try to work with my screen native reso if possible.

I'm OK with using not-square pixels though, I think.

My screen is set to not scale at all. If I set the MiSTer reso to e.g. 1280 x 1024 I get a centered smaller-than-the-screen-area image.

[dfilskov]

It's not my intention to further complicate this topic but I found no mention of pixel clock in this discussion. Pixel clock is important when calculating perfect aspect ratio's since it influences the amount of pixels per line. The aspect ratio calculator in my signature takes this into account.

Thanks, morf77! - could you be kind enough to explain how to use it to get the largest integer image with aspect ratio closest to the original? - e.g. for SNES I get these data:

MiSTer video info
512x224 - 15.75 KHz - 60.1 Hz
1920x1920 - 230.87 MHz - 60.1 Hz

Mister Aspect Ratio Calculator
Pixel clock in mhz
230.87

Visible pixels
512

Visible lines
224

Video type
NTSC

Calculation Results:
Dots per line: 12039.000000, Lines: 240.000000 Standard: NTSC
Aspect ratio: (235:3868) (Floating Point:0.060755 Fraction:0.060755)
Square aspect ratio: (12039:224)
480p
Integer aspect ratio for 480p (0x and 2x): (0:1) minimum required pixel clock: 0.000000 mhz
720p
Integer aspect ratio for 720p (0x and 3x): (0:1) minimum required pixel clock: 0.000000 mhz
1080p
Integer aspect ratio for 1080p (0x and 4x): (0:1) minimum required pixel clock: 0.000000 mhz
1200p
Integer aspect ratio for 1200p (0x and 5x): (0:1) minimum required pixel clock: 0.000000 mhz
1440p
Integer aspect ratio for 1440p (1x and 6x): (8:21) minimum required pixel clock: 48.389161 mhz
1576p
Integer aspect ratio for 1576p (2x and 7x): (32:49)

[dfilskov]

OK, I got SNES to work now but not quite as I'd have liked.

I either had to make the ratio narrower in 1920x1920 - just like Yim wrote - as square pixels exceeded the width - thus the largest image had slim / narrow pixels (7:8 perhaps or maybe even 4:5) and stretched graphics - or - I had to go to a lower but less tall 4:3 resolution 1920 x 1440 (too "fat" IMO as it seems my monitor neither sopports 1920x1600 / 6:5 - nor 1920x1536 / 5:4 - work) wasting a LOT of screen estate making the graphics unnecessarily small as seen below.

sharpness trade-off.png (3.31 MiB) Viewed 17515 times

[dfilskov]

FINALLY. I have a working solution now which should work on all screens.

It's close to Yim's method but use vscale_mode=0 and set the vscale_border to shrink the image to the exact resolution I want.

Note that for some reason vscale_border is disabled when vscale_mode is other than 0 (I'm hoping sorgelig will change this behaviour) so that's why I had to set vscale_mode=0

E.g. for NES (on my 1920x1920 setup) I set vscale_border=64 and custom_ratio_1=1792:1440 ; (256*7 : 240*6) - causing graphics to be just a little compressed heightwise
and
custom_ratio_2=1792:1680 ; (256*7 : 240*7) making the graphic too tall in my opiion.

That's perfect sharpness but largest possible image. I'll try to update the guide with this procedure.

Note that for some reason vscale_border is disabled when vscale_mode is other than 0 (I'm hoping sorgelig will change this behaviour) so that's why I had to set vscale_mode=0

[LamerDeluxe]

Cool idea. Using the vscale_border should make it possible to let my custom aspect ratio method for integer horizontal scaling have correct aspect ratio as well. I use that to get perfect vertical scan lines for vertically oriented arcade games.

It is a lot of hassle though. An option to set horizontal integer scaling, or for both directions, would be much easier.

[Yim]

It is a lot of hassle though. An option to set horizontal integer scaling, or for both directions, would be much easier.

I may have some good news for you.

[pbsk8]

I have a humble monitor 27'' lcd fullHD 60Hz screen to play the mister via hdmi

So far didn't mess with any of those options for scale, etc.

I still don't understand how I would have to change the NES parameters for my case.

[LamerDeluxe]

It is a lot of hassle though. An option to set horizontal integer scaling, or for both directions, would be much easier.

I may have some good news for you.

That would be awesome!

[orange428]

Hi dfilskov, I've been looking at an Eizo monitor and saw it looks like you have one based on the picture you attached. From reading the manuals it looks like you need to use dual-link DVI-D or DisplayPort to get 60Hz at native resolution. How are you connecting yours to the MiSTer's HDMI port? Can you please let me know what adapters or cables you had to get to make the connection from HDMI to either dual-link DVI-D or DisplayPort? Thanks so much!

[dfilskov]

Eizo square monitor

Hi - yes. I couldn't make Displayport work so I use DVI-D - tested a some hdmi-to-DVI-D cables and found two that work without flickering.
They're both short (1.5 m) and have magnetic "filters" (magnet rings) at each end close the connectors. Actually one of them was an HDMI-HDMI with a connector-converter to DVI-D I think but I use a native HDMI-to-DVI-D cable.
The only issue I have with the disolay is that it can't show 50 Hz so you need to set refresh_min to 60 (or perhaps 56) ruining smoth scrolling on PAL resoutions.

[orange428]

Hi dfilskov, thanks for letting me know you're just using a HDMI-DVI-D cable and no special adapters. I ordered two cables to test with (https://www.amazon.com/dp/B0713SJ1KW and https://www.amazon.com/dp/B00DI89J58/).

If you could, can you please share your:

video_mode
vsync_adjust
refresh_max

parameters you're using to get 1920x1920 resolution at 60Hz? Thanks so much for your help.

[orange428]

Hi, just an update on my end. My Eizo monitor is a 2048x1536 native model from 2013, so it's a bit older than yours. I tried using video_mode=13 and also calculating my own custom video_modes using reduced blanking with both CVT 1.1 and CVT 1.2. The pixel clock was just too high for 2048x1536 at 60Hz. It was strange though, I did get a picture which looked good and it did seem to work but my monitor constantly displayed a pixel clock warning in the upper right corner. If only the warning would go away but it never does.

My monitor does support 1:1 scaling though so I ended up just using 1024x768 resolution and let the monitor 1:1 scale it to fill up the screen. This worked out since the native resolution is exactly double of 1024x768.

The only downside is the monitor is almost 10 years old so the response time isn't great, 20ms.