Correct Color Palette

[James-F2]

@ekeeke
Please add the correct 4bit PPU color palette to this excellent emulator.
As I understand the Genesis has 3bit and 1bit emphasis, and currently the image looks darker than what the real mega drive PPU outputs.

Here is the correct LUT in the MiSTer Genesis core:
MiSTer-devel/Genesis_MiSTer@a297b52

[ekeeke]

Well, I have compared the output of this emulator on Wii with my MD1/MD2 output on same TV, both with RGB scart and it looked quite exactly the same, and certainly not darker.

I know about the correct RGB ramp as it was first discussed on spritesmind forums long time ago (see http://gendev.spritesmind.net/forum/viewtopic.php?f=22&t=2188) but it has nothing to do with making image darker and it didn't make any noticeable differences in any games when I tried to implement it so I ditched it.

As for the 'darker' colors thing, it's more related to the fact that most recent emulators (including FPGA implementations) assume the VDP max level corresponds to 255 in RGB888 encoding scheme, i e that white output by MD console is 'pure white' as defined by video standards, while this emulator rather use max measured level ratio compared to standard video RGB level (4.7V for max level at VDP output, scaled down to 0.9Vpp at video encoder input vs 5V for VDP reference voltage which would give 1Vpp at encoder input and correspond to 'pure white' voltage in encoder datasheet). As said above, my experiments showed that (at least with my french RGB MD models) my assumptions were correct and white level output by MD was not 'pure white'. Using max RGB888 range in emulator showed very minimal difference (making screen indeed slightly brighter) but still enough difference with MD console output used as reference to make me think current model I use is the correct one.

That said, I understand this is mostly an user preference thing at the end so I am not against adding an option to let user choose between 'limited' and 'full' RGB range. Same for letting users choosing 'linear' or 'original' RGB levels and play the 'spot the differences' game by themselves ;-)

[James-F2]

It made very noticeable difference when it was implemented in the MiSTer, and looks much closer to what I see from my console with the OSSC.
That would be very awesome if it will be switchable and accurate to the original in GPGX too.
Thank you ekeeke.

[ekeeke]

Could you provide info on which games it made noticeable differences to you or better, screen shots made with core before and after correct levels implementation ?

It really wasn't much noticeable from what I remember when I tried it so maybe Mister initial implementation was wrong: it is indeed the case with some older emulators which get shadow/highlight color levels wrong or use much reduced rgb range, i.e they simply shift color RAM RGB 3-bit value left to get RGB 8-bit value, thus setting max level to 224 instead of 255 (or 238 in this emulator) and indeed getting a noticeably darker image.

[James-F2]

It made difference in all games.
Mister uses very close Shadow and HiLite values from the hardware measurements you linked.
Though Kitrinx can't remember what emulator she took the values from.

Mister Aladdin Old

Mister Aladdin Correct

Mister Sonic Old

Mister Sonic Correct

[James-F2]

Got it.
Blast Em.

static uint8_t levels[] = {0, 27, 49, 71, 87, 103, 119, 130, 146, 157, 174, 190, 206, 228, 255};

https://github.com/AmatCoder/blastem-gtk/blob/37cf19bfbe8963490a7919f120ab1523abc727a4/vdp.c#L67

Here is the Highlight and Shadow implementation:
https://github.com/AmatCoder/blastem-gtk/blob/37cf19bfbe8963490a7919f120ab1523abc727a4/vdp.c#L164

[ekeeke]

Yes, those are the numbers derived from the voltage measurements in spritesmind thread, scaled to maximal rgb 8-bit range (255). Mister use a 16-entries lut because it's likely easier for implementation, although there are only 15 levels in reality, hence the duplicated 255 value.

Looking at this Mister commit you linked above and Mister video mixer implementation, it seems that Mister older implementation was using same max value as Genesis Plus GX (238 i.e 11101110b) and I can confirm the differences you see in those pictures are similar to the ones I noticed on my TV when trying out 255 as RGB max value (instead of 238) and I remember testing the exact same games. But like I said also, comparison with my RGB MD models on the same TV showed that 238 as max RGB value was more correct and gave closer results to my original console. Maybe it's different when using non-RGB models which output composite video encoded from VDP RGB output ?

[James-F2]

Can't say about Composite because I haven't used that in ages.
But I can say that Mister IO Board has Analog RGB output that is connected to my european 29" Sony Trinitron with a VGA to Scart cable, and the current Mister palette is very similar to the Genesis when I switch between the two with my Scart switcher.
The real Genesis might be slightly brighter, but the resulting gamma curve is very similar,, certainly closer than the old mister implementation.

Anyway, it would be very useful to have the original measured levels available in GPGX.

[ekeeke]

Can't say about Composite because I haven't used that in ages.

What I meant is that unless you are using a French RGB Megadrive with official RGB scart, your console still outputs a composite video signal that is converted back to RGB by an external hardware converter. Depending how these recreated RGB signals are amplified by those converters (and they likely are amplified to match RGB scart spec), the end result could be brighter than with original RGB consoles (which uses its own official RGB adapter box).

Even RGB-modded consoles, which use third-party circuitry to bypass the video encoder and adapt the VDP RGB output for RGB scart standard probably have different amplification levels than the official RGB console.

That said, I understand that people like the slightly brighter image those modern RGB solutions provide and also, most emulator users run them through modern screens, where a given RGB range probably looks different than with CRT TVs, so the most logical is indeed to provide both options (limited or full RGB range) in the emulator.

[James-F2]

Thank you for considering to implement this option.

I think it has to do more with 'linear' vs 'original' grey scale, than 'limited' vs 'full' range.
I don't mind slightly darker whites, but I do mind if the emulator does not match the original VDP grey scale tracking.
Maybe I'm misunderstanding this, and they are one and the same.

[p1pkin]

@James-F2

but I do mind if the emulator does not match the original VDP grey scale tracking.

we don't know how exactly original VDP works, there required chip decap and its analysis to get all the details.
so, at this point it is more like "one approximation VS another approximation", where one is not any "better" or "more accurate" than another.

[James-F2]

The equivalent to 'Full Range' + 'Original Grey' most certainly looks closer to what the Genesis outputs than what GPGX has now.
It is also how the Genesis looks with Scart RGB + OSSC.
BlastEm and MiSTer use that approach too, because it looks authentic to the real console output.

I hope to see that implemented in GPGX soon.

[eadmaster]

The Master System also has a non-linear blue channel that should be reproduced.

[James-F2]

Bumping this.
It would be lovely to see correct original Genesis colors in GPGX.
The correct values are well known and can be taken directly from Blast Em.

[p1pkin]

@James-F2

The correct values are well known

they are not, BlastEm have just another (quite rough) approximation, or in other words "palette which likes its author".

[James-F2]

No need to go as far as decapping, a real world comparison will do.
What I see with my Genesis (RGB) into a RetroTink 5x vs MiSTer (BlastEm palette) is basically identical to my eye when switching between the HDMI inputs, but GPGX is much darker with wrong colors.

Genesis has a certain look, and I think GPGX should at least provide an option to get close to it with the knowledge we have today.

[mikepavone]

BlastEm and MiSTer use that approach too, because it looks authentic to the real console output.

For what it's worth, I did not pick 255 as the max level in BlastEm because I did any real comparison to hardware. My general thinking here is that since max brightness can vary significantly between displays (especially since most displays have brightness controls) that there is not really any ground truth when it comes to max brightness and so I went with 255 because it was my personal preference.

There is some more objective truth with respect to the relative values, but as has already been pointed out the non-linearity is relatively subtle compared to the choice of max brightness. Further, what I'm doing in BlastEm is not really fully correct either as NTSC and PAL have different gamma curves than an sRGB display and I do not currently account for that at all.

[birdybro]

I agree with @ekeeke that a French Megadrive doing SCART RGB is going to differ significantly from one that uses an RGB modification on it. Additionally even different regions and models had different levels and different encoders, see this article for more information @James-F2 :

https://medium.com/rgb-inside/comparing-native-rgb-video-quality-of-different-models-of-mega-drive-and-sega-genesis-english-e3d3703270ba

I have a JVC X'Eye, MegaJet, an MD1VA3 and an MD1VA6. I also grew up with a 32x on top of a Genesis Model 2. I can confirm they all look different. So which one is "correct"? The answer is, none. Here's the variety of RGB output compared in the article so you don't have to scroll through it and so it doesn't disappear when the article goes away someday:

Japanese MD1

Japanese MD2

US MD1 "High Definition"

US MD1 no "High Definition" written on it

US MD1 Modded by cutting the leg of the composite video signal to clean up RGB

Tec Toy Brazilian MD3 (actually an MD2 but with a Fujitsu video encoder)

Tec Toy Brazilian CDX

32x (yes it modifies the image quality)

Mega Jet

JVC X'Eye

The likely solution would be to have a variety of color palettes to be selectable like other emulators have done before (usually NES emulators, because of that system's uniquely strange color situation). You can really get into the weeds on this stuff. The audio differences between every variation of Sega Genesis are a ridiculously long list. :P

[Dogway]

To add to the conversation I don't know how accurate BlastEm is regarding color output but from some tests it's nothing fancy, just lowering contrast in a sigmoidal fashion in RGB, so saturation is decreased along luma contrast.

pseudo-call:
sigmoidal_inv(rgb, strength=4.0, pivot=0.52)

I added an option to mimic the look in grade shader, along the blue push for SMS. Will upload in a few days.

[mikepavone]

Literally all I did in BlastEm was took the voltages others had measured for each of the Mode 5 RGB levels, divided by the max measured voltage and multiplied by 255. Those values are in levels in vdp.c and get used for translating palette entries. Nothing fancy at all.

[Dogway]

Well so it seems legit, at least it's based on measurements. Would be nice to add source reference so we (ekeeke included) can have a look and maybe implement it.

[ekeeke]

@Dogway: source is already referenced, see spritesmind forum link I posted in second message. The values used in Blastem are also referenced somewhere above. As Mask of Destiny explained, there is nothing complicated, it's just that I never took the time implementing it (it also requires some design changes in vdp rendering code). This will be done someday, eventually...

[KrokusPokus]

@ekeeke: Is adding an option for a 255 max level still considered? My main concern is using the rather deplorable backlight output of my laptop's TFT screen to its fullest. With a max of 238, a good 7% of the light output gets wasted. (Well, technically more, because of gamma...)
(Also, with back-lit displays in general, having to turn up the backlight to compensate for not using full range increases backlight bleeding, making blacks more greyish., lowering contrast more than neccessary..)