Can we make a simple protocol or API for simple displays in Elixir?

[lawik]

The idea here would be that for any display that we can drive we would be able to implement a small API where most displays can be treated the same and things like pixel rendering of primitives, Scenic Drivers, host development simulators and maybe even test rigs can be compatible with multiple display libraries.

For now we have the OLED and ssd1306 libraries which both run the SSD1306 OLED display and we have Inky.

I think they cover enough ground that we can figure out some of the requirements if we would like to do this. For example, Inky refreshes slowly and any changes should be batched to avoid being constantly changing and useless. I think the OLED library happily refreshes line-by-line or something, from a brief browse of the code.

If we decide on some standard approach to throw pixels at the display and letting the display library figure out how to run the device well we can also start working on common tooling, such as Scenic drivers and drawing primitives. groldan has a working prototype library for rendering pixel fonts without scenic, which is a big part of what we'd need to implement a non-OpenGL Scenic Driver that never touches the framebuffer and can fairly efficiently run these displays.

I think it seems fun and makes sense. So I mostly want people to throw thoughts, hats and opinions into the ring.

Thoughts?

@nyaray @luisgabrielroldan @fhunleth @mobileoverlord
(Frank and Justin, we know you have plenty of other stuff, just want to keep you in the loop)

[nyaray]

I'd almost forgot about this idea, excellent!

I think we can base it on https://github.com/pappersverk/inky/blob/master/lib/inky.ex and https://github.com/pappersverk/inky/blob/master/lib/hal/hal.ex, but would like input from someone who's built another kind of driver.

[lawik]

I think the OLED does some stuff line-by-line? Or at least can do things line-by-line in a way Inky can't.

And I figure if we can unify on something minimal to implement like a module with two public functions that would be ideal:

• set pixels
• push pixels to display

Then the display library can choose a default strategy for how to best handle this simple protocol internally. In Inkys case, we'd want to throttle the rate of updates and batch them. We maintain a pixel buffer as state. In a fast display you might just want to refresh according to the max potential refresh rate or just push as soon as possible, you may or may not need to maintain the state.

And then we have color, which can be tricky, how do we want to define pixel colors. Generally I'd say the display library should do any downsizing of the color values provided. So for a red Inky we'd likely clamp any reddish value that isn't more black or white to red. The protocol would need to define the basic format for color in my mind.

[luisgabrielroldan]

I think the OLED does some stuff line-by-line? Or at least can do things line-by-line in a way Inky can't.

Not sure what do you mean. The OLED lib just has a backbuffer to draw and push to the display. that's all.

In fact.. some operations are complicated because it works with the backbuffer in the same format as the display.

It has some optimized functions to draw lines... but there's nothing special in that...

And I figure if we can unify on something minimal to implement like a module with two public functions that would be ideal:

• set pixels
• push pixels to display

Yes, those are the basic operation for screen devices. The set pixel and flush operations.

And then we have color, which can be tricky, how do we want to define pixel colors. Generally I'd say the display library should do any downsizing of the color values provided. So for a red Inky we'd likely clamp any reddish value that isn't more black or white to red. The protocol would need to define the basic format for color in my mind.

Yes... you need to introduce color modes...
B&W, Grayscale, RGB565, RGB24... and be prepared to convert one to other...

In practice you probably want to work in the specific mode for your display to avoid any overhead...

Drivers could support something like a common set of features, the devices has to deal with data efficently. Having a to convert a backbuffer to the destiny format on each update could be expensive...

[lawik]

Ah, I thought you might have something like scanlines or such on the OLED. I only skimmed the code.

I'm thinking we could default to a certain color mode (one with plenty of colors) and let the libs figure out what they want to represent it as. If we want to keep it stupid simple.

What would you want out of a standard protocol/behavior like this?

[nyaray]

Isn't colour mode something that the display-specific code has to care about? In other words, something that is to be hidden in the separate implementations of the "set pixels"/"flush pixels" behaviour.

[lawik]

I mean we need to pass in some form of color information. And if we want to take information from Scenic for example and pass it in a generalized way to a display library we will get some kind of RGB value that the Scenic driver renders to a pixel color value. Passing that pixel on through our behaviour we'd want the standard to help with making sure the display library knows what to expect right?

If not in the formal typespec of the behaviour then at least in the documentation. "Your library should expect RGB integer values from 0-255." or something similar.

Or do you have another thought on how that would work?

I think the format of the pixel has to be in the standard, if the standard is going to provide any particular value. And when the display library receives values outside its capabilities it will adapt them to the best of its ability. Or it needs to allow color mode options but I think that might be overkill.

[nyaray]

I agree, something about the range of values in the docs would be required. Spec'ing it out properly wouldn't hurt... and we'd probably specify 0.0-1.0 or something, given that 8 bits of depth would be a bit weak for some displays :).

I think the format of the pixel has to be in the standard, if the standard is going to provide any particular value. And when the display library receives values outside its capabilities it will adapt them to the best of its ability. Or it needs to allow color mode options but I think that might be overkill.

Indeed, how we represent pixels needs to be specified but I think we should leave it up to the display driver to decide how it deals with pixel colours, dithering and other aspects related to downgrading the content to within the screen's ability, if required.