Chromium Embedders Web Engines Hackfest 2021

Chromium Embedders - Web Engines Hackfest 2021

[zakhar presents]

• Video: https://www.youtube.com/watch?v=M9EuTkViITQ&list=PL4sEzdAGvRgCTD9dJtGHTpeh9W6Wx7aLX
• Slides: https://webengineshackfest.org/2021/slides/kiosk-mode-browser-using-chromium-embedded-framework-cef-by-zakhar-voit.pdf

BK: I see a lot of similar-ish things with the way that web engines are deployed to solve problems that aren't a full "browser" - "the button for the internet" that people use on their desktop or mobile device

nick: What about the usage of CEF today... My impression is that it is mainly used for desktop applications, is it used on embedded devices?

z: I was asked this question a few months ago and yes, most of the examples I know of are desktop usages but I don't reall know - if someone has more information or examples I would love to hear them?

Tyler Wilcock When would one use CEF over something like WPE?

z: I don't have experience on the other, so I would love to hear myself. I was hoping that the presentation would raise some issues. I think in one of the other talks we are going to talk about some uses (AGL), and will talk about some reasons why we like CEF as well.

Lorenzo: we will talk about how we were using chromium before and why we like CEF better there. I think your example of using CEF for kiosk mode very natural, as is a good way to embed Chromium capabilities, and integrate some other more specific features on top of it.

z: The documentation was very helpful. I was able to mostly focus on our own features that we needed to implement, but the things like when you want to download documents were a little tricky. Most of the things though were actually well documented and pretty straightforward.

bk: I guess... was the question about CEF specifically? because chromium is certainly used on a number of embedded devices.

Gustavo: I guess, what even is embedded now.. I worked on a device that could be considered embedded, but I am not sure if it would be. We used CEF for that, it was the primary user interface... I'm not sure if that counts though.

Loren: I think sometimes you just don't need the whole chromium, but just some parts - and that works well with CEF I think.

Alex D: I participated in AGL at some point... Why I think CEF is not really widespread for embedded platforms is that the use cases usually imply porting chromium, and it needs to be done anyway... But if you want to use CEF, you kind of need to port CEF as well. You still have to maintain your own port or convince the CEF owners to accept your patches. Maybe it just seems to some people it is not worth the effort.

Gustavo: yeah, one thing we did was porting Igalia's work to our own. It was a non-trivial amount of work. It was eventually upstreamed. I don't think it is hard to port CEF if chromium itself is ported.

z: There was a recent ARM support port - it took a year or a couple of years. It certainly is possible and if chromium itself supports the architecture it seems they are trying to as well

Gustavo: The project I am talking about was using ARM, I guess maybe that was some very specific ARM thing? I think the M1 is crashing.

z: I could be wrong, it seems the patch started in 2016 for linux ARM support. https://bitbucket.org/chromiumembedded/cef/issues/1990/linux-add-arm-build-support

Gustavo: You can see on that bug there's a patch from 2016 and someone talking about rpi and it working right out of the box - so I think when we are talking about porting it's about more specific features. I think AGL might even have some ARM based experience on that too, right?

Lorenzo: Yes... I dont want to step too much on my talk later, but yes, it does. If that's ok with you we can come back to that after.

Gustavo: My bank uses CEF for their desktop and mobile applications and it's been a year it still doesn't work on the M1.

Lorenzo: Administrations are sometimes stuck for a long time with certain bugs. Is it a big bank?

Gustavo: It is brazil's biggest bank and also in some other places like Chile.

Lorenzo: It is always suprising to me when there are hundreds of thousands or millions of users living with a bug.

Brian: Certainly there are not millions of people in Brazil using the bank and an M1 chip.

Gustavo: Absolutely. And bugs happen, we know that. There are two other things worth discussing - like Electron, and also how many browser engines do we need on a computer these days. Couldn't we just have one?

(silence)

Gustavo: I’ll take that as a no.

Brian: this gets into the question I was trying to ask earlier. On desktop, you can have multiple engines. Are you suggesting there should only be one engine per operating system, or are you asking something more subtle?

Gustavo: Web engines are too monolithic, it would be better to have the system give a rendering frawmework so you don’t have to embed Electron in VS Code, for example.

Brian: That’s the case in iOS. In both iOS and Android, you can install things that are web views, but there’s also a way to say “hand this to The Browser”.

Zakhar: I was going to bring up PWA on desktop, and seems like what we want here for use cases like Electron. We’d like to have web sites installable like applications on desktop.

Brian: It is very much like a PWA, and if you look at a lot of stuff on embedded devices, a lot of them want something like that. Right now, everything has apps, your toaster has apps. And now people are talking about "mini-apps", which are basically PWAs for specific platforms (so they can have more privileges and access to device-specific stuff). It feels like lots of people are trying to solve the same problem: I don’t want a whole OS, I want something that can install these PWA-lik things. Seems like there could be a standardized way to allow it.

Brian: My questions to Gustavo is, would you agree it’s good to have multiple web engines?

Gustavo: Yes, absolutely. Having only one gets you into the early 2000s IE scenario. My thing is more, how many versions of the same web engine do we need? I have multiple versions of Chromium.

Brian: This is a fair question in all kinds of ways. Something like 95% of all sites used jQuery at one point, and everyone was like "gee...is there a reason it isn’t built into browsers? shouldn't there at least be a way to really share that download so every single website on earth doesn't have so many k downloaded and copies in cache"

Gustavo: I have Signal, I have Spotify, I have VC Code, I have Chromium itself. That’s four versions of Chromium. Is there a way forward to let various apps use the same Chromium, instead of having to bundle its own Electron?

Nick: This is more of a long term effort. The developers of these different frameworks have different needs. But I agree that it's far from ideal as things are now. Something like PWA wrapping could be a way forward. I always end up thinking about this as well because I have a lot of apps using Chromium under the hood. Maybe AGL is a good model, being based on webOS. On the embedded use case, I don’t think we have these on desktop, but maybe the PWA model is the way forward.

Brian: If you’re not familiar with PWAs, primarily that’s using a service worker and a site offers a way to “install itself” to your home screen and to most people it’s indistinguishable from a native app. Effectively once it lauches, it takes you to a URL that has a reverse proxy around the cache object, so it can have a little JS server that can consult local storage to cut down on network calls. That has almost all you need, but may lack embedded security models and a high-level channel to allow a PWA to install a sub-app or otherwise communicate with the computer, but lets itself run as a browser in kiosk mode.

Zakhar: Jitsi offers a way to install locally.

Brian: We managed to fill the break, so let’s get restarted.

Lorenzo: (briefly and smartly recaps above discussion for those who stepped away)

[Lorenzo presents]

• Video: https://www.youtube.com/watch?v=lx6aXffPL0k&list=PL4sEzdAGvRgCTD9dJtGHTpeh9W6Wx7aLX
• Slides: https://webengineshackfest.org/2021/slides/embedding-chromium-into-agl-demo-platform-with-wam-by-lorenzo-tilve.pdf

Eric Meyer: Thanks! Also, the idea of including a video game in an automotive embedded system kind of worries me.

Gustavo Noronha (kov) not a problem if it's running on the backseat screen for the kids ;D

Mario: I believe Tesla already does that, although only available while the car is off (e.g. racing games you play using the actual wheel)

Gustavo Noronha (kov): indeed, I heard of that

Lorenzo: It’s a good point that having a driving video game in a car is a little worrying. Point it that you can ship whatever HTML allows in an AGL embedded device. WebGL is available, and so on. It’s a good example of isolation and security.

Lorenzo: When you have a web runtime in your embedded device, in order to be able to interact with different complements and APIs that exist, Chromium should have a superset of them in order to be able to talk to everything. Some web apps need to be able to consume information from the car’s interfaces, but something like a game should never be able to consume those things.

Brian: If you weren’t here during the break, this is sort of the thing I was hinting at. We have things like WebOS and the Web Application Manager because pretty much every device that wants to have levels of access like this has to re-figure this out. There are lots of ways to get this wrong but then there are questions like how do you update the device. I can see there being useful standardization and definitively answering questions about security models and so on.

Lorenzo: In desktop, there are things that are like a webview but to the user they’re indistinguishable. It’s true there’s nothing standard enough yet, we can have different engines in the same enviroment. We were using Pi 3 that were able to run the AGL thing I showed. There’s reference hardward based on that.

Lorenzo: You can put CEF recipes for that in your build process. In something like an interface for a car or a settop box or other domestic device, it would be good to have a common security model, something more related to mobile PWAs than desktop apps. We need something more amibitious for the embedded world.

Brian: Things like games, if you use WebGL you can have a PWA. You can think about as, you have all these apps and maybe something that syncs with email and so on, but you only have one audio channel. Some phones detect if you’re driving and will go into distraction-free mode. You can imagine someone making an app that requests other apps mute themselves.

Lorenzo: That makes a good point. So far PWAs are geared to provide full functionality. Embedded is a little different. There are clear examples in cars that show this, like disabling media apps while in motion. The navigator should have priority on the audio pipeline. Same thing for notifications. More critical services like collision alerts or flat tire notice should have different levels of priority. Something in the middle should manage this. If every app makes its own decisions, then you can’t ensure the GPS app has more priority than the music app.

Brian: That tmakes sense. There’s a question of isolation, but also a need for a certain kind of communication between apps and devices. This keeps having to get solved and re-solved in different devices. I can imagine a world in which I can get webbiness from a device, but create the best interface ever for HVAC systems so every HVAC device can install that and it has a way to communicate and be installed and be approved in different stores.

Lorenzo: In certain scenarios like this, the complexity of installation is controversial. There should be alternative security levels. You can run into problems when things are deployed in containers. You have to be more careful about how you put security in place.

Gustavo: Tracking against upgrades to chromium is something that concerns me about CEF. It’s kind of a one-person project, right? What happens if this guy stops working on it?

Lorenzo: That’s true, there’s more room for improvement. It would be good to more people working on it. In any scneario, it should be possible to get the maintainer of CEF more open to get more contributions landed.

Brian: Let me rewind and throw out an example: many/most UIs that employ a web engine want to make all of their interfaces out of web engines. We can talk about how you install PWAs and service workers. Service workers talk about a cache. If you send someone something with your infotainment system, you want the system’s interface to be a PWA that can communicate with other systems. To fill the cache of a PWA, you have to make an HTTP request. So we need answers how to prefill things in case HTTP isn’t avaialable. There could be one answer to the question of how to do this.

Lorenzo: Any AGL app can do token validation and run some apps locally. For a PWA, there are things around bundling and offline usage that could help with this. Not sure how they could work in this context but I agree it’s something we should find a solution for. If your tablet is using local apps, you need to use them offline sometimes.

Brian: If you install a Chromium browser, they usually have profiles and that’s where it looks for a cache. So you can prefill a profile. It feels like it would be really good to solve this problem for the more general case.

Lorenzo: You do have access to profiles and do keep certain information there in some web apps. That’s an existing approaach that is available in embedded.

Lorenzo: Gustavo, do you have experience with approaching a community that is very small?

Gustavo: It would probably fix itself in a way, if necessary. Several people are relying on CEF so if the maintainer drops out, others would step up.

Gustavo: I’m amazed one guy can keep up with Chromium. It’s a massive amount of work to keep CEF up to date.

Lorenzo: Quite a lot changed during CEF 2 development. But I mostly agree with what you’re saying. I think it would be easier to grow the community maintaining CEF if people contribute patches.

Brian: API stability is less of a problem for WebKit, as I understand it. True?

Gustavo: It depends on on the specific WK port. Some are stable with specific breaking points. Apple never broke anything. You can still use WebKit 1 even though WebKit 2 is available. It took a long time for WebKitGTK to obsolete an older version.

Mario: The WebKit engine is designed to be the boundary, but in chromium the internals are changing very fast and you can take what you need. The purpose of the architectures are very different. WebKit was designed right from the beginning to be embeddable everywhere.

Lorenzo: Zkhar, what are the challenges you see in this idea?

Zakhar: In terms of breaking APIs, we didn’t have much problem with that in CEF. But it’s more of a high-level framework and pretty much everything works out of the box. If something doesn’t, you probably have to contribute your own patches. Could be a problem keeping those patches up to date, because Chromium APIs change a lot and so CEF does at well.

Zakhar: Find out how well does what you’re trying to do fit with the CEF architecture.

Lorenzo: Hopefully, the number of things that don’t fit isn’t very high.

Brian: I thought this was a fantastic discussion and great presentations. I learned things! Thanks to everyone for coming, and thanks to our speakers. Hope you’re signed up for other HackFest sessions!

[room thanks the room and says goodbye]