2023 Wolvic: An open source XR browser

Wolvic: An open source XR browser

• Date: 7th June 2023
• Issue: https://github.com/Igalia/webengineshackfest/issues/17

Felipe: welcome, will start with the presentation. One of the people at Igalia working on Wolvic browser for VR. Today we will talk about Wolvic. (presents slides)

alex: About the last slide you mentioned (eye tracking). How do you think this is integrated in browser?

Felipe: sure: good question. I'm not really sure.

Juanjo: Typically done in two browsers, we integrate it in Android at the 2D level and then for the 3D experience. Probably same for eye tracking. This is interesting because main model is not eye tracking (complex system with the position of the body, arms...) for us but Apple is thinking about that.

: Not only in browser level but also in sensors...

Juanjo: most of the case uses OpenXR to get the data.

????: Is the long term plan to maintain 3 backends?

Juanjo: it could be. we don't know yet. at the moment we are in middle of the chromium experience. showing very good perf improvements. This is open source project, future will depend on interest, contributors etc we will experiment in webkit too.

javiF : Some users value a lot features like adblockers that are provided by Gecko, privacy etc it's difficult to handle different behaviors for the various backends.

brian: it's a great question about the backend. it's also really tricky. a bunch of works come from downstream, currently in firefox which not the most invested things. in chromium there are more contributors so more care work for them. hopefully true for webkit too but it won't be for android. really tricky square circle. not necessary to be i chromium but also good reason for doing it to try to be competive. Firefox for Android has a very minor use, been there for long and hasn't raised.

juanjo: we discussed AR support and want it. We have a bit that in Android, we want it in the browser to be able to do some of the things felipe demoed. In chromium, the support for AR is really advanced. for gecko that would require more investment.

nick: I was curious about the QA support.

juanjo: at the end we rely on the QA effort of each backends, currently doing a lot of manual testing. we have'nt worked a lot on automating it, but it's also not easy. low priority now as we don't have a lot to test, but need to think about it when that gets bigger.

javiF: there is no WPT tests for this kind of features. perhaps using webdriver or other kind of framework could help to test the features, but so far it's under-tested. Some point that I have here, among the challenges we have need to find a UI for multitasking. Felipe has done good work on that but hopefully we can have improvements at the end of year.

???: possible to see some information?

juanjo: can you share docs?

felipe. not currently but can talk about it. currently wolvic UI positions things around a cylinder. you have 3 windows at the same distance and you cannot change any of this. one way to improve the UX is to make it more flexible; allow different distances, positions around you etc in a way that makes sense for the work you are doing. e.g. good to have a small window on the side like a chat, aware of it but not taking full place. so have this kind of flexible organizations, and easy for people to make sense of it. I have the intuition that with the right UI, virtual reality could be a very good way to browse the web: have a lot of flexibility to organize your work and not constrained by the screen, can understand better etc would me more pleasant and more productive way to use a web browser. not a certainty but very interssting line of reaseachr;

????: my question too. even looking at the demo, it looks like a 2D space is not taking full advantage of things. what about making web content a more 3D way?

felipe: we are limited by what's the web content itself can provide. tools like webxr can provide more features like the things I demoed (immersive video etc...)

german: for now only discussed web content. did we explore changing the UI itself to rely more on 3D?

felipe: one way it could work could be for the navigation page to page, like showing the history in a different space. or the 3-style tabs extension firefox providing a new visualization to organize things. having something organized automatically without the user having to do it themselves.

felipe: brian asked if you were talking about the reader mode

german: can be in that case too. basically presenting things in different way. but one issue could be information overload.

brian: reader mode is an opt-in kind of things. you don't mess with the content of normal web page. intersting to explore. could we make web pages more compelling to explore?

juanjo: could be an extension of what we are doing. our focus is more on the web content. we could do for the all menus we have could be a research field to explore.

german: challenge to make a new backend? to make it work?

juanjo: shin can explain? we need to do a lot of work on the wolvic side, almost a port of chromium. also take care of performance. somethings considering on weblayers which has no promises to be kept in chromium, so risky. for firefox, it was bit easier, since Firefox for android already had some stuff.

zahkar: current state of the chromium backend, we can open multiple tabs, show webgl content etc good perf but not perfect. some things are working some others not. in term of challenges, redesign our multibackends architecture as current one (based on gecko) does not allow for performance improvements.

???: I could imagine a whole bunch of interesting stuff you could do in 3D, using an existing web rendeirng engines limit you. have you any idea about long term things? it's a cool testbed for 3D interact...

felipe: our role is to support webXR to let web sites experiment approaches and provide ideas.

javiF: it's yet to see what the web content in 3D will be in the next year. we have to admit it is still at early stage. but we don't know, for now provide the spec and let's see what the authors would do with it.

tiago: any hints of what content is used the most?

javiF: no idea personally.

juanjo: what are the main use cases. we have regular browsing or videos. but gaming and more advanced is new. many areas are planning to migrate to webxr, but things are really early. they show the potential but that's the beginning of something. another use case is education, simulating the kind of knowledge you have to get etc it's getting quite a lot of importance. everything early stage.

juanjo: brian said ??? is very popular. There is a question here: how much scope is actually shared between backends? what is not the web engine is not changing at all. quite a lot code is shared.

zahkar: probably 90%.

dape: I'm checking some stats... diff is pretty small for chromium vs edge.

javiF: one topic I want to bring and related to pres apple did this week. they mentioned CSS features interesting for 3D. and also accessibility which is a hot topic in AR, currently akwoledgemnt it's important but no clear solution yet.

brian: how many of people on this room own a device?

juanjo: 3 people raised hand.

brian: do you agree that when you experiment one it's wouah this is going to be something. anyone want to talk about that?

javiF: agree. really skeptical at the beginning. after trying I was impressed. I wanted to say that we have the responsibility or the challenge to make the web there. currently lot of native application, web need to find a place there given it's importance. we need to make an effort to help with that.

mario: I have headset. this kind of device are very hard to perceive how they work. you can see videos, how it works. but trying is different. I recommend people to try. I tried the web experience and was quite surprise as they are many existing web content, not as impressive as the native ones, but still.

juanjo: according to public estimation, meta has a lot of usages, but it's not very constant. challenge is that hardware is interesting, potential there, but things of do is still limited. currently appstores not full of things yet, but can already try. wolvic is helping people to test immediately. some devices don't have a browser yet, so wolvic could help.

alex: some people are very skepitcal....

juanjo: we would need a whole hackfest to discuss to make sure everybody is very comfortable with the device. it's an open debate. true that today heavy, unconfortable etc

brian: it's okay to have different tools to accomodate needs of different people. things are improving over time. first game I played was not so pleasant to me. its getting better.

tiago: there are some experiences that don't require the display. e.g. medication stuff? space & audio already gives pretty good things.

mario: it's not many years apart that things get much better. the great improvement was to remove connection to a separate device. now you can just put the glass and everything is there. my experience after trying the latest ones was ouah. not sure how much improvement there will be in the future.

Gyuyoung: main benefit of multiple backends.

juanjo: multiple angles. 1 angle: maintain diversity in the web platform, not pushing in one direction. 2 angle, we still need to see how chromium would be. We try to be a bit more open. ALso understanding that some users of wolvic are willing to not use chromium only, so let's see how far we can get.

zahkar: chromium shows promising performance. the other new backends have only been experiments, so we will see.

tiago: video perform is it in wolvic or engine?

zahkar: engine.

javiF: multimedia pipelinee provided by gecko and would be the same for chromium (blink).

rego: thank you everyone. we are done.

clapping