Onshape Support? Or More Generally Browser Support

[misterjtc]

Are there are plans to update spacenavd to support Onshape or other browser-based CAD software. I see that there are a number of posts related to 3Dconnexion support on Onshapes forums.

That said, I thought I would post about it here.

Thank you in advance for developing this software, I am currently using it and it works great! Just looking for support for browser-based systems as I am currently using Onshape.

[jtsiomb]

I've been asked about this a lot lately, which is why I've recently added a FAQ item about it: http://spacenav.sourceforge.net/faq.html#faq15

In short: no, 3dconnexion cooked up a new API based on websockets to serve 6dof input to onshape, which is completely different for obvious reasons, from any previous protocols designed for native applications, and spacenavd does not support it.

I'm open to integrating support for it in spacenavd, but since I'm not at all familiar with web protocols, nor at all motivated to familiarize myself with such, someone else needs to step up and do most of the work, and I can help with integration. Until that happens, I'm afraid it's very unlikely that onshape will be supported by spacenavd.

Marking this as a feature request.

[pkucmus]

@jtsiomb I would like to help. I can only do Python and I did some initial probing. I'd like to run a Python server that would receive events from your driver (possible?) and push it into a websocket that Onshape seems to be listening to for the official Windows driver.
It seems that they are serving a http server on https://127.51.68.120:8181 (still a 127.x.x.x loopback) which on a request responds with a wss port. That wss server then emits events from the device.

It would not be a "general browser" solution as such thing would not exist. It's dependent on the app and from what I can gather Onshape is leveraging 3dconnexion.js that tries to listen to the local Websocket server.

More or less I see it like this:

[jtsiomb]

Sure, I guess ti makes more sense to have this as a separate service, rather than putting a tiny web server into spacenavd. So it might as well be written in python, or whatever else makes it easy. I'd prefer a native C version just to avoid any unnecessary runtime dependencies, but it's more important to get something working first, since a lot of people have expressed interest for using 6dof devices with onshape.

I expect it should be easy for a python program to connect to the spacenavd UNIX domain socket and receive 6dof events that way. Even though I'm not very familiar with python, I'm sure I can cook up some rough example python code for how to talk to spacenavd if that would make it easier for you to get started. Let me know.

Btw I posted a question about the onshape API to the 3dconnexion forums a few months ago, and they pointed me towards examples in the 3dconnexion SDK. They said they provide sample javascript programs and documentation in their windows SDK under the web directory: https://forum.3dconnexion.com/viewtopic.php?f=21&t=39679 It seemed like a reasonable starting point for figuring out the API we need to reproduce.

[pkucmus]

Even if my service will end up being just a Proof Of Concept I'm still eager to do that. Will dockerize it so it's going to be one command to run (or will be "deamonized" with --restart=unless-stopped which would require no manual execution from the user).

UNIX sockets were my first guess as well, I don't have much experience with those but I'm sure we can figure something out.
I saw that post, thank you, from there I already got the files and was able to poke around them. But those seem incomplete (I'll try to figure it out). I also asked Onshape for guidance since it seems that they are not even trying to get a hold of a websocket connection on Linux.
I'm riding on an anger wave right now. Got the device yesterday only to learn that there's yet another company making rubbish claims like "there's not enough 'professional' users on Linux for us to support that", well... let's change that :)

[jtsiomb]

My only request if it's going to be included with the spacenav project, is to keep it simple where possible. I don't know what dockerization entails, but I'd like the option to be able to just get the python program and run it normally without having to get some sort of special bundle with a million depdencies built-in to it. This means that I'd also like to avoid unnecessary dependencies where possible, so that instructions can be "use your package manager or pip or whatever to install 1-2 python libraries at most, chmod +x it, and run it".

Hopefully that's not too much trouble. If it is, I guess do what you feel is best for the prototype, and we'll see how to simplify it for distribution from there.

[pkucmus]

I think it does not have to be included in the spacenav project (although it would be cool if it made it there).
Treat the Python version as a proof of concept (Python deps will always be hard to deal with), but I already heard from a buddy "we could then rewrite it to Rust and make binaries". That said I should also be able to keep it all within:

pip3 install {the_name_for_the_package}
python3 -m {the_name_for_the_package} start

That said if you were to make a well defined "API" of what would be written to the UNIX socket then people could come and implement all sorts of nice things - even outside of our "Onshape spacenav service". I mean someone could easily come in and write something better than I will in Python. The key is that we document what we'll have figured out.

Am I making sense?

1. How would this work on your end?
2. Would spnavcfg get a input field to specify a socket file for services to read from?
3. What would be written to the socket?

I still need to reverse engineer the webhook API between Onshape and the local server Windows users get from the official driver - this is going to be the majority of the work.

[jtsiomb]

Rust is definitely not acceptable for inclusion :) In any case as I said, prototype any way you like, we'll figure the rest when we have something working.

When I mentioned UNIX domain sockets previously, I wasn't talking about a new interface (we could add a new interface if it's necessary to make this work, but I can't imagine why it should), I was talking about the currently implemented method spacenavd uses to talk to clients. It's a very simple mechanism: the client connects the spacenavd socket (/var/run/spnav.sock) and spacenavd feeds it button and motion events. The client side of this is implemented in libspnav, which is a simple C library native clients use to talk to spacenavd, but I assume for a python program it would be simpler to just rewrite that bit in python.

You can use the libspnav code as a reference for the structure layout of the events, or as I said previously, if you prefer I could try and hack a python version of the client code for you to use as a starting point.

[pkucmus]

Oh I did not get the part that there's a socket already. Awesome. I'll start poking around.

[pkucmus]

I pushed a simple web app here (for those reading in the future - this is just a proof of concept): https://github.com/pkucmus/spacenav-ws which means we are controlling a browser app with spacenav now.

Now I need to figure out how to make Onshape connect to this websocket and what payloads it needs.

[pkucmus]

The websocket payloads are complex and would require math that is most likely beyond me. Here's an example of a view.affine event:

[
    8,
    "3dconnexion:3dcontroller/3566065573492",
    [
        2,
        "vhj1WFOb1aGDXI7d",
        "self:update",
        "",
        "view.affine",
        [
            0.9271885408792452,
            -0.3671419119088283,
            0.07435204221907192,
            0,
            0.08448485590856766,
            0.3983266795284326,
            0.9133444944258232,
            0,
            -0.3649434460081796,
            -0.8405609275333846,
            0.4003418643108554,
            0,
            0.08627579426964002,
            -0.2486249618828344,
            0.1137656502994606,
            1
        ]
    ]
]

It's some kind of 4x4 matrix of where the camera is to be. How to calculate that out of movement reads from the driver is going to be difficult to me. Any idea on how to approach that?

[jtsiomb]

That's pretty standard 3D math. 6dof devices provide translation (Tx, Ty, Tz) and rotation (Rx, Ry, Rz) input. The best way to accumulate rotations is to convert them to an axis-angle form, and use that to rotate a quaternion representing the "current" orientation. The orientation quaternion can be converted to a 3x3 rotation matrix, which expanded to 4x4 can be then concatenated with a translation matrix derived from the current "position" vector which accumulates the translation events. The result is a combined matrix specifying both the position and the orientation of the manipulated object in 3D space. You can see an example of this in the "cube" sample program that comes with libspnav.

The "payload" you quoted is most likely meant to be used for a viewing transformation, so I assume it's going to be the inverse of that matrix.

If you're having trouble, make a list of all the math the 3D connexion web interface needs to handle, and I'll take a stab at writing some example code for how to implement them. It seems they defined a higher level interface for web apps, than their regular native raw 6dof input their usual APIs provide.

If the higher level API is much more extensive than just calculating a couple of matrices, maybe it makes sense for me to see if I can add that functionality in spacenavd, rather than implement them in the python code that handles the websocket communication. But again let's prototype first, see what's needed, and I can decide later on that.

[MarcelRobitaille]

I'm riding on an anger wave right now. Got the device yesterday only to learn that there's yet another company making rubbish claims like "there's not enough 'professional' users on Linux for us to support that", well... let's change that :)

I got mine today and I feel the same way. Let me know if there's something I can do to help, I have a fair bit of websocket experience.

[pkucmus]

I was trying to make it work with no success but I learned a few things:

1. What is going to be a pain to setup for an end user is a self-signed cert for the WSS and HTTPS service - I suspect the Windows driver adds their self-signed cert to the installed browsers upon installation (if forces you to kill Firefox while installing driver - I guess it's then). This has no workaround since no one will issue a certificate for an IP address and the first call to https://127.51.68.120:8181/3dconnexion/nlproxy is hardcoded. One will have to trust a cert provided by us or make one on his own.
2. The 3dconnexion.js client uses some (I think) archaic autobahn.js version to handle the transport and the WAMP Websocket sub protocol. WAMP is probably what is producing all that meta junk around the 4x4 matrix in the example above .
3. There's Autobahn libraries for different languages: https://crossbar.io/autobahn/ (Python, C, C++, JS, Java).
4. One can use https://3dconnexion.com/technical_support/web_threejs.html for testing. It makes the call to the https://127.51.68.120:8181/3dconnexion/nlproxy service to get {"port": PORT_INT} where PORT_INT is the port on 127.51.68.120 that listens for WSS.
5. I'm attaching 2 files examples.zip that will show the exchange that is happening on the beginning - right after you load the page and in the other file what happened once I tapped the space mouse top to bottom - there's a lot of back and forth between the 3d view and the websocket server, I'm guessing that in Onshape's example, Onshape is informing the server about the selected part around which the pivot should happen (for the server to do the math?).

As you can most likely tell I'm guessing a lot in the above so further investigation maybe needed.
This seems to be the "the 3Dconnexion" standard of talking with web software. So there is potential that once implemented it'll work with not only Onshape (although I have no idea what else there is).

I would also like to explore the idea of not following the standard (and just making it simpler...) which could mean emulating the mouse in Onshape's viewport but in both cases be may hit a wall without help from someone from Onshape.

[jtsiomb]

As long as we provide a documented way of how to make it work, I don't think we need to care, at the moment, if there are extra steps required about adding certificates to the system or whatever. If we can streamline it further down the road, sure, but I don't think right now we need be concerned with setup ease.

Emulating mouse input is not worth the effort in my opinion. I don't consider anything less than correct 6dof input worthwhile.

For the web stuff I can't help, coordinate with @MarcelRobitaille who offered to help with that part if you can. But as I said previously, if you reach a point where you can communicate with onshape, and you're missing the necessary math algorithms, give me a list of what you need and I'll try and provide example implementations (in C) or add them as a secondary higher level API to spacenavd.

Also if I may offer a suggestion, github is nice for bug reports, but really bad for maintaining extensive conversations about how to implement things. I would ask both of you guys to subscribe to the spacenav-devel mailing list, where it will be easier to coordinate and discuss how to implement this. https://sourceforge.net/projects/spacenav/lists/spacenav-devel

[misterjtc]

I agree with @jtsiomb and I am also a big supporter of docker and use it extensively. That said, having to run a container to connect spavenavd to Onshape seems pretty reasonable.

I have subscribed to the mailing list and look forward to following this. I don't have any experience in C or python, but I am happy to support with testing or documentation. Don't hesitate to ask.

[MarcelRobitaille]

Thanks @pkucmus for the very detailed reply. Regarding the self-signed cert, could letsencrypt help us there? With letsencrypt / certbot, we could generate a certificate that browsers will trust. I agree with @jtsiomb though, I'd like to get it working so I can start playing with it 😉.

@jtsiomb Thanks for the suggestion, I have joined the mailing list.

Do we have a copy of 3dconnexion.js? I can't find it in the network tab or the debugger tab in devtools on onshape.

[pkucmus]

@jtsiomb I subscribed but it's the first time I'm using a mailing list (am I too young?) and have no idea how to use it (sorry for being a problem).

Not to leave a thread with an unanswered question: @MarcelRobitaille No one will issue a certificate for a bare IP, Letsencrypt included (I tried). I'll prepare a docker compose for us to work on, later we'll worry about setup ease (per @jtsiomb comment).
You can sign up for a SDK kit on 3dconnexion's site. You will see nothing in that regards on the network tab in Onshape from Linux (they are not even executing the code on Linux, changing user-agent does not help :|)

[MarcelRobitaille]

No one will issue a certificate for a bare IP

Ah true. I didn't think of that

[jtsiomb]

@pkucmus You just send emails to [email protected], and your email goes to everyone who is subscribed to the list. Then people can reply, and you receive their responses, and so on. It helps if you use some sort of filter in your email client that sends emails with "To: [email protected]" to a dedicated mailbox, so they don't get mixed up with your regular emails.

Also make sure to follow some basic rules of email nettiquette:

• send text emails, not html, and not multipart emails if at all possible. Only text.
• do not send large attachments or binary files
• do not "top-post", i.e. when you reply to some other email, include the relevant part quoted with '>' and then write your reply after that quote. Keep the attribution lines intact so it's obvious who said what. And also do not just quote the whole email, only the relevant parts.

And @MarcelRobitaille make sure you always include a subject line, don't leave it empty please.

[Pace17881]

Hi,

Are there any news on that topic? I'am using spacenavd with my "SpaceMouse Wireless" on Debian bullseye. It works without problems with freecad. But this is not my preferred tool for cad. At the moment I love to work with "Onshape". Because it is easy to handle with it.

Is there any chance to get it working with browser based cad tools?

Thanks a lot for your current implementation. Would be nicer to use it with browsers ;)

bg

Sebastian.

[TechplexEngineer]

In the event that others end up in this thread like I did, there is some discussion on the mailing list about a working proof of concept: https://sourceforge.net/p/spacenav/mailman/spacenav-devel/?viewmonth=202101#msg37204049

[Crono141]

The last message on this issue in the mailing list was in January 2021. Has this project died?

[jtsiomb]

I assume so, I haven't heard from them since then.

[VolkerLieber]

I'm currently looking into developing the ws server for communicating with onshape. The communication is done via the wamp1 ws subprotocol and isn't a problem.
But since it's required to calculate the viewmatrix based on the current values fetched from the client, my question would be if you have looked into 3DconnexionJS.pdf from the sdk and have implemented the needed functionality already or if that is something that would need to be done?

[jtsiomb]

Current libspnav (>= 1.0) provides utility functions for constructing a view matrix out of the stream of spnav input events.

Search for "posrot" in the spnav.h header file, and also take a look at the fly and cube examples which demonstrate how to use this new API for either manipulating an object (cube), or the camera (fly). Both spnav.h and the example programs are heavily commented to explain the usage, but feel free to ask followup questions if you need to.

If there's something else needed for the ws server let me know, and we can decide how to go about implementing it.

[baranner]

Is there any update on this topic or has anyone else started working on it?
I have a SpaceMouse compact and would love to use it with OnShape in Firefox.
I have a little experience in python, maybe there is something i could do to contribute?

[VolkerLieber]

I've encountered some problems with the 3D calculations.@jtsiomb kindly offered his help, but we both have relatively full calendars, so it might take a few more weeks before I can give you another update.

[jtsiomb]

Yeah sorry about the delay, I didn't get around to it yet, but I have it noted down right in front of me. I'll get back to you soon.

[timblakely]

Picked this up to see if I could actually get it working with the 3dconnexion demo app.

Current status:

1. Implemented WAMP v1 protocol
2. Bi-directional communication between reverse-proxied python server and web client
3. The demo app kind of works; rotations work and translation in XY works, but it seems to use extents for Z (?) which I haven't implemented.
4. I can get OnShape to connect to the server and do some (very) basic motions, assuming I apply the navigator.__defineGetter__('platform', function(){ return 'Windows' }); during page load.

My 3D graphics knowledge is about two decades rusty :/. I don't have the cycles to look into exactly how to set up the navigation library to mimic the original one at the moment, and unfortunately I can't seem to find any python wrappers around libspnav (know of any? 😅). Hoping this may help someone else with more 3D graphics knowledge in their brain-cache :)

[jtsiomb]

Nice to see this moving again. @timblakely maybe you could coordinate with @VolkerLieber who is also working on this issue.

I've sent him a C example program the other day, with the kinds of algorithms needed to implement some of the navlib functionality. I've attached it here in case it helps: navlibtest.tar.gz. Unfortunately I didn't have the time to make it exactly right, there are missing things which I intended to tackle another time, but haven't done so yet. Here's some relevant fragments of my email to @VolkerLieber for context:

I've attached a first approximation to the
necessary calculations, but it's not done or entirely correct yet. I
hope it will serve as a starting point to get something working until
I can find the time to do a second pass.

The example code I'm working on (see attached navlibtest.tar.gz) is
written as a standalone C OpenGL program, in two conceptual parts:
main.c plays the role of the "client" program, and nav.c plays the role
of the navlib library which responds to input and does all the math.

The "communication" between the two is initiated by the client
calling start_frame, which triggers the nav module to query the current
state (view.affine, selection.affine, etc), and use any input received
since the last frame to calculate the new state and update them. If no
input was received since the last frame, motion_pending is false, which
is equivalent to sending back "motion: false" and not updating anything.

I made it cycle through different navigation modes by pressing space.
I'm pretty sure the "object" mode corresponds to the "selection.affine"
property, and the view manipulation modes (I have "fly" and "target")
correspond to the "view.affine" property, but I'm not entirely sure yet
how the client signals which mode to use, I'll have to go through the
3dconnexion docs and examples again to find out. Target is unimplemented
in my code for now, fly manipulates the view, and obj manipulates an
object (the teapot) pretending to be the current selection.

See the comment blocks I added in nav.c for details on what I'm
currently doing, and what remains to be done or doesn't work entirely
correctly at the moment.

[timblakely]

Sounds good; I'll poke around when I have time (a bit cycle-starved at the moment) but I'll see what I can do. Ideally I'd put some pybind11 wrappers around the navlib code... 🤔

Quick question: I've noticed in the OnShape back and forth over the websocket that there's some image info that comes through, which at least on Windows appears to be intended to be displayed on the SpaceMouse Enterprise screen in a CAD-context-dependent way. Haven't had a chance to dig into spacenavd's source at more than a passing glance; is writing to the screen's image supported in some way? Just trying to determine if I should spend the time trying to grok those messages vs just dropping them if there isn't any support for it in the first place. :)

[jtsiomb]

The enterprise display isn't supported currently. Feel free to just drop them.

[Pace17881]

Hi, what's the current state?

[edutchie]

I don't have the skills necessary to figure this out, but I would like to support this in any way I can as I would see great benefit from OnShape compatibility. Is there anything I can do, learn, or give to further this side project?

[OmegaRogue]

Whats the current state of this for non onshape browser support with a custom web app?

[jtsiomb]

What you see in this thread is the current state.

[robots]

@timblakely Your demo doesn't work anymore, it seems they updated their web demo (and API?)

For rest of the world:
The spacenav_ws can be executed in userspace (without docker or nginx) by adding https certificated and changing port (main.py file):

uvicorn.run(
"spacenav_ws.main:app",
host="0.0.0.0",
port=8181,
reload=True,
log_level="debug",
ssl_keyfile="certs/cert.key",
ssl_certfile="certs/cert.crt"
)

You still need to generate new self-signed certificate and have firefox accept it.

Generate certificate:
openssl genrsa -out cert.key 2048
openssl req -new -key cert.key -out cert.csr
openssl x509 -req -days 3650 -in cert.csr -signkey cert.key -out cert.crt

Firefox:
Preferences -> Privacy & Security -> View Certificates, choose Servers Tab and click Add Exception, Enter https://127.51.68.120:8181 and get certificate, confirm exception

Run the spacenav_ws like this from main repository directory:
PYTHONPATH="src" python src/spacenav_ws/main.py
(you need to install python-fastapi and uvicorn)

Have fun :)

[grapemix]

Just somethings to consider:

1. If we go to let's encrypt route, the cert will eventually expired and we have to rotate it. A legit homelab should have mechanism to auto-rotate the cert. But some users might feel frustrate for the process.
2. For the self signed cert route, we will have to click the exception button every time, but it is more newbie setup friendly.
3. I don't know about the detail of implementation, but we can turn off the SSL on some protocols during testing. I've tried GRPC, S3/minio/ceph and probably wss too (long times ago). Can we downgrade wss to ws or https to http? Lots of lib has a flag for the SSL. If the traffic is only within a desktop's localhost, I can't see much security threats.

Anyway, great work!

[Arbel-arad]

@grapemix have you tried installing your root CA (the one used for local signing) in the browser?
it would remove the warning without requiring a registered domain.

[grapemix]

I think I tried installing the root CA for my firefox once and success. Then, I can't install the cert afterwards while I rebuilding my homelab multiple times. Finally, I totally rework my entire network and have a legit let's encrypted setup and auto-rotate mechanism. So no more self-signed cert for me.

Anyway, thanks for offering your help. @Arbel-arad

[ruevs]

For cross reference:
https://discourse.threejs.org/t/spacemouse-liberated-at-last/37146
https://forum.3dconnexion.com/viewtopic.php?p=110179#p110179

This uses WebHID and therefore does not solve the problem with replicating the 3Dconnexion websocket 6dof device interface, and thus would not help with Onshape. However it is a viable alternative for other web applications. At this moment WebHID is implemented only on Chromium based browsers.

[Arbel-arad]

For cross reference: https://discourse.threejs.org/t/spacemouse-liberated-at-last/37146 https://forum.3dconnexion.com/viewtopic.php?p=110179#p110179

This uses WebHID and therefore does not solve the problem with replicating the 3Dconnexion websocket 6dof device interface, and thus would not help with Onshape. However it is a viable alternative for other web applications. At this moment WebHID is implemented only on Chromium based browsers.

this doesn't work for me, requesting device access only lists the unifying receiver for my normal mouse.

[jtsiomb]

Let's not try to debug that unrelated project in this issue please.