Adding package to Xcode project.

[bhartsb]

First thank you for this great project!

I've created a MacOS command line tool app in Xcode and attempting to add the swift-mujoco package via Xcode:

After doing this and attempting to build I'm seeing:

My longer term hope is to be able to show mujoco in a MTKView or SCNView. I read that NSOpenGLView is deprecated, but maybe it could be used as an interim step to porting Mujoco's OpenGL code.

[liuliu]

You need to install glfw locally for the GLContext object dependency.

I would also recommend you first to try the swift build --target simulate to see if it works locally before adding as a target to Xcode.

To install glfw on macOS, brew install glfw

If you want to replace with NSOpenGLView, you basically need to write your own GLContext, at that point, I recommend fork / edit and then put up a PR to introduce the new GLXXXContext back.

[bhartsb]

I had previously installed glfw 3.3.8.
I can build simulate. If I try to run it in Terminal I get:
main.swift:3:10: error: no such module 'MuJoCo'
Probably not related to the linking error.
I'm not clear on the steps to adding simulate as a target.
Simulate is added as a framework when I added the MujoCo package:

[liuliu]

simulate and codegen are not frameworks, but a binary target (one is the same as samples/simulate.cc from main MuJoCo repo, the other is the code generator).

Can you copy the full command line errors? Make sure you run the command from swift-mujoco repo:

git clone https://github.com/liuliu/swift-mujoco.git
cd swift-mujoco
swift build --target simulate

[bhartsb]

Yes I see .o files and swift modules. Here is the output from building simulate. Warnings not errors. The only error is the no such module.
buildMujoSwiftOutput.txt

[liuliu]

If

swift build --target simulate

works. You should be able to find the binary under .build/debug/simulate. Alternatively, you should be able to run the simulate app by:

swift run simulate

If all these works, then the library itself is good on your macOS. There are probably just some setup issues on the Xcode end if it fails for your project.

I am trying to teething out your macOS dependency issues v.s. the Xcode project setup issues. If swift run simulate shows the simulate app window properly, we can rule out former and focus on what happens to the latter.

[bhartsb]

So swift run simulate works, but there are some issues that the UI's left panel is very small. Resizing GLContext from 1280X720 to 1080p did not help as the panel doesn't get bigger. I encountered multiple issues so I will give the screenshots to illustrate:

1080p left panel is still too small since it doesn't resize:

Dragging to resize causes display issues:

fullscreen looks good, but can't exit fullscreen:

After killing simulate the desktop drawing is broken. Just showing the top left corner of my desktop in this screenshot:

I'm starting to get a slightly better feel for how the project is organized.
I hadn't initially realized that the simulate target main.swift file in /Examples/simulate was specified in Packages.swift.
Also I'm not familiar with Bazel, only a tiny familiar with SPM, so figuring out what was happening with swift build --target simulate and swift run simulate also was a bit challenging. I didn't realize that the "simulate" unix executable wasn't created until the "swift run simulate" step. There are a lot of files generated in the .build folder. What helped me sort it out was to use FileMerge to diff between the cloned swift-mujoco folder, the folder post build, and the folder post run. Also looking at the swift symbolic links: ls -lsa /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/

I just added this extra information in case it helps someone else who is new to this. I'm sure that you are already aware of the symbolic linking to swift-frontend.

[liuliu]

Yeah. Bazel is not required to know if you don't develop swift-mujoco library yourself. SwiftPM is required and recommended way to introduce swift-mujoco as a dependency for your project.

As for the UI, try to drag & drop a model into it and see what happens (./Examples/assets/).

[bhartsb]

Tried the ant asset. It is not working. I compared it to the non swift Mujoco and definitely a lot of features aren't working. The Fullscreen and resizing as mentioned is broken. When in fullscreen swiping gets one out of FS as does touching some of the menus
. Which don't have anything to do with exiting Fullscreen so gestures are mixed up.

Per the above the skeleton is not recalculating skinning as it does in the C version of Mujoco.
I haven't really explored the C version of MujoCo yet so it is challenging to make comparisons.

Here is the C version with the ant dropped in:

[liuliu]

OK. This looks like simply the event loop is not running, hence no event processed. Could be a new issue with Xcode 14 (which I haven't tried) since we use some of Swift Concurrency primitives. What's your environment? (Which Xcode version, what kind of Mac (M1 / M2 / Intel?)).

[liuliu]

Just tried with Xcode 14.0.1, on a Intel-based Mac Mini, cannot reprod this issue.

[bhartsb]

MacOS BigSur 11.6.7
swift-driver version: 1.26.21 Apple Swift version 5.5.2 (swiftlang-1300.0.47.5 clang-1300.0.29.30)
Xcode Version 13.2.1
Hardware:
Model Name: MacBook Pro
Model Identifier: MacBookPro16,1
Processor Name: 8-Core Intel Core i9
Processor Speed: 2.3 GHz
Number of Processors: 1
Total Number of Cores: 8
L2 Cache (per Core): 256 KB
L3 Cache: 16 MB
Hyper-Threading Technology: Enabled
Memory: 64 GB

[liuliu]

Is it possible for you to upgrade? I don't plan to support 5.5 (5.6.x and above, yes) and it would be easier for me to verify (also BigSur is a bit dicey, as I am not sure how compatible some of the concurrency primitives are).

[bhartsb]

I'm not sure. I think I have a dependency related to Maya 2019. I will have to carefully think about it. I have another MacBook that I just got back but needs a complete OS reinstall so either I can try that route or do a careful upgrade. Either way it will take some time.

[bhartsb]

I upgraded to Monterey, Xcode 14 and Swift 5.7. Things are working better but there are still issues. It doesn't work with dual monitors or fullscreen mode. With dual monitors I had to disconnect my external one and then it looks okay on my laptop screen. If I have external monitor attached then the UI is not legible on either screen.

[liuliu]

Right. I didn't optimize for dual monitor or fullscreen. These should act the same as simulate.cc