SceneKit annotation view

[1ec5]

Let’s add an example of using SceneKit content in an annotation view. For example, we could use an SCNView as the user location annotation view to turn the user puck into a 3D car. We could also do the same in the iOS navigation SDK’s example application as well.

This blog post shows how to sync an SCNView with an MGLMapView, although it doesn’t associate the view with an annotation. This Apple sample project provides some inspiration for using a car model to represent the user. Freely licensed 3D car models in COLLADA format aren’t easy to find, but apparently the Unity Asset Store has models in FBX format that can be converted to COLLADA using Maya or another 3D application.

/ref mapbox/mapbox-navigation-ios#402 (comment)
/cc @mapbox/ios @mapbox/navigation-ios @david-rhodes @kronick

[1ec5]

Some more possible sources for models from @isiyu, some of which may be openly licensed:

https://www.blender-models.com/?s=car
https://www.cgtrader.com/free-3d-models?keywords=car
https://www.turbosquid.com/3d-model/car

However, this repository has a CC0 dedication; it might be difficult to find a public domain model. Might need to make one ourselves.

[1ec5]

NASA is a U.S. federal government agency, so all the models at NASA 3D Resources are in the public domain. Selecting space-themed models gives us an opportunity to demonstrate the concept of putting 3D onto the map without so literally evoking a car navigation use case. This crawler model is in OBJ format, while this Curiosity rover is in Blender format. The OBJ file can be imported directly into an asset catalog and loaded into an SCNScene in one line of code. The Blender file requires exporting COLLADA from Blender, which is a bit more involved.

That’s the good news. The bad news is that there isn’t really a direct way to treat an SCNNode itself as an element in the Core Graphics–backed view hierarchy. I had mistakenly assumed that there would be an SCNLayer on iOS as there is on macOS, which would’ve made it much easier to integrate. This SceneKit demonstration (source code) works by overlaying a transparent SCNView atop the entire map view, manually synchronizing the SCNView with the camera. That’s great, but at the same time it doesn’t really take advantage of MGLAnnotationView.

I did notice that SCNRenderer is specifically intended to render 3D content into a Metal or OpenGL context, which means it might be possible to use SceneKit content as a style layer via MGLOpenGLStyleLayer. Not only does MGLOpenGLStyleLayer have access to the render transform matrix for a more consistent tilting effect (mapbox/mapbox-gl-native#10498), but it also synchronizes with the map to avoid the panning lag described in mapbox/mapbox-gl-native#5489. As a style layer, the 3D content could also appear beneath other style layers containing labels and shields.

/cc @avi-c

[1ec5]

I did notice that SCNRenderer is specifically intended to render 3D content into a Metal or OpenGL context, which means it might be possible to use SceneKit content as a style layer via MGLOpenGLStyleLayer. Not only does MGLOpenGLStyleLayer have access to the render transform matrix for a more consistent tilting effect (mapbox/mapbox-gl-native#10498), but it also synchronizes with the map to avoid the panning lag described in mapbox/mapbox-gl-native#5489. As a style layer, the 3D content could also appear beneath other style layers containing labels and shields.

Sketched out what this integration could look like on the 1ec5-scenekit-111 branch.

Along the way, I had to work around mapbox/mapbox-gl-native#12944 and mapbox/mapbox-gl-native#12945.

[1024kilobyte]

I'm also surprised that I could not find an example, as it is recommended to use MGLAnnotationView for SceneKit:
https://docs.mapbox.com/ios/maps/overview/markers-and-annotations/

But there are no obvious properties or methods exposed by MGLAnnotationView to use it with SceneKit. Am I missing sth. ?

[1ec5]

But there are no obvious properties or methods exposed by MGLAnnotationView to use it with SceneKit. Am I missing sth. ?

That’s correct, it isn’t possible to use MGLAnnotationView directly with SceneKit, as I had originally thought. However, the approach described in #111 (comment) does work: create an MGLOpenGLStyleLayer that manages an SCNRenderer, which in turn is created using the map view’s context. Getting the transform right requires reverse-engineering a bit of the map SDK’s internal transform code, but there are similar examples for working with GL JS’s custom layers: mapbox/mapbox-gl-js/pull/7039. This gist demonstrates the approach with the macOS map SDK. However, for iOS, there are some additional considerations like ensuring the right frame rate.

[captainbarbosa]

Better examples/documentation of how to integrate SceneKit are better suited for this repo: https://github.com/mapbox/mapbox-scenekit.

[1ec5]

That repository is unrelated to the map SDK.