iOS

August 29, 2019

Set up a basic ARKit project in Xcode

With this blog post, I will aim to explain how ARKit works and how it should be used. I will also show a practical example of how to set it up.

As with many Apple frameworks, ARKit makes use of the delegation pattern which I have previously written about here.

The ViewController

Let’s begin by setting up the basic view controller. The first step is to replace the view controller’s view with the ARKit view. ARKit provides a class ‘ARSCNView’ which should be used.

NOTE: Usually, an AR app’s view takes up the entire content on the screen. It is also common that an AR app starts out in the AR Scene right away.

Let’s start with modifying the ViewController created when setting up a single view project to look like the following.

1
2
3
4
5
6
7
8
9
10
import ARKit

class ViewController: UIViewController {
  var sceneView = ARSCNView()

  override func viewDidLoad() {
    super.viewDidLoad()
    view = sceneView // Replace the current view with ARSCNView
  }
}

The sceneView needs a scene as well. This scene can be created as a regular SceneKit scene in the assets folder and loaded in the code like this:

1
2
let scene = SCNScene(named: "art.scnassets/pingpong.scn")!
sceneView.scene = scene

Note that we force-unwrap the SCNScene. That is because we will always expect the scene to exist. If, for some reason, the scene doesn’t exist we want the application to crash so that we are aware of the issue and can fix it.

If you want to have some initial models loaded onto your scene, the scene file is a great place to put them. There you can visually edit the file in 3D.

If you do that, keep in mind that the origin will be the device’s position when it is loaded. That will affect where the models are visible. If you instead wish to place the models on a flat surface it’s better to load them while detecting a plane (described further down below).

Setting up the configuration

Before we can start the scene’s session, ARSession needs a configuration to know what it should track. For that there is the ARConfiguration and its subclasses. Apple defines this configuration object as:

An object that defines the particular ARKit features you enable in your session at a given time.

There are a few options for the configuration that is set up with ARKit. Apple also provide a list for all the options in the documentation. Basically there are the following options:

  • World Tracking (Most common)
  • Orientation Tracking
  • Face Tracking
  • Image Tracking
  • Object Scanning

The most common ones to use are the world tracking or the face tracking. But for some cases the other ones might be more suited.

The object scanning configuration is used for scanning a real object that then later can be detected and tracked in the world tracking configuration. Apple has a great demo project on it here.

With ARKit 3 introduced there is also BodyTracking.

Read more about the different configurations here.

Starting the Session

Now that the scene is loaded we need to start the session. We will do this in viewWillAppear. It is equally important to remember to pause the scene whenever the view disapears. So every time the ARScene is off-screen we don’t waste any processing power, and whenever it comes back we start up again. Be careful not to put the configuration setup in viewDidLoad as it will only be called once. If you do that, your ARSession will not run again after being paused.

1
2
3
4
5
6
7
8
9
  override func viewWillAppear() {
    let configuration = ARWorldTrackingConfiguration()
    sceneView.session.run(configuration)
  }

  override func viewWillDisapear() {
    sceneView.session.pause()
  }
}

Detecting planes

When ARKit detects things such as planes, 2D images or objects it will call the session delegate. A common way to catch those events is to make the ViewController conform to the ARSessionDelegate protocol.

1
2
3
4
5
6
7
8
9
extension ViewController: ARSessionDelegate {
  func session(_ session: ARSession, didAdd anchors: [ARAnchor]) {
    for anchor in anchors {
      if let planeAnchor = anchor as? ARPlaneAnchor {
        // Do something when a plane anchor is added
      }
    }
  }
}

(In the example, I’m looking for plane anchors, but it is also possible to look for other types of anchors such as images or objects. We will implement that in the coming sections.)

This lets you take action whenever the AR session detects planes, objects, images or faces. However, if you want to add some geometry (which is quite common in an AR app) it is better to access the added anchors through the ARSCNViewDelegate.

This is what Apple says in their documentation:

If you display an AR experience using SceneKit or SpriteKit, you can instead implement one of the following methods instead to track not only the addition of anchors to the session but also how to add SceneKit or SpriteKit content to the corresponding scene.

Below that text they are referring to methods in the ARSCNViewDelegate.

Let’s implement that instead.

1
2
3
4
5
6
7
8
9
10
11
12
extension ViewController: ARSCNViewDelegate {
  func renderer(_ renderer: SCNSceneRenderer, didAdd node: SCNNode, for anchor: ARAnchor) {
    if let planeAnchor = anchor as? ARPlaneAnchor {
      // Add a node on a detected plane
      let customerNode = SCNNode()

      // Your custom configuration

      node.addChildNode(customNode)
    }
  }
}

Detecting images

To start detecting images, we need to inform ARKit of what images to look for. To do that, we set the configuration’s ‘detectionImages’. It can be set by modifying the code in viewDidAppear where the ARConfiguration is setup.

1
2
3
4
5
6
7
8
9
override func viewWillAppear() {
  let configuration = ARWorldTrackingConfiguration()
  guard let referenceImages =
  ARReferenceImage.referenceImages(inGroupNamed: "DetectionImages", bundle: nil) else {
    fatalError("Group DetectionImages does not exist.")
  }
  configuration.detectionImages = referenceImages
  sceneView.session.run(configuration)
}

Now we just need to create a group with the name ‘DetectionImages’ and put some images there and ARKit will do the rest for us.

This is what it looks like in Xcode when an AR Reference Image is added to an AR Reference Group. Don’t forget to set the physical sizes for the image.

The AR session will automatically start to identify the images in the group. An ARImageAnchor will be added to the session every time an image is identified. Just like with planes, we can capture that in the delegate and perhaps add our own nodes on that anchor.

1
2
3
4
5
6
7
func renderer(_ renderer: SCNSceneRenderer, didAdd node: SCNNode, for anchor: ARAnchor) {
  if let planeAnchor= anchor as? ARPlaneAnchor {
    // Add a node on a detected plane
  } else if let imageAnchor = anchor as? ARImageAnchor {
    // Code for adding a node on the image position
  }
}

Detecting objects

Setting up for detecting objects is almost exactly the same as for images. Add a new AR Resource Group and call it “DetectionObjects”. Instead of AR Reference Image, add a AR Reference Object.

The easiest way to create AR Reference Objects is to install Apple’s demo project as mentioned earlier.

Now we can magically start to detect real objects in the scene with our renderer function.

1
2
3
4
5
6
7
8
9
func renderer(_ renderer: SCNSceneRenderer, didAdd node: SCNNode, for anchor: ARAnchor) {
  if let planeAnchor= anchor as? ARPlaneAnchor {
    // Add a sphere node on a detected plane
  } else if let imageAnchor = anchor as? ARImageAnchor {
    // Code for adding a node on the image position
  } else if let objectAnchor = anchor as? ARObjectAnchor {
    // Code for adding a node on the objects position
  }
}

Adding an object without an anchor

The last thing I want to talk about is appending a node that is not anchored to any real world object. For example, suppose that you would like to add some floating text one meter directly in front of the camera. The easiest way to do that is to add a node as a child node directly to the scene’s root node.

Here is a quick example of how to accomplish that.

1
2
3
4
5
6
7
if let camera = sceneView.pointOfView {
  let localPosition = SCNVector3(x: 0, y: 0, z: -0.5) // Notice the minus sign
  let scenePosition = camera.convertPosition(localPosition, to: nil) // nil means that we are converting the position to world space
  let node = makeText("Hello world!")

  sceneView.scene.rootNode.addChildNode(node)
}

The makeText function could look something like this.

1
2
3
4
5
6
7
8
9
func makeText(text: String) -> SCNNode {
  let geometry = SCNText(string: text, extrusionDepth: 5)
  let textNode = SCNNode(geometry: geometry)
  textNode.scale = SCNVector3(x: 0.005, y: 0.005, z: 0.005)
       
  let node = SCNNode()
  node.addChildNode(textNode)
  return node
}

With the above code you can add nodes to your scene at any point in time. It doesn’t require ARKit to detect something first. In the above case, the node is not anchored to anything. However, if possible, it should be preferred to add anchors to the session instead, and from the delegate append any nodes you may want.

You may also like...

Leave a Reply

Your email address will not be published. Required fields are marked *