The Complete ARKit tutorial

So you’ve decided to take the plunge into the exciting world of ARKit. Welcome.

I will try to cover as many aspects of this popular and exciting framework as I can. We will cover a fun tabletop game, portals and more.

So for a moment let us rewind and start at the beginning.

Just what is AR?

Well, I’m glad you asked!

AR is short for Augmented reality. It’s a technology that will shake the digital world and create new ones in the process. With the advances in artificial intelligence, specifically machine learning. We are beginning to learn that both AR and AI can work together to produce an amazing new world. I can’t help but think about Bladerunner when envisioning the future AR.

How does it work?

Simply put, AR comes from overlaying virtual content on top of a live image feed from a camera. The result is an augmented (having made something greater) the world around us (reality) producing a mind-blowing experience.

For AR to be effective it’s vitally important that the position, rotation, and scale of the virtual content is updated constantly at a consistent rate using real-time motion data from the device as it moves through the real world.

What makes ARKit different?

ARKit is Apple’s baby. I say, baby because I believe we are still at the beginning of this revolutionary technology. It allows you to create an immersive experience with both 2D and 3D content with the built-in camera. What makes ARKit special is that it performs markerless tracking. So if you have an A9 chip or greater (from iPhone 6s, 6s plus, iPhone SE and iPads (2017 +)) then you already have access to this amazing bit of tech.

What can it do for me?

ARKit does all the heavy lifting for you. So with just a few lines of code, you can create a truly immersive AR experience

Tracking

ARKit tracks device movement in a real-time.  It uses motion sampling to automatically update the virtual content. It achieves this by moving a virtual camera’s point of view in the virtual scene that equal that of the device.

Scene understanding & interactions

ARKit creates and manages its own virtual map of the world it sees. It detects surfaces and responds to them instantly. Giving you an immense amount of control over the scene including how your virtual content interacts with it.

Because you are ARKit is embedded in Apple’s eco-system you can combine the use of ARKit with other great frameworks like CoreML, SiriKit, and even simple gestures. Therefore creating a truly unrestricted way to build your AR world.

Rendering integrations

ARKit doesn’t use its own graphics API. The framework only provides you with surface detection and tracking info. This allows you to combine it with Metal, SceneKit or Spritekit. You may even see it used with Unity or the Unreal engine.

Lighting

ARKit uses the devices video frame to estimate the lighting conditions within your current environment – in real time (WOW). This enables the automatic adjustments of brightness levels of your virtual content to match that of the conditions in the real world. We will learn more about this later when we take a look at how to use physical based materials.

Metric scaling system

ARKit uses a pretty accurate metric scaling system to calculate distance and sizes within your camera’s scene. It doesn’t matter what graphics API you are using. ARKit uses 1 unit per meter.

Any limitations?

Sure, like with most new technologies and frameworks there are a few issues with ARKit. That being said WWDC 2018 have greatly improved this framework with the arrival of ARKit 2.0. But for now be aware of the following:-

  • Surface detection takes time
  • There’s processing lag when the device is moved excessively
  • Low-light conditions will cause ARKit to struggle with scene detection (Have you ever tried to walk around in the dark?)
  • Smooth or shiny surfaces such as a mirror confuse it
  • Spooky happenings with your virtual content, where it may pass through real-world physical objects.

In my next post on this series, we will create our first basic ARKit application. Exciting 😀