From a tap on the screen to a point in the room

Span is an AR tape measure I built in Swift and Kotlin. You aim a crosshair at something, tap to drop a point, move, and tap again. Before Span, I had never thought about what that tap actually has to do.

The screen is two dimensional. The room is not. A tap gives you an x and a y in points on the glass, and a measurement needs an x, y and z in metres in the real world. Everything else in the app depends on getting that step right.

Casting a ray

The phone knows where its camera is and which way it faces, because ARKit and ARCore track that all the time. So a point on the screen can be turned into a ray: a line that starts at the camera and goes out through that point into the room. The question becomes where that ray first hits a surface.

AR frameworks find surfaces as planes, like the floor, a table top or a wall. Span asks for a hit in a fixed order:

let targets: [ARRaycastQuery.Target] = [.existingPlaneGeometry, .estimatedPlane]
for target in targets {
    if let result = arView.raycast(from: point, allowing: target, alignment: .any).first {
        let column = result.worldTransform.columns.3
        return Vec3(column.x, column.y, column.z)
    }
}

Existing plane geometry is a surface ARKit has properly detected, with a real outline. It is the most accurate, but it takes a moment to appear. An estimated plane is ARKit’s quicker guess. It works sooner and on more surfaces, but it is less sure of itself. Trying the accurate one first and falling back to the guess gives the best of both.

The hit comes back as a transform, a 4 by 4 matrix that holds a position and a rotation. The position is the last column, which is why the code reads columns.3.

Snapping

Even with a good hit, a finger is not precise. If you are closing a shape, you want the last tap to land exactly on the first corner, not 1 cm next to it. If you are measuring along a table, you want the point on the table’s edge.

So every hit is checked against two things before it is used:

• Existing points. If the hit is within 2.5 cm of a point already placed, it becomes that point
• Plane edges. If it is within 2.5 cm of the outline of a detected surface, it moves onto the closest spot on that edge

2.5 cm is roughly the width of a finger on screen at normal measuring distance. Points win over edges, because closing a shape on its first corner matters more than landing on a plane boundary that happens to be close. When a snap happens, the app shows a small “Snapped to edge” label and gives a light haptic, so the person knows it was not their exact tap.

Finding the closest spot on an edge was a small piece of maths worth knowing. You project the point onto the line, then clamp the result so it stays between the two ends, because a plane outline is made of segments, not endless lines:

let direction = end - start
let t = simd_clamp(simd_dot(p - start, direction) / simd_length_squared(direction), 0, 1)
return start + direction * t

Points that move after you place them

The last thing I learned is that the world keeps shifting under you. As you walk around, ARKit learns the room better and corrects its idea of where things are. Each point in Span is an anchor, and ARKit can move an anchor after it was placed.

Span keeps both positions for every point: where it was placed, and where its anchor is now. Lengths follow the current position. The gap between the two is drift, and when any point has moved 1.5 cm or more, the app warns that the measurement may be off and suggests measuring again.

So a single tap goes through a ray, a surface, a snap and an anchor before it becomes a number on the screen. Each of those steps can be a little wrong, which is why the app also shows a ± range. That part has its own note.