Testing an AR app without AR

Span measures through the camera, using ARKit on iOS and ARCore on Android. Neither runs in the iPhone simulator, and the Android emulator on my Mac cannot run real AR either. Real AR needs a real phone.

That would mean every change to a screen, a button or a rule needed a phone in hand. Instead, the app is built so that only a small part of it knows AR exists.

One protocol, two versions

Everything the app needs from AR is described in one protocol, ARMeasuring: the camera view, where the crosshair hits, the tracking quality and the detected surface edges. There are two versions of it.

• RealARMeasuring uses ARKit, and only runs on a phone
• SimulatedARMeasuring draws a fake room at a fixed scale, 250 points to a metre, and dragging a finger moves the “phone”

The rest of the app cannot tell which one it has. A tap, a snap, a closed area and a saved project all run through the same code. Android has the same split.

This proves the app’s logic. It proves nothing about real AR accuracy or real AR behaviour, and that difference turned out to matter. A crash in the room scan only happened on a real iPhone, because the simulator’s fake scanner never ran Apple’s code. That bug has its own note.

Opening any screen directly

Some states are hard to reach on purpose, like tracking lost mid-measure, a measurement that drifted or AR not supported on this phone. To compare them with the design, the app accepts a launch argument naming the screen, -screen on iOS and an intent extra on Android. A script opens each one on a simulator the same size as the design and takes a screenshot. Debug screens use a database held only in memory, so they can never touch real saved work.

Bugs the fake room still found

• On Android, a 125 dp drag did not move 125 dp. Compose’s drag detector ignores the first few dp as “touch slop” before it decides a drag has started. That is right for a normal drag, but in the fake room every dp is a measurement. It now tracks the finger’s distance from where it first touched
• On Android, the forced “install AR” and “can’t measure” screens opened as the normal Measure screen. Returning to the app rechecks ARCore, and on the emulator that always says ready. The recheck stays for real phones, and only the debug screens opt out
• On iOS, an end to end UI test found that tapping the active tab did nothing, where every iOS app scrolls back to the top. The system tab bar gives you that for free, and my custom one did not

Bugs that were not bugs

Some failures looked like app bugs and were not. On iOS, four screenshots came out black, and on Android every screenshot showed the splash screen. Both looked like crashes. Both were the app simply not having finished launching when the screenshot fired, because the Android emulator on this Mac renders in software. Waiting longer fixed both. After the second time, I check timing before I debug the app.

Two flaky UI tests had the same flavour. One typed into a field before the keyboard was ready, on a busy simulator. Another typed a project name into the search field behind a dialog, because it picked the first text field it found. In both cases the app was fine and the test was wrong.