Automatically Turn UI Tests Into Screen Recordings

In a previous post, I talked about making your status bar nice when automating your App Store screenshots. These days, images alone are not enough to tell your app’s story. The App Store supports preview videos, and if you’re trying to create one of those for every combination of screen size and language that your app supports, it can be a chore!

Video can also be incredibly effective at presenting your work to others. At Detroit Labs, we pride ourselves in polished demos of our work to our clients. One of the best ways to do that is to embed videos of the app in a slide deck and go over it during the presentation. This avoids awkwardness in live demos and allows the video to loop as the client asks questions (you should also have the code ready to run in case the client asks about something that isn’t in the video).

One downside of these videos is the amount of time it takes to prepare them. If you’re diligent, you can record these videos for each individual pull request, then at the end of a sprint, collect the videos and prepare the presentation from them. Even if that happens, you can still fall out of sync. If pull request A shows some new functionality but is merged before pull request B, which updates the client’s logo to their new branding, their old branding in the video for pull request A will confuse them and distract from the actual feature during the demo. It’s best to compile all of the videos from a single, stable codebase, and deliver that exact code to the client along with the demo so there are no surprises.

Manually running all of the code and capturing video is a chore, especially for large, productive teams. I wanted to figure out a way to automate these video captures, and last week while in my hotel room at CodeMash, I figured out how to do it. Xcode ships with the simctl utility, and you can use it to record video from the command line:

xcrun simctl io <device> recordVideo <options> <path>

Once this starts running, it’ll record the given device’s screen to the given file. All we need to do is trigger this command. Unfortunately, the UI tests that Xcode runs run as iOS applications, so we don’t have access to the host Mac to run simctl. That’s where SBTUITestTunnelHost comes in! Using this framework and the associated Mac app, we can pass messages locally from our UI test suite to our Mac. One of the things it can do is execute command-line utilities. This is perfect for our needs… almost.

Because simctl will keep recording video until you hit ^C in Terminal, we need to leave it open. SBTUITestTunelHost, by default, executes a command, waits for it to finish, and then returns. To get around this, I modified the server (and sent a pull request) to enable starting asynchronous commands, then later terminating them. With this in place and running, recording an individual unit test is straightforward! Here’s an example test suite:

import SBTUITestTunnelHost
import XCTest

class InstantReplay_iOSUITests: XCTestCase {
    
    var commandID: UUID!
    
    override func setUp() {
        // Put setup code here. This method is called before the invocation of each test method in the class.

        // In UI tests it is usually best to stop immediately when a failure occurs.
        continueAfterFailure = false
        
        // UI tests must launch the application that they test.
        let app = XCUIApplication()
        app.launch()

        commandID = host.launchCommand(
            "xcrun simctl io booted recordVideo --codec=h264 --mask=black --force /Users/Jeff/Desktop/test.mov"
        )
        
        _ = app.wait(for: .runningForeground, timeout: 5)
        
        _ = app.textFields.firstMatch.waitForExistence(timeout: 0.5)

        if let commandID = commandID {
            addTeardownBlock { [unowned self] in
                self.host.interruptCommand(with: commandID)
            }
        }
        else {
            print("No status!")
        }
    }

    func testExample() {
        let app = XCUIApplication()
            
        app.textFields.firstMatch.tap()
        
        let keyDelay = { return TimeInterval.random(in: 0...0.15) }

        "UITestExample@example.com".map(String.init).forEach {
            if ["@", "."].contains($0) {
                app.keys["more"].tap()
                XCTAssertTrue(app.keys[$0].waitForExistence(timeout: keyDelay()))
                app.keys[$0].tap()
                app.keys["more"].tap()
            }
            else if $0 == $0.uppercased(), !app.keys[$0].exists {
                app.buttons["shift"].tap()
                XCTAssertTrue(app.keys[$0].waitForExistence(timeout: keyDelay()))
                app.keys[$0].tap()
            }
            else {
                XCTAssertTrue(app.keys[$0].waitForExistence(timeout: keyDelay()))
                app.keys[$0].tap()
            }
        }
        
        app.buttons["Return"].tap()
        
        XCTAssertTrue(app.textFields.firstMatch.waitForExistence(timeout: 0.5))
    }
    
}

This suite will automatically run, perform its tests, and record the output to a video on my Desktop. Sure enough, once I’d given the host app sufficient permissions in macOS Catalina, it worked, and while I was drinking my coffee after pressing ⌘U in Xcode, this video appeared on my desktop:

This is useful for pull requests, demos, embedding videos into Deckset presentations, you name it! Because it’s generating new videos every time, you never have to worry about your demo being out of date, or spending all of your time creating videos for every language you support. Just set it and forget it!

Published by

Jeff Kelley

I make iOS apps for Detroit Labs.