Category: watchOS Programming

Since the first release of watchOS, it’s been a unique platform among Apple’s in that Xcode doesn’t support running unit test targets for it. This is of course a hindrance to writing maintainable code for the watch; with no tests, there’s a lot of manual testing involved. It’s hard to even debug some aspects of watch code—since the debugger requires the watch to be connected to the iPhone, which is in turn connected to the Mac via USB, you can’t debug a watch app’s behavior when it’s disconnected from the phone.

There have been efforts to allow limited testing of WatchKit code, but so far everything I’ve seen has one crucial flaw: the tests are actually running on iOS, either by testing a shared framework or by simulating WatchKit on iOS. This is all well and good, and certainly better than nothing, but it doesn’t allow you to test any platform-specific code. Inspired by other efforts to make cross-platform frameworks well-tested, I wondered what it would take to run actual tests on watchOS.

A Tale of Two XCTests

Much like PivotalCoreKit re-implements some of WatchKit to let an iOS target use it, my first thought was to re-implement some of XCTest to get existing test code to build under watchOS. As I dug through XCTest, however, I realized that it’s actually a pretty complex framework, and a complete reimplementation doesn’t make sense when Apple has already begun the task and has a perfectly good open-source repository just sitting there waiting to be used.

To use Apple’s XCTest reimplementation, I first had to create a podspec file to allow CocoaPods to set up my Xcode project. It looks like so:

Pod::Spec.new do |s|
  s.name         = "XCTest"
  s.version      = "3.0.1"
  s.summary      = "A watchOS compilation of Apple’s open-source XCTest."

  s.description  = <<-DESC
            A watchOS compilation of Apple’s open-source XCTest.
                   DESC

  s.homepage     = "https://github.com/apple/swift-corelibs-xctest"
  s.license      = "Apache License, Version 2.0"
  s.author    = "Apple"

  s.watchos.deployment_target = "3.0"

  s.source       = { :git => "https://github.com/apple/swift-corelibs-xctest.git", :tag => "swift-" + s.version.to_s + "-RELEASE" }

  s.source_files  = "Sources/**/*.swift"

  s.framework  = "Foundation"

  s.prepare_command = <<-CMD
                        find Sources/ -type f -name "*.swift" | xargs sed -e 's/import SwiftFoundation/import Foundation/g' -i ""
                        sed -i "" -e 's/usingBlock:/using:/' Sources/XCTest/Public/XCTestCase+Asynchronous.swift
                      CMD
end

While this is a fairly straightforward podspec, I did two interesting things in its prepare_command to get it to work:

• Because the version I’m targeting uses the also-open-source SwiftFoundation instead of just Foundation, I use sed to change the import statement to point at regular Foundation. This has since been fixed, so this part will become unnecessary.
• I used sed again to fix a method that had been renamed since this tag was created.

The prepare_command part of a podspec is often-overlooked as a way to fix up a pod without making a fork of your own to maintain.

Now that I had a version of XCTest that would build for watchOS, I set up a new target in my Xcode project called “WatchTests Test Runner WatchKit App” (with a corresponding WatchKit Extension target). This target does what it says on the tin: run it to run the tests. The Xcode project also has an iOS unit test target, and the goal is to share those tests with watchOS, so I simply linked the test files with the new WatchKit extension. When those files use import XCTest, they’ll be pulling from the Swift version automatically.

Running The Tests

To run the tests, you call XCTMain() with an array of XCTestCaseEntry objects that represent the test classes you’d like to test. As of right now, however, it’s still a goal of the Swift XCTest project to enable test method discovery without the Objective-C runtime, which means we need to do it. The easiest way is for each test class to implement a property called allTests, wherein you manually enumerate test methods. The final Swift test class might look something like this:

import XCTest

class WatchTestsTests: XCTestCase {
    
    #if os(watchOS)
    static var allTests = {
        return [
            ("testPassingTest", testPassingTest),
        ]
    }()
    #endif
    
    func testPassingTest() {
        XCTAssertTrue(true)
    }
    
}

This test can be compiled on iOS and watchOS. The extension delegate of the test runner WatchKit App then runs the tests:

import WatchKit
import XCTest

class ExtensionDelegate: NSObject, WKExtensionDelegate {

    func applicationDidFinishLaunching() {
        XCTMain([testCase(WatchTestsTests.allTests)])
    }
    
}

XCTMain will call exit with an exit status, so you can see the results in Xcode’s console:

Test Suite 'All tests' started at 22:36:14.852
Test Suite 'WatchTestsTestRunner WatchKit Extension.appex.xctest' started at 22:36:14.854
Test Suite 'WatchTestsTests' started at 22:36:14.854
Test Case 'WatchTestsTests.testPassingTest' started at 22:36:14.854
Test Case 'WatchTestsTests.testPassingTest' passed (0.001 seconds).
Test Suite 'WatchTestsTests' passed at 22:36:14.856
     Executed 1 test, with 0 failures (0 unexpected) in 0.001 (0.001) seconds
Test Suite 'WatchTestsTestRunner WatchKit Extension.appex.xctest' passed at 22:36:14.856
     Executed 1 test, with 0 failures (0 unexpected) in 0.001 (0.001) seconds
Test Suite 'All tests' passed at 22:36:14.856
     Executed 1 test, with 0 failures (0 unexpected) in 0.001 (0.001) seconds
Program ended with exit code: 0

As you can see, this is exactly what you’d get out of XCTest on iOS, except it’s running in a watchOS target!

Limitations

This trivial example works well, but there are some limitations with this approach:

• Because I’m using the Swift implementation of XCTest, the only existing tests that could possibly run are those written in Swift. You have to subclass XCTestCase to use XCTest, and you can’t make an Objective-C subclass of a Swift class, so for now at least, no Objective-C tests can run on watchOS. If Swift and Objective-C interoperability improves to the point where you can subclass a Swift class in Objective-C, look for this to improve. Alternatively, XCTestCase could be a protocol exposed to Objective-C, which would allow more flexibility in that regard.
• Since we’re just running the tests directly, we don’t get any fancy Xcode integration. Failed tests won’t color a line red, nor will they run when you press ⌘U.
• Our XCTestCase subclasses are in a unique position: they are subclasses of an Objective-C class on iOS, and subclasses of a Swift class on watchOS. This can confuse the compiler and leave you without code completion.

Nevertheless, I’m excited to start tinkering with tests on watchOS. I have a couple areas I’d like to explore:

• Integrating these tests with CI of some sort. If I can get a shell script to return the watch target’s exit status, then I can get a Jenkins or Travis build to fail if the watch tests fail.
• Testing the watch UI further. Right now the shared WKExtension’s rootInterfaceController property is nil during testing. Waiting for the UI to load before calling XCTMain() will probably help here.

I’d love to hear suggestions or feedback on this. The more tests are written for watchOS, the better and more maintainable our watch apps will be! If you want to poke around with my code, you can find it on GitHub.