Build an iOS Launch Performance Regression Gate on a Cloud Mac

After a homepage refactor was merged, all functional tests passed, yet users began reporting slower first launches. Launch performance is easy to overlook in day-to-day development: a new database migration, synchronous configuration read, analytics initialization, or large image load can quietly add work to the main thread. Instead of waiting for someone to notice the slowdown manually, turn cold-launch measurement into a merge gate on a dedicated cloud Mac.

Define a Comparable Launch Scenario First

“Opening the app” is not a complete test definition. At a minimum, pin the build configuration, Simulator model, OS runtime, initial app data, and launch arguments. Keep the test target narrowly scoped as well. For example, measure only the interval from process creation until the first interactive screen appears, without mixing login requests or test-data downloads into the result.

Create a dedicated Performance.xctestplan, enable only the launch performance test, and disable randomized execution. Use launch arguments to place the test environment on a deterministic screen:

final class LaunchPerformanceTests: XCTestCase {
    override func setUp() {
        continueAfterFailure = false
    }

    func testColdLaunch() {
        let app = XCUIApplication()
        app.launchArguments = ["-uiTesting", "-resetLocalState"]

        let options = XCTMeasureOptions()
        options.iterationCount = 8

        measure(
            metrics: [XCTApplicationLaunchMetric(waitUntilResponsive: true)],
            options: options
        ) {
            app.launch()
        }
    }
}

The app’s test entry point should handle -resetLocalState and clear only local state that can be recreated. Do not use this argument to bypass initialization that genuinely needs to be measured in the production code path. Otherwise, the result reflects a “test-only launch speed.”

Pin the Cloud Mac and Simulator State

A performance gate needs a dedicated physical machine, not a shared environment where unknown workloads compete for CPU and disk resources. Before each run, verify the active Xcode path, available Simulators, and remaining disk space:

set -euo pipefail

xcode-select -p
xcodebuild -version
xcrun simctl list devices available
df -h "$HOME"

Select a specific installed Simulator in the test plan. Do not let the script automatically choose the “first available device,” because image changes will make historical results incomparable. While the gate is running, also avoid parallel archive jobs, dependency updates, and large-scale indexing tasks.

Performance data describes the environment before it describes the code. A number that cannot be tied to a specific runner, build configuration, and Simulator state should not be used directly to block a merge.

When establishing the initial baseline, repeat the test on the same commit. If the variance is clearly too high, investigate background tasks, disk pressure, and test-data reset logic before loosening the threshold.

Run a Single Gate with xcodebuild

The command-line job should run only the target test and preserve the result bundle as a pipeline artifact. Replace the device name with the model pinned in the actual test plan:

set -o pipefail
mkdir -p Artifacts

xcodebuild test \
  -workspace App.xcworkspace \
  -scheme App \
  -testPlan Performance \
  -destination 'platform=iOS Simulator,name=iPhone 16' \
  -only-testing:AppUITests/LaunchPerformanceTests/testColdLaunch \
  -resultBundlePath Artifacts/Launch.xcresult \
  | tee Artifacts/launch-test.log

set -o pipefail is essential. Without it, the trailing tee may allow the job to report success even after xcodebuild fails. Preserve both Launch.xcresult and the text log: the former provides access to individual measurements and test attachments, while the latter makes compilation and launch-stage errors easier to locate quickly.

Manage a Baseline Instead of Chasing Individual Numbers

Create a performance baseline for the specified test in Xcode’s test report, and commit the generated shared baseline files to version control. Baseline changes should be reviewed like dependency lockfiles: the commit description must explain the source of the change, the commit used for verification, and the expected impact. “Update performance data” is not sufficient.

Use separate thresholds for warnings and merge blockers. Small fluctuations can remain visible for trend monitoring, while a change that still exceeds the tolerance after repeated runs can fail the gate. The real criterion is not the fastest result from one run, but whether repeated iterations in the same environment consistently deviate from the reviewed baseline.

Do not update the baseline directly in any of the following situations:

• Other resource-intensive jobs were running on the test machine;
• The Simulator model or runtime changed;
• Dependencies were still being downloaded for the first time, or indexing had not finished;
• Initialization steps were skipped solely to make the test pass;
• Only one run was abnormal while the remaining results were normal.

Preserve Evidence After a Failure and Identify the Responsible Path

When the gate fails, rerun it once on the same cloud Mac and the same commit. If the result is reproducible, break the launch path into stages: process entry, dependency injection, database opening, initial-screen model construction, resource decoding, and first-frame rendering. Add consistent os_signpost instrumentation at these boundaries so later analysis shows where the time was spent instead of relying on guesses based on the size of a commit.

Start the investigation with the most recent changes, but do not focus exclusively on application code. Changes to build configuration, debug diagnostics, resource size, and test fixtures can also alter the launch path. After the fix, retain both the failing and passing xcresult bundles and record the comparison commits in the merge request.

The value of a reliable gate is not that it assigns an absolute label to launch time, but that it keeps the same scenario comparable over time. By pinning the environment, versioning the baseline, preserving evidence, and reviewing baseline updates, launch performance becomes a traceable engineering constraint rather than a subjective impression.

Frequently asked questions