$ ssh somehost
somehost$ sleep 15 &
somehost$ exit

$ ssh somehost
somehost$ sleep 15 &
somehost$ exit

If you run that command sequence, you’ll find that it makes the SSH connection and then exits immediately – and it kills the background job (sleep 15) as it exits. Which I find intuitive if only because that’s obviously how it’s always worked.

But now run it this way:

$ ssh somehost 'sleep 15 &'

$ ssh somehost 'sleep 15 &'

Now it makes the SSH connection, waits fifteen seconds, and only then exits.

This subtle difference matters a lot if you’re e.g. writing an automation system that happens to sometimes spawn background jobs on remote machines via SSH, since it can lead to SSH commands hanging (potentially forever, depending on the nature of the background jobs).

And it’s very hard to debug, because no matter what debugging aids you put into your SSH session’s remote command – e.g. an explicit exit 0 at the end, set -xv, or echo 'Okay, going away now!' as the last command – it’ll still just hang instead of completing.

Turns out this is because when run without a remote command, SSH implicitly creates a pseudo-terminal. And the pseudo-terminal provides the behaviour of killing all subprocesses (including un-detached background jobs) when the main process ends.

When you run SSH with a remote command, SSH does not create a pseudo-terminal. I’m guessing technically what it’s doing is waiting for the remote side to close the connection, which the remote side (sshd) won’t do until all child processes have exited (or at least closed their stdin, stdout, and stderr?).

One workaround is to use the -t argument to SSH, to force creation of a pseudo-terminal. The downside is that remote programs may then assume an interactive session, so they may prompt for input in cases where they wouldn’t otherwise. That could cause misalignment between what you write through to the remote command and what it’s expecting input for, or may cause a hang (if you don’t write anything but keep the remote side’s stdin open), or may cause the remote command to abort because it detects the end of stdin.

Another option is to manually kill all background jobs before exiting the main command, with e.g.:

jobs -p | egrep '^\d' | xargs kill

jobs -p | egrep '^\d' | xargs kill

…or just kill all subprocesses:

pkill -P $$

Or, if you can precisely control where you create background jobs (and are confident they don’t spawn background jobs recursively), you can do:

some background command &
BACKGROUND_PID=$!

… # Rest of script.

kill ${BACKGROUND_PID}
# *Now* we can exit without hanging.

some background command &
BACKGROUND_PID=$!

… # Rest of script.

kill ${BACKGROUND_PID}
# *Now* we can exit without hanging.

Just beware with this more precise approach that it’s more fragile – subprocesses could get spawned in ways you didn’t anticipate (if not now, then maybe in future versions of your code / production environment), and that sometimes the PID returned is effectively invalid because it’s merely the PID of some transient subprocess anyway (not the long-lived subprocess(es)).

  1> "\r".count
$R0: Int = 1
  2> "\n".count 
$R1: Int = 1
  3> "\r\n".count 
$R2: Int = 1

Yes, Swift treats the two bytes “\r\n” as a single character.  This is actually super convenient a lot of the time, because it means algorithms that look for line breaks with isNewline just work, even on “Windows”-style text.  Otherwise, you’d have to explicitly look for two isNewline characters in sequence and check if they are exactly “\r\n”.

But it does lead to some potentially surprising side-effects, like:

 4> x.unicodeScalars.count
$R3: Int = 2
 5> Array(x.unicodeScalars)
$R4: [String.UnicodeScalarView.Element] = 2 values {
  [0] = U'\r'
  [1] = U'\n'
}

This isn’t surprising if you’re pretty familiar with how Unicode actually works – starting with the difference between graphemes (approximately what Swift calls a Character) and “scalars”, but I suspect it’ll catch some people out.

extension AVPlayer {
    func fadeAudio(from startVolume: Float, to endVolume: Float, duration: Double) {
        let audioMix = AVMutableAudioMix()

        audioMix.inputParameters = (player.currentItem?.tracks ?? [])
                                    .compactMap(\.assetTrack)
                                    .filter({ $0.mediaType == .audio })
                                    .map { track in
            let currentTime = player.currentTime()

            let parameters = AVMutableAudioMixInputParameters(track: track)

            parameters.setVolumeRamp(fromStartVolume: startVolume,
                                     toEndVolume: endVolume,
                                     timeRange: CMTimeRange(start: currentTime,
                                                            duration: CMTime(seconds: duration,
                                                                             preferredTimescale: currentTime.timeScale)))

            return parameters
        }

        player.currentItem?.audioMix = audioMix
    }
}

I’m not certain what curve this implements, but to my ears it doesn’t sound quite as harsh as a naive linear ramp, so perhaps it’s an S-curve or similar.

Edge cases not handled

Where there’s less than duration time left in the track(s).

How you want to handle that might vary depending on context. e.g. you could clamp the duration to the remaining duration (but you have to think about whether your individual tracks all have the same duration, whether they match the duration of the overall playback item, and whether they’re all aligned within the playback sequence), or wrap it around to the beginning (if you’re looping), or carry the fade through to the next item (if you’re playing a sequence of items), etc. Alas this must be left as an exercise to you, the reader.

Starting a fade while another one is still in progress.

It will halt the previous fade, immediately jump to startVolume, and perform the new fade. If you know that a prior fade is in progress you could potentially extrapolate the current volume and start from there instead (though beware of non-linear ramping).

If you move the playhead backwards (e.g. skimming, or looped playback).

The mix will stay in place, and result in wonky volume levels on the subsequent plays through. To work around that, you can add:

player.currentItem?.audioMix = nil
player.volume = endVolume

…wherever you restart playback at the beginning or move playback earlier than the end of the fade.

Future work?

You can work around these limitations by doing a timer-based fade (i.e. player.volume += smallIncrement at regular, short intervals). However, the problem with that approach is that it’s not synchronised to actual playback – e.g. if the audio is paused, stutters, or faces an initial loading delay, your fade won’t wait for it, potentially resulting in no fade at all (e.g. it takes five seconds to buffer the audio before playback starts, at which point your five second “fade” has run to completion, so your audio starts playing abruptly at full volume).

There’s very likely a third option that addresses all these shortcomings, but I explored that a bit and concluded that it’d be a lot more work. If someone wants to explore that all the way, I’d be interested to see the result. But for many purposes the above code is quite sufficient.

If you’re using AVQueuePlayer, then just use AVPlayerLooper. Easy. But if for some reason you want to use AVPlayer specifically (e.g. you need to do additional things anyway when playback loops back around), read on.

AVPlayer itself doesn’t really help you here – it’s AVPlayerItem that you need to look at, as it has several notifications associated with it that can be very useful – most relevant here is the didPlayToEndTimeNotification. Simply observe that and restart the player explicitly, like so:

let player = AVPlayer(…)  // You'll need to keep a reference to this somewhere, for use in the notification handler, as the notification doesn't provide it.

void startPlayer() {  // Or wherever / however you start playback.
    NotificationCenter.default.addObserver(self,
                                           selector: #selector(playedToEnd(_:)),
                                           name: .AVPlayerItemDidPlayToEndTime,
                                           object: player.currentItem)

    player.play()
} 

@objc func playedToEnd(_ notification: Notification) {
    assert(player.currentItem == notification.object as? AVPlayerItem, "AVPlayer \(player)'s current item (\(player.currentItem)) doesn't match the one in the notification object (\(notification.object)).")

    player.seek(to: .zero)
    player.play()
}

Again, considering using AVQueuePlayer if possible, but the above works too.

If you’re determined, read on.

Screen savers are basically just applications. Same bundle format and structure, just with a “saver” extension instead of “app”.

Setting up the Xcode project

In Xcode, create a new project using the Screen Saver template.

Delete the Objective-C code & header files it creates by default (unless, I suppose, you want to write your screen saver in Objective-C – woo, retro! 😆).

You need to import the ScreenSaver module (framework), subclass ScreenSaverView, and implement a couple of method overrides. Here’s the basic skeleton:

import ScreenSaver

class MyScreenSaver: ScreenSaverView {
    override init?(frame: NSRect, isPreview: Bool) {
        super.init(frame: frame, isPreview: isPreview)
        setup()
    }

    required init?(coder: NSCoder) {
        super.init(coder: coder)
        setup()
    }

    private func setup() {
        // TODO
    }

    override func startAnimation() {
        super.startAnimation()

        // TODO
    }

    override func animateOneFrame() {  // Optional.
        // TODO
    }

    override func stopAnimation() { //  Only for the live preview in System Settings.
        // TODO

        super.stopAnimation()
    }

    override var hasConfigureSheet: Bool {
        true
    }

    private var configureSheetController: ConfigureSheetController?

    override var configureSheet: NSWindow? {
        configureSheetController = ConfigureSheetController(windowNibName: "ConfigureSheet")
        return configureSheetController?.window
    }
}

class ConfigureSheetController: NSWindowController {
    override var windowNibName: NSNib.Name? {
        return "ConfigureSheet"
    }

    override func windowDidLoad() {
        super.windowDidLoad()

        // TODO
    }

    @IBAction func okButtonClicked(_ sender: NSButton) {
        // TODO

        window!.sheetParent!.endSheet(window!, returnCode: .OK)
    }
}

import ScreenSaver

class MyScreenSaver: ScreenSaverView {
    override init?(frame: NSRect, isPreview: Bool) {
        super.init(frame: frame, isPreview: isPreview)
        setup()
    }

    required init?(coder: NSCoder) {
        super.init(coder: coder)
        setup()
    }

    private func setup() {
        // TODO
    }

    override func startAnimation() {
        super.startAnimation()

        // TODO
    }

    override func animateOneFrame() {  // Optional.
        // TODO
    }

    override func stopAnimation() { //  Only for the live preview in System Settings.
        // TODO

        super.stopAnimation()
    }

    override var hasConfigureSheet: Bool {
        true
    }

    private var configureSheetController: ConfigureSheetController?

    override var configureSheet: NSWindow? {
        configureSheetController = ConfigureSheetController(windowNibName: "ConfigureSheet")
        return configureSheetController?.window
    }
}

class ConfigureSheetController: NSWindowController {
    override var windowNibName: NSNib.Name? {
        return "ConfigureSheet"
    }

    override func windowDidLoad() {
        super.windowDidLoad()

        // TODO
    }

    @IBAction func okButtonClicked(_ sender: NSButton) {
        // TODO

        window!.sheetParent!.endSheet(window!, returnCode: .OK)
    }
}

Providing a preferences sheet

If you don’t have any options to configure, you can of course change hasConfigureSheet to return false. Otherwise, you’ll need to create a “ConfigureSheet” xib file with a window in it containing your screen saver’s settings. You can use UserDefaults to save your settings (if you wish), same as any other app. And you’ll need to add an “Okay” or “Save” or similar button to dismiss the sheet.

Getting ready to render

The key methods to implement are setup, with any initial configuration you wish to do (e.g. allocate image or video views, load assets, set up the view hierarchy, etc).

ScreenSaverView is an NSView subclass with a flat black background by default, onto which you can add subviews. Typically for a screen saver you have a very simple view hierarchy – often just a single view or CoreAnimation layer that you’re rendering to – but you can load a xib and insert elements from it into the view if you like.

Rendering

startAnimation is where you should actually start displaying things. e.g. if you’re using an AVPlayer, this is where you actually start it playing (after making it visible, e.g. setting its opacity to 1).

animateOneFrame is optional, and is only called if you set the animationTimeInterval property (on self) to a finite, non-zero value in setup (in which case it’ll be called at intervals at least that long – it might not be called as often as you desire if previous calls overrun or there’s other bottlenecks in the screen saver framework). It’s essentially just a minor convenience vs having to explicitly set up an NSTimer.

Given how buggy Apple’s screen saver framework is, I suggest not relying on animateOneFrame if you can at all avoid it. Even if that means setting up your own timer. That way when they likely break that too in some future macOS release, your screen saver won’t necessarily break as well.

Bonus topic: fading in

Unless your screen saver inherently appears gently (e.g. starts rendering with a flat black view and only slowly adds to it), it’s nice to add a fade-in. You can do that using CoreAnimation on the view’s layer:

override func startAnimation() {
    // Other code…
    
    if let layer {
        layer.opacity = 0.0  // Should already be zero from `setup`, but just to be sure.

        let fadeAnimation = CABasicAnimation(keyPath: "opacity")
        fadeAnimation.fromValue = 0.0
        fadeAnimation.toValue = 1.0
        fadeAnimation.duration = 5  // Seconds.

        // Essential settings to keep the final state
        fadeAnimation.fillMode = .forwards
        fadeAnimation.isRemovedOnCompletion = false

        layer.add(fadeAnimation, forKey: "fadeAnimation")
    }
    
    // Other code…
}

override func startAnimation() {
    // Other code…
    
    if let layer {
        layer.opacity = 0.0  // Should already be zero from `setup`, but just to be sure.

        let fadeAnimation = CABasicAnimation(keyPath: "opacity")
        fadeAnimation.fromValue = 0.0
        fadeAnimation.toValue = 1.0
        fadeAnimation.duration = 5  // Seconds.

        // Essential settings to keep the final state
        fadeAnimation.fillMode = .forwards
        fadeAnimation.isRemovedOnCompletion = false

        layer.add(fadeAnimation, forKey: "fadeAnimation")
    }
    
    // Other code…
}

Note that you cannot implement a fade-out when the screen saver exits, because macOS hides your screen saver immediately. Plus, the user might not want a fade-out as they may be in a rush to do something on their computer.

Previewing

You can determine if you’re running for real or only the preview via the isPreview property (on self). Many screen savers don’t care, but particularly if you save any persistent state, you might want to avoid doing that during preview. For example, in a screen saver which plays a looping video and resumes where it last left off, you probably don’t want the preview to quietly advance the video.

Stopping

stopAnimation is only used for the live preview thumbnail shown in the Screen Saver System Settings pane. It is never called in normal operation of the screen saver (contrary to what Apple’s documentation says – Apple broke that in macOS Sonoma and later).

And that leads to the first path off the official track. When the screen saver is dismissed by the user, nothing in Apple’s framework code does anything. Your view continues to exist, animateOneFrame continues getting called, etc. Your screen saver just runs in the background, its output not visible, but wasting CPU cycles and RAM. Worse, if you have sound, that keeps playing.

To get around that, you need to register for the com.apple.screensaver.willstop notification, in setup, like so:

  private func setup() {
    DistributedNotificationCenter.default.addObserver(self,
                                                      selector: #selector(willStop(_:)),
                                                      name: Notification.Name("com.apple.screensaver.willstop"),
                                                      object: nil)
}
    
@objc func willStop(_ notification: Notification) {
    stopAnimation()
}

  private func setup() {
    DistributedNotificationCenter.default.addObserver(self,
                                                      selector: #selector(willStop(_:)),
                                                      name: Notification.Name("com.apple.screensaver.willstop"),
                                                      object: nil)
}
    
@objc func willStop(_ notification: Notification) {
    stopAnimation()
}

Note that you still need stopAnimation specifically, because in the live preview in System Settings you won’t receive that com.apple.screensaver.willstop notification (from the system’s point of view, the screen saver isn’t running – it’s merely previewing).

Handling resumption

Here’s the second big bug in Apple’s screen saver framework – every time the screen saver starts, your ScreenSaverView subclass is created again. But the old one doesn’t go anywhere. So now you have two copies running simultaneously, which is at the very least wasteful, and can easily lead to gnarly bugs and weird behaviour (e.g. if both are playing sound, or both modify persistent state).

There are essentially two ways to handle this:

1. Kill your own process every time you stop animating.
2. Manually kill or lame-duck older views when a new one is initialised.

Note that you cannot simply check at MyScreenSaver initialisation time if an instance already exists and if so fail initialisation (as is prescribed by this otherwise excellent write-up of this problem), because if you don’t correctly initialise you’ll sometimes end up with nothing rendering or running (the screen saver framework appears to not gracefully handle initialisation failures).

Killing your own process can work but has some perils:

• If you kill your process in stopAnimation the screen will flash black momentarily before actually exiting screen saver mode, which is visually annoying.
• If the screen saver is restarted rapidly after being interrupted, sometimes you’ll end up with nothing but a black screen (with no screen saver running). There’s evidently some race condition in Apple’s screen saver system between screen saver processes exiting and being [re]launched.

So I recommend not taking that approach. Instead, you can lame-duck the old view instances. They’ll stick around, which is a little wasteful of RAM, but as long as they’re not rendering or otherwise doing anything, they’re benign.

There are various ways to implement that, but one of the simpler ones is simply a notification between instances:

static let NewInstanceNotification = "com.myapp.MyScreenSaver.NewInstance";

var lameDuck = false

private func setup() {
    // Initial setup…
    
    NotificationCenter.default.post(name: MyScreenSaver.NewInstanceNotification, object: self)

    NotificationCenter.default.addObserver(self,
                                           selector: #selector(neuter(_:)),
                                           name: MyScreenSaver.NewInstanceNotification,
                                           object: nil)

    // Further setup…
}

@objc func neuter(_ notification: Notification) {
    lameDuck = true

    stopAnimation()

    self.removeFromSuperview()

    // TODO: any additional cleanup you can, e.g. release image & video files, throw out transient models and state, etc.

    NotificationCenter.default.removeObserver(self)
    DistributedNotificationCenter.default().removeObserver(self)
}

static let NewInstanceNotification = "com.myapp.MyScreenSaver.NewInstance";

var lameDuck = false

private func setup() {
    // Initial setup…
    
    NotificationCenter.default.post(name: MyScreenSaver.NewInstanceNotification, object: self)

    NotificationCenter.default.addObserver(self,
                                           selector: #selector(neuter(_:)),
                                           name: MyScreenSaver.NewInstanceNotification,
                                           object: nil)

    // Further setup…
}

@objc func neuter(_ notification: Notification) {
    lameDuck = true

    stopAnimation()

    self.removeFromSuperview()

    // TODO: any additional cleanup you can, e.g. release image & video files, throw out transient models and state, etc.

    NotificationCenter.default.removeObserver(self)
    DistributedNotificationCenter.default().removeObserver(self)
}

You should check lameDuck at the start of methods like startAnimation or animateOneFrame and exit immediately if it’s set to true. Unfortunately, Apple’s screen saver framework will still call those methods on old instances.

Exiting

Unfortunately Apple’s screen saver system will never terminate your screen saver process. Worse, even if you do nothing yourself, Apple’s screen saver framework code will run in an infinite loop, wasting [a small amount of] CPU time. So it’s not great to leave your screen saver process running indefinitely.

Thus, I implement an idle timeout in my screen savers, to have them exit if they’re not active for a while. This can be done like:

@MainActor var idleTimeoutWorkItem: DispatchWorkItem? = nil

override func startAnimation() {
    // Other code…
    
    DispatchQueue.main.async {
        if let idleTimeoutWorkItem {
            idleTimeoutWorkItem.cancel()
        }

        idleTimeoutWorkItem = nil
    }
    
    // Other code…
}

override func stopAnimation() {
    // Other code…
    
    if !lameDuck {
        DispatchQueue.main.async {
            idleTimeoutWorkItem?.cancel()

            let workItem = DispatchWorkItem(block: {
                NSApplication.shared.terminate(nil)
            })
            
            idleTimeoutWorkItem = workItem
            
            DispatchQueue.main.asyncAfter(wallDeadline: .now() + 65, execute: workItem)
        }
    }
    
    // Other code…
}

@MainActor var idleTimeoutWorkItem: DispatchWorkItem? = nil

override func startAnimation() {
    // Other code…
    
    DispatchQueue.main.async {
        if let idleTimeoutWorkItem {
            idleTimeoutWorkItem.cancel()
        }

        idleTimeoutWorkItem = nil
    }
    
    // Other code…
}

override func stopAnimation() {
    // Other code…
    
    if !lameDuck {
        DispatchQueue.main.async {
            idleTimeoutWorkItem?.cancel()

            let workItem = DispatchWorkItem(block: {
                NSApplication.shared.terminate(nil)
            })
            
            idleTimeoutWorkItem = workItem
            
            DispatchQueue.main.asyncAfter(wallDeadline: .now() + 65, execute: workItem)
        }
    }
    
    // Other code…
}

I chose the 65 second timeout somewhat arbitrarily. I figure there’s a reasonable chance a user will unlock their screen to do something quick, then engage the screen saver again – all in the less than a minute – and the cost of idling in the background for an extra minute is small, compared to the cost of relaunching the whole app and reinitialising your renderer.

I added five extra seconds to reduce the probability of aligning with some one-minute timer (e.g. a spurious wake with the screen saver set to start automatically after one minute of no user activity).

You can adjust it however you like.

Testing your screen saver

Double-clicking the built product (your “.saver” app) will prompt the user to install it, replacing an old version if necessary. So that works, though I find it faster to just manually copy the “.saver” app to ~/Library/Screen Savers. Just make sure to kill the existing legacyScreenSaver process, if necessary.

You can test it in System Settings, in the Screen Saver pane. That’s the only place you can test the live preview part.

But otherwise, I found it easiest to just set one of the screen hot corners to start the screen saver, and use that immediately after copying the new “.saver” file into place.

Just be aware that the first time any new copy of the screen saver runs, macOS runs a verification on the bundle, which can take a while if your screen saver is non-trivial in size (e.g. if you bundle large image or video resources). You’ll get a black screen with nothing happening, after invoking the screen saver, while that verification is running.

Distributing your screen saver

You don’t have to sign your screen saver, necessarily, but users will get some annoying error dialogs trying to run it, and will have to fiddle with things in System Settings – or, if they’re on a corporate Mac, they might not be able to run it at all. So it’s preferable to just sign it.

Xcode doesn’t support signing screen savers like it does for plain app targets and the like. So you have to do it manually via the command line, on the built product (your “.saver” app). Thankfully it’s just two commands (once you have the appropriate stuff set up in your developer account – note that you will need a paid Apple Developer account, at $99/year).

Follow the instructions here, but note that they’re missing the final step:

xcrun stapler staple -v MyScreenSaver.saver

Note that you run it against the screen saver itself, not the zip file. The zip file’s just a hack to get Apple’s notary tool to accept the submission. It’s the screen saver bundle itself that’s actually notarised and signed.

Conclusion

That’s “it” in a superficial sense – if you’ve followed all this so far, you have a roughly working screen saver.

But there are a lot more bugs and nuances thereof that may afflict you, depending on what you’re doing in your screen saver. So good luck. 😣

Addendum: SaveHollywood

I found that looking at existing open source screen savers was partially helpful, but also sometimes misleading. e.g. for my most recent screen saver I basically just play a video in a loop (which should be embarrassingly trivial but took two weeks to get working properly, thanks to the aforementioned bugs in Apple’s frameworks, among many others). In that case I looked at SaveHollywood, a similar screen saver, for aid and ideas.

Unfortunately, SaveHollywood is abandoned and doesn’t work on recent versions of macOS. The way it does some things is archaic and either not the best way or not functional at all.

Nonetheless, it did help with some of the higher-level aspects, above the screen saver machinery itself, like how to use AVPlayer in a screen saver.

So, do check out similar, existing screen savers (and of course just use them directly if they suit your needs!) but beware of obsolete or otherwise incorrect code.

Addendum: What this says about Apple

What really troubles me about the screen saver system is what it says about Apple’s approach to the Mac, software, and their users. Which is sadly just the same thing we’ve been seeing for years now.

Screen savers used to work fine. There was an API established long ago that was lightweight, straight-forward, and effective. All Apple had to do was not break it.

And how they broke it is troubling. Perhaps it was prompted by some otherwise unrelated but well-meaning refactor – pulling screen saver code out into a separate, sandboxed process, perhaps, for improved system security. Fine. But if it’s worth changing it’s worth changing properly. It’s very clear that whomever did the changes either (a) didn’t care that they broke things badly, or (b) didn’t care to check.

It’s that recurring theme of not caring that’s most disappointing in today’s Apple.

What’s the difference between presentedWindowStyle(_:) and windowStyle(_:)?

Well, one does something, the other doesn’t, basically.

I tried using the former, and observed that it never has any effect. I filed FB14892608 about it, a month ago. While the long delay isn’t great, the response I got today was actually pretty helpful:

We’re sorry you ran into trouble with that API.

To adjust the style of a window group’s windows, you have a couple of options. If all of the windows in the group should have the same style, then using the windowStyle() scene modifier is what you want:

struct MyApp: App {
    var body: some Scene {
        WindowGroup {
            ContentView()
        }
        .windowStyle(.hiddenTitleBar)
    }
}

If you need to adjust the style on an individual basis using state for that window, there are also some related view modifiers:
struct MyApp: App {
    var body: some Scene {
        WindowGroup {
            ContentView()
                .toolbar(removing: .title)
                .toolbarBackgroundVisibility(.hidden, for: .windowToolbar)
        }
    }
}

A simple apology followed by clear and specific instructions on how to actually achieve what I wanted. I genuinely applaud Apple – or perhaps I should more specifically thank the individual(s) within DTS that actually provided this response. Either way, it was a very pleasant surprise.

Now, it still leaves unanswered the question of what presentedWindowStyle(_:) is supposed to be for, and in what situations it actually does anything. So not a perfect reply, per se, but really the proper solution for that is either:

1. Fix the API to not have such confusingly similar modifiers.
2. Improve the documentation to:
   • Explicitly point out the other, similarly-named modifier.
   • Distinguish them.

Currently the documentation is just:

Versus:

The latter might technically convey something important here, in the created by interacting with this view part, but so much in SwiftUI is modifiers stuck haphazardly in weird and arbitrary places, that I’m been conditioned to ignore the fact that I’m often applying modifiers to views that don’t actually affect those views. And for all I know in SwiftUI parlance views do “create” and/or “interact” with their parent windows (a lot about SwiftUI is backwards, part of its nature as a declarative API).

Especially if that’s the first candidate API you stumble across, when searching for a way to style a window, it’s very easy to presume it’s the right API. Why would there be multiple APIs for styling the window; why would you continue searching after finding one?

Similarly, the presentedWindowStyle(_:) variant is the only one that appears in Xcode’s auto-complete if you try to add the modifier to your main view, which is both where you sometimes have to add window-level modifiers and also just a really easy mistake to make (instead of adding the modifier to the WindowGroup, one indentation level up).

Lastly, it doesn’t help that I can’t find any situation in which presentedWindowStyle(_:) has any effect, even knowing it’s not intended to style the parent window. One might assume it’s somehow related to sheets or somesuch, but apparently not? Presumably I’m overlooking some use-case – or maybe it doesn’t actually do anything on macOS, only iDevices? I welcome clues or tips.

If any view in the [active] window contains a Toggle – even one that’s disabled or hidden – then Continuity Camera (re. ImportFromDevicesCommands and importableFromServices) doesn’t work; all the submenu items under “Import from iPhone or iPad” are disabled.

I don’t know if this is truly specific to Toggle, that’s just the example case I happen to have isolated [first?].

What’s really weird is that once a Toggle has ever been displayed, even if you subsequently remove it from the view hierarchy entirely the “Import from iPhone or iPad” submenu items all remain disabled.

import SwiftUI

@main
struct Example: App {
    @State var breakImportFromiDevice = true
    @State var text = ""

    var body: some Scene {
        WindowGroup {
            VStack {
                TextField("Input", text: $text) // Doesn't break anything.

                if breakImportFromiDevice {
                    Toggle("Break import from iDevice", isOn: $breakImportFromiDevice)
                }
            }
            .importableFromServices(action: { (images: [NSImage]) -> Bool in
                print("Load image!")
                return true
            })
        }.commands {
            ImportFromDevicesCommands()
        }
    }
}

There are some circumstances in which this gets “unbroken” during interactions with other views and so forth, which is 100% reproducible in my real app but I have no idea what the reason is. The steps involved in my real app are kinda ridiculous (and not in any way remotely a viable workaround) and make absolutely no sense – it only “unbreaks” when a specific view is in a specific weird state (itself kind of the result of a bug, albeit a benign one). And that weird state is merely whether it’s displaying one image or another – which as far as SwiftUI is concerned is not even a difference in view state, since I’m just swapping NSImages under the cover.

I figured it must be something to do with view focus, but after much experimentation I believe I can conclusively rule that out. No matter which view has focus, or how focus is configured, or which views are even focusable at all, the problem persists. Likewise for window focus and key state. And, [accessability-]focusable views other than Toggle – e.g. TextField – don’t cause any issues.

Frankly it’s baffling, and a mite infuriating. I can’t even conceive of how SwiftUI can be so incredibly broken regarding such basic functionality, and the apparent interaction of GUI elements that have absolutely no business together.

Tangentially, a few things I’ve noticed about this Continuity Camera feature:

• “Add Sketch” doesn’t do anything. Unlike the other options, which open the camera app on the target iDevice, it has no effect. 🤷‍♂️
• Within the Finder, you can right-click empty whitespace within a folder, and the contextual menu has this “Import from iPhone or iPad” option at the bottom. That’s pretty handy – until now I’d been doing it the “hard” way by taking a photo on my iPhone and AirDropping it across to my Mac.
  
  I’d never noticed that feature prior to debugging this problem (I wanted to confirm that multiple other apps worked just fine; that it wasn’t an issue with Continuity Camera system-wide).
• TextEdit has the same feature but tweaks the wording to “Insert…” rather than “Import…”, which I thought is both a nice touch and frustratingly not something you can do in your own apps (at least, not in SwiftUI). 😕

NSPasteboard mutates itself simultaneously from the main thread and the global concurrent Dispatch pool, w.r.t. to its internal type cache. This is surprisingly trivial to reproduce (sample code below) by just dropping, e.g. a file promise (such as by opening a PNG in Preview, revealing the thumbnails sidebar, and then dragging the thumbnail onto the sample project’s window).

import SwiftUI

struct ContentView: View {
    var body: some View {
        Rectangle().onDrop(of: NSImage.imageTypes, isTargeted: nil) { _ in
            let pb = NSPasteboard(name: .drag)

            _ = pb.pasteboardItems // Seems to be necessary for the crash.

            _ = NSImage.imageTypes // Not strictly necessary for the crash, but seems to make it more likely. 🤷‍♂️

            return true
        }
    }
}

Judging from the callstack that runs in the concurrent pool, this is specific to file promises (and that seems to match my experience – it only crashes for some test cases, all of which involve file promises being present in the drag pasteboard at the time of the drop).

Since this bug causes semi-random memory corruption, it manifests in a large number of ways – not all of which are all that helpful. But at least one case I’ve seen a few times is helpful, as it clearly shows the offending internal NSPasteboard code running concurrently with itself, e.g.:

Main queue / thread:
#0	0x00007ff80108dab5 in _platform_bzero$VARIANT$Haswell ()
#1	0x000000010df8774d in GuardMalloc_mallocInternal ()
#2	0x00007ff90792542f in stack_logging_lite_malloc ()
#3	0x00007ff800ea8733 in _malloc_zone_malloc_instrumented_or_legacy ()
#4	0x00007ff800f39a72 in _vasprintf ()
#5	0x00007ff800f16922 in asprintf ()
#6	0x00007ff80125655a in -[NSObject(NSObject) __dealloc_zombie] ()
#7	0x00007ff8020e039a in -[NSConcreteMapTable dealloc] ()
#8	0x00007ff804876983 in -[NSPasteboard _updateTypeCacheIfNeeded] ()
#9	0x00007ff8048763df in -[NSPasteboard _typesAtIndex:combinesItems:] ()
#10	0x00007ff804aad148 in NSCoreDragReceiveMessageProc ()
#11	0x00007ff807517b1a in CallReceiveMessageCollectionWithMessage ()
#12	0x00007ff8075124fa in DoMultipartDropMessage ()
#13	0x00007ff8075122ce in DoDropMessage ()
#14	0x00007ff8075159a9 in CoreDragMessageHandler ()
#15	0x00007ff8011d776b in __CFMessagePortPerform ()
#16	0x00007ff80113e5b7 in __CFRUNLOOP_IS_CALLING_OUT_TO_A_SOURCE1_PERFORM_FUNCTION__ ()
#17	0x00007ff80113e4ee in __CFRunLoopDoSource1 ()
#18	0x00007ff80113d166 in __CFRunLoopRun ()
#19	0x00007ff80113c112 in CFRunLoopRunSpecific ()
#20	0x00007ff80bb55a09 in RunCurrentEventLoopInMode ()
#21	0x00007ff80bb55646 in ReceiveNextEventCommon ()
#22	0x00007ff80bb55561 in _BlockUntilNextEventMatchingListInModeWithFilter ()
#23	0x00007ff8047acc61 in _DPSNextEvent ()
#24	0x00007ff8050c0dc0 in -[NSApplication(NSEventRouting) _nextEventMatchingEventMask:untilDate:inMode:dequeue:] ()
#25	0x00007ff80479e075 in -[NSApplication run] ()
#26	0x00007ff804771ff3 in NSApplicationMain ()
#27	0x00007ff90dc24557 in ___lldb_unnamed_symbol57096 ()
#28	0x00007ff90e31fe64 in ___lldb_unnamed_symbol104448 ()
#29	0x00007ff90e6e63ff in static SwiftUI.App.main() -> () ()
#30	0x000000010dfa5cce in static NSPasteboardItem_CrashApp.$main() ()
#31	0x000000010dfa5d69 in main at /Users/SadPanda/Documents/NSPasteboardItem Crash/NSPasteboardItem Crash/NSPasteboardItem_CrashApp.swift:11
#32	0x00007ff800cd5366 in start ()

Dispatch concurrent queue (default QoS):
#0	0x00007ff80111b45c in -[__NSSetM addObject:] ()
#1	0x00007ff80487692e in -[NSPasteboard _updateTypeCacheIfNeeded] ()
#2	0x00007ff8048763df in -[NSPasteboard _typesAtIndex:combinesItems:] ()
#3	0x00007ff804aa9597 in -[NSPasteboard _canRequestDataForType:index:usesPboardTypes:combinesItems:] ()
#4	0x00007ff804fdd161 in -[NSPasteboard _dataForType:index:usesPboardTypes:combinesItems:securityScoped:] ()
#5	0x00007ff804aa7c4b in -[NSPasteboardItem dataForType:] ()
#6	0x00007ff8055804af in -[NSFilePromiseReceiver receivePromisedFilesAtDestination:options:operationQueue:reader:] ()
#7	0x00007ff90e787b51 in ___lldb_unnamed_symbol131674 ()
#8	0x00007ff90e9bc340 in ___lldb_unnamed_symbol148147 ()
#9	0x00007ff8020d00ba in __NSBLOCKOPERATION_IS_CALLING_OUT_TO_A_BLOCK__ ()
#10	0x00007ff8020cffb8 in -[NSBlockOperation main] ()
#11	0x00007ff8020cff4b in __NSOPERATION_IS_INVOKING_MAIN__ ()
#12	0x00007ff8020cf1ec in -[NSOperation start] ()
#13	0x00007ff8020cef0d in __NSOPERATIONQUEUE_IS_STARTING_AN_OPERATION__ ()
#14	0x00007ff8020cedde in __NSOQSchedule_f ()
#15	0x000000010e68ce7d in _dispatch_block_async_invoke2 ()
#16	0x000000010e67ca7b in _dispatch_client_callout ()
#17	0x000000010e67fa09 in _dispatch_continuation_pop ()
#18	0x000000010e67eae8 in _dispatch_async_redirect_invoke ()
#19	0x000000010e6906a9 in _dispatch_root_queue_drain ()
#20	0x000000010e6911ba in _dispatch_worker_thread2 ()
#21	0x000000010dfb832f in _pthread_wqthread ()
#22	0x000000010dfbebeb in start_wqthread ()

There doesn’t appear to be any workaround (short of not supporting drops at all!).

The more complicated the drop handler the more likely it is to promptly crash upon drop – in my real code with a non-trivial handler, it’s virtually guaranteed to crash on the second drop containing a file promise, while in the vastly reduced sample code (above) it can take dozens of drops before it finally crashes outright.

I have not directly tested whether this NSPasteboard bug occurs in the absence of SwiftUI, so I don’t strictly know if the root cause is in AppKit or SwiftUI. However, since most SwiftUI apps don’t support drag-and-drop, but plenty of AppKit ones do and manage to not crash when given the exact same test cases, I do strongly suspect SwiftUI is causing this somehow.

Follow-up (September 12th, 2024)

I actually received a response from Apple, from a real human (or at least a convincing AI). Ultimately their response didn’t help as it contained some mistakes, but I’m hopeful there’ll be more follow-up and a productive conclusion. And in the interim, they did assert a few things which are important to know, and are not otherwise documented by Apple:

• SwiftUI’s onDrop(of:isTargeted:perform:) method makes no claims or promises as to what thread / queue it executes the closure on, and in fact according to the anonymous Apple engineer it never executes the closure on the main thread.
  
  Now, while that may be the intent, the reality of that is wrong – in my experience it always executes the closure on the main thread (which makes a lot of sense to me as drag-and-drop event handling in AppKit has always been on the main thread in practice).
  
  Nonetheless, Apple says one cannot rely on the current behaviour and should in fact assume it never executes on the main thread (though in practice that means you have to check, not assume, since if you blindly do something like DispatchQueue.main.sync { … } in your drop handler your code will deadlock, today).
• NSPasteboard is not safe to use outside the main thread / queue.
  
  This isn’t documented anywhere public – not in NSPasteboard‘s documentation itself, nor the ancient Application Kit Framework Thread Safety documentation.
  
  I wouldn’t be surprised if it’s broadly true, as the AppKit APIs involving it always seemed main-thread centric anyway (all the handlers and delegate methods involving pasteboards are invoked on the main thread, in my experience). And it’s generally best to assume everything in AppKit is main-thread-only unless it’s explicitly documented otherwise.
  
  However, it’s important to note that Apple’s own code doesn’t follow this rule – e.g. NSFilePromiseReceiver, internally, uses NSPasteboard from the global concurrent queue.

Even though Apple’s initial response to this bug report hasn’t been all that fruitful, I do want to emphasise the fact that they did respond, which was a pleasant surprise and very much appreciated.

I find my hand is often forced by APIs I don’t control (most often Apple’s APIs). e.g. data source or delegate callbacks that are synchronous¹ and require you to return a value, but in order to obtain that value you have to run async code (perhaps because yet again that’s all you’re given by 3rd parties, or because that code makes sense to be async and is used happily as such in other places and you don’t want to have to duplicate it in perpetuity just to have a sync version).

If that asynchronosity is achieved through e.g. GCD or NSRunLoop or NSProcess or NSTask or NSThread or pthreads, it’s easy. There are numerous ways to synchronously wait on their tasks. In contrast, Swift Concurrency really doesn’t want you to do this. The language and standard library take an adamant idealogical position on this – one which is unfortunately impractical; a spherical chicken in a vacuum².

Nonetheless, despite Swift’s best efforts to prevent me, I believe I’ve come up with a way to do this. It appears to work reliably, given fairly extensive testing. Nonetheless, I do not make any promises. Use at your own risk.

If you know of a better way, please do let me know (e.g. in the comments below).

import Dispatch

extension Task {
    /// Executes the given async closure synchronously, waiting for it to finish before returning.
    ///
    /// **Warning**: Do not call this from a thread used by Swift Concurrency (e.g. an actor, including global actors like MainActor) if the closure - or anything it calls transitively via `await` - might be bound to that same isolation context.  Doing so may result in deadlock.
    static func sync(_ code: sending () async throws(Failure) -> Success) throws(Failure) -> Success { // 1
        let semaphore = DispatchSemaphore(value: 0)

        nonisolated(unsafe) var result: Result<Success, Failure>? = nil // 2

        withoutActuallyEscaping(code) { // 3
            nonisolated(unsafe) let sendableCode = $0 // 4

            let coreTask = Task<Void, Never>.detached(priority: .userInitiated) { @Sendable () async -> Void in // 5
                do {
                    result = .success(try await sendableCode())
                } catch {
                    result = .failure(error as! Failure)
                }
            }

            Task<Void, Never>.detached(priority: .userInitiated) { // 6
                await coreTask.value
                semaphore.signal()
            }

            semaphore.wait()
        }

        return try result!.get() // 7
    }
}

Elaborating on some of the odder or less than self-explanatory aspects of this:

1. The closure parameter must be sending otherwise this deadlocks if e.g. you call it from the main thread (even if the closure, and all its transitive async calls, are not isolated to the main thread). I don’t understand why this happens – it’s possibly explicable and working as intended, but I wonder if it’s simply a bug. Nobody has been able to explain why it happens.
   
   Note: in the initial version of this post I accidentally omitted this essential keyword. I apologise for the error, and hope it didn’t cause grief for anyone.
2. Since there’s no sync way to retrieve the result of a Task, the result has to be passed out through a side-channel. The nonisolated(unsafe) is to silence the Swift 6 compiler’s erroneous error diagnostics about concurrent mutation of shared state.
3. Task constructors only accept escaping closures, even though as they’re used here the closure never actually escapes. Fortunately the withoutActuallyEscaping escape hatch is available.
4. code isn’t @Sendable – since it doesn’t actually have to be sent in the sense of executing concurrently – so trying to use it in the Task closure below, which is @Sendable, results in an erroneous compiler error (“Capture of 'code' with non-sendable type '() async throws(Failure) -> Success' in a @Sendable closure“). Assigning to a variable lets us apply nonisolated(unsafe) to disable the incorrect compiler diagnostic.
5. Several key aspects happen on this line:
   • It’s important to use a detached task, in case we’re already running in an isolated context (e.g. the MainActor) as we’re going to block the current thread waiting on the task to finish, via the semaphore.
   • The task logically needs to be run at the current task’s priority (or higher) in order to ensure it does actually run (re. priority inversion problems), although I’m not sure that technically matters here since we’re blocking in a non-await way anyway. One could use Task.currentPriority here, but I’ve chosen to hard-code the highest priority (userInitiated) because it’s not great to block (in a non-await manner) on async code; although async code isn’t necessarily slow, I feel it’s wise to eliminate task prioritisation as a variable.
   • This closure must be explicitly marked as @Sendable as by default the compiler mistakenly infers it to be not @Sendable, even though all closure arguments to Task initialisers have to be @Sendable. The compiler diagnostics in this case are frustratingly obtuse and misleading (although the sad saving grace is that this is a relatively common problem, so once you hit it enough times you start to develop a spidey sense for it).
6. This otherwise pointless second Task is critical to prevent withoutActuallyEscaping from crashing.
   
   withoutActuallyEscaping basically relies on reference-counting – it records the retain count of its primary argument going in (code in this case) and compares that to the retain count going out – if they disagree, it crashes. There’s no way to disable or directly work around this³.
   
   That’s a problem here because if we just signal the semaphore in the first task, right before exiting the task, we have a race – maybe the task will actually exit before the signal is acted on (semaphore.wait() returns and allows execution to exit the withoutActuallyEscaping block), but maybe it won’t. Since the task is retaining the closure, it must exit before we wake up from the semaphore and exit the withoutActuallyEscaping block, otherwise crash.
   
   The only way I found to ensure the problematic task has fully exited – after hours of experimenting, covering numerous methods – is to wait for it in a second task. Surprisingly, the second task – despite having a strong reference to the first task – seemingly doesn’t prevent the first task from being cleaned up. This makes me suspicious, but despite extensive testing I’m unable to get withoutActuallyEscaping to crash when using this workaround.
7. There’s no practical way to avoid this forced unwrap, even though it’s impossible for it to fail unless something goes very wrong with Swift’s built-ins (like withoutActuallyEscaping and Task).
   
   If you don’t wish to use typed throws, you could unwrap it more gently and throw an error of your own type if it’s nil, but it’s extra work and a loss of type safety for something that realistically cannot happen.

1. Specifically meaning “not run through Swift Concurrency, as async functions / closures”. Lots of APIs will execute the callback on the main thread, which is the most difficult case, but even those that execute on a user-specified GCD queue aren’t helpful here – at least, not until assumeIsolated actually works. ↩︎
2. Incidentally, Wikipedia seems to think the canonical version of the joke is about spherical cows, but I’ve only ever heard it about chickens. Indeed the very first known instance of the joke used chickens, and all the pop-culture uses of it that I could find use chickens (most notably the ninth episode of The Big Bang Theory). ↩︎
3. There’s no unsafeWithoutActuallyEscaping that forgoes the runtime checking, nor any environment variables or similar that influence it; the check is always included and cannot be disabled at runtime. Even when it’s unnecessary or – as in this case – outright erroneous.
   
   Nor is there a way to replicate withoutActuallyEscaping‘s core functionality of merely adding @escaping to the closure’s signature (e.g. via unsafeBitCast or similar) because it’s a special interaction with the compiler’s escape checker which is evaluated purely at compile-time (whether a closure is escaping or not is not actually encoded into the output binary nor memory representation of closures – the only time escapingness ever leaks into the binary is when you use withoutActuallyEscaping and the compiler inserts the special runtime assertion). ↩︎

What NSCopyObject does

Its implementation currently starts with essentially:

id NSCopyObject(id object, NSUInteger extraBytes, NSZone *zone) {
    if (nil == object) {
        return nil;
    }
    
    id copy = object_copy(object, extraBytes);
    object_setClass(copy, objc_opt_class(object));
    return copy;
}

…where object_copy et al are part of the Objective-C runtime. object_copy and its callees are not trivial, so I won’t repeat them here. The key parts are:

1. The malloc in _class_createInstance (allocate space for the copy).
2. The memmove in object_copy (naively copy the raw bytes over).
3. The call from object_copy to fixupCopiedIvars (half-heartedly attempt to fix the damage).

fixupCopiedIvars is notable. It was added by necessity when ARC was introduced to the Objective-C runtime, in Mac OS X 10.6 (Snow Leopard) in 2006. ARC added metadata to Objective-C classes to convey which instance variables were retain-counted object references, so that it could manage them automagically at runtime (not just for copying objects, but more importantly for deallocating them). fixupCopiedIvars uses that metadata to identify things it has to retain (strongly or weakly) in the new copy.

So that should work great, right? The copy operation increments the retain count of all shared objects the new copy references, like you’d expect?

That metadata is incomplete. It only works for Objective-C ivars managed by ARC. i.e. not C++ ivars or Swift stored properties, nor even Objective-C ivars that aren’t using ARC¹.

But I don’t use NSCopyObject…?

Almost nobody intentionally uses NSCopyObject, but your superclass might, and therefore you might. Ever subclassed NSCell or NSAnimation, for example?

I happened to hit this when subclassing NSBitmapImageRep (and I’m very grateful to Kyle Sluder for so quickly identifying the problem – it could have taken me forever to figure it out, otherwise).

If your superclass uses NSCopyObject, it’s now your problem just as much as if you’d used NSCopyObject directly, whether you like it or not.

And even more problematically, whether you know it or not. If your superclass is defined by a 3rd party framework / library, or anything that’s closed source, you might have no idea whether it uses NSCopyObject currently. Worse, you have no control over whether it will or will not use it in future (though anyone that adds a use of NSCopyObject at this point had better hope the atheists are right).

So how do I defend against NSCopyObject?

Objective-C

Pre-ARC it used to be relatively easy to work around this, in Objective-C. You “just” had to manually retain all your subclasses’ reference ivars – and manually copy some others, like non-ref-counted mutable or mortal buffers, etc.

But that generally isn’t possible with ARC – under which you cannot explicitly call retain. Worse:

• There’s still prominent guides scattered about the web that push you unequivocally to use retain, which is not just impossible to do directly under ARC, but flat-out wrong even if you do figure out one of the “clever” ways to do it (you’ll end up over-retaining your ARC-managed references, causing memory leaks).
• There’s also pages lingering on the web that claim that merely turning on ARC will magically solve the problem (it might, but it’s not a panacea).

Some guides specify a better method, which is to manually zero out the copied object’s ivars and then repopulate them via formal property setters. That actually works with or without ARC, although it may break – causing memory leaks – if the superclass ever stops using NSCopyObject (or if NSCopyObject ever gets upgraded to understand reference-counted ivars that it currently does not). It’s also only possible in Objective-C because Swift doesn’t provide direct access to instance variables.

Keep in mind that any reference-typed ivars which are not strong or weak Objective-C objects managed by ARC will still, always need to be handled manually. e.g. pointers to manually-managed memory buffers.

Swift

Ironically, Swift’s attempts to prevent incorrect code actually make it harder to write correct code in this case. What you want to do is – like the Objective-C implementation – to just zero out the references and re-assign them like normal properties. Zeroing them out without triggering a release basically undoes the mistaken memcpy that NSCopyObject did. But Swift won’t let you.

Worse, this has been known for most of Swift’s existence and nothing has been done about it.

Simply setting the property to nil will cause it to be erroneously released, which may immediately deallocate the object and ultimately cause a crash or memory corruption. Even if it doesn’t happen to deallocate the object, it’ll negate the retain you do during the assignment, making all your effort moot.

That’s pretty heavy-handed, though, and not always possible (e.g. NSImageRep, as used by e.g. NSBitmapImageRep, does some special magic in its copy implementation which you cannot practically replicate).

It appears that the best you can do is assume the superclass will always use NSCopyObject, if it does currently, and just manually increment the retain count. Like Objective-C with ARC, the language & standard library really don’t want you to actually do this, but at least in Swift it’s relatively straightforward:

override func copy(with zone: NSZone? = nil) -> Any {
    let result = super.copy(with: zone)
    
    if result.myProperty === self.myProperty {
        _ = Unmanaged.passRetained(myProperty)
    } else {
        result.myProperty = self.myProperty
    }
}

The conditional might help protect you if the superclass stops using NSCopyObject in future – in that case, it’ll probably cause myProperty to default to nil (or to be assigned to some other instance, which you can discard), in which case you just want to assign to it normally.

In the interim – while NSCopyObject is in use, at least – the myProperty pointer will be copied verbatim and you have to assume it requires the extra, manual retain. It’s not future-proof – it’s possible for the superclass to copy the pointer verbatim and increment the retain count for you – but at least in that case you “merely” get a memory leak, rather than a crash or memory corruption.

Do as Apple says, not as Apple does

The most frustrating part of all of this is that this is entirely Apple’s fault. Sure, you can argue it’s not their fault that NeXT added this vile function to Foundation; that Apple “merely” inherited it and were “forced” to keep for backwards compatibility. But it’s entirely Apple’s choice to have kept using it all this time, in their core frameworks, even while they’ve been telling everyone else to never use it.

NSCopyObject has been a known problem-maker pretty much forever – it was a terrible idea right from the outset. Blindly copying the bytes of an object instance, and just hoping that somehow that works correctly – in an object-oriented language derived from Smalltalk where even numbers are often reference types – is farcical.

The introduction of ARC (in 2008) didn’t really help anything, as although it changed NSCopyObject to properly retain ARC-managed ivars, it did nothing for non-ARC-managed ivars (remember that ARC can be enabled in one library but not in another, and libraries can subclass each others’ classes).

NSCopyObject has been officially deprecated since 2012:

The NSCopyObject() function has been deprecated. It has always been a dangerous function to use, except in the implementation of copy methods², and only then with care.

Foundation Release Notes for OS X 10.8 Mountain Lion and iOS 6

…though Apple officially told everyone not to use it in 2008:

This function is dangerous and very difficult to use correctly. It’s [sic] use as part of -copyWithZone: by any class that can be subclassed, is highly error prone. This function is known to fail for objects with embedded retain count ivars, singletons, and C++ ivars, and other circumstances.

Foundation Release Notes for Mac OS X 10.6 Snow Leopard

And this was all still a decade or more after it was known that NSCopyObject was fundamentally evil, e.g. NSCell, and GnuStep’s broken NSControl.

And yet, Apple still use NSCopyObject themselves to this very day, in their own applications and frameworks – including major frameworks like AppKit that almost all 3rd party developers rely on. NSCell is still broken, three decades later, as is NSImage & NSImageRep, and NSAnimation. Most of those are explicitly designed to be subclassed, despite Apple’s own very clear instructions to never mix subclassing with NSCopyObject.

Admittedly it’s not trivial for Apple to remove the NSCopyObject use – alas, because people have had to code myriad hacky workarounds to it, Apple now has to be careful not to break those workarounds. That might even preclude fixing the existing code paths; it might require a replacement copy mechanism. Which leads to…

Tangent: NSCopying considered harmful

The big driver of NSCopyObject use has long been NSCopying. Classes that intend to be subclassed – but also semantically should support copying i.e. NSCopying – have long been making the mistake of thinking that means using NSCopyObject. One need only read the NSCopying documentation, even from before Mac OS X was even publicly released, to see how dangerously fragile and error-prone NSCopying has always been.

Compounding the problem is that NSCopying doesn’t work, by default, on subclasses. You have to override copy(with:) in every subclass³, but the compiler does not enforce this, because in Objective-C (and alas Swift) protocol conformance is assumed inherited even when it cannot correctly be without explicit, extra work by the subclass.

1. Yes, it’s still possible to this day to write Objective-C without using ARC – -fno-objc-arc / CLANG_ENABLE_OBJC_ARC. There might even be valid (albeit unfortunate) use-cases for having to do so, such as for performance.
   
   And even with ARC, it’s of course possible to have pointers to things which aren’t NSObjects and therefore aren’t handled by ARC, such as raw malloc allocations. ↩︎
2. This is false and always has been (that it was safe to use in copy methods). Apple’s false statements in the deprecation notices may ironically have caused even more instances of people using NSCopyObject. ↩︎
3. Any and all that add retain-counted ivars, Swift stored properties, or ivars of C++ types that have destructors. ↩︎

I don’t really remember the timing – sometime between 2006 and 2010 – but presumably around 2006 as I recall it was when Core Data was still relatively new. For whatever reason, there was a huge push internal to Apple to use Core Data everywhere. People were running around all over the place asking “can it be made to use Core Data?”, for Apple’s frameworks and applications.

Keep in mind that Core Data at that time was similar to SwiftData now – very limited functionality, and chock full of bugs. But of course it’s the nature of ‘shiny’ new things that their proponents think it’s the second coming and the cure for all ills.

So, I recall sitting down with a couple of folks from the Core Data team, that were there to see if Shark could adopt Core Data. A little like letting the missionaries in, if only out of morbid curiosity.

Have you heard the good news? Core Data is here to save your very data. It’s effortless and divine and its unintuitive, thread-unsafe API will definitely not be the bane of all its users for the next fifteen years.

Jokes aside, they were in fact earnestly curious if Shark could use Core Data, instead of its own purpose-built binary formats, for storing & querying its profiling data. It was perhaps the classic case of naively underestimating the complexity of a foreign domain. By my recollection, they assumed our profiling data was just a small handful of homogenous, relatively trivial records. “At second N, the program ran the function named XYZ” or somesuch.

I think we (Shark engineers) tried to be open-minded and kind. We were sceptical, but you never know until you actually look. We could see some potential for a more general query capability, for example. But of course the first and most obvious hurdle was: how well does Core Data handle sizeable numbers of records? Oh yes, was the response, it’s great even with tens of thousands of records.

We asked how it did with tens of millions of records, and that was pretty much the end of the conversation.

It wasn’t just the volume of data, it was also the dramatic difference in representation efficiency. The in-memory representation in Shark was basically as efficient as it could be – basically just arrays of compact structs, sometimes with pointers to other arrays (which might share a malloc block to avoid the overhead of small allocations) which were usually just of uint32_t or uint64_t. The most important operations – indexing to an arbitrary point in the profile’s timeline, then scanning forward over the data – were about as fast as they can possibly be.

In contrast, Core Data would have required an entire object (NSManagedObject subclass) for at least every sample, if not every uintXX_t in the callstack (depending on how ‘pure’ you wanted the design to be). It would have increased memory usage by at least an order of magnitude – and Shark already struggled with big profiles on the hardware of the day, which typically had just a couple of GiB of RAM. Even the most trivial operations – like reading the data in from disk and iterating it sequentially would have been thousands of times slower.

In defence of the Core Data folks in the meeting – and I don’t remember who specifically it was – they never tried to misrepresent or exaggerate what Core Data could do. I seem to recall them being quite nice people. But as soon as we started explaining the type and volume of data that we worked with, they clearly gave up on any kind of pitch. Core Data was designed for developer convenience, not runtime efficiency or performance.

It’s never ceased to surprise and disappoint me how many folks try to arbitrarily apply generalised data storage systems – particularly SQLite and MySQL, or wrappers thereover. Usually for the same reasons – perceived convenience to them, right now, not necessarily efficiency (nor the convenience of their successors).

I guess by modern standards SQLite is considered efficient and fast, but – hah – back in my day SQLite was what you used when you didn’t have time to write your own, efficient and fast persistent data management system.

See also JSON and its older sister XML. 😔

Sadly it’s still not quite that easy (and some Python package is squatting on the swift package name 🤨).

You can still install from source, in principle, although I’m not aware of any current instructions on how to do so. The buildSwiftOnARM project has seemingly been abandoned. I haven’t tested if their now-years-old instructions still work for the latest versions of Swift (e.g. 5.10, 6.0).

But, you shouldn’t need to build from source anymore – now you have many better options:

Swift.org (Apple-provided options)

• If you happen to be using a yum-centric platform, like RHEL / Amazon Linux / CentOS, you can (finally!) just install RPMs.
• If you happen to use one of the officially supported Linux distributions, you can install via Docker. Alas many major distros are notably absent from that list, like Debian¹ (and therefore Raspbian). 😔
  
  It’s unfortunate that Swift.org pushes this as the preferred way to install Swift on Linux. I avoid Docker because it tends to just make everything more complex (and error-prone), and encourage bad software practices (“it works in my docker image 🖕”).
  • You can bypass the Docker requirement via the relatively primitive method of splatting a tarball into your system. It works, it’s pretty quick, but it’s a maintenance hassle (no version management, no reliable update method, etc).

Swift Community Apt Repository

Thanks to the efforts of community volunteers – in particular the folks at FutureJones – there is the Swift Community Apt Repository, which fills the massive void left by Apple’s apparent hatred for apt-based distros:

1. curl -s https://archive.swiftlang.xyz/install.sh | sudo bash
2. sudo apt install swiftlang
3. There is no step 3!

More details (including the list of supported distros).

It’s a shame that it falls to third parties to make Swift on Linux easy (on major distros, at least). Don’t forget to support them if you benefit from their work!

1. Though it appears Debian support is in the works. ↩︎

1234.formatted(.byteCount(style: .decimal))

This is just a small subset of Swift’s FormatStyle-based formatting capabilities, with which I have a bit of a love-hate relationship even when they’re working correctly.

Alas, they don’t always work correctly; some of these formatters contain egregious bugs.

In particular, ByteCountFormatStyle pretends to support multiple numeric bases – decimal and binary – but it doesn’t, because what it renders for binary is:

1234.formatted(.byteCount(style: .binary))

Note how it still uses decimal units, “kB”. Decimal is not binary. I mean, duh, right? But apparently Apple don’t know this.

So what?

While the above example is tolerable because, given the rounding applied, the numeric result (“1”) is technically corrected in both bases, see what happens when you use larger values:

Once you get up to non-trivial byte counts, you start to get different results even with the heavy rounding that’s applied by default. By the time you’re talking about GBs the error is on the order of 10%. And the error just gets proportionately larger as the input number increases. Wikipedia has a neat little table showing this.

Okay, but that’s just binary, what about file and memory?

Those are just aliases to decimal and binary, respectively. So memory is right out.

You might think file is still okay, because it essentially means decimal. But that’s only currently. The entire point of its existence is to allow its behaviour to change over time, following whatever convention Apple believes is most appropriate for files. In the past that was in fact binary, as noted previously (pre-Snow Leopard). It might be binary again in future. So it’s dangerous to use since you can neither know what units it’s going to use nor whether Apple will have fixed their formatters by the time it switches off of decimal.

But context will save us!

Probably not. Go look at file sizes in the macOS Finder. Can you tell whether they’re actual decimal (as they appear) or binary?

They’re decimal, but they used to be binary, up until Mac OS X Snow Leopard (10.6). And the Finder used the same (decimal) unit abbreviations throughout. So at least it’s using the correct units now – almost by accident – but it created a confused transition and history.

Worse and more presently pertinent, not everything Finder-like uses decimal, nor correct units when they don’t. e.g. Synology’s products, not to mention countless websites. That creates a lot of confusion which bleeds across into even well-behaved software.

Point being, you both can’t rely on context to help, because context includes things you can’t control like other software, and by getting the units wrong you’re making it worse for everyone, not just yourself.

Why is this still happening?

Going back twenty years, this kind of error – confusing decimal with binary w.r.t. unit prefixes – was both the norm and somewhat excusable – binary prefixes were only formally standardised upon in 1999, so it’s only to be expected that it will take some time for everyone to adopt them.

But it’s been a quarter of a century already. There is absolutely no excuse anymore for getting this wrong.

1. If you’re paying attention you’ll have noticed “k” vs “K”, i.e. the difference in case. This is not arbitrary – it’s because units have to be distinct to be functional, and “K” is the abbreviation for Kelvin (and apparently wins because it’s one of the seven base physical units, from which pretty much all other physical units are derived).
   
   “Ki” is fine because it’s a distinct prefix (and “i” is not a valid abbreviation by itself so there’s no potential confusion about whether it means “Kelvin • i” or “kibi”). ↩︎

Text(progress.formatted(.percent.precision(.significantDigits(2))))

On the pro side:

• They’re better than rolling your own formatting. Not just in terms of convenience, but in terms of correctness – they support localisation, for example. “42%” in English, “٤٢٪؜” in العربية (Arabic), “৪২%” in অসমীয়া (Assamese), “42 %” in Deutsch (German), etc.
• They’re terser than using their otherwise more powerful cousins the Formatters, as they support a “fluent” style of property-based access, which tends to read more naturally and usually avoids having to define variables to hold the formatter.

But the cons are rough:

• They almost always break Xcode’s auto-complete, which is a problem since their syntax is non-trivial and unintuitive.
• They’re hard to understand – and to even find in Apple’s official documentation – because there’s so many protocols and indirection involved.
  
  It’s particularly hard to tell where the inexplicable gaps are. e.g. Double doesn’t support ByteCountFormatStyle, even though logically it should and Xcode will sometimes auto-complete as if it does. Worse, there’s often no valid compiler diagnostic if you try to use them together – you just get a long hang and eventually the dreaded “unable to type check in a reasonable time” cop-out.

All in all, there’s a reason fuckingformatstyle.com¹ exists (and don’t forget to tip!).

1. There’s also the alias goshdarnformatstyle.com if you want to pretend anybody doesn’t know exactly what you actually mean. I assure you that the stronger expletive is by far the most appropriate, once you have experience with FormatStyle. ↩︎

This is kind of a sister function to hard links, which similarly avoid copying the file’s contents but where modifications apply to all copies. See also this article on the differences, including versus aliases and symlinks.

Copy-on-write is beneficial for several reasons:

• Copies don’t take up any significant space (just whatever tiny amount is necessary for their metadata).
• The initial copy operation is practically instantaneous (just a few small metadata writes & updates).
• Deferring (if not entirely avoiding) the actual disk I/O reduces wear on the disk.
• The copies can share common segments, saving disk space even when they’re not ultimately identical copies.

It does have some potential downsides:

• You don’t get the increased data redundancy and error resilience that actual copies provide (although if you’re aiming for data redundancy or backup, you should be using separate physical disks anyway).
• It can make subsequent modifications of the file slower, as even just modifying a single byte can trigger the actual copy to be performed.

And some basic limitations:

• It’s only supported on APFS (and maybe additional file systems added by 3rd party extensions, but I haven’t tested nor can I find any accounts of this).
• It only works within individual volumes (it doesn’t work even between two volumes in the same APFS container, or sharing the same physical disk).

Contrary to what I saw online in a few places, copy-on-write works on all APFS volumes, irrespective of whether they are backed by SSDs, HDDs, or some other type of storage.

Should I use it?

Yes!

For most purposes those downsides aren’t an issue, and the limitations merely mean that it’s wise to have a fallback option (of just copying the actual file contents) whenever copy-on-write isn’t available. And many of the tools & APIs fallback automatically (unless you explicitly require them not to, such as with COPYFILE_CLONE_FORCE to copyfile).

How do I use it?

Testing method

Not that this is interesting, just for posterity and to show my work a bit, in case I made a mistake.

I created large-enough files on each of my test volumes (APFS on SSD, APFS on HDD, HFS+ on SSD) that an actual copy would take tens of seconds at least, using dd e.g.:

I then ran the various command line tools on these files, attempting to clone the file to a different name in the same folder, and observed disk I/O activity with Activity Monitor and iStat Menus.

For actual copy-on-writes I would observe that the program successfully concluded practically instantly, and there’d be at most a small blip of disk writes (for metadata modifications).

For failed copy-on-writes I would observe that the program would not exit promptly and I’d see voluminous disk writes (hundreds of megabytes to gigabytes per second, depending on the disk, sustained for many seconds until I was satisfied with the results and killed the test).

For API tests the overall approach was the same, but the APIs were invoked from inside swift repl where possible, and from a throw-away Swift script otherwise³.

1. I haven’t tested it, but as far as I’ve heard APFS does not actually check if the modifications actually diverge the files. e.g. if you “modify” a byte of the file to the value it already has – a pointless but technically possible operation that leaves both copies still identical – APFS will still copy the modified block.
   
   Furthermore, APFS & Apple’s operating systems appear to have no tools (nor APIs) to deduplicate files – e.g. to detect full or partial copies and deduplicate their actual storage on disk. Not even tools that you could invoke manually if you do the hard work of first determining that two files’ contents are identical.
   
   APFS / Apple’s operating systems rely entirely on user applications using copy-on-write explicitly and upfront. ↩︎
2. Curiously, clonefile first shipped in macOS 10.12 (Sierra), according to its man page, which is the release preceding the introduction of APFS in macOS 10.13 (High Sierra). Yet, as far as I can tell HFS+ doesn’t and never did support cloning – nor do any of the other file systems supported by macOS 10.12 (e.g. FAT16 & FAT32, exFAT, NFS). It’s possible it was just convenient for Apple to include it in the prior release as they were probably testing APFS with it internally, during APFS’s development.
   
   Update: Michael Tsai pointed out that Sierra included APFS support in beta form (e.g. you could create disk images with it, but not use it for a boot disk). Thus why clonefile (and other APFS-related tools) were included in Sierra. ↩︎
3. While the C APIs worked just fine inside swift repl irrespective of what volumes I was targeting, the Foundation APIs oddly refused to work whenever the files were not on the boot volume, throwing “Operation not permitted” errors (NSCocoaErrorDomain 513, with no user info). If it weren’t for the C APIs working just fine I’d think it’s a sandboxing issue, but clearly it’s not (and swift repl -disable-sandbox makes no difference). 🤔 ↩︎

Tangentially, I want to highlight that Axel’s comparison is notable because he is interested in efficiency, not mere brute performance. The two are usually correlated but not always the same. He correctly noted that electricity is a major and increasingly large part of server costs (see my prior post for why it’s even worse than you likely realise). That said, while he did take RAM and power measurements, his benchmark and analysis didn’t go into detail about energy efficiency.

Benchmark method & apparatus

Axel wanted to see how a very simple web server performed in:

• FPM w/ NGINX (PHP).
• Helidon (Kotlin / Java¹).
• Node.js (JavaScript).
• Vapor (Swift).

He was particularly interested in throughput & latency vs RAM & power usage. All are important metrics in their own right, but are most useful in light of each other.

He chose to use Fibonacci sequence calculation as the load. Choosing a load for any web server benchmark is always highly contentious, and not the focus of this post. Whether you think Fibonacci’s a good choice or not, read on to see why really it didn’t matter.

He did use very old hardware, though – an Intel Core i3-550 from over a decade ago. Fortunately it didn’t turn out to materially impact the relative results nor behaviours of the benchmark, but it’s usually unwise to add unnecessary [potential] variables to your setup, like unusual hardware.

In my own debugging and profiling, I used my also very old 10-core iMac Pro. It’s at least a Xeon? 😅

Benchmark results

In words:

• Helidon (Kotlin / Java) had the highest throughput and lowest latency at low (and arguably more reasonable) loads, but used by far the most RAM, and the most power. Consequently it handled the most load before requests started failing (timing out).
• Node.js (JavaScript) was qualitatively very similar to Helidon (Kotlin / Java) but less in all metrics – less throughput, less peak load capacity, but also less RAM and very slightly less power used.
• FPM + NGINX (PHP) followed the pattern.
• Vapor (Swift) did not – it had higher throughput than PHP yet requests started failing much sooner as load increased. It used the least RAM and least power, though, and kept on trucking irrespective of the load.

Many people would have left it at that – obviously the results make sense for the first three (“everyone knows” that Kotlin / Java’s faster than JavaScript that’s faster than PHP) and Vapor / Swift apparently just isn’t fast and has weird reliability behaviours. QED, right?

To his credit, Axel wasn’t so sure – he felt that the results he was seeing were suspicious, and he sought help from the Swift Forums in explaining or correcting them.

While Axel did suspect something was wrong – noting the oddly small but persistent failure rate – he missed the most obvious proof of wrongness – logically impossible results. Doing 80,000 continuous concurrent streams of requests with ~98% of those requests completing within the two second time limit means the server must have a throughput of at least 39,000 requests per second. Yet the benchmark tool reported a mere ~8,000 requests per second.

Sadly, though I pointed this out as the very first response to the thread, it seemed to be overlooked by everyone (even myself!), even though it clearly fingered the benchmark tool itself as the problem (which is only partially correct, as we’ll see later, but in any case was the exact right place to start looking).

Debugging the benchmark

Domain experts weigh in

The Swift Forum post immediately attracted relevant people: folks that work on Vapor and NIO, and folks that have experience using them. However, ironically this didn’t initially help – they tended to assume the problem was in Vapor (or its networking library, SwiftNIO) or how Vapor was being configured. It turned out none of this was really true – there was a small optimisation made to Vapor as a result of all this, which did marginally improve performance (in specific circumstances), but ultimately Vapor & NIO were not the problem, nor was the benchmark’s configuration and use of them.

There were some assertions that the results were plausible and just how Vapor performs, and that the “problem” was the choice of Vapor rather than some other web server framework (e.g. Hummingbird).

Others similarly asserted that the results were plausible because Swift uses reference-counting for memory management whereas PHP, JavaScript, and Kotlin / Java use garbage collection. It was presented as “common knowledge” that garbage collection has inherent benefits for some programs, like web servers, because it makes memory allocation super cheap.

Examining the load

Even though it was clear that something was wrong with the actual measurements, a lot of the early discussion revolved around the load used (Fibonacci sequence calculation), particularly regarding whether it was:

The “right” load

A few folks asserted that the CPU-heavy nature of calculating Fibonacci numbers isn’t representative of web servers generally. Multiple people noted that – in the Swift implementation, at least – the majority of the CPU time was spent doing the Fibonacci calculation. Some felt this was therefore not a useful benchmark of Vapor itself.

A lot of this boiled down to the “no true Scotsman” problem, which is very common in benchmarking, with a bit of perfect world logical fallacy peppered in, trying to identify the One True Representative Benchmark. See the earlier point about fixating on such matters rather than whether the benchmark is useful.

A “fair” load

Accusations were made pretty quickly that the benchmark is “unfair” to Swift because Swift doesn’t – it was asserted – have a properly-optimised “BigInt” implementation, unlike all the other languages tested.

No real evidence was given for this. Even if it were true, it doesn’t invalidate the benchmark – in fact, it just makes the benchmark more successful because it’s then highlighted an area where Swift is lacking.

The BigInt library that Axel used, attaswift/BigInt, is by far the most popular available for Swift, as judged by things like GitHub stars, forks, & contributor counts, ranking in web & GitHub searches, etc. There are quite a few others, though.

It wasn’t until actual evidence was presented, that the discussion made progress.

It was shown that in fact the BigInt implementation in question was significantly slower than it could be, because JavaScript’s implementation of addition was much faster. Some additional simple tests showed even wider performance gaps regarding the other key operation: rendering to strings. It was that data that turned out to be critical – I myself happened to have implemented BigInt string rendering for Apple’s new Foundation, and then saw it dramatically optimised by Oscar Byström Ericsson, whom has his own BigInt package for Swift, Numberick. So I had a pretty darn good idea of where I might find a faster package… 😆

You can read more about that specific bit of serendipity in the Swift Forums thread.

It was trivial to do the package switch, and it quickly improved Vapor/Swift’s showing in the benchmark manyfold – in combination with some other simple and reasonable tweaks, it was five times faster!

a += b
swap(&a, &b)

let c = a + b
a = b
b = c

Axel posted a follow-up with additional data (with the aforementioned changes and optimisations). That showed Swift now beating out the other three frameworks / languages, with the highest throughput and lowest latency (and still the lowest RAM and power usage).

So, all done, right? Turns out, Vapor/Swift wins, yeah?

Well, maybe.

Do these improvements apply to the other cases too?

That is yet to be examined. Because only Swift seemed to be producing odd results, Axel only put the benchmark to the Swift community for deeper analysis. It’s quite possible that doing the same with the other web frameworks & languages would similarly reveal potential improvements.

Still, the results are useful as they stand. Some simple and very plausible – even for a Swift beginner – optimisations made a big difference, though of course the biggest difference was simply using a different 3rd party package. There are a lot of useful lessons in that, both in the specifics as already covered and as general best practices.

…but… why is the success rate still weird?

Despite the improved performance, a fundamental problem remained: the numbers still didn’t make sense.

The success rates are slightly different but not materially – as concurrent requests go up, the throughput plateaus very quickly, yet success rate remains about the same. It’s exactly the same problem as at the outset – these results cannot possibly be correct.

Despite all the community’s efforts, we hadn’t actually figured out the real problem. We’d merely made Swift look better, without actually providing confidence in the accuracy of the results.

In fairness to myself, I was well aware that we weren’t done, I was just struggling to understand what was really going on, as I noted here.

Examining the benchmark tool

While there’d been some tangential questions about wrk, the benchmarking tool Axel used, it had largely been ignored thus far.

Ironically (as you’ll soon see) Axel chose wrk specifically because he didn’t like the behaviour he saw with ApacheBench. Mostly its lack of HTTP/1.1 connection reuse (a subjective but valid methodology choice on Axel’s part) but also because it sounds like he saw some inexplicable results from it too. In hindsight, that might have been a clue that something more pervasive was wrong.

In retrospect there were a few tangential comments in the Swift Forums thread that were on the right track, e.g.:

…when a new connection comes in, the server needs to make a decision: It can

• Either accept the new connection immediately, slowing the existing connections down a little (because now there are more connections to service with the same resources as before)
• Or it can prioritise the existing connections and slow the connection acceptance (increasing the latency of the first request in the new connection which now has to wait).

Johannes Weiss, Swift Forums post

As a little spoiler, it seems apparent that the other three web frameworks all accept incoming connections virtually immediately with priority over any existing connections & request handling (even though they don’t necessarily attempt to serve all those connections’ requests simultaneously). Vapor does not.

Suspicions did [correctly] develop around the opening of the connections themselves, which triggered testing with longer timeouts in a somewhat blind attempt to cover-up the “spurious” first moments of the test.

Characterising the failure mode(s)

Though admittedly I wasn’t fully conscious of what I was doing at the time, the next breakthrough came from simply gathering more data and analysing it qualitatively. This helped in two key ways:

• It better defined and pinned down the circumstances in which things appear to go wrong with the benchmark itself.
  
  It separated out a whole bunch of test configurations that seemingly weren’t interesting (as they behaved in line with intuition / expectations, and similarly across all four web servers).

• It provided hints and potential insight into the nature of the problem.
  
  It showed that there was some kind of variability (in time) in the benchmark’s behaviour, with three very distinct modes (including one which was basically the benchmark actually working as expected, the existence of which had been unknown until that point!).

Interestingly although ultimately only tangentially, this modality finding prompted quite a few “me too!” responses from other folks, about a variety of use-cases involving Vapor or NIO. I took that as affirmation that I was onto something real, but in retrospect that should have been an even better clue: the fact that some people had seen this issue without Vapor involved – the only common denominator was NIO. Even though it turns out NIO itself wasn’t doing any wrong, it was on the right path to answers. This was specifically pointed out to everyone, even.

Overlooked clues

At this point there were a bunch of discussions about benchmark tool configuration, hardware arrangement, whether TLS should be used, etc. I’m going to skim over it, because there’s not much to ultimately say about it – it turned out to not be on the right track in this case, or purely tangential, but it was entirely reasonable to investigate & discuss those aspects. Such is debug life.

What’s interesting is that yet another key clue was mentioned in the Swift Forums thread, yet was overlooked because it was attributed incorrectly and the mechanics miscategorised:

Don’t test with more than 128 connections. You will get read errors. This is due to the file descriptor limit applied to each process on macOS. As @johannesweiss mentioned earlier the default for this is 256. You can change this but it involves disabling the System Integrity Protection.

Adam Fowler, Swift Forums thread

The 128 connections & read errors parts were spot on, in hindsight. But the rest was incorrect (it’s not about the file descriptor ulimit) and in particular the incorrect statement about having to disable SIP perhaps further distracted readers (corrections were posted in reply, which perhaps steered the thread away from what actually mattered).

I’m not sure what precisely the lesson is here… if Adam had better understood the behaviour he’d seen previously (re. 128 connections being the apparent limit) he might have been able to immediately point out one of the key problems. But who can say why he didn’t quite understand that limit correctly, or whether he should have. This sort of thing happens, and maybe it suggests a failure to properly diagnose problems previously, but mostly I’d just point out that the discrepancy here – between 128 and 256 – should have been noticed, and had it been questioned it would have accelerated progress towards the root cause.

Speaking just for myself, I think I (erroneously) dismissed Adam’s comment because I already knew that the default file descriptor limit is not actually 256 (it’s 2,560 on macOS, mostly) and so I assumed the whole comment was wrong and irrelevant.

Another clue was put forth, yet again essentially by accident (without understanding its significance, at the time):

Yes, the reason I used wrk, is that it uses pipelining. That’s why ab (apachebench) had such terrible performance: it opened a new socket for each request. And then it overloaded the system by throwing

• socket: Too many open files
• apr_socket_recv: Connection reset by peer 

errors.

I raised the ulimit -n to 10240, but still apr_socket_recv: Connection reset by peer (104) occurred occasionally.

Axel Roest, Swift Forums thread

This hinted very directly at the second major problem, but it seems nobody in the forum thread realised it. I think there was still a pre-occupation with the file descriptor ulimit.

A little logic applied at the time of Axel’s comment should have revealed its mistaken presumption: that opening new TCP connections for each HTTP request will inevitably cause connection failures. Sure, it will if you give it enough concurrent connection attempts, but real-world web servers operate at huge loads that are basically one HTTP request per connection, without any significant reliability problems. In hindsight, it’s clear that Axel’s dismissal of this behaviour as in any way normal was a mistake – as was everyone else in the thread going along with that dismissal.

A misunderstood workaround

In parallel to all of the above discussion in the Swift Forums thread, I’d been diving into wrk to see what it was really doing. I discovered a way to eliminate the errors: by opening all the TCP connections in advance in a way that happened to limit how many were attempted concurrently by wrk thread count which happened to be low enough in my use of wrk to not hit the magic 128 limit (more on that later). As you can see in my forum post on this, I initially misunderstood how wrk functioned and misattributed the root cause as bugs / bad design in wrk.

In my defence, wrk isn’t written very well, eschewing such outrageous and bourgeois software engineering practices as, you know, actually checking for errors. So it wasn’t unreasonable to believe it was ultimately just broken, given plenty of evidence that it was at least partly broken (which it was & is), but it was ultimately a mistake to let that cloud my judgement of each individual behaviour.

Then again, if I hadn’t been so appalled by the bad code in wrk, and taken it upon myself to rewrite key parts of it, I might not have stumbled onto the above “fix” and therefore also not found the true cause, later.

It’s never the compiler or the kernel… except when it is

At the time I did think I’d actually fixed wrk; I didn’t realise I’d merely found an imperfect workaround. I’d solved the connection errors (not really)! But, I was still curious about one thing – something pretty much everyone had kinda ignored this whole time:

Though those lingering few read/write errors still bother me. I might look into them later.

Me, Swift Forums thread

Tracing those reported errors to their cause was quite a challenge. The only known way to reproduce the errors was to use a very high number of concurrent TCP connections (several thousand), which made it hard to follow any single connection through its lifecycle using any low-brow methods (printf debugging etc). I eventually managed using System Trace² (lamenting, the entire time I used Instruments, that it would have been so much easier in Shark).

Unfortunately, what I was seeing – while in fact correct – did not make sense to me, so I was hesitant to take it on face value.

The lack of any error reporting on the server side, because Vapor lacks it completely, was also both a known problem at the time and also a problem in hindsight. Had Vapor/NIO actually reported the errors they were encountering, it would have partially validated what I was seeing in the system traces – in fact, it would probably have saved me from having to capture & analyse system traces.

Alas I don’t actually remember now precisely what led me to the final answers and root causes. I know it involved many hours of experimenting, exploring hypotheses, and in generally fiddling with everything I could think of.

Somehow or other, I did finally cotton on to a key configuration parameter: kern.ipc.somaxconn.

That controls how many connection requests can be pending (not formally accepted by the server) at one time. It defaults to 128 on macOS. Remember that number, 128?

Once I had figured out that kern.ipc.somaxconn directly controlled the problematic behaviour, the rest followed pretty naturally and quickly – I realised that what I saw in the system traces was in fact accurate, and that in turn revealed that the macOS kernel contains multiple surprisingly blatant and serious bugs (or at the very least dubious design choices, and lying documentation) regarding TCP sockets in non-blocking mode. I wrote that up in some detail in the second half of this Swift Forums post.

As a sidenote, that darn magic number that everyone kept ignoring – 128 – cropped up yet again, in the listen man page, though by the time I saw it there it was merely a confirmation of what I’d already discovered, than a helpful clue. Still, perhaps there’s a lesson there: read the man page. 😆

Conclusion

All told, the major problems identified by the benchmark were (and not all of these were mentioned above, but you can find all the details in the Swift Forums thread):

• The particular 3rd party library used for BigInt support in Swift, attaswift/BigInt, performs quite poorly.
• Vapor would accept too few connections per cycle of its event loop (promptly fixed, in 4.96.0).
• The benchmark tool used, wrk, has numerous bugs:
  • It doesn’t always use the configured number of concurrent connections.
  • It doesn’t measure latency correctly.
  • It doesn’t report errors correctly (in the sense both that it miscategorises them, e.g. connect vs read/write, and that it doesn’t provide enough detail to understand what they are, such as by including the errno).
• The macOS kernel (and seemingly Linux kernel likewise) has multiple bugs:
  • Connection errors are reported incorrectly (as ECONNRESET or EBADF, instead of ECONNREFUSED).
  • kqueue (kevents) behaves as if all connections are always accepted, even when they are not. Put another way, you cannot actually tell if a connection was successful when using non-blocking sockets on macOS.
• Key network configuration on macOS & Linux is way too restrictive:
  • Maximum file descriptors per process is only 2,560 generally on macOS, and even less (256) in GUI apps. It may vary on Linux, but on Axel’s particular server it was 1,024.
  • Maximum number of unaccepted connection requests (the kern.ipc.somaxconn sysctl on macOS, /proc/sys/net/core/somaxconn on Linux) is only 128 on macOS. It may vary on Linux.

It appears that the kernel bugs apply to Linux as well (although it’s not known if kqueue was in use there, as wrk also supports epoll and select), as the behaviour seems to be the same between macOS and Linux.

With the above issues fixed or worked around, his benchmark produces more explicable results (but keep in mind that the difference in connection acceptance behaviour is real and reflects a different design trade-off in Vapor, which may be a problem for real-world use if you don’t raise somaxconn and the listen backlog limit enough).

And that’s all just the problems Axel’s benchmark surfaced – there was a whole host of other interesting lessons taken away from all this (only a fraction of which were highlighted in this post – many more can be found in Axel’s posts and the Swift Forums thread).

Nominally the end result is also a benchmark that shows Vapor (Swift) out-performing other popular web frameworks in other languages. Hugely out-performing them, if you factor in not just throughput & latency but RAM & power usage. But, to reiterate what I pointed out earlier, take that with a grain of salt.

So, for a benchmark that many initially decried as unrealistic or plain poorly conceived, it turned out to be pretty darn useful, I think. And if that doesn’t make it a successful benchmark, I don’t know what does.

Addendum: post title

Looking at the comments about this post on HackerNews etc, I feel like I have to explain the title a little. I was quite pleased with myself when I came up with it (admittedly by accident), because it’s subtle and I think kinda clever, but perhaps too subtle.

“Swift sucks at web serving… or does it?” is a [platonic] double entendre.

On face value it’s alluding to the more typical type of post that is both (a) click-baity and (b) a standard “turns out” story where actually Swift is awesome at web server and haha to all those who doubted it. (where one can replace the word “Swift” with basically any programming technology, because benchmarking brings out some ugly competitiveness from the community)

But really what it means here, if you read the whole post, is that actually we still don’t know. It’s alluding to the oft-overlooked fact that benchmarks are rarely as conclusive as they’re presented. Which I thought was quite clever because it reiterates, at a meta level, my whole point about learning being more important than competing.

At least, that was the idea. 😆

1. Helidon itself is written in Java, but Axel used Kotlin for his little web server implementation – including most crucially the Fibonacci calculations. Both interoperate atop the JVM and plenty of “Java” libraries are partly written in Kotlin, or have dependencies written in Kotlin – and vice versa. A little like Objective-C and Swift interoperate such that many Mac / iDevice apps use a rich mix of both and you don’t typically need to care which language is used for any particular piece. ↩︎
2. I always endeavour to link to the things I mention, but in this case there’s nothing to link to – Apple don’t provide any actual documentation of the System Trace tool in Instruments, and there’s not even any usable 3rd party guide to it, that I can find. It’s a sad demonstration of Apple’s general indifference to performance tools. 😔
   
   Apple don’t even have a proper product page for Instruments itself – the closest you can find is merely its Help. ↩︎

var previous = UIntXL(0)
var current = UIntXL(1)

print(previous)
print(current)

while true {
    let next = previous + current
    previous = current
    current = next

    print(next)
}

Short of using a different algorithm entirely (there are much smarter ways to calculate Fibonacci numbers), you’d think it’d be optimally fast – I mean, how can the above example get any better, really?

Well, by doing this:

var previous = UIntXL(0)
var current = UIntXL(1)

print(previous)
print(current)

while true {
    previous += current
    swap(&previous, &current)

    print(next)
}

Sometimes this version is significantly faster (e.g. 30% in a recent example, and I’ve seen up to 14x in some of my own, similar cases).

Admittedly, sometimes it’s merely as fast – sometimes the Swift compiler optimises the first version into the second for us. But the compiler’s optimiser is unreliable and unpredictable. So if performance is important to you, it’s best to use swap explicitly rather than hang your hopes on the compiler.

What does swap do?

Its purpose is pretty obvious – it swaps the contents of two variables. Logically it’s equivalent to¹:

func swap(_ a: inout T, _ b: inout T) {
    let tmp = a
    a = b
    b = tmp
}

But that’s not how it’s actually implemented. The implementation starts here, though the pertinent details are hidden behind compiler built-ins. Suffice to know that it boils down to simply swapping the raw bytes.

Why is that better?

It avoids unnecessary work: temporary object initialisation and deinitialisation, memory allocation (and frees), reference-counting, etc. That can be from the more efficient swap itself, and/or from mutating one of the values in place rather than creating a temporary third value.

It can also prevent unnecessary copy-on-write behaviour, if you’re mixing the swap in amongst other mutations on value types that implement reference semantics² (like the standard Array, Set, etc). That usually saves time but can also, in some relatively rare circumstances, save memory too (by not leaving duplicate copies of internal buffers in memory indefinitely).

It also works for ~Copyable types without any extra effort.

Is it always optimal?

For actually swapping two values in RAM, yes.

Well, maybe. Mostly? It turns out – after I initially published this post – that it’s a bit more complicated than that. Sometimes an alternative method is slightly faster: the “tuple swap”. e.g.:

var previous = UIntXL(0)
var current = UIntXL(1)

print(previous)
print(current)

while true {
    previous += current
    (previous, current) = (current, previous)

    print(next)
}

In some benchmarks that’s faster, in some it’s slower³. It seems to always be faster than using a named temporary, just like swap, although I don’t believe that’s guaranteed behaviour. Furthermore, it relies on the compiler successfully recognising that particular expression of the swap intent and optimising it appropriately. Plus, using swap is shorter, clearer, and less error-prone. So I think an explicit call to swap is still the best option.

In any case, there are sometimes faster methods which eliminate the need to actually swap the bytes around. e.g. using a pointer to toggle between the two values. To my knowledge, Swift doesn’t provide any way to actually do that for value types, since Swift tries to pretend that pointers don’t exist for them. 😔

Hypothetically the optimiser could perform that optimisation for any code like the above examples, and others, although I’ve never seen it do that.

1. Ignoring support for ~Copyable types, which can be done but requires a more complicated implementation than we need to worry about here. ↩︎
2. Many value types in Swift are not actually just simple sequences of bytes. They often contain pointers to potentially-shared reference objects (e.g. classes or actors), or raw memory buffers. When you copy the value – which includes simply assigning it to a new variable – there can be a lot of work involved in incrementing reference counts for these now-shared internal objects & buffers (which also means time spent decrementing those counts some time later).
   
   And that’s assuming a type that uses copy-on-write (or similar) optimisations – otherwise, you’re going to have to copy all those internal memory buffers as well, which can be tremendously expensive.
   
   Even mere reference counting operations are actually pretty expensive – tens to hundreds of instructions each, typically. Usually in Swift we don’t notice them because the optimiser works very hard to actually remove them, as much as possible. ↩︎
3. I put together some benchmarks in researching & writing this post, with the intent of using them as specific examples of the performance differences. Unfortunately they are apparently too simple, and the optimiser is generally able to optimise the named temporary method into using swap or the “tuple swap” method (even though in real-world code it usually fails to do so, in my experience). ↩︎

There’s two key decisions you must make: which specific URLSession API will you use, and how will you access the bytes themselves.

Measurements

Each benchmark was run a hundred times or for 30 seconds (whichever limit was hit first). I’m highlighting here just the medians (in general there wasn’t much variation anyway), but you can dig into the other percentiles & metrics via the disclosure triangles, if you like.

I’m pretty sure the reads were all served out of the kernel’s in-memory file system cache, judging by the lack of SSD read I/O reported by iStat Menus. But I didn’t go out of my way to verify this.

M2 MacBook Air

bytewise read using bytes(from:) and for loop
╒════════════════════════════╤═════════╤═════════╤═════════╤═════════╤═════════╤═════════╤═════════╤═════════╕
│ Metric                     │      p0 │     p25 │     p50 │     p75 │     p90 │     p99 │    p100 │ Samples │
╞════════════════════════════╪═════════╪═════════╪═════════╪═════════╪═════════╪═════════╪═════════╪═════════╡
│ Malloc (total) (K)         │      35 │      36 │      37 │      37 │      37 │      67 │      69 │     100 │
├────────────────────────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┤
│ Memory (resident peak) (M) │      68 │      91 │      91 │      91 │      94 │      94 │      94 │     100 │
├────────────────────────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┤
│ Releases                   │    1841 │    1869 │    1883 │    1911 │    1925 │    1939 │    1953 │     100 │
├────────────────────────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┤
│ Retains                    │    1312 │    1332 │    1342 │    1362 │    1372 │    1382 │    1392 │     100 │
├────────────────────────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┤
│ Syscalls (total) (K)       │      24 │      25 │      25 │      25 │      25 │      25 │      25 │     100 │
├────────────────────────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┤
│ Time (total CPU) (ms)      │     136 │     137 │     138 │     138 │     138 │     139 │     140 │     100 │
├────────────────────────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┤
│ Time (wall clock) (ms)     │      79 │      79 │      79 │      79 │      80 │      81 │      81 │     100 │
╘════════════════════════════╧═════════╧═════════╧═════════╧═════════╧═════════╧═════════╧═════════╧═════════╛

bytewise read using bytes(from:) and reduce
╒════════════════════════════╤═════════╤═════════╤═════════╤═════════╤═════════╤═════════╤═════════╤═════════╕
│ Metric                     │      p0 │     p25 │     p50 │     p75 │     p90 │     p99 │    p100 │ Samples │
╞════════════════════════════╪═════════╪═════════╪═════════╪═════════╪═════════╪═════════╪═════════╪═════════╡
│ Malloc (total) (K)         │      35 │      36 │      37 │      37 │      37 │      67 │      69 │     100 │
├────────────────────────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┤
│ Memory (resident peak) (M) │      68 │      83 │      84 │      85 │      88 │      88 │      88 │     100 │
├────────────────────────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┤
│ Releases                   │    1840 │    1869 │    1897 │    1911 │    1925 │    1953 │    1967 │     100 │
├────────────────────────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┤
│ Retains                    │    1312 │    1332 │    1352 │    1362 │    1372 │    1392 │    1402 │     100 │
├────────────────────────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┤
│ Syscalls (total) (K)       │      24 │      25 │      25 │      25 │      25 │      25 │      25 │     100 │
├────────────────────────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┤
│ Time (total CPU) (ms)      │     137 │     138 │     138 │     138 │     138 │     139 │     140 │     100 │
├────────────────────────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┤
│ Time (wall clock) (ms)     │      79 │      79 │      79 │      79 │      80 │      80 │      81 │     100 │
╘════════════════════════════╧═════════╧═════════╧═════════╧═════════╧═════════╧═════════╧═════════╧═════════╛

bytewise read using data(from:) and for loop
╒════════════════════════════╤═════════╤═════════╤═════════╤═════════╤═════════╤═════════╤═════════╤═════════╕
│ Metric                     │      p0 │     p25 │     p50 │     p75 │     p90 │     p99 │    p100 │ Samples │
╞════════════════════════════╪═════════╪═════════╪═════════╪═════════╪═════════╪═════════╪═════════╪═════════╡
│ Malloc (total) (K)         │      40 │      43 │      44 │      45 │      47 │      76 │      76 │      50 │
├────────────────────────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┤
│ Memory (resident peak) (M) │     167 │     230 │     265 │     278 │     286 │     297 │     297 │      50 │
├────────────────────────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┤
│ Releases (K)               │    4194 │    4194 │    4194 │    4194 │    4194 │    4194 │    4194 │      50 │
├────────────────────────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┤
│ Retains (K)                │    4194 │    4194 │    4194 │    4194 │    4194 │    4194 │    4194 │      50 │
├────────────────────────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┤
│ Syscalls (total) (K)       │      21 │      23 │      23 │      23 │      23 │      23 │      23 │      50 │
├────────────────────────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┤
│ Time (total CPU) (ms)      │     631 │     639 │     641 │     647 │     659 │     676 │     676 │      50 │
├────────────────────────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┤
│ Time (wall clock) (ms)     │     600 │     603 │     605 │     610 │     625 │     642 │     642 │      50 │
╘════════════════════════════╧═════════╧═════════╧═════════╧═════════╧═════════╧═════════╧═════════╧═════════╛

bytewise read using data(from:) and for loop inside withUnsafeBytes
╒════════════════════════════╤═════════╤═════════╤═════════╤═════════╤═════════╤═════════╤═════════╤═════════╕
│ Metric                     │      p0 │     p25 │     p50 │     p75 │     p90 │     p99 │    p100 │ Samples │
╞════════════════════════════╪═════════╪═════════╪═════════╪═════════╪═════════╪═════════╪═════════╪═════════╡
│ Malloc (total) (K)         │      39 │      44 │      44 │      45 │      47 │      75 │      76 │     100 │
├────────────────────────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┤
│ Memory (resident peak) (M) │     150 │     293 │     338 │     398 │     419 │     439 │     445 │     100 │
├────────────────────────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┤
│ Releases                   │      11 │      11 │      11 │      11 │      11 │      11 │      11 │     100 │
├────────────────────────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┤
│ Retains                    │       5 │       5 │       5 │       5 │       5 │       5 │       5 │     100 │
├────────────────────────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┤
│ Syscalls (total) (K)       │      21 │      22 │      22 │      23 │      23 │      23 │      23 │     100 │
├────────────────────────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┤
│ Time (total CPU) (ms)      │      79 │      93 │      95 │      96 │      98 │     106 │     110 │     100 │
├────────────────────────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┤
│ Time (wall clock) (ms)     │      49 │      59 │      60 │      61 │      62 │      68 │      71 │     100 │
╘════════════════════════════╧═════════╧═════════╧═════════╧═════════╧═════════╧═════════╧═════════╧═════════╛

bytewise read using data(from:) and forEach
╒════════════════════════════╤═════════╤═════════╤═════════╤═════════╤═════════╤═════════╤═════════╤═════════╕
│ Metric                     │      p0 │     p25 │     p50 │     p75 │     p90 │     p99 │    p100 │ Samples │
╞════════════════════════════╪═════════╪═════════╪═════════╪═════════╪═════════╪═════════╪═════════╪═════════╡
│ Malloc (total) (K)         │      41 │      43 │      44 │      45 │      47 │      77 │      77 │      40 │
├────────────────────────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┤
│ Memory (resident peak) (M) │     145 │     228 │     262 │     279 │     292 │     296 │     296 │      40 │
├────────────────────────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┤
│ Releases (K)               │    4194 │    4194 │    4194 │    4194 │    4194 │    4194 │    4194 │      40 │
├────────────────────────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┤
│ Retains (K)                │    4194 │    4194 │    4194 │    4194 │    4194 │    4194 │    4194 │      40 │
├────────────────────────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┤
│ Syscalls (total) (K)       │      22 │      22 │      23 │      23 │      23 │      23 │      23 │      40 │
├────────────────────────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┤
│ Time (total CPU) (ms)      │     782 │     794 │     800 │     805 │     806 │     816 │     816 │      40 │
├────────────────────────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┤
│ Time (wall clock) (ms)     │     750 │     760 │     765 │     771 │     773 │     783 │     783 │      40 │
╘════════════════════════════╧═════════╧═════════╧═════════╧═════════╧═════════╧═════════╧═════════╧═════════╛

bytewise read using data(from:) and reduce
╒════════════════════════════╤═════════╤═════════╤═════════╤═════════╤═════════╤═════════╤═════════╤═════════╕
│ Metric                     │      p0 │     p25 │     p50 │     p75 │     p90 │     p99 │    p100 │ Samples │
╞════════════════════════════╪═════════╪═════════╪═════════╪═════════╪═════════╪═════════╪═════════╪═════════╡
│ Malloc (total) (K)         │      41 │      44 │      44 │      45 │      47 │      77 │      77 │      40 │
├────────────────────────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┤
│ Memory (resident peak) (M) │     165 │     254 │     290 │     338 │     353 │     361 │     361 │      40 │
├────────────────────────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┤
│ Releases (K)               │    4194 │    4194 │    4194 │    4194 │    4194 │    4194 │    4194 │      40 │
├────────────────────────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┤
│ Retains (K)                │    4194 │    4194 │    4194 │    4194 │    4194 │    4194 │    4194 │      40 │
├────────────────────────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┤
│ Syscalls (total) (K)       │      22 │      22 │      22 │      23 │      23 │      23 │      23 │      40 │
├────────────────────────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┤
│ Time (total CPU) (ms)      │     771 │     777 │     784 │     789 │     792 │     796 │     796 │      40 │
├────────────────────────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┤
│ Time (wall clock) (ms)     │     738 │     742 │     750 │     756 │     758 │     760 │     760 │      40 │
╘════════════════════════════╧═════════╧═════════╧═════════╧═════════╧═════════╧═════════╧═════════╧═════════╛

bytewise read using dataTask(with:) and an incremental delegate with for loop
╒════════════════════════════╤═════════╤═════════╤═════════╤═════════╤═════════╤═════════╤═════════╤═════════╕
│ Metric                     │      p0 │     p25 │     p50 │     p75 │     p90 │     p99 │    p100 │ Samples │
╞════════════════════════════╪═════════╪═════════╪═════════╪═════════╪═════════╪═════════╪═════════╪═════════╡
│ Malloc (total) (K)         │      34 │      35 │      36 │      36 │      37 │      69 │      69 │      54 │
├────────────────────────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┤
│ Memory (resident peak) (M) │      52 │      52 │      53 │      57 │      57 │      57 │      57 │      54 │
├────────────────────────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┤
│ Releases (K)               │    4194 │    4194 │    4194 │    4194 │    4194 │    4194 │    4194 │      54 │
├────────────────────────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┤
│ Retains (K)                │    4194 │    4194 │    4194 │    4194 │    4194 │    4194 │    4194 │      54 │
├────────────────────────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┤
│ Syscalls (total) (K)       │      26 │      26 │      27 │      27 │      27 │      27 │      27 │      54 │
├────────────────────────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┤
│ Time (total CPU) (ms)      │     616 │     616 │     617 │     617 │     617 │     620 │     620 │      54 │
├────────────────────────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┤
│ Time (wall clock) (ms)     │     559 │     559 │     560 │     560 │     560 │     564 │     564 │      54 │
╘════════════════════════════╧═════════╧═════════╧═════════╧═════════╧═════════╧═════════╧═════════╧═════════╛

bytewise read using dataTask(with:) and an incremental delegate with for loop inside withUnsafeBytes
╒════════════════════════════╤═════════╤═════════╤═════════╤═════════╤═════════╤═════════╤═════════╤═════════╕
│ Metric                     │      p0 │     p25 │     p50 │     p75 │     p90 │     p99 │    p100 │ Samples │
╞════════════════════════════╪═════════╪═════════╪═════════╪═════════╪═════════╪═════════╪═════════╪═════════╡
│ Malloc (total) (K)         │      47 │      51 │      53 │      55 │      57 │      87 │      87 │     100 │
├────────────────────────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┤
│ Memory (resident peak) (M) │      22 │      26 │      26 │      27 │      27 │      27 │      27 │     100 │
├────────────────────────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┤
│ Releases                   │    6135 │    6147 │    6147 │    6147 │    6150 │    6150 │    6150 │     100 │
├────────────────────────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┤
│ Retains                    │    2046 │    2051 │    2051 │    2051 │    2051 │    2051 │    2051 │     100 │
├────────────────────────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┤
│ Syscalls (total) (K)       │      35 │      35 │      35 │      35 │      35 │      35 │      35 │     100 │
├────────────────────────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┤
│ Time (total CPU) (ms)      │      73 │      74 │      75 │      75 │      76 │      76 │      76 │     100 │
├────────────────────────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┤
│ Time (wall clock) (ms)     │      35 │      36 │      36 │      36 │      36 │      37 │      37 │     100 │
╘════════════════════════════╧═════════╧═════════╧═════════╧═════════╧═════════╧═════════╧═════════╧═════════╛

bytewise read using dataTask(with:) and an incremental delegate with forEach
╒════════════════════════════╤═════════╤═════════╤═════════╤═════════╤═════════╤═════════╤═════════╤═════════╕
│ Metric                     │      p0 │     p25 │     p50 │     p75 │     p90 │     p99 │    p100 │ Samples │
╞════════════════════════════╪═════════╪═════════╪═════════╪═════════╪═════════╪═════════╪═════════╪═════════╡
│ Malloc (total) (K)         │      34 │      35 │      36 │      36 │      37 │      68 │      68 │      42 │
├────────────────────────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┤
│ Memory (resident peak) (M) │      47 │      50 │      51 │      51 │      51 │      51 │      51 │      42 │
├────────────────────────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┤
│ Releases (K)               │    4194 │    4194 │    4194 │    4194 │    4194 │    4194 │    4194 │      42 │
├────────────────────────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┤
│ Retains (K)                │    4194 │    4194 │    4194 │    4194 │    4194 │    4194 │    4194 │      42 │
├────────────────────────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┤
│ Syscalls (total) (K)       │      26 │      27 │      27 │      27 │      27 │      27 │      27 │      42 │
├────────────────────────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┤
│ Time (total CPU) (ms)      │     773 │     774 │     775 │     775 │     776 │     779 │     779 │      42 │
├────────────────────────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┤
│ Time (wall clock) (ms)     │     717 │     718 │     719 │     719 │     719 │     722 │     722 │      42 │
╘════════════════════════════╧═════════╧═════════╧═════════╧═════════╧═════════╧═════════╧═════════╧═════════╛

bytewise read using dataTask(with:) and an incremental delegate with reduce
╒════════════════════════════╤═════════╤═════════╤═════════╤═════════╤═════════╤═════════╤═════════╤═════════╕
│ Metric                     │      p0 │     p25 │     p50 │     p75 │     p90 │     p99 │    p100 │ Samples │
╞════════════════════════════╪═════════╪═════════╪═════════╪═════════╪═════════╪═════════╪═════════╪═════════╡
│ Malloc (total) (K)         │      34 │      35 │      36 │      36 │      37 │      70 │      70 │      43 │
├────────────────────────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┤
│ Memory (resident peak) (M) │      45 │      45 │      45 │      45 │      45 │      45 │      45 │      43 │
├────────────────────────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┤
│ Releases (K)               │    4194 │    4194 │    4194 │    4194 │    4194 │    4194 │    4194 │      43 │
├────────────────────────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┤
│ Retains (K)                │    4194 │    4194 │    4194 │    4194 │    4194 │    4194 │    4194 │      43 │
├────────────────────────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┤
│ Syscalls (total) (K)       │      26 │      26 │      27 │      27 │      27 │      27 │      27 │      43 │
├────────────────────────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┤
│ Time (total CPU) (ms)      │     763 │     764 │     765 │     766 │     766 │     768 │     768 │      43 │
├────────────────────────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┤
│ Time (wall clock) (ms)     │     707 │     709 │     709 │     710 │     710 │     714 │     714 │      43 │
╘════════════════════════════╧═════════╧═════════╧═════════╧═════════╧═════════╧═════════╧═════════╧═════════╛

bytewise read using dataTask(with:completionHandler:) and for loop 
╒════════════════════════════╤═════════╤═════════╤═════════╤═════════╤═════════╤═════════╤═════════╤═════════╕
│ Metric                     │      p0 │     p25 │     p50 │     p75 │     p90 │     p99 │    p100 │ Samples │
╞════════════════════════════╪═════════╪═════════╪═════════╪═════════╪═════════╪═════════╪═════════╪═════════╡
│ Malloc (total) (K)         │      39 │      43 │      44 │      47 │      50 │      79 │      79 │      51 │
├────────────────────────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┤
│ Memory (resident peak) (M) │     378 │     410 │     442 │     471 │     496 │     528 │     528 │      51 │
├────────────────────────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┤
│ Releases (K)               │    4194 │    4194 │    4194 │    4194 │    4194 │    4194 │    4194 │      51 │
├────────────────────────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┤
│ Retains (K)                │    4194 │    4194 │    4194 │    4194 │    4194 │    4194 │    4194 │      51 │
├────────────────────────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┤
│ Syscalls (total) (K)       │      27 │      27 │      28 │      28 │      28 │      28 │      28 │      51 │
├────────────────────────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┤
│ Time (total CPU) (ms)      │     624 │     627 │     630 │     632 │     634 │     640 │     640 │      51 │
├────────────────────────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┤
│ Time (wall clock) (ms)     │     587 │     589 │     590 │     594 │     597 │     601 │     601 │      51 │
╘════════════════════════════╧═════════╧═════════╧═════════╧═════════╧═════════╧═════════╧═════════╧═════════╛

bytewise read using dataTask(with:completionHandler:) and for loop inside withUnsafeBytes
╒════════════════════════════╤═════════╤═════════╤═════════╤═════════╤═════════╤═════════╤═════════╤═════════╕
│ Metric                     │      p0 │     p25 │     p50 │     p75 │     p90 │     p99 │    p100 │ Samples │
╞════════════════════════════╪═════════╪═════════╪═════════╪═════════╪═════════╪═════════╪═════════╪═════════╡
│ Malloc (total) (K)         │      38 │      43 │      45 │      46 │      48 │      76 │      76 │     100 │
├────────────────────────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┤
│ Memory (resident peak) (M) │     394 │     462 │     504 │     525 │     547 │     583 │     587 │     100 │
├────────────────────────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┤
│ Releases                   │       5 │       5 │       5 │       5 │       5 │       5 │       5 │     100 │
├────────────────────────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┤
│ Retains                    │       3 │       3 │       3 │       3 │       3 │       3 │       3 │     100 │
├────────────────────────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┤
│ Syscalls (total) (K)       │      27 │      27 │      28 │      28 │      28 │      28 │      28 │     100 │
├────────────────────────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┤
│ Time (total CPU) (ms)      │      92 │      96 │      98 │     100 │     102 │     105 │     106 │     100 │
├────────────────────────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┤
│ Time (wall clock) (ms)     │      55 │      57 │      57 │      58 │      59 │      61 │      63 │     100 │
╘════════════════════════════╧═════════╧═════════╧═════════╧═════════╧═════════╧═════════╧═════════╧═════════╛

bytewise read using dataTask(with:completionHandler:) and forEach
╒════════════════════════════╤═════════╤═════════╤═════════╤═════════╤═════════╤═════════╤═════════╤═════════╕
│ Metric                     │      p0 │     p25 │     p50 │     p75 │     p90 │     p99 │    p100 │ Samples │
╞════════════════════════════╪═════════╪═════════╪═════════╪═════════╪═════════╪═════════╪═════════╪═════════╡
│ Malloc (total) (K)         │      38 │      41 │      44 │      47 │      51 │      74 │      74 │      41 │
├────────────────────────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┤
│ Memory (resident peak) (M) │     391 │     430 │     470 │     560 │     587 │     617 │     617 │      41 │
├────────────────────────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┤
│ Releases (K)               │    4194 │    4194 │    4194 │    4194 │    4194 │    4194 │    4194 │      41 │
├────────────────────────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┤
│ Retains (K)                │    4194 │    4194 │    4194 │    4194 │    4194 │    4194 │    4194 │      41 │
├────────────────────────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┤
│ Syscalls (total) (K)       │      27 │      28 │      28 │      28 │      28 │      28 │      28 │      41 │
├────────────────────────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┤
│ Time (total CPU) (ms)      │     773 │     776 │     783 │     789 │     790 │     795 │     795 │      41 │
├────────────────────────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┤
│ Time (wall clock) (ms)     │     734 │     737 │     742 │     749 │     751 │     755 │     755 │      41 │
╘════════════════════════════╧═════════╧═════════╧═════════╧═════════╧═════════╧═════════╧═════════╧═════════╛

bytewise read using dataTask(with:completionHandler:) and reduce
╒════════════════════════════╤═════════╤═════════╤═════════╤═════════╤═════════╤═════════╤═════════╤═════════╕
│ Metric                     │      p0 │     p25 │     p50 │     p75 │     p90 │     p99 │    p100 │ Samples │
╞════════════════════════════╪═════════╪═════════╪═════════╪═════════╪═════════╪═════════╪═════════╪═════════╡
│ Malloc (total) (K)         │      38 │      43 │      44 │      46 │      51 │      77 │      77 │      41 │
├────────────────────────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┤
│ Memory (resident peak) (M) │     348 │     408 │     451 │     485 │     499 │     548 │     548 │      41 │
├────────────────────────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┤
│ Releases (K)               │    4194 │    4194 │    4194 │    4194 │    4194 │    4194 │    4194 │      41 │
├────────────────────────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┤
│ Retains (K)                │    4194 │    4194 │    4194 │    4194 │    4194 │    4194 │    4194 │      41 │
├────────────────────────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┤
│ Syscalls (total) (K)       │      27 │      28 │      28 │      28 │      28 │      28 │      28 │      41 │
├────────────────────────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┤
│ Time (total CPU) (ms)      │     770 │     776 │     786 │     788 │     793 │     796 │     796 │      41 │
├────────────────────────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┤
│ Time (wall clock) (ms)     │     732 │     737 │     742 │     749 │     752 │     755 │     755 │      41 │
╘════════════════════════════╧═════════╧═════════╧═════════╧═════════╧═════════╧═════════╧═════════╧═════════╛

10-core iMac Pro

bytewise read using bytes(from:) and for loop
╒════════════════════════════╤═════════╤═════════╤═════════╤═════════╤═════════╤═════════╤═════════╤═════════╕
│ Metric                     │      p0 │     p25 │     p50 │     p75 │     p90 │     p99 │    p100 │ Samples │
╞════════════════════════════╪═════════╪═════════╪═════════╪═════════╪═════════╪═════════╪═════════╪═════════╡
│ Malloc (total) (K)         │      36 │      37 │      37 │      37 │      37 │      68 │      69 │     100 │
├────────────────────────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┤
│ Memory (resident peak) (M) │      45 │      51 │      54 │      56 │      62 │      66 │      66 │     100 │
├────────────────────────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┤
│ Releases                   │    1952 │    2023 │    2051 │    2079 │    2093 │    2135 │    2135 │     100 │
├────────────────────────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┤
│ Retains                    │    1392 │    1442 │    1462 │    1482 │    1492 │    1522 │    1522 │     100 │
├────────────────────────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┤
│ Syscalls (total) (K)       │      21 │      21 │      21 │      21 │      21 │      21 │      21 │     100 │
├────────────────────────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┤
│ Time (total CPU) (ms)      │     245 │     248 │     250 │     251 │     254 │     259 │     267 │     100 │
├────────────────────────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┤
│ Time (wall clock) (ms)     │     136 │     138 │     138 │     140 │     141 │     145 │     146 │     100 │
╘════════════════════════════╧═════════╧═════════╧═════════╧═════════╧═════════╧═════════╧═════════╧═════════╛

bytewise read using bytes(from:) and reduce
╒════════════════════════════╤═════════╤═════════╤═════════╤═════════╤═════════╤═════════╤═════════╤═════════╕
│ Metric                     │      p0 │     p25 │     p50 │     p75 │     p90 │     p99 │    p100 │ Samples │
╞════════════════════════════╪═════════╪═════════╪═════════╪═════════╪═════════╪═════════╪═════════╪═════════╡
│ Malloc (total) (K)         │      36 │      37 │      37 │      37 │      37 │      67 │      69 │     100 │
├────────────────────────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┤
│ Memory (resident peak) (M) │      48 │      52 │      53 │      54 │      56 │      58 │      60 │     100 │
├────────────────────────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┤
│ Releases                   │    1953 │    2009 │    2037 │    2065 │    2079 │    2121 │    2121 │     100 │
├────────────────────────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┤
│ Retains                    │    1392 │    1432 │    1452 │    1472 │    1482 │    1512 │    1512 │     100 │
├────────────────────────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┤
│ Syscalls (total) (K)       │      21 │      21 │      21 │      21 │      21 │      21 │      21 │     100 │
├────────────────────────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┤
│ Time (total CPU) (ms)      │     246 │     249 │     251 │     253 │     256 │     274 │     278 │     100 │
├────────────────────────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┤
│ Time (wall clock) (ms)     │     136 │     138 │     139 │     140 │     141 │     150 │     151 │     100 │
╘════════════════════════════╧═════════╧═════════╧═════════╧═════════╧═════════╧═════════╧═════════╧═════════╛

bytewise read using data(from:) and for loop
╒════════════════════════════╤═════════╤═════════╤═════════╤═════════╤═════════╤═════════╤═════════╤═════════╕
│ Metric                     │      p0 │     p25 │     p50 │     p75 │     p90 │     p99 │    p100 │ Samples │
╞════════════════════════════╪═════════╪═════════╪═════════╪═════════╪═════════╪═════════╪═════════╪═════════╡
│ Malloc (total) (K)         │      41 │      42 │      43 │      43 │      72 │      76 │      76 │      32 │
├────────────────────────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┤
│ Memory (resident peak) (M) │     188 │     243 │     257 │     266 │     279 │     316 │     316 │      32 │
├────────────────────────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┤
│ Releases (K)               │    4194 │    4194 │    4194 │    4194 │    4194 │    4194 │    4194 │      32 │
├────────────────────────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┤
│ Retains (K)                │    4194 │    4194 │    4194 │    4194 │    4194 │    4194 │    4194 │      32 │
├────────────────────────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┤
│ Syscalls (total) (K)       │      18 │      18 │      19 │      19 │      19 │      19 │      19 │      32 │
├────────────────────────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┤
│ Time (total CPU) (ms)      │    1010 │    1015 │    1023 │    1031 │    1040 │    1060 │    1060 │      32 │
├────────────────────────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┤
│ Time (wall clock) (ms)     │     941 │     946 │     953 │     958 │     968 │     985 │     985 │      32 │
╘════════════════════════════╧═════════╧═════════╧═════════╧═════════╧═════════╧═════════╧═════════╧═════════╛

bytewise read using data(from:) and for loop inside withUnsafeBytes
╒════════════════════════════╤═════════╤═════════╤═════════╤═════════╤═════════╤═════════╤═════════╤═════════╕
│ Metric                     │      p0 │     p25 │     p50 │     p75 │     p90 │     p99 │    p100 │ Samples │
╞════════════════════════════╪═════════╪═════════╪═════════╪═════════╪═════════╪═════════╪═════════╪═════════╡
│ Malloc (total) (K)         │      38 │      42 │      43 │      44 │      47 │      72 │      73 │     100 │
├────────────────────────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┤
│ Memory (resident peak) (M) │     244 │     328 │     344 │     352 │     358 │     369 │     370 │     100 │
├────────────────────────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┤
│ Releases                   │      11 │      11 │      11 │      11 │      11 │      11 │      11 │     100 │
├────────────────────────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┤
│ Retains                    │       5 │       5 │       5 │       5 │       5 │       5 │       5 │     100 │
├────────────────────────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┤
│ Syscalls (total) (K)       │      18 │      18 │      18 │      18 │      18 │      19 │      19 │     100 │
├────────────────────────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┤
│ Time (total CPU) (ms)      │     203 │     207 │     208 │     210 │     213 │     221 │     234 │     100 │
├────────────────────────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┤
│ Time (wall clock) (ms)     │     135 │     139 │     140 │     141 │     143 │     148 │     159 │     100 │
╘════════════════════════════╧═════════╧═════════╧═════════╧═════════╧═════════╧═════════╧═════════╧═════════╛

bytewise read using data(from:) and forEach
╒════════════════════════════╤═════════╤═════════╤═════════╤═════════╤═════════╤═════════╤═════════╤═════════╕
│ Metric                     │      p0 │     p25 │     p50 │     p75 │     p90 │     p99 │    p100 │ Samples │
╞════════════════════════════╪═════════╪═════════╪═════════╪═════════╪═════════╪═════════╪═════════╪═════════╡
│ Malloc (total) (K)         │      41 │      42 │      43 │      43 │      73 │      75 │      75 │      26 │
├────────────────────────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┤
│ Memory (resident peak) (M) │     171 │     225 │     236 │     263 │     283 │     288 │     288 │      26 │
├────────────────────────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┤
│ Releases (K)               │    4194 │    4194 │    4194 │    4194 │    4194 │    4194 │    4194 │      26 │
├────────────────────────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┤
│ Retains (K)                │    4194 │    4194 │    4194 │    4194 │    4194 │    4194 │    4194 │      26 │
├────────────────────────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┤
│ Syscalls (total) (K)       │      18 │      18 │      18 │      19 │      19 │      19 │      19 │      26 │
├────────────────────────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┤
│ Time (total CPU) (ms)      │    1247 │    1250 │    1254 │    1266 │    1271 │    1284 │    1284 │      26 │
├────────────────────────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┤
│ Time (wall clock) (ms)     │    1177 │    1179 │    1181 │    1194 │    1199 │    1211 │    1211 │      26 │
╘════════════════════════════╧═════════╧═════════╧═════════╧═════════╧═════════╧═════════╧═════════╧═════════╛

bytewise read using data(from:) and reduce
╒════════════════════════════╤═════════╤═════════╤═════════╤═════════╤═════════╤═════════╤═════════╤═════════╕
│ Metric                     │      p0 │     p25 │     p50 │     p75 │     p90 │     p99 │    p100 │ Samples │
╞════════════════════════════╪═════════╪═════════╪═════════╪═════════╪═════════╪═════════╪═════════╪═════════╡
│ Malloc (total) (K)         │      39 │      42 │      43 │      43 │      72 │      79 │      79 │      26 │
├────────────────────────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┤
│ Memory (resident peak) (M) │     207 │     226 │     232 │     243 │     260 │     278 │     278 │      26 │
├────────────────────────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┤
│ Releases (K)               │    4194 │    4194 │    4194 │    4194 │    4194 │    4194 │    4194 │      26 │
├────────────────────────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┤
│ Retains (K)                │    4194 │    4194 │    4194 │    4194 │    4194 │    4194 │    4194 │      26 │
├────────────────────────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┤
│ Syscalls (total) (K)       │      18 │      18 │      18 │      19 │      19 │      19 │      19 │      26 │
├────────────────────────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┤
│ Time (total CPU) (ms)      │    1225 │    1228 │    1233 │    1243 │    1250 │    1269 │    1269 │      26 │
├────────────────────────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┤
│ Time (wall clock) (ms)     │    1157 │    1160 │    1163 │    1172 │    1178 │    1198 │    1198 │      26 │
╘════════════════════════════╧═════════╧═════════╧═════════╧═════════╧═════════╧═════════╧═════════╧═════════╛

bytewise read using dataTask(with:) and an incremental delegate with for loop
╒════════════════════════════╤═════════╤═════════╤═════════╤═════════╤═════════╤═════════╤═════════╤═════════╕
│ Metric                     │      p0 │     p25 │     p50 │     p75 │     p90 │     p99 │    p100 │ Samples │
╞════════════════════════════╪═════════╪═════════╪═════════╪═════════╪═════════╪═════════╪═════════╪═════════╡
│ Malloc (total) (K)         │      35 │      35 │      36 │      36 │      65 │      68 │      68 │      36 │
├────────────────────────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┤
│ Memory (resident peak) (M) │      32 │      36 │      40 │      45 │      48 │      52 │      52 │      36 │
├────────────────────────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┤
│ Releases (K)               │    4194 │    4194 │    4194 │    4194 │    4194 │    4194 │    4194 │      36 │
├────────────────────────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┤
│ Retains (K)                │    4194 │    4194 │    4194 │    4194 │    4194 │    4194 │    4194 │      36 │
├────────────────────────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┤
│ Syscalls (total) (K)       │      23 │      23 │      23 │      23 │      23 │      23 │      23 │      36 │
├────────────────────────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┤
│ Time (total CPU) (ms)      │     954 │     958 │     964 │     965 │     975 │     992 │     992 │      36 │
├────────────────────────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┤
│ Time (wall clock) (ms)     │     841 │     845 │     848 │     851 │     857 │     871 │     871 │      36 │
╘════════════════════════════╧═════════╧═════════╧═════════╧═════════╧═════════╧═════════╧═════════╧═════════╛

bytewise read using dataTask(with:) and an incremental delegate with for loop inside withUnsafeBytes
╒════════════════════════════╤═════════╤═════════╤═════════╤═════════╤═════════╤═════════╤═════════╤═════════╕
│ Metric                     │      p0 │     p25 │     p50 │     p75 │     p90 │     p99 │    p100 │ Samples │
╞════════════════════════════╪═════════╪═════════╪═════════╪═════════╪═════════╪═════════╪═════════╪═════════╡
│ Malloc (total) (K)         │      47 │      51 │      53 │      54 │      56 │      83 │      84 │     100 │
├────────────────────────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┤
│ Memory (resident peak) (M) │      17 │      20 │      23 │      24 │      25 │      25 │      25 │     100 │
├────────────────────────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┤
│ Releases                   │    5895 │    6067 │    6087 │    6099 │    6111 │    6123 │    6123 │     100 │
├────────────────────────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┤
│ Retains                    │    1966 │    2024 │    2031 │    2035 │    2039 │    2043 │    2043 │     100 │
├────────────────────────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┤
│ Syscalls (total) (K)       │      32 │      32 │      32 │      32 │      33 │      33 │      33 │     100 │
├────────────────────────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┤
│ Time (total CPU) (ms)      │     137 │     139 │     140 │     142 │     147 │     152 │     155 │     100 │
├────────────────────────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┤
│ Time (wall clock) (ms)     │      55 │      55 │      56 │      56 │      58 │      60 │      62 │     100 │
╘════════════════════════════╧═════════╧═════════╧═════════╧═════════╧═════════╧═════════╧═════════╧═════════╛

bytewise read using dataTask(with:) and an incremental delegate with forEach
╒════════════════════════════╤═════════╤═════════╤═════════╤═════════╤═════════╤═════════╤═════════╤═════════╕
│ Metric                     │      p0 │     p25 │     p50 │     p75 │     p90 │     p99 │    p100 │ Samples │
╞════════════════════════════╪═════════╪═════════╪═════════╪═════════╪═════════╪═════════╪═════════╪═════════╡
│ Malloc (total) (K)         │      35 │      35 │      36 │      36 │      65 │      69 │      69 │      29 │
├────────────────────────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┤
│ Memory (resident peak) (M) │      29 │      33 │      35 │      37 │      39 │      41 │      41 │      29 │
├────────────────────────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┤
│ Releases (K)               │    4194 │    4194 │    4194 │    4194 │    4194 │    4194 │    4194 │      29 │
├────────────────────────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┤
│ Retains (K)                │    4194 │    4194 │    4194 │    4194 │    4194 │    4194 │    4194 │      29 │
├────────────────────────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┤
│ Syscalls (total) (K)       │      23 │      23 │      23 │      23 │      23 │      23 │      23 │      29 │
├────────────────────────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┤
│ Time (total CPU) (ms)      │    1169 │    1179 │    1181 │    1184 │    1190 │    1205 │    1205 │      29 │
├────────────────────────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┤
│ Time (wall clock) (ms)     │    1056 │    1065 │    1066 │    1069 │    1074 │    1086 │    1086 │      29 │
╘════════════════════════════╧═════════╧═════════╧═════════╧═════════╧═════════╧═════════╧═════════╧═════════╛

bytewise read using dataTask(with:) and an incremental delegate with reduce
╒════════════════════════════╤═════════╤═════════╤═════════╤═════════╤═════════╤═════════╤═════════╤═════════╕
│ Metric                     │      p0 │     p25 │     p50 │     p75 │     p90 │     p99 │    p100 │ Samples │
╞════════════════════════════╪═════════╪═════════╪═════════╪═════════╪═════════╪═════════╪═════════╪═════════╡
│ Malloc (total) (K)         │      35 │      35 │      35 │      36 │      65 │      68 │      68 │      28 │
├────────────────────────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┤
│ Memory (resident peak) (M) │      30 │      35 │      43 │      46 │      48 │      52 │      52 │      28 │
├────────────────────────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┤
│ Releases (K)               │    4194 │    4194 │    4194 │    4194 │    4194 │    4194 │    4194 │      28 │
├────────────────────────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┤
│ Retains (K)                │    4194 │    4194 │    4194 │    4194 │    4194 │    4194 │    4194 │      28 │
├────────────────────────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┤
│ Syscalls (total) (K)       │      23 │      23 │      23 │      23 │      23 │      23 │      23 │      28 │
├────────────────────────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┤
│ Time (total CPU) (ms)      │    1176 │    1182 │    1185 │    1190 │    1198 │    1200 │    1200 │      28 │
├────────────────────────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┤
│ Time (wall clock) (ms)     │    1061 │    1067 │    1072 │    1075 │    1080 │    1083 │    1083 │      28 │
╘════════════════════════════╧═════════╧═════════╧═════════╧═════════╧═════════╧═════════╧═════════╧═════════╛

bytewise read using dataTask(with:completionHandler:) and for loop 
╒════════════════════════════╤═════════╤═════════╤═════════╤═════════╤═════════╤═════════╤═════════╤═════════╕
│ Metric                     │      p0 │     p25 │     p50 │     p75 │     p90 │     p99 │    p100 │ Samples │
╞════════════════════════════╪═════════╪═════════╪═════════╪═════════╪═════════╪═════════╪═════════╪═════════╡
│ Malloc (total) (K)         │      40 │      42 │      43 │      43 │      72 │      75 │      75 │      32 │
├────────────────────────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┤
│ Memory (resident peak) (M) │     356 │     368 │     375 │     386 │     400 │     407 │     407 │      32 │
├────────────────────────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┤
│ Releases (K)               │    4194 │    4194 │    4194 │    4194 │    4194 │    4194 │    4194 │      32 │
├────────────────────────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┤
│ Retains (K)                │    4194 │    4194 │    4194 │    4194 │    4194 │    4194 │    4194 │      32 │
├────────────────────────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┤
│ Syscalls (total) (K)       │      23 │      24 │      24 │      24 │      24 │      24 │      24 │      32 │
├────────────────────────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┤
│ Time (total CPU) (ms)      │    1016 │    1023 │    1026 │    1032 │    1041 │    1048 │    1048 │      32 │
├────────────────────────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┤
│ Time (wall clock) (ms)     │     940 │     946 │     948 │     952 │     961 │     967 │     967 │      32 │
╘════════════════════════════╧═════════╧═════════╧═════════╧═════════╧═════════╧═════════╧═════════╧═════════╛

bytewise read using dataTask(with:completionHandler:) and for loop inside withUnsafeBytes
╒════════════════════════════╤═════════╤═════════╤═════════╤═════════╤═════════╤═════════╤═════════╤═════════╕
│ Metric                     │      p0 │     p25 │     p50 │     p75 │     p90 │     p99 │    p100 │ Samples │
╞════════════════════════════╪═════════╪═════════╪═════════╪═════════╪═════════╪═════════╪═════════╪═════════╡
│ Malloc (total) (K)         │      40 │      42 │      43 │      43 │      44 │      74 │      74 │     100 │
├────────────────────────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┤
│ Memory (resident peak) (M) │     370 │     580 │     591 │     599 │     607 │     613 │     614 │     100 │
├────────────────────────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┤
│ Releases                   │       5 │       5 │       5 │       5 │       5 │       5 │       5 │     100 │
├────────────────────────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┤
│ Retains                    │       3 │       3 │       3 │       3 │       3 │       3 │       3 │     100 │
├────────────────────────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┤
│ Syscalls (total) (K)       │      23 │      23 │      24 │      24 │      24 │      24 │      24 │     100 │
├────────────────────────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┤
│ Time (total CPU) (ms)      │     210 │     213 │     215 │     217 │     220 │     236 │     238 │     100 │
├────────────────────────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┤
│ Time (wall clock) (ms)     │     133 │     135 │     137 │     138 │     140 │     153 │     154 │     100 │
╘════════════════════════════╧═════════╧═════════╧═════════╧═════════╧═════════╧═════════╧═════════╧═════════╛

bytewise read using dataTask(with:completionHandler:) and forEach
╒════════════════════════════╤═════════╤═════════╤═════════╤═════════╤═════════╤═════════╤═════════╤═════════╕
│ Metric                     │      p0 │     p25 │     p50 │     p75 │     p90 │     p99 │    p100 │ Samples │
╞════════════════════════════╪═════════╪═════════╪═════════╪═════════╪═════════╪═════════╪═════════╪═════════╡
│ Malloc (total) (K)         │      41 │      42 │      42 │      43 │      71 │      75 │      75 │      26 │
├────────────────────────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┤
│ Memory (resident peak) (M) │     332 │     364 │     370 │     384 │     415 │     418 │     418 │      26 │
├────────────────────────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┤
│ Releases (K)               │    4194 │    4194 │    4194 │    4194 │    4194 │    4194 │    4194 │      26 │
├────────────────────────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┤
│ Retains (K)                │    4194 │    4194 │    4194 │    4194 │    4194 │    4194 │    4194 │      26 │
├────────────────────────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┤
│ Syscalls (total) (K)       │      23 │      24 │      24 │      24 │      24 │      24 │      24 │      26 │
├────────────────────────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┤
│ Time (total CPU) (ms)      │    1243 │    1253 │    1258 │    1268 │    1270 │    1277 │    1277 │      26 │
├────────────────────────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┤
│ Time (wall clock) (ms)     │    1164 │    1175 │    1179 │    1188 │    1190 │    1201 │    1201 │      26 │
╘════════════════════════════╧═════════╧═════════╧═════════╧═════════╧═════════╧═════════╧═════════╧═════════╛

bytewise read using dataTask(with:completionHandler:) and reduce
╒════════════════════════════╤═════════╤═════════╤═════════╤═════════╤═════════╤═════════╤═════════╤═════════╕
│ Metric                     │      p0 │     p25 │     p50 │     p75 │     p90 │     p99 │    p100 │ Samples │
╞════════════════════════════╪═════════╪═════════╪═════════╪═════════╪═════════╪═════════╪═════════╪═════════╡
│ Malloc (total) (K)         │      40 │      42 │      42 │      43 │      72 │      75 │      75 │      26 │
├────────────────────────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┤
│ Memory (resident peak) (M) │     353 │     392 │     416 │     452 │     472 │     473 │     473 │      26 │
├────────────────────────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┤
│ Releases (K)               │    4194 │    4194 │    4194 │    4194 │    4194 │    4194 │    4194 │      26 │
├────────────────────────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┤
│ Retains (K)                │    4194 │    4194 │    4194 │    4194 │    4194 │    4194 │    4194 │      26 │
├────────────────────────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┤
│ Syscalls (total) (K)       │      23 │      24 │      24 │      24 │      24 │      24 │      24 │      26 │
├────────────────────────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┤
│ Time (total CPU) (ms)      │    1241 │    1250 │    1254 │    1266 │    1278 │    1281 │    1281 │      26 │
├────────────────────────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┤
│ Time (wall clock) (ms)     │    1164 │    1172 │    1176 │    1185 │    1195 │    1200 │    1200 │      26 │
╘════════════════════════════╧═════════╧═════════╧═════════╧═════════╧═════════╧═════════╧═════════╧═════════╛

Observations

Incremental reads + withUnsafeBytes is unequivocally the best method

It’s dramatically faster than any other approach – both in wall time and overall CPU usage – and uses the least amount of memory by far.

Within the Data-centric methods this doesn’t surprise me – with the incremental approach URLSession can just hand data back as it comes in, in whatever chunk sizes are most convenient. In the other Data-based approaches it has to aggregate everything into one final contiguous blob.

Implementation note

I utilised the incremental API by basically adapting it to invoke a closure (shown below), mainly because it made it easier to then test different byte enumeration approaches within it, but the results should hold for typical implementations of URLSessionDataDelegate.

class IncrementalDataDelegate: NSObject, URLSessionDataDelegate {
    private let task: URLSessionTask
    private let handler: (Data) -> ()
    private let done = NSCondition()

    init(_ task: URLSessionTask,
         handler: @escaping (Data) -> ()) {
        self.task = task
        self.handler = handler
        super.init()
    }

    func urlSession(_ session: URLSession, dataTask: URLSessionDataTask, didReceive data: Data) {
        precondition(self.task == dataTask)
        self.handler(data)
    }

    func urlSession(_ session: URLSession, task: URLSessionTask, didCompleteWithError error: (any Error)?) {
        precondition(self.task == task)

        if let error {
            preconditionFailure("Error: \(error)")
        }

        self.done.broadcast()
    }

    func wait() {
        self.done.wait()
    }
}

AsyncBytes (bytes(from:)) is surprisingly not bad – in this specific case

This was surprising to me because generally I’ve seen Swift’s AsyncSequence stuff – especially for operating on individual bytes – being unusably slow and inefficient. It’s actually what prompted me to do these benchmarks, because I stubbornly tried using bytes(from:) in a current project and the performance in that real app was god-awful. These benchmarks demonstrate that it doesn’t necessarily have to be, and there’s something more complicated going on. I’m yet to get to the bottom of that.

The problem seems to be that async code in general – but especially anything involving AsyncSequences – is terribly dependent on the compiler’s optimiser. If the optimiser does anything less than an astounding job on it, the performance can drop off a cliff.

So, while these results nominally recommend bytes(from:) as a decent way to use URLSession – being a respectable second-fastest in these benchmarks and noticeably easier to use than the fastest method – I’d be very cautious about it and test the performance early and often.

withUnsafeBytes is way faster than “safe” access to Data‘s contents

It’s an order of magnitude faster an Apple Silicon, and ‘merely’ seven to nine times faster on Intel.

This isn’t surprising – Data‘s regular APIs involve actual function calls (if not also Objective-C message sends, depending on what exactly is being returned by URLSession (native Data or actual NSData) and how Swift imports NSData from Objective-C. withUnsafeBytes provides basically direct memory access, with practically zero overhead.

However, it has one notable downside…

withUnsafeBytes doubles the memory usage of the target Data

This surprised and disappointed me – Data is supposed to already be a contiguous array of bytes, internally, so accessing those bytes with withUnsafeBytes should be nothing more than returning a pointer to that internal storage. But in at least some cases, it doesn’t – instead, it allocates a whole new memory allocation, copies its contents to that allocation, and then provides that instead (and releases it afterwards – so repeated calls will incur this overhead every time).

This isn’t a huge issue when the Datas in question are small – clearly it doesn’t hamper performance all that much, since it’s still the fastest way to access the bytes of even a 128 MiB Data – but it can be an issue when the Datas in question are not small. If you run out of free RAM, the cost of the kernel’s in-memory compression or swapping is very likely going to cripple the performance far beyond the degrees seen here by using the slow APIs.

Reading a file with URLSession takes more than one CPU core

I pointedly included both wall time and CPU time to highlight that there’s multiple cores engaged simultaneously for a single read. This isn’t surprising, but it’s important to remember if you’re doing lots of parallel I/O – you can’t just naively allocate one read operation per CPU core and expect linear scaling (notwithstanding CPU frequency scaling etc anyway).

Though, that’s generally true anyway because most systems don’t have enough disk or network I/O to keep up with the CPU anyway.

for loops are faster than forEach & reduce

This might surprise some folks. It’s surely a bit of a sore point with functional programming dogmatists. The difference isn’t massive – in these benchmarks it’s only about 20%. Still, it’s measurable and noticeable.

I find the for-loop approach easier to write and read anyway, so IMO this is just another reason to favour that instead of functional programming styles. But not a reason to unilaterally favour one over the other.

forEach & reduce perform the same

Not surprising or news, but worth noting. In principle the optimiser should reduce them to the exact same machine code in the end.

Similar performance characteristics between [old] Intel Xeons and Apple Silicon

The M2 was faster, of course, but maybe not as much faster than one would expect – at best only twice as fast, which (subjectively) feels underwhelming given Apple’s numerous manufacturing and design advantages versus my iMac Pro’s old Xeon.

What I mean, though, is that the relative performance of the different methods is about the same irrespective of the platform. What’s good (or bad) on an M2 is likewise on an Intel CPU. Which is worth appreciating – although x86 is rapidly fading into irrelevance, it’s not quite there yet, and it’s always unpleasant when optimising for one architecture has the opposite effect on another.

You’re probably not going to be I/O limited on Apple SSDs

(for a single read at a time, that is)

Even in the very best case shown here – and despite the very light workload these benchmarks impose, on the actual file data – the best throughput was merely ~3.6 GB/s. That’s not bad, of course – only a few years ago that would have easily saturated any consumer storage device, even a big Thunderbolt RAID array of SSDs. But these days most of Apple’s computers have PCIe 4, quad-lane SSDs that have read speeds of about 7 GB/s.

And, this is all ignoring the fact that actually reading from an SSD has more overhead than merely reading from the kernel’s file system cache, as these benchmarks almost certainly did. So actual SSD read performance is very likely lower than what these benchmarks achieved.

That all said, it doesn’t necessarily take much to be I/O-limited – two or three concurrent reads, efficiently implemented, would probably do it. Certainly if you just spin up an operation per CPU core and they all try to do I/O at once, even a rather inefficient implementation will hit the SSD’s limits.

In absolute terms they don’t have much overhead – think sub-microsecond for most uses. Which makes them perfectly acceptable when they’re used sporadically (e.g. only a few times per second).

However, if you need to deal with time and timing more frequently, their inefficiency can become a serious bottleneck.

I stumbled into this because of a fairly common and otherwise uninteresting pattern – throttling UI updates on an I/O operation’s progress. This might look something like:

struct Example: View {
    let bytes: AsyncSequence<UInt8>

    @State var byteCount = 0

    var body: some View {
        Text("Bytes so far: \(byteCount.formatted(.byteCount(style: .binary)))")
            .task {
                var unpostedByteCount = 0
                let clock = ContinuousClock()
                var lastUpdate = clock.now

                for try await byte in bytes {
                    … // Do something with the byte.

                    unpostedByteCount += 1

                    let now = clock.now
                    let delta = now - lastUpdate

                    if (    delta > .seconds(1)
                         || (    (delta > .milliseconds(100)
                              && 1_000_000 <= unpostedByteCount))) {
                        byteCount += unpostedByteCount
                        unpostedByteCount = 0
                        lastUpdate = now
                    }
                }
            }
    }
}

The above seems fairly straightforward, but if you run it and have any non-trivial I/O rate – even just a few hundred kilobytes per second – you’ll find that it saturates an entire CPU core, not just wasting CPU time but limiting the I/O rate severely.

Using a SuspendingClock makes no difference.

In a nutshell, the problem is that Swift’s Clock protocol has significant overheads by design¹. If you look at a time profile of code like this, you’ll see things like:

That’s a lot of time wasted in function calls and struct initialisation and type conversion and protocol witnesses and all that guff. The only part that’s actually retrieving the time is the swift_get_time call (which is just a wrapper over clock_gettime, which is just a wrapper over clock_gettime_nsec_np(CLOCK_UPTIME_RAW), which is just a wrapper over mach_absolute_time).

I wrote some simple benchmarks of various alternative time-tracking methods, with these results with Swift 5.10 (showing the median runtime of the benchmark, which is a million iterations of checking the time):

All these alternative methods are well over an order of magnitude faster than Swift’s native clock APIs, showing just how dreadfully inefficient the Swift Clock API is.

mach_absolute_time for the win

Unsurprisingly, mach_absolute_time is the fastest. It is what all these other APIs are actually based on; it is the lowest level of the time stack.

The downside to calling mach_absolute_time directly, though, is that it’s on Apple’s “naughty” list – apparently it’s been abused for device fingerprinting, so Apple require you to beg for special permission if you want to use it (even though it’s used by all these other APIs anyway, as the basis for their implementations, and there’s nothing you can get from mach_absolute_time that you can’t get from them too 🤨).

Date surprisingly not bad

I was quite surprised to see good ol’ Date performing competitively with the traditional C-level APIs, at least on x86-64. Even on arm64 it’s not bad, at still a third to half the speed of the C APIs. This surprised me because it has the overhead of at least one Objective-C message send (for timeIntervalSinceNow), unless somehow the Swift compiler is optimising that into a static function call, or inlining it entirely…?

I certainly wouldn’t be afraid to use Date broadly, going down to lower APIs only when truly necessary – which is pretty rarely, I’d wager; we’re talking a mere 19 to 30 nanoseconds to get the time elapsed since a reference date and compare it to a threshold. If that’s too slow, it might be an indication that there’s a bigger problem (like transferring data a single byte at a time, as in the example that started this post – but more on that in the next post).

Follow-up

This post got some attention on HackerNews. Pleasingly, the comments there were almost all well-intentioned and interesting. It’s a bit beyond me to try to address all of them, but a few in particular raised good points that I would like to answer / clarify:

• A lot of folks were curious about mach_absolute_time being on Apple’s naughty list. I don’t know for sure why it is either, but I think it’s very likely that it’s primarily because it essentially provides a reference time point, that’s very precise and pretty unique between computers. It’s not the boot time necessarily – because the timer pauses whenever the system is put to sleep – but even so it provides a simple way to nearly if not exactly identify an individual machine session (between boots & sleeps). It probably wouldn’t take many other fingerprinting data points to reliably pin-point a specific machine.
  
  Secondarily, because it provides very precise timing capabilities (e.g. nanosecond-resolution on x86), it could possibly be a key component of timing attacks and broader device fingerprinting based on timing information (e.g. measuring how long it takes to perform an otherwise innocuous operation).
  
  That all said, the only difference between it and some of the higher-level APIs wrapping it is their overhead. And it’s not apparent to me that merely making the “get-time” functionality 2x slower is going to magically mitigate all the above concerns, especially when we’re still talking just a few nanoseconds.
• Admittedly my phrasing regarding Apple’s policies on mach_absolute_time – “beg for permission to use it” – is a little melodramatic. It’s revealing something of my personal opinions on certain Apple “security” practices. I love that Apple genuinely care about protecting everyone’s privacy, but sometimes I chaff at what feels like capricious or impractical specific policies.
  
  In this particular case, it’s not apparent to me why this sort of protection is needed for native apps. In a web browser, sure, you’re running untrustworthy, essentially arbitrary code from all over the place, a lot of which is openly malicious (thanks, Google & Facebook, for your pervasive trackers – fuck you too). But a native app – or heck, even a dodgy non-native one like an Electron app – must be explicitly installed by the end user, among other barriers like code signing.
• A few folks looked at the example case, of iterating a single byte at a time, and were suspicious of how performant that could possibly be anyway. This is a very fair reaction – it’s my ingrained instinct as well, from years of C/C++/Objective-C – but it’s relying on a few outdated assumptions. My next post already covered this for the most part, but in short here:
  
  Through inlining, that code basically optimises down to an outer loop that fetches a new chunk of data (a pointer & length) plus an inner loop to iterate over that as direct memory access. The chunks are typically tens of kilobytes to megabytes, in my experience (depending on the source, e.g. network vs local storage, and the buffer sizes chosen by Apple’s framework code). So it actually is quite performant and essentially what you’d conventionally write in a file descriptor read loop. If and when it happens to optimise correctly. That’s the major caveat – sometimes the Swift compiler fails to properly optimise code like this, and then indeed the performance can really suck. But for simple cases like in this post’s example code, the optimiser has no trouble with it.
• Similarly, a few folks questioned the need to check the clock on every byte, as in the example. That’s a valid critique of this sort of code in many contexts, and I concur that where possible one should try to be smarter about such things – i.e. use sequences of bunches of bytes, not sequences of individual bytes. e.g. with URLSession you can, and indeed it is faster to do it smarter like that. But, you can get acceptable real-world performance with this code, even in high-throughput cases, and it’s relatively simple and intuitive to write, so it’s not uncommon or necessarily unreasonable.
  
  In addition, sometimes you’re at the mercy of the APIs available – e.g. sometimes you can only get an AsyncSequence<UInt8>. If you don’t care about complete accuracy, you can do things like only considering UI updates every N bytes. You’ll save CPU time and nobody will notice the difference for small enough N on a fast enough iteration, but if those prerequisites aren’t met you might read e.g. N-1 bytes and then hit a long pause, during which time you have the extra N-1 bytes in hand but you’re not showing as such in your UI.
• Some folks noted that are a lot of other clock APIs from Apple’s frameworks, like DispatchTime and CACurrentMediaTime. I didn’t include those in the benchmark because I just didn’t think of them at the time. If anyone wants to send me a pull request adding them to the code, I’d be very happy to accept it.
  
  I haven’t checked all those other APIs specifically, but I can pretty much guarantee they’re all built on mach_absolute_time too (possibly via one or more of the other C APIs already covered in this post). In fact those two examples just mentioned are explicitly documented as using mach_absolute_time.
• Kallikrates quietly pointed to a very interesting recent change in Apple’s Swift standard library code, Make static [milli/micro/nano]seconds members on Duration inlinable. It’s paired with another patch that together seem very specifically aimed at eliminating some of the absurd overhead in Swift’s ContinuousClock & SuspendingClock implementations. The timing is a bit interesting – I don’t know if they were prompted by this post, but it’d be an unlikely coincidence otherwise.
  
  In any case, I suspect it is possible to eliminate the overheads – there’s no apparent reason why they can’t be at least as efficient as Date already is – and so I hope that is what’s happening. Hopefully I’ll be able to re-run these benchmarks in a few months, with Swift 6, and see the performance gap eliminated. 🤞

1. One might quibble with the “by design” assertion. What I mean is that because it uses a protocol it’s susceptible to significant overheads – as is seen in these benchmarks – and because its internal implementation (a private _Int128 type, inside the standard library) is kept hidden, it limits the compiler’s ability to inline, which is in turn critical to eliminating what’s technically a lot of boilerplate. In contrast, if it were simply a struct using only public types internally, it would have avoided most of these overheads and been more amenable to inlining.
   
   It’s not an irredeemable design (I think) – and that’s what the recent patches seem to be banking on, by tweaking the design in order to allow inlining and thus hopefully eliminate almost all the overhead. ↩︎

Surely that’s exactly what the hasPrefix(_:) method is for:

let greeting = "Hello"
let sentence = "hello, this is dog."


if sentence.hasPrefix(greeting) {
    print("Hi!  Nice to meet you!")
} else {
    print("No can haz etiquette?")
}

Wot?

The problem is that hasPrefix is not meant for general use with human text; it’s barely better than a byte-wise comparison. It only guarantees that it won’t be fooled by mere differences in Unicode encoding, which is a good start, but not remotely sufficient for general use.

Let’s step back a bit, and first consider the slightly simpler case of just comparing two whole strings. We can worry about the prefix-matching aspect later.

NSString (from Foundation) provides Swift Strings with a variety of more powerful comparison methods, such as caseInsensitiveCompare(_:) (which is really just an alias for compare(_:options:range:locale:) with the caseInsensitive option).

let a = "hello"
let b = "Hello"

if a.caseInsensitiveCompare(b) {
    print("Hello indeed.")
} else {
    print("Hmph… rude.")
}

So that works. For case sensitivity. But what about other situations?

let plain = "cafe"
let fancy = "café"

if plain.caseInsensitiveCompare(fancy) {
    print("Right, either way it's a shop that sells coffee.")
} else {
    print("So… no coffee, then?")
}

Well, shit.

Ah, but wait, the documentation for caseInsensitiveCompare(_:) has a footnote which surely addresses exactly this problem, albeit obliquely:

Important

When working with text that’s presented to the user, use the localizedCaseInsensitiveCompare(_:) method instead.

No worries – we’ll just use that instead:

let plain = "cafe"
let fancy = "café"

if plain.localizedCaseInsensitiveCompare(fancy) {
    print("Finally!")
} else {
    print("Oh come on!")
}

It turns out this mistake is made by most of the String / NSString methods of similar ilk. And the discrepancies are inscrutable – e.g. localizedStandardCompare(_:) doesn’t handle accents correctly but localizedStandardRange(of:) does.

Long story short, you need to base most (if not all) your string comparison on compare(_:options:range:locale:) or its sibling range(of:options:range:locale:), because the other string methods don’t work properly.

So, with compare(…) you can do e.g.:

let plain = "cafe"
let fancy = "Café"

if .orderedSame == plain.compare(fancy,
                                 options: [.caseInsensitive,
                                           .diacriticInsensitive],
                                 locale: .current) {
    print("Finally!")
} else {
    print("😤")
}

But there’s two other options you should almost always use, which are easy to overlook:

• widthInsensitive.
  
  In all the Latin-alphabet languages of which I’m aware, there is no notion of “width” and therefore no issues with width [in]sensitivity. It seems it most-often comes up in Japanese, where for historical reasons there were multiple versions of the same character that merely different in their visual dimensions. e.g. “カ” and “ｶ”. They are semantically the exact same character, even moreso than “a” is to “A”.
  
  Even if the locale uses a Latin alphabet, there may still be mixed character sets and languages in the text your app processes – e.g. someone writing mostly in English but including Japanese names.
• numeric.
  
  There are more numeric systems than just the modern Arabic numerals as used in English. e.g. “٤٢” is 42 in Eastern Arabic. What matters is usually their meaning (i.e. numeric value), not their representation, just like the other factors we’ve already covered.

So, incorporating all that, the magic incantation required to correctly compare two human pieces of English text, in Swift, is:

let plain = "cafe カ 42"
let fancy = "Café ｶ ٤٢"

if .orderedSame == plain.compare(fancy,
                                 options: [.caseInsensitive,
                                           .diacriticInsensitive,
                                           .numeric,
                                           .widthInsensitive],
                                 locale: .current) {
    print("Actually equivalent.")
} else {
    print("Not equivalent")
}

Note that the explicit locale argument may be important for some use-cases, for two reasons:

• Generally it seems to turn on some – but not all – locale-appropriate options, in addition to any you specify explicitly.
  
  While that may be redundant when you’re explicitly turning them on anyway, it’s possible it will have additional effects that aren’t expressible with the options parameter. You’ll probably want those too, as if they exist they’ll be things like special handling of unusual cases and exceptions to the rules.
• Sometimes it turns hidden options off, such as whether to consider superscripts & subscripts equivalent. This might be a reason to not use it sometimes, if you don’t like the end result.

For reference, here’s a breakdown of the behaviour of some key String / NSString methods (as tested in the en_AU locale), to help you decide what specific incantation you need in a given situation:

Now, the real challenge is making code that works across all locales. In a nutshell, that’s practically impossible with Swift’s standard libraries today – they just don’t support it. To do it right, you’d have to determine what the appropriate comparison options are for every possible locale, manually, and bundle that database with your app.

But, given it’s usually better anyway to err on the side of matching rather than not matching, you can get pretty far by just assuming insensitivity to the above five factors.

Even in cases where this does cause mistakes – e.g. conflating “Maßen” (in moderation) with “Massen” (en masse) in German – it’s potentially unavoidable without deeper, context-specific knowledge anyway (since “ß” is normally equivalent to “ss” in German, just not regarding those specific two words – you can read more about this unpleasant situation on e.g. Wikipedia).

So, back to prefixes…

Fortunately, compare(_:options:range:locale:) and range(of:options:range:locale:) have a couple of additional options which make them easier to apply to situations other than just comparing whole strings.

Checking for a specific prefix

There is an anchored option which is perfect for this – it restricts the match to the start of the receiving string. e.g.:

let reaction = "😁👍"

if nil != reaction.range(of: "😁",
                         options: [.anchored,
                                   .caseInsensitive,
                                   .diacriticInsensitive,
                                   .numeric,
                                   .widthInsensitive],
                         locale: .current)) {
    print("Happy!")
} else {
    print("Sad¡")
}

Note that you must use the range(of:…) variant, not compare(…), because the latter essentially requires that the two strings fully match, not merely that one is a prefix of the other (more on that later, in case you’re not convinced).

Finding common prefixes

Fortunately, there’s a convenience method for exactly this, commonPrefix(with:options:):

let happy = "😁👍"
let party = "😁🎉"

print("Similarities:", happy.commonPrefix(with: party,
                                          options: [.caseInsensitive,
                                                    .diacriticInsensitive,
                                                    .numeric,
                                                    .widthInsensitive]))

Do not use this if you merely want to see if they share a specific prefix, because:

• It’s more efficient to just check that directly on each string separately (rather than allocating and returning an intermediary string).
• You still have to use compare(…) with the full set of options to check the result.

Note also that it does not have a locale parameter, so you cannot opt in to any system-default options defined for the current locale; you must explicitly specify every option you need.

Working with suffixes

You can of course reverse both strings and then compare what are now their prefixes, but this is expensive and awkward, since the result of String‘s reversed() method is a ReversedCollection<String>, not a String or even a Substring, and it does not have the necessary comparison methods, so you have to convert it to a real String first.

Far easier and more efficient is to make use the backwards option to range(of:…). e.g.:

let word = "doing"

if nil != word.range(of: "ing",
                     options: [.anchored,
                               .backwards,
                               .caseInsensitive,
                               .diacriticInsensitive,
                               .numeric,
                               .widthInsensitive],
                     locale: .current)) {
    print("It's an 'ing' word.")
} else {
    print("Nyet.")
}

Note how – conveniently – it does not require reversal of the argument string (“ing” in the above example).

Beware the range parameter

The compare(…) and range(of:…) methods also have a range parameter. This seems like a great idea – you can specify which specific subset of a string you care about, without having to actually break it out into a whole new String instance.

However, the range parameter is both a little unintuitive in its behaviour and fundamentally hard to use correctly.

On the first aspect, it’s critical to realise that it specifies the range within only the receiver (“happy” in the example below). It has no effect on the argument string (“party” in the example below). So you might innocently write the following:

let happy = "😁👍"
let party = "😁🎉"

if .orderedSame == happy.compare(party,
                                 options: [.caseInsensitive,
                                           .diacriticInsensitive,
                                           .numeric,
                                           .widthInsensitive],
                                 range: happy.startIndex ..< happy.index(after: happy.startIndex),
                                 locale: .current)) {
    print("Grins all round.")
} else {
    print("…not happy?")
}

If you want to compare subsets of both strings, you need to explicitly slice the second string, e.g.:

let happy = "😁👍"
let party = "😁🎉"

if .orderedSame == happy.compare(party.prefix(1),
                                 options: [.caseInsensitive,
                                           .diacriticInsensitive,
                                           .numeric,
                                           .widthInsensitive],
                                 range: happy.startIndex ..< happy.index(after: happy.startIndex),
                                 locale: .current)) {
    print("Grins all round.")
} else {
    print("…not happy?")
}

Or, more simply:

let happy = "😁👍"
let party = "😁🎉"

if .orderedSame == happy.prefix(1).compare(party.prefix(1),
                                           options: [.caseInsensitive,
                                                     .diacriticInsensitive,
                                                     .numeric,
                                                     .widthInsensitive],
                                           locale: .current)) {
    print("Grins all round.")
} else {
    print("…not happy?")
}

But, you should rarely if ever actually do the above, because of the second aspect: slicing strings is actually really hard. Not technically, obviously, but if you want to do it correctly. The crux of the challenge is that two strings can have different lengths but still be equivalent (e.g. “ß” and “ss” in German), so slicing them independently is error-prone, unless you somehow account for the specific differences in their encoding. If you naively assume things like a specific length for a target string (e.g. the single character of “ß”) and apply that length to the input string, you might get incorrect results. e.g.:

let input = "Füssen"
let target = "Füß"

if .orderedSame == input.prefix(target.count).compare(target,
                                                      options: [.caseInsensitive,
                                                                .diacriticInsensitive,
                                                                .numeric,
                                                                .widthInsensitive],
                                                      locale: .current)) {
    print("Something about feet.")
} else {
    print("Nothing afoot.")
}

(in case you don’t speak German, that’s the wrong result logically – Füß is a prefix of Füssen)

1. Yes, really. I have no idea why commonPrefix(with:options:) doesn’t work correctly with the numeric option, given it presumably uses compare(_:options:range:locale:) or range(of:options:range:locale:) under the hood. Possibly some bad interaction with locale-specific settings, given it doesn’t let you specify the locale and it doesn’t document what it hard-codes it to. ↩︎
2. Yes, really. I don’t know why using the current locale (en_AU in this case) turns off baseline insensitivity when the numeric option is used, whereas it turns it on otherwise. Seems like a bug in Apple’s framework. ↩︎

Long story short, on at least some bitmap images, as soon as you call getBitmapDataPlanes it somehow permanently breaks that NSImage and NSBitmapImageRep. The telltale sign of this – aside from the image rendering incorrectly – is the stderr message from AppKit:

Failed to extract pixel data from NSBitmapImageRep. Error: -21778

This definitely occurs with 10-bit AVIFs, but it’s not limited to that bit depth nor that file format, because I found the one other mention of this error message online, where the source image was 1-bit.

I’ve never seen this happen with 8-bit, 12-bit, or 16-bit images, AVIF or otherwise.

This makes me suspect it’s tied to the internal pixel format – 10-bit AVIFs always load as 10 / 40 (bits per sample / bits per pixel), irrespective of whether they have an alpha channel. 8-bit AVIFs are always 8 / 32, 12-bit are always 12 / 48. Maybe NSBitmapImageRep has a bug when the bits per pixel isn’t a multiple of 16?

If you call bitmapData instead you get the same error message but it does not stop the NSImage / NSBitmapImageRep from rendering correctly. But the pointer returned from bitmapData points to opaque black. It’s particularly weird that it returns a pointer to a valid memory allocation, of at least the expected size, yet the contents are nothing but zeroes. Seems like it pre-allocates some output buffer, as zeroed memory, and then fails to actually write to that buffer. Yet it returns it anyway, instead of returning nil. 😕

FB13693411.

Partial workaround

If you call recache on the NSImage afterwards, the image is capable of rendering correctly again. That doesn’t help if your objective is to access the bitmap bytes, but at least if you don’t – e.g. you’re encountering this only because some library code is triggering the bug – you might be able to work around it through recache.

A brief history of Contextual Menus

Contextual menus were introduced [to the Mac] in 1997 with Mac OS 8, with the new Contextual Menu Manager system extension and associated Contextual Menu Modules. See also the Mac OS 8 edition of the HIG (page 93).

Support varied a lot in the early days, though, and some user-interface conventions didn’t solidify until a few years later. e.g. SimpleText didn’t support contextual menus at all in Mac OS 8.0, and while 3rd party apps like BBEdit eventually did, it took longer still for now-standard items to become commonplace, like Cut / Copy / Paste.

Broadly-speaking, contextual menus have changed very little over the decades. At some point the app-specific commands were separated from the system-wide commands, the latter becoming relegated to the current “Services” submenu. And the specific menu items have of course evolved over time, but the basic idea has persisted: to provide app-specific or built-in commands followed by commands proffered by system-wide extensions.

It’s chuckle-worthy to remember that when Mac OS 8, with Contextual Menus, was introduced, Macs still had one-button mice. I don’t recall but assume it was possible to connect a mouse with multiple buttons, to a Mac, but remember that no Macs had USB at this time (that didn’t arrive until the iMac in August 1998, nearly a year later). Prior to USB, Macs used different I/O ports than PCs (ADB vs PS/2, for the most part), so there were a lot fewer mouse (and keyboard) options for Macs. Thus, contextual menus were originally – and to this day still also – invoked by control-clicking. You might have encountered the term “secondary-click[ing]”, which arose in that era to abstract over whether it was a physically distinct mouse button or just a modified left click¹.

It apparently wasn’t until the Mighty Mouse in 2005 that Apple actually shipped a multi-button mouse. (I say apparently because I don’t personally recall, and I had switched to 3rd party mice long before then anyway so I probably didn’t even notice at the time)

Are Services still relevant?

I assume that the Services submenu – whether through contextual menus or the app menu – is not used by most Mac users today, if only because the Mac user-base has expanded massively and most people barely leave their web browsers. I’ve heard Services derided or overlooked as a “power-user” or “niche” feature. Which is sad, because they can be very handy.

More importantly, some of your app’s users might rely heavily on Services as part of their personal workflow and choice, and it’s really not our place – as native Mac application developers – to tell them they shouldn’t use a standard system feature.

It’s frustrating for end-users to encounter applications which don’t support standard Mac features, like Services, and makes such applications stand out as non-native or otherwise broken.

It is sad that SwiftUI is in the general company of Electron and its ilk, here.

So how’s it done?

There’s probably at least two ways to do this, one of them being to manually insert the Services menu item into an otherwise vanilla SwiftUI menu. But I quickly ran into non-trivial challenges in pursuing that avenue, as you can’t just ask AppKit for the Services menu item, or its contents. Alas.

Ultimately I found it easier – and more in keeping with the grain – to instead just not use SwiftUI for contextual menus at all. But fortunately that doesn’t mean abandoning SwiftUI entirely, merely intermingling some AppKit into it.

The basic design is the standard AppKit sandwich²: an NSViewRepresentable containing an NSHostingView.

The first part is particularly easy, and just the usual annoying boilerplate:

struct ContextualMenuView<Content: View>: NSViewRepresentable {
    let viewContent: () -> Content
    let textProvider: @MainActor () -> String

    init(@ViewBuilder viewContent: @escaping () -> Content,
         text: @autoclosure @escaping @MainActor () -> String) {
        self.viewContent = viewContent
        self.textProvider = text
    }

    func updateNSView(_ nsView: ContextualMenuViewImplementation<Content>,
                      context: NSViewRepresentableContext<ContextualMenuView>) {
        nsView.rootView = self.viewContent()
        nsView.textProvider = self.textProvider
    }

    func makeNSView(context: Context) -> ContextualMenuViewImplementation<Content> {
        ContextualMenuViewImplementation(rootView: viewContent(),
                                         textProvider: textProvider)
    }

    func sizeThatFits(_ proposal: ProposedViewSize,
                      nsView: ContextualMenuViewImplementation<Content>,
                      context: Context) -> CGSize? {
        return nsView.fittingSize
    }
}

I’ve hard-coded it for text in this example, for simplicity, but you can adjust that to suite your needs.

For convenience it’s nice to add a view modifier for this too:

extension View {
    func contextualMenu(for textProvider: @autoclosure @MainActor () -> String) -> ContextualMenuView<Self> {
        ContextualMenuView(viewContent: { self },
                           text: textProvider)
    }
}

And now on to the real guts of all this, the custom NSView subclass that will define the contextual menu. Fortunately, NSView has very straightforward built-in support for contextual menus, so you don’t need to worry about mouse-event handling – you just provide it a non-nil NSMenu and it does the rest.

class ContextualMenuViewImplementation<Content: View>: NSHostingView<Content>,
                                                       NSServicesMenuRequestor {
    @MainActor fileprivate var textProvider: @MainActor () -> String

    @MainActor required init(rootView: Content,
                             text: @autoclosure @MainActor () -> String) {
        self.textProvider = text
        super.init(rootView: rootView)
    }
    
    // Mandated by NSHostingView, but not actually necessary for our purposes here.  But feel free to give this a real implementation, if that makes sense for your use and needs.
    @MainActor @objc required dynamic init?(coder aDecoder: NSCoder) {
        fatalError("init(coder:) has not been implemented for ContextualMenuView")
    }

    // As above.
    @MainActor required init(rootView: Content) {
        fatalError("init(rootView:) has not been implemented for ContextualMenuView")
    }

    @objc override func validRequestor(forSendType sendType: NSPasteboard.PasteboardType?,
                                       returnType: NSPasteboard.PasteboardType?) -> Any? {
        guard sendType == .string || sendType == .init("NSStringPboardType") else {
            return super.validRequestor(forSendType: sendType, returnType: returnType)
        }

        return self
    }

    @objc nonisolated func writeSelection(to pboard: NSPasteboard,
                                          types: [NSPasteboard.PasteboardType]) -> Bool {
        guard types.contains(.string) || types.contains(.init("NSStringPboardType")) else {
            return false
        }
        
        let text = if Thread.isMainThread {
            self.textProvider()
        } else {
            DispatchQueue.main.sync {
                self.textProvider()
            }
        }

        pboard.setString(text, forType: .string)
        return true
    }

    @objc override func menu(for event: NSEvent) -> NSMenu? {
        NSApplication.shared.registerServicesMenuSendTypes([.string, .init("NSStringPboardType")],
                                                           returnTypes: [])

        let menu = NSMenu()
        menu.allowsContextMenuPlugIns = true

        // Insert other menu items here.

        return menu
    }
}

The only real complexity is in the NSServicesMenuRequestor delegate method writeSelection(to:types:). Because it’s declared nonisolated by the NSServicesMenuRequestor protocol, you’re forced to assume it can be called in any isolation context; from any thread. Unfortunately, at runtime it’s sometimes – but not always! – called from the main thread. Swift doesn’t have an elegant way to say “run this synchronously on the main thread / actor” – if you naively call DispatchQueue.main.sync(…) and you’re already on the main queue, it crashes your application! So you must manually check for being on the main thread, and handle that specially. 😒

The other thing you might want to consider, not shown in the simple example above, is whether you want to support Services sending data back to your view. e.g. they might transform the text and return the new text to you, or generate / source text from somewhere else entirely for you. If you wish to support that, you need to populate the returnTypes argument to registerServicesMenuSendTypes(_:returnTypes:) and implement the readSelection(from:) delegate method.

How’s it done better?

The above works – and is the canonical way to do it. It results in the same plain Services submenu as you’ll see throughout Mac apps.

But that menu’s not great. It just dumps everything it a big amorphous list, with no delineation and merely alphabetical ordering.

Whereas if you look the Services submenu in the application menu:

…it is superior in many ways:

• There are more items.
• There’s a link to System Preferences / Settings to adjust which services are shown.
• There’s dividers in appropriate places, with subheadings, to help visually organise everything.
• App icons are shown to better visually distinguish each service.
• The grouping is somewhat alphabetical but with all services from a given app shown contiguously.

Some of the items shown aren’t actually context-specific – or at least, to no more detail than merely which application is targeted – but they’re intentionally relegated to the bottom, and could be handy anyway. e.g. if you can see any right-clickable area of any window of an application, you can quickly start profiling that application in Instruments.

It’s not hard, per se, to get the better menu. But it’s not documented, not officially supported, and has some broken edge cases. It is how many applications insert the Services menu into their menus (in fact it was the Electron source which helped me figure out how to do this in the first place), so it’s highly unlikely Apple will break it in the foreseeable future. Nonetheless, be warned.

The main edge case / bug that I’ve encountered is that this better Services submenu only works if the window is key. And there’s no way to force the window to be key (all of the AppKit APIs which seem specifically for that purpose flat-out do not work, such as makeKeyWindow). So you have to fall back to the lesser version of the Services menu in those cases. Which is basically whenever the view’s window is not the active window when the right-click occurs (or, whenever that macOS bug hits whereby the window is shown as if it’s the key window but actually isn’t 😤).

Anyway, with all that in mind and apparently not having dissuaded you, here’s the code to go in menu(for:):

menu.allowsContextMenuPlugIns = !(self.window?.isKeyWindow ?? false)

if !menu.allowsContextMenuPlugIns && !(self.window?.makeFirstResponder(self) ?? false) {
    print("Unable to make self the first responder - reverting to built-in Services submenu.")
    menu.allowsContextMenuPlugIns = true
}

// Add all your other items here.

if !menu.allowsContextMenuPlugIns {
    menu.addItem(NSMenuItem.separator())

    let services = NSMenuItem(title: "Services", action: nil, keyEquivalent: "")
    let serviceSubmenu = NSMenu()
    services.submenu = serviceSubmenu
    services.representedObject = textItem.provider
    NSApplication.shared.servicesMenu = serviceSubmenu
    menu.addItem(services)
}

Fortunately – and thanks to the deep elegance of AppKit’s design – that’s it! It otherwise uses all the same machinery as before (like writeSelection(to:returnType:)).

⚠️ macOS 14 Sonoma has a rendering bug when first opening the Services submenu within a given application session, as shown below. It’s not a big deal insofar as most of the items still work, and dismissing the menu and re-opening it fixes the rendering.

Bonuses

Adding a Copy menu item

Since the contextual menu is empty except for the default Services menu, you’ll probably want to add in some of the other options that are typically found in the contextual menu – Copy being perhaps the most prominent and often-used.

Fortunately, it’s trivial:

@MainActor @objc func doCopy(_ menuItem: NSMenuItem) {
    let pb = NSPasteboard.general
    pb.prepareForNewContents(with: nil)
    pb.setString(self.textProvider(), forType: .string)
}

// Then, in your `menu(for:)` method:
do {
    let copyMenuItem = NSMenuItem(title: "Copy", action: #selector(Self.doCopy(_:)), keyEquivalent: "")
    copyMenuItem.target = self
    copyMenuItem.isEnabled = true
    menu.addItem(copyMenuItem)
}

One obvious missing piece is localisation of the “Copy” label; left as an exercise for the reader.

Adding a Share menu item

It’s similarly simple to add a standard Share menu item:

@MainActor @objc func showSharePopup(_ menuItem: NSMenuItem) {
    let picker = NSSharingServicePicker(items: [self.textProvider()])
    picker.show(relativeTo: self.bounds, of: self, preferredEdge: .maxY)
}

// Then, in your `menu(for:)` method:
do {
    let picker = NSSharingServicePicker(items: ["🐞"])
    let pickerMenuItem = picker.standardShareMenuItem
    pickerMenuItem.target = self
    pickerMenuItem.action = #selector(Self.showSharePopup(_:))
    pickerMenuItem.isEnabled = true
    menu.addItem(pickerMenuItem)
}

This one benefits from using the standard, AppKit-provided menu item, so you don’t need to handle localising its label.

The 🐞 is there for two reasons:

1. You have to provide NSSharingServicePicker the nominal item(s) to share up front, in order to initialise it (and so it can tailor its display to the content). standardShareMenuItem should actually be a class member variable, not an instance member variable – a design flaw in AppKit.
   
   You don’t want to invoke the textProvider closure unless you really need to (it could be time-consuming to run, so you don’t want to run it unnecessarily nor while the user is trying to navigate the contextual menu lest it cause GUI hiccups). So an equivalent placeholder value – any other string, in this case – is better, and suffices.
2. I use the ladybug emoji so that it stands out if the value ever somehow gets shown to the user (it’s a bug, get it? 😄).

Adding a Look Up menu item

This one’s a little bit more iffy. If you think you genuinely have need of a Look Up item, consider whether you should be instead making the text selectable and simply utilising the built-in contextual menu support that selectable text views have in SwiftUI.

@MainActor @objc func lookUp(_ menuItem: NSMenuItem) {
    self.showDefinition(for: NSAttributedString(string: self.textProvider()),
                        at: NSPoint(x: CGFloat.infinity, y: CGFloat.infinity))
}

// Then, in your `menu(for:)` method:
do {
    let lookupItem = NSMenuItem(title: submenu == menu ? "Look Up “\(self.textProvider())”" : "Look Up",
                                action: #selector(Self.lookUp(_:)),
                                keyEquivalent: "")
    lookupItem.target = self
    lookupItem.isEnabled = true
    menu.addItem(lookupItem)
}

This doesn’t display perfectly. The use of infinities for the at argument causes it to draw a small yellow box at the top left of the view, layered underneath the contextual menu but usually still visible. That’s a hack for cases where you can’t determine where the text is, or the text being looked up doesn’t exactly match what’s shown in the view.

If you know the actual location of the selected text, you can specify that instead to get the yellow text drawn in that location. But beware: the text from textProvider must exactly match what’s rendered in the view, otherwise the yellow overlaid box – intended to look like a highlight effect – will look weird, because it [re]renders the text based on what textProvider returned. It also might not correctly match the font, in any case.

Thus why I caution against using the reinvention of this particular wheel.

1. Possibly borrowed from earlier computers that had multi-mouse-button support, such as some Unixes, but I suspect just coincidence. I vaguely recall that they were the philosophical antithesis of Apple w.r.t. mouse buttons, with some *nix GUIs requiring at least three mouse buttons for their basic function. I seem to recall some of them labelling the buttons primary, secondary, and tertiary. ↩︎
2. I doubt I came up with this metaphor – although it’s pretty obvious in any case – but it’s worth considering if it’s more than just a cute superficial analogy. AppKit forms the meaningful contents of the sandwich, providing its flavour, substance, and value, while SwiftUI serves only as the bread, there mainly just to convey the contents. 🤔 ↩︎

👾 /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/ld -v
@(#)PROGRAM:ld PROJECT:ld-1053.12
BUILD 15:44:24 Feb  3 2024
configured to support archs: armv6 armv7 armv7s arm64 arm64e arm64_32 i386 x86_64 x86_64h armv6m armv7k armv7m armv7em
will use ld-classic for: armv6 armv7 armv7s arm64_32 i386 armv6m armv7k armv7m armv7em
LTO support using: LLVM version 15.0.0 (static support for 29, runtime is 29)
TAPI support using: Apple TAPI version 15.0.0 (tapi-1500.3.2.2)

👾 /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/ld-classic -v
@(#)PROGRAM:ld-classic  PROJECT:ld64-951.9
BUILD 15:44:42 Feb  3 2024
configured to support archs: armv6 armv7 armv7s arm64 arm64e arm64_32 i386 x86_64 x86_64h armv6m armv7k armv7m armv7em
LTO support using: LLVM version 15.0.0 (static support for 29, runtime is 29)
TAPI support using: Apple TAPI version 15.0.0 (tapi-1500.3.2.2)

Apple’s new linker appears to be much more pedantic than the old one – it warns about a lot of things that the old one didn’t care about. One of these is missing platform load commands:

/Users/SadPanda/Documents/vmaf/libvmaf/ld:1:1: no platform load command found in 'src/libvmaf.a[62](cpuid.obj)', assuming: macOS

This doesn’t technically break anything – assuming it guessed the platform correctly, which I suspect it just takes as being the host’s platform – but it’s super annoying because it’s emitted for every afflicted object file the linker sees (that’s individual files, even if they’re buried in archive files – e.g. libfoo.a). You can have hundreds or even thousands of these warnings for a single library. Worse, they’re emitted when you link against the library, not just when you build it. And with nested static libraries they can propagate up a build chain endlessly.

If we look at the offending file with otool, we see that indeed it has no platform load command:

👾 otool -vl cpuid.obj
Load command 0
      cmd LC_SEGMENT_64
  cmdsize 152
  segname 
   vmaddr 0x0000000000000000
   vmsize 0x000000000000002d
  fileoff 208
 filesize 45
  maxprot rwx
 initprot rwx
   nsects 1
    flags (none)
Section
  sectname __text
   segname __TEXT
      addr 0x0000000000000000
      size 0x000000000000002d
    offset 208
     align 2^4 (16)
    reloff 0
    nreloc 0
      type S_REGULAR
attributes PURE_INSTRUCTIONS SOME_INSTRUCTIONS
 reserved1 0
 reserved2 0
Load command 1
     cmd LC_SYMTAB
 cmdsize 24
  symoff 256
   nsyms 4
  stroff 320
 strsize 56

For reference, here’s an example of the load command it’s looking for:

Load command 1
      cmd LC_BUILD_VERSION
  cmdsize 24
 platform MACOS
    minos 14.2
      sdk 14.4
   ntools 0

Why is it missing?

There’s likely many reasons for this, but they all come down to the same general reason: something is wrong with one [or more] of the tools in the toolchain that created the object file. All object files should contain this load command, on Apple platforms. Though the linker currently treats it as optional, the warning is likely a hint from Apple that it may become required in future. More importantly, without it the linker cannot know which SDK the file is compatible with, and therefore cannot validate that it’s linking the right things.

In the example case above, the problem tool is nasm. It’s an open-source, non-Apple x86-family assembler. It’s been around for nearly thirty years – its first platform was DOS. Though it supports Apple’s platforms, it is predominately focused on Linux & Windows. So it’s not entirely surprising that it’s causing issues.

It also hasn’t released even a minor patch update since 2022 (with version 2.16.01), so nominally it predates and doesn’t support Xcode 15 (announced & released mid-2023).

However, the absence of the LC_BUILD_VERSION command was in fact a problem even before the new Apple linker in Xcode 15, because of cross-compiling (e.g. targeting the iOS simulator on a Mac). There was even a patch to address the problem, submitted by Byoungchan Lee way back in 2021, which was never accepted (it seems to have been completely ignored – not even a response on the mailing list).

Workarounds?

No good ones, that I can find. Stoically ignoring the warning for now is probably the best option.

It is still possible (in Xcode 15, at least) to switch to using the old linker instead, using the -Wl,-ld_classic flags, although that’s not a long-term solution and may have downsides (e.g. some folks report significantly smaller object files with the new linker).

It’s probably possible to manually insert the missing load command – if your build process is amenable to that – although it’s not apparent to me what tool or command invocation you would use to do that. Possibly ld itself, though its documentation provides no insight into this.

For some projects, you might be able to switch to a different assembler (e.g. Apple’s native as from Clang, or Yasm, etc). Apple’s toolchains don’t have this problem (so far as I’ve seen) but are of course not available on other platforms (though, being based on Clang, you can probably get essentially the same thing direct from Clang). I haven’t tested whether Yasm has the issue.

You might have noticed that the primitive types e.g.:

• Bool
• Int & UInt (and all fixed-sized variants, e.g. Int8)
• String
• Float
• Double
• Nil

…all conform to Codable (which is just a convenience protocol combining Encodable & Decodable). That means they have two methods:

func encode(to encoder: any Encoder) throws
init(from decoder: any Decoder) throws

This is good – it ensures you can reason about Encodable and Decodable types and not have any weird exceptions for these primitives, e.g. an Array<MyCodableStruct> is just as Codable as Array<String>. Array doesn’t have to special-case the primitive types, it just calls encode on its Elements irrespectively.

However, if you stop and think about it, it might seem like there’s an infinite loop here. For example, if you call encode(to:) on UInt64, it has only the Encoder APIs to work with – it has to use those APIs, because it doesn’t actually know how to serialise itself, because it has no idea what the serialisation format is – that’s defined by the particular Encoder / Decoder in use.

So, basically UInt64 has to call an encode method somewhere, like this one, which would surely then just call encode(to: self) on the original UInt64, right? Infinite recursion!

Of course, no, thankfully.

The first part of that thinking is correct – see for example UInt64s actual implementation of Codable:

extension UInt64: Codable {
  public init(from decoder: any Decoder) throws {
    self = try decoder.singleValueContainer().decode(UInt64.self)
  }

  public func encode(to encoder: any Encoder) throws {
    var container = encoder.singleValueContainer()
    try container.encode(self)
  }
}

The trick to avoiding self-recursion is that encoders & decoders must special-case the primitive types. They may never call encode(to: self) / init(from: self) on the primitive types.

The way they do this is two-fold:

1. There are specific overloads of encode (and decode) for the primitive types.
   
   An encoder / decoder doesn’t technically have to provide those specialised overloads – it can just implement the generic version, which will satisfy the protocol constraints – but this is ultimately only an optimisation because either way it must serialise those primitive values intrinsically…
2. The implementation of the generic methods must special-case the primitive types. e.g. the pertinent bit of JSONEncoder‘s implementation:

func wrapGeneric<T: Encodable>(_ value: T, for node: _CodingPathNode, _ additionalKey: (some CodingKey)? = _CodingKey?.none) throws -> JSONReference? {
    switch T.self {
    case is Date.Type:
        // Respect Date encoding strategy
        return try self.wrap(value as! Date, for: node, additionalKey)
    case is Data.Type:
        // Respect Data encoding strategy
        return try self.wrap(value as! Data, for: node, additionalKey)
#if FOUNDATION_FRAMEWORK // TODO: Reenable once URL and Decimal are moved
    case is URL.Type:
        // Encode URLs as single strings.
        let url = value as! URL
        return self.wrap(url.absoluteString)
    case is Decimal.Type:
        let decimal = value as! Decimal
        return .number(decimal.description)
#endif // FOUNDATION_FRAMEWORK
    case is _JSONStringDictionaryEncodableMarker.Type:
        return try self.wrap(value as! [String : Encodable], for: node, additionalKey)
    case is _JSONDirectArrayEncodable.Type:
        let array = value as! _JSONDirectArrayEncodable
        if options.outputFormatting.contains(.prettyPrinted) {
            return .init(.directArray(array.individualElementRepresentation(options: options)))
        } else {
            return .init(.nonPrettyDirectArray(array.nonPrettyJSONRepresentation(options: options)))
        }
    default:
        break
    }

    return try _wrapGeneric({
        try value.encode(to: $0)
    }, for: node, additionalKey)
}