Contrary to what you might read online or what the dumb robots might tell you (because they just plagiarise it from those same Reddit threads and YouTube videos), you can’t simply “Expand Audio” and apply the Cross Dissolve to just the video lane¹. You can detach the audio entirely, but that’s heavy-handed and may make your life much harder downstream (since Final Cut Pro will then treat the separated audio and video as completely independent clips).

Actually, there’s a simple ‘hack’ which works:

1. Drag the Cross Dissolve (or similar) onto the clip.
2. Select the clip and “Expand Audio” (from the Clip menu, contextual pop-up menu, or by pressing ⌃S). This is really just to get access to the audio track that’s otherwise hidden under the Cross Dissolve.
3. Carefully drag the audio fade-in handle to the edge of the clip, to exactly where the tooltip says the offset is zero. Don’t go too far – if you go over the end of the clip it’ll re-install the audio fade!

It feels like that shouldn’t work – like it’s actually a subtle bug where Final Cut Pro actually intends to ignore your zero-duration fade-in and reset it back to the full length, like it does if you drag just a pixel too far. But hey, thank goodness for some bugs!

1. Maybe this did work in an earlier version of Final Cut Pro, but it definitely does not in 11.2. ↩︎

Thankfully, any USB-C or USB-C-to-Lightning cable will do

You just connect one end of the USB-C cable to the Portacapture X8 (to its built-in USB port) and the other to your iPhone (USB-C or Lightning work, as suits your particular model of iPhone).

The Portacapture X8 can both record audio via USB and output it. It will automatically output it to any connected device that accepts it. As far as I can tell it uses the master track mix (there doesn’t seem to be any way to send multiple tracks simultaneously – I assume the USB audio protocol only allows a plain stereo transmission).

You must use 48kHz sampling

If you don’t, you’ll quickly get an error dialog on the Portacapture X8 saying “USB FS Mismatch”. That’s its daft way of trying to say that the receiving USB device won’t accept the sampling frequency it’s outputting. Why it can’t have a coherent error message, we’ll probably never know.

You can set the sampling rate in the “General Settings” app, under “Rec Settings” (along with the file format and bit depth / rate, though it doesn’t seem to matter what those are set to w.r.t. iPhone compatibility – they affect only recording to the microSD card).

You don’t have to use the iPhone to provide power

By default the Portacapture X8 will try to use the iPhone to provide power, rather than its batteries. But it doesn’t trust USB power sources – it will ask, via a dialog, “Is the AC adapter 1.5A or more?”. On USB-C iPhones you can power the Portacapture X8 over USB – by selecting “Yes” – although it will drain your iPhone’s battery. But whichever option you choose, the Portacapture X8 will then refuse to provide phantom power to your mics. If you’re not using phantom power then no worries, but if you are you must change the Portacapture’s settings – in “General Settings” app, under “Power/Display”, you must set “Power Source Select” to “Battery” instead of “Auto”. That will essentially turn off the Portacapture’s desire for power from USB, leaving USB as an audio channel only. With it powering itself from its batteries, the phantom outputs will work like normal.

You don’t have to record on the Portacapture X8

It passes audio through to the iPhone automatically, even if you’re not actively recording.

Thankfully the workaround is simple – disable tethering, or enable Airplane mode, while you’re using Image Capture.

Steps:

1. Duplicate the clip (option-drag it in the timeline view) and place the duplicate above¹ the original. Ensure it’s perfectly aligned on the timeline’s X (time) axis, otherwise it won’t show up as a tracking source in step 3.2.
2. On the duplicate clip (make sure it’s selected and your playback position is within it so that you can see what you’re doing!):
   1. Invert the scale (e.g. make it -100% instead of the default 100%).
   2. Invert the X & Y offsets (if necessary – if they were non-zero beforehand).
   3. Add an Object Tracker (the + icon to the right of the “Tracker” titlebar at the bottom of the Video Inspector (right-hand pane)).
   4. A white grid – your anchor section – should appear over your clip. Move and resize it to have it cover an appropriate part of the clip (a subsection that’s contrasty and contains object(s) that are stable – in appearance and position – within the world space of the scene, and ideally are never obscured by anything during the clip).
      • ☝️ You can adjust the timeline position of the clip to find the optimum frame in which to identify your anchor section.
   5. Click Analyze.
      • ⚠️ This will sometimes not work if certain other operations are outstanding, such as dominant motion analysis for the clip – you have to either cancel those background tasks or wait for them to finish. One of Final Cut Pro’s many irritating bugs.
      • ⚠️ Watch carefully as it works through the clip (forwards from your starting point, then backwards, as necessary), for:
        • Your anchor section being intruded upon by any moving objects within the scene. If that happens, try to go back and refine your anchor section placement so that it won’t be intruded upon. If that’s impossible, you can proceed but be aware that the results may be subpar.
        • The anchor section changing in position and size – the more it ‘wobbles’, the worse the final results are likely to be. Consider different anchor section placement, or try a different analysis method (in the Video Inspector, for your Object Track, change the Analysis Method from the default, “Automatic”, to another option – note that you must click Analyze again after changing this, for it to take effect).
3. On the original clip:
   1. Open the Transform viewer (the rectangle icon in the “Transform” titlebar in the Video Inspector).
   2. Click the downward chevron next to “Tracker” text on the Tracker tab button (at the top of the video preview view), to bring up the configuration pop-up:
      • Set Tracker Source to your duplicate clip.
      • Set Tracker to the Object Tracker (or whatever you renamed it to).
      • Set the axes you want stabilisation to apply to (the Apply Tracker To checkboxes).
4. Disable the duplicate (e.g. V key while it’s selected).
   • ☝️ You cannot have it disabled before you select it as the Tracker Source, as it won’t show up in the pop-up menu while disabled.
5. On the original clip:
   1. Set the scale to -100%.
   2. Adjust the X & Y positions to correct the framing, if necessary (if your X & Y position were both 0 on the original clip, this shouldn’t be necessary). Note that this usually isn’t a simple sign inversion as in similar previous steps.

Optional sixth step: file a bug report or suggestion with Apple asking for them to make their built-in image stabilisation work better, and/or make the object tracker GUI easier to use (it could be just one or two clicks, to enable it and say “lock this part of the scene in place”).

1. Technically it doesn’t matter if it’s above or below, I just find it slightly more convenient when it’s above as usually it’ll be disabled (showing the original, now-stabilised version from below) but I access it easily by simply enabling it (V key), such as if I want to adjust the tracking. ↩︎

The primary reason to extract the images exactly as is, bit-for-bit-identical, is that they’re typically already lossy-compressed (usually JPEG). Recompressing them will introduce further losses or increase the file size¹, or both.

Kudos to Josef Habr for suggesting The Unarchiver on StackExchange – I would never have found it on my own, even though I already had it installed and use it occasionally (for more traditional archive file formats, like Zip or StuffIt).

Frustratingly, Josef’s post aside, none of the recommendations you read online mention The Unarchiver, pointing instead to other options which are harder to install, harder to use, and don’t extract the images correctly. Worst of all, many people falsely claim that their suggested approach will extract the images losslessly. Examples include:

• pdfimages from Poppler – silently re-encodes images in some cases (contrary to what its documentation and users claim), such as if they have non-sRGB colour profiles, and fails to preserve the embedded ICC profile. Worse, the developers have known about this for nearly a decade and refuse to fix it.
• pdftoppm (et al) – explicitly convert the embedded images into another format, which while usually a lossless format (e.g. PNG or PPM) by default, still requires you to then re-encode them for use online etc. Plus, they typically don’t preserve ICC profiles.
• ImageMagick – doesn’t extract the images, merely renders the whole PDF page(s) as images, requiring further post-processing and inevitably reducing the image quality (due to mismatched output resolution and pixel alignment vs the embedded images’).
• Exporting pages as images from Preview or Acrobat Reader – obviously doesn’t preserve the extracted images as-is, requires re-encoding them with additional compression losses, etc.
• Screenshots via Preview or Acrobat Reader – ugh, I can’t even.
• Various websites – I mean, they might work, but why upload your personal data to some skeezy website when it’s easy and fast to just use The Unarchiver locally?

I saw a recommendation for File Juicer, but unfortunately the free trial doesn’t work for me – it claims it’s already expired – so I was unable to check that it actually works. Plus, it’s not free (USD$19 at time of writing) so that’s a strong disincentive compared to The Unarchiver.

1. It is possible, with some newer formats like AVIF, to recompress a JPEG in a way that arguably improves the image quality while also reducing the file size. AVIF encoders typically have some built-in smarts to recognise JPEG artefacts specifically, and try to remove them – a direct benefit visually since the artefacts are ugly and a benefit to [re]compression since the encoder then doesn’t need to waste time & output bits trying to preserve the artefacts.
   
   But, utilising this feature can require more care during compression to find the right trade-offs and ensure the result is in fact as good or better than the original – and in any case, AVIF recompression will work better from the original JPEG than a mangled version. ↩︎

extension AVPlayer {
    func fadeAudio(from startVolume: Float, to endVolume: Float, duration: Double) {
        let audioMix = AVMutableAudioMix()

        audioMix.inputParameters = (player.currentItem?.tracks ?? [])
                                    .compactMap(\.assetTrack)
                                    .filter({ $0.mediaType == .audio })
                                    .map { track in
            let currentTime = player.currentTime()

            let parameters = AVMutableAudioMixInputParameters(track: track)

            parameters.setVolumeRamp(fromStartVolume: startVolume,
                                     toEndVolume: endVolume,
                                     timeRange: CMTimeRange(start: currentTime,
                                                            duration: CMTime(seconds: duration,
                                                                             preferredTimescale: currentTime.timeScale)))

            return parameters
        }

        player.currentItem?.audioMix = audioMix
    }
}

I’m not certain what curve this implements, but to my ears it doesn’t sound quite as harsh as a naive linear ramp, so perhaps it’s an S-curve or similar.

Edge cases not handled

Where there’s less than duration time left in the track(s).

How you want to handle that might vary depending on context. e.g. you could clamp the duration to the remaining duration (but you have to think about whether your individual tracks all have the same duration, whether they match the duration of the overall playback item, and whether they’re all aligned within the playback sequence), or wrap it around to the beginning (if you’re looping), or carry the fade through to the next item (if you’re playing a sequence of items), etc. Alas this must be left as an exercise to you, the reader.

Starting a fade while another one is still in progress.

It will halt the previous fade, immediately jump to startVolume, and perform the new fade. If you know that a prior fade is in progress you could potentially extrapolate the current volume and start from there instead (though beware of non-linear ramping).

If you move the playhead backwards (e.g. skimming, or looped playback).

The mix will stay in place, and result in wonky volume levels on the subsequent plays through. To work around that, you can add:

player.currentItem?.audioMix = nil
player.volume = endVolume

…wherever you restart playback at the beginning or move playback earlier than the end of the fade.

Future work?

You can work around these limitations by doing a timer-based fade (i.e. player.volume += smallIncrement at regular, short intervals). However, the problem with that approach is that it’s not synchronised to actual playback – e.g. if the audio is paused, stutters, or faces an initial loading delay, your fade won’t wait for it, potentially resulting in no fade at all (e.g. it takes five seconds to buffer the audio before playback starts, at which point your five second “fade” has run to completion, so your audio starts playing abruptly at full volume).

There’s very likely a third option that addresses all these shortcomings, but I explored that a bit and concluded that it’d be a lot more work. If someone wants to explore that all the way, I’d be interested to see the result. But for many purposes the above code is quite sufficient.

If you’re using AVQueuePlayer, then just use AVPlayerLooper. Easy. But if for some reason you want to use AVPlayer specifically (e.g. you need to do additional things anyway when playback loops back around), read on.

AVPlayer itself doesn’t really help you here – it’s AVPlayerItem that you need to look at, as it has several notifications associated with it that can be very useful – most relevant here is the didPlayToEndTimeNotification. Simply observe that and restart the player explicitly, like so:

let player = AVPlayer(…)  // You'll need to keep a reference to this somewhere, for use in the notification handler, as the notification doesn't provide it.

void startPlayer() {  // Or wherever / however you start playback.
    NotificationCenter.default.addObserver(self,
                                           selector: #selector(playedToEnd(_:)),
                                           name: .AVPlayerItemDidPlayToEndTime,
                                           object: player.currentItem)

    player.play()
} 

@objc func playedToEnd(_ notification: Notification) {
    assert(player.currentItem == notification.object as? AVPlayerItem, "AVPlayer \(player)'s current item (\(player.currentItem)) doesn't match the one in the notification object (\(notification.object)).")

    player.seek(to: .zero)
    player.play()
}

Again, considering using AVQueuePlayer if possible, but the above works too.

If you’re determined, read on.

Screen savers are basically just applications. Same bundle format and structure, just with a “saver” extension instead of “app”.

Setting up the Xcode project

In Xcode, create a new project using the Screen Saver template.

Delete the Objective-C code & header files it creates by default (unless, I suppose, you want to write your screen saver in Objective-C – woo, retro! 😆).

You need to import the ScreenSaver module (framework), subclass ScreenSaverView, and implement a couple of method overrides. Here’s the basic skeleton:

import ScreenSaver

class MyScreenSaver: ScreenSaverView {
    override init?(frame: NSRect, isPreview: Bool) {
        super.init(frame: frame, isPreview: isPreview)
        setup()
    }

    required init?(coder: NSCoder) {
        super.init(coder: coder)
        setup()
    }

    private func setup() {
        // TODO
    }

    override func startAnimation() {
        super.startAnimation()

        // TODO
    }

    override func animateOneFrame() {  // Optional.
        // TODO
    }

    override func stopAnimation() { //  Only for the live preview in System Settings.
        // TODO

        super.stopAnimation()
    }

    override var hasConfigureSheet: Bool {
        true
    }

    private var configureSheetController: ConfigureSheetController?

    override var configureSheet: NSWindow? {
        configureSheetController = ConfigureSheetController(windowNibName: "ConfigureSheet")
        return configureSheetController?.window
    }
}

class ConfigureSheetController: NSWindowController {
    override var windowNibName: NSNib.Name? {
        return "ConfigureSheet"
    }

    override func windowDidLoad() {
        super.windowDidLoad()

        // TODO
    }

    @IBAction func okButtonClicked(_ sender: NSButton) {
        // TODO

        window!.sheetParent!.endSheet(window!, returnCode: .OK)
    }
}

import ScreenSaver

class MyScreenSaver: ScreenSaverView {
    override init?(frame: NSRect, isPreview: Bool) {
        super.init(frame: frame, isPreview: isPreview)
        setup()
    }

    required init?(coder: NSCoder) {
        super.init(coder: coder)
        setup()
    }

    private func setup() {
        // TODO
    }

    override func startAnimation() {
        super.startAnimation()

        // TODO
    }

    override func animateOneFrame() {  // Optional.
        // TODO
    }

    override func stopAnimation() { //  Only for the live preview in System Settings.
        // TODO

        super.stopAnimation()
    }

    override var hasConfigureSheet: Bool {
        true
    }

    private var configureSheetController: ConfigureSheetController?

    override var configureSheet: NSWindow? {
        configureSheetController = ConfigureSheetController(windowNibName: "ConfigureSheet")
        return configureSheetController?.window
    }
}

class ConfigureSheetController: NSWindowController {
    override var windowNibName: NSNib.Name? {
        return "ConfigureSheet"
    }

    override func windowDidLoad() {
        super.windowDidLoad()

        // TODO
    }

    @IBAction func okButtonClicked(_ sender: NSButton) {
        // TODO

        window!.sheetParent!.endSheet(window!, returnCode: .OK)
    }
}

Providing a preferences sheet

If you don’t have any options to configure, you can of course change hasConfigureSheet to return false. Otherwise, you’ll need to create a “ConfigureSheet” xib file with a window in it containing your screen saver’s settings. You can use UserDefaults to save your settings (if you wish), same as any other app. And you’ll need to add an “Okay” or “Save” or similar button to dismiss the sheet.

Getting ready to render

The key methods to implement are setup, with any initial configuration you wish to do (e.g. allocate image or video views, load assets, set up the view hierarchy, etc).

ScreenSaverView is an NSView subclass with a flat black background by default, onto which you can add subviews. Typically for a screen saver you have a very simple view hierarchy – often just a single view or CoreAnimation layer that you’re rendering to – but you can load a xib and insert elements from it into the view if you like.

Rendering

startAnimation is where you should actually start displaying things. e.g. if you’re using an AVPlayer, this is where you actually start it playing (after making it visible, e.g. setting its opacity to 1).

animateOneFrame is optional, and is only called if you set the animationTimeInterval property (on self) to a finite, non-zero value in setup (in which case it’ll be called at intervals at least that long – it might not be called as often as you desire if previous calls overrun or there’s other bottlenecks in the screen saver framework). It’s essentially just a minor convenience vs having to explicitly set up an NSTimer.

Given how buggy Apple’s screen saver framework is, I suggest not relying on animateOneFrame if you can at all avoid it. Even if that means setting up your own timer. That way when they likely break that too in some future macOS release, your screen saver won’t necessarily break as well.

Bonus topic: fading in

Unless your screen saver inherently appears gently (e.g. starts rendering with a flat black view and only slowly adds to it), it’s nice to add a fade-in. You can do that using CoreAnimation on the view’s layer:

override func startAnimation() {
    // Other code…
    
    if let layer {
        layer.opacity = 0.0  // Should already be zero from `setup`, but just to be sure.

        let fadeAnimation = CABasicAnimation(keyPath: "opacity")
        fadeAnimation.fromValue = 0.0
        fadeAnimation.toValue = 1.0
        fadeAnimation.duration = 5  // Seconds.

        // Essential settings to keep the final state
        fadeAnimation.fillMode = .forwards
        fadeAnimation.isRemovedOnCompletion = false

        layer.add(fadeAnimation, forKey: "fadeAnimation")
    }
    
    // Other code…
}

override func startAnimation() {
    // Other code…
    
    if let layer {
        layer.opacity = 0.0  // Should already be zero from `setup`, but just to be sure.

        let fadeAnimation = CABasicAnimation(keyPath: "opacity")
        fadeAnimation.fromValue = 0.0
        fadeAnimation.toValue = 1.0
        fadeAnimation.duration = 5  // Seconds.

        // Essential settings to keep the final state
        fadeAnimation.fillMode = .forwards
        fadeAnimation.isRemovedOnCompletion = false

        layer.add(fadeAnimation, forKey: "fadeAnimation")
    }
    
    // Other code…
}

Note that you cannot implement a fade-out when the screen saver exits, because macOS hides your screen saver immediately. Plus, the user might not want a fade-out as they may be in a rush to do something on their computer.

Previewing

You can determine if you’re running for real or only the preview via the isPreview property (on self). Many screen savers don’t care, but particularly if you save any persistent state, you might want to avoid doing that during preview. For example, in a screen saver which plays a looping video and resumes where it last left off, you probably don’t want the preview to quietly advance the video.

Stopping

stopAnimation is only used for the live preview thumbnail shown in the Screen Saver System Settings pane. It is never called in normal operation of the screen saver (contrary to what Apple’s documentation says – Apple broke that in macOS Sonoma and later).

And that leads to the first path off the official track. When the screen saver is dismissed by the user, nothing in Apple’s framework code does anything. Your view continues to exist, animateOneFrame continues getting called, etc. Your screen saver just runs in the background, its output not visible, but wasting CPU cycles and RAM. Worse, if you have sound, that keeps playing.

To get around that, you need to register for the com.apple.screensaver.willstop notification, in setup, like so:

  private func setup() {
    DistributedNotificationCenter.default.addObserver(self,
                                                      selector: #selector(willStop(_:)),
                                                      name: Notification.Name("com.apple.screensaver.willstop"),
                                                      object: nil)
}
    
@objc func willStop(_ notification: Notification) {
    stopAnimation()
}

  private func setup() {
    DistributedNotificationCenter.default.addObserver(self,
                                                      selector: #selector(willStop(_:)),
                                                      name: Notification.Name("com.apple.screensaver.willstop"),
                                                      object: nil)
}
    
@objc func willStop(_ notification: Notification) {
    stopAnimation()
}

Note that you still need stopAnimation specifically, because in the live preview in System Settings you won’t receive that com.apple.screensaver.willstop notification (from the system’s point of view, the screen saver isn’t running – it’s merely previewing).

Handling resumption

Here’s the second big bug in Apple’s screen saver framework – every time the screen saver starts, your ScreenSaverView subclass is created again. But the old one doesn’t go anywhere. So now you have two copies running simultaneously, which is at the very least wasteful, and can easily lead to gnarly bugs and weird behaviour (e.g. if both are playing sound, or both modify persistent state).

There are essentially two ways to handle this:

1. Kill your own process every time you stop animating.
2. Manually kill or lame-duck older views when a new one is initialised.

Note that you cannot simply check at MyScreenSaver initialisation time if an instance already exists and if so fail initialisation (as is prescribed by this otherwise excellent write-up of this problem), because if you don’t correctly initialise you’ll sometimes end up with nothing rendering or running (the screen saver framework appears to not gracefully handle initialisation failures).

Killing your own process can work but has some perils:

• If you kill your process in stopAnimation the screen will flash black momentarily before actually exiting screen saver mode, which is visually annoying.
• If the screen saver is restarted rapidly after being interrupted, sometimes you’ll end up with nothing but a black screen (with no screen saver running). There’s evidently some race condition in Apple’s screen saver system between screen saver processes exiting and being [re]launched.

So I recommend not taking that approach. Instead, you can lame-duck the old view instances. They’ll stick around, which is a little wasteful of RAM, but as long as they’re not rendering or otherwise doing anything, they’re benign.

There are various ways to implement that, but one of the simpler ones is simply a notification between instances:

static let NewInstanceNotification = "com.myapp.MyScreenSaver.NewInstance";

var lameDuck = false

private func setup() {
    // Initial setup…
    
    NotificationCenter.default.post(name: MyScreenSaver.NewInstanceNotification, object: self)

    NotificationCenter.default.addObserver(self,
                                           selector: #selector(neuter(_:)),
                                           name: MyScreenSaver.NewInstanceNotification,
                                           object: nil)

    // Further setup…
}

@objc func neuter(_ notification: Notification) {
    lameDuck = true

    stopAnimation()

    self.removeFromSuperview()

    // TODO: any additional cleanup you can, e.g. release image & video files, throw out transient models and state, etc.

    NotificationCenter.default.removeObserver(self)
    DistributedNotificationCenter.default().removeObserver(self)
}

static let NewInstanceNotification = "com.myapp.MyScreenSaver.NewInstance";

var lameDuck = false

private func setup() {
    // Initial setup…
    
    NotificationCenter.default.post(name: MyScreenSaver.NewInstanceNotification, object: self)

    NotificationCenter.default.addObserver(self,
                                           selector: #selector(neuter(_:)),
                                           name: MyScreenSaver.NewInstanceNotification,
                                           object: nil)

    // Further setup…
}

@objc func neuter(_ notification: Notification) {
    lameDuck = true

    stopAnimation()

    self.removeFromSuperview()

    // TODO: any additional cleanup you can, e.g. release image & video files, throw out transient models and state, etc.

    NotificationCenter.default.removeObserver(self)
    DistributedNotificationCenter.default().removeObserver(self)
}

You should check lameDuck at the start of methods like startAnimation or animateOneFrame and exit immediately if it’s set to true. Unfortunately, Apple’s screen saver framework will still call those methods on old instances.

Exiting

Unfortunately Apple’s screen saver system will never terminate your screen saver process. Worse, even if you do nothing yourself, Apple’s screen saver framework code will run in an infinite loop, wasting [a small amount of] CPU time. So it’s not great to leave your screen saver process running indefinitely.

Thus, I implement an idle timeout in my screen savers, to have them exit if they’re not active for a while. This can be done like:

@MainActor var idleTimeoutWorkItem: DispatchWorkItem? = nil

override func startAnimation() {
    // Other code…
    
    DispatchQueue.main.async {
        if let idleTimeoutWorkItem {
            idleTimeoutWorkItem.cancel()
        }

        idleTimeoutWorkItem = nil
    }
    
    // Other code…
}

override func stopAnimation() {
    // Other code…
    
    if !lameDuck {
        DispatchQueue.main.async {
            idleTimeoutWorkItem?.cancel()

            let workItem = DispatchWorkItem(block: {
                NSApplication.shared.terminate(nil)
            })
            
            idleTimeoutWorkItem = workItem
            
            DispatchQueue.main.asyncAfter(wallDeadline: .now() + 65, execute: workItem)
        }
    }
    
    // Other code…
}

@MainActor var idleTimeoutWorkItem: DispatchWorkItem? = nil

override func startAnimation() {
    // Other code…
    
    DispatchQueue.main.async {
        if let idleTimeoutWorkItem {
            idleTimeoutWorkItem.cancel()
        }

        idleTimeoutWorkItem = nil
    }
    
    // Other code…
}

override func stopAnimation() {
    // Other code…
    
    if !lameDuck {
        DispatchQueue.main.async {
            idleTimeoutWorkItem?.cancel()

            let workItem = DispatchWorkItem(block: {
                NSApplication.shared.terminate(nil)
            })
            
            idleTimeoutWorkItem = workItem
            
            DispatchQueue.main.asyncAfter(wallDeadline: .now() + 65, execute: workItem)
        }
    }
    
    // Other code…
}

I chose the 65 second timeout somewhat arbitrarily. I figure there’s a reasonable chance a user will unlock their screen to do something quick, then engage the screen saver again – all in the less than a minute – and the cost of idling in the background for an extra minute is small, compared to the cost of relaunching the whole app and reinitialising your renderer.

I added five extra seconds to reduce the probability of aligning with some one-minute timer (e.g. a spurious wake with the screen saver set to start automatically after one minute of no user activity).

You can adjust it however you like.

Testing your screen saver

Double-clicking the built product (your “.saver” app) will prompt the user to install it, replacing an old version if necessary. So that works, though I find it faster to just manually copy the “.saver” app to ~/Library/Screen Savers. Just make sure to kill the existing legacyScreenSaver process, if necessary.

You can test it in System Settings, in the Screen Saver pane. That’s the only place you can test the live preview part.

But otherwise, I found it easiest to just set one of the screen hot corners to start the screen saver, and use that immediately after copying the new “.saver” file into place.

Just be aware that the first time any new copy of the screen saver runs, macOS runs a verification on the bundle, which can take a while if your screen saver is non-trivial in size (e.g. if you bundle large image or video resources). You’ll get a black screen with nothing happening, after invoking the screen saver, while that verification is running.

Distributing your screen saver

You don’t have to sign your screen saver, necessarily, but users will get some annoying error dialogs trying to run it, and will have to fiddle with things in System Settings – or, if they’re on a corporate Mac, they might not be able to run it at all. So it’s preferable to just sign it.

Xcode doesn’t support signing screen savers like it does for plain app targets and the like. So you have to do it manually via the command line, on the built product (your “.saver” app). Thankfully it’s just two commands (once you have the appropriate stuff set up in your developer account – note that you will need a paid Apple Developer account, at $99/year).

Follow the instructions here, but note that they’re missing the final step:

xcrun stapler staple -v MyScreenSaver.saver

Note that you run it against the screen saver itself, not the zip file. The zip file’s just a hack to get Apple’s notary tool to accept the submission. It’s the screen saver bundle itself that’s actually notarised and signed.

Conclusion

That’s “it” in a superficial sense – if you’ve followed all this so far, you have a roughly working screen saver.

But there are a lot more bugs and nuances thereof that may afflict you, depending on what you’re doing in your screen saver. So good luck. 😣

Addendum: SaveHollywood

I found that looking at existing open source screen savers was partially helpful, but also sometimes misleading. e.g. for my most recent screen saver I basically just play a video in a loop (which should be embarrassingly trivial but took two weeks to get working properly, thanks to the aforementioned bugs in Apple’s frameworks, among many others). In that case I looked at SaveHollywood, a similar screen saver, for aid and ideas.

Unfortunately, SaveHollywood is abandoned and doesn’t work on recent versions of macOS. The way it does some things is archaic and either not the best way or not functional at all.

Nonetheless, it did help with some of the higher-level aspects, above the screen saver machinery itself, like how to use AVPlayer in a screen saver.

So, do check out similar, existing screen savers (and of course just use them directly if they suit your needs!) but beware of obsolete or otherwise incorrect code.

Addendum: What this says about Apple

What really troubles me about the screen saver system is what it says about Apple’s approach to the Mac, software, and their users. Which is sadly just the same thing we’ve been seeing for years now.

Screen savers used to work fine. There was an API established long ago that was lightweight, straight-forward, and effective. All Apple had to do was not break it.

And how they broke it is troubling. Perhaps it was prompted by some otherwise unrelated but well-meaning refactor – pulling screen saver code out into a separate, sandboxed process, perhaps, for improved system security. Fine. But if it’s worth changing it’s worth changing properly. It’s very clear that whomever did the changes either (a) didn’t care that they broke things badly, or (b) didn’t care to check.

It’s that recurring theme of not caring that’s most disappointing in today’s Apple.

In short:

1. File > Library Properties (⌃⌘J).
2. Click “Modify Settings” next to “Storage Locations”.
3. Set the “Backups” option to “Do Not Save”.

My projects are already covered by Time Machine and other backup mechanisms, and I tried to tolerate Final Cut Pros built-in backup system, but the damn thing runs so often, which I know about very well because whenever it runs Final Cut Pro becomes completely unusable for several minutes. It’s immensely irritating when it kicks in while I’m in the middle of editing.

However, I seem to have found a fairly reliable workaround:

1. If you’re directly plugging in a removable SSD, as your means of moving files between computers, then skip to step 2.
   
   Copy the catalog-to-be-imported, along with all the original files it references, to local storage on the target computer. You must preserve their relative paths, so it’s easiest if you pre-arrange your source catalog’s files (the “.lrcat” file and its entourage) to be in the same root folder as your original files (images etc)¹.
   
   Any attempt to import directly from a network drive will fail, always.
   
   All the following steps are performed on the target computer.
2. Open the catalog-to-be-imported in Lightroom on the target computer.
   
   This will automatically close whatever other catalog you have open, first.
3. Choose “Export as catalog…” from the File menu.
4. Adjust settings to suit, and export to a new catalog.
   
   Note that this will duplicate all the files referenced by the catalog, into the new catalog. So it might take a while even though it’s all localised to the one computer (and even if it’s on the same volume – Lightroom is not smart enough to perform APFS COW clones).
5. Open the target catalog.
6. Import the catalog you just exported (“Import from Another Catalog…” in the File menu). Make sure to choose to copy the files to a new destination, not just reference them.
7. Delete the temporary catalog.

In my experience you must perform the export-to-an-otherwise-pointless-new-catalog after copying everything to the target computer. Somehow, something about copying Lightroom’s files from one computer to another [over a network] “breaks” them such that Lightroom will refuse to import them.

1. You can do this by – on the source computer – selecting all the photos in the catalog and using the “Folders” subsection of the left panel to adjust their location on disk. Typically by selecting an existing location, right-clicking, and selecting “Move Selected Photos to this Folder”. If necessary, you can first add the desired location by clicking the ‘plus’ icon to the right of the “Folders” section header, and choosing “Add Folder…”.
   
   Yes, Lightroom’s file management UI is a pain in the arse, and badly designed. ↩︎

Sadly it’s still not quite that easy (and some Python package is squatting on the swift package name 🤨).

You can still install from source, in principle, although I’m not aware of any current instructions on how to do so. The buildSwiftOnARM project has seemingly been abandoned. I haven’t tested if their now-years-old instructions still work for the latest versions of Swift (e.g. 5.10, 6.0).

But, you shouldn’t need to build from source anymore – now you have many better options:

Swift.org (Apple-provided options)

• If you happen to be using a yum-centric platform, like RHEL / Amazon Linux / CentOS, you can (finally!) just install RPMs.
• If you happen to use one of the officially supported Linux distributions, you can install via Docker. Alas many major distros are notably absent from that list, like Debian¹ (and therefore Raspbian). 😔
  
  It’s unfortunate that Swift.org pushes this as the preferred way to install Swift on Linux. I avoid Docker because it tends to just make everything more complex (and error-prone), and encourage bad software practices (“it works in my docker image 🖕”).
  • You can bypass the Docker requirement via the relatively primitive method of splatting a tarball into your system. It works, it’s pretty quick, but it’s a maintenance hassle (no version management, no reliable update method, etc).

Swift Community Apt Repository

Thanks to the efforts of community volunteers – in particular the folks at FutureJones – there is the Swift Community Apt Repository, which fills the massive void left by Apple’s apparent hatred for apt-based distros:

1. curl -s https://archive.swiftlang.xyz/install.sh | sudo bash
2. sudo apt install swiftlang
3. There is no step 3!

More details (including the list of supported distros).

It’s a shame that it falls to third parties to make Swift on Linux easy (on major distros, at least). Don’t forget to support them if you benefit from their work!

1. Though it appears Debian support is in the works. ↩︎

This is kind of a sister function to hard links, which similarly avoid copying the file’s contents but where modifications apply to all copies. See also this article on the differences, including versus aliases and symlinks.

Copy-on-write is beneficial for several reasons:

• Copies don’t take up any significant space (just whatever tiny amount is necessary for their metadata).
• The initial copy operation is practically instantaneous (just a few small metadata writes & updates).
• Deferring (if not entirely avoiding) the actual disk I/O reduces wear on the disk.
• The copies can share common segments, saving disk space even when they’re not ultimately identical copies.

It does have some potential downsides:

• You don’t get the increased data redundancy and error resilience that actual copies provide (although if you’re aiming for data redundancy or backup, you should be using separate physical disks anyway).
• It can make subsequent modifications of the file slower, as even just modifying a single byte can trigger the actual copy to be performed.

And some basic limitations:

• It’s only supported on APFS (and maybe additional file systems added by 3rd party extensions, but I haven’t tested nor can I find any accounts of this).
• It only works within individual volumes (it doesn’t work even between two volumes in the same APFS container, or sharing the same physical disk).

Contrary to what I saw online in a few places, copy-on-write works on all APFS volumes, irrespective of whether they are backed by SSDs, HDDs, or some other type of storage.

Should I use it?

Yes!

For most purposes those downsides aren’t an issue, and the limitations merely mean that it’s wise to have a fallback option (of just copying the actual file contents) whenever copy-on-write isn’t available. And many of the tools & APIs fallback automatically (unless you explicitly require them not to, such as with COPYFILE_CLONE_FORCE to copyfile).

How do I use it?

Testing method

Not that this is interesting, just for posterity and to show my work a bit, in case I made a mistake.

I created large-enough files on each of my test volumes (APFS on SSD, APFS on HDD, HFS+ on SSD) that an actual copy would take tens of seconds at least, using dd e.g.:

I then ran the various command line tools on these files, attempting to clone the file to a different name in the same folder, and observed disk I/O activity with Activity Monitor and iStat Menus.

For actual copy-on-writes I would observe that the program successfully concluded practically instantly, and there’d be at most a small blip of disk writes (for metadata modifications).

For failed copy-on-writes I would observe that the program would not exit promptly and I’d see voluminous disk writes (hundreds of megabytes to gigabytes per second, depending on the disk, sustained for many seconds until I was satisfied with the results and killed the test).

For API tests the overall approach was the same, but the APIs were invoked from inside swift repl where possible, and from a throw-away Swift script otherwise³.

1. I haven’t tested it, but as far as I’ve heard APFS does not actually check if the modifications actually diverge the files. e.g. if you “modify” a byte of the file to the value it already has – a pointless but technically possible operation that leaves both copies still identical – APFS will still copy the modified block.
   
   Furthermore, APFS & Apple’s operating systems appear to have no tools (nor APIs) to deduplicate files – e.g. to detect full or partial copies and deduplicate their actual storage on disk. Not even tools that you could invoke manually if you do the hard work of first determining that two files’ contents are identical.
   
   APFS / Apple’s operating systems rely entirely on user applications using copy-on-write explicitly and upfront. ↩︎
2. Curiously, clonefile first shipped in macOS 10.12 (Sierra), according to its man page, which is the release preceding the introduction of APFS in macOS 10.13 (High Sierra). Yet, as far as I can tell HFS+ doesn’t and never did support cloning – nor do any of the other file systems supported by macOS 10.12 (e.g. FAT16 & FAT32, exFAT, NFS). It’s possible it was just convenient for Apple to include it in the prior release as they were probably testing APFS with it internally, during APFS’s development.
   
   Update: Michael Tsai pointed out that Sierra included APFS support in beta form (e.g. you could create disk images with it, but not use it for a boot disk). Thus why clonefile (and other APFS-related tools) were included in Sierra. ↩︎
3. While the C APIs worked just fine inside swift repl irrespective of what volumes I was targeting, the Foundation APIs oddly refused to work whenever the files were not on the boot volume, throwing “Operation not permitted” errors (NSCocoaErrorDomain 513, with no user info). If it weren’t for the C APIs working just fine I’d think it’s a sandboxing issue, but clearly it’s not (and swift repl -disable-sandbox makes no difference). 🤔 ↩︎

Surely that’s exactly what the hasPrefix(_:) method is for:

let greeting = "Hello"
let sentence = "hello, this is dog."


if sentence.hasPrefix(greeting) {
    print("Hi!  Nice to meet you!")
} else {
    print("No can haz etiquette?")
}

Wot?

The problem is that hasPrefix is not meant for general use with human text; it’s barely better than a byte-wise comparison. It only guarantees that it won’t be fooled by mere differences in Unicode encoding, which is a good start, but not remotely sufficient for general use.

Let’s step back a bit, and first consider the slightly simpler case of just comparing two whole strings. We can worry about the prefix-matching aspect later.

NSString (from Foundation) provides Swift Strings with a variety of more powerful comparison methods, such as caseInsensitiveCompare(_:) (which is really just an alias for compare(_:options:range:locale:) with the caseInsensitive option).

let a = "hello"
let b = "Hello"

if a.caseInsensitiveCompare(b) {
    print("Hello indeed.")
} else {
    print("Hmph… rude.")
}

So that works. For case sensitivity. But what about other situations?

let plain = "cafe"
let fancy = "café"

if plain.caseInsensitiveCompare(fancy) {
    print("Right, either way it's a shop that sells coffee.")
} else {
    print("So… no coffee, then?")
}

Well, shit.

Ah, but wait, the documentation for caseInsensitiveCompare(_:) has a footnote which surely addresses exactly this problem, albeit obliquely:

Important

When working with text that’s presented to the user, use the localizedCaseInsensitiveCompare(_:) method instead.

No worries – we’ll just use that instead:

let plain = "cafe"
let fancy = "café"

if plain.localizedCaseInsensitiveCompare(fancy) {
    print("Finally!")
} else {
    print("Oh come on!")
}

It turns out this mistake is made by most of the String / NSString methods of similar ilk. And the discrepancies are inscrutable – e.g. localizedStandardCompare(_:) doesn’t handle accents correctly but localizedStandardRange(of:) does.

Long story short, you need to base most (if not all) your string comparison on compare(_:options:range:locale:) or its sibling range(of:options:range:locale:), because the other string methods don’t work properly.

So, with compare(…) you can do e.g.:

let plain = "cafe"
let fancy = "Café"

if .orderedSame == plain.compare(fancy,
                                 options: [.caseInsensitive,
                                           .diacriticInsensitive],
                                 locale: .current) {
    print("Finally!")
} else {
    print("😤")
}

But there’s two other options you should almost always use, which are easy to overlook:

• widthInsensitive.
  
  In all the Latin-alphabet languages of which I’m aware, there is no notion of “width” and therefore no issues with width [in]sensitivity. It seems it most-often comes up in Japanese, where for historical reasons there were multiple versions of the same character that merely different in their visual dimensions. e.g. “カ” and “ｶ”. They are semantically the exact same character, even moreso than “a” is to “A”.
  
  Even if the locale uses a Latin alphabet, there may still be mixed character sets and languages in the text your app processes – e.g. someone writing mostly in English but including Japanese names.
• numeric.
  
  There are more numeric systems than just the modern Arabic numerals as used in English. e.g. “٤٢” is 42 in Eastern Arabic. What matters is usually their meaning (i.e. numeric value), not their representation, just like the other factors we’ve already covered.

So, incorporating all that, the magic incantation required to correctly compare two human pieces of English text, in Swift, is:

let plain = "cafe カ 42"
let fancy = "Café ｶ ٤٢"

if .orderedSame == plain.compare(fancy,
                                 options: [.caseInsensitive,
                                           .diacriticInsensitive,
                                           .numeric,
                                           .widthInsensitive],
                                 locale: .current) {
    print("Actually equivalent.")
} else {
    print("Not equivalent")
}

Note that the explicit locale argument may be important for some use-cases, for two reasons:

• Generally it seems to turn on some – but not all – locale-appropriate options, in addition to any you specify explicitly.
  
  While that may be redundant when you’re explicitly turning them on anyway, it’s possible it will have additional effects that aren’t expressible with the options parameter. You’ll probably want those too, as if they exist they’ll be things like special handling of unusual cases and exceptions to the rules.
• Sometimes it turns hidden options off, such as whether to consider superscripts & subscripts equivalent. This might be a reason to not use it sometimes, if you don’t like the end result.

For reference, here’s a breakdown of the behaviour of some key String / NSString methods (as tested in the en_AU locale), to help you decide what specific incantation you need in a given situation:

Now, the real challenge is making code that works across all locales. In a nutshell, that’s practically impossible with Swift’s standard libraries today – they just don’t support it. To do it right, you’d have to determine what the appropriate comparison options are for every possible locale, manually, and bundle that database with your app.

But, given it’s usually better anyway to err on the side of matching rather than not matching, you can get pretty far by just assuming insensitivity to the above five factors.

Even in cases where this does cause mistakes – e.g. conflating “Maßen” (in moderation) with “Massen” (en masse) in German – it’s potentially unavoidable without deeper, context-specific knowledge anyway (since “ß” is normally equivalent to “ss” in German, just not regarding those specific two words – you can read more about this unpleasant situation on e.g. Wikipedia).

So, back to prefixes…

Fortunately, compare(_:options:range:locale:) and range(of:options:range:locale:) have a couple of additional options which make them easier to apply to situations other than just comparing whole strings.

Checking for a specific prefix

There is an anchored option which is perfect for this – it restricts the match to the start of the receiving string. e.g.:

let reaction = "😁👍"

if nil != reaction.range(of: "😁",
                         options: [.anchored,
                                   .caseInsensitive,
                                   .diacriticInsensitive,
                                   .numeric,
                                   .widthInsensitive],
                         locale: .current)) {
    print("Happy!")
} else {
    print("Sad¡")
}

Note that you must use the range(of:…) variant, not compare(…), because the latter essentially requires that the two strings fully match, not merely that one is a prefix of the other (more on that later, in case you’re not convinced).

Finding common prefixes

Fortunately, there’s a convenience method for exactly this, commonPrefix(with:options:):

let happy = "😁👍"
let party = "😁🎉"

print("Similarities:", happy.commonPrefix(with: party,
                                          options: [.caseInsensitive,
                                                    .diacriticInsensitive,
                                                    .numeric,
                                                    .widthInsensitive]))

Do not use this if you merely want to see if they share a specific prefix, because:

• It’s more efficient to just check that directly on each string separately (rather than allocating and returning an intermediary string).
• You still have to use compare(…) with the full set of options to check the result.

Note also that it does not have a locale parameter, so you cannot opt in to any system-default options defined for the current locale; you must explicitly specify every option you need.

Working with suffixes

You can of course reverse both strings and then compare what are now their prefixes, but this is expensive and awkward, since the result of String‘s reversed() method is a ReversedCollection<String>, not a String or even a Substring, and it does not have the necessary comparison methods, so you have to convert it to a real String first.

Far easier and more efficient is to make use the backwards option to range(of:…). e.g.:

let word = "doing"

if nil != word.range(of: "ing",
                     options: [.anchored,
                               .backwards,
                               .caseInsensitive,
                               .diacriticInsensitive,
                               .numeric,
                               .widthInsensitive],
                     locale: .current)) {
    print("It's an 'ing' word.")
} else {
    print("Nyet.")
}

Note how – conveniently – it does not require reversal of the argument string (“ing” in the above example).

Beware the range parameter

The compare(…) and range(of:…) methods also have a range parameter. This seems like a great idea – you can specify which specific subset of a string you care about, without having to actually break it out into a whole new String instance.

However, the range parameter is both a little unintuitive in its behaviour and fundamentally hard to use correctly.

On the first aspect, it’s critical to realise that it specifies the range within only the receiver (“happy” in the example below). It has no effect on the argument string (“party” in the example below). So you might innocently write the following:

let happy = "😁👍"
let party = "😁🎉"

if .orderedSame == happy.compare(party,
                                 options: [.caseInsensitive,
                                           .diacriticInsensitive,
                                           .numeric,
                                           .widthInsensitive],
                                 range: happy.startIndex ..< happy.index(after: happy.startIndex),
                                 locale: .current)) {
    print("Grins all round.")
} else {
    print("…not happy?")
}

If you want to compare subsets of both strings, you need to explicitly slice the second string, e.g.:

let happy = "😁👍"
let party = "😁🎉"

if .orderedSame == happy.compare(party.prefix(1),
                                 options: [.caseInsensitive,
                                           .diacriticInsensitive,
                                           .numeric,
                                           .widthInsensitive],
                                 range: happy.startIndex ..< happy.index(after: happy.startIndex),
                                 locale: .current)) {
    print("Grins all round.")
} else {
    print("…not happy?")
}

Or, more simply:

let happy = "😁👍"
let party = "😁🎉"

if .orderedSame == happy.prefix(1).compare(party.prefix(1),
                                           options: [.caseInsensitive,
                                                     .diacriticInsensitive,
                                                     .numeric,
                                                     .widthInsensitive],
                                           locale: .current)) {
    print("Grins all round.")
} else {
    print("…not happy?")
}

But, you should rarely if ever actually do the above, because of the second aspect: slicing strings is actually really hard. Not technically, obviously, but if you want to do it correctly. The crux of the challenge is that two strings can have different lengths but still be equivalent (e.g. “ß” and “ss” in German), so slicing them independently is error-prone, unless you somehow account for the specific differences in their encoding. If you naively assume things like a specific length for a target string (e.g. the single character of “ß”) and apply that length to the input string, you might get incorrect results. e.g.:

let input = "Füssen"
let target = "Füß"

if .orderedSame == input.prefix(target.count).compare(target,
                                                      options: [.caseInsensitive,
                                                                .diacriticInsensitive,
                                                                .numeric,
                                                                .widthInsensitive],
                                                      locale: .current)) {
    print("Something about feet.")
} else {
    print("Nothing afoot.")
}

(in case you don’t speak German, that’s the wrong result logically – Füß is a prefix of Füssen)

1. Yes, really. I have no idea why commonPrefix(with:options:) doesn’t work correctly with the numeric option, given it presumably uses compare(_:options:range:locale:) or range(of:options:range:locale:) under the hood. Possibly some bad interaction with locale-specific settings, given it doesn’t let you specify the locale and it doesn’t document what it hard-codes it to. ↩︎
2. Yes, really. I don’t know why using the current locale (en_AU in this case) turns off baseline insensitivity when the numeric option is used, whereas it turns it on otherwise. Seems like a bug in Apple’s framework. ↩︎

A brief history of Contextual Menus

Contextual menus were introduced [to the Mac] in 1997 with Mac OS 8, with the new Contextual Menu Manager system extension and associated Contextual Menu Modules. See also the Mac OS 8 edition of the HIG (page 93).

Support varied a lot in the early days, though, and some user-interface conventions didn’t solidify until a few years later. e.g. SimpleText didn’t support contextual menus at all in Mac OS 8.0, and while 3rd party apps like BBEdit eventually did, it took longer still for now-standard items to become commonplace, like Cut / Copy / Paste.

Broadly-speaking, contextual menus have changed very little over the decades. At some point the app-specific commands were separated from the system-wide commands, the latter becoming relegated to the current “Services” submenu. And the specific menu items have of course evolved over time, but the basic idea has persisted: to provide app-specific or built-in commands followed by commands proffered by system-wide extensions.

It’s chuckle-worthy to remember that when Mac OS 8, with Contextual Menus, was introduced, Macs still had one-button mice. I don’t recall but assume it was possible to connect a mouse with multiple buttons, to a Mac, but remember that no Macs had USB at this time (that didn’t arrive until the iMac in August 1998, nearly a year later). Prior to USB, Macs used different I/O ports than PCs (ADB vs PS/2, for the most part), so there were a lot fewer mouse (and keyboard) options for Macs. Thus, contextual menus were originally – and to this day still also – invoked by control-clicking. You might have encountered the term “secondary-click[ing]”, which arose in that era to abstract over whether it was a physically distinct mouse button or just a modified left click¹.

It apparently wasn’t until the Mighty Mouse in 2005 that Apple actually shipped a multi-button mouse. (I say apparently because I don’t personally recall, and I had switched to 3rd party mice long before then anyway so I probably didn’t even notice at the time)

Are Services still relevant?

I assume that the Services submenu – whether through contextual menus or the app menu – is not used by most Mac users today, if only because the Mac user-base has expanded massively and most people barely leave their web browsers. I’ve heard Services derided or overlooked as a “power-user” or “niche” feature. Which is sad, because they can be very handy.

More importantly, some of your app’s users might rely heavily on Services as part of their personal workflow and choice, and it’s really not our place – as native Mac application developers – to tell them they shouldn’t use a standard system feature.

It’s frustrating for end-users to encounter applications which don’t support standard Mac features, like Services, and makes such applications stand out as non-native or otherwise broken.

It is sad that SwiftUI is in the general company of Electron and its ilk, here.

So how’s it done?

There’s probably at least two ways to do this, one of them being to manually insert the Services menu item into an otherwise vanilla SwiftUI menu. But I quickly ran into non-trivial challenges in pursuing that avenue, as you can’t just ask AppKit for the Services menu item, or its contents. Alas.

Ultimately I found it easier – and more in keeping with the grain – to instead just not use SwiftUI for contextual menus at all. But fortunately that doesn’t mean abandoning SwiftUI entirely, merely intermingling some AppKit into it.

The basic design is the standard AppKit sandwich²: an NSViewRepresentable containing an NSHostingView.

The first part is particularly easy, and just the usual annoying boilerplate:

struct ContextualMenuView<Content: View>: NSViewRepresentable {
    let viewContent: () -> Content
    let textProvider: @MainActor () -> String

    init(@ViewBuilder viewContent: @escaping () -> Content,
         text: @autoclosure @escaping @MainActor () -> String) {
        self.viewContent = viewContent
        self.textProvider = text
    }

    func updateNSView(_ nsView: ContextualMenuViewImplementation<Content>,
                      context: NSViewRepresentableContext<ContextualMenuView>) {
        nsView.rootView = self.viewContent()
        nsView.textProvider = self.textProvider
    }

    func makeNSView(context: Context) -> ContextualMenuViewImplementation<Content> {
        ContextualMenuViewImplementation(rootView: viewContent(),
                                         textProvider: textProvider)
    }

    func sizeThatFits(_ proposal: ProposedViewSize,
                      nsView: ContextualMenuViewImplementation<Content>,
                      context: Context) -> CGSize? {
        return nsView.fittingSize
    }
}

I’ve hard-coded it for text in this example, for simplicity, but you can adjust that to suite your needs.

For convenience it’s nice to add a view modifier for this too:

extension View {
    func contextualMenu(for textProvider: @autoclosure @MainActor () -> String) -> ContextualMenuView<Self> {
        ContextualMenuView(viewContent: { self },
                           text: textProvider)
    }
}

And now on to the real guts of all this, the custom NSView subclass that will define the contextual menu. Fortunately, NSView has very straightforward built-in support for contextual menus, so you don’t need to worry about mouse-event handling – you just provide it a non-nil NSMenu and it does the rest.

class ContextualMenuViewImplementation<Content: View>: NSHostingView<Content>,
                                                       NSServicesMenuRequestor {
    @MainActor fileprivate var textProvider: @MainActor () -> String

    @MainActor required init(rootView: Content,
                             text: @autoclosure @MainActor () -> String) {
        self.textProvider = text
        super.init(rootView: rootView)
    }
    
    // Mandated by NSHostingView, but not actually necessary for our purposes here.  But feel free to give this a real implementation, if that makes sense for your use and needs.
    @MainActor @objc required dynamic init?(coder aDecoder: NSCoder) {
        fatalError("init(coder:) has not been implemented for ContextualMenuView")
    }

    // As above.
    @MainActor required init(rootView: Content) {
        fatalError("init(rootView:) has not been implemented for ContextualMenuView")
    }

    @objc override func validRequestor(forSendType sendType: NSPasteboard.PasteboardType?,
                                       returnType: NSPasteboard.PasteboardType?) -> Any? {
        guard sendType == .string || sendType == .init("NSStringPboardType") else {
            return super.validRequestor(forSendType: sendType, returnType: returnType)
        }

        return self
    }

    @objc nonisolated func writeSelection(to pboard: NSPasteboard,
                                          types: [NSPasteboard.PasteboardType]) -> Bool {
        guard types.contains(.string) || types.contains(.init("NSStringPboardType")) else {
            return false
        }
        
        let text = if Thread.isMainThread {
            self.textProvider()
        } else {
            DispatchQueue.main.sync {
                self.textProvider()
            }
        }

        pboard.setString(text, forType: .string)
        return true
    }

    @objc override func menu(for event: NSEvent) -> NSMenu? {
        NSApplication.shared.registerServicesMenuSendTypes([.string, .init("NSStringPboardType")],
                                                           returnTypes: [])

        let menu = NSMenu()
        menu.allowsContextMenuPlugIns = true

        // Insert other menu items here.

        return menu
    }
}

The only real complexity is in the NSServicesMenuRequestor delegate method writeSelection(to:types:). Because it’s declared nonisolated by the NSServicesMenuRequestor protocol, you’re forced to assume it can be called in any isolation context; from any thread. Unfortunately, at runtime it’s sometimes – but not always! – called from the main thread. Swift doesn’t have an elegant way to say “run this synchronously on the main thread / actor” – if you naively call DispatchQueue.main.sync(…) and you’re already on the main queue, it crashes your application! So you must manually check for being on the main thread, and handle that specially. 😒

The other thing you might want to consider, not shown in the simple example above, is whether you want to support Services sending data back to your view. e.g. they might transform the text and return the new text to you, or generate / source text from somewhere else entirely for you. If you wish to support that, you need to populate the returnTypes argument to registerServicesMenuSendTypes(_:returnTypes:) and implement the readSelection(from:) delegate method.

How’s it done better?

The above works – and is the canonical way to do it. It results in the same plain Services submenu as you’ll see throughout Mac apps.

But that menu’s not great. It just dumps everything it a big amorphous list, with no delineation and merely alphabetical ordering.

Whereas if you look the Services submenu in the application menu:

…it is superior in many ways:

• There are more items.
• There’s a link to System Preferences / Settings to adjust which services are shown.
• There’s dividers in appropriate places, with subheadings, to help visually organise everything.
• App icons are shown to better visually distinguish each service.
• The grouping is somewhat alphabetical but with all services from a given app shown contiguously.

Some of the items shown aren’t actually context-specific – or at least, to no more detail than merely which application is targeted – but they’re intentionally relegated to the bottom, and could be handy anyway. e.g. if you can see any right-clickable area of any window of an application, you can quickly start profiling that application in Instruments.

It’s not hard, per se, to get the better menu. But it’s not documented, not officially supported, and has some broken edge cases. It is how many applications insert the Services menu into their menus (in fact it was the Electron source which helped me figure out how to do this in the first place), so it’s highly unlikely Apple will break it in the foreseeable future. Nonetheless, be warned.

The main edge case / bug that I’ve encountered is that this better Services submenu only works if the window is key. And there’s no way to force the window to be key (all of the AppKit APIs which seem specifically for that purpose flat-out do not work, such as makeKeyWindow). So you have to fall back to the lesser version of the Services menu in those cases. Which is basically whenever the view’s window is not the active window when the right-click occurs (or, whenever that macOS bug hits whereby the window is shown as if it’s the key window but actually isn’t 😤).

Anyway, with all that in mind and apparently not having dissuaded you, here’s the code to go in menu(for:):

menu.allowsContextMenuPlugIns = !(self.window?.isKeyWindow ?? false)

if !menu.allowsContextMenuPlugIns && !(self.window?.makeFirstResponder(self) ?? false) {
    print("Unable to make self the first responder - reverting to built-in Services submenu.")
    menu.allowsContextMenuPlugIns = true
}

// Add all your other items here.

if !menu.allowsContextMenuPlugIns {
    menu.addItem(NSMenuItem.separator())

    let services = NSMenuItem(title: "Services", action: nil, keyEquivalent: "")
    let serviceSubmenu = NSMenu()
    services.submenu = serviceSubmenu
    services.representedObject = textItem.provider
    NSApplication.shared.servicesMenu = serviceSubmenu
    menu.addItem(services)
}

Fortunately – and thanks to the deep elegance of AppKit’s design – that’s it! It otherwise uses all the same machinery as before (like writeSelection(to:returnType:)).

⚠️ macOS 14 Sonoma has a rendering bug when first opening the Services submenu within a given application session, as shown below. It’s not a big deal insofar as most of the items still work, and dismissing the menu and re-opening it fixes the rendering.

Bonuses

Adding a Copy menu item

Since the contextual menu is empty except for the default Services menu, you’ll probably want to add in some of the other options that are typically found in the contextual menu – Copy being perhaps the most prominent and often-used.

Fortunately, it’s trivial:

@MainActor @objc func doCopy(_ menuItem: NSMenuItem) {
    let pb = NSPasteboard.general
    pb.prepareForNewContents(with: nil)
    pb.setString(self.textProvider(), forType: .string)
}

// Then, in your `menu(for:)` method:
do {
    let copyMenuItem = NSMenuItem(title: "Copy", action: #selector(Self.doCopy(_:)), keyEquivalent: "")
    copyMenuItem.target = self
    copyMenuItem.isEnabled = true
    menu.addItem(copyMenuItem)
}

One obvious missing piece is localisation of the “Copy” label; left as an exercise for the reader.

Adding a Share menu item

It’s similarly simple to add a standard Share menu item:

@MainActor @objc func showSharePopup(_ menuItem: NSMenuItem) {
    let picker = NSSharingServicePicker(items: [self.textProvider()])
    picker.show(relativeTo: self.bounds, of: self, preferredEdge: .maxY)
}

// Then, in your `menu(for:)` method:
do {
    let picker = NSSharingServicePicker(items: ["🐞"])
    let pickerMenuItem = picker.standardShareMenuItem
    pickerMenuItem.target = self
    pickerMenuItem.action = #selector(Self.showSharePopup(_:))
    pickerMenuItem.isEnabled = true
    menu.addItem(pickerMenuItem)
}

This one benefits from using the standard, AppKit-provided menu item, so you don’t need to handle localising its label.

The 🐞 is there for two reasons:

1. You have to provide NSSharingServicePicker the nominal item(s) to share up front, in order to initialise it (and so it can tailor its display to the content). standardShareMenuItem should actually be a class member variable, not an instance member variable – a design flaw in AppKit.
   
   You don’t want to invoke the textProvider closure unless you really need to (it could be time-consuming to run, so you don’t want to run it unnecessarily nor while the user is trying to navigate the contextual menu lest it cause GUI hiccups). So an equivalent placeholder value – any other string, in this case – is better, and suffices.
2. I use the ladybug emoji so that it stands out if the value ever somehow gets shown to the user (it’s a bug, get it? 😄).

Adding a Look Up menu item

This one’s a little bit more iffy. If you think you genuinely have need of a Look Up item, consider whether you should be instead making the text selectable and simply utilising the built-in contextual menu support that selectable text views have in SwiftUI.

@MainActor @objc func lookUp(_ menuItem: NSMenuItem) {
    self.showDefinition(for: NSAttributedString(string: self.textProvider()),
                        at: NSPoint(x: CGFloat.infinity, y: CGFloat.infinity))
}

// Then, in your `menu(for:)` method:
do {
    let lookupItem = NSMenuItem(title: submenu == menu ? "Look Up “\(self.textProvider())”" : "Look Up",
                                action: #selector(Self.lookUp(_:)),
                                keyEquivalent: "")
    lookupItem.target = self
    lookupItem.isEnabled = true
    menu.addItem(lookupItem)
}

This doesn’t display perfectly. The use of infinities for the at argument causes it to draw a small yellow box at the top left of the view, layered underneath the contextual menu but usually still visible. That’s a hack for cases where you can’t determine where the text is, or the text being looked up doesn’t exactly match what’s shown in the view.

If you know the actual location of the selected text, you can specify that instead to get the yellow text drawn in that location. But beware: the text from textProvider must exactly match what’s rendered in the view, otherwise the yellow overlaid box – intended to look like a highlight effect – will look weird, because it [re]renders the text based on what textProvider returned. It also might not correctly match the font, in any case.

Thus why I caution against using the reinvention of this particular wheel.

1. Possibly borrowed from earlier computers that had multi-mouse-button support, such as some Unixes, but I suspect just coincidence. I vaguely recall that they were the philosophical antithesis of Apple w.r.t. mouse buttons, with some *nix GUIs requiring at least three mouse buttons for their basic function. I seem to recall some of them labelling the buttons primary, secondary, and tertiary. ↩︎
2. I doubt I came up with this metaphor – although it’s pretty obvious in any case – but it’s worth considering if it’s more than just a cute superficial analogy. AppKit forms the meaningful contents of the sandwich, providing its flavour, substance, and value, while SwiftUI serves only as the bread, there mainly just to convey the contents. 🤔 ↩︎

I have been unable to deduce any reason for this.

To make it invisible you need only set its opacity to zero or use the hidden modifier. But the view will still be laid out just the same, and take up space in the GUI.

If you want to actually hide the view, so that it disappears completely, you can either not emit the view at all (e.g. conditionalise its existence with if or switch) or you can replace it with EmptyView.

EmptyView is pretty marvellous in this respect. Notice how even though it’s a real view – an actual value that you emit from a view builder – layout collections (e.g. HStack) don’t “see” it – there’s no visible gap, from doubling up on the cell padding, like there would be if they simply treated EmptyViews like 0 wide ⨉ 0 high views.

You might ask, when would you actually need EmptyView? Wouldn’t you’d just use conditional code to hide a view, e.g.:

struct MyView: View {
    let model: Model?
    
    var body: some View {
        if let model {
            Text(model.text)
        }
    }
}

// Now you can just use MyView(model: someOptional),
// without having to unwrap it before every use.

The above is making use of the “ViewBuilder mode”, to implicitly yield zero or more views which are automagically aggregated into a TupleView (if you emit more than one), an optional view (if you dynamically emit zero or one), etc. That’s very convenient, in simple cases like that.

But, “ViewBuilder mode” comes with a lot of limitations, including:

• Compiler diagnostics are far inferior to regular Swift code.
• You cannot return early (i.e. use return statements) except by throwing an exception, which is not always semantically appropriate.
• You cannot have nested functions.
• You sometimes cannot use full Swift control flow syntax (e.g. switch statements).

Fortunately, you can opt out of “ViewBuilder mode” by simply using an explicit return statement, but then you have to follow the usual rules for opaque return types, i.e. that the value be of the same type for every return statement. You can use AnyView in concert with EmptyView to straighten the compiler’s knickers regarding the return type¹, e.g.:

struct MyView: View {
    let model: Model?
    
    var body: some View {
        guard let model else {
            return AnyView(EmptyView())
        }
        
        return AnyView(<actual view contents>)
    }
}

…or – if you don’t mind the return type being Optional, which SwiftUI itself doesn’t – you can use a partially opaque return type:

struct MyView: View {
    let model: Model?
    
    var body: Optional<some View> {
        guard let model else {
            return Body.none
        }
        
        return <actual view contents>
    }
}

Thanks to Dima Galimzianov for helping me figure out how to do that.

Note that in the trivial cases shown above there’s no particularly compelling reason to use these forms instead of the “ViewBuilder mode”, but you could hopefully imagine how, as the conditional logic gets more complicated, it becomes increasingly impractical to avoid using guard, or early returns otherwise.

In case you’re wondering, there’s no difference in show/hide behaviour either – it just works!

It’s possible that there’s some situations in which using EmptyView might cause animation issues, by confusing SwiftUI as to what the relationship is between view hierarchies over time, but I haven’t found that to be the case in practice. And if it ever does crop up, you can always just explicitly id your views.

EmptyView is one of those little sleeper features that seems irrelevant until you stumble upon a need for it, and then it’s really nice to have.

1. There’s technically another option: use a concrete return type instead. But, that’s usually impractical – SwiftUI uses opaque return types a lot, such as for virtually all view modifiers, which force you to use opaque return types in turn. And even when that isn’t an issue, good luck deducing the correct fully-qualified type name even for something as simple as a LabeledContent. ↩︎
2. Literally, “Fear, Uncertainty, and Doubt”. Used here in that face-value sense, not to suggest any malice on anyone’s part. ↩︎

It’s easy to see how some UI designer thought this was a good idea. Surely if you move the mouse near the edge of the window (or the screen, in fullscreen mode) and rest it there, it’s because you’re looking forlornly for your lost sidebar? What could be more helpful and delightful than your missing sidebar popping into view?!

Unfortunately, they have ignored that fact that there is usually already other GUI controls at the edge of the window, not the least of which being the window edge itself (for drag-resizing of the window). Scrollbars are another common inhabitant of window edges.

“Proactive Peek” is the worst of these two because not only does it change what’s under the mouse cursor just as you’re likely to click, stealing the click away from its true target, but it actually shrinks the window’s visible contents. This leads to layout changes and motion noise, particularly in web pages where it can have knock-on effects like mucking with the scroll position or causing major changes by crossing some “responsive design” threshold.

Unfortunately there’s no way to turn this poorly-considered ‘feature’ off completely, although you can effectively disable the “Reveal on Edge Hover” piece:

If you’re an app developer it looks like (I haven’t tested it) you can disable these ‘features’ within your own app, at least, by implementing the private NSSplitView delegate method _splitView:canProactivePeekArrangedView: and setting the NSSplitView revealsOnEdgeHoverInFullscreen property to NO (or the NSSplitViewController private property  _hasItemToRevealOnEdgeHover to NO, if you’re using NSSplitViewController). Or subclassing NSSplitView and overriding _canDoSidebarProactivePeek and _canDoInspectorProactivePeek to return NO – though that only applies to Proactive Peek.

FB13583826.

Is that it?

I know that’s not a very helpful in some sense, but I wasted days trying to figure out how to implement this very basic drag and drop functionality, in SwiftUI. From what I found online, I’m not the only person who’s struggled with this. So hopefully this post saves others from long and frustrating searches.

A big part of the difficulty in answering this simple question – does SwiftUI support this or not – is that a lot of documentation (first & third party) around SwiftUI’s APIs uses the word “promise” but not in the same way. They merely mean that the pasteboard data is populated on first access, not that it actually follows the file promise protocol.

Best alternative (other than ditching SwiftUI)

The closest you can get is to write data into some arbitrary location that you have to choose blindly (not having any idea where the actual destination is), and then rely on the receiving app to move or copy the file from there. It doesn’t matter, in this regard, whether you use the Transferable-based APIs or the NSItemProvider-based APIs.

view.onDrag {
    let provider = NSItemProvider()
    
    provider.registerDataRepresentation(for: UTType.fileURL) {
        do {
            let tmpDir = try FileManager.default.url(for: .itemReplacementDirectory,
                                                     in: .userDomainMask,
                                                     appropriateFor: URL.temporaryDirectory,
                                                     create: true)

            // You'll need to provide the `suggestedFileName` based on the particulars of your use-case.
            let file = tmpDir.appending(component: suggestedFileName)

            try bytes.write(to: file, options: .withoutOverwriting)

            $0(file.dataRepresentation, nil)
        } catch {
            $0(nil, error)
        }

        return nil
    }
    
    return provider
}

This makes it impossible to avoid unnecessary file copies, and requires you to keep the file around permanently, since you have no idea when the receiver is done with it.

e.g. if the Finder is the drop destination, you have no idea what volume the promised file was dropped on, so you don’t know which volume to create it on. If you guess wrong, the Finder is forced to copy the file. Not only does this confuse the user (by showing a copy badge on the mouse pointer) but it wastes time (in performing the copy) and the original file is left in place, requiring manual clean-up (which is impossible to do correctly because you have no idea when the receiving application is done with the temporary file – and the receiver might hold a reference to it permanently).

Understanding the SwiftUI drag & drop APIs

Tangentially, I found most documentation on SwiftUI’s drag & drop APIs – especially Apple’s – to be very poorly written, which is particularly frustrating in this case because the APIs are not intuitive in the slightest.

And most code / usage examples focus exclusively on trivial, disinteresting cases (e.g. dragging images around within a single window).

Only after I’d spend days reverse-engineering the APIs to figure out how they actually work and how they’re supposed to be used, did I stumble upon Dave Rahardja‘s All about Item Providers. It’s by far the best documentation I’ve found on drag & drop in SwiftUI. Note though that Dave also uses the word “promise” a lot even though he’s never actually talking about actual file promises.

Also, I found that the newer and ostensibly better Transferable-based API was harder to understand and harder to use than the NSItemProvider-based APIs. I recommend going straight to, and sticking with, the latter. Your code will be simpler and more reliable.

So you’d think it’d be trivial to do correctly. Alas, it is not.

What is the danger?

There are two key things to watch out for when creating files:

1. Security flaws due to incorrect use of, or badly designed, file system APIs. This is especially a concern for privileged applications (e.g. setuid) or those that ever run with elevated privileges (e.g. via sudo or by admin users). Unintentional reuse of existing files can be (and has been) the cause of major security vulnerabilities. See the appendix for more details.
   
   Another security concern is leaking sensitive information to other programs (or users, on a shared computer). This can easily happen if files are created in places other programs or users can access, such as shared folders. The files may be inadvertently created with inappropriately broad permissions (e.g. world-readable) or the parent folder’s permissions might permit others to change the permissions of the file after the fact even if they don’t own it (e.g. a world-writable folder without the sticky bit set).
2. Data loss risks due to inadvertent overwrites or modifications of existing files. If you’re not certain you know what file is already at a given path on disk, and that you should be allowed to overwrite it, then you should not. Usually, the user has to give explicit permission (e.g. Save dialogs that explicitly ask the user if they intend to overwrite an existing file).
   
   This risk is greatest for persistent files, e.g. files in your Documents folder. Those are usually where the user stores their most important data.
   
   For temporary files the level of danger is generally lower but not zero. If you’re using a system-designated temporaries folder, then in principle anything in there is unimportant anyway. However, randomly mucking with temporary files can still cause data corruption or loss, depending on how those files are used by applications (including your own). e.g. they might store autosaves of the current document in temporary files, and directly copy / move those files when the user formally saves. Thus, modifying the temporary file might end up modifying the user’s actual save file.

How are these dangers mitigated?

App Sandboxing helps

As annoying & limiting as App Sandboxing can be in other regards, it is one of the best single steps an application author can take to improve file security. For the most part, your application is the only (non-system & non-privileged¹) application that can write within its sandbox, and you’ll usually be creating files only within your own sandbox.

Avoid /private/tmp (a.k.a. /tmp)

/tmp is merely a symlink to /private/tmp.

/private/tmp is world-readable: every program on the computer can access its contents. Thankfully, it’s not as bad it first appears – /private/tmp is special in that it has the “sticky bit” set, which imposes some key restrictions on what users may do to each other’s files (most importantly, that they can’t delete them even though /tmp is world-writable).

Still, it’s better to use the tmp folder designated via FileManager.default.temporaryDirectory or URL.temporaryDirectory. Though the location and security properties of that folder varies:

• If App Sandboxing is in use, it’s an application-specific folder like /Users/SadPanda/Library/Containers/com.sadpanda.MyApp/Data/tmp/. No other (unprivileged) application can access that folder.
• If App Sandboxing is not in use, it’s a user-specific folder like /var/folders/v3/8anbad56df64adf3_35gj346jg13v19a/T/ (the exact path varies between user accounts and computers, by design for security). That folder is still insecure – it is accessible to all programs of the same user – but at least you don’t have to worry about other [unprivileged] users.

If used correctly, /private/tmp has essentially the same properties as the user-specific temporary folders. Using it correctly starts with ensuring files are created with no group or ‘other’ privileges, but the full complexities are beyond the scope of this post.

Consider setting a restrictive umask

When you create a file on any Linux or Unix system, such as macOS, its permissions are set based on a combination of the specific API used and the process-wide umask. Good APIs will require you to specify the initial privileges of the file, but some do not (e.g. fopen) and instead use some arbitrary default, such as creating files as readable and writable by anyone (0666). That is bad – usually you don’t want your files readable by any other users.

While you should generally avoid APIs that don’t provide proper control over file permissions, you realistically may not even know if you’re using such APIs because they might be employed by library code you don’t control (including Apple’s).

While it’s possible that the parent folder(s) will protect a given file (by preventing access to their contents by other users), it’s safest to not assume that.

The umask can help mitigate the dangers of bad APIs by specifying which privileges are not to be granted by default².

The umask defaults to denying write access to groups and other users (0022), which is a start but still not good – read access to private user data is still a concern.

However – beyond the overly permissive default setting – there at two problems with umask:

1. It is a process-wide global. Modifications to it apply to all threads in your process, which makes it dangerous to modify. e.g. you might be creating a particularly sensitive file and need the umask to be 0177, so you set it to that, but before you actually get to execute the file creation another thread sets the umask to 0000 because it wants to create an otherwise unrelated file that’s world-writable.
   
   So unfortunately the only safe way to use umask is to set it very early in process launch before any additional threads are created³.
   
   You can try to enforce your own mutual exclusion around umask, e.g. with a global lock, but beware of 3rd party code (including Apple’s) that might modify umask without following your mutual exclusion protocol. Generally umask isn’t modified often, so this arguably is possible to achieve in practice, but proving that there are no missed calls to umask can be practically impossible.
2. Lots of existing code, that you might be unwittingly using via libraries or frameworks, assumes the umask remains at its default. Thus, making it more restrictive might break things in ways & places that are difficult to foresee.
   
   In general the more complicated your program, that more of a concern this is. e.g. most command-line programs can modify umask without causing issues, but GUI programs pull in a lot of framework code and functionality, some of which might implicitly rely on certain umask bits. Unfortunately the only way to find out is experimentally.

So umask is not a panacea. Still, setting a restrictive umask improves security if you can get away with it.

Correctly use the right file creation APIs

In short, creation of files needs to (generally) not extend nor replace existing files, and ensure correct initial file permissions.

There are quite a lot of APIs for creating a file in macOS. I’m going to enumerate only the most common ones provided by the system libraries & frameworks.

⚠️ open

Nominally this is the low-level API for opening (existing or new) files, although as you’ll see later it’s not actually the only one.

It has a flags parameter, which is how you tell it what to actually do, between opening existing files, creating new ones, etc. Two of the most important flags are O_CREAT and O_EXCL. O_CREAT tells open to create the file. If O_EXCL is specified, open will fail if a file already exists at the target location. If O_EXCL is not specified, O_CREAT is interperted as “create the file if necessary” – meaning, it will actually open the existing file if it exists, and not create a new file.

You should almost never use O_CREAT without O_EXCL. If you really do intend to overwrite the existing file, then it’s safer to remove the existing file first (e.g. with unlink), and then create your new file (with O_EXCL to ensure no other file appears at the target location in the interim). That way you ensure the new file has your expected location (not a symlink) and attributes (e.g. file permissions).

Note that O_EXCL will fail if the target is a symlink, so you don’t need to specify O_NOFOLLOW (although it doesn’t hurt).

open requires you to specify the new file’s permissions if you use the O_CREAT option (and respects the umask), which is good as it makes you think about what the permissions should be, and lets you set them to something suitable for each use case. These permissions are only applied to new files, so in a nutshell you cannot rely on them if you don’t use O_EXCL.

For that reason also, you typically should not use O_TRUNC. If you don’t need the contents of the existing file, delete the file first. “Reusing it” via O_TRUNC also reuses its permissions and other attributes, which might not be set correctly for your intentions (e.g. a malicious program might have pre-created the file as world-readable, even though you intend it to be readable only by the current user and have otherwise done the right things such as set the umask to ensure that).

One reason you might validly use O_TRUNC is if you anticipate there being multiple hard links to the file, and you want to modify the file as seen by the other links too. This is very rare. Be sure that’s necessary before you use O_TRUNC, and consider putting validations in place to ensure the file you’ve just opened for reuse has the expected permissions and attributes (e.g. via fstat64 and similar APIs that operate on the file descriptor – do not use lstat or any other path-based APIs).

⚠️ fopen

This is essentially just a wrapper atop open, with the “w” and “a” flags mapping to O_CREAT (essentially) and the “x” flag mapping to O_EXCL. The same rules apply, so you should generally never use “w” without the “x” flag as well, nor “a” without “+”.

fopen does not let you specify the permissions of the created file, instead defaulting to 0666 (readable & writable by everyone) which is a terrible default. It does respect umask, so by default it will create files as 0644 which is marginally better. But, as discussed previously, it is difficult to guarantee what the umask actually is at any particular point in time. So in general you should prefer open instead of fopen.

❌ OutputStream(toFileAtPath:append:) & friends

This ultimately (when you call the open method) calls open with the flags O_WRONLY | O_CREAT (plus O_TRUNC if the append argument is false). As such it will always modify an existing file if present.

It also does not let you specify the permissions of the new file, instead defaulting arbitrarily to 0666 (but respecting umask, at least).

It is an unsafe API and should not be used.

⚠️ NSData.write(toFile:options:) & friends

This family of methods all ultimately call open_dprotected_np (implementation), a variant of open specific to Apple platforms which adds Apple-specific functionality regarding file encryption and isolation (see e.g. the protection-related flags within NSData.WritingOptions). It takes the same flags as the regular open, and write(toFile:options:) by default uses O_CREAT | O_TRUNC. If you use the withoutOverwriting option, it adds O_EXCL. So you should usually use withoutOverwriting.

If you use the atomic flag, it creates the file using a private function _NSCreateTemporaryFile_Protected which obtains a file path using mktemp ⚠️ and calls open_dprotected_np with the flags O_CREAT | O_EXCL | O_RDWR⁴. Once it has created & written to that temporary file, it uses rename to move it into place. rename just silently deletes any existing file at the destination path. So it’s safe against race attacks, but susceptible to data loss bugs from unintentionally overwriting existing files. As such, use atomic only with caution.

It also does not let you specify the permissions of the new file, instead defaulting arbitrarily to 0666 (but respecting umask, at least). For files containing sensitive data (such as private user data), this API should generally not be used.

❌ NSString.write(toFile:atomically:encoding:) & friends

These are essentially just wrappers over NSData.write(toFile:options:) & friends, where the atomically argument maps to the atomic option. They provide no way to use the withoutOverwriting option, so they should not be used in most cases.

Use randomised names for transient files

Whenever the file name doesn’t actually matter – i.e. it’s not chosen by the user and isn’t pre-defined by some system requirement – it’s best to use a randomised name. This serves two purposes:

1. If your code has any bugs that allow it to erroneously overwrite existing files, using random names at least makes it a lot less likely that you’ll trigger those bugs, by greatly reducing the odds of using the same name twice.
2. It makes it harder (if not impossible) for attackers to predict the file names, and therefore to attack them.

There are many ways to obtain a randomised name, but it’s wise to use one of the canonical methods detailed below.

⚠️ mktemp

mktemp is infamously a source of security vulnerabilities, because it doesn’t actually create the file (or folder) but merely returns a path. The caller is responsible for securely creating the file (or folder). This can be done safely – by following the guidance earlier in this post, in particular around the O_EXCL open flag – but it’s easy to screw up.

Generally it’s preferable to use mkstemp / mkdtemp & friends. Unfortunately, if you want to use higher-level file APIs on Apple’s platforms that’s a problem because most of those APIs don’t support initialisation from a file descriptor, only a path (or equivalently, URL). So you may find that you still need to use mktemp. If so, be very careful about how you actually create the files & folders.

✅ mkstemp / mkdtemp & friends

These replacements for mktemp actually create the file / folder (respectively), in a safe way.

mkstemp creates the file and returns the corresponding open file descriptor, instead of merely returning a path and leaving it to the caller to get the creation step right. It will never overwrite (nor open) an existing file. It also sets the file’s initial permissions to 0600 (i.e. read-writable but only by the current user), which is a pretty safe default.

mkdtemp still returns a path (not a file descriptor), like mktemp, but it ensures the folder was actually created (and not previously existent) with the permissions set to 0700 (i.e. usable only by the current user).

Neither make any guarantees regarding security – or lack thereof – due to parent folder permissions. The caller still needs to ensure necessary security protections for those (whether by choosing a suitable system-provided folder, or manually checking permissions and symlinks in the path). URL.temporaryDirectory is a good starting point.

Using these from Swift is a little awkward because they mutate their primary argument (the path template), but here’s an example. Once you have the file descriptor (in the mkstemp case) you can wrap it in e.g. a FileHandle and work with it at a slightly higher level.

⚠️ FileManager.default.url(for:in:appropriateFor:create:)

In its special mode where ‘for’ is .itemReplacementDirectory and ‘in’ is .userDomainMask, this behaves like mkdtemp; it creates a randomly-named folder and returns the URL to it (note that it completely ignores the ‘create’ argument in this case – there is no way to have it not create the temporary folder).

This API seems to presume the use of App Sandboxing to mitigate its problems – and when App Sandboxing is enabled, it’s the best way to obtain a temporary file or folder path, though only if you use a suitable value for the ‘appropriateFor’ parameter, such as URL.temporaryDirectory⁵. When App Sandboxing is enabled, that will result in a path inside the sandbox, which is the most secure place an unprivileged application can use.

However, if instead a URL is provided which points to a different volume, it returns a path to a user-specific temporary folder on that volume, e.g. /Volumes/Example/.TemporaryItems/folders.501/TemporaryItems/. While that does exclude (unprivileged) other users, it’s still a big step down from a location inside the app’s sandbox.

Appendix: File system races in more detail

Generally-speaking, the file system is a shared resource. Multiple programs can access it simultaneously with no coordination required⁶ between them. That opens the door for races – where the state of the world changes in-between file system operations that a program might mistakenly assume are atomic.

In general, any single call to a low-level file system API – e.g. open – is atomic. Most such APIs correspond to a single syscall into the kernel, and the operation inside the kernel is wrapped inside a lock (conceptually if not also literally).

Conversely, any operation that takes multiple calls to a file system API is never atomic.

A textbook security vulnerability arises when you do something like:

1. Make up some random file name (e.g. with mktemp).
2. Check that it doesn’t exist (one syscall), and see that it doesn’t.
3. Write to that file (a separate syscall).

A malicious program could inject its own file in-between steps two and three – or even more dangerously, a symlink – and cause your program to overwrite something (many file APIs will automatically follow symlinks and open existing files, if not used correctly as detailed in this post).

The classic concern in this regard is with privileged programs that have the ability to overwrite sensitive files, e.g. /etc/passwd. Tricking them into doing so can cause major damage to the system (e.g. nobody can login anymore!) in the best case, and in the worst case – where the attacker can also influence the contents of the file, or those contents are conveniently just what the attacker wants – they might be able to implement a more subtle attack that doesn’t merely break the system but instead e.g. changes the root password, giving them superuser access to the computer.

Even for unprivileged applications, it can still be a concern. e.g. they might be tricked into writing a bunch of private user data into a shared location from where the attacker can exfiltrate it.

Appendix: File writing APIs that cannot create new files

These APIs are nominally irrelevant since they can’t be used to create new files, but it can be useful to know that fact, for use in converse scenarios where you do not want to create a file.

FileHandle(forWritingAtPath:)

…ultimately calls [[NSConcreteFileHandle alloc] initWithPath:… flags:0x1 createMode:0 error:nil], which turns the path string into a URL using -[NSURL fileURLWithPath:] and calls -[NSConcreteFileHandle initWithURL:flags:createMode:error:], which calls _NSOpenFileDescriptor to do the actual file system calls. That calls open with only the flag O_WRONLY; it does not pass O_CREAT nor O_EXCL. So FileHandle cannot create new files (which I find a bit unintuitive, as nothing in the name really suggests that limitation).

NSData(contentsOfFile:options:) & friends

…ultimately call open with no flags (meaning they can only read existing files, not even modify them). So again, cannot create new files. Which is perhaps implied and obvious from the name, but it’s good to be certain.

1. In principle system programs & privileged (e.g. root & admin) programs should be defended against too, but it’s often impractically difficult to do so, and beyond the scope of this post to try to explain how.
   
   Root is of course the most impractical to defend against – even though the root user is not a traditional “God” user on macOS, Apple’s nerfing of root is designed to protect Apple’s programs, not yours. Root (and admin users) can ultimately still access any files your application(s) create and there’s nothing you can do about it (other than potentially through orthogonal protections, such as encryption). ↩︎
2. It is of course possible for libraries to override the umask, but that would be particularly foul of them and none that I’ve surveyed do that, thankfully. ↩︎
3. This can be hard to guarantee, in a non-trivial program. You can use an assertion or precondition on the return value of pthread_is_threaded_np to help ensure you’re modifying umask before additional threads are created. ↩︎
4. It’s not apparent to me why it needs read access to the file as well – that appears to be a bug. ↩︎
5. I’m not sure what happens if the App Sandbox container is not on the boot volume. ↩︎
6. There are various mechanism for voluntary coordination, such as NSFileCoordinator and flock, but programs are not required to use them (and malicious programs happily won’t). ↩︎

$ ls -lh /System/Library/Frameworks/AppKit.framework/Versions/Current/AppKit
ls: /System/Library/Frameworks/AppKit.framework/Versions/Current/AppKit: No such file or directory

WTF, mate?

Invariably I spend half an hour websearching around to try to figure out how the hell my system got so broken, and how it’s possible to even boot macOS in such a corrupt state, until finally I chance upon Michael Tsai’s excellent summary of how Apple broke their frameworks starting in Big Sur.

The good news for Hopper is that it has since been updated to work around this – you can access the Apple framework binaries through File > Read File from DYLD Cache… There’s also tools like dyld-shared-cache-extractor which can resurrect the binaries from the cache.

Note also that in Sonoma, at least, the cache lives at /System/Volumes/Preboot/Cryptexes/OS/System/Library/dyld/ (in previous macOS releases it was apparently in /System/Library/dyld/).

127.0.1.1 {{fqdn}} {{hostname}}

I have no idea what it’s even trying to do with that.

What it does do is break a lot of things, because while 127.0.1.1 is a valid address (albeit weird to use instead of the canonical 127.0.0.1), in a standard Plesk system Nginx (that sits in front of Apache typically) is configured to listen only on the network interface(s) associated with your web server’s public IP address(es), not localhost¹. So if you try e.g. curl mydomain.com from inside your server hosting that very domain, you get a connection refused error (or it just times out with no response, depending on firewall settings).

The way I found out about this was when WordPress’s “cron” system silently stopped running tasks (because I use a real cron job to trigger it periodically, rather than relying on WordPress’s flaky built-in system). I subsequently also noticed that Broken Link Checker mistakenly reported every link within my own site as broken.

The solution

…is fairly simple – modify the template to remove the offending line. You then have to either reboot or similarly manually modify /etc/hosts to actually apply the fix.

If for some reason you cannot do that, there are potentially workarounds. For example, for curl commands you can make curl navigate around the problem this like:

In a non-Plesk environment you might also be able to just change the Nginx configuration (somewhere under /etc/nginx). That’s ill-advised for Plesk-operated servers as Plesk will inevitably revert your changes (and as far as I can tell there is no configuration option, or workaround otherwise, to make Plesk configure Nginx to listen on localhost as well 😕).

1. I suspect this is more a side-effect than intentional. Plesk chooses the interface only implicitly, when it specifies that it listens on specific addresses – those assigned to the website in question (Nginx can host multiple otherwise unrelated websites, and each is configured independently within its settings). Since Plesk supports hosting multiple websites, each with potentially different IP addresses, it makes sense that it would want to keep them separate. Otherwise, in a shared hosting situation you could connect to the IP address for somehost.com yet issue a HTTP request for unrelatedhost.com and actually get a response, which is weird, at least. ↩︎

WordPress relies on either ImageMagick or GD for its ability to read & write images. It prefers ImageMagick, as ImageMagick supports a much wider range of files, tends to be faster, and some assert that it produces higher quality files.

ImageMagick 6 was superseded by ImageMagick 7 nearly a decade ago. Version 6 is officially in legacy mode and while still technically supported (in a long term support sense) it generally receives only bug fixes – it fundamentally lacks many features, and most crucially it doesn’t support modern image formats correctly, such as AVIF¹.

Additionally, ImageMagick is a common source of serious security exploits. It’s very important to run the latest version (or very close to) at all times, to have as many critical bug fixes as possible.

Note that, depending on how you operate your WordPress site(s), you may or may not be seriously exposed to ImageMagick security problems. For simple single-author sites, where only the admin can upload anything to the site (images or otherwise), it’s arguably not a huge concern. But for more complex sites with multiple contributors – or worse, sites which accept images from general users or the public – using an outdated ImageMagick release is a serious risk.

Why isn’t this easy?

On some Linux distros, and when Plesk isn’t involved, it is easy – you just apt install imagemagick or yum install ImageMagick or at worst download the RPM and install that. Generally that installs ImageMagick 7 (and if not the very latest release, at least a recent one). You subsequently just run update on your package manager periodically and ImageMagick is kept reasonably up to date.

Ubuntu does not like ImageMagick 7, for some reason. No version of Ubuntu supports it officially, in the sense that it is not in any of the official package repos. If you apt install imagemagick, you’ll get version 6 – and not even a recent version 6, but 6.9.11-60 which is over three years old! That’s three years of well-known bugs – some of them potential security exploits.

So, unfortunately, the only way to install a modern version of ImageMagick on Ubuntu is manually.

Consequently, you have to manually install a suitable version of Imagick as well, and you have to do it into Plesk’s special PHP installation(s).

It took me several hours to figure all of this out, which is why I’ve written this post – so that nobody else ever has to.

The procedure

If you run into any issues, please let me know in the comments section at the bottom of the page. I endeavour to correct any mistakes, oversights, or confusion.

Note that most of these commands must be run as root. Either run them inside a root shell (e.g. su, or sudo bash) or prefix them with sudo.

Install ImageMagick 7

Fortunately, ImageMagick Easy Install (IMEI) exists to greatly simplify this step.

First, as a precaution IMEI insists you remove any existing ImageMagick installation(s). Manually installing a build of ImageMagick over the formally-packaged Ubuntu version could cause problems.

Next, download and install ImageMagick:

By default it’ll install a very recent version – usually the very latest release – but you can customise it if you wish with the --imagemagick-version argument to imei.sh.

Note: I had to use the --force argument to get imei.sh to actually build & install all the components – for some reason it skipped the dependent libraries (for AVIF & JXL support) and just installed ImageMagick, without AVIF and JXL support. Try it without first, since that’s supposed to be the happy path, but check that it says [OK] for every item; that nothing is [SKIPPED].

Once it’s completed successfully – which can take tens of minutes if you have a wimpy server such as is often used for WordPress hosting – double-check that you have a good version installed:

At time of writing that says 7.1.1-26 for me, but of course for you it’ll probably be something newer. You can consult ImageMagick’s download page to find out what the newest version is.

Install Imagick

First, download Imagick:

Next, you need to set the version string to the actual version – otherwise, at the very end of this process, you’ll find that WordPress quietly refuses to use ImageMagick / Imagick! Do that by editing the php_imagick.h file. You should see, somewhere near the top of the file, something like:

#define PHP_IMAGICK_VERSION    "@PACKAGE_VERSION@"
#define PHP_IMAGICK_EXTNUM     30700

You need to change that first string to the version declared on that second line. 30700 means version 3.7.0. So in my case I replaced "@PACKAGE_VERSION@" with "3.7.0".

Now you need to determine where the relevant PHP installation is – the version that Plesk is using to run WordPress for your website(s). You can find that out in various ways, perhaps the simplest being to log in to your Plesk dashboard, open the “Websites & Domains” section, and look at the PHP version listed for each website of interest.

In my case I’m using PHP 8.2. So now I know that the PHP install of interest is located at /opt/plesk/php/8.2/ (Plesk installs all its versions of PHP in /opt/plesk/php). I’ll use that in the subsequent commands shown here, but make sure to adjust that to suit whatever version of PHP you’re using.

To build Imagick, you need the PHP developer tools – for your particular Plesk PHP installation(s). e.g. in my case I’m using PHP 8.2, so I need to run:

Now you need to configure the Imagick build:

Don’t forget to change “8.2” to your version of PHP, if it’s different!

It should only take ten seconds or so to configure, build, & install Imagick.

Now, rename the built library so that Plesk updates won’t clobber it:

Lastly, although it’s technically optional, it’s nice to fix the permissions of the installed library:

Load Imagick into PHP

Annoyingly, PHP won’t automatically notice Imagick is installed – you have to tell it so, manually. Though this is pretty easy to do. There are a few ways to do it, the easiest being through Plesk: go to the “Tools & Settings” section of your Plesk dashboard, click “PHP Settings” (from the “General Settings” category), then click on the relevant handler in the list (e.g. “FPM application”). If you’re not sure which is relevant, look for a non-zero number in the “Domains” column.

Click on “php.ini” to switch to that tab, and you should see a text field containing the contents of the relevant php.ini file. Somewhere in this file – technically anywhere, but I like to scroll down to the section where all the other extensions are listed – you need to add the line:

extension=imagick_custom.so

Then, click the “OK” button to save your changes.

In theory Plesk will now restart the relevant server daemons, to have them pick up the change, but if you subsequently find that WordPress doesn’t seem to see the new ImageMagick installation (or you edited php.ini directly from the command line), you can give it a more forceful shove:

You should now be set – you can test your upgrade in a variety of ways, such as:

• Try uploading an image in a format, or using features such as animation or transparency, that wasn’t previously supported. e.g. AVIFs. Note that you might need additional WordPress plug-ins to enable use of the new formats, e.g. AVIF Support or any of the numerous SVG support plug-ins.
  
  See also my earlier post on my Image workflow for WordPress – particularly the section on AVIF.
• Check phpinfo – in your Plesk dashboard, go to your website(s) in the “Websites & Domains” section, click “PHP” (under the “Dev Tools” category), then click the subtle “View the phpinfo() page” link to the right of the first pop-up menus. Search for “imagick” – you should find an entry for it, with a table of plug-in information such as its version, the version of ImageMagick being used, and the file formats supported.

Addendum: updating ImageMagick & Magick

You can of course just repeat all the above steps to perform a fresh install, incorporating any updates since you last installed everything. But you can save a little time by keeping the imei and imagick folders around, and just doing:

1. git pull --rebase in each, then re-run their respective build commands. For Imagick you’ll probably have to git stash first, in order to do the git pull, and then restore your version string patch with git stash apply.
   
   Don’t forget to check if you need to update the version string in php_imagick.h.
2. Rename the built imagick.so again, and fix its permissions.
3. Restart the PHP daemon.

1. Some versions & builds nominally support AVIF, but this support is almost by accident – what they actually support formally is HEIF, for which they use libheif, which can be built with AVIF support as well, but in my experience that support is buggy and incomplete – e.g. alpha channels are not supported. ↩︎
2. Alas I don’t actually know if / how often it was successfully exploited, as far as Google saw. I have to imagine it was not uncommon, though, given the large attack surface that is ImageMagick’s APIs and image plug-ins, not to mention how lucrative a target Google is. ↩︎

var body: some View {
    SomeRootView {
        …
    }.task {
        let approximateGranularity = Duration.milliseconds(10)
        let threshold = Duration.milliseconds(50)

        let clock = SuspendingClock()
        var lastIteration = clock.now

        while !Task.isCancelled {
            try? await Task.sleep(for: approximateGranularity,
                                  tolerance: approximateGranularity / 2,
                                  clock: clock)

            let now = clock.now

            if now - lastIteration > threshold {
                print("Main thread hung for ",
                      (now - lastIteration).formatted(.units(width: .wide,
                                                             fractionalPart: .show(length: 2))),
                      ".",
                      separator: "")
            }

            lastIteration = now
        }
    }
}

You can adjust the two parameters – approximateGranularity and threshold – to suit your preferences. The overhead is quite tiny in CPU-usage terms, although be aware that this will cause the main thread to wake up frequently so it may have a noticeable, detrimental energy-usage impact. I suggest not deploying this to your users.

Perhaps it goes without saying, but a breakpoint set on the print statement enables you to debug deeper into hangs. Even without that, though, it can be illuminating just to have the log message – oftentimes you don’t notice that your app is hanging, because you don’t happen to be actively interacting with it in that moment. But your users might.

My input images generally fall into three categories:

1. Screenshots, as PNGs.
2. Random 3rd party images, typically as JPEGs but sometimes PNGs or other formats.
3. Photos, as Nikon NEFs.

Some of these formats can be published directly into WordPress – mainly PNG and JPEG – but not all, and neither PNG nor JPEG are the best file formats for anything these days. In terms of compression efficiency, there are numerous superior formats – JPEG 2000, JPEG XL, HEIF¹, WebP, AVIF, etc.

Using an appropriate and efficient format – and suitable compression quality settings – is important to me to save disk space, save bandwidth, minimise page load times, and ensure images are seen the way I want them to be (re. lossless vs lossy).

Best image codecs

Legacy options

PNG and JPEG are supported in practically any & every web browser still in real use today. As is GIF, technically, although it’s strictly inferior to PNG [for static images] and actual video formats [for moving images]. It certainly is possible to still use them, but it wastes storage space & bandwidth, and hurts page load times.

That said, PNG is not the worst option if you use appropriate tooling to maximise the compression. The level of compression you achieve ‘out of the box’ with most applications – e.g. Photoshop, Apple’s apps, etc – is actually quite poor. There are numerous specialised tools which can greatly improve the compression. I recommend ImageOptim – it’s free and open source. It typically reduces PNG files sizes by a further 20% to 50% (depending on how much fine detail or noise the image contains) without any loss of image quality. It actually makes PNG competitive with the modern formats to a degree, although ultimately still not the best.

It’s quite configurable, including with regard to stripping metadata or not – although, alas, it offers no nuance in this, such as keeping copyright metadata but removing spurious camera metadata, or geolocations.

It’s also possible to use such specialised tools for lossy PNG compression. PNG is not inherently lossy, but by manipulating the image before compression it is possible to greatly improve its compressibility. e.g. by reducing the colour palette, adding or removing dithering, or simply removing fine detail. Various tools – including ImageOptim – will do this for you if you wish, although be warned that the resulting image quality is unpredictable and sometimes very poor (exhibiting obvious colour changes, banding, or visual artefacts otherwise).

The example below is a fairly typical “good” result – the file size is cut by 70%, but the lossy PNG compression has introduced noticeable banding and dithering. You might consider it worth the trade-off but consider that a lossy WebP version, also saving 70% on file size, doesn’t have any noticeable image degradation. And an AVIF version with similar image quality cuts the file size by 80%. So while you might think it’s neat to use a ‘lossy’ PNG and get – by PNG standards – dramatically higher compression, you’re probably better off just using WebP or AVIF.

Non-options

Despite its age JPEG 2000 is surprisingly good in my experience – I used it many years ago on this website for my Raw converter comparison as an alternative to TIFF and PNG, as it yielded file sizes about half that of PNG. It’s competitive even with newer formats like WebP and AVIF. JPEG XL is purportedly even better again. Alas, no browser other than Safari supports either of them.

HEIF exists and is “modern” but isn’t really in the same ballpark – while it is better than JPEG, that’s a low bar; it achieves worse results even than JPEG 2000, a codec that’s twenty years older. Its only claim to any fame is that it’s the native source format for many photos today, thanks to its use on Apple iDevices. In any case, the only current-version web browser which supports it is Safari.

Viable options

The only modern format which is universally supported² is WebP.

AVIF is nearly universally supported, with the unfortunately notable exception of Edge.

I choose to use AVIF nonetheless, as my audience are unlikely to be using Edge anyway, since they’re unlikely to be using Windows at all. I also like to assume that Edge will, eventually, get with the program – although the fact that AVIF support was apparently incorporated in versions 114 to 117, but then removed, is concerning.

So those are the two options, basically. For better or worse, neither one is strictly superior to the other.

It is case-dependent and a little subjective as they have noticeably different methodologies:

• AVIF tends to be heavy-handed with ‘noise’ removal, which seems to be a significant part of how it achieves better compression.
• WebP will more faithfully preserve noise (and artefacts from earlier lossy compressions), which can be counter-productive.

AVIF really excels – compared to WebP and earlier codecs – with more aggressive compression settings. It’s much better under tight file size constraints. It also performs particularly well for images which aren’t so clean to begin with – where they do have some actual image noise, or visible compression artefacts from an earlier lossy compression stage, or otherwise fine details that aren’t important to preserve.

WebP behaves more “traditionally” – like JPEG – with regard to increase blockiness and banding as you crank up the compression. AVIF does a much better job of degrading image quality more ‘smoothly’ across the image – so that particularly bad, noticeable artefacts don’t develop as easily. Even while it preserves more genuine detail.

WebP is more competitive with high quality settings or high quality, ‘smooth’ images (e.g. screenshots or computer renderings). In lossless mode WebP universally outperforms AVIF, often by a significant margin. In fact even PNG typically outperforms AVIF for lossless compression!

I’ve also noticed that WebP is substantially better at compressing repetitive patterns (even if they’re not perfectly identical).

AVIF seems particularly good at reducing visible artefacts resulting from JPEG over-compression, and in avoiding prominent compression artefacts in general – such as in the example below, where the file sizes are very similar (14 KB for AVIF, 13 KB for WebP) but AVIF does more with it – preserving more genuine detail such as in the camera lens and IR sensor even while doing a better job of removing the JPEG artefacts (noise, blockiness, and banding).

While generally I avoid re-compressing from an already lossy source since it tends to just compound the compression artefacts, AVIF can be useful for replacing low-quality JPEGs when you don’t have a quality original available. It can actually improve subjective image quality – by correcting JPEG artefacts – even while substantially shrinking the file size.

Image format conversion

There’s basically two schools of thought on converting images to their serving format – do it in advance of uploading the image to WordPress, or have WordPress do it (via one or more plug-ins). I prefer the former as:

1. I have more control over the images by doing the conversion myself.
   • I can pick the best format for each situation.
   • I can customise compression settings to suit the situation (e.g. use more aggressive compression if the source image isn’t high quality to begin with) and use (e.g. use lossless for images which are about comparing image formats, RAW converters, or other such situations where viewers need to see the image exactly).
2. I have more reliably-available computing horsepower locally; I can run pretty much any conversion or optimisation process and it’ll just work. In contrast, WordPress-hosted processing is subject to various limits on runtime and RAM usage, which can cause things to fail in all sorts of ways both obvious and nefarious. Most web servers [used for WordPress sites] have way less RAM available than even an entry-level personal computer. Processing of “large” images, such as photos from a camera, will fail with default WordPress & PHP settings.
3. Every additional WordPress plug-in increases the possibility of problems, whether due to bugs in the plug-in, the possibility of incompatibilities between plug-ins, etc.
4. WordPress plug-ins don’t always age well; each one I adopt is yet another plug-in which I might have to migrate off of in future.
5. WordPress plug-ins for image format conversion & optimisation tend, in my experience, to be both buggy and money-grubbing. Many are tied to 3rd party services like CDNs or “Cloud”-based image processing, and try to lock you into pricey and unnecessary paid subscriptions.

Nominally having WordPress do it means you can just upload any random image without care for its format or compression, and that over time you can easily adopt new image formats just by installing new plug-ins, with all existing images converted retroactively. In reality, new image formats come along very rarely and popular ones – like PNG, JPEG and WebP – are supported basically forever.

You do have to be a little careful, however, to not over-compress your original uploads (or, retain higher-quality versions locally). Especially if you’re churning through image conversions & uploads rapidly, you might not immediately notice compression artefacts. Having the original available in order to redo the compression is valuable.

Advance conversion

There’s a further choice as to whether you do the processing locally or use a website. I prefer to just do it locally, for some of the same reasons as not using WordPress, and also because in my experience I can achieve better results – websites that do image conversion / recompression, especially those that do it for free, use weak compression in order to minimise processing time, and consequently do not achieve the best results.

The best tool I’ve found for local image processing is Image Tool+. It supports all the formats you’ll likely need and is designed for easy batch conversions.

In years gone by I used GraphicConverter, but it doesn’t support AVIF.

It’s important to note that the PNG files produced by Image Tool+ are not very efficiently compressed – you should run them through ImageOptim if you actually intend to use them.

Publishing images in WordPress

WordPress supports some image formats natively, and does a passable job of handling them by default. So you don’t have to do anything additional, necessarily. However, there’s several plug-ins I recommend which significantly improve its capabilities.

AVIF Support

As it says on the tin, it adds support for AVIF to WordPress, which sadly lacks support otherwise. Without this plug-in, WordPress will simply refuse to accept AVIF uploads.

It’s free, open source, and works pretty seamlessly, as long as your web server and WordPress installation are sufficiently modern. The easiest way to determine if that’s the case it to simply install the plug-in – its Settings page will tell you if AVIF is supported or not. If it’s not, only then will you need to do some package upgrading. In particular, you need (as best I can tell from experience and what I’ve read online):

• PHP 8.2 (or later).
• libavif and suitably-compiled versions of GD and PHP.
• ImageMagick 7.0.25 (or later)
  
  …or ImageMagick 6 with a suitably new and compiled libheif w/ AVIF support. I haven’t been able to pin down specific version numbers, and it might vary by package repo (as libheif has to be compiled with certain flags to include the AVIF support). In a nutshell you can tell if your system has it by following the dependency graph from your libmagickcore-6… library (via apt info <package> or similar) and seeing if it includes libheifN with in turn an [installed] dependency on svt-av1 or libaomN. Where ‘N’ may be any number.
  
  Note: ImageMagick 6 doesn’t support alpha channels in AVIF images (see Bug: Alpha channels, below). So use ImageMagick 7 if possible.

Even though all those versions are quite old, I actually had to switch web hosts in order to get them, as you’d be sadly surprised how many WordPress hosting services are running wildly ancient versions of Linux (kernel & distro) and PHP. Now that I have my own Linux VPS, however, it was quite trivial – I basically just used apt and Plesk to install the latest versions of all the relevant software (which you should be frequently doing anyway, for security patches and bug fixes). Although sadly that still only gets me ImageMagick 6 (and an outdated GD that doesn’t support AVIF) – Plesk PHP builds are pretty far behind, and Ubuntu doesn’t officially support ImageMagick 7 at all! 😠

Note that you only need one of GD or ImageMagick to have the necessary AVIF support, which is fortunate in case one of the two is difficult to upgrade with your particular system (such as if you use Plesk).

ImageMagick is apparently superior:

It’s recommended though to use Imagick lib. It gives way better results in terms of speed, quality and size.

AVIF Support author (GrandPlugins)

I haven’t tested that claim myself, but you’ll find it repeated in numerous places online, and not just in the context of AVIF. It sounds like the common wisdom is that GD just isn’t very good³.

Bug: AVIF over-compression

At time of writing, it has a bug whereby it doesn’t respect image quality settings, neither the WordPress defaults nor any customisation you might do with other plug-ins. Instead, it always uses a 30% quality setting for AVIF compression (e.g. for thumbnails) which often results in noticeable visual artefacts from the over-compression. I’ve reported this to the developer and 🤞 they address this soon, as it looks like a trivial one-line patch.

Update: the plug-in author fixed this in version 1.0.5, January 2nd 2024. As a bonus they also added plug-in settings for specifying the desired image quality and (for GD) ‘speed’ (how hard the AVIF encoder tries, trading off compression speed for effectiveness).

Bug: Alpha channels

AVIFs with alpha channels don’t work with ImageMagick 6, at time of writing – any time WordPress re-encodes the image, such as for thumbnail generation, it removes the alpha channel. Typically this results in the transparent sections turning black instead.

For now, I use WebP when transparency is involved, as I’m hamstrung by Plesk’s lack of support for ImageMagick 7 in their PHP packages. 😕

I ultimately upgraded to ImageMagick 7 to fix this bug. See my upgrade guide if you’re on Ubuntu or using Plesk, or otherwise can’t seem to get ImageMagick 7 from your normal package management process.

CDN incompatibility

Note that some CDNs still don’t support AVIF, and there’s nothing the AVIF Support plug-in can do about that. You might be using such a CDN without even realising it – e.g. the popular Jetpack plug-in secretly uses the wp.com CDN, so AVIF images don’t work in some places, such as Jetpack’s “Related posts” feature. Alas there’s not much you can do other than not use such broken CDNs – nor plug-ins that rely on them, like Jetpack.

In general you don’t want to use such CDNs anyway, because their lack of support for AVIF – or any other content format – is usually only a result of them mucking with your images in some way. Not all CDNs are bad, though – e.g. Cloudflare has no issues with AVIF because they don’t try to mess with your files.

Enable Media Replace

This lets you replace an existing image without breaking any use of it in existing posts, pages, etc. Including with a different file format. That latter detail is something most competing image replacement plug-ins do not support [reliably], but it’s crucial if you want to upgrade any existing images on your website.

It’s free and open source.

Image Regenerate & Select Crop

This lets you perform a bunch of helpful little things, including:

• You can define custom image “thumbnail” sizes (and modify WordPress’s defaults), which can be handy for tailoring your images to your actual presentation sizes. The WordPress defaults are pretty arbitrary and typically don’t match what your theme desires, let-alone any other plug-ins’ needs (such as for galleries, slideshows, etc).
• You can adjust the default image compression quality setting (for each thumbnail size individually).
  
  Note however that this doesn’t always apply, e.g. currently the AVIF Support plug-in has a bug whereby it ignores this setting. The setting seems to work for the built-in-supported image formats, though, like JPEG and WebP.
• You can manually delete or [re]generate thumbnails in each of the possible sizes, for images both individually or en masse. This can be handy if you change relevant settings (like compression quality, above) or in debugging situations (such as if you’re experimenting with other image plug-ins and they muck up your thumbnails).

It’s free (albeit with optional paid ‘advanced’ features).

Perfect Images

It’s baffling that to this day WordPress doesn’t support modern displays – and I use ‘modern’ in a very liberal sense, since we’re talking circa 2010 onwards. That is, displays designed with a high pixel density that use a non-1-to-1 ratio of points to pixels. i.e. any smartphone or quality computer display in the last decade.

The result is images that render at a quarter (or worse) of full resolution, looking pixelated or soft.

If you care enough about image quality to even consider what image formats or compression settings you’re using, you absolutely should be using Perfect Images.

Basically it fixes & extends WordPress’s image support to – mostly automagically and invisibly – just work with full-resolution images. It’s particularly noticeable and essential for things like screenshots, or any images containing text, as those are especially obvious if rendered incorrectly.

Its core functionality is free – and you can get a fair way with just the free version – but the pro version is worth it, even if a bit pricey.

Usage tips

Fix WordPress’s image size limit

Make sure to check the “Image Threshold … Disable” checkbox in the General settings. This fixes a truly obnoxious WordPress bug whereby it mangles uploads of large images.

Upload ‘Retina’ images specially

For images which are explicitly “2x” – such as screenshots – it’s important to upload them ‘manually’ using Media > Add New Media File, as opposed to e.g. just drag-and-dropping them into a post. The plug-in adds an “Upload New Retina Image” option to the dedicated upload page, which registers the uploaded image with WordPress at technically a quarter of its actual size. That basically tricks WordPress into using the correct dimensions for the image – otherwise, WordPress mistakes the pixel dimensions for the point dimensions, and will naively try to display the image at four times its natural size.

Don’t worry, Perfect Images preserves the real, full-resolution version as the “2x” version of the upload, and will display that instead to your visitors.

This particular upload method is the only way to get images to render correctly in all cases. In some places – e.g. embedded in posts – you can manually specify the correct image dimensions, but that’s both tedious and error-prone, and not possible in all uses.

The only caveat is that those images will look ugly and blurry in the post & page editors. Don’t worry, they’ll render correctly in the actual published (and preview) version of your posts & pages.

For images which don’t have an intrinsic pixel density – such as photos – you typically don’t have to worry about this, as you’ll usually just let them render at whatever size the viewport & layout permit, up to their natural full size.

Limitations

The core functionality works reliably in my experience (over many years now), but some of the peripheral features are broken or notably buggy:

• Image replacement will not work correctly if the file format changes – and it won’t even recognise this case and prevent it, instead leaving you with broken images. Use Enable Media Replace for this instead.
• Batch regeneration of thumbnails doesn’t work reliably – it will silently fail midway if an error occurs, such as it encountering a video file. Use Image Regenerate & Select Crop for this instead.
• Custom Image Sizes don’t work correctly – they always force cropping of the image (“resize to fill”), rather than resizing to fit. This is almost never what you want. Use Image Regenerate & Select Crop for this instead.
• For the Pro version [only], auto-update doesn’t work. You have to periodically check the website for new versions.
• It sometimes reports “issues” for images, in its special dashboard under the Media section, when there aren’t any. This is of course pretty insignificant – you can just ignore it – although it might hide any actual issues if they occur. I did find a way to clear these bogus issues, but it involved manually deleting all thumbnails for all my images and regenerating them, which was a bit of an ordeal due to other, aforementioned bugs. So I don’t recommend mucking with it.

1. Note that HEIF does support lossless encoding but support for this amongst actual encoding applications is poor to non-existent. As such it’s generally not a suitable replacement for PNG in practice. ↩︎
2. As the matrix shows, no version of Internet Explorer supports WebP. But – though there apparently is some non-trivial number of people still using it – I don’t consider Internet Explorer a browser of concern anymore, given it has an official and straightforward successor in Edge, and its last version was released a decade ago in 2013. ↩︎
3. Although for image compression, specifically, this is odd, since there’s only a couple of underlying libraries they could possibly be using, and while they do vary significantly in speed they should produce very similar outputs. ↩︎

There was of course the bug whereby encrypted external drives no longer mounted automatically. That was pretty special, because it reveals that Apple doesn’t test encrypted external drives anywhere in their QA process – possibly not even ad-hoc internally, since it’s hard to image that an Apple employee stumbling on this bug by accident would not bother to report it. Or maybe they did but even Apple’s own employees’ bug reports are ignored. Consistent, at least.

Since virtually every external drive should be encrypted, this suggests Apple either believes nobody actually uses external drives, or that they don’t care. At least that bug was mostly fixed in 14.1, although it took Apple a full month to get around to it.

Radical pink redesign

But perhaps the most glaring bug, in a very literal sense, was the rampant pinkification of the GUI.

As you can see, Sonoma brought with it a penchant for tinting things a gross light pink colour, including the rectangular extents of images that are otherwise invisible because they’re transparent.

WTF?!

I spent a lot of time debugging this. Over days, weeks, and months. Trying out new avenues as they came to me, even repeating some web & forum searches over and over again in frustrated desperation.

I only just today figured out what’s going on. 84 days later.

One of my first instincts – right on day 0 – was that it was some Quartz Debug thing, because it looks a lot like some of the tinting options Quartz Debug has for things like flashing screen updates or showing opaque regions. But the Quartz Debug app insisted everything was disabled. I enabled everything anyway, and then disabled it all again, in case that would jog something free. No bueno.

I then spent the better part of three months trying every superstition, every rumour or snippet of hearsay online, that might in any way related to this problem or conceivably fix it. Nothing helped.

Epiphany

It was last night, when I was fiddling with some images in a gallery here on my website, that I discovered that adjusting the CSS dimensions of an image by just a single pixel could pinkify them. Most importantly, restoring the dimensions to the actual image dimensions would fix them. I’d already observed something similar to that, in various apps and websites, but it felt different to be directly triggering the problem myself.

That kept me up half the night, tossing and turning in bed, highly suspicious that this was a pivotal clue, but unsure what it pointed to.

Eventually, it led me right back to my very first instinct – Quartz Debug – and the tentative conclusion that this was in fact a Quartz Debug problem even if Quartz Debug claimed otherwise. I become suspicious that the app was either not showing me the full range of settings, or just plain wrong about them.

And that was the insight I needed – right on the very first page of search results for “quartz debug defaults” was a link to Testing and Troubleshooting High-Resolution Content. Buried behind a disclosure triangle near the bottom is the magic string:

CGContextHighlight2xScaledImages

That was set to YES, on my system.

Deleting it fixes the issue once affected apps are restarted (something Apple’s own documentation fails to mention – kind of an important step, Apple!).

defaults delete -g CGContextHighlight2xScaledImages

The magic incantation to exorcise Barbie

In theory you can also disable this from the Quartz Debug app itself, it’s just hidden – in the Tools menu, not the actual settings window, is an item labelled “Color 1x Artwork”. Note how the setting applies even when Quartz Debugging is disabled.

So, wait, how did this come about?

I have no idea why this only cropped up when Sonoma was installed. Either Sonoma turned this on (unlikely…?) or it was already on yet wasn’t having any effect in prior OS releases (from 2017 onwards, at least, since my iMac Pro never exhibited this issue before Sonoma).

I know for sure that this setting did work at least going back over a decade, as I remember using this – on a different computer – for doing then-new-fangled Retina development… but I’m pretty sure I’ve never set this on my iMac Pro since it way post-dates the introduction of Retina. Conceivably it was pulled across by Apple’s Migration Assistant. Though, again, why did no earlier version of the OS, going back over a decade, exhibit this behaviour if it was supposedly enabled the whole time?

I know it’s not just me that’s been afflicted by this to date, as I found a sporadic few reports of this going back years, e.g. Highly abnormal graphic glitch on yosemite / retina: all white backgrounds appear pink. It pisses me off immensely that this page has exactly the keywords I was searching for right from the start – right in its title! – yet neither Bing nor Google can find it.

Apple’s response? “Fuck you” in mime.

I actually filed a pretty detailed bug report with Apple about this, the day I discovered it (September 19th, 2023). FB13190068.

Of course, they never responded in the slightest.

I provided a bunch of example images, even a screen recording to show how the pink comes and goes in certain circumstances.

I studiously tested every new macOS update and updated the bug report to note that they each had not fixed it.

The morale of this is, as always, that filing bug reports with Apple is an infuriating waste of time. Mea culpa.