All in all… meh.

28% fewer pixels for 34% 42% fewer dollars (47% if you’re talking nano-textured) – so technically better value, if you don’t really care about screen real-estate. But that extra real estate is really valuable, and Apple have now apparently ceded the large display market to… well, mostly the tumbleweeds. Sure, there’s technically other 6k displays, like the LG, the Dell, or the Asus, but while they have some advantages – less than half the price, most notably – they have real big disadvantages – like low brightness and poor contrast ratios.

4⨉ the backlight zones is a significant improvement, I’ll grant Apple that. But it doesn’t eliminate the blooming that was problematic with the Pro Display XDR, merely reduces it. In an era of OLED displays – hell, my old LG TV from nearly a decade ago has an OLED display; this ain’t new technology – a brand new, ostensibly-high-end “studio” display still running on LED backlighting is just sad.

The extra brightness of the Studio Display XDR is merely nice – an extra 25% isn’t a big difference (certainly nothing like the +200% or so in going from a competing 6k display to the Apple Pro Display XDR). Props to Apple for the improvement, but it’s minor.

It’s interesting that the new, smaller, lower-resolution Studio Display XDR is nearly the same spatial volume as its big sister (and not as much lighter as its reduced resolution and screen dimensions would suggest). I wonder if that’s dictated by thermals?

I didn’t bother including the audio & camera aspects because I’m genuinely confused as to who, in the market for an expensive display, would care? If you’re doing photography there’s no sound anyway, and if you’re doing videography in this price range you should be using real speakers or headphones.

And the camera… sigh… I miss the iSight, which gave you a much better camera – thanks to physics – that you could optionally buy and use. And look at it – it was a beautiful design that functioned really well, that would have fit in perfectly with the Pro Display XDR!

I’m also choosing to overlook the firmware, which I assume uses the same weird, bastardised, glitchy version of iOS as the prior Studio Display model.

After more than six years, I was hoping for an improved Pro Display XDR, not merely a small version.

1. Only when used with Macs which support DSC (Display Stream Compression), otherwise the USB-C ports are limited to USB 2.0 (400 Mb/s). ↩︎

The externalities cost estimates might be a little tongue-in-cheek, but honestly, are they all that wrong? One small irritation at the wrong moment can ricochet my happy mood off into the doldrums, and Apple’s products produce a hundred “small” irritations every day – which compound in their irritation when you see them software update after software update, year after year, product after product. It’s hard not to take it personally. Like Apple is deliberately being cruel.

Daniel Kennett wrote, in his memorial to Aperture, about how the Mac-using experience wasn’t actually rainbows and perfection even back in whatever you personally feel was the golden age (I’m with him that circa System 7.1 was glorious, though the early days of Mac OS X were also very special to me). Which is true – fire up your favourite old Mac on Infinite Mac and see the strength of your rose-tinted nostalgia glasses.

But, the big difference is that Apple back then was a relatively tiny company struggling just to survive in a brutal industry dominated by soulless, greedy monsters.

There is a point at which mere indifference or incompetence transitions into negligence, and it’s long before you become one of the wealthiest companies on the planet with a veritable army of engineers.

Having worked at Apple – among other big tech companies – I can say with confidence that there’s no valid reason why they cannot fix long-standing, infamous bugs. It’s. Not. That. Hard. One half-decent engineer could fix everything listed on Bugs Apple Loves in six months, single-handed.

It’s not apparent why that doesn’t happen, but it’s not that Apple are technically incapable of it, and it cannot be that they’re unaware, so it must be that they’re choosing not to.

The primary reason to extract the images exactly as is, bit-for-bit-identical, is that they’re typically already lossy-compressed (usually JPEG). Recompressing them will introduce further losses or increase the file size¹, or both.

Kudos to Josef Habr for suggesting The Unarchiver on StackExchange – I would never have found it on my own, even though I already had it installed and use it occasionally (for more traditional archive file formats, like Zip or StuffIt).

Frustratingly, Josef’s post aside, none of the recommendations you read online mention The Unarchiver, pointing instead to other options which are harder to install, harder to use, and don’t extract the images correctly. Worst of all, many people falsely claim that their suggested approach will extract the images losslessly. Examples include:

• pdfimages from Poppler – silently re-encodes images in some cases (contrary to what its documentation and users claim), such as if they have non-sRGB colour profiles, and fails to preserve the embedded ICC profile. Worse, the developers have known about this for nearly a decade and refuse to fix it.
• pdftoppm (et al) – explicitly convert the embedded images into another format, which while usually a lossless format (e.g. PNG or PPM) by default, still requires you to then re-encode them for use online etc. Plus, they typically don’t preserve ICC profiles.
• ImageMagick – doesn’t extract the images, merely renders the whole PDF page(s) as images, requiring further post-processing and inevitably reducing the image quality (due to mismatched output resolution and pixel alignment vs the embedded images’).
• Exporting pages as images from Preview or Acrobat Reader – obviously doesn’t preserve the extracted images as-is, requires re-encoding them with additional compression losses, etc.
• Screenshots via Preview or Acrobat Reader – ugh, I can’t even.
• Various websites – I mean, they might work, but why upload your personal data to some skeezy website when it’s easy and fast to just use The Unarchiver locally?

I saw a recommendation for File Juicer, but unfortunately the free trial doesn’t work for me – it claims it’s already expired – so I was unable to check that it actually works. Plus, it’s not free (USD$19 at time of writing) so that’s a strong disincentive compared to The Unarchiver.

1. It is possible, with some newer formats like AVIF, to recompress a JPEG in a way that arguably improves the image quality while also reducing the file size. AVIF encoders typically have some built-in smarts to recognise JPEG artefacts specifically, and try to remove them – a direct benefit visually since the artefacts are ugly and a benefit to [re]compression since the encoder then doesn’t need to waste time & output bits trying to preserve the artefacts.
   
   But, utilising this feature can require more care during compression to find the right trade-offs and ensure the result is in fact as good or better than the original – and in any case, AVIF recompression will work better from the original JPEG than a mangled version. ↩︎

Using their Explainfile tool to dig into the log files, the clue seems to be:

  - line 288 - 2024-12-16 16:16:17 0000000646 - ERROR: UpdateBzDoneRegardingFlsToBeExp - Z_B_TOO_MANY_CHUNKS bz_done_ line chunk related, numBytesInLargeFile=1099512156951, totNumChunks=104858, bz_done_line_is: 5	! …
  - line 522 - 2024-12-16 16:18:46 0000000646 - ERROR - bz_done_ INCONSISTENCY_FOUND - 20241216161846 - BadBadBadChunkRecord hexAsciiVal=0x78 - AfterBzdoneLargeFileAnalysis: chunkSeq=100001, highestChunkSeqSeen=104857, fileIdOfLargeFile=00000000002c53cd, dateTimeOfLargeFile=20231217091843, XYXBXXX_FILE_NAME: …

Admittedly I’m guessing somewhat, since that’s a rather reader-hostile log message, but the combination of the Z_B_TOO_MANY_CHUNKS error mnemonic and chunkSeq=100001 (because of its proximity to the arbitrary round number 100,000) strongly suggests that Backblaze is imposing a 100,000 chunk limit. Since chunks are 10 MB each, that’s exactly 1 TB.

This is unequivocally at odds with what they claim repeatedly on their website, on pages like What Backblaze Backs Up and File Sizes.

It’s not clear to me why this is suddenly a problem; is this a newly-imposed limit? It’s possible that a month ago I removed some exclusion on the file, but I don’t remembering doing that and I can see no reason why I would have excluded it to begin with. If it is newly imposed, that would imply it’s also retroactive – that Backblaze actually deleted the existing backup of the file from their servers, thus causing the client app to try uploading it again.

I reached out to their technical support, of course, but thus far have only received mindless responses – restart your computer, reinstall Backblaze, etc.

Update

I received no further response from Backblaze’s technical support. They asked me to send them the log files, which I did on January 2nd, 2025, and they never responded again.

As of this update (February 4th 2025) their website still falsely advertises support for files of any size.

Addendum

I was surprised to see that many folks on HackerNews were surprised by the idea of a 1 TiB file. I certainly agree that’s large, but it doesn’t seem unusual or inexplicable to me. In my case, this particular “problem” file is an encrypted, compressed disk image of the boot drive of a prior computer, that I saved when I upgraded to my current computer.

It’s true that I could probably throw it out at this point – it was just a precaution in case I forgot to migrate something over, so now (eight years later) it seems that either I made no such mistake or whatever I forgot to migrate doesn’t matter anyway. For now I’ve just manually excluded it from the backup, to work around Backblaze’s bugs.

There are other cases in which I’ve had files over 1 TiB, though – e.g. video files:

• With some cameras and recording modes (e.g. documentarian, interviews) it’s in principle easy to exceed 1 TiB per file. e.g. the Nikon Z9 & Z8 record around 700 MB/s for 8k60 N-RAW, which is about 24 minutes per TiB.
  
  Note that I don’t recall if I personally have ever actually exceeded 1 TiB this way. I mention it mainly for illustration. It’s also possible that the Z9 & Z8 shard large recordings into multiple files (I don’t recall seeing this in years – not since the 4 GiB per-file limit of cameras a decade ago – but perhaps I’ve just not had a single recording large enough).
• Usually (for me) it’s output files that are largest, since they can combine many clips. I use Final Cut Pro and its video compression capabilities aren’t great, so I export essentially lossless ProRes and then use ffmpeg or Handbrake for the real compression. ProRes 422 HQ is nearly a gigabyte per second for 8k60, so it takes less than twenty minutes of video to exceed 1 TiB. Fortunately these large intermediaries only have to live as long as the final compression takes (though that can be days, especially with the latest formats like AV1).

If any view in the [active] window contains a Toggle – even one that’s disabled or hidden – then Continuity Camera (re. ImportFromDevicesCommands and importableFromServices) doesn’t work; all the submenu items under “Import from iPhone or iPad” are disabled.

I don’t know if this is truly specific to Toggle, that’s just the example case I happen to have isolated [first?].

What’s really weird is that once a Toggle has ever been displayed, even if you subsequently remove it from the view hierarchy entirely the “Import from iPhone or iPad” submenu items all remain disabled.

import SwiftUI

@main
struct Example: App {
    @State var breakImportFromiDevice = true
    @State var text = ""

    var body: some Scene {
        WindowGroup {
            VStack {
                TextField("Input", text: $text) // Doesn't break anything.

                if breakImportFromiDevice {
                    Toggle("Break import from iDevice", isOn: $breakImportFromiDevice)
                }
            }
            .importableFromServices(action: { (images: [NSImage]) -> Bool in
                print("Load image!")
                return true
            })
        }.commands {
            ImportFromDevicesCommands()
        }
    }
}

There are some circumstances in which this gets “unbroken” during interactions with other views and so forth, which is 100% reproducible in my real app but I have no idea what the reason is. The steps involved in my real app are kinda ridiculous (and not in any way remotely a viable workaround) and make absolutely no sense – it only “unbreaks” when a specific view is in a specific weird state (itself kind of the result of a bug, albeit a benign one). And that weird state is merely whether it’s displaying one image or another – which as far as SwiftUI is concerned is not even a difference in view state, since I’m just swapping NSImages under the cover.

I figured it must be something to do with view focus, but after much experimentation I believe I can conclusively rule that out. No matter which view has focus, or how focus is configured, or which views are even focusable at all, the problem persists. Likewise for window focus and key state. And, [accessability-]focusable views other than Toggle – e.g. TextField – don’t cause any issues.

Frankly it’s baffling, and a mite infuriating. I can’t even conceive of how SwiftUI can be so incredibly broken regarding such basic functionality, and the apparent interaction of GUI elements that have absolutely no business together.

Tangentially, a few things I’ve noticed about this Continuity Camera feature:

• “Add Sketch” doesn’t do anything. Unlike the other options, which open the camera app on the target iDevice, it has no effect. 🤷‍♂️
• Within the Finder, you can right-click empty whitespace within a folder, and the contextual menu has this “Import from iPhone or iPad” option at the bottom. That’s pretty handy – until now I’d been doing it the “hard” way by taking a photo on my iPhone and AirDropping it across to my Mac.
  
  I’d never noticed that feature prior to debugging this problem (I wanted to confirm that multiple other apps worked just fine; that it wasn’t an issue with Continuity Camera system-wide).
• TextEdit has the same feature but tweaks the wording to “Insert…” rather than “Import…”, which I thought is both a nice touch and frustratingly not something you can do in your own apps (at least, not in SwiftUI). 😕

NSPasteboard mutates itself simultaneously from the main thread and the global concurrent Dispatch pool, w.r.t. to its internal type cache. This is surprisingly trivial to reproduce (sample code below) by just dropping, e.g. a file promise (such as by opening a PNG in Preview, revealing the thumbnails sidebar, and then dragging the thumbnail onto the sample project’s window).

import SwiftUI

struct ContentView: View {
    var body: some View {
        Rectangle().onDrop(of: NSImage.imageTypes, isTargeted: nil) { _ in
            let pb = NSPasteboard(name: .drag)

            _ = pb.pasteboardItems // Seems to be necessary for the crash.

            _ = NSImage.imageTypes // Not strictly necessary for the crash, but seems to make it more likely. 🤷‍♂️

            return true
        }
    }
}

Judging from the callstack that runs in the concurrent pool, this is specific to file promises (and that seems to match my experience – it only crashes for some test cases, all of which involve file promises being present in the drag pasteboard at the time of the drop).

Since this bug causes semi-random memory corruption, it manifests in a large number of ways – not all of which are all that helpful. But at least one case I’ve seen a few times is helpful, as it clearly shows the offending internal NSPasteboard code running concurrently with itself, e.g.:

Main queue / thread:
#0	0x00007ff80108dab5 in _platform_bzero$VARIANT$Haswell ()
#1	0x000000010df8774d in GuardMalloc_mallocInternal ()
#2	0x00007ff90792542f in stack_logging_lite_malloc ()
#3	0x00007ff800ea8733 in _malloc_zone_malloc_instrumented_or_legacy ()
#4	0x00007ff800f39a72 in _vasprintf ()
#5	0x00007ff800f16922 in asprintf ()
#6	0x00007ff80125655a in -[NSObject(NSObject) __dealloc_zombie] ()
#7	0x00007ff8020e039a in -[NSConcreteMapTable dealloc] ()
#8	0x00007ff804876983 in -[NSPasteboard _updateTypeCacheIfNeeded] ()
#9	0x00007ff8048763df in -[NSPasteboard _typesAtIndex:combinesItems:] ()
#10	0x00007ff804aad148 in NSCoreDragReceiveMessageProc ()
#11	0x00007ff807517b1a in CallReceiveMessageCollectionWithMessage ()
#12	0x00007ff8075124fa in DoMultipartDropMessage ()
#13	0x00007ff8075122ce in DoDropMessage ()
#14	0x00007ff8075159a9 in CoreDragMessageHandler ()
#15	0x00007ff8011d776b in __CFMessagePortPerform ()
#16	0x00007ff80113e5b7 in __CFRUNLOOP_IS_CALLING_OUT_TO_A_SOURCE1_PERFORM_FUNCTION__ ()
#17	0x00007ff80113e4ee in __CFRunLoopDoSource1 ()
#18	0x00007ff80113d166 in __CFRunLoopRun ()
#19	0x00007ff80113c112 in CFRunLoopRunSpecific ()
#20	0x00007ff80bb55a09 in RunCurrentEventLoopInMode ()
#21	0x00007ff80bb55646 in ReceiveNextEventCommon ()
#22	0x00007ff80bb55561 in _BlockUntilNextEventMatchingListInModeWithFilter ()
#23	0x00007ff8047acc61 in _DPSNextEvent ()
#24	0x00007ff8050c0dc0 in -[NSApplication(NSEventRouting) _nextEventMatchingEventMask:untilDate:inMode:dequeue:] ()
#25	0x00007ff80479e075 in -[NSApplication run] ()
#26	0x00007ff804771ff3 in NSApplicationMain ()
#27	0x00007ff90dc24557 in ___lldb_unnamed_symbol57096 ()
#28	0x00007ff90e31fe64 in ___lldb_unnamed_symbol104448 ()
#29	0x00007ff90e6e63ff in static SwiftUI.App.main() -> () ()
#30	0x000000010dfa5cce in static NSPasteboardItem_CrashApp.$main() ()
#31	0x000000010dfa5d69 in main at /Users/SadPanda/Documents/NSPasteboardItem Crash/NSPasteboardItem Crash/NSPasteboardItem_CrashApp.swift:11
#32	0x00007ff800cd5366 in start ()

Dispatch concurrent queue (default QoS):
#0	0x00007ff80111b45c in -[__NSSetM addObject:] ()
#1	0x00007ff80487692e in -[NSPasteboard _updateTypeCacheIfNeeded] ()
#2	0x00007ff8048763df in -[NSPasteboard _typesAtIndex:combinesItems:] ()
#3	0x00007ff804aa9597 in -[NSPasteboard _canRequestDataForType:index:usesPboardTypes:combinesItems:] ()
#4	0x00007ff804fdd161 in -[NSPasteboard _dataForType:index:usesPboardTypes:combinesItems:securityScoped:] ()
#5	0x00007ff804aa7c4b in -[NSPasteboardItem dataForType:] ()
#6	0x00007ff8055804af in -[NSFilePromiseReceiver receivePromisedFilesAtDestination:options:operationQueue:reader:] ()
#7	0x00007ff90e787b51 in ___lldb_unnamed_symbol131674 ()
#8	0x00007ff90e9bc340 in ___lldb_unnamed_symbol148147 ()
#9	0x00007ff8020d00ba in __NSBLOCKOPERATION_IS_CALLING_OUT_TO_A_BLOCK__ ()
#10	0x00007ff8020cffb8 in -[NSBlockOperation main] ()
#11	0x00007ff8020cff4b in __NSOPERATION_IS_INVOKING_MAIN__ ()
#12	0x00007ff8020cf1ec in -[NSOperation start] ()
#13	0x00007ff8020cef0d in __NSOPERATIONQUEUE_IS_STARTING_AN_OPERATION__ ()
#14	0x00007ff8020cedde in __NSOQSchedule_f ()
#15	0x000000010e68ce7d in _dispatch_block_async_invoke2 ()
#16	0x000000010e67ca7b in _dispatch_client_callout ()
#17	0x000000010e67fa09 in _dispatch_continuation_pop ()
#18	0x000000010e67eae8 in _dispatch_async_redirect_invoke ()
#19	0x000000010e6906a9 in _dispatch_root_queue_drain ()
#20	0x000000010e6911ba in _dispatch_worker_thread2 ()
#21	0x000000010dfb832f in _pthread_wqthread ()
#22	0x000000010dfbebeb in start_wqthread ()

There doesn’t appear to be any workaround (short of not supporting drops at all!).

The more complicated the drop handler the more likely it is to promptly crash upon drop – in my real code with a non-trivial handler, it’s virtually guaranteed to crash on the second drop containing a file promise, while in the vastly reduced sample code (above) it can take dozens of drops before it finally crashes outright.

I have not directly tested whether this NSPasteboard bug occurs in the absence of SwiftUI, so I don’t strictly know if the root cause is in AppKit or SwiftUI. However, since most SwiftUI apps don’t support drag-and-drop, but plenty of AppKit ones do and manage to not crash when given the exact same test cases, I do strongly suspect SwiftUI is causing this somehow.

Follow-up (September 12th, 2024)

I actually received a response from Apple, from a real human (or at least a convincing AI). Ultimately their response didn’t help as it contained some mistakes, but I’m hopeful there’ll be more follow-up and a productive conclusion. And in the interim, they did assert a few things which are important to know, and are not otherwise documented by Apple:

• SwiftUI’s onDrop(of:isTargeted:perform:) method makes no claims or promises as to what thread / queue it executes the closure on, and in fact according to the anonymous Apple engineer it never executes the closure on the main thread.
  
  Now, while that may be the intent, the reality of that is wrong – in my experience it always executes the closure on the main thread (which makes a lot of sense to me as drag-and-drop event handling in AppKit has always been on the main thread in practice).
  
  Nonetheless, Apple says one cannot rely on the current behaviour and should in fact assume it never executes on the main thread (though in practice that means you have to check, not assume, since if you blindly do something like DispatchQueue.main.sync { … } in your drop handler your code will deadlock, today).
• NSPasteboard is not safe to use outside the main thread / queue.
  
  This isn’t documented anywhere public – not in NSPasteboard‘s documentation itself, nor the ancient Application Kit Framework Thread Safety documentation.
  
  I wouldn’t be surprised if it’s broadly true, as the AppKit APIs involving it always seemed main-thread centric anyway (all the handlers and delegate methods involving pasteboards are invoked on the main thread, in my experience). And it’s generally best to assume everything in AppKit is main-thread-only unless it’s explicitly documented otherwise.
  
  However, it’s important to note that Apple’s own code doesn’t follow this rule – e.g. NSFilePromiseReceiver, internally, uses NSPasteboard from the global concurrent queue.

Even though Apple’s initial response to this bug report hasn’t been all that fruitful, I do want to emphasise the fact that they did respond, which was a pleasant surprise and very much appreciated.

I find my hand is often forced by APIs I don’t control (most often Apple’s APIs). e.g. data source or delegate callbacks that are synchronous¹ and require you to return a value, but in order to obtain that value you have to run async code (perhaps because yet again that’s all you’re given by 3rd parties, or because that code makes sense to be async and is used happily as such in other places and you don’t want to have to duplicate it in perpetuity just to have a sync version).

If that asynchronosity is achieved through e.g. GCD or NSRunLoop or NSProcess or NSTask or NSThread or pthreads, it’s easy. There are numerous ways to synchronously wait on their tasks. In contrast, Swift Concurrency really doesn’t want you to do this. The language and standard library take an adamant idealogical position on this – one which is unfortunately impractical; a spherical chicken in a vacuum².

Nonetheless, despite Swift’s best efforts to prevent me, I believe I’ve come up with a way to do this. It appears to work reliably, given fairly extensive testing. Nonetheless, I do not make any promises. Use at your own risk.

If you know of a better way, please do let me know (e.g. in the comments below).

import Dispatch

extension Task {
    /// Executes the given async closure synchronously, waiting for it to finish before returning.
    ///
    /// **Warning**: Do not call this from a thread used by Swift Concurrency (e.g. an actor, including global actors like MainActor) if the closure - or anything it calls transitively via `await` - might be bound to that same isolation context.  Doing so may result in deadlock.
    static func sync(_ code: sending () async throws(Failure) -> Success) throws(Failure) -> Success { // 1
        let semaphore = DispatchSemaphore(value: 0)

        nonisolated(unsafe) var result: Result<Success, Failure>? = nil // 2

        withoutActuallyEscaping(code) { // 3
            nonisolated(unsafe) let sendableCode = $0 // 4

            let coreTask = Task<Void, Never>.detached(priority: .userInitiated) { @Sendable () async -> Void in // 5
                do {
                    result = .success(try await sendableCode())
                } catch {
                    result = .failure(error as! Failure)
                }
            }

            Task<Void, Never>.detached(priority: .userInitiated) { // 6
                await coreTask.value
                semaphore.signal()
            }

            semaphore.wait()
        }

        return try result!.get() // 7
    }
}

Elaborating on some of the odder or less than self-explanatory aspects of this:

1. The closure parameter must be sending otherwise this deadlocks if e.g. you call it from the main thread (even if the closure, and all its transitive async calls, are not isolated to the main thread). I don’t understand why this happens – it’s possibly explicable and working as intended, but I wonder if it’s simply a bug. Nobody has been able to explain why it happens.
   
   Note: in the initial version of this post I accidentally omitted this essential keyword. I apologise for the error, and hope it didn’t cause grief for anyone.
2. Since there’s no sync way to retrieve the result of a Task, the result has to be passed out through a side-channel. The nonisolated(unsafe) is to silence the Swift 6 compiler’s erroneous error diagnostics about concurrent mutation of shared state.
3. Task constructors only accept escaping closures, even though as they’re used here the closure never actually escapes. Fortunately the withoutActuallyEscaping escape hatch is available.
4. code isn’t @Sendable – since it doesn’t actually have to be sent in the sense of executing concurrently – so trying to use it in the Task closure below, which is @Sendable, results in an erroneous compiler error (“Capture of 'code' with non-sendable type '() async throws(Failure) -> Success' in a @Sendable closure“). Assigning to a variable lets us apply nonisolated(unsafe) to disable the incorrect compiler diagnostic.
5. Several key aspects happen on this line:
   • It’s important to use a detached task, in case we’re already running in an isolated context (e.g. the MainActor) as we’re going to block the current thread waiting on the task to finish, via the semaphore.
   • The task logically needs to be run at the current task’s priority (or higher) in order to ensure it does actually run (re. priority inversion problems), although I’m not sure that technically matters here since we’re blocking in a non-await way anyway. One could use Task.currentPriority here, but I’ve chosen to hard-code the highest priority (userInitiated) because it’s not great to block (in a non-await manner) on async code; although async code isn’t necessarily slow, I feel it’s wise to eliminate task prioritisation as a variable.
   • This closure must be explicitly marked as @Sendable as by default the compiler mistakenly infers it to be not @Sendable, even though all closure arguments to Task initialisers have to be @Sendable. The compiler diagnostics in this case are frustratingly obtuse and misleading (although the sad saving grace is that this is a relatively common problem, so once you hit it enough times you start to develop a spidey sense for it).
6. This otherwise pointless second Task is critical to prevent withoutActuallyEscaping from crashing.
   
   withoutActuallyEscaping basically relies on reference-counting – it records the retain count of its primary argument going in (code in this case) and compares that to the retain count going out – if they disagree, it crashes. There’s no way to disable or directly work around this³.
   
   That’s a problem here because if we just signal the semaphore in the first task, right before exiting the task, we have a race – maybe the task will actually exit before the signal is acted on (semaphore.wait() returns and allows execution to exit the withoutActuallyEscaping block), but maybe it won’t. Since the task is retaining the closure, it must exit before we wake up from the semaphore and exit the withoutActuallyEscaping block, otherwise crash.
   
   The only way I found to ensure the problematic task has fully exited – after hours of experimenting, covering numerous methods – is to wait for it in a second task. Surprisingly, the second task – despite having a strong reference to the first task – seemingly doesn’t prevent the first task from being cleaned up. This makes me suspicious, but despite extensive testing I’m unable to get withoutActuallyEscaping to crash when using this workaround.
7. There’s no practical way to avoid this forced unwrap, even though it’s impossible for it to fail unless something goes very wrong with Swift’s built-ins (like withoutActuallyEscaping and Task).
   
   If you don’t wish to use typed throws, you could unwrap it more gently and throw an error of your own type if it’s nil, but it’s extra work and a loss of type safety for something that realistically cannot happen.

1. Specifically meaning “not run through Swift Concurrency, as async functions / closures”. Lots of APIs will execute the callback on the main thread, which is the most difficult case, but even those that execute on a user-specified GCD queue aren’t helpful here – at least, not until assumeIsolated actually works. ↩︎
2. Incidentally, Wikipedia seems to think the canonical version of the joke is about spherical cows, but I’ve only ever heard it about chickens. Indeed the very first known instance of the joke used chickens, and all the pop-culture uses of it that I could find use chickens (most notably the ninth episode of The Big Bang Theory). ↩︎
3. There’s no unsafeWithoutActuallyEscaping that forgoes the runtime checking, nor any environment variables or similar that influence it; the check is always included and cannot be disabled at runtime. Even when it’s unnecessary or – as in this case – outright erroneous.
   
   Nor is there a way to replicate withoutActuallyEscaping‘s core functionality of merely adding @escaping to the closure’s signature (e.g. via unsafeBitCast or similar) because it’s a special interaction with the compiler’s escape checker which is evaluated purely at compile-time (whether a closure is escaping or not is not actually encoded into the output binary nor memory representation of closures – the only time escapingness ever leaks into the binary is when you use withoutActuallyEscaping and the compiler inserts the special runtime assertion). ↩︎

Circa 2011, not long after I joined Google, I pitched a 20% project idea to my boss. In a nutshell, I wanted to optimise traffic light systems to improve traffic throughput. My motivation was to reduce wasted fuel – and therefore pollution and cost to drivers. Granted I didn’t have much more than the goal at that point – with some ideas about starting with simulations and models, and working with the Google Maps & Streetview teams, if I recall correctly. In any case, my boss said no. That wasn’t relevant to Google nor something Google was interested in, I was told.

Well, apparently now it is – Google has a publicly disclosed project “Green Light” to do exactly that. Evidently as far more than merely a 20% project (although I’m curious if that’s how it started).

I always felt the denial by management was misguided. It seemed at odds with official company policy. But, contrary to what this post might seem to imply, it didn’t weigh on me. I was a bit miffed, but largely forgot about it until I happened to hear about Green Light and was reminded. Now it’s perhaps a sliding doors thought exercise.

Executive decrees, employee handbooks, even public promises, are all weak against undermining management.

Canary Wharf traffic lights photo by Martin Pearce, CC BY-ND 2.0.

1. The ambiguity there reflects the wildly differing opinions I encountered while at Google starting in 2010. Though there was some official guidance about 20% time (e.g. in the employee handbook), asserting in writing that it did in fact exist and was supposedly encouraged, in my first- and second-hand experience management were not at all keen on it. I met few Googlers who actually worked on a 20% project, and some kept it secret from management. ↩︎

Nice gal Amanda

I submitted the following feedback, not really expecting much of it. In fact, I fully expected some useless, boilerplate response that completely ignored the point and directed me to the upload requirements page (despite it being the very first thing I myself pointed to) or was nothing but mail-merge platitudes.

And in retrospect I could have phrased this more respectfully, too. So I wasn’t even going in as a great customer.

What I got surprised me greatly, in the best way:

Holy shit. A humane response from a real human. Better yet, a couple of hours later that was followed by (unprompted by me):

Even though these responses made no conclusive promises (regarding support for these file formats), I was thrilled. Beyond the pleasant tone and clear demonstration that Amanda actually read and comprehended my feedback, just knowing that my feedback actually got to the intended recipients (the dev team) is heartening; it made me feel good about choosing to use Flickr for all these years.

But then…

Bad guy Marc

Coincidentally, while Amanda was getting back to me I ran into a bug in Flickr’s file upload API. It was apparent from the logs that Flickr was the cause of the issue, but nonetheless I took the time to first confirm that with the Photo Upload plug-in’s author. Then, I submitted the bug report to Flickr:

HTTP/1.1 504 Gateway Time-out
Content-Type: text/html
Content-Length: 1033
Connection: keep-alive
Server: CloudFront
Date: Thu, 18 Jul 2024 01:42:43 GMT
X-Cache: Error from cloudfront
Via: 1.1 CENSORED.cloudfront.net (CloudFront)
X-Amz-Cf-Pop: CENSORED
X-Amz-Cf-Id: CENSORED

<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" "http://www.w3.org/TR/html4/loose.dtd">
<HTML><HEAD><META HTTP-EQUIV="Content-Type" CONTENT="text/html; charset=iso-8859-1">
<TITLE>ERROR: The request could not be satisfied</TITLE>
</HEAD><BODY>
<H1>504 ERROR</H1>
<H2>The request could not be satisfied.</H2>
<HR noshade size="1px">
CloudFront attempted to establish a connection with the origin, but either the attempt failed or the origin closed the connection.
We can't connect to the server for this app or website at this time. There might be too much traffic or a configuration error. Try again later, or contact the app or website owner.
<BR clear="all">
If you provide content to customers through CloudFront, you can find steps to troubleshoot and help prevent this error by reviewing the CloudFront documentation.
<BR clear="all">
<HR noshade size="1px">
<PRE>
Generated by cloudfront (CloudFront)
Request ID: CENSORED
</PRE>
<ADDRESS>
</ADDRESS>
</BODY></HTML>

* Host up.flickr.com:443 was resolved.
* IPv6: (none)
* IPv4: 13.227.78.140
*   Trying 13.227.78.140:443...
* Connected to up.flickr.com (13.227.78.140) port 443
* ALPN: curl offers http/1.1
* (304) (OUT), TLS handshake, Client hello (1):
} [315 bytes data]
* (304) (IN), TLS handshake, Server hello (2):
{ [122 bytes data]
* (304) (IN), TLS handshake, Unknown (8):
{ [25 bytes data]
* (304) (IN), TLS handshake, Certificate (11):
{ [4967 bytes data]
* (304) (IN), TLS handshake, CERT verify (15):
{ [264 bytes data]
* (304) (IN), TLS handshake, Finished (20):
{ [36 bytes data]
* (304) (OUT), TLS handshake, Finished (20):
} [36 bytes data]
* SSL connection using TLSv1.3 / AEAD-AES128-GCM-SHA256 / [blank] / UNDEF
* ALPN: server accepted http/1.1
* Server certificate:
*  subject: CN=flickr.com
*  start date: Feb  5 00:00:00 2024 GMT
*  expire date: Mar  4 23:59:59 2025 GMT
*  issuer: C=US; O=Amazon; CN=Amazon RSA 2048 M02
*  SSL certificate verify ok.
* using HTTP/1.x
> POST /services/upload/ HTTP/1.1
> Host: up.flickr.com
> User-Agent: curl/8.6.0
> Accept: */*
> Authorization: OAuth oauth_signature="CENSORED",oauth_token="CENSORED",oauth_version="1.0",oauth_nonce="CENSORED",oauth_timestamp="CENSORED",oauth_signature_method="HMAC-SHA1",oauth_consumer_key="CENSORED"
> Content-Length: 123113221
> Content-Type: multipart/form-data; boundary=------------------------cbLPLbBocYE2oLsFDbSmbX
> 
} [233 bytes data]

  0  117M    0     0    0  128k      0   453k  0:04:25 --:--:--  0:04:25  452k
  5  117M    0     0    5 6848k      0  5670k  0:00:21  0:00:01  0:00:20 5669k
  6  117M    0     0    6 7360k      0  3068k  0:00:39  0:00:02  0:00:37 3068k
  6  117M    0     0    6 7360k      0  2165k  0:00:55  0:00:03  0:00:52 2165k
  6  117M    0     0    6 7488k      0  1780k  0:01:07  0:00:04  0:01:03 1780k
  7  117M    0     0    7 9280k      0  1773k  0:01:07  0:00:05  0:01:02 1848k
  9  117M    0     0    9 11.5M      0  1906k  0:01:03  0:00:06  0:00:57  997k
 12  117M    0     0   12 14.3M      0  2025k  0:00:59  0:00:07  0:00:52 1508k
 14  117M    0     0   14 17.4M      0  2175k  0:00:55  0:00:08  0:00:47 2182k
 17  117M    0     0   17 20.4M      0  2270k  0:00:52  0:00:09  0:00:43 2682k
 19  117M    0     0   19 22.8M      0  2295k  0:00:52  0:00:10  0:00:42 2843k
 21  117M    0     0   21 24.9M      0  2274k  0:00:52  0:00:11  0:00:41 2730k
 23  117M    0     0   23 27.1M      0  2279k  0:00:52  0:00:12  0:00:40 2648k
 25  117M    0     0   25 30.0M      0  2324k  0:00:51  0:00:13  0:00:38 2567k
 27  117M    0     0   27 32.6M      0  2342k  0:00:51  0:00:14  0:00:37 2472k
 29  117M    0     0   29 34.9M      0  2350k  0:00:51  0:00:15  0:00:36 2462k
 32  117M    0     0   32 38.0M      0  2399k  0:00:50  0:00:16  0:00:34 2679k
 34  117M    0     0   34 40.5M      0  2403k  0:00:50  0:00:17  0:00:33 2705k
 36  117M    0     0   36 42.6M      0  2400k  0:00:50  0:00:18  0:00:32 2601k
 38  117M    0     0   38 45.6M      0  2432k  0:00:49  0:00:19  0:00:30 2691k
 41  117M    0     0   41 48.5M      0  2456k  0:00:48  0:00:20  0:00:28 2780k
 43  117M    0     0   43 51.5M      0  2487k  0:00:48  0:00:21  0:00:27 2772k
 46  117M    0     0   46 54.2M      0  2499k  0:00:48  0:00:22  0:00:26 2830k
 48  117M    0     0   48 56.5M      0  2489k  0:00:48  0:00:23  0:00:25 2810k
 50  117M    0     0   50 59.2M      0  2504k  0:00:48  0:00:24  0:00:24 2782k
 52  117M    0     0   52 62.1M      0  2526k  0:00:47  0:00:25  0:00:22 2811k
 55  117M    0     0   55 65.1M      0  2547k  0:00:47  0:00:26  0:00:21 2803k
 58  117M    0     0   58 68.8M      0  2589k  0:00:46  0:00:27  0:00:19 2990k
 61  117M    0     0   61 72.1M      0  2617k  0:00:45  0:00:28  0:00:17 3221k
 62  117M    0     0   62 73.8M      0  2556k  0:00:47  0:00:29  0:00:18 2791k
 65  117M    0     0   65 77.3M      0  2620k  0:00:45  0:00:30  0:00:15 3091k
 67  117M    0     0   67 79.5M      0  2600k  0:00:46  0:00:31  0:00:15 2873k
 71  117M    0     0   71 83.3M      0  2651k  0:00:45  0:00:32  0:00:13 2987k
 72  117M    0     0   72 85.5M      0  2638k  0:00:45  0:00:33  0:00:12 2753k
 73  117M    0     0   73 85.7M      0  2557k  0:00:47  0:00:34  0:00:13 2565k
 73  117M    0     0   73 85.8M      0  2494k  0:00:48  0:00:35  0:00:13 1728k
 76  117M    0     0   76 89.3M      0  2506k  0:00:47  0:00:36  0:00:11 1934k
 76  117M    0     0   76 89.3M      0  2439k  0:00:49  0:00:37  0:00:12 1150k
 76  117M    0     0   76 89.3M      0  2375k  0:00:50  0:00:38  0:00:12  726k
 76  117M    0     0   76 89.3M      0  2315k  0:00:51  0:00:39  0:00:12  705k
 76  117M    0     0   76 89.3M      0  2258k  0:00:53  0:00:40  0:00:13  680k
 76  117M    0     0   76 89.3M      0  2203k  0:00:54  0:00:41  0:00:13     0
 76  117M    0     0   76 89.3M      0  2151k  0:00:55  0:00:42  0:00:13     0
 76  117M    0     0   76 89.3M      0  2102k  0:00:57  0:00:43  0:00:14     0
 76  117M    0     0   76 89.3M      0  2054k  0:00:58  0:00:44  0:00:14     0
 76  117M    0     0   76 89.3M      0  2009k  0:00:59  0:00:45  0:00:14     0
 76  117M    0     0   76 89.3M      0  1966k  0:01:01  0:00:46  0:00:15     0
 76  117M    0     0   76 89.3M      0  1924k  0:01:02  0:00:47  0:00:15     0
 76  117M    0     0   76 89.3M      0  1885k  0:01:03  0:00:48  0:00:15     0
 76  117M    0     0   76 89.3M      0  1846k  0:01:05  0:00:49  0:00:16     0
 76  117M    0     0   76 89.3M      0  1810k  0:01:06  0:00:50  0:00:16     0
 76  117M    0     0   76 89.3M      0  1775k  0:01:07  0:00:51  0:00:16     0
 76  117M    0     0   76 89.3M      0  1741k  0:01:09  0:00:52  0:00:17     0
 76  117M    0     0   76 90.3M      0  1739k  0:01:09  0:00:53  0:00:16  231k
 77  117M    0     0   77 91.3M      0  1724k  0:01:09  0:00:54  0:00:15  436k
 78  117M    0     0   78 92.3M      0  1711k  0:01:10  0:00:55  0:00:15  652k
 79  117M    0     0   79 93.3M      0  1700k  0:01:10  0:00:56  0:00:14  886k
 80  117M    0     0   80 94.7M      0  1695k  0:01:10  0:00:57  0:00:13 1186k
 81  117M    0     0   81 96.0M      0  1689k  0:01:11  0:00:58  0:00:13 1166k
 83  117M    0     0   83 97.7M      0  1690k  0:01:11  0:00:59  0:00:12 1319k
 84  117M    0     0   84 99.5M      0  1692k  0:01:11  0:01:00  0:00:11 1478k
 87  117M    0     0   87  102M      0  1714k  0:01:10  0:01:01  0:00:09 1870k
 89  117M    0     0   89  104M      0  1719k  0:01:09  0:01:02  0:00:07 1995k
 90  117M    0     0   90  106M      0  1725k  0:01:09  0:01:03  0:00:06 2137k
 91  117M    0     0   91  107M      0  1718k  0:01:09  0:01:04  0:00:05 2042k
 92  117M    0     0   92  108M      0  1705k  0:01:10  0:01:05  0:00:05 1861k
 93  117M    0     0   93  109M      0  1700k  0:01:10  0:01:06  0:00:04 1521k
 94  117M    0     0   94  110M      0  1683k  0:01:11  0:01:07  0:00:04 1237k
 94  117M    0     0   94  110M      0  1664k  0:01:12  0:01:08  0:00:04  896k
 95  117M    0     0   95  111M      0  1651k  0:01:12  0:01:09  0:00:03  803k
 95  117M    0     0   95  112M      0  1621k  0:01:14  0:01:10  0:00:04  633k
 95  117M    0     0   95  112M      0  1615k  0:01:14  0:01:11  0:00:03  497k
 95  117M    0     0   95  112M      0  1595k  0:01:15  0:01:12  0:00:03  419k
 96  117M    0     0   96  113M      0  1585k  0:01:15  0:01:13  0:00:02  510k
 97  117M    0     0   97  114M      0  1574k  0:01:16  0:01:14  0:00:02  498k
 97  117M    0     0   97  114M      0  1541k  0:01:18  0:01:16  0:00:02  462k
 97  117M    0     0   97  114M      0  1536k  0:01:18  0:01:16  0:00:02  419k
 98  117M    0     0   98  115M      0  1530k  0:01:18  0:01:17  0:00:01  581k
 98  117M    0     0   98  116M      0  1520k  0:01:19  0:01:18  0:00:01  564k
 99  117M    0     0   99  117M      0  1513k  0:01:19  0:01:19 --:--:--  614k
* We are completely uploaded and fine
100  117M    0     0  100  117M      0  1495k  0:01:20  0:01:20 --:--:--  695k
100  117M    0     0  100  117M      0  1476k  0:01:21  0:01:21 --:--:--  589k
100  117M    0     0  100  117M      0  1458k  0:01:22  0:01:22 --:--:--  388k
100  117M    0     0  100  117M      0  1441k  0:01:23  0:01:23 --:--:--  252k
100  117M    0     0  100  117M      0  1424k  0:01:24  0:01:24 --:--:-- 57489
100  117M    0     0  100  117M      0  1407k  0:01:25  0:01:25 --:--:--     0
100  117M    0     0  100  117M      0  1391k  0:01:26  0:01:26 --:--:--     0
100  117M    0     0  100  117M      0  1375k  0:01:27  0:01:27 --:--:--     0
100  117M    0     0  100  117M      0  1359k  0:01:28  0:01:28 --:--:--     0
100  117M    0     0  100  117M      0  1344k  0:01:29  0:01:29 --:--:--     0
100  117M    0     0  100  117M      0  1329k  0:01:30  0:01:30 --:--:--     0
100  117M    0     0  100  117M      0  1314k  0:01:31  0:01:31 --:--:--     0
100  117M    0     0  100  117M      0  1300k  0:01:32  0:01:32 --:--:--     0
100  117M    0     0  100  117M      0  1286k  0:01:33  0:01:33 --:--:--     0
100  117M    0     0  100  117M      0  1273k  0:01:34  0:01:34 --:--:--     0
100  117M    0     0  100  117M      0  1259k  0:01:35  0:01:35 --:--:--     0
100  117M    0     0  100  117M      0  1246k  0:01:36  0:01:36 --:--:--     0
100  117M    0     0  100  117M      0  1233k  0:01:37  0:01:37 --:--:--     0
100  117M    0     0  100  117M      0  1221k  0:01:38  0:01:38 --:--:--     0
100  117M    0     0  100  117M      0  1209k  0:01:39  0:01:39 --:--:--     0
100  117M    0     0  100  117M      0  1196k  0:01:40  0:01:40 --:--:--     0
100  117M    0     0  100  117M      0  1185k  0:01:41  0:01:41 --:--:--     0
100  117M    0     0  100  117M      0  1173k  0:01:42  0:01:42 --:--:--     0
100  117M    0     0  100  117M      0  1162k  0:01:43  0:01:43 --:--:--     0
100  117M    0     0  100  117M      0  1151k  0:01:44  0:01:44 --:--:--     0
100  117M    0     0  100  117M      0  1140k  0:01:45  0:01:45 --:--:--     0
100  117M    0     0  100  117M      0  1129k  0:01:46  0:01:46 --:--:--     0
100  117M    0     0  100  117M      0  1118k  0:01:47  0:01:47 --:--:--     0
100  117M    0     0  100  117M      0  1108k  0:01:48  0:01:48 --:--:--     0
100  117M    0     0  100  117M      0  1098k  0:01:49  0:01:49 --:--:--     0
100  117M    0     0  100  117M      0  1088k  0:01:50  0:01:50 --:--:--     0
100  117M    0     0  100  117M      0  1078k  0:01:51  0:01:51 --:--:--     0
100  117M    0     0  100  117M      0  1069k  0:01:52  0:01:52 --:--:--     0
100  117M    0     0  100  117M      0  1059k  0:01:53  0:01:53 --:--:--     0
100  117M    0     0  100  117M      0  1050k  0:01:54  0:01:54 --:--:--     0
100  117M    0     0  100  117M      0  1041k  0:01:55  0:01:55 --:--:--     0
100  117M    0     0  100  117M      0  1032k  0:01:56  0:01:56 --:--:--     0
100  117M    0     0  100  117M      0  1023k  0:01:57  0:01:57 --:--:--     0
100  117M    0     0  100  117M      0  1014k  0:01:58  0:01:58 --:--:--     0
100  117M    0     0  100  117M      0  1006k  0:01:59  0:01:59 --:--:--     0
100  117M    0     0  100  117M      0   997k  0:02:00  0:02:00 --:--:--     0
100  117M    0     0  100  117M      0   989k  0:02:01  0:02:01 --:--:--     0
100  117M    0     0  100  117M      0   981k  0:02:02  0:02:02 --:--:--     0
100  117M    0     0  100  117M      0   973k  0:02:03  0:02:03 --:--:--     0
100  117M    0     0  100  117M      0   965k  0:02:04  0:02:04 --:--:--     0
100  117M    0     0  100  117M      0   958k  0:02:05  0:02:05 --:--:--     0
100  117M    0     0  100  117M      0   950k  0:02:06  0:02:06 --:--:--     0
100  117M    0     0  100  117M      0   943k  0:02:07  0:02:07 --:--:--     0
100  117M    0     0  100  117M      0   935k  0:02:08  0:02:08 --:--:--     0
100  117M    0     0  100  117M      0   928k  0:02:09  0:02:09 --:--:--     0
100  117M    0     0  100  117M      0   921k  0:02:10  0:02:10 --:--:--     0
100  117M    0     0  100  117M      0   914k  0:02:11  0:02:11 --:--:--     0
100  117M    0     0  100  117M      0   907k  0:02:12  0:02:12 --:--:--     0
100  117M    0     0  100  117M      0   900k  0:02:13  0:02:13 --:--:--     0
100  117M    0     0  100  117M      0   893k  0:02:14  0:02:14 --:--:--     0
100  117M    0     0  100  117M      0   887k  0:02:15  0:02:15 --:--:--     0
100  117M    0     0  100  117M      0   880k  0:02:16  0:02:16 --:--:--     0
100  117M    0     0  100  117M      0   874k  0:02:17  0:02:17 --:--:--     0
100  117M    0     0  100  117M      0   868k  0:02:18  0:02:18 --:--:--     0
100  117M    0     0  100  117M      0   861k  0:02:19  0:02:19 --:--:--     0
100  117M    0     0  100  117M      0   855k  0:02:20  0:02:20 --:--:--     0
< HTTP/1.1 504 Gateway Time-out
< Content-Type: text/html
< Content-Length: 1033
< Connection: keep-alive
< Server: CloudFront
< Date: Thu, 18 Jul 2024 01:42:43 GMT
< X-Cache: Error from cloudfront
< Via: 1.1 CENSORED.cloudfront.net (CloudFront)
< X-Amz-Cf-Pop: CENSORED
< X-Amz-Cf-Id: CENSORED
< 
{ [1033 bytes data]

100  117M  100  1033  100  117M      7   855k  0:02:27  0:02:20  0:00:07   253
* Connection #0 to host up.flickr.com left intact

And the response was:

What the fuck happened? You were doing so well, Flickr. Here I am going out of my way to diagnose and report – with all pertinent details available to me – a bug in your API servers, and I get a robotic, bullshit response that’s purely about refusing to accept responsibility. This is the kind of response which makes me think your support staff are paid based solely on how fast they close tickets.

And what the hell is Marc ‘appreciating’ here? Amanda’s “Warmly” valediction actually feels genuine, in context (and is human even out of context). “Appreciatively” feels like corporate innuendo.

We humans are weird

Admittedly I started writing this out of catharsis, but it really got me thinking.

Why should I care that Flickr delivered miserable customer support in this second case? Shouldn’t I just quietly move on, like I would if it were from most other companies? Shouldn’t I be thrilled it’s at least not a bug in Lightroom itself, since Adobe’s customer support is a hundred times worse in every case; among the most aggressively evasive and pre-emptively hostile I’ve ever encountered?

I think we’re all pretty conditioned to expect terrible experiences with so-called customer support from large software companies, like Adobe, or Google. We expect it and tolerate it, against all justice and our own interests.

Instead, it’s often inconsistency in a single company’s behaviour that’s the most infuriating and raises our ire. This makes no logical nor rational sense – and is very bad from a game theory perspective, as it encourages companies to be consistently dumb and evil.

It seems akin to how most folks pay little attention to e.g. Facebook doing yet another horrible thing – the headlines might as well all be “Facebook acts like Facebook yet again” – but if e.g. Apple does something a bit clueless, the world gets up in arms.

I guess it boils down to hope, ironically. If we see a company – or a person – demonstrate that they can do better, then we raise the bar for them. And are then ripe to be disappointed if they merely behave like most of their peers, subsequently.

Which is stupid, really.

What NSCopyObject does

Its implementation currently starts with essentially:

id NSCopyObject(id object, NSUInteger extraBytes, NSZone *zone) {
    if (nil == object) {
        return nil;
    }
    
    id copy = object_copy(object, extraBytes);
    object_setClass(copy, objc_opt_class(object));
    return copy;
}

…where object_copy et al are part of the Objective-C runtime. object_copy and its callees are not trivial, so I won’t repeat them here. The key parts are:

1. The malloc in _class_createInstance (allocate space for the copy).
2. The memmove in object_copy (naively copy the raw bytes over).
3. The call from object_copy to fixupCopiedIvars (half-heartedly attempt to fix the damage).

fixupCopiedIvars is notable. It was added by necessity when ARC was introduced to the Objective-C runtime, in Mac OS X 10.6 (Snow Leopard) in 2006. ARC added metadata to Objective-C classes to convey which instance variables were retain-counted object references, so that it could manage them automagically at runtime (not just for copying objects, but more importantly for deallocating them). fixupCopiedIvars uses that metadata to identify things it has to retain (strongly or weakly) in the new copy.

So that should work great, right? The copy operation increments the retain count of all shared objects the new copy references, like you’d expect?

That metadata is incomplete. It only works for Objective-C ivars managed by ARC. i.e. not C++ ivars or Swift stored properties, nor even Objective-C ivars that aren’t using ARC¹.

But I don’t use NSCopyObject…?

Almost nobody intentionally uses NSCopyObject, but your superclass might, and therefore you might. Ever subclassed NSCell or NSAnimation, for example?

I happened to hit this when subclassing NSBitmapImageRep (and I’m very grateful to Kyle Sluder for so quickly identifying the problem – it could have taken me forever to figure it out, otherwise).

If your superclass uses NSCopyObject, it’s now your problem just as much as if you’d used NSCopyObject directly, whether you like it or not.

And even more problematically, whether you know it or not. If your superclass is defined by a 3rd party framework / library, or anything that’s closed source, you might have no idea whether it uses NSCopyObject currently. Worse, you have no control over whether it will or will not use it in future (though anyone that adds a use of NSCopyObject at this point had better hope the atheists are right).

So how do I defend against NSCopyObject?

Objective-C

Pre-ARC it used to be relatively easy to work around this, in Objective-C. You “just” had to manually retain all your subclasses’ reference ivars – and manually copy some others, like non-ref-counted mutable or mortal buffers, etc.

But that generally isn’t possible with ARC – under which you cannot explicitly call retain. Worse:

• There’s still prominent guides scattered about the web that push you unequivocally to use retain, which is not just impossible to do directly under ARC, but flat-out wrong even if you do figure out one of the “clever” ways to do it (you’ll end up over-retaining your ARC-managed references, causing memory leaks).
• There’s also pages lingering on the web that claim that merely turning on ARC will magically solve the problem (it might, but it’s not a panacea).

Some guides specify a better method, which is to manually zero out the copied object’s ivars and then repopulate them via formal property setters. That actually works with or without ARC, although it may break – causing memory leaks – if the superclass ever stops using NSCopyObject (or if NSCopyObject ever gets upgraded to understand reference-counted ivars that it currently does not). It’s also only possible in Objective-C because Swift doesn’t provide direct access to instance variables.

Keep in mind that any reference-typed ivars which are not strong or weak Objective-C objects managed by ARC will still, always need to be handled manually. e.g. pointers to manually-managed memory buffers.

Swift

Ironically, Swift’s attempts to prevent incorrect code actually make it harder to write correct code in this case. What you want to do is – like the Objective-C implementation – to just zero out the references and re-assign them like normal properties. Zeroing them out without triggering a release basically undoes the mistaken memcpy that NSCopyObject did. But Swift won’t let you.

Worse, this has been known for most of Swift’s existence and nothing has been done about it.

Simply setting the property to nil will cause it to be erroneously released, which may immediately deallocate the object and ultimately cause a crash or memory corruption. Even if it doesn’t happen to deallocate the object, it’ll negate the retain you do during the assignment, making all your effort moot.

That’s pretty heavy-handed, though, and not always possible (e.g. NSImageRep, as used by e.g. NSBitmapImageRep, does some special magic in its copy implementation which you cannot practically replicate).

It appears that the best you can do is assume the superclass will always use NSCopyObject, if it does currently, and just manually increment the retain count. Like Objective-C with ARC, the language & standard library really don’t want you to actually do this, but at least in Swift it’s relatively straightforward:

override func copy(with zone: NSZone? = nil) -> Any {
    let result = super.copy(with: zone)
    
    if result.myProperty === self.myProperty {
        _ = Unmanaged.passRetained(myProperty)
    } else {
        result.myProperty = self.myProperty
    }
}

The conditional might help protect you if the superclass stops using NSCopyObject in future – in that case, it’ll probably cause myProperty to default to nil (or to be assigned to some other instance, which you can discard), in which case you just want to assign to it normally.

In the interim – while NSCopyObject is in use, at least – the myProperty pointer will be copied verbatim and you have to assume it requires the extra, manual retain. It’s not future-proof – it’s possible for the superclass to copy the pointer verbatim and increment the retain count for you – but at least in that case you “merely” get a memory leak, rather than a crash or memory corruption.

Do as Apple says, not as Apple does

The most frustrating part of all of this is that this is entirely Apple’s fault. Sure, you can argue it’s not their fault that NeXT added this vile function to Foundation; that Apple “merely” inherited it and were “forced” to keep for backwards compatibility. But it’s entirely Apple’s choice to have kept using it all this time, in their core frameworks, even while they’ve been telling everyone else to never use it.

NSCopyObject has been a known problem-maker pretty much forever – it was a terrible idea right from the outset. Blindly copying the bytes of an object instance, and just hoping that somehow that works correctly – in an object-oriented language derived from Smalltalk where even numbers are often reference types – is farcical.

The introduction of ARC (in 2008) didn’t really help anything, as although it changed NSCopyObject to properly retain ARC-managed ivars, it did nothing for non-ARC-managed ivars (remember that ARC can be enabled in one library but not in another, and libraries can subclass each others’ classes).

NSCopyObject has been officially deprecated since 2012:

The NSCopyObject() function has been deprecated. It has always been a dangerous function to use, except in the implementation of copy methods², and only then with care.

Foundation Release Notes for OS X 10.8 Mountain Lion and iOS 6

…though Apple officially told everyone not to use it in 2008:

This function is dangerous and very difficult to use correctly. It’s [sic] use as part of -copyWithZone: by any class that can be subclassed, is highly error prone. This function is known to fail for objects with embedded retain count ivars, singletons, and C++ ivars, and other circumstances.

Foundation Release Notes for Mac OS X 10.6 Snow Leopard

And this was all still a decade or more after it was known that NSCopyObject was fundamentally evil, e.g. NSCell, and GnuStep’s broken NSControl.

And yet, Apple still use NSCopyObject themselves to this very day, in their own applications and frameworks – including major frameworks like AppKit that almost all 3rd party developers rely on. NSCell is still broken, three decades later, as is NSImage & NSImageRep, and NSAnimation. Most of those are explicitly designed to be subclassed, despite Apple’s own very clear instructions to never mix subclassing with NSCopyObject.

Admittedly it’s not trivial for Apple to remove the NSCopyObject use – alas, because people have had to code myriad hacky workarounds to it, Apple now has to be careful not to break those workarounds. That might even preclude fixing the existing code paths; it might require a replacement copy mechanism. Which leads to…

Tangent: NSCopying considered harmful

The big driver of NSCopyObject use has long been NSCopying. Classes that intend to be subclassed – but also semantically should support copying i.e. NSCopying – have long been making the mistake of thinking that means using NSCopyObject. One need only read the NSCopying documentation, even from before Mac OS X was even publicly released, to see how dangerously fragile and error-prone NSCopying has always been.

Compounding the problem is that NSCopying doesn’t work, by default, on subclasses. You have to override copy(with:) in every subclass³, but the compiler does not enforce this, because in Objective-C (and alas Swift) protocol conformance is assumed inherited even when it cannot correctly be without explicit, extra work by the subclass.

1. Yes, it’s still possible to this day to write Objective-C without using ARC – -fno-objc-arc / CLANG_ENABLE_OBJC_ARC. There might even be valid (albeit unfortunate) use-cases for having to do so, such as for performance.
   
   And even with ARC, it’s of course possible to have pointers to things which aren’t NSObjects and therefore aren’t handled by ARC, such as raw malloc allocations. ↩︎
2. This is false and always has been (that it was safe to use in copy methods). Apple’s false statements in the deprecation notices may ironically have caused even more instances of people using NSCopyObject. ↩︎
3. Any and all that add retain-counted ivars, Swift stored properties, or ivars of C++ types that have destructors. ↩︎

In absolute terms they don’t have much overhead – think sub-microsecond for most uses. Which makes them perfectly acceptable when they’re used sporadically (e.g. only a few times per second).

However, if you need to deal with time and timing more frequently, their inefficiency can become a serious bottleneck.

I stumbled into this because of a fairly common and otherwise uninteresting pattern – throttling UI updates on an I/O operation’s progress. This might look something like:

struct Example: View {
    let bytes: AsyncSequence<UInt8>

    @State var byteCount = 0

    var body: some View {
        Text("Bytes so far: \(byteCount.formatted(.byteCount(style: .binary)))")
            .task {
                var unpostedByteCount = 0
                let clock = ContinuousClock()
                var lastUpdate = clock.now

                for try await byte in bytes {
                    … // Do something with the byte.

                    unpostedByteCount += 1

                    let now = clock.now
                    let delta = now - lastUpdate

                    if (    delta > .seconds(1)
                         || (    (delta > .milliseconds(100)
                              && 1_000_000 <= unpostedByteCount))) {
                        byteCount += unpostedByteCount
                        unpostedByteCount = 0
                        lastUpdate = now
                    }
                }
            }
    }
}

The above seems fairly straightforward, but if you run it and have any non-trivial I/O rate – even just a few hundred kilobytes per second – you’ll find that it saturates an entire CPU core, not just wasting CPU time but limiting the I/O rate severely.

Using a SuspendingClock makes no difference.

In a nutshell, the problem is that Swift’s Clock protocol has significant overheads by design¹. If you look at a time profile of code like this, you’ll see things like:

That’s a lot of time wasted in function calls and struct initialisation and type conversion and protocol witnesses and all that guff. The only part that’s actually retrieving the time is the swift_get_time call (which is just a wrapper over clock_gettime, which is just a wrapper over clock_gettime_nsec_np(CLOCK_UPTIME_RAW), which is just a wrapper over mach_absolute_time).

I wrote some simple benchmarks of various alternative time-tracking methods, with these results with Swift 5.10 (showing the median runtime of the benchmark, which is a million iterations of checking the time):

All these alternative methods are well over an order of magnitude faster than Swift’s native clock APIs, showing just how dreadfully inefficient the Swift Clock API is.

mach_absolute_time for the win

Unsurprisingly, mach_absolute_time is the fastest. It is what all these other APIs are actually based on; it is the lowest level of the time stack.

The downside to calling mach_absolute_time directly, though, is that it’s on Apple’s “naughty” list – apparently it’s been abused for device fingerprinting, so Apple require you to beg for special permission if you want to use it (even though it’s used by all these other APIs anyway, as the basis for their implementations, and there’s nothing you can get from mach_absolute_time that you can’t get from them too 🤨).

Date surprisingly not bad

I was quite surprised to see good ol’ Date performing competitively with the traditional C-level APIs, at least on x86-64. Even on arm64 it’s not bad, at still a third to half the speed of the C APIs. This surprised me because it has the overhead of at least one Objective-C message send (for timeIntervalSinceNow), unless somehow the Swift compiler is optimising that into a static function call, or inlining it entirely…?

I certainly wouldn’t be afraid to use Date broadly, going down to lower APIs only when truly necessary – which is pretty rarely, I’d wager; we’re talking a mere 19 to 30 nanoseconds to get the time elapsed since a reference date and compare it to a threshold. If that’s too slow, it might be an indication that there’s a bigger problem (like transferring data a single byte at a time, as in the example that started this post – but more on that in the next post).

Follow-up

This post got some attention on HackerNews. Pleasingly, the comments there were almost all well-intentioned and interesting. It’s a bit beyond me to try to address all of them, but a few in particular raised good points that I would like to answer / clarify:

• A lot of folks were curious about mach_absolute_time being on Apple’s naughty list. I don’t know for sure why it is either, but I think it’s very likely that it’s primarily because it essentially provides a reference time point, that’s very precise and pretty unique between computers. It’s not the boot time necessarily – because the timer pauses whenever the system is put to sleep – but even so it provides a simple way to nearly if not exactly identify an individual machine session (between boots & sleeps). It probably wouldn’t take many other fingerprinting data points to reliably pin-point a specific machine.
  
  Secondarily, because it provides very precise timing capabilities (e.g. nanosecond-resolution on x86), it could possibly be a key component of timing attacks and broader device fingerprinting based on timing information (e.g. measuring how long it takes to perform an otherwise innocuous operation).
  
  That all said, the only difference between it and some of the higher-level APIs wrapping it is their overhead. And it’s not apparent to me that merely making the “get-time” functionality 2x slower is going to magically mitigate all the above concerns, especially when we’re still talking just a few nanoseconds.
• Admittedly my phrasing regarding Apple’s policies on mach_absolute_time – “beg for permission to use it” – is a little melodramatic. It’s revealing something of my personal opinions on certain Apple “security” practices. I love that Apple genuinely care about protecting everyone’s privacy, but sometimes I chaff at what feels like capricious or impractical specific policies.
  
  In this particular case, it’s not apparent to me why this sort of protection is needed for native apps. In a web browser, sure, you’re running untrustworthy, essentially arbitrary code from all over the place, a lot of which is openly malicious (thanks, Google & Facebook, for your pervasive trackers – fuck you too). But a native app – or heck, even a dodgy non-native one like an Electron app – must be explicitly installed by the end user, among other barriers like code signing.
• A few folks looked at the example case, of iterating a single byte at a time, and were suspicious of how performant that could possibly be anyway. This is a very fair reaction – it’s my ingrained instinct as well, from years of C/C++/Objective-C – but it’s relying on a few outdated assumptions. My next post already covered this for the most part, but in short here:
  
  Through inlining, that code basically optimises down to an outer loop that fetches a new chunk of data (a pointer & length) plus an inner loop to iterate over that as direct memory access. The chunks are typically tens of kilobytes to megabytes, in my experience (depending on the source, e.g. network vs local storage, and the buffer sizes chosen by Apple’s framework code). So it actually is quite performant and essentially what you’d conventionally write in a file descriptor read loop. If and when it happens to optimise correctly. That’s the major caveat – sometimes the Swift compiler fails to properly optimise code like this, and then indeed the performance can really suck. But for simple cases like in this post’s example code, the optimiser has no trouble with it.
• Similarly, a few folks questioned the need to check the clock on every byte, as in the example. That’s a valid critique of this sort of code in many contexts, and I concur that where possible one should try to be smarter about such things – i.e. use sequences of bunches of bytes, not sequences of individual bytes. e.g. with URLSession you can, and indeed it is faster to do it smarter like that. But, you can get acceptable real-world performance with this code, even in high-throughput cases, and it’s relatively simple and intuitive to write, so it’s not uncommon or necessarily unreasonable.
  
  In addition, sometimes you’re at the mercy of the APIs available – e.g. sometimes you can only get an AsyncSequence<UInt8>. If you don’t care about complete accuracy, you can do things like only considering UI updates every N bytes. You’ll save CPU time and nobody will notice the difference for small enough N on a fast enough iteration, but if those prerequisites aren’t met you might read e.g. N-1 bytes and then hit a long pause, during which time you have the extra N-1 bytes in hand but you’re not showing as such in your UI.
• Some folks noted that are a lot of other clock APIs from Apple’s frameworks, like DispatchTime and CACurrentMediaTime. I didn’t include those in the benchmark because I just didn’t think of them at the time. If anyone wants to send me a pull request adding them to the code, I’d be very happy to accept it.
  
  I haven’t checked all those other APIs specifically, but I can pretty much guarantee they’re all built on mach_absolute_time too (possibly via one or more of the other C APIs already covered in this post). In fact those two examples just mentioned are explicitly documented as using mach_absolute_time.
• Kallikrates quietly pointed to a very interesting recent change in Apple’s Swift standard library code, Make static [milli/micro/nano]seconds members on Duration inlinable. It’s paired with another patch that together seem very specifically aimed at eliminating some of the absurd overhead in Swift’s ContinuousClock & SuspendingClock implementations. The timing is a bit interesting – I don’t know if they were prompted by this post, but it’d be an unlikely coincidence otherwise.
  
  In any case, I suspect it is possible to eliminate the overheads – there’s no apparent reason why they can’t be at least as efficient as Date already is – and so I hope that is what’s happening. Hopefully I’ll be able to re-run these benchmarks in a few months, with Swift 6, and see the performance gap eliminated. 🤞

1. One might quibble with the “by design” assertion. What I mean is that because it uses a protocol it’s susceptible to significant overheads – as is seen in these benchmarks – and because its internal implementation (a private _Int128 type, inside the standard library) is kept hidden, it limits the compiler’s ability to inline, which is in turn critical to eliminating what’s technically a lot of boilerplate. In contrast, if it were simply a struct using only public types internally, it would have avoided most of these overheads and been more amenable to inlining.
   
   It’s not an irredeemable design (I think) – and that’s what the recent patches seem to be banking on, by tweaking the design in order to allow inlining and thus hopefully eliminate almost all the overhead. ↩︎

Long story short, on at least some bitmap images, as soon as you call getBitmapDataPlanes it somehow permanently breaks that NSImage and NSBitmapImageRep. The telltale sign of this – aside from the image rendering incorrectly – is the stderr message from AppKit:

Failed to extract pixel data from NSBitmapImageRep. Error: -21778

This definitely occurs with 10-bit AVIFs, but it’s not limited to that bit depth nor that file format, because I found the one other mention of this error message online, where the source image was 1-bit.

I’ve never seen this happen with 8-bit, 12-bit, or 16-bit images, AVIF or otherwise.

This makes me suspect it’s tied to the internal pixel format – 10-bit AVIFs always load as 10 / 40 (bits per sample / bits per pixel), irrespective of whether they have an alpha channel. 8-bit AVIFs are always 8 / 32, 12-bit are always 12 / 48. Maybe NSBitmapImageRep has a bug when the bits per pixel isn’t a multiple of 16?

If you call bitmapData instead you get the same error message but it does not stop the NSImage / NSBitmapImageRep from rendering correctly. But the pointer returned from bitmapData points to opaque black. It’s particularly weird that it returns a pointer to a valid memory allocation, of at least the expected size, yet the contents are nothing but zeroes. Seems like it pre-allocates some output buffer, as zeroed memory, and then fails to actually write to that buffer. Yet it returns it anyway, instead of returning nil. 😕

FB13693411.

Partial workaround

If you call recache on the NSImage afterwards, the image is capable of rendering correctly again. That doesn’t help if your objective is to access the bitmap bytes, but at least if you don’t – e.g. you’re encountering this only because some library code is triggering the bug – you might be able to work around it through recache.

It’s easy to see how some UI designer thought this was a good idea. Surely if you move the mouse near the edge of the window (or the screen, in fullscreen mode) and rest it there, it’s because you’re looking forlornly for your lost sidebar? What could be more helpful and delightful than your missing sidebar popping into view?!

Unfortunately, they have ignored that fact that there is usually already other GUI controls at the edge of the window, not the least of which being the window edge itself (for drag-resizing of the window). Scrollbars are another common inhabitant of window edges.

“Proactive Peek” is the worst of these two because not only does it change what’s under the mouse cursor just as you’re likely to click, stealing the click away from its true target, but it actually shrinks the window’s visible contents. This leads to layout changes and motion noise, particularly in web pages where it can have knock-on effects like mucking with the scroll position or causing major changes by crossing some “responsive design” threshold.

Unfortunately there’s no way to turn this poorly-considered ‘feature’ off completely, although you can effectively disable the “Reveal on Edge Hover” piece:

If you’re an app developer it looks like (I haven’t tested it) you can disable these ‘features’ within your own app, at least, by implementing the private NSSplitView delegate method _splitView:canProactivePeekArrangedView: and setting the NSSplitView revealsOnEdgeHoverInFullscreen property to NO (or the NSSplitViewController private property  _hasItemToRevealOnEdgeHover to NO, if you’re using NSSplitViewController). Or subclassing NSSplitView and overriding _canDoSidebarProactivePeek and _canDoInspectorProactivePeek to return NO – though that only applies to Proactive Peek.

FB13583826.

Is that it?

I know that’s not a very helpful in some sense, but I wasted days trying to figure out how to implement this very basic drag and drop functionality, in SwiftUI. From what I found online, I’m not the only person who’s struggled with this. So hopefully this post saves others from long and frustrating searches.

A big part of the difficulty in answering this simple question – does SwiftUI support this or not – is that a lot of documentation (first & third party) around SwiftUI’s APIs uses the word “promise” but not in the same way. They merely mean that the pasteboard data is populated on first access, not that it actually follows the file promise protocol.

Best alternative (other than ditching SwiftUI)

The closest you can get is to write data into some arbitrary location that you have to choose blindly (not having any idea where the actual destination is), and then rely on the receiving app to move or copy the file from there. It doesn’t matter, in this regard, whether you use the Transferable-based APIs or the NSItemProvider-based APIs.

view.onDrag {
    let provider = NSItemProvider()
    
    provider.registerDataRepresentation(for: UTType.fileURL) {
        do {
            let tmpDir = try FileManager.default.url(for: .itemReplacementDirectory,
                                                     in: .userDomainMask,
                                                     appropriateFor: URL.temporaryDirectory,
                                                     create: true)

            // You'll need to provide the `suggestedFileName` based on the particulars of your use-case.
            let file = tmpDir.appending(component: suggestedFileName)

            try bytes.write(to: file, options: .withoutOverwriting)

            $0(file.dataRepresentation, nil)
        } catch {
            $0(nil, error)
        }

        return nil
    }
    
    return provider
}

This makes it impossible to avoid unnecessary file copies, and requires you to keep the file around permanently, since you have no idea when the receiver is done with it.

e.g. if the Finder is the drop destination, you have no idea what volume the promised file was dropped on, so you don’t know which volume to create it on. If you guess wrong, the Finder is forced to copy the file. Not only does this confuse the user (by showing a copy badge on the mouse pointer) but it wastes time (in performing the copy) and the original file is left in place, requiring manual clean-up (which is impossible to do correctly because you have no idea when the receiving application is done with the temporary file – and the receiver might hold a reference to it permanently).

Understanding the SwiftUI drag & drop APIs

Tangentially, I found most documentation on SwiftUI’s drag & drop APIs – especially Apple’s – to be very poorly written, which is particularly frustrating in this case because the APIs are not intuitive in the slightest.

And most code / usage examples focus exclusively on trivial, disinteresting cases (e.g. dragging images around within a single window).

Only after I’d spend days reverse-engineering the APIs to figure out how they actually work and how they’re supposed to be used, did I stumble upon Dave Rahardja‘s All about Item Providers. It’s by far the best documentation I’ve found on drag & drop in SwiftUI. Note though that Dave also uses the word “promise” a lot even though he’s never actually talking about actual file promises.

Also, I found that the newer and ostensibly better Transferable-based API was harder to understand and harder to use than the NSItemProvider-based APIs. I recommend going straight to, and sticking with, the latter. Your code will be simpler and more reliable.

It is an example of bad API design. Specifically, regarding cohesion: the principle that an API should have one purpose. A litmus test for this is whether all the method parameters are always applicable¹.

It wasn’t until I wrote a test driver which explores its entire parameter space, that I was finally able to grok what the hell it’s doing and delineate its multiple modes of operation.

I’ve contrasted it with the results from its sibling urls(for:in:), to better understand what it’s doing (and expose some more of its flaws).

let fm = FileManager.default

let searchPathDirectories = [("applicationDirectory", FileManager.SearchPathDirectory.applicationDirectory),
                             ("demoApplicationDirectory", FileManager.SearchPathDirectory.demoApplicationDirectory),
                             ("developerApplicationDirectory", FileManager.SearchPathDirectory.developerApplicationDirectory),
                             ("adminApplicationDirectory", FileManager.SearchPathDirectory.adminApplicationDirectory),
                             ("libraryDirectory", FileManager.SearchPathDirectory.libraryDirectory),
                             ("developerDirectory", FileManager.SearchPathDirectory.developerDirectory),
                             ("userDirectory", FileManager.SearchPathDirectory.userDirectory),
                             ("documentationDirectory", FileManager.SearchPathDirectory.documentationDirectory),
                             ("documentDirectory", FileManager.SearchPathDirectory.documentDirectory),
                             ("coreServiceDirectory", FileManager.SearchPathDirectory.coreServiceDirectory),
                             ("autosavedInformationDirectory", FileManager.SearchPathDirectory.autosavedInformationDirectory),
                             ("desktopDirectory", FileManager.SearchPathDirectory.desktopDirectory),
                             ("cachesDirectory", FileManager.SearchPathDirectory.cachesDirectory),
                             ("applicationSupportDirectory", FileManager.SearchPathDirectory.applicationSupportDirectory),
                             ("downloadsDirectory", FileManager.SearchPathDirectory.downloadsDirectory),
                             ("inputMethodsDirectory", FileManager.SearchPathDirectory.inputMethodsDirectory),
                             ("moviesDirectory", FileManager.SearchPathDirectory.moviesDirectory),
                             ("musicDirectory", FileManager.SearchPathDirectory.musicDirectory),
                             ("picturesDirectory", FileManager.SearchPathDirectory.picturesDirectory),
                             ("printerDescriptionDirectory", FileManager.SearchPathDirectory.printerDescriptionDirectory),
                             ("sharedPublicDirectory", FileManager.SearchPathDirectory.sharedPublicDirectory),
                             ("preferencePanesDirectory", FileManager.SearchPathDirectory.preferencePanesDirectory),
                             ("applicationScriptsDirectory", FileManager.SearchPathDirectory.applicationScriptsDirectory),
                             ("itemReplacementDirectory", FileManager.SearchPathDirectory.itemReplacementDirectory),
                             ("allApplicationsDirectory", FileManager.SearchPathDirectory.allApplicationsDirectory),
                             ("allLibrariesDirectory", FileManager.SearchPathDirectory.allLibrariesDirectory),
                             ("trashDirectory", FileManager.SearchPathDirectory.trashDirectory)]

let searchPathDomainMasks = [("userDomainMask", FileManager.SearchPathDomainMask.userDomainMask),
                             ("localDomainMask", FileManager.SearchPathDomainMask.localDomainMask),
                             ("systemDomainMask", FileManager.SearchPathDomainMask.systemDomainMask),
                             ("networkDomainMask", FileManager.SearchPathDomainMask.networkDomainMask),
                             /*("allDomainsMask", FileManager.SearchPathDomainMask.allDomainsMask)*/]

print("urls(for:in:):")

for (dirName, dir) in searchPathDirectories {
    print("\n\(dirName):")

    for (domainName, domain) in searchPathDomainMasks {
        let dirs = fm.urls(for: dir, in: domain)
            .map { $0.path(percentEncoded: false) }
            .joined(separator: "\n" + String(repeating: " ", count: 23))

        print("    \(domainName): \(String(repeating: " ", count: 17 - domainName.count))\(dirs)")
    }
}

print("\n\nurl(for:in:appropriateFor:create:):")

let paths = [nil,
             URL(filePath: "/"),
             FileManager.default.temporaryDirectory,
             URL(filePath: "/Volumes/Flash/")]
let pathDesc: (URL?) -> String = { $0?.path(percentEncoded: false) ?? "nil" }
let maxPathLength = paths.map { pathDesc($0).count }.max() ?? 0

for (dirName, dir) in searchPathDirectories {
    print("\n\(dirName):")

    for (domainName, domain) in searchPathDomainMasks {
        var results = [String: String]()

        for appropriateForPath in paths {
            let path: String

            do {
                let folderURL = try FileManager.default.url(for: dir,
                                                            in: domain,
                                                            appropriateFor: appropriateForPath,
                                                            create: false)
                path = pathDesc(folderURL)
            } catch {
                path = "ERROR (\(error))"
            }

            results[pathDesc(appropriateForPath)] = path
        }

        let uniquePaths = Set(results.values)

        if 1 == uniquePaths.count {
            print("    \(domainName): \(String(repeating: " ", count: 17 - domainName.count))\(uniquePaths.first!)")
        } else {
            print("    \(domainName):")

            for (appropriateForPath, path) in results.sorted(by: { $0.key < $1.key }) {
                print("        \(appropriateForPath): \(String(repeating: " ", count: maxPathLength - appropriateForPath.count))\(path)")
            }
        }
    }
}

And here’s the output:

urls(for:in:):

applicationDirectory:
    userDomainMask:    /Users/SadPanda/Library/Containers/com.SadPanda.MyApp/Data/Applications/
    localDomainMask:   /Applications/
    systemDomainMask:  /System/Applications/
                       /System/Cryptexes/App/System/Applications/
    networkDomainMask: /Network/Applications/

demoApplicationDirectory:
    userDomainMask:    /Users/SadPanda/Library/Containers/com.SadPanda.MyApp/Data/Applications/Demos/
    localDomainMask:   /Applications/Demos/
    systemDomainMask:  /Applications/Demos/
    networkDomainMask: /Network/Applications/Demos/

developerApplicationDirectory:
    userDomainMask:    /Users/SadPanda/Library/Containers/com.SadPanda.MyApp/Data/Developer/Applications/
    localDomainMask:   /Developer/Applications/
    systemDomainMask:  /Developer/Applications/
    networkDomainMask: /Network/Developer/Applications/

adminApplicationDirectory:
    userDomainMask:    /Users/SadPanda/Library/Containers/com.SadPanda.MyApp/Data/Applications/Utilities/
    localDomainMask:   /Applications/Utilities/
    systemDomainMask:  /System/Applications/Utilities/
                       /System/Cryptexes/App/System/Applications/Utilities/
    networkDomainMask: /Network/Applications/Utilities/

libraryDirectory:
    userDomainMask:    /Users/SadPanda/Library/Containers/com.SadPanda.MyApp/Data/Library/
    localDomainMask:   /Library/
    systemDomainMask:  /System/Library/
                       /System/Cryptexes/App/System/Library/
    networkDomainMask: /Network/Library/

developerDirectory:
    userDomainMask:    /Users/SadPanda/Library/Containers/com.SadPanda.MyApp/Data/Developer/
    localDomainMask:   /Developer/
    systemDomainMask:  /Developer/
    networkDomainMask: /Network/Developer/

userDirectory:
    userDomainMask:    
    localDomainMask:   /Users/
    systemDomainMask:  
    networkDomainMask: /Network/Users/

documentationDirectory:
    userDomainMask:    /Users/SadPanda/Library/Containers/com.SadPanda.MyApp/Data/Library/Documentation/
    localDomainMask:   /Library/Documentation/
    systemDomainMask:  /System/Library/Documentation/
                       /System/Cryptexes/App/System/Library/Documentation/
    networkDomainMask: /Network/Library/Documentation/

documentDirectory:
    userDomainMask:    /Users/SadPanda/Library/Containers/com.SadPanda.MyApp/Data/Documents/
    localDomainMask:   
    systemDomainMask:  
    networkDomainMask: 

coreServiceDirectory:
    userDomainMask:    
    localDomainMask:   
    systemDomainMask:  /System/Library/CoreServices/
                       /System/Cryptexes/App/System/Library/CoreServices/
    networkDomainMask: 

autosavedInformationDirectory:
    userDomainMask:    /Users/SadPanda/Library/Containers/com.SadPanda.MyApp/Data/Library/Autosave Information/
    localDomainMask:   
    systemDomainMask:  
    networkDomainMask: 

desktopDirectory:
    userDomainMask:    /Users/SadPanda/Library/Containers/com.SadPanda.MyApp/Data/Desktop/
    localDomainMask:   
    systemDomainMask:  
    networkDomainMask: 

cachesDirectory:
    userDomainMask:    /Users/SadPanda/Library/Containers/com.SadPanda.MyApp/Data/Library/Caches/
    localDomainMask:   /Library/Caches/
    systemDomainMask:  /System/Library/Caches/
    networkDomainMask: 

applicationSupportDirectory:
    userDomainMask:    /Users/SadPanda/Library/Containers/com.SadPanda.MyApp/Data/Library/Application Support/
    localDomainMask:   /Library/Application Support/
    systemDomainMask:  /Library/Application Support/
    networkDomainMask: /Network/Library/Application Support/

downloadsDirectory:
    userDomainMask:    /Users/SadPanda/Library/Containers/com.SadPanda.MyApp/Data/Downloads/
    localDomainMask:   
    systemDomainMask:  
    networkDomainMask: 

inputMethodsDirectory:
    userDomainMask:    /Users/SadPanda/Library/Containers/com.SadPanda.MyApp/Data/Library/Input Methods/
    localDomainMask:   /Library/Input Methods/
    systemDomainMask:  /System/Library/Input Methods/
                       /System/Cryptexes/App/System/Library/Input Methods/
    networkDomainMask: /Network/Library/Input Methods/

moviesDirectory:
    userDomainMask:    /Users/SadPanda/Library/Containers/com.SadPanda.MyApp/Data/Movies/
    localDomainMask:   
    systemDomainMask:  
    networkDomainMask: 

musicDirectory:
    userDomainMask:    /Users/SadPanda/Library/Containers/com.SadPanda.MyApp/Data/Music/
    localDomainMask:   
    systemDomainMask:  
    networkDomainMask: 

picturesDirectory:
    userDomainMask:    /Users/SadPanda/Library/Containers/com.SadPanda.MyApp/Data/Pictures/
    localDomainMask:   
    systemDomainMask:  
    networkDomainMask: 

printerDescriptionDirectory:
    userDomainMask:    
    localDomainMask:   
    systemDomainMask:  /System/Library/Printers/PPDs/
                       /System/Cryptexes/App/System/Library/Printers/PPDs/
    networkDomainMask: 

sharedPublicDirectory:
    userDomainMask:    /Users/SadPanda/Library/Containers/com.SadPanda.MyApp/Data/Public/
    localDomainMask:   
    systemDomainMask:  
    networkDomainMask: 

preferencePanesDirectory:
    userDomainMask:    /Users/SadPanda/Library/Containers/com.SadPanda.MyApp/Data/Library/PreferencePanes/
    localDomainMask:   /Library/PreferencePanes/
    systemDomainMask:  /System/Library/PreferencePanes/
                       /System/Cryptexes/App/System/Library/PreferencePanes/
    networkDomainMask: 

applicationScriptsDirectory:
    userDomainMask:    /Users/SadPanda/Library/Application Scripts/com.SadPanda.MyApp/
    localDomainMask:   
    systemDomainMask:  
    networkDomainMask: 

itemReplacementDirectory:
    userDomainMask:    
    localDomainMask:   
    systemDomainMask:  
    networkDomainMask: 

allApplicationsDirectory:
    userDomainMask:    /Users/SadPanda/Library/Containers/com.SadPanda.MyApp/Data/Applications/
                       /Users/SadPanda/Library/Containers/com.SadPanda.MyApp/Data/Applications/Utilities/
                       /Users/SadPanda/Library/Containers/com.SadPanda.MyApp/Data/Developer/Applications/
                       /Users/SadPanda/Library/Containers/com.SadPanda.MyApp/Data/Applications/Demos/
    localDomainMask:   /Applications/
                       /Applications/Utilities/
                       /Developer/Applications/
                       /Applications/Demos/
    systemDomainMask:  /System/Applications/
                       /System/Applications/Utilities/
                       /System/Developer/Applications/
                       /System/Applications/Demos/
                       /System/Cryptexes/App/System/Applications/
                       /System/Cryptexes/App/System/Applications/Utilities/
                       /System/Cryptexes/App/System/Developer/Applications/
                       /System/Cryptexes/App/System/Applications/Demos/
    networkDomainMask: /Network/Applications/
                       /Network/Applications/Utilities/
                       /Network/Developer/Applications/
                       /Network/Applications/Demos/

allLibrariesDirectory:
    userDomainMask:    /Users/SadPanda/Library/Containers/com.SadPanda.MyApp/Data/Library/
                       /Users/SadPanda/Library/Containers/com.SadPanda.MyApp/Data/Developer/
    localDomainMask:   /Library/
                       /Developer/
    systemDomainMask:  /System/Library/
                       /Developer/
                       /System/Cryptexes/App/System/Library/
                       /System/Cryptexes/App/System/Developer/
    networkDomainMask: /Network/Library/
                       /Network/Developer/

trashDirectory:
    userDomainMask:    /Users/SadPanda/.Trash/
    localDomainMask:   /Users/SadPanda/.Trash/
    systemDomainMask:  
    networkDomainMask: 


url(for:in:appropriateFor:create:):

applicationDirectory:
    userDomainMask:    /Users/SadPanda/Library/Containers/com.SadPanda.MyApp/Data/Applications/
    localDomainMask:   /Applications/
    systemDomainMask:  /System/Applications/
    networkDomainMask: /Network/Applications/

demoApplicationDirectory:
    userDomainMask:    /Users/SadPanda/Library/Containers/com.SadPanda.MyApp/Data/Applications/Demos/
    localDomainMask:   /Applications/Demos/
    systemDomainMask:  /Applications/Demos/
    networkDomainMask: /Network/Applications/Demos/

developerApplicationDirectory:
    userDomainMask:    /Users/SadPanda/Library/Containers/com.SadPanda.MyApp/Data/Developer/Applications/
    localDomainMask:   /Developer/Applications/
    systemDomainMask:  /Developer/Applications/
    networkDomainMask: /Network/Developer/Applications/

adminApplicationDirectory:
    userDomainMask:    /Users/SadPanda/Library/Containers/com.SadPanda.MyApp/Data/Applications/Utilities/
    localDomainMask:   /Applications/Utilities/
    systemDomainMask:  /System/Applications/Utilities/
    networkDomainMask: /Network/Applications/Utilities/

libraryDirectory:
    userDomainMask:    /Users/SadPanda/Library/Containers/com.SadPanda.MyApp/Data/Library/
    localDomainMask:   /Library/
    systemDomainMask:  /System/Library/
    networkDomainMask: /Network/Library/

developerDirectory:
    userDomainMask:    /Users/SadPanda/Library/Containers/com.SadPanda.MyApp/Data/Developer/
    localDomainMask:   /Developer/
    systemDomainMask:  /Developer/
    networkDomainMask: /Network/Developer/

userDirectory:
    userDomainMask:    ERROR (nilError)
    localDomainMask:   /Users/
    systemDomainMask:  ERROR (nilError)
    networkDomainMask: /Network/Users/

documentationDirectory:
    userDomainMask:    /Users/SadPanda/Library/Containers/com.SadPanda.MyApp/Data/Library/Documentation/
    localDomainMask:   /Library/Documentation/
    systemDomainMask:  /System/Library/Documentation/
    networkDomainMask: /Network/Library/Documentation/

documentDirectory:
    userDomainMask:    /Users/SadPanda/Library/Containers/com.SadPanda.MyApp/Data/Documents/
    localDomainMask:   ERROR (nilError)
    systemDomainMask:  ERROR (nilError)
    networkDomainMask: ERROR (nilError)

coreServiceDirectory:
    userDomainMask:    ERROR (nilError)
    localDomainMask:   ERROR (nilError)
    systemDomainMask:  /System/Library/CoreServices/
    networkDomainMask: ERROR (nilError)

autosavedInformationDirectory:
    userDomainMask:    /Users/SadPanda/Library/Containers/com.SadPanda.MyApp/Data/Library/Autosave Information/
    localDomainMask:   ERROR (nilError)
    systemDomainMask:  ERROR (nilError)
    networkDomainMask: ERROR (nilError)

desktopDirectory:
    userDomainMask:    /Users/SadPanda/Library/Containers/com.SadPanda.MyApp/Data/Desktop/
    localDomainMask:   ERROR (nilError)
    systemDomainMask:  ERROR (nilError)
    networkDomainMask: ERROR (nilError)

cachesDirectory:
    userDomainMask:    /Users/SadPanda/Library/Containers/com.SadPanda.MyApp/Data/Library/Caches/
    localDomainMask:   /Library/Caches/
    systemDomainMask:  /System/Library/Caches/
    networkDomainMask: ERROR (nilError)

applicationSupportDirectory:
    userDomainMask:    /Users/SadPanda/Library/Containers/com.SadPanda.MyApp/Data/Library/Application Support/
    localDomainMask:   /Library/Application Support/
    systemDomainMask:  /Library/Application Support/
    networkDomainMask: /Network/Library/Application Support/

downloadsDirectory:
    userDomainMask:    /Users/SadPanda/Library/Containers/com.SadPanda.MyApp/Data/Downloads/
    localDomainMask:   ERROR (nilError)
    systemDomainMask:  ERROR (nilError)
    networkDomainMask: ERROR (nilError)

inputMethodsDirectory:
    userDomainMask:    /Users/SadPanda/Library/Containers/com.SadPanda.MyApp/Data/Library/Input Methods/
    localDomainMask:   /Library/Input Methods/
    systemDomainMask:  /System/Library/Input Methods/
    networkDomainMask: /Network/Library/Input Methods/

moviesDirectory:
    userDomainMask:    /Users/SadPanda/Library/Containers/com.SadPanda.MyApp/Data/Movies/
    localDomainMask:   ERROR (nilError)
    systemDomainMask:  ERROR (nilError)
    networkDomainMask: ERROR (nilError)

musicDirectory:
    userDomainMask:    /Users/SadPanda/Library/Containers/com.SadPanda.MyApp/Data/Music/
    localDomainMask:   ERROR (nilError)
    systemDomainMask:  ERROR (nilError)
    networkDomainMask: ERROR (nilError)

picturesDirectory:
    userDomainMask:    /Users/SadPanda/Library/Containers/com.SadPanda.MyApp/Data/Pictures/
    localDomainMask:   ERROR (nilError)
    systemDomainMask:  ERROR (nilError)
    networkDomainMask: ERROR (nilError)

printerDescriptionDirectory:
    userDomainMask:    ERROR (nilError)
    localDomainMask:   ERROR (nilError)
    systemDomainMask:  /System/Library/Printers/PPDs/
    networkDomainMask: ERROR (nilError)

sharedPublicDirectory:
    userDomainMask:    /Users/SadPanda/Library/Containers/com.SadPanda.MyApp/Data/Public/
    localDomainMask:   ERROR (nilError)
    systemDomainMask:  ERROR (nilError)
    networkDomainMask: ERROR (nilError)

preferencePanesDirectory:
    userDomainMask:    /Users/SadPanda/Library/Containers/com.SadPanda.MyApp/Data/Library/PreferencePanes/
    localDomainMask:   /Library/PreferencePanes/
    systemDomainMask:  /System/Library/PreferencePanes/
    networkDomainMask: ERROR (nilError)

applicationScriptsDirectory:
    userDomainMask:    /Users/SadPanda/Library/Application Scripts/com.SadPanda.MyApp/
    localDomainMask:   ERROR (nilError)
    systemDomainMask:  ERROR (nilError)
    networkDomainMask: ERROR (nilError)

itemReplacementDirectory:
    userDomainMask:
        /:                                                               /Users/SadPanda/Library/Containers/com.SadPanda.MyApp/Data/tmp/TemporaryItems/NSIRD_MyApp_oflT6r/
        /Users/SadPanda/Library/Containers/com.SadPanda.MyApp/Data/tmp/: /Users/SadPanda/Library/Containers/com.SadPanda.MyApp/Data/tmp/TemporaryItems/NSIRD_MyApp_xkFfky/
        /Volumes/Other/:                                                 /Volumes/Other/.TemporaryItems/folders.501/TemporaryItems/NSIRD_MyApp_bLj7gn/
        nil:                                                             ERROR (nilError)
    localDomainMask:   ERROR (nilError)
    systemDomainMask:  ERROR (nilError)
    networkDomainMask: ERROR (nilError)

allApplicationsDirectory:
    userDomainMask:    /Users/SadPanda/Library/Containers/com.SadPanda.MyApp/Data/Applications/
    localDomainMask:   /Applications/
    systemDomainMask:  /System/Applications/Demos/
    networkDomainMask: /Network/Applications/

allLibrariesDirectory:
    userDomainMask:    /Users/SadPanda/Library/Containers/com.SadPanda.MyApp/Data/Library/
    localDomainMask:   /Library/
    systemDomainMask:  /Developer/
    networkDomainMask: /Network/Library/

trashDirectory:
    userDomainMask:
        /:                                                               /Users/SadPanda/.Trash/
        /Users/SadPanda/Library/Containers/com.SadPanda.MyApp/Data/tmp/: /Users/SadPanda/.Trash/
        /Volumes/Other/:                                                 /Volumes/Other/.Trashes/501/
        nil:                                                             /Users/SadPanda/.Trash/
    localDomainMask:
        /:                                                               /Users/SadPanda/.Trash/
        /Users/SadPanda/Library/Containers/com.SadPanda.MyApp/Data/tmp/: /Users/SadPanda/.Trash/
        /Volumes/Other/:                                                 /Volumes/Other/.Trashes/501/
        nil:                                                             /Users/SadPanda/.Trash/
    systemDomainMask:
        /:                                                               /Users/SadPanda/.Trash/
        /Users/SadPanda/Library/Containers/com.SadPanda.MyApp/Data/tmp/: /Users/SadPanda/.Trash/
        /Volumes/Other/:                                                 /Volumes/Other/.Trashes/501/
        nil:                                                             ERROR (nilError)
    networkDomainMask:
        /:                                                               /Users/SadPanda/.Trash/
        /Users/SadPanda/Library/Containers/com.SadPanda.MyApp/Data/tmp/: /Users/SadPanda/.Trash/
        /Volumes/Other/:                                                 /Volumes/Other/.Trashes/501/
        nil:                                                             ERROR (nilError)

The output outside of an App Sandbox is pretty similar, just different paths in some cases as you’d expect.

urls(for:in:):

applicationDirectory:
    userDomainMask:    /Users/SadPanda/Applications/
    localDomainMask:   /Applications/
    systemDomainMask:  /System/Applications/
                       /System/Cryptexes/App/System/Applications/
    networkDomainMask: /Network/Applications/

demoApplicationDirectory:
    userDomainMask:    /Users/SadPanda/Applications/Demos/
    localDomainMask:   /Applications/Demos/
    systemDomainMask:  /Applications/Demos/
    networkDomainMask: /Network/Applications/Demos/

developerApplicationDirectory:
    userDomainMask:    /Users/SadPanda/Developer/Applications/
    localDomainMask:   /Developer/Applications/
    systemDomainMask:  /Developer/Applications/
    networkDomainMask: /Network/Developer/Applications/

adminApplicationDirectory:
    userDomainMask:    /Users/SadPanda/Applications/Utilities/
    localDomainMask:   /Applications/Utilities/
    systemDomainMask:  /System/Applications/Utilities/
                       /System/Cryptexes/App/System/Applications/Utilities/
    networkDomainMask: /Network/Applications/Utilities/

libraryDirectory:
    userDomainMask:    /Users/SadPanda/Library/
    localDomainMask:   /Library/
    systemDomainMask:  /System/Library/
                       /System/Cryptexes/App/System/Library/
    networkDomainMask: /Network/Library/

developerDirectory:
    userDomainMask:    /Users/SadPanda/Developer/
    localDomainMask:   /Developer/
    systemDomainMask:  /Developer/
    networkDomainMask: /Network/Developer/

userDirectory:
    userDomainMask:    
    localDomainMask:   /Users/
    systemDomainMask:  
    networkDomainMask: /Network/Users/

documentationDirectory:
    userDomainMask:    /Users/SadPanda/Library/Documentation/
    localDomainMask:   /Library/Documentation/
    systemDomainMask:  /System/Library/Documentation/
                       /System/Cryptexes/App/System/Library/Documentation/
    networkDomainMask: /Network/Library/Documentation/

documentDirectory:
    userDomainMask:    /Users/SadPanda/Documents/
    localDomainMask:   
    systemDomainMask:  
    networkDomainMask: 

coreServiceDirectory:
    userDomainMask:    
    localDomainMask:   
    systemDomainMask:  /System/Library/CoreServices/
                       /System/Cryptexes/App/System/Library/CoreServices/
    networkDomainMask: 

autosavedInformationDirectory:
    userDomainMask:    /Users/SadPanda/Library/Autosave Information/
    localDomainMask:   
    systemDomainMask:  
    networkDomainMask: 

desktopDirectory:
    userDomainMask:    /Users/SadPanda/Desktop/
    localDomainMask:   
    systemDomainMask:  
    networkDomainMask: 

cachesDirectory:
    userDomainMask:    /Users/SadPanda/Library/Caches/
    localDomainMask:   /Library/Caches/
    systemDomainMask:  /System/Library/Caches/
    networkDomainMask: 

applicationSupportDirectory:
    userDomainMask:    /Users/SadPanda/Library/Application Support/
    localDomainMask:   /Library/Application Support/
    systemDomainMask:  /Library/Application Support/
    networkDomainMask: /Network/Library/Application Support/

downloadsDirectory:
    userDomainMask:    /Users/SadPanda/Downloads/
    localDomainMask:   
    systemDomainMask:  
    networkDomainMask: 

inputMethodsDirectory:
    userDomainMask:    /Users/SadPanda/Library/Input Methods/
    localDomainMask:   /Library/Input Methods/
    systemDomainMask:  /System/Library/Input Methods/
                       /System/Cryptexes/App/System/Library/Input Methods/
    networkDomainMask: /Network/Library/Input Methods/

moviesDirectory:
    userDomainMask:    /Users/SadPanda/Movies/
    localDomainMask:   
    systemDomainMask:  
    networkDomainMask: 

musicDirectory:
    userDomainMask:    /Users/SadPanda/Music/
    localDomainMask:   
    systemDomainMask:  
    networkDomainMask: 

picturesDirectory:
    userDomainMask:    /Users/SadPanda/Pictures/
    localDomainMask:   
    systemDomainMask:  
    networkDomainMask: 

printerDescriptionDirectory:
    userDomainMask:    
    localDomainMask:   
    systemDomainMask:  /System/Library/Printers/PPDs/
                       /System/Cryptexes/App/System/Library/Printers/PPDs/
    networkDomainMask: 

sharedPublicDirectory:
    userDomainMask:    /Users/SadPanda/Public/
    localDomainMask:   
    systemDomainMask:  
    networkDomainMask: 

preferencePanesDirectory:
    userDomainMask:    /Users/SadPanda/Library/PreferencePanes/
    localDomainMask:   /Library/PreferencePanes/
    systemDomainMask:  /System/Library/PreferencePanes/
                       /System/Cryptexes/App/System/Library/PreferencePanes/
    networkDomainMask: 

applicationScriptsDirectory:
    userDomainMask:    /Users/SadPanda/Library/Application Scripts/com.SadPanda.MyApp/
    localDomainMask:   
    systemDomainMask:  
    networkDomainMask: 

itemReplacementDirectory:
    userDomainMask:    
    localDomainMask:   
    systemDomainMask:  
    networkDomainMask: 

allApplicationsDirectory:
    userDomainMask:    /Users/SadPanda/Applications/
                       /Users/SadPanda/Applications/Utilities/
                       /Users/SadPanda/Developer/Applications/
                       /Users/SadPanda/Applications/Demos/
    localDomainMask:   /Applications/
                       /Applications/Utilities/
                       /Developer/Applications/
                       /Applications/Demos/
    systemDomainMask:  /System/Applications/
                       /System/Applications/Utilities/
                       /System/Developer/Applications/
                       /System/Applications/Demos/
                       /System/Cryptexes/App/System/Applications/
                       /System/Cryptexes/App/System/Applications/Utilities/
                       /System/Cryptexes/App/System/Developer/Applications/
                       /System/Cryptexes/App/System/Applications/Demos/
    networkDomainMask: /Network/Applications/
                       /Network/Applications/Utilities/
                       /Network/Developer/Applications/
                       /Network/Applications/Demos/

allLibrariesDirectory:
    userDomainMask:    /Users/SadPanda/Library/
                       /Users/SadPanda/Developer/
    localDomainMask:   /Library/
                       /Developer/
    systemDomainMask:  /System/Library/
                       /Developer/
                       /System/Cryptexes/App/System/Library/
                       /System/Cryptexes/App/System/Developer/
    networkDomainMask: /Network/Library/
                       /Network/Developer/

trashDirectory:
    userDomainMask:    /Users/SadPanda/.Trash/
    localDomainMask:   /Users/SadPanda/.Trash/
    systemDomainMask:  
    networkDomainMask: 


url(for:in:appropriateFor:create:):

applicationDirectory:
    userDomainMask:    /Users/SadPanda/Applications/
    localDomainMask:   /Applications/
    systemDomainMask:  /System/Applications/
    networkDomainMask: /Network/Applications/

demoApplicationDirectory:
    userDomainMask:    /Users/SadPanda/Applications/Demos/
    localDomainMask:   /Applications/Demos/
    systemDomainMask:  /Applications/Demos/
    networkDomainMask: /Network/Applications/Demos/

developerApplicationDirectory:
    userDomainMask:    /Users/SadPanda/Developer/Applications/
    localDomainMask:   /Developer/Applications/
    systemDomainMask:  /Developer/Applications/
    networkDomainMask: /Network/Developer/Applications/

adminApplicationDirectory:
    userDomainMask:    /Users/SadPanda/Applications/Utilities/
    localDomainMask:   /Applications/Utilities/
    systemDomainMask:  /System/Applications/Utilities/
    networkDomainMask: /Network/Applications/Utilities/

libraryDirectory:
    userDomainMask:    /Users/SadPanda/Library/
    localDomainMask:   /Library/
    systemDomainMask:  /System/Library/
    networkDomainMask: /Network/Library/

developerDirectory:
    userDomainMask:    /Users/SadPanda/Developer/
    localDomainMask:   /Developer/
    systemDomainMask:  /Developer/
    networkDomainMask: /Network/Developer/

userDirectory:
    userDomainMask:    ERROR (nilError)
    localDomainMask:   /Users/
    systemDomainMask:  ERROR (nilError)
    networkDomainMask: /Network/Users/

documentationDirectory:
    userDomainMask:    /Users/SadPanda/Library/Documentation/
    localDomainMask:   /Library/Documentation/
    systemDomainMask:  /System/Library/Documentation/
    networkDomainMask: /Network/Library/Documentation/

documentDirectory:
    userDomainMask:    /Users/SadPanda/Documents/
    localDomainMask:   ERROR (nilError)
    systemDomainMask:  ERROR (nilError)
    networkDomainMask: ERROR (nilError)

coreServiceDirectory:
    userDomainMask:    ERROR (nilError)
    localDomainMask:   ERROR (nilError)
    systemDomainMask:  /System/Library/CoreServices/
    networkDomainMask: ERROR (nilError)

autosavedInformationDirectory:
    userDomainMask:    /Users/SadPanda/Library/Autosave Information/
    localDomainMask:   ERROR (nilError)
    systemDomainMask:  ERROR (nilError)
    networkDomainMask: ERROR (nilError)

desktopDirectory:
    userDomainMask:    /Users/SadPanda/Desktop/
    localDomainMask:   ERROR (nilError)
    systemDomainMask:  ERROR (nilError)
    networkDomainMask: ERROR (nilError)

cachesDirectory:
    userDomainMask:    /Users/SadPanda/Library/Caches/
    localDomainMask:   /Library/Caches/
    systemDomainMask:  /System/Library/Caches/
    networkDomainMask: ERROR (nilError)

applicationSupportDirectory:
    userDomainMask:    /Users/SadPanda/Library/Application Support/
    localDomainMask:   /Library/Application Support/
    systemDomainMask:  /Library/Application Support/
    networkDomainMask: /Network/Library/Application Support/

downloadsDirectory:
    userDomainMask:    /Users/SadPanda/Downloads/
    localDomainMask:   ERROR (nilError)
    systemDomainMask:  ERROR (nilError)
    networkDomainMask: ERROR (nilError)

inputMethodsDirectory:
    userDomainMask:    /Users/SadPanda/Library/Input Methods/
    localDomainMask:   /Library/Input Methods/
    systemDomainMask:  /System/Library/Input Methods/
    networkDomainMask: /Network/Library/Input Methods/

moviesDirectory:
    userDomainMask:    /Users/SadPanda/Movies/
    localDomainMask:   ERROR (nilError)
    systemDomainMask:  ERROR (nilError)
    networkDomainMask: ERROR (nilError)

musicDirectory:
    userDomainMask:    /Users/SadPanda/Music/
    localDomainMask:   ERROR (nilError)
    systemDomainMask:  ERROR (nilError)
    networkDomainMask: ERROR (nilError)

picturesDirectory:
    userDomainMask:    /Users/SadPanda/Pictures/
    localDomainMask:   ERROR (nilError)
    systemDomainMask:  ERROR (nilError)
    networkDomainMask: ERROR (nilError)

printerDescriptionDirectory:
    userDomainMask:    ERROR (nilError)
    localDomainMask:   ERROR (nilError)
    systemDomainMask:  /System/Library/Printers/PPDs/
    networkDomainMask: ERROR (nilError)

sharedPublicDirectory:
    userDomainMask:    /Users/SadPanda/Public/
    localDomainMask:   ERROR (nilError)
    systemDomainMask:  ERROR (nilError)
    networkDomainMask: ERROR (nilError)

preferencePanesDirectory:
    userDomainMask:    /Users/SadPanda/Library/PreferencePanes/
    localDomainMask:   /Library/PreferencePanes/
    systemDomainMask:  /System/Library/PreferencePanes/
    networkDomainMask: ERROR (nilError)

applicationScriptsDirectory:
    userDomainMask:    /Users/SadPanda/Library/Application Scripts/com.SadPanda.MyApp/
    localDomainMask:   ERROR (nilError)
    systemDomainMask:  ERROR (nilError)
    networkDomainMask: ERROR (nilError)

itemReplacementDirectory:
    userDomainMask:
        /:                                                 /var/folders/v3/8anb56f64adf3_35gj346jg13000xa/T/TemporaryItems/NSIRD_MyApp_6D4CLt/
        /Volumes/Other/:                                   /Volumes/Other/.TemporaryItems/folders.501/TemporaryItems/NSIRD_MyApp_fdKnpT/
        /var/folders/v3/8anb56f64adf3_35gj346jg13000xa/T/: /var/folders/v3/8anb56f64adf3_35gj346jg13000xa/T/TemporaryItems/NSIRD_MyApp_bcHf11/
        nil:                                               ERROR (nilError)
    localDomainMask:   ERROR (nilError)
    systemDomainMask:  ERROR (nilError)
    networkDomainMask: ERROR (nilError)

allApplicationsDirectory:
    userDomainMask:    /Users/SadPanda/Applications/
    localDomainMask:   /Applications/
    systemDomainMask:  /System/Applications/Demos/
    networkDomainMask: /Network/Applications/

allLibrariesDirectory:
    userDomainMask:    /Users/SadPanda/Library/
    localDomainMask:   /Library/
    systemDomainMask:  /Developer/
    networkDomainMask: /Network/Library/

trashDirectory:
    userDomainMask:
        /:                                                 /Users/SadPanda/.Trash/
        /Volumes/Other/:                                   /Volumes/Other/.Trashes/501/
        /var/folders/v3/8anb56f64adf3_35gj346jg13000xa/T/: /Users/SadPanda/.Trash/
        nil:                                               /Users/SadPanda/.Trash/
    localDomainMask:
        /:                                                 /Users/SadPanda/.Trash/
        /Volumes/Other/:                                   /Volumes/Other/.Trashes/501/
        /var/folders/v3/8anb56f64adf3_35gj346jg13000xa/T/: /Users/SadPanda/.Trash/
        nil:                                               /Users/SadPanda/.Trash/
    systemDomainMask:
        /:                                                 /Users/SadPanda/.Trash/
        /Volumes/Other/:                                   /Volumes/Other/.Trashes/501/
        /var/folders/v3/8anb56f64adf3_35gj346jg13000xa/T/: /Users/SadPanda/.Trash/
        nil:                                               ERROR (nilError)
    networkDomainMask:
        /:                                                 /Users/SadPanda/.Trash/
        /Volumes/Other/:                                   /Volumes/Other/.Trashes/501/
        /var/folders/v3/8anb56f64adf3_35gj346jg13000xa/T/: /Users/SadPanda/.Trash/
        nil:                                               ERROR (nilError)

Note how urls(for:in:) works for every SearchPathDirectory except itemReplacementDirectory.

cannot open file at line 49259 of [1b37c146ee]
os_unix.c:49259: (0) open(/private/var/db/DetachedSignatures) - Undefined error: 0

This method should not be called on the main thread as it may lead to UI unresponsiveness.

Some observations about urls(for:in:appropriateFor:create:):

• It returns a single URL, even though there are often multiple folders for a given search path. Note how it never returns a path to a Cryptex folder, for example.
  
  If you’re looking for search paths – the set of folders to search for a resource – use urls(for:in:) only.
• The appropriateFor argument is completely irrelevant to every SearchPathDirectory except itemReplacementDirectory and trashDirectory.
• It will only ever return a path inside the App Sandbox for userDomainMask (if it doesn’t fail by throwing an exception).
• It won’t allow nil as an argument for appropriateFor when targeting itemReplacementDirectory, yet it will allow nil when targeting trashDirectory, but only for userDomainMask and localDomainMask. I cannot think of any explanation for this inconsistency.
  
  If you ignore that inconsistency, the SearchPathDomainMask is irrelevant to trashDirectory (as it logically should be – bin folders are tied to volumes, not apps, users, or computers.
• It always throws an exception for itemReplacementDirectory if the SearchPathDomainMask is not userDomainMask. This might actually be explicable, even if a bit unintuitive and misguided – it seems to be presuming that, because you’re running inside an App Sandbox, you cannot modify any files except the current user’s. I’m not sure that’s not accurate – there’s the possibility of special entitlements and exclusions – although nor do I know that it’s not.
• Whenever it fails, it throws a nilError exception which is largely useless. Even the name doesn’t provide any real insight into what its problem is.
  
  It’s unsurprising to me that it does such a poor job, given the rest of the API – good error messages are a hallmark of good API design, or put conversely, happy path programming produces bad code.

I did not explore use of the create argument (I always left it as false). I didn’t need to in order to know it has its own problems, too – the documentation states that it also has inconsistent behaviour, in that it is completely ignored for itemReplacementDirectory (and only that SearchPathDirectory).

The root of all these problems is the API’s bad design:

• itemReplacementDirectory and trashDirectory should not be part of SearchPathDirectory. They do not behave like any of the other cases; they do not have broadly correct generic values, requiring instead some context regarding the items being replaced or trashed.
• url(for:in:appropriateFor:create) should be three independent methods – one of which already exists, in the form of urls(for:in:). The other two should handle each of the special cases of itemReplacementDirectory and trashDirectory. i.e.:

// Existing method
func urls(for directory: FileManager.SearchPathDirectory,
          in domainMask: FileManager.SearchPathDomainMask) -> [URL]

// New methods
func replacementItemFolder(appropriateFor url: URL) throws(InvalidURLError) -> URL

func trashFolder(appropriateFor url: URL) throws(InvalidURLError) -> URL

So much clearer and easier to use than what we have now.

The two specialised methods still need to throw, because they take URLs which could be invalid, but that’s now the only reason they might throw (I’ve used a hypothetical InvalidURLError type to express that formally).

They could arguably be combined into one method (taking an enum that delineates the two cases), but I see little practical value in that – it’s simpler and easier to read if they’re simply distinct methods, even though their shape is similar. I can’t imagine any reasonable scenario where you only decide at runtime if you’re going to replace something or delete it.

☝️ I’ve removed the create parameter entirely, as I think it presumes too much (e.g. what if you want to create a file with the replacement item URL, not a folder?), though arguably it could be added back in (though if it were, it should actually be obeyed).

1. Note that I wrote applicable, not necessarily used. The distinction is important. e.g. it makes sense to supply the request headers for a HTTP request to an API which makes such requests, even if the request might be served from a local cache in which case those headers aren’t actually used, in that instance.
   
   Another way to frame this is that it’s a design flaw to have parameters whose utility depends only on the value of other parameters.
   
   If you can know at compile time that some parameters are unnecessary, then you shouldn’t be forced to provide them. ↩︎

The following classes and functions are generally not thread-safe. In most cases, you can use these classes from any thread as long as you use them from only one thread at a time. Check the class documentation for additional details.

Apple’s Threading Programming Guide > Appendix A: Thread Safety Summary, subsection Application Kit Framework Thread Safety

What “in most cases” means is left to the reader’s imagination. Apple adds a little addendum for NSImage specifically:

One thread can create an NSImage object, draw to the image buffer, and pass it off to the main thread for drawing. The underlying image cache is shared among all threads. For more information about images and how caching works, see Cocoa Drawing Guide.

NSImage Restrictions

For a start, it’s talking only about creating an NSImage from scratch, not loading it from serialised form (e.g. a file, a pasteboard, etc). It doesn’t even deign to mention those other, much more common cases.

And even for that one mentioned use case, what does it mean, exactly? What “image cache” is it referring to?

I don’t have authoritative answers. The documentation is so infuriatingly vague that Apple could do basically anything to the implementation, between macOS updates, and claim to have broken no promises.

What I do have is some empirical data and the results of some reverse engineering (shout out to Hopper), from macOS 14.2 Sonoma.

NSImage 101

NSImages can represent a wide range of imagery. Most uses of them are probably for bitmap data (i.e. what you find in common image formats like WebP & AVIF), but NSImage also supports ‘raw’ images (e.g. Nikon NEF) as well as vector data (e.g. SVG & PDF). It also has a plug-in mechanism of sorts, so the supported image formats can be extended dynamically at runtime.

You can fetch the full list of supported types from NSImage.imageTypes – although it doesn’t distinguish between those it can read vs those it can write. Fortunately, sips --formats in Terminal gives you the same list with additional metadata. On my machine that list happens to be:

Supported Formats:
-------------------------------------------
com.adobe.pdf                pdf   Writable
com.adobe.photoshop-image    psd   Writable
com.adobe.raw-image          dng   
com.apple.atx                --    Writable
com.apple.icns               icns  Writable
com.apple.pict               pict  
com.canon.cr2-raw-image      cr2   
com.canon.cr3-raw-image      cr3   
com.canon.crw-raw-image      crw   
com.canon.tif-raw-image      tif   
com.compuserve.gif           gif   Writable
com.dxo.raw-image            dxo   
com.epson.raw-image          erf   
com.fuji.raw-image           raf   
com.hasselblad.3fr-raw-image 3fr   
com.hasselblad.fff-raw-image fff   
com.ilm.openexr-image        exr   Writable
com.kodak.raw-image          dcr   
com.konicaminolta.raw-image  mrw   
com.leafamerica.raw-image    mos   
com.leica.raw-image          raw   
com.leica.rwl-raw-image      rwl   
com.microsoft.bmp            bmp   Writable
com.microsoft.cur            --    
com.microsoft.dds            dds   Writable
com.microsoft.ico            ico   Writable
com.nikon.nrw-raw-image      nrw   
com.nikon.raw-image          nef   
com.olympus.or-raw-image     orf   
com.olympus.raw-image        orf   
com.olympus.sr-raw-image     orf   
com.panasonic.raw-image      raw   
com.panasonic.rw2-raw-image  rw2   
com.pentax.raw-image         pef   
com.phaseone.raw-image       iiq   
com.samsung.raw-image        srw   
com.sgi.sgi-image            sgi   
com.sony.arw-raw-image       arw   
com.sony.raw-image           srf   
com.sony.sr2-raw-image       sr2   
com.truevision.tga-image     tga   Writable
org.khronos.astc             astc  Writable
org.khronos.ktx              ktx   Writable
org.khronos.ktx2             --    Writable
org.webmproject.webp         webp  
public.avci                  avci  
public.avif                  avif  
public.avis                  --    
public.heic                  heic  Writable
public.heics                 heics Writable
public.heif                  heif  
public.jpeg                  jpeg  Writable
public.jpeg-2000             jp2   Writable
public.jpeg-xl               jxl   
public.mpo-image             mpo   
public.pbm                   pbm   Writable
public.png                   png   Writable
public.pvr                   pvr   Writable
public.radiance              pic   
public.svg-image             svg   
public.tiff                  tiff  Writable

An NSImage can have multiple ‘representations‘. These are essentially just different versions of the image. One representation might be the original vector form (e.g. an NSPDFImageRep). Another might be a rasterisation of that at a certain resolution. Yet another might be a rasterisation at a different resolution.

It gets more complicated, however, because some representations are merely proxies for other frameworks’ representations, like CGImage, CIImage or somesuch.

Nonetheless, for the simple case of a static bitmap image read from a file, generally NSImage produces just one representation, which is the full bitmap (as an NSBitmapImageRep). That’s the only case I’m going to discuss here, for simplicity’s sake (though likely the lessons apply to the other cases too).

Accessing / using an NSImage

Generally to use an NSImage you need a bitmap representation. e.g. to actually draw it to the screen.

Unless you manually create an NSImage from an in-memory bitmap, the bitmap representation is not loaded initially, but rather only when first needed.

Ultimately the load is triggered when something asks for the raw bytes of the bitmap (either you, directly in your code, or indirectly when you e.g. ask AppKit to draw the image).

Fetching the raw bytes of a bitmap from NSBitmapImageRep is not a trivial exercise. I’m going to gloss over the complexities (like planar data formats) and just talk about the common case of single-plane (“interleaved channels”) bitmaps.

For those, you can access the raw bytes via the bitmapData property.

Nominally, bitmapData just returns a uint8_t*.

In fact, bitmapData calls a lot of internal methods in a complicated fashion, with many possible code paths. What exactly it does depends on the underlying source of data, but in a nutshell it checks if the desired data has already been created / loaded (it’s cached in an instance variable) and if not it loads it, e.g. by calling CGImageSourceCreateImageAtIndex.

That’s what makes NSImage dangerous to use from multiple threads simultaneously.

Reading the cached bitmap data is essentially read-only (just some retain/autorelease traffic) so it’s safe to call from multiple threads concurrently (as long as something ensures the NSBitmapImageRep is kept alive the whole time and not mutated)

But loading or modifying it is not. As such, if you call bitmapData from multiple threads concurrently, and you don’t know for sure that it’s already fully loaded, you get a data race (also known as a “WTF does my app crash randomly?!” condition).

The consequences of that race vary. Maybe you “win” the race – one thread happens to run virtually to completion of bitmapData first, storing the fresh backing data into the caching instance member, and then all the other threads run and just return that same value – the ideal situation as everything works as intended.

Maybe you “lose” the race: every concurrent thread checks simultaneously and sees there’s no cached value, so they all – in parallel and redundant to each other – load the bitmap data and store it into the cache. They each return the one they created, even though ultimately only one thread wins – the last one to write into the cache – and the duplicate bitmap data that all the earlier threads created is deallocated. Even though pointers to them have been returned to you. And you might be in the middle of using them. Causing you to crash with a memory protection fault (or worse, read from some random other memory allocation that happened to be placed at the same address afterwards, reading essentially garbage).

Hypocritically, this is because Apple don’t follow their own rules on escaping pointers:

The pointer you pass to the function is only guaranteed to be valid for the duration of the function call. Do not persist the pointer and access it after the function has returned.

Apple’s Swift Standard Library > Manual Memory Management > Calling Functions With Pointer Parameters, subsection Pass a Constant Pointer as a Parameter.

The source code for the relevant NSBitmapImageRep methods is essentially:

- (uint8_t*)bitmapData {
    uint8_t **result = nil;
    [self getBitmapDataPlanes:&result];
    return *result;
}

- (void)getBitmapDataPlanes:(uint8_t***)output {
    [self _performBlockUsingBackingMutableData:^void (uint8_t* dataPlanes[5]) {
        *output = dataPlanes;
    }];
}

🤦‍♂️

Unfortunately, while _performBlockUsingBackingMutableData and its many similar siblings can be made safe, they are (a) all private and (b) not currently enforcing the necessary mutual exclusion. This could be corrected by Apple in future (although I wouldn’t hold your breath).

Why does it matter that NSImage is not thread-safe?

The crux of the problem is that you often have no good choice about it, because NSImage is a currency type used widely throughout Apple’s own frameworks, and you often have no control over what thread it’s created or used on.

For example, even using the very latest Apple APIs such as SwiftUI’s dropDestination(for:action:isTargeted:), you cannot control which thread the NSImages are created on (it appears to be the main thread, although that API provides no guarantees).

Similarly you have no control over where those images are used if you pass them to 3rd party code – including Apple’s. e.g. Image(nsImage:); possibly that only uses them on the main thread, but it might be pre-rendered the image a separate thread for better performance (to avoid blocking the main thread and causing the app to hang). In fact it should, in principle.

Loading an image – actually reading it from a file or URL and decompressing it into a raw bitmap suitable for drawing to the screen – should never be done on the main thread, because it can take a long time. A 1 GiB TIFF takes nearly 30 seconds on my iMac Pro, for example (and TIFF uses very lightweight compression, so it’s a relatively fast-to-read format). Anything involving the network could take an unbounded amount of time. Even small files – like a 20 MiB NEF – can take seconds to render because they are non-trivial to decode and/or decompress.

So you’re screwed on multiple levels: not only can you generally not guarantee what thread NSImage is born on nor used on, you can’t use it exclusively on the main thread because that will cause a terrible user experience.

What can you do?

In simple terms, the best you can realistically do is try ensure that all modifications to the NSImage (including implicit ones, such as loading a bitmap representation upon first use) happen exclusively in one thread [at a time]. For example, in my experiments, putting a lock around otherwise concurrent calls to bitmapData is enough to prevent any data races. Although I am mystified as to how I can still let the main thread draw the image concurrently without any apparent problems. 🤔

If you want to play it really safe, you have to create & pre-load each NSImage on a single thread (not the main thread), before ever sending it to another isolation domain. That means manually reimplementing things like drag and drop of images (because you can no longer work directly with NSImage with any drag & drop APIs, you have to instead use only – and all – the things that could potentially be images, like URLs or data blobs, and then translate those to & from NSImages manually).

Complicating all this is that NSImage is unclear about how its caching works. For starters, does this caching apply to the representations or is it a hidden, orthogonal system? Is it thread-safe? Etc.

You can supposedly modify the caching behaviour via the cacheMode property, but in my experience there is no apparent effect no matter what it is set it to (not on the image’s representations nor on render performance in the GUI).

It’s a shame that NSImage has been so neglected, and has so many glaring problems. Over the years Apple have seemingly tried to replace it, introducing new image types over and over again, but all that’s done is made everything more complicated.

$ ls -lh /System/Library/Frameworks/AppKit.framework/Versions/Current/AppKit
ls: /System/Library/Frameworks/AppKit.framework/Versions/Current/AppKit: No such file or directory

WTF, mate?

Invariably I spend half an hour websearching around to try to figure out how the hell my system got so broken, and how it’s possible to even boot macOS in such a corrupt state, until finally I chance upon Michael Tsai’s excellent summary of how Apple broke their frameworks starting in Big Sur.

The good news for Hopper is that it has since been updated to work around this – you can access the Apple framework binaries through File > Read File from DYLD Cache… There’s also tools like dyld-shared-cache-extractor which can resurrect the binaries from the cache.

Note also that in Sonoma, at least, the cache lives at /System/Volumes/Preboot/Cryptexes/OS/System/Library/dyld/ (in previous macOS releases it was apparently in /System/Library/dyld/).

Right from there, you know you’re going to have a bad day. 😔

Then you try to actually use the file dropped on your app, and you get:

Upload preparation for claim 1C0F9013-4DEB-4E5D-8896-F522AA979BA6 completed with error: Error Domain=NSCocoaErrorDomain Code=513 "“Example.jpg” couldn’t be copied because you don’t have permission to access “CoordinatedZipFilejxc2lC”." UserInfo={NSSourceFilePathErrorKey=/Users/SadPanda/Pictures/Example.jpg, NSUserStringVariant=(
    Copy
), NSDestinationFilePath=/Users/SadPanda/Library/Containers/com.me.MyApp/Data/tmp/CoordinatedZipFilejxc2lC/Example.jpg, NSFilePath=/Users/SadPanda/Pictures/Example.jpg, NSUnderlyingError=0x600000ad0cf0 {Error Domain=NSPOSIXErrorDomain Code=1 "Operation not permitted"}}

There’s a variety of ways to run afoul of this. Here’s a particular one, for illustration.

Noticing when drag & drop operations start

Say you want to have a drop zone in your app that draws attention to itself whenever the user starts dragging a relevant file. Merely intelligent, courteous UI.

There’s no built-in facility for this – both AppKit and SwiftUI drag & drop APIs only start functioning once the dragged item enters the drop zone itself (the view, or at best window, that you’ve made a drop receiver).

There is a convention on how to work around this – you can deduce that a drag is occurring by watching mouse events and the drag pasteboard.

Deducing what’s going on by spying on global mouse events

When you see a drag event you don’t immediately know anything more than that the mouse moved with the left mouse button held down. That can mean anything – the user might be pulling out a selection rectangle, or drawing in a graphics application, or just randomly dragging the mouse around.

Since the contents of the drag pasteboard are left there indefinitely after a drag concludes, you can’t just use the existence of something on the drag pasteboard as an indication that a drag is in progress.

However, pasteboards have a “change count”. What this is actually counting is ill-defined and seemingly not about the actual contents of the pasteboard, but in a nutshell it can be used to mean essentially that – the count might change even if the contents don’t change, but it seems it will always change if the contents do change. So, some false positives, but no false negatives, which is the important thing.

So, when you see a mouse drag event, you can look at the drag pasteboard’s change count and see if it changed since the last mouse drag event. If it did, that’s a pretty strong suggestion – although admittedly not a guarantee – that the user is performing a drag & drop operation.

The essential code is:

let dragPasteboard = NSPasteboard(name: .drag)
var lastDragPasteboardChangeCount: Int = dragPasteboard.changeCount

let mouseDragWatcher = NSEvent.addGlobalMonitorForEvents(matching: [.leftMouseDragged],
                                                         handler: { event in
    let currentChangeCount = dragPasteboard.changeCount

    if lastDragPasteboardChangeCount != currentChangeCount {
        // The user very likely just started dragging something.
    }

    lastDragPasteboardChangeCount = currentChangeCount
}

let mouseUpWatcher = NSEvent.addGlobalMonitorForEvents(matching: [.leftMouseUp],
                                                       handler: { event in
    // The drag ended.
}

And the above code works fine as presented. The problem arises if & when you start actually looking at the pasteboard during the drag…

You get only one look at the dragged files

Files, when dragged, are represented mainly as their paths (technically, NSURLs). That’s the “public.file-url” UTI (and the Apple URL pasteboard type for backwards compatibility). Though you’ll also see a bunch of other types reported if you ask the pasteboard what it’s carrying, e.g.:

Apple URL pasteboard type
CorePasteboardFlavorType 0x6675726C
NSFilenamesPboardType
com.apple.finder.node
dyn.ah62d4rv4gu8y6y4grf0gn5xbrzw1gydcr7u1e3cytf2gn
dyn.ah62d4rv4gu8yc6durvwwaznwmuuha2pxsvw0e55bsmwca7d3sbwu
public.file-url

Merely inspecting the UTIs in the pasteboard is fine – that doesn’t interfere with anything. So if all you care about is if any kind of file (or folder) is being dragged, you’re set. But if you want to only react to some types of files or folders, you need to know more.

If you ask for the URL – even without actually using it – you trigger some behind the scenes activity involving app sandboxing. This prevents the file being made accessible to your app if & when it actually is dropped into your app.

When things are working correctly, when a file is dragged and dropped onto a receptive view in your app a link to that file is created inside your own app’s container. It’s that link that you actually have access to – the original file cannot be accessed directly. That link seems to persist for a while – perhaps until your app is quit – so once you have it you’re set.

I don’t know why merely peeking at the file path (URL) prevents this link being created, but it does.

Sigh.

FB13520048.

Failed workarounds

Sadly the only meaningful existing documentation of this problem that Bing or Google can find is this one StackOverflow question. Which is unresolved – the claimed hacks & workarounds don’t actually work, at least not anymore.

Supposedly on some older versions of macOS you could create a bookmark of the file (presumably a security-scoped one) and that would implicitly secure your access to it. But that seems pretty clearly to no longer work – assuming it ever did – because you can’t create a bookmark for files you can’t read, and until the drop occurs you don’t have read access to the file.

Similarly, you can supposedly use startAccessingSecurityScopedResource to indefinitely lock in your access to the file, but this doesn’t apply if you don’t have access to it to begin with. So, again, not useful here.

Partial workaround

While I knew that using canReadObject(forClasses: [NSImage.self]) ran afoul of this, I happened to notice there’s a logically equivalent method on NSImage itself – NSImage.canInit(with:) – which I tried, and lo and behold it works! No sandboxing issues.

I disassembled it (thanks Hopper!) and saw that it’s basically just doing some UTI pre-checks – already known to be safe – to see if there’s a file URL in the pasteboard, and then it fetches that file URL using readObjects(forClasses: [NSURL.self]), the same method that if you or I call results in this bug surfacing. The difference is it uses a secret reading option key – “NSPasteboardURLReadingSecurityScopedFileURLsKey”! Apparently that’s the magic sauce which allows you to read the URL without triggering the bug. You still can’t actually access the file, but at least you can e.g. examine its file extension (which is essentially what NSImage.canInit(with:) does).

if let URLs = dragPasteboard.readObjects(forClasses: [NSURL.self],
                                         options: [NSPasteboard.ReadingOptionKey(rawValue: "NSPasteboardURLReadingSecurityScopedFileURLsKey") : kCFBooleanTrue]) {
    for u in URLs {
        // Go nuts!
    }
}

Incidentally, I noticed it also calls startAccessingSecurityScopedResource but in my testing it always gets a failure response from it too. So that apparently is irrelevant.

Addendum: Michael Tsai’s Blog

I was reading through articles in NetNewsWire, including a few of the latest from Michael Tsai, and for one in particular I thought, “this sounds really familiar for some reason”. Hah, because he was quoting part of this post.

I’m very flattered to be linked to by Michael – I have no idea how he found my obscure website to begin with – as his blog is one of my favourite Apple developer news / discussion feeds. He does such a great job of collecting valuable links and collating them together on timely and/or valuable topics. His own commentary – if he adds any – is concise and valuable. If you’re not following his blog, you should rectify that. 🙂

I also now wish I’d written this post better. 😝

WP Dark Mode had ‘flashing’ problems, though – where the page loads first in normal, ‘light’ mode and only then re-renders into dark mode. Possibly just a result of interference by WP Fastest Cache, which I later realised was probably deferring WP Dark Mode’s JavaScript files past initial page load. Irrespective, I found that Dracula Dark Mode worked better – though still not flawlessly in this regard – so I switched to that.

However, I eventually noticed that Dracula Dark Mode adds 415 KiB to every single page load, with its JavaScript and CSS files. That doubles the total size for some pages, even with WordPress & Jetpack bloating the baseline. Its own analytics suggested that basically nobody uses dark mode, at least amongst people that browse my website. So that does not seem like a reasonable trade-off.

Thus, with a bit of sadness, I’ve disabled the plug-in. Maybe in future the plug-in’s authors can find a way to optimise things such that its JS & CSS is only loaded if dark mode is actually in use.

Turns out, though, that almost no websites support dark mode anyway – I discovered as much while having dark mode enabled during my own testing. So it doesn’t seem like I’m an odd one out, and my tiny website is not going to start any trends. From the looks of it the way most people actually tackle the web in dark mode is through browser plug-ins which basically do the same thing as the WordPress plug-ins, but for every website. And only users actually interested in dark mode have to pay their price.

I haven’t tried any of them, but there’s numerous Safari plug-ins for dark mode, e.g. Dark Reader, Noir, Dark Mode for Safari, etc.

It does mean I can’t do image substitutions for dark mode users, but I couldn’t find any dark mode WordPress plug-in that works correctly in that regard anyway (they seem to do naive URL substitutions, without accounting for images having various replicas of different sizes per WordPress standard practice, and certainly without supporting Perfect Images). It’s also rare that I have a dark mode version of an image available – though I can at least be better about using actual alpha channels rather than fixed white backgrounds, going forward.

★☆☆☆☆

I love the idea, but unsurprisingly – given it hasn’t been updated in over seven years – this no longer works. With my site, running WordPress 6.4.2, it broke it so badly that I couldn’t even login in recovery mode. I had to SFTP in and manually delete the plug-in to get my site back up.

It was immediately removed with the statement:

I have removed your review. Please do not use the review section for support.

Anonymous WordPress.org moderator

I think merely noting that an abandoned, seven-years-out-of-date plug-in is broken – and completely breaks WordPress if it’s installed & activated – is not unreasonable to note on the plug-in’s page. Putting that information in the Support forums is pointless since (a) nobody’s going to check those before installing the plug-in, and (b) it’s abandoned; there is no support.

Quite a disappointing move by the WordPress.org folks. 😕

Curiously, although the review itself was removed, it seems to still be counted in the stars histogram. Which is ironic since the 1-star rating is really not the important aspect of the review.

Addendum

Another review of mine was just removed by a different moderator for basically the same purported reason:

…you are 100% doing it wrong. Do not use a review to get support again.

Jan Dembowski

Seems a bit aggressive. I was, of course, not “getting support”, I was merely noting that another old, abandoned plug-in (LH Wayback Machine) only partially works now.

The first instance of this was disappointing but conceivably random, but this second removal suggests a systematic problem with WordPress.org’s review system.

So I didn’t buy an M1 Ultra.

The M2 was announced in July 2022. The M2 Ultra wasn’t announced for another eleven months, in June 2023, and offered only a small performance gain over its predecessor. There were rumours that the M3 was right around the corner (and it was, released just four months later). Worse, the M3 Pro and Max were released simultaneous to the base M3, and the M3 Max actually outperforms the M2 Ultra in many workloads.

So I didn’t buy an M2 Ultra.

So now the question is: when will the M3 Ultra come out? If it takes eleven to sixteen months, like its predecessors, then we’ll be expecting the M4 by then, and this sad cycle will have repeated yet again.

Will I buy an M3 Ultra?

I hope so, but it’s up to Apple.

e.g. I thought I’d give it a whirl as I was flying out of Houston, out over the Gulf of Mexico. Alas all I got was a blunt denial:

So don’t actually travel to remote places and expect this feature to work. 😔

A major reason I purchased an Apple Watch Ultra was for its loudly advertised ability to function as a dive computer, much like some Garmins.

It’s been a rough and disappointing road.

Right out of the gate, it didn’t work. It requires a 3rd-party application, Oceanic+, which didn’t exist at Apple Watch Ultra release time. It was over two months before Oceanic+ was finally released, on November 28th.

As it happens, my dive trip plans were delayed a bit anyway, resulting in my first dive [since purchasing the Apple Watch Ultra] being on that exact day. Hallelujah, I thought. I was able to hastily install the app in the morning, before leaving for a remote, internet-less island for a week.

But then there was the surprise that it basically requires an expensive subscription ($80/year, or even more if you commit to less than a whole year). Without it you don’t get tissue load and NDL tracking – critical functions of a dive computer. I was not aware before purchase, from any of the Apple Watch Ultra advertising or product pages on Apple’s website, that this subscription would be required. It feels surprisingly shifty and dishonest from a company like Apple.

Arguably the above frustrations (and costs) could be overlooked if it actually worked well. Unfortunately, it does not.

I discovered serious flaws with Oceanic+ right from the first dive. Flaws that any qualified diver would immediately recognise, which begs the question of why Apple & Oceanic+ somehow haven’t.

There are two major design flaws in its most basic function, the recording of dives:

1. It doesn’t start recording automatically.
   
   You have to hit the “Action” button to acknowledge a lawyer-smelling disclaimer that you’re “fit to dive”. If you forget, it doesn’t record.
   
   I’m not aware of any other dive computer that does this. e.g. my Shearwater Peregrine autodetects submersion below one metre even if it’s not on. It turns itself on and starts recording automatically.
2. If you ascend above one metre, even just for a split second, it immediately ends recording.
   
   This makes shore entries, in particular, likely to go unrecorded (unless you’re able to swim out to deep water and descend rapidly, which isn’t always an option or the best dive plan). Descents in significant swell, currents, or surge could also fall victim to this design flaw.
   
   So on my first dive I got five “dives” recorded, representing the four times where the swell floated me up to just above one metre (I never actually broke the surface). Even just reaching up momentarily with your watch hand, such as to grab a line or brush away a fin, could trigger it to fail.
   
   Furthermore, there’s no way to merge these together in Oceanic+ – you can either keep them, messing up your dive counts and stats, or delete them, throwing away [parts of] actual dive records.
   
   And on later dives I didn’t always notice it had failed and stopped recording, so it basically didn’t record the dive at all. This will be less of an issue if you’re using it as your only computer, since you’ll be looking at it periodically throughout your dive (I was using a separate dive computer as my primary, since I wisely didn’t trust the Apple Watch Ultra untested).
   
   Again, I’m not aware of any other dive computer that has this flaw. e.g. my Peregrine allows a sixty second grace period before ending the dive (configurable for up to ten minutes).

These two combine to make it unsafe for diving. It might be better than nothing, or acceptable as a backup computer (as long as you’re religious about ensuring it’s recording all the time), but it’s arguably worse than no dive computer at all in that it provides a false sense of security – you might plan many dives in one day, relying on the Apple Watch Ultra to precisely track your tissue loading, but have it fail midway and leave you with a dangerous decision to make.

What makes this all the more frustrating is that there’s a lot of things to like about it otherwise:

• The Apple Watch Ultra screen is so much better than what you find on most dive watches – clear and readable, with relatively low glare, even in harsh daylight. Not to mention that it’s a touch screen, so [when dry, on the surface] it’s faster to change your settings, review your dive log, etc.
• The Apple Watch Ultra is a lot smaller than most dive computers. Even those that are nominally intended to dual-purpose as watches.
• The information display during diving is well-designed and user-friendly (certainly not as powerful as what you can get on other dive computers, but quite sufficient for basic recreational diving).

• The iPhone integration is smoother and easier than with other dive computers – dives appear on your phone automatically (albeit sometimes after a short delay).
• It records the geographic location of the dive, which many dive computers do not.
• Its configurable alerts – e.g. for depth, dive duration, etc – are nice and clear when they trigger underwater, with a clear and prominent visual display and strong vibration. I find that its alert vibration is much more likely to actually get my attention than that of my Peregrine.

It feels like it’s actually close to being a pretty good dive computer & companion app, if not for a handful of bizarrely obvious, serious flaws.

It feels, in fact, like it was very rushed – from the obviously daft design flaws noted above, to even just simple things like bad grammar, poor alignment, and broken layout in the GUI (it smells like they’re using SwiftUI and haven’t figure out how to work around all its layout problems). It seems they put more time into their slick website than their actual product.

And of course there’s the other hint that maybe real-world testing was skipped – the fact that the iOS Oceanic+ app crashes on launch if you don’t have a good internet connection. On a boat, far from land? Or a remote island? Or in Airplane mode? Or just in an area with poor internet connectivity? No app for you. Crash on launch, every time. Forget about entering your dive details into the log while you actually remember them.

This remains the case even after six app updates over a month. Apparently Oceanic+ either don’t care that their app usually crashes on launch, or are incapable of fixing it.

Maybe there’s hope that in time they’ll be able to straighten all this out. But until then, I cannot in good conscience recommend the Apple Watch Ultra for diving. (it remains a fantastic watch for health-tracking and hiking, though)

For completeness, a list of other miscellaneous flaws and limitations:

• Any time you ascend past six metres, it throws up an alert about a safety stop. Which keeps buzzing at you forever, until you hit the action button. It’s hard to overstate how annoying this is when doing shallow dives. It is super distracting and may put you in harms way (e.g. if you’re constantly having to fiddle with the Apple Watch Ultra instead of paying attention to the reef around you).
  
  I simply can’t fathom why they feel the need to alert for this at all. An alert would be warranted for skipping your safety stop, yet it doesn’t do that.
• It vibrates the Apple Watch Ultra frequently without any indication why (nothing changes on the display). Observationally, I suspect it’s something to do with ascending “too fast”, but if so then it’s way too sensitive to small depth changes – it vibrates at me when simply ascending less than a metre (even when tens of metres deep, where a metre makes very little difference in pressure).
  
  Overall, the Apple Watch Ultra is too chatty. It’s a classic boy-who-cried wolf problem waiting to happen.
• It doesn’t show CNS, current PPO₂, [surface] GF, etc. Especially when using enriched air (Nitrox), nearing no-decompression limits, or deep diving, these are important for safety. They are purely software features so it’s especially odd that they’re not included.
  
  Surface GF is arguably the most important of these – it basically tells you how dangerous it is to surface immediately. When everything goes smoothly – and assuming you’re planning safe, conservative dives – you don’t need to worry much about it. But when things go awry it can be critical in helping you make the right decision under pressure.
• There’s no Oceanic+ Mac app, or even a web version, which strongly discourages actually using Oceanic+ as your full dive log. Entering all the details of your dive – gear, notes, etc – is very tedious on an iPhone.
  
  Some other dive computer manufacturers do have Mac apps (e.g. Shearwater, albeit one that’s clearly made in some janky cross-platform framework and looks like something you’d find in X11 in the 90s), and there are a few viable 3rd party options (e.g. MacDive, Subsurface).
• The map it shows, of your entry & exit points, is useless most of the time, because it just shows as flat blue (for water) with no identifying geography. There’s no way to switch it to anything useful, like a satellite view that would actually reveal the reefs, sand bars, atolls, etc.
  
  This is exacerbated by Apple Maps’ limitations. Google Maps, for example, often does show atolls & islands at least, and other surface features. Apple Maps simply lacks actual maps for most of the world’s oceans.

• The iOS Oceanic+ app lets you record what gear you were using, but bizarrely requires you to pick from a pre-defined list and that list is missing gear from major brands (e.g. Aqua Lung, Cressi). Another reason why it won’t be your real dive log.
• There’s no way to import or export dive data. This is both a lock-in concern – your dive data will be deleted if you end your subscription – and also a roadblock to using the Oceanic+ app as your real dive log, unless you have never and will never dive without an Apple Watch Ultra.
• You can’t change the activity type – dive vs snorkelling – underwater, nor after the fact. So if you forget to change it before going for a snorkel, you’ll forever have a bogus “dive” in your log (or you can delete the record entirely, but then you lose record of any free diving you do).
• There’s five entries in the main menu in the Oceanic+ watch app, which are arranged as a scrolling carousel… which is just weird and annoying since they’d fit all on one screen as simple buttons, which would make navigation much faster and easier.
• A lot of actions on the Oceanic+ watch app require more steps than seem necessary. e.g. changing numeric values requires not just selecting a different value but also tapping back to the previous screen (which also feels unnatural, like I’m backing out of the change without applying it). There are menus trees five or more levels deep, whereas it seems like they could be flattened into just two or three levels.

• Relatedly, you can’t edit any of your settings on your iPhone, only the watch. You can view the settings on the iPhone, which just makes it even more baffling why you can’t edit them there. Editing them on the watch is great in a pinch – you might not have your phone with you – but it’s a pain compared to on an iPhone.
• The Oceanic+ iPhone app “Home” screen – what’s displayed when you launch the app – just shows a handful of stats of dubious merit. Minimum temperature over the last four weeks? Who cares. Cumulative total max depth? That clearly has no purpose. It seems like they knew they needed to show the most basic numbers – total dive count and duration – and felt compelled to stuff in a bunch more numbers for some reason. It’s also unclear why they think these are more important than your actual dive logs, or the dive planner.

• There’s supposedly seven watch complications available once you have Oceanic+ installed, but on my watch only two are available (“Max altitude” and “Oceanic Launcher”). This might be because I use a digital time display (the “Modular” face), something Apple seems to hate.
• The battery life is surprisingly short – about five hours of dive time. Given that the Apple Watch Ultra can record workarounds that are 16 hours long (at least – longer if you use energy-saving features), it’s a bit of a mystery to me why it chews through the battery so fast while diving. It’s not even monitoring your heart rate or other health metrics – just water pressure – and the screen brightness tends to be low since you’re underwater in low light.

Here’s a few things which are more just wishlist items (or: things you can get from some other dive computers, though you usually pay a lot more for those than you do an Apple Watch Ultra):

• Gas usage recording. Upmarket dive computers support wireless communication with a dongle that attaches to your first stage regulator. This means you can forgo a whole separate hose and dangly, annoying air gauge, and have a unified view of your dive status. It also means you get more accurate tracking of tissue loading, and more advanced functionality like gas consumption rates (great for extra safety – know if you’re going to run out of air too early – and for optimising your gas consumption over time).
  
  Radio protocols like Bluetooth don’t work underwater – lower frequencies are required. So the Apple Watch Ultra would require an additional built-in antenna. It’s arguably reasonable to omit this in a watch that’s not intended solely for diving, given the cost or other trade-offs it might require.
  
  That said, some air-integrated dive computers use sound instead of electromagnetism, and I suspect it’s not hard to support the necessary, inaudible frequencies in the Apple Watch Ultra’s microphone(s). Maybe this support is already present in hardware, and a transmitter dongle will be released later?
• Multi-gas support. Most dive computers support this, even those that are much cheaper than an Apple Watch Ultra. This is arguably a more “serious” or “technical” feature, that most recreational divers won’t ever need, but it’s also easy to do – it’s purely a software feature.
• The ability to enter key dive details on the Apple Watch Ultra (as opposed to an iPhone), such as gas start & end pressures. It’s all too easy to forget these in-between the dive itself and when you get back to land and your iPhone.
• The dive planner functionality is pretty rudimentary. e.g. you can’t do anything like actually enter a dive depth profile (whether as a squiggle with your finger, to give the rough idea, or importing it from a previous dive at the same site). The GUI is also a bit obtuse, especially in the Oceanic+ iOS app, as rather than showing a simple table or chart of depth vs no-deco times it makes you pick a single depth and gives you a single no-deco time. For planning you often want to consider multiple depth options and pick the right trade-off against dive (or at least bottom) duration.
• It might actually be nice to have some “social” functionality. Not for bragging and other vanity purposes – I’m not talking about inane integrations with Facebook or whatever – but for sharing amongst dive buddies and the like. I suspect there’s some neat, innovative possibilities here (e.g. automatically detect physically- & temporally-nearby dive friends, and be able to automagically see their photos & notes on what they saw on what’s presumably the same dive with you – maybe even add your own notes on theirs like “don’t forget about that Whale Shark eating that Orca!”).

And lastly just some miscellanea:

• Apple / Oceanic say that it won’t record below forty metres, but it does. Thankfully – the last thing you want if you do a deep dive, intentionally or especially unintentionally, is for your dive computer to not record your dive profile correctly.

I don’t regret getting my Apple Watch Ultra – it’s proven a worthwhile upgrade even just for its other features like battery life and a relatively large screen – but I am sad that I can’t actually rely on it for diving. And that I had to spend a lot of extra money to get a dive computer – the Peregrine – that I can rely on.

For reference, I’ve completed about fifty dives with the Apple Watch Ultra.

Addendum (May 2023)

I’ve now had the Apple Watch Ultra accompany me on about 150 dives. Sadly, despite it being a long six months later, little has changed. The Oceanic+ app is still awkward and very rudimentary, with the same data and platform lock-in problems. The Apple Watch component is almost unchanged – same limitations and GUI frustrations.

Two things which did improve at some point:

• Recording now happens when submerged even if you haven’t clicked through the lawyer screen. This is a significant safety improvement.
  
  However, if you go the entire dive and return to the surface without clicking through the lawyer screen, the recording is discarded. So there’s still danger here. You just get (a lot) more time to realise the watch is being obstinate (and you really should be looking at your Apple Watch Ultra at least once during the whole dive anyway, even if it’s just your backup, to ensure it’s working and to check its data against your primary).
• The Oceanic+ app seems to have fixed many of the glaring GUI bugs – e.g. the charts that rendered in the wrong places on the screen (or not at all), inconsistent font sizes and baselines, etc.

I haven’t systematically re-reviewed the Oceanic+ app, so perhaps there’s been additional fixes or improvements too. I’ve barely used it since my initial review, since I can only rely on my primary dive computer (Shearwater Peregrine) anyway. For what it’s worth, I use MacDive on my iPhone & Mac and am reasonably happy with it.

I wish I’d just bought the Shearwater Teric, though – the Peregrine was me hedging my bets and hoping that the Apple Watch Ultra would prove sufficient alone. The Peregrine is good but the Teric is much nicer (and for clear reason by far the most popular dive computer on the six boats I’ve dived from).

In short, the Mojave update installer does not work (on High Sierra at least) if you have ‘Optimize Mac Storage’ enabled for iCloud Drive (System Preferences > iCloud pane > iCloud Drive Options… button > Documents tab > Optimize Mac Storage checkbox).

Specifically, the installer reports:

Installation requires downloading important content. That content can’t be downloaded at this time. Try again later.

…and indeed fails to download the actual Mojave update files (the installer app as ‘installed’ via the App Store is merely a 22 MB bootstrapping app, that downloads the actual image only after you run it & start the installation).

Even more obnoxiously, if you use the dosdude1 Mojave Patcher Tool to force-download the entire installer, as soon as it completes the 6.5 GB download and produces the ‘Install macOS Mojave’ app in /Applications, the system deletes the downloaded installation files out from under that app, rendering it just as broken as the official App Store version. Infuriating.

Aside: to be clear, turning off ‘Optimize Mac Storage’ enabled me to produce – and keep – a working installer as downloaded by dosdude1’s tool. I did not verify that it also fixes the regular installer as downloaded via the App Store.

I also ran into the “The recovery server could not be contacted” error message even before all the above, but thankfully that was fixable via the means normally prescribed online – running “sudo ntpdate -u time.apple.com”.