The following classes and functions are generally not thread-safe. In most cases, you can use these classes from any thread as long as you use them from only one thread at a time. Check the class documentation for additional details.

Apple’s Threading Programming Guide > Appendix A: Thread Safety Summary, subsection Application Kit Framework Thread Safety

What “in most cases” means is left to the reader’s imagination. Apple adds a little addendum for NSImage specifically:

One thread can create an NSImage object, draw to the image buffer, and pass it off to the main thread for drawing. The underlying image cache is shared among all threads. For more information about images and how caching works, see Cocoa Drawing Guide.

NSImage Restrictions

For a start, it’s talking only about creating an NSImage from scratch, not loading it from serialised form (e.g. a file, a pasteboard, etc). It doesn’t even deign to mention those other, much more common cases.

And even for that one mentioned use case, what does it mean, exactly? What “image cache” is it referring to?

I don’t have authoritative answers. The documentation is so infuriatingly vague that Apple could do basically anything to the implementation, between macOS updates, and claim to have broken no promises.

What I do have is some empirical data and the results of some reverse engineering (shout out to Hopper), from macOS 14.2 Sonoma.

NSImage 101

NSImages can represent a wide range of imagery. Most uses of them are probably for bitmap data (i.e. what you find in common image formats like WebP & AVIF), but NSImage also supports ‘raw’ images (e.g. Nikon NEF) as well as vector data (e.g. SVG & PDF). It also has a plug-in mechanism of sorts, so the supported image formats can be extended dynamically at runtime.

You can fetch the full list of supported types from NSImage.imageTypes – although it doesn’t distinguish between those it can read vs those it can write. Fortunately, sips --formats in Terminal gives you the same list with additional metadata. On my machine that list happens to be:

Supported Formats:
-------------------------------------------
com.adobe.pdf                pdf   Writable
com.adobe.photoshop-image    psd   Writable
com.adobe.raw-image          dng   
com.apple.atx                --    Writable
com.apple.icns               icns  Writable
com.apple.pict               pict  
com.canon.cr2-raw-image      cr2   
com.canon.cr3-raw-image      cr3   
com.canon.crw-raw-image      crw   
com.canon.tif-raw-image      tif   
com.compuserve.gif           gif   Writable
com.dxo.raw-image            dxo   
com.epson.raw-image          erf   
com.fuji.raw-image           raf   
com.hasselblad.3fr-raw-image 3fr   
com.hasselblad.fff-raw-image fff   
com.ilm.openexr-image        exr   Writable
com.kodak.raw-image          dcr   
com.konicaminolta.raw-image  mrw   
com.leafamerica.raw-image    mos   
com.leica.raw-image          raw   
com.leica.rwl-raw-image      rwl   
com.microsoft.bmp            bmp   Writable
com.microsoft.cur            --    
com.microsoft.dds            dds   Writable
com.microsoft.ico            ico   Writable
com.nikon.nrw-raw-image      nrw   
com.nikon.raw-image          nef   
com.olympus.or-raw-image     orf   
com.olympus.raw-image        orf   
com.olympus.sr-raw-image     orf   
com.panasonic.raw-image      raw   
com.panasonic.rw2-raw-image  rw2   
com.pentax.raw-image         pef   
com.phaseone.raw-image       iiq   
com.samsung.raw-image        srw   
com.sgi.sgi-image            sgi   
com.sony.arw-raw-image       arw   
com.sony.raw-image           srf   
com.sony.sr2-raw-image       sr2   
com.truevision.tga-image     tga   Writable
org.khronos.astc             astc  Writable
org.khronos.ktx              ktx   Writable
org.khronos.ktx2             --    Writable
org.webmproject.webp         webp  
public.avci                  avci  
public.avif                  avif  
public.avis                  --    
public.heic                  heic  Writable
public.heics                 heics Writable
public.heif                  heif  
public.jpeg                  jpeg  Writable
public.jpeg-2000             jp2   Writable
public.jpeg-xl               jxl   
public.mpo-image             mpo   
public.pbm                   pbm   Writable
public.png                   png   Writable
public.pvr                   pvr   Writable
public.radiance              pic   
public.svg-image             svg   
public.tiff                  tiff  Writable

An NSImage can have multiple ‘representations‘. These are essentially just different versions of the image. One representation might be the original vector form (e.g. an NSPDFImageRep). Another might be a rasterisation of that at a certain resolution. Yet another might be a rasterisation at a different resolution.

It gets more complicated, however, because some representations are merely proxies for other frameworks’ representations, like CGImage, CIImage or somesuch.

Nonetheless, for the simple case of a static bitmap image read from a file, generally NSImage produces just one representation, which is the full bitmap (as an NSBitmapImageRep). That’s the only case I’m going to discuss here, for simplicity’s sake (though likely the lessons apply to the other cases too).

Accessing / using an NSImage

Generally to use an NSImage you need a bitmap representation. e.g. to actually draw it to the screen.

Unless you manually create an NSImage from an in-memory bitmap, the bitmap representation is not loaded initially, but rather only when first needed.

Ultimately the load is triggered when something asks for the raw bytes of the bitmap (either you, directly in your code, or indirectly when you e.g. ask AppKit to draw the image).

Fetching the raw bytes of a bitmap from NSBitmapImageRep is not a trivial exercise. I’m going to gloss over the complexities (like planar data formats) and just talk about the common case of single-plane (“interleaved channels”) bitmaps.

For those, you can access the raw bytes via the bitmapData property.

Nominally, bitmapData just returns a uint8_t*.

In fact, bitmapData calls a lot of internal methods in a complicated fashion, with many possible code paths. What exactly it does depends on the underlying source of data, but in a nutshell it checks if the desired data has already been created / loaded (it’s cached in an instance variable) and if not it loads it, e.g. by calling CGImageSourceCreateImageAtIndex.

That’s what makes NSImage dangerous to use from multiple threads simultaneously.

Reading the cached bitmap data is essentially read-only (just some retain/autorelease traffic) so it’s safe to call from multiple threads concurrently (as long as something ensures the NSBitmapImageRep is kept alive the whole time and not mutated)

But loading or modifying it is not. As such, if you call bitmapData from multiple threads concurrently, and you don’t know for sure that it’s already fully loaded, you get a data race (also known as a “WTF does my app crash randomly?!” condition).

The consequences of that race vary. Maybe you “win” the race – one thread happens to run virtually to completion of bitmapData first, storing the fresh backing data into the caching instance member, and then all the other threads run and just return that same value – the ideal situation as everything works as intended.

Maybe you “lose” the race: every concurrent thread checks simultaneously and sees there’s no cached value, so they all – in parallel and redundant to each other – load the bitmap data and store it into the cache. They each return the one they created, even though ultimately only one thread wins – the last one to write into the cache – and the duplicate bitmap data that all the earlier threads created is deallocated. Even though pointers to them have been returned to you. And you might be in the middle of using them. Causing you to crash with a memory protection fault (or worse, read from some random other memory allocation that happened to be placed at the same address afterwards, reading essentially garbage).

Hypocritically, this is because Apple don’t follow their own rules on escaping pointers:

The pointer you pass to the function is only guaranteed to be valid for the duration of the function call. Do not persist the pointer and access it after the function has returned.

Apple’s Swift Standard Library > Manual Memory Management > Calling Functions With Pointer Parameters, subsection Pass a Constant Pointer as a Parameter.

The source code for the relevant NSBitmapImageRep methods is essentially:

- (uint8_t*)bitmapData {
    uint8_t **result = nil;
    [self getBitmapDataPlanes:&result];
    return *result;
}

- (void)getBitmapDataPlanes:(uint8_t***)output {
    [self _performBlockUsingBackingMutableData:^void (uint8_t* dataPlanes[5]) {
        *output = dataPlanes;
    }];
}

🤦‍♂️

Unfortunately, while _performBlockUsingBackingMutableData and its many similar siblings can be made safe, they are (a) all private and (b) not currently enforcing the necessary mutual exclusion. This could be corrected by Apple in future (although I wouldn’t hold your breath).

Why does it matter that NSImage is not thread-safe?

The crux of the problem is that you often have no good choice about it, because NSImage is a currency type used widely throughout Apple’s own frameworks, and you often have no control over what thread it’s created or used on.

For example, even using the very latest Apple APIs such as SwiftUI’s dropDestination(for:action:isTargeted:), you cannot control which thread the NSImages are created on (it appears to be the main thread, although that API provides no guarantees).

Similarly you have no control over where those images are used if you pass them to 3rd party code – including Apple’s. e.g. Image(nsImage:); possibly that only uses them on the main thread, but it might be pre-rendered the image a separate thread for better performance (to avoid blocking the main thread and causing the app to hang). In fact it should, in principle.

Loading an image – actually reading it from a file or URL and decompressing it into a raw bitmap suitable for drawing to the screen – should never be done on the main thread, because it can take a long time. A 1 GiB TIFF takes nearly 30 seconds on my iMac Pro, for example (and TIFF uses very lightweight compression, so it’s a relatively fast-to-read format). Anything involving the network could take an unbounded amount of time. Even small files – like a 20 MiB NEF – can take seconds to render because they are non-trivial to decode and/or decompress.

So you’re screwed on multiple levels: not only can you generally not guarantee what thread NSImage is born on nor used on, you can’t use it exclusively on the main thread because that will cause a terrible user experience.

What can you do?

In simple terms, the best you can realistically do is try ensure that all modifications to the NSImage (including implicit ones, such as loading a bitmap representation upon first use) happen exclusively in one thread [at a time]. For example, in my experiments, putting a lock around otherwise concurrent calls to bitmapData is enough to prevent any data races. Although I am mystified as to how I can still let the main thread draw the image concurrently without any apparent problems. 🤔

If you want to play it really safe, you have to create & pre-load each NSImage on a single thread (not the main thread), before ever sending it to another isolation domain. That means manually reimplementing things like drag and drop of images (because you can no longer work directly with NSImage with any drag & drop APIs, you have to instead use only – and all – the things that could potentially be images, like URLs or data blobs, and then translate those to & from NSImages manually).

Complicating all this is that NSImage is unclear about how its caching works. For starters, does this caching apply to the representations or is it a hidden, orthogonal system? Is it thread-safe? Etc.

You can supposedly modify the caching behaviour via the cacheMode property, but in my experience there is no apparent effect no matter what it is set it to (not on the image’s representations nor on render performance in the GUI).

It’s a shame that NSImage has been so neglected, and has so many glaring problems. Over the years Apple have seemingly tried to replace it, introducing new image types over and over again, but all that’s done is made everything more complicated.

You could punt the problem to some other code, outside the actor – i.e. make something else periodically call some method on the actor. Sometimes this is a suitable approach, but sometimes you just want your actor to be self-contained and not require external aid. And if the work takes time anyway (e.g. network activity) what’s the point of having some external entity calling in and having to wait on your actor – easier if your actor just notifies the external entity whenever new data is available.

Timer ❌

You might think to just use Timer (formerly NSTimer) – and most of the internet would strongly encourage you to do so, based on the search results you’ll find for this topic.

Except it doesn’t work.

Timer relies on RunLoop (NSRunLoop) to actually schedule itself and be executed. RunLoops are normally created automatically in most cases and kind of just work magically in the background.

Actors don’t have runloops. More accurately – the magical, mutually-exclusive context in which an actor runs is not a RunLoop and does not interact with RunLoops. Actors use a relatively new Executor class family, but at least for now (2022) those are basically useless – literally the only method they have is to add a task to be executed as soon as possible. They provide no way to schedule a task after a delay.

Confusingly, if you access RunLoop.current from an actor context, it will return some non-nil value, but it’s useless because nothing ever runs that particular RunLoop. And how would you run it yourself – calling its run method from within the actor’s context would block the actor indefinitely and cause deadlock.

You could use the main runloop that most Swift apps have by default, but you shouldn’t – that main runloop is intended for user interaction only. You should never put random background tasks on it, as they can interfere with your UI and make your app stutter.

You could spawn your own Thread to actually run a RunLoop, but it’s unsafe – you cannot interact with any runloop except the current runloop – i.e. you have to be in a runloop in order to touch it – and you can’t get into such a runloop in an easy way from an actor context. Seemingly obvious and innocuous methods like calling RunLoop.current from actor context are dangerous because they’re not guaranteed to return the same runloop each time. Instead, you have to explicitly bridge to the runloop via some bespoke message system that you have to pre-install into that runloop. Ick.

It’s also very inefficient, if you don’t actually need to use a whole CPU core most of the time – you’ll need a dedicated thread for every actor instance, and threads aren’t completely free to create & maintain – plus there’s a limit to how many you can have alive at one time.

Dispatch ✔️

A better approach is to use Dispatch (formerly Grand Central Dispatch) – specifically the convenient methods on DispatchQueue that allow you to set up delayed and/or scheduled tasks. There’s the various async… methods for doing something once after a certain time period, and also schedule… methods which also support recurring, cancellable timers. Granted they aren’t tied to the actor’s context – you still have to bridge back into the actor via Task or similar – but at least they actually work.

e.g.:

let timer = DispatchQueue
    .global(qos: .utility)
    .schedule(after: DispatchQueue.SchedulerTimeType(.now()),
              interval: .seconds(refreshInterval),
              tolerance: .seconds(refreshInterval / 5)) { [weak self] in
    guard let self else { return }
    Task { await self.getLatestMeasurement() } // Trampoline back into the actor context.
}

// Keep "timer" around as e.g. a member variable if you want to be able to cancel it.

If you use the schedule… methods you may need to import Combine, as the Cancellable type that they return is actually defined in Combine. But you don’t have to actually use Combine otherwise.

Structured Concurrency (Task) ✔️

An alternative approach is to just use Swift Concurrency primitives. This is arguably “purer” – no need to pull in the Dispatch & Combine libraries – but requires a bit more manual labour if you need to support cancellation.

Basically you can create a new Task which just sits in a loop running the desired operation endlessly (or until cancelled), manually sleeping between executions.

e.g.:

self.timer = Task(priority: .utility) { [weak self] in
    while !Task.isCancelled {
          guard let self else { return }
          await self.getLatestMeasurement()
          try? await Task.sleep(nanoseconds: UInt64(refreshInterval * 1_000_000_000))
    }
}

It doesn’t look too bad, but keep in mind this loses some important functionality vs the Dispatch approach – there’s no way to specify a tolerance to the timer, so this is not as good for energy efficiency (in cases where you don’t need exact timing, which is most cases).

There is a new variant of the Task.sleep method potentially coming in iOS 16, macOS 13, etc which does allow you to specify a tolerance, but that of course requires a minimum deployment target of essentially 2023 or later.