NSCopyObject, the griefer that keeps on griefing

NSCopyObject is a very old Foundation function – pre-dating Mac OS X entirely; from the NeXT era – that was originally basically just memcpy, but now it’s complicated. A lot more complicated.

What NSCopyObject does

Its implementation currently starts with essentially:

id NSCopyObject(id object, NSUInteger extraBytes, NSZone *zone) {
    if (nil == object) {
        return nil;
    }
    
    id copy = object_copy(object, extraBytes);
    object_setClass(copy, objc_opt_class(object));
    return copy;
}

…where object_copy et al are part of the Objective-C runtime. object_copy and its callees are not trivial, so I won’t repeat them here. The key parts are:

1. The malloc in _class_createInstance (allocate space for the copy).
2. The memmove in object_copy (naively copy the raw bytes over).
3. The call from object_copy to fixupCopiedIvars (half-heartedly attempt to fix the damage).

fixupCopiedIvars is notable. It was added by necessity when ARC was introduced to the Objective-C runtime, in Mac OS X 10.6 (Snow Leopard) in 2006. ARC added metadata to Objective-C classes to convey which instance variables were retain-counted object references, so that it could manage them automagically at runtime (not just for copying objects, but more importantly for deallocating them). fixupCopiedIvars uses that metadata to identify things it has to retain (strongly or weakly) in the new copy.

So that should work great, right? The copy operation increments the retain count of all shared objects the new copy references, like you’d expect?

That metadata is incomplete. It only works for Objective-C ivars managed by ARC. i.e. not C++ ivars or Swift stored properties, nor even Objective-C ivars that aren’t using ARC¹.

But I don’t use NSCopyObject…?

Almost nobody intentionally uses NSCopyObject, but your superclass might, and therefore you might. Ever subclassed NSCell or NSAnimation, for example?

I happened to hit this when subclassing NSBitmapImageRep (and I’m very grateful to Kyle Sluder for so quickly identifying the problem – it could have taken me forever to figure it out, otherwise).

If your superclass uses NSCopyObject, it’s now your problem just as much as if you’d used NSCopyObject directly, whether you like it or not.

And even more problematically, whether you know it or not. If your superclass is defined by a 3rd party framework / library, or anything that’s closed source, you might have no idea whether it uses NSCopyObject currently. Worse, you have no control over whether it will or will not use it in future (though anyone that adds a use of NSCopyObject at this point had better hope the atheists are right).

So how do I defend against NSCopyObject?

Objective-C

Pre-ARC it used to be relatively easy to work around this, in Objective-C. You “just” had to manually retain all your subclasses’ reference ivars – and manually copy some others, like non-ref-counted mutable or mortal buffers, etc.

But that generally isn’t possible with ARC – under which you cannot explicitly call retain. Worse:

• There’s still prominent guides scattered about the web that push you unequivocally to use retain, which is not just impossible to do directly under ARC, but flat-out wrong even if you do figure out one of the “clever” ways to do it (you’ll end up over-retaining your ARC-managed references, causing memory leaks).
• There’s also pages lingering on the web that claim that merely turning on ARC will magically solve the problem (it might, but it’s not a panacea).

Some guides specify a better method, which is to manually zero out the copied object’s ivars and then repopulate them via formal property setters. That actually works with or without ARC, although it may break – causing memory leaks – if the superclass ever stops using NSCopyObject (or if NSCopyObject ever gets upgraded to understand reference-counted ivars that it currently does not). It’s also only possible in Objective-C because Swift doesn’t provide direct access to instance variables.

Keep in mind that any reference-typed ivars which are not strong or weak Objective-C objects managed by ARC will still, always need to be handled manually. e.g. pointers to manually-managed memory buffers.

Swift

Ironically, Swift’s attempts to prevent incorrect code actually make it harder to write correct code in this case. What you want to do is – like the Objective-C implementation – to just zero out the references and re-assign them like normal properties. Zeroing them out without triggering a release basically undoes the mistaken memcpy that NSCopyObject did. But Swift won’t let you.

Worse, this has been known for most of Swift’s existence and nothing has been done about it.

Simply setting the property to nil will cause it to be erroneously released, which may immediately deallocate the object and ultimately cause a crash or memory corruption. Even if it doesn’t happen to deallocate the object, it’ll negate the retain you do during the assignment, making all your effort moot.

That’s pretty heavy-handed, though, and not always possible (e.g. NSImageRep, as used by e.g. NSBitmapImageRep, does some special magic in its copy implementation which you cannot practically replicate).

It appears that the best you can do is assume the superclass will always use NSCopyObject, if it does currently, and just manually increment the retain count. Like Objective-C with ARC, the language & standard library really don’t want you to actually do this, but at least in Swift it’s relatively straightforward:

override func copy(with zone: NSZone? = nil) -> Any {
    let result = super.copy(with: zone)
    
    if result.myProperty === self.myProperty {
        _ = Unmanaged.passRetained(myProperty)
    } else {
        result.myProperty = self.myProperty
    }
}

The conditional might help protect you if the superclass stops using NSCopyObject in future – in that case, it’ll probably cause myProperty to default to nil (or to be assigned to some other instance, which you can discard), in which case you just want to assign to it normally.

In the interim – while NSCopyObject is in use, at least – the myProperty pointer will be copied verbatim and you have to assume it requires the extra, manual retain. It’s not future-proof – it’s possible for the superclass to copy the pointer verbatim and increment the retain count for you – but at least in that case you “merely” get a memory leak, rather than a crash or memory corruption.

Do as Apple says, not as Apple does

The most frustrating part of all of this is that this is entirely Apple’s fault. Sure, you can argue it’s not their fault that NeXT added this vile function to Foundation; that Apple “merely” inherited it and were “forced” to keep for backwards compatibility. But it’s entirely Apple’s choice to have kept using it all this time, in their core frameworks, even while they’ve been telling everyone else to never use it.

NSCopyObject has been a known problem-maker pretty much forever – it was a terrible idea right from the outset. Blindly copying the bytes of an object instance, and just hoping that somehow that works correctly – in an object-oriented language derived from Smalltalk where even numbers are often reference types – is farcical.

The introduction of ARC (in 2008) didn’t really help anything, as although it changed NSCopyObject to properly retain ARC-managed ivars, it did nothing for non-ARC-managed ivars (remember that ARC can be enabled in one library but not in another, and libraries can subclass each others’ classes).

NSCopyObject has been officially deprecated since 2012:

The NSCopyObject() function has been deprecated. It has always been a dangerous function to use, except in the implementation of copy methods², and only then with care.

Foundation Release Notes for OS X 10.8 Mountain Lion and iOS 6

…though Apple officially told everyone not to use it in 2008:

This function is dangerous and very difficult to use correctly. It’s [sic] use as part of -copyWithZone: by any class that can be subclassed, is highly error prone. This function is known to fail for objects with embedded retain count ivars, singletons, and C++ ivars, and other circumstances.

Foundation Release Notes for Mac OS X 10.6 Snow Leopard

And this was all still a decade or more after it was known that NSCopyObject was fundamentally evil, e.g. NSCell, and GnuStep’s broken NSControl.

And yet, Apple still use NSCopyObject themselves to this very day, in their own applications and frameworks – including major frameworks like AppKit that almost all 3rd party developers rely on. NSCell is still broken, three decades later, as is NSImage & NSImageRep, and NSAnimation. Most of those are explicitly designed to be subclassed, despite Apple’s own very clear instructions to never mix subclassing with NSCopyObject.

Admittedly it’s not trivial for Apple to remove the NSCopyObject use – alas, because people have had to code myriad hacky workarounds to it, Apple now has to be careful not to break those workarounds. That might even preclude fixing the existing code paths; it might require a replacement copy mechanism. Which leads to…

Tangent: NSCopying considered harmful

The big driver of NSCopyObject use has long been NSCopying. Classes that intend to be subclassed – but also semantically should support copying i.e. NSCopying – have long been making the mistake of thinking that means using NSCopyObject. One need only read the NSCopying documentation, even from before Mac OS X was even publicly released, to see how dangerously fragile and error-prone NSCopying has always been.

Compounding the problem is that NSCopying doesn’t work, by default, on subclasses. You have to override copy(with:) in every subclass³, but the compiler does not enforce this, because in Objective-C (and alas Swift) protocol conformance is assumed inherited even when it cannot correctly be without explicit, extra work by the subclass.

1. Yes, it’s still possible to this day to write Objective-C without using ARC – -fno-objc-arc / CLANG_ENABLE_OBJC_ARC. There might even be valid (albeit unfortunate) use-cases for having to do so, such as for performance.
   
   And even with ARC, it’s of course possible to have pointers to things which aren’t NSObjects and therefore aren’t handled by ARC, such as raw malloc allocations. ↩︎
2. This is false and always has been (that it was safe to use in copy methods). Apple’s false statements in the deprecation notices may ironically have caused even more instances of people using NSCopyObject. ↩︎
3. Any and all that add retain-counted ivars, Swift stored properties, or ivars of C++ types that have destructors. ↩︎