<?xml version="1.0" encoding="UTF-8"?><rss version="2.0"
	xmlns:content="http://purl.org/rss/1.0/modules/content/"
	xmlns:wfw="http://wellformedweb.org/CommentAPI/"
	xmlns:dc="http://purl.org/dc/elements/1.1/"
	xmlns:atom="http://www.w3.org/2005/Atom"
	xmlns:sy="http://purl.org/rss/1.0/modules/syndication/"
	xmlns:slash="http://purl.org/rss/1.0/modules/slash/"
	xmlns:media="http://search.yahoo.com/mrss/"
>

<channel>
	<title>Foundation &#8211; Wade Tregaskis</title>
	<atom:link href="https://wadetregaskis.com/tags/foundation/feed/" rel="self" type="application/rss+xml" />
	<link>https://wadetregaskis.com</link>
	<description></description>
	<lastBuildDate>Mon, 20 May 2024 03:37:59 +0000</lastBuildDate>
	<language>en-US</language>
	<sy:updatePeriod>
	hourly	</sy:updatePeriod>
	<sy:updateFrequency>
	1	</sy:updateFrequency>
	

<image>
	<url>https://wadetregaskis.com/wp-content/uploads/2016/03/Stitch-512x512-1-256x256.png</url>
	<title>Foundation &#8211; Wade Tregaskis</title>
	<link>https://wadetregaskis.com</link>
	<width>32</width>
	<height>32</height>
</image> 
<site xmlns="com-wordpress:feed-additions:1">226351702</site>	<item>
		<title>Copy-on-write on APFS</title>
		<link>https://wadetregaskis.com/copy-on-write-on-apfs/</link>
					<comments>https://wadetregaskis.com/copy-on-write-on-apfs/#respond</comments>
		
		<dc:creator><![CDATA[]]></dc:creator>
		<pubDate>Thu, 16 May 2024 21:14:10 +0000</pubDate>
				<category><![CDATA[Coding]]></category>
		<category><![CDATA[Education]]></category>
		<category><![CDATA[Howto]]></category>
		<category><![CDATA[APFS]]></category>
		<category><![CDATA[clonefile]]></category>
		<category><![CDATA[copy-on-write]]></category>
		<category><![CDATA[copyfile]]></category>
		<category><![CDATA[FileManager]]></category>
		<category><![CDATA[Foundation]]></category>
		<guid isPermaLink="false">https://wadetregaskis.com/?p=8138</guid>

					<description><![CDATA[APFS (like many modern file systems but unlike its predecessor HFS+) supports copy-on-write. This means you can logically copy a file &#8211; it looks and behaves like a distinct file &#8211; but it doesn&#8217;t immediately copy the file&#8217;s contents on disk &#8211; it merely shares them with the original. Only if and as you modify&#8230; <a class="read-more-link" href="https://wadetregaskis.com/copy-on-write-on-apfs/" data-wpel-link="internal">Read more</a>]]></description>
										<content:encoded><![CDATA[
<p><a href="https://en.wikipedia.org/wiki/Apple_File_System" data-wpel-link="external" target="_blank" rel="external noopener">APFS</a> (like many modern file systems but unlike its predecessor <a href="https://en.wikipedia.org/wiki/HFS_Plus" data-wpel-link="external" target="_blank" rel="external noopener">HFS+</a>) supports <a href="https://eclecticlight.co/2017/06/23/what-is-copy-on-write-and-how-is-it-good/" data-wpel-link="external" target="_blank" rel="external noopener">copy-on-write</a>.  This means you can <em>logically</em> copy a file &#8211; it <em>looks</em> and <em>behaves</em> like a distinct file &#8211; but it doesn&#8217;t <em>immediately</em> copy the file&#8217;s contents on disk &#8211; it merely shares them with the original.  Only if and as you modify either version do they start to diverge on disk, with APFS dynamically allocating new storage for the modified parts<sup data-fn="fd9542d5-ca23-49bc-ae00-3d2b57caf906" class="fn"><a href="#fd9542d5-ca23-49bc-ae00-3d2b57caf906" id="fd9542d5-ca23-49bc-ae00-3d2b57caf906-link">1</a></sup>.</p>



<p>This is kind of a sister function to hard links, which similarly avoid copying the file&#8217;s contents <em>but</em> where modifications apply to <em>all</em> copies.  See also <a href="https://eclecticlight.co/2019/01/05/aliases-hard-links-symlinks-and-copies-in-mojaves-apfs/" data-wpel-link="external" target="_blank" rel="external noopener">this article on the differences</a>, including versus aliases and symlinks.</p>



<p>Copy-on-write is beneficial for several reasons:</p>



<ul class="wp-block-list">
<li>Copies don&#8217;t take up any significant space (just whatever tiny amount is necessary for their metadata).</li>



<li>The initial copy operation is practically instantaneous (just a few small metadata writes &amp; updates).</li>



<li>Deferring (if not entirely avoiding) the actual disk I/O reduces wear on the disk.</li>



<li>The copies can share common segments, saving disk space even when they&#8217;re not ultimately identical copies.</li>
</ul>



<p>It does have some potential downsides:</p>



<ul class="wp-block-list">
<li>You don&#8217;t get the increased data redundancy and error resilience that <em>actual</em> copies provide (although if you&#8217;re aiming for data redundancy or backup, you should be using separate physical disks anyway).</li>



<li>It can make subsequent modifications of the file slower, as even just modifying a single byte can trigger the actual copy to be performed.</li>
</ul>



<p>And some basic limitations:</p>



<ul class="wp-block-list">
<li>It&#8217;s only supported on APFS (and <em>maybe</em> additional file systems added by 3rd party extensions, but I haven&#8217;t tested nor can I find any accounts of this).</li>



<li>It only works within individual volumes (it doesn&#8217;t work even between two volumes in the same APFS container, or sharing the same physical disk).</li>
</ul>



<p>Contrary to what I saw online in a few places, copy-on-write works on <em>all</em> APFS volumes, irrespective of whether they are backed by SSDs, HDDs, or some other type of storage.</p>



<h2 class="wp-block-heading">Should I use it?</h2>



<p>Yes!</p>



<p>For most purposes those downsides aren&#8217;t an issue, and the limitations merely mean that it&#8217;s wise to have a fallback option (of just copying the actual file contents) whenever copy-on-write isn&#8217;t available.  And many of the tools &amp; APIs fallback automatically (unless you explicitly require them not to, such as with <code>COPYFILE_CLONE_FORCE</code> to <code>copyfile</code>).</p>



<h2 class="wp-block-heading">How do I use it?</h2>



<figure class="wp-block-table aligncenter"><table><thead><tr><th>Method</th><th class="has-text-align-center" data-align="center">Uses copy-on-write<br>(where possible)</th><th class="has-text-align-center" data-align="center">Does actual copy<br>if copy-on-write<br> isn&#8217;t available</th></tr></thead><tbody><tr><td>cp</td><td class="has-text-align-center" data-align="center">❌</td><td class="has-text-align-center" data-align="center">N/A</td></tr><tr><td>cp -c</td><td class="has-text-align-center" data-align="center">✅</td><td class="has-text-align-center" data-align="center">❌</td></tr><tr><td>ditto</td><td class="has-text-align-center" data-align="center">❌</td><td class="has-text-align-center" data-align="center">N/A</td></tr><tr><td>ditto &#8211;clone</td><td class="has-text-align-center" data-align="center">❌</td><td class="has-text-align-center" data-align="center">N/A</td></tr><tr><td>dd</td><td class="has-text-align-center" data-align="center">❌</td><td class="has-text-align-center" data-align="center">N/A</td></tr><tr><td>scp</td><td class="has-text-align-center" data-align="center">❌</td><td class="has-text-align-center" data-align="center">N/A</td></tr><tr><td>Finder Copy then Paste</td><td class="has-text-align-center" data-align="center">✅</td><td class="has-text-align-center" data-align="center">✅</td></tr><tr><td>Finder Duplicate</td><td class="has-text-align-center" data-align="center">✅</td><td class="has-text-align-center" data-align="center">✅</td></tr><tr><td>Finder ⌥-drag</td><td class="has-text-align-center" data-align="center">✅</td><td class="has-text-align-center" data-align="center">✅</td></tr><tr><td><code><a href="https://www.manpagez.com/man/2/clonefile/" data-wpel-link="external" target="_blank" rel="external noopener">clonefile</a></code><sup data-fn="1bebdb6f-7e69-4ebb-bca6-692795e4e8f7" class="fn"><a href="#1bebdb6f-7e69-4ebb-bca6-692795e4e8f7" id="1bebdb6f-7e69-4ebb-bca6-692795e4e8f7-link">2</a></sup></td><td class="has-text-align-center" data-align="center">✅</td><td class="has-text-align-center" data-align="center">❌</td></tr><tr><td><code><a href="https://keith.github.io/xcode-man-pages/copyfile.3.html" data-wpel-link="external" target="_blank" rel="external noopener">copyfile</a>(…, …, …, COPYFILE_DATA)</code></td><td class="has-text-align-center" data-align="center">❌</td><td class="has-text-align-center" data-align="center">N/A</td></tr><tr><td><code><a href="https://keith.github.io/xcode-man-pages/copyfile.3.html" data-wpel-link="external" target="_blank" rel="external noopener">copyfile</a>(…, …, …, COPYFILE_CLONE)</code></td><td class="has-text-align-center" data-align="center">✅</td><td class="has-text-align-center" data-align="center">✅</td></tr><tr><td><code><a href="https://keith.github.io/xcode-man-pages/copyfile.3.html" data-wpel-link="external" target="_blank" rel="external noopener">copyfile</a>(…, …, …, COPYFILE_CLONE_FORCE)</code></td><td class="has-text-align-center" data-align="center">✅</td><td class="has-text-align-center" data-align="center">❌</td></tr><tr><td><code><a href="https://developer.apple.com/documentation/foundation/filemanager/1412957-copyitem" data-wpel-link="external" target="_blank" rel="external noopener">FileManager.copyItem(at:to:)</a></code></td><td class="has-text-align-center" data-align="center">✅</td><td class="has-text-align-center" data-align="center">✅</td></tr><tr><td><code><a href="https://developer.apple.com/documentation/foundation/filemanager/1407903-copyitem" data-wpel-link="external" target="_blank" rel="external noopener">FileManager.copyItem(atPath:toPath:)</a></code></td><td class="has-text-align-center" data-align="center">✅</td><td class="has-text-align-center" data-align="center">✅</td></tr></tbody></table><figcaption class="wp-element-caption">This is accurate for macOS 14.5 (23F79).  The behaviour might vary across OS releases.</figcaption></figure>



<hr class="wp-block-separator has-alpha-channel-opacity is-style-dots"/>



<h3 class="wp-block-heading">Testing method</h3>



<p>Not that this is interesting, just for posterity and to show my work a bit, in case I made a mistake.</p>



<p>I created large-enough files on each of my test volumes (APFS on SSD, APFS on HDD, HFS+ on SSD) that an actual copy would take tens of seconds at least, using <code>dd</code> e.g.:</p>



<figure class="wp-block-pullquote"><blockquote><p><code>dd if=/dev/random of=/tmp/bigfile oflag=direct status=progress bs=1k count=104857600</code></p></blockquote></figure>



<p>I then ran the various command line tools on these files, attempting to clone the file to a different name in the same folder, and observed disk I/O activity with Activity Monitor and iStat Menus.</p>



<p>For actual copy-on-writes I would observe that the program successfully concluded practically instantly, and there&#8217;d be at most a small blip of disk writes (for metadata modifications).</p>



<p>For failed copy-on-writes I would observe that the program would not exit promptly and I&#8217;d see voluminous disk writes (hundreds of megabytes to gigabytes per second, depending on the disk, sustained for many seconds until I was satisfied with the results and killed the test).</p>



<p>For API tests the overall approach was the same, but the APIs were invoked from inside <code>swift repl</code> where possible, and from a throw-away Swift script otherwise<sup data-fn="4f593842-632c-4279-83ba-435dde24c411" class="fn"><a href="#4f593842-632c-4279-83ba-435dde24c411" id="4f593842-632c-4279-83ba-435dde24c411-link">3</a></sup>.</p>


<ol class="wp-block-footnotes"><li id="fd9542d5-ca23-49bc-ae00-3d2b57caf906">I haven&#8217;t tested it, but as far as I&#8217;ve heard APFS does <em>not</em> actually check if the modifications actually diverge the files.  e.g. if you &#8220;modify&#8221; a byte of the file to the value it already has &#8211; a pointless but technically possible operation that leaves both copies still identical &#8211; APFS <em>will still copy the modified block</em>.<br><br>Furthermore, APFS &amp; Apple&#8217;s operating systems appear to have no tools (nor APIs) to deduplicate files &#8211; e.g. to detect full or partial copies and deduplicate their actual storage on disk.  Not even tools that you could invoke manually if you do the hard work of first determining that two files&#8217; contents are identical.<br><br>APFS / Apple&#8217;s operating systems rely entirely on user applications using copy-on-write explicitly and upfront. <a href="#fd9542d5-ca23-49bc-ae00-3d2b57caf906-link" aria-label="Jump to footnote reference 1">↩︎</a></li><li id="1bebdb6f-7e69-4ebb-bca6-692795e4e8f7">Curiously, <code>clonefile</code> first shipped in macOS 10.12 (Sierra), <a href="https://www.manpagez.com/man/2/clonefile/" data-wpel-link="external" target="_blank" rel="external noopener">according to its man page</a>, which is the release <em>preceding</em> the introduction of APFS in macOS 10.13 (High Sierra).  Yet, as far as I can tell HFS+ doesn&#8217;t and never did support cloning &#8211; nor do any of the other file systems supported by macOS 10.12 (e.g. FAT16 &amp; FAT32, exFAT, NFS).  It&#8217;s possible it was just convenient for Apple to include it in the prior release as they were probably testing APFS with it internally, during APFS&#8217;s development.<br><br><strong>Update</strong>: <a href="https://mjtsai.com" data-wpel-link="external" target="_blank" rel="external noopener">Michael Tsai</a> <a href="https://mastodon.social/@mjtsai/112456863768132668" data-wpel-link="external" target="_blank" rel="external noopener">pointed out</a> that <a href="https://www.pcmag.com/news/what-macos-sierras-new-apfs-file-system-means-to-you" data-wpel-link="external" target="_blank" rel="external noopener">Sierra included APFS support in beta form</a> (e.g. you could create disk images with it, but not use it for a boot disk).  Thus why <code>clonefile</code> (and other APFS-related tools) were included in Sierra.  <a href="#1bebdb6f-7e69-4ebb-bca6-692795e4e8f7-link" aria-label="Jump to footnote reference 2">↩︎</a></li><li id="4f593842-632c-4279-83ba-435dde24c411">While the C APIs worked just fine inside <code>swift repl</code> irrespective of what volumes I was targeting, the Foundation APIs oddly refused to work whenever the files were not on the boot volume, throwing &#8220;Operation not permitted&#8221; errors (NSCocoaErrorDomain 513, with no user info).  If it weren&#8217;t for the C APIs working just fine I&#8217;d think it&#8217;s a sandboxing issue, but clearly it&#8217;s not (and <code>swift repl -disable-sandbox</code> makes no difference). 🤔 <a href="#4f593842-632c-4279-83ba-435dde24c411-link" aria-label="Jump to footnote reference 3">↩︎</a></li></ol>]]></content:encoded>
					
					<wfw:commentRss>https://wadetregaskis.com/copy-on-write-on-apfs/feed/</wfw:commentRss>
			<slash:comments>0</slash:comments>
		
		
		<post-id xmlns="com-wordpress:feed-additions:1">8138</post-id>	</item>
	</channel>
</rss>
