<?xml version="1.0" encoding="UTF-8"?><rss version="2.0"
	xmlns:content="http://purl.org/rss/1.0/modules/content/"
	xmlns:wfw="http://wellformedweb.org/CommentAPI/"
	xmlns:dc="http://purl.org/dc/elements/1.1/"
	xmlns:atom="http://www.w3.org/2005/Atom"
	xmlns:sy="http://purl.org/rss/1.0/modules/syndication/"
	xmlns:slash="http://purl.org/rss/1.0/modules/slash/"
	xmlns:media="http://search.yahoo.com/mrss/"
>

<channel>
	<title>APFS &#8211; Wade Tregaskis</title>
	<atom:link href="https://wadetregaskis.com/tags/apfs/feed/" rel="self" type="application/rss+xml" />
	<link>https://wadetregaskis.com</link>
	<description></description>
	<lastBuildDate>Mon, 20 May 2024 03:37:59 +0000</lastBuildDate>
	<language>en-US</language>
	<sy:updatePeriod>
	hourly	</sy:updatePeriod>
	<sy:updateFrequency>
	1	</sy:updateFrequency>
	

<image>
	<url>https://wadetregaskis.com/wp-content/uploads/2016/03/Stitch-512x512-1-256x256.png</url>
	<title>APFS &#8211; Wade Tregaskis</title>
	<link>https://wadetregaskis.com</link>
	<width>32</width>
	<height>32</height>
</image> 
<site xmlns="com-wordpress:feed-additions:1">226351702</site>	<item>
		<title>Copy-on-write on APFS</title>
		<link>https://wadetregaskis.com/copy-on-write-on-apfs/</link>
					<comments>https://wadetregaskis.com/copy-on-write-on-apfs/#respond</comments>
		
		<dc:creator><![CDATA[]]></dc:creator>
		<pubDate>Thu, 16 May 2024 21:14:10 +0000</pubDate>
				<category><![CDATA[Coding]]></category>
		<category><![CDATA[Education]]></category>
		<category><![CDATA[Howto]]></category>
		<category><![CDATA[APFS]]></category>
		<category><![CDATA[clonefile]]></category>
		<category><![CDATA[copy-on-write]]></category>
		<category><![CDATA[copyfile]]></category>
		<category><![CDATA[FileManager]]></category>
		<category><![CDATA[Foundation]]></category>
		<guid isPermaLink="false">https://wadetregaskis.com/?p=8138</guid>

					<description><![CDATA[APFS (like many modern file systems but unlike its predecessor HFS+) supports copy-on-write. This means you can logically copy a file &#8211; it looks and behaves like a distinct file &#8211; but it doesn&#8217;t immediately copy the file&#8217;s contents on disk &#8211; it merely shares them with the original. Only if and as you modify&#8230; <a class="read-more-link" href="https://wadetregaskis.com/copy-on-write-on-apfs/" data-wpel-link="internal">Read more</a>]]></description>
										<content:encoded><![CDATA[
<p><a href="https://en.wikipedia.org/wiki/Apple_File_System" data-wpel-link="external" target="_blank" rel="external noopener">APFS</a> (like many modern file systems but unlike its predecessor <a href="https://en.wikipedia.org/wiki/HFS_Plus" data-wpel-link="external" target="_blank" rel="external noopener">HFS+</a>) supports <a href="https://eclecticlight.co/2017/06/23/what-is-copy-on-write-and-how-is-it-good/" data-wpel-link="external" target="_blank" rel="external noopener">copy-on-write</a>.  This means you can <em>logically</em> copy a file &#8211; it <em>looks</em> and <em>behaves</em> like a distinct file &#8211; but it doesn&#8217;t <em>immediately</em> copy the file&#8217;s contents on disk &#8211; it merely shares them with the original.  Only if and as you modify either version do they start to diverge on disk, with APFS dynamically allocating new storage for the modified parts<sup data-fn="fd9542d5-ca23-49bc-ae00-3d2b57caf906" class="fn"><a href="#fd9542d5-ca23-49bc-ae00-3d2b57caf906" id="fd9542d5-ca23-49bc-ae00-3d2b57caf906-link">1</a></sup>.</p>



<p>This is kind of a sister function to hard links, which similarly avoid copying the file&#8217;s contents <em>but</em> where modifications apply to <em>all</em> copies.  See also <a href="https://eclecticlight.co/2019/01/05/aliases-hard-links-symlinks-and-copies-in-mojaves-apfs/" data-wpel-link="external" target="_blank" rel="external noopener">this article on the differences</a>, including versus aliases and symlinks.</p>



<p>Copy-on-write is beneficial for several reasons:</p>



<ul class="wp-block-list">
<li>Copies don&#8217;t take up any significant space (just whatever tiny amount is necessary for their metadata).</li>



<li>The initial copy operation is practically instantaneous (just a few small metadata writes &amp; updates).</li>



<li>Deferring (if not entirely avoiding) the actual disk I/O reduces wear on the disk.</li>



<li>The copies can share common segments, saving disk space even when they&#8217;re not ultimately identical copies.</li>
</ul>



<p>It does have some potential downsides:</p>



<ul class="wp-block-list">
<li>You don&#8217;t get the increased data redundancy and error resilience that <em>actual</em> copies provide (although if you&#8217;re aiming for data redundancy or backup, you should be using separate physical disks anyway).</li>



<li>It can make subsequent modifications of the file slower, as even just modifying a single byte can trigger the actual copy to be performed.</li>
</ul>



<p>And some basic limitations:</p>



<ul class="wp-block-list">
<li>It&#8217;s only supported on APFS (and <em>maybe</em> additional file systems added by 3rd party extensions, but I haven&#8217;t tested nor can I find any accounts of this).</li>



<li>It only works within individual volumes (it doesn&#8217;t work even between two volumes in the same APFS container, or sharing the same physical disk).</li>
</ul>



<p>Contrary to what I saw online in a few places, copy-on-write works on <em>all</em> APFS volumes, irrespective of whether they are backed by SSDs, HDDs, or some other type of storage.</p>



<h2 class="wp-block-heading">Should I use it?</h2>



<p>Yes!</p>



<p>For most purposes those downsides aren&#8217;t an issue, and the limitations merely mean that it&#8217;s wise to have a fallback option (of just copying the actual file contents) whenever copy-on-write isn&#8217;t available.  And many of the tools &amp; APIs fallback automatically (unless you explicitly require them not to, such as with <code>COPYFILE_CLONE_FORCE</code> to <code>copyfile</code>).</p>



<h2 class="wp-block-heading">How do I use it?</h2>



<figure class="wp-block-table aligncenter"><table><thead><tr><th>Method</th><th class="has-text-align-center" data-align="center">Uses copy-on-write<br>(where possible)</th><th class="has-text-align-center" data-align="center">Does actual copy<br>if copy-on-write<br> isn&#8217;t available</th></tr></thead><tbody><tr><td>cp</td><td class="has-text-align-center" data-align="center">❌</td><td class="has-text-align-center" data-align="center">N/A</td></tr><tr><td>cp -c</td><td class="has-text-align-center" data-align="center">✅</td><td class="has-text-align-center" data-align="center">❌</td></tr><tr><td>ditto</td><td class="has-text-align-center" data-align="center">❌</td><td class="has-text-align-center" data-align="center">N/A</td></tr><tr><td>ditto &#8211;clone</td><td class="has-text-align-center" data-align="center">❌</td><td class="has-text-align-center" data-align="center">N/A</td></tr><tr><td>dd</td><td class="has-text-align-center" data-align="center">❌</td><td class="has-text-align-center" data-align="center">N/A</td></tr><tr><td>scp</td><td class="has-text-align-center" data-align="center">❌</td><td class="has-text-align-center" data-align="center">N/A</td></tr><tr><td>Finder Copy then Paste</td><td class="has-text-align-center" data-align="center">✅</td><td class="has-text-align-center" data-align="center">✅</td></tr><tr><td>Finder Duplicate</td><td class="has-text-align-center" data-align="center">✅</td><td class="has-text-align-center" data-align="center">✅</td></tr><tr><td>Finder ⌥-drag</td><td class="has-text-align-center" data-align="center">✅</td><td class="has-text-align-center" data-align="center">✅</td></tr><tr><td><code><a href="https://www.manpagez.com/man/2/clonefile/" data-wpel-link="external" target="_blank" rel="external noopener">clonefile</a></code><sup data-fn="1bebdb6f-7e69-4ebb-bca6-692795e4e8f7" class="fn"><a href="#1bebdb6f-7e69-4ebb-bca6-692795e4e8f7" id="1bebdb6f-7e69-4ebb-bca6-692795e4e8f7-link">2</a></sup></td><td class="has-text-align-center" data-align="center">✅</td><td class="has-text-align-center" data-align="center">❌</td></tr><tr><td><code><a href="https://keith.github.io/xcode-man-pages/copyfile.3.html" data-wpel-link="external" target="_blank" rel="external noopener">copyfile</a>(…, …, …, COPYFILE_DATA)</code></td><td class="has-text-align-center" data-align="center">❌</td><td class="has-text-align-center" data-align="center">N/A</td></tr><tr><td><code><a href="https://keith.github.io/xcode-man-pages/copyfile.3.html" data-wpel-link="external" target="_blank" rel="external noopener">copyfile</a>(…, …, …, COPYFILE_CLONE)</code></td><td class="has-text-align-center" data-align="center">✅</td><td class="has-text-align-center" data-align="center">✅</td></tr><tr><td><code><a href="https://keith.github.io/xcode-man-pages/copyfile.3.html" data-wpel-link="external" target="_blank" rel="external noopener">copyfile</a>(…, …, …, COPYFILE_CLONE_FORCE)</code></td><td class="has-text-align-center" data-align="center">✅</td><td class="has-text-align-center" data-align="center">❌</td></tr><tr><td><code><a href="https://developer.apple.com/documentation/foundation/filemanager/1412957-copyitem" data-wpel-link="external" target="_blank" rel="external noopener">FileManager.copyItem(at:to:)</a></code></td><td class="has-text-align-center" data-align="center">✅</td><td class="has-text-align-center" data-align="center">✅</td></tr><tr><td><code><a href="https://developer.apple.com/documentation/foundation/filemanager/1407903-copyitem" data-wpel-link="external" target="_blank" rel="external noopener">FileManager.copyItem(atPath:toPath:)</a></code></td><td class="has-text-align-center" data-align="center">✅</td><td class="has-text-align-center" data-align="center">✅</td></tr></tbody></table><figcaption class="wp-element-caption">This is accurate for macOS 14.5 (23F79).  The behaviour might vary across OS releases.</figcaption></figure>



<hr class="wp-block-separator has-alpha-channel-opacity is-style-dots"/>



<h3 class="wp-block-heading">Testing method</h3>



<p>Not that this is interesting, just for posterity and to show my work a bit, in case I made a mistake.</p>



<p>I created large-enough files on each of my test volumes (APFS on SSD, APFS on HDD, HFS+ on SSD) that an actual copy would take tens of seconds at least, using <code>dd</code> e.g.:</p>



<figure class="wp-block-pullquote"><blockquote><p><code>dd if=/dev/random of=/tmp/bigfile oflag=direct status=progress bs=1k count=104857600</code></p></blockquote></figure>



<p>I then ran the various command line tools on these files, attempting to clone the file to a different name in the same folder, and observed disk I/O activity with Activity Monitor and iStat Menus.</p>



<p>For actual copy-on-writes I would observe that the program successfully concluded practically instantly, and there&#8217;d be at most a small blip of disk writes (for metadata modifications).</p>



<p>For failed copy-on-writes I would observe that the program would not exit promptly and I&#8217;d see voluminous disk writes (hundreds of megabytes to gigabytes per second, depending on the disk, sustained for many seconds until I was satisfied with the results and killed the test).</p>



<p>For API tests the overall approach was the same, but the APIs were invoked from inside <code>swift repl</code> where possible, and from a throw-away Swift script otherwise<sup data-fn="4f593842-632c-4279-83ba-435dde24c411" class="fn"><a href="#4f593842-632c-4279-83ba-435dde24c411" id="4f593842-632c-4279-83ba-435dde24c411-link">3</a></sup>.</p>


<ol class="wp-block-footnotes"><li id="fd9542d5-ca23-49bc-ae00-3d2b57caf906">I haven&#8217;t tested it, but as far as I&#8217;ve heard APFS does <em>not</em> actually check if the modifications actually diverge the files.  e.g. if you &#8220;modify&#8221; a byte of the file to the value it already has &#8211; a pointless but technically possible operation that leaves both copies still identical &#8211; APFS <em>will still copy the modified block</em>.<br><br>Furthermore, APFS &amp; Apple&#8217;s operating systems appear to have no tools (nor APIs) to deduplicate files &#8211; e.g. to detect full or partial copies and deduplicate their actual storage on disk.  Not even tools that you could invoke manually if you do the hard work of first determining that two files&#8217; contents are identical.<br><br>APFS / Apple&#8217;s operating systems rely entirely on user applications using copy-on-write explicitly and upfront. <a href="#fd9542d5-ca23-49bc-ae00-3d2b57caf906-link" aria-label="Jump to footnote reference 1">↩︎</a></li><li id="1bebdb6f-7e69-4ebb-bca6-692795e4e8f7">Curiously, <code>clonefile</code> first shipped in macOS 10.12 (Sierra), <a href="https://www.manpagez.com/man/2/clonefile/" data-wpel-link="external" target="_blank" rel="external noopener">according to its man page</a>, which is the release <em>preceding</em> the introduction of APFS in macOS 10.13 (High Sierra).  Yet, as far as I can tell HFS+ doesn&#8217;t and never did support cloning &#8211; nor do any of the other file systems supported by macOS 10.12 (e.g. FAT16 &amp; FAT32, exFAT, NFS).  It&#8217;s possible it was just convenient for Apple to include it in the prior release as they were probably testing APFS with it internally, during APFS&#8217;s development.<br><br><strong>Update</strong>: <a href="https://mjtsai.com" data-wpel-link="external" target="_blank" rel="external noopener">Michael Tsai</a> <a href="https://mastodon.social/@mjtsai/112456863768132668" data-wpel-link="external" target="_blank" rel="external noopener">pointed out</a> that <a href="https://www.pcmag.com/news/what-macos-sierras-new-apfs-file-system-means-to-you" data-wpel-link="external" target="_blank" rel="external noopener">Sierra included APFS support in beta form</a> (e.g. you could create disk images with it, but not use it for a boot disk).  Thus why <code>clonefile</code> (and other APFS-related tools) were included in Sierra.  <a href="#1bebdb6f-7e69-4ebb-bca6-692795e4e8f7-link" aria-label="Jump to footnote reference 2">↩︎</a></li><li id="4f593842-632c-4279-83ba-435dde24c411">While the C APIs worked just fine inside <code>swift repl</code> irrespective of what volumes I was targeting, the Foundation APIs oddly refused to work whenever the files were not on the boot volume, throwing &#8220;Operation not permitted&#8221; errors (NSCocoaErrorDomain 513, with no user info).  If it weren&#8217;t for the C APIs working just fine I&#8217;d think it&#8217;s a sandboxing issue, but clearly it&#8217;s not (and <code>swift repl -disable-sandbox</code> makes no difference). 🤔 <a href="#4f593842-632c-4279-83ba-435dde24c411-link" aria-label="Jump to footnote reference 3">↩︎</a></li></ol>]]></content:encoded>
					
					<wfw:commentRss>https://wadetregaskis.com/copy-on-write-on-apfs/feed/</wfw:commentRss>
			<slash:comments>0</slash:comments>
		
		
		<post-id xmlns="com-wordpress:feed-additions:1">8138</post-id>	</item>
		<item>
		<title>Creating a Fusion Drive from an existing disk, without erasing it</title>
		<link>https://wadetregaskis.com/creating-a-fusion-drive-from-an-existing-disk-without-erasing-it/</link>
					<comments>https://wadetregaskis.com/creating-a-fusion-drive-from-an-existing-disk-without-erasing-it/#respond</comments>
		
		<dc:creator><![CDATA[]]></dc:creator>
		<pubDate>Sun, 22 Jul 2018 18:01:02 +0000</pubDate>
				<category><![CDATA[Howto]]></category>
		<category><![CDATA[APFS]]></category>
		<category><![CDATA[CoreStorage]]></category>
		<category><![CDATA[Finder]]></category>
		<category><![CDATA[macOS]]></category>
		<category><![CDATA[RAID]]></category>
		<category><![CDATA[Terminal]]></category>
		<guid isPermaLink="false">https://blog.wadetregaskis.com/?p=4160</guid>

					<description><![CDATA[Note: this guide was written circa macOS 10.13 High Sierra, in 2018. Its accuracy has not been verified for newer macOS releases. Curiously there&#8217;s very little information out on the web (at time of writing) on how to create (or expand) a Fusion Drive without erasing its contents first. &#160;It&#8217;s entirely possible to do so&#8230; <a class="read-more-link" href="https://wadetregaskis.com/creating-a-fusion-drive-from-an-existing-disk-without-erasing-it/" data-wpel-link="internal">Read more</a>]]></description>
										<content:encoded><![CDATA[
<div class="wp-block-group"><div class="wp-block-group__inner-container is-layout-constrained wp-block-group-is-layout-constrained">
<p><strong>Note</strong>:  this guide was written circa macOS 10.13 High Sierra, in 2018.  Its accuracy has not been verified for newer macOS releases.</p>
</div></div>



<p>Curiously there&#8217;s very little information out on the web (at time of writing) on how to create (or expand) a Fusion Drive <em>without</em> erasing its contents first. &nbsp;It&#8217;s entirely possible to do so &#8211; in <em>some</em> circumstances &#8211; and is probably what most people want, yet almost every &#8216;create your own Fusion Drive&#8217; guide presumes you&#8217;re happy losing all your data in the process.&nbsp;🤪</p>



<p>So here&#8217;s how to do it, step-by-step. &nbsp;The process is fairly straightforward, but there are some edge cases that might apply to you. &nbsp;The obvious &amp; most common ones are covered by this guide, but if you find yourself in a situation that isn&#8217;t, please leave a comment at the bottom of this guide explaining what&#8217;s not working for you, and I&#8217;ll try to help you out.</p>



<p>Don&#8217;t be intimidated by the length of this guide &#8211; in the common case it&#8217;s only six steps, each a single operation, and someone familiar with the steps involved can complete this whole process in just a couple of minutes. &nbsp;The verbosity of this guide is because it takes great pains to explain precisely what&#8217;s going on in order to prevent confusion &amp; error. &nbsp;Read it carefully, and take it slow &#8211; there is no undo option for any of these commands.</p>



<h2 class="wp-block-heading">Seriously important warning</h2>



<p class="has-drop-cap">⚠️Manipulating your computer&#8217;s storage at the low level described in this guide is <strong>dangerous</strong>. &nbsp;In <em>principle</em> it&#8217;s safe as long as you enter all the right commands. &nbsp;However,&nbsp;<strong>a single typo in the wrong place can destroy all your data</strong>&nbsp;&#8211; e.g. a &#8216;2&#8217; instead of a &#8216;1&#8217; when typing a disk identifier. &nbsp;Furthermore, in practice Apple&#8217;s tools are not completely reliable, and it&#8217;s possible that even with <em>you</em> doing everything exactly right, something will still go wrong and your data might be lost.</p>



<p>Make sure all of your data is reliably &amp; <em>verifiably</em> backed up before attempting any of this. &nbsp;<strong>Make sure your backups are not connected to your computer while you attempt this</strong> &#8211; if they are, they&#8217;re also just one typo or software bug or hardware glitch away from being destroyed too.</p>



<p>It&#8217;s recommended that you disconnect all other disks while you go through this process. &nbsp;That way you eliminate the possibility of damaging them, whether as the result of a typo, confusion between similar disks, or a software bug.</p>



<h2 class="wp-block-heading">A quick word on critical terminology</h2>



<ul class="wp-block-list">
<li><strong>&#8220;Volume&#8221;</strong> in this context is what you might casually call a &#8220;disk&#8221; &#8211; it&#8217;s what you see as a disk icon on your desktop, for example, or listed under the &#8216;Devices&#8217; section in the Finder window sidebars.<br><br>All Macs have at least one volume &#8211; the boot volume, usually the internal SSD, Fusion Drive, or hard disk &#8211; but you can of course add any number more of them via USB, Thunderbolt, Firewire, or even internally if you have an old enough Mac.<br><br>&#8220;Volume&#8221; is used rather than &#8220;disk&#8221; because the latter is ambiguous &#8211; lots of things are called &#8220;disks&#8221;, including many things that aren&#8217;t actually disks in any physical or practical sense. &nbsp;But &#8220;volume&#8221; has a specific meaning in the Mac storage lexicon, and you&#8217;ll see the word used frequently by Apple&#8217;s tools. &nbsp;There are different types of volumes &#8211; &#8220;physical&#8221;, &#8220;logical&#8221;, etc. &nbsp;For the most part you don&#8217;t need to worry about those distinctions here &#8211; it&#8217;ll be pointed out specifically if &amp; when you do. &nbsp;Similarly there are related constructs like &#8220;volume groups&#8221; and &#8220;volume families&#8221;, but likewise you don&#8217;t need to know exactly what those mean for the purposes of this task. &nbsp;Just be aware that these distinctions exist, and read the instructions very carefully to see exactly which one is needed &#8211; particularly when dealing with UUIDs, as you&#8217;ll see.</li>



<li><strong>&#8220;Disk&#8221;</strong> is thus used only to refer to physical disks &#8211; things you can hold in your hand. &nbsp;A disk can contain many volumes (sometimes called partitions).</li>



<li><strong>&#8220;Target volume&#8221;</strong> is used to denote the particular volume you want converted into a Fusion Drive. &nbsp;It should be your <em>existing</em> volume, the one with your data already on it &#8211; data that you want preserved.</li>



<li><strong>&#8220;Additional disk&#8221;</strong> is used to denote the SSD (or similar) that you want to fuse into your target volume, to create your Fusion Drive. &nbsp;In contrast to your target volume, its contents will <em>not</em> be preserved. &nbsp;Make sure you move any data on that disk somewhere safe before starting this process. &nbsp;Also note the distinction between disk and volume here &#8211; you can only add whole disks to Fusion Drives, not volumes &#8211; if your additional disk has multiple volumes on it (e.g. a Boot Camp partition in addition to a Mac volume) <em>all</em> of those will be erased during the fusion process.</li>
</ul>



<h2 class="wp-block-heading">Procedure</h2>


<div class="wp-block-image">
<figure class="alignright size-thumbnail is-resized"><img fetchpriority="high" decoding="async" width="256" height="256" src="https://wadetregaskis.com/wp-content/uploads/2018/07/Terminal-icon-256x256.webp" alt="Terminal app icon" class="wp-image-4161" style="object-fit:cover;width:128px;height:128px" srcset="https://wadetregaskis.com/wp-content/uploads/2018/07/Terminal-icon-256x256.webp 256w, https://wadetregaskis.com/wp-content/uploads/2018/07/Terminal-icon-512x512.webp 512w, https://wadetregaskis.com/wp-content/uploads/2018/07/Terminal-icon.webp 1024w" sizes="(max-width: 256px) 100vw, 256px" /></figure>
</div>


<h3 class="wp-block-heading">1. &nbsp;Open Terminal.</h3>



<p>All of the following commands are entered in Terminal, unless otherwise specified. &nbsp;If you&#8217;re an admin user you won&#8217;t need to do anything more than what&#8217;s listed here, nor enter your login password at any point (nor, similarly, use sudo at any point). &nbsp;If you&#8217;re not an admin user, you might be prompted for an admin&#8217;s credentials, or you might have to run all the following commands via sudo (ideally as an admin, or as root if necessary).</p>



<h3 class="wp-block-heading">2. &nbsp;List your existing <a href="https://en.wikipedia.org/wiki/Core_Storage" data-wpel-link="external" target="_blank" rel="external noopener">Core Storage</a> volumes.</h3>



<p>Enter the following command in Terminal:</p>



<blockquote class="wp-block-quote is-layout-flow wp-block-quote-is-layout-flow">
<p><code>diskutil cs list</code></p>
</blockquote>



<p>The output will ideally look something like:</p>



<pre class="wp-block-preformatted">CoreStorage logical volume groups (1 found)
|
+-- Logical Volume Group CF936785-3F07-4414-8512-F94E26F0ABC3
 &nbsp; =========================================================
 &nbsp; Name: &nbsp; &nbsp; &nbsp; &nbsp; Macintosh HD
 &nbsp; Status: &nbsp; &nbsp; &nbsp; Online
 &nbsp; Size: &nbsp; &nbsp; &nbsp; &nbsp; 6000831135744 B (6.0 TB)
 &nbsp; Free Space: &nbsp; 15101952 B (15.1 MB)
 &nbsp; |
 &nbsp; +-&lt; Physical Volume C68C0FB4-B85F-4376-B9C4-BC744520A5FA
 &nbsp; | &nbsp; ----------------------------------------------------
 &nbsp; | &nbsp; Index:&nbsp; &nbsp; 0
 &nbsp; | &nbsp; Disk: &nbsp; &nbsp; disk99s2
 &nbsp; | &nbsp; Status: &nbsp; Online
 &nbsp; | &nbsp; Size: &nbsp; &nbsp; 6000831135744 B (6.0 TB)
 &nbsp; |
 &nbsp; +-&gt; Logical Volume Family 33A1CC8F-A3AF-4B5D-8EA0-379F4C9D6D0E
 &nbsp;   &nbsp; ----------------------------------------------------------
 &nbsp; &nbsp; &nbsp; Encryption Type: &nbsp; &nbsp; &nbsp; &nbsp; AES-XTS
 &nbsp; &nbsp; &nbsp; Encryption Status: &nbsp; &nbsp; &nbsp; Unlocked
 &nbsp; &nbsp; &nbsp; Conversion Status: &nbsp; &nbsp; &nbsp; Complete
 &nbsp; &nbsp; &nbsp; High Level Queries:&nbsp; &nbsp; &nbsp; Fully Secure
 &nbsp; &nbsp; &nbsp; |&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; Passphrase Required
 &nbsp; &nbsp; &nbsp; |&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; Accepts New Users
 &nbsp; &nbsp; &nbsp; |&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; Has Visible Users
 &nbsp; &nbsp; &nbsp; |&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; Has Volume Key
 &nbsp; &nbsp; &nbsp; |
 &nbsp; &nbsp; &nbsp; +-&gt; Logical Volume 10DE6E37-6701-46F8-8E1F-42759C42B5A0
 &nbsp; &nbsp; &nbsp;   &nbsp; ---------------------------------------------------
 &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; Disk:&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; disk100
 &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; Status:&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; Online
 &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; Size (Total):&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; 6000471965696 B (6.0 TB)
 &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; Revertible:&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; No
 &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; LV Name: &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; Macintosh HD
 &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; Volume Name: &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; Macintosh HD
 &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; Content Hint:&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; Apple_HFS</pre>



<p>If you see your target volume in there as a Logical Volume &#8211; e.g. like &#8220;Macintosh HD&#8221; in the above example &#8211; then you&#8217;re all set to move on to step 3, making note of two things:</p>



<ol class="wp-block-list">
<li>The target logical volume&#8217;s UUID (10DE6E37-6701-46F8-8E1F-42759C42B5A0 in the above example). &nbsp;You&#8217;ll need this in step 6.</li>



<li>The target logical volume&#8217;s volume group UUID&nbsp;(CF936785-3F07-4414-8512-F94E26F0ABC3 in the above example). &nbsp;You&#8217;ll need this in step 5.</li>
</ol>



<p>If instead there are no volumes listed, or your target volume isn&#8217;t one of them, then you will need to perform the extra steps below.</p>



<p>If your target volume&#8217;s name does appear, but not on any Logical Volume &#8211; perhaps only as a Logical Volume Group &#8211; then do not proceed; your scenario is not covered by this guide. &nbsp;Please leave a comment at the bottom of this guide, so I can help you figure out what&#8217;s going on and what to do in your specific circumstances.</p>



<h4 class="wp-block-heading">2B. &nbsp;List non-Core Storage volumes.</h4>



<blockquote class="wp-block-quote is-layout-flow wp-block-quote-is-layout-flow">
<p><code>diskutil list</code></p>
</blockquote>



<p>You&#8217;ll see a list of one or more disks &amp; their constituent volumes, e.g.:</p>



<pre class="wp-block-preformatted">/dev/disk99 (internal):
   #:                       TYPE NAME                    SIZE       IDENTIFIER
   0:      GUID_partition_scheme                         6.0 TB     disk99
   1:                        EFI EFI                     314.6 MB   disk99s1
   2:                  Apple_HFS Macintosh HD            6.0 TB     disk99s2
   3:                 Apple_Boot Boot OS X               134.2 MB   disk99s3</pre>



<p>Make sure your target volume appears somewhere in that list. &nbsp;If it doesn&#8217;t… then I&#8217;m not sure what to tell you &#8211; your target volume appears to not be connected to your computer.</p>



<p>You&#8217;d typically be using Core Storage as a side-effect of encrypting your volumes (<em>iff</em> they&#8217;re HFS-formatted), or if you&#8217;re using Apple&#8217;s software RAID. &nbsp;If your target volume is APFS-formatted, however, and it&#8217;s <em>not</em> already using Core Storage underneath, you might be out of luck &#8211; you can try following 2C Option 2 below on the APFS Container volume, but I haven&#8217;t tried it myself so I don&#8217;t know if Apple&#8217;s tools support it (nor whether they can do the conversion correctly &amp; reliably &#8211; proceed at your own risk).</p>



<h4 class="wp-block-heading">2C Option 1: &nbsp;Encrypt your target volume.</h4>



<p>The easiest and most fool-proof way to do this is to right-click on the target volume anywhere it appears in the Finder, and choose the &#8216;<em>Encrypt &#8220;&lt;volume name&gt;</em>&#8220;…&#8217; option from the contextual menu. &nbsp;You&#8217;ll be prompted to enter a passphrase with which to encrypt the volume, and the operation might take some time, depending on the speed &amp; size of your target volume.</p>



<p>This is also an excellent option because you should <em>always</em> encrypt your volumes, for your own security &amp; privacy.</p>



<p>Alternatively, you can use Terminal:</p>



<blockquote class="wp-block-quote is-layout-flow wp-block-quote-is-layout-flow">
<p><code>diskutil cs convert &lt;disk identifier> -stdinpassphrase</code></p>
</blockquote>



<p>The disk identifier is the name associated with your target volume that looks like &#8220;diskXsY&#8221; where X &amp; Y are numbers (e.g. disk99s2 in the prior example).</p>



<p>You&#8217;ll be prompted to enter the passphrase to encrypt the volume with.</p>



<p>Again, the operation might take some time, depending on the speed &amp; size of your target volume.</p>



<h4 class="wp-block-heading">2C Option 2: &nbsp;Convert your target volume to Core Storage without encrypting it.</h4>



<blockquote class="wp-block-quote is-layout-flow wp-block-quote-is-layout-flow">
<p><code>diskutil cs convert &lt;disk identifier></code></p>
</blockquote>



<p>The disk identifier is the name associated with your target volume that looks like &#8220;diskXsY&#8221; where X &amp; Y are numbers (e.g. disk99s2 in the prior example) &#8211; see under the &#8216;IDENTIFIER&#8217; column in the output of diskutil list from step 2B.</p>



<p>The operation might take some time, depending on the speed &amp; size of your target volume.</p>



<h4 class="wp-block-heading">2D. &nbsp;Verify the conversion succeeded.</h4>



<blockquote class="wp-block-quote is-layout-flow wp-block-quote-is-layout-flow">
<p><code>diskutil cs list</code></p>
</blockquote>



<p>As per the instructions in step 2, you should see your target volume now listed as a Logical Volume.</p>



<h3 class="wp-block-heading">3. &nbsp;Determine your additional disk&#8217;s identifier.</h3>



<p>This is the disk you want to fuse into your target volume in order to create your Fusion Drive. &nbsp;It will ideally be blank to begin with &#8211; e.g. you just bought it &amp; connected it to your computer, and have no yet formatted it or otherwise used it.</p>



<p>If it is not blank, you need to move all your data off of it &#8211; <strong>it will be essentially erased as part of the fusion process</strong> (your target volume won&#8217;t be, so potentially you can move the data to there).</p>



<p>Also note the distinction between disk and volume here &#8211; you can only add whole disks to Fusion Drives, not volumes &#8211; if your additional disk has multiple volumes on it (e.g. a Boot Camp partition in addition to a Mac volume)&nbsp;<span style="font-style: italic;">all</span>&nbsp;of those will be erased during the fusion process. &nbsp;Make sure to move your data off of&nbsp;<em>all</em> those volumes.</p>



<p>To determine the disk identifier for your additional disk, run:</p>



<blockquote class="wp-block-quote is-layout-flow wp-block-quote-is-layout-flow">
<p><code>diskutil list</code></p>
</blockquote>



<p>The output should look something like the following:</p>



<pre class="wp-block-preformatted">/dev/disk99 (internal):
   #:                       TYPE NAME                    SIZE       IDENTIFIER
   0:      GUID_partition_scheme                         6.0 TB     disk99
   1:                        EFI EFI                     314.6 MB   disk99s1
   2:          Apple_CoreStorage Macintosh HD            6.0 TB     disk99s2
/dev/disk100 (internal, virtual):
   #:                       TYPE NAME                    SIZE       IDENTIFIER
   0:                  Apple_HFS Macintosh HD           +16.0 TB    disk100
                                 Logical Volume on disk99
                                 10DE6E37-6701-46F8-8E1F-42759C42B5A0
                                 Unlocked Encrypted
/dev/disk101 (external, physical):</pre>



<p>In the above example, the last entry is the additional disk &#8211; it has the disk identifier disk101. &nbsp;In this example it is not formatted, so it has no volumes listed on it.</p>



<h3 class="wp-block-heading">4. &nbsp;Check that you&#8217;ve identified the correct additional disk.</h3>



<blockquote class="wp-block-quote is-layout-flow wp-block-quote-is-layout-flow">
<p><code>diskutil info &lt;disk identifier></code></p>
</blockquote>



<p>The output should look something like:</p>



<pre class="wp-block-preformatted">   Device Identifier:        disk101
   Device Node:              /dev/disk101
   Whole:                    Yes
   Part of Whole:            disk101
   Device / Media Name:      CT2000MX500SSD1

   Volume Name:              Not applicable (no file system)
   Mounted:                  Not applicable (no file system)
   File System:              None

   Content (IOContent):      GUID_partition_scheme
   OS Can Be Installed:      No
   Media Type:               Generic
   Protocol:                 SATA
   SMART Status:             Verified

   Disk Size:                2.0 TB (2000398934016 Bytes) (exactly 3907029168 512-Byte-Units)
   Device Block Size:        512 Bytes

   Read-Only Media:          No
   Read-Only Volume:         Not applicable (no file system)

   Device Location:          External
   Removable Media:          Fixed

   Solid State:              Yes
   Virtual:                  No
   Hardware AES Support:     No</pre>



<p>Check that info to make sure it matches what you&#8217;re expecting &#8211; in particular things like the device / media name (model number), and the disk&#8217;s size. &nbsp;Be extremely careful if you have multiple disks connected to your computer of the same type or size &#8211; remember that this disk will be erased during the fusion process, so you definitely do not want to pick the wrong disk by mistake. &nbsp;If in doubt &#8211; in fact as a best practice regardless &#8211; disconnect all other disks first, to ensure you can&#8217;t target them by mistake.</p>



<h3 class="wp-block-heading">5. &nbsp;Add your additional disk to your target volume&#8217;s Core Storage volume group.</h3>



<blockquote class="wp-block-quote is-layout-flow wp-block-quote-is-layout-flow">
<p><code>diskutil cs addDisk &lt;target logical volume's volume group UUID> &lt;additional disk's identifier></code></p>
</blockquote>



<p>If all goes well, you&#8217;ll see something like:</p>



<pre class="wp-block-preformatted">Started CoreStorage operation on disk101
Unmounting disk101
Repartitioning disk101
Unmounting disk
Creating the partition map
Rediscovering disk101
Adding disk101s2 to Logical Volume Group
Switching disk101s2 to Core Storage
Waiting for Logical Volume Group to come back online
Core Storage PV UUID: D0E04199-9402-46EF-8481-98C8BADEF8FE
Finished CoreStorage operation on disk101
</pre>



<p>If you wish you can also confirm that the logical volume group now has your additional disk listed as a physical volume underneath it:</p>



<blockquote class="wp-block-quote is-layout-flow wp-block-quote-is-layout-flow">
<p><code>diskutil cs list</code></p>
</blockquote>



<p>You should now see something like:</p>



<pre class="wp-block-preformatted">CoreStorage logical volume groups (1 found)
|
+-- Logical Volume Group CF936785-3F07-4414-8512-F94E26F0ABC3
   =========================================================
   Name:         Macintosh HD
   Status:       Online
   Size:         6000831135744 B (6.0 TB)
   Free Space:   15101952 B (15.1 MB)
   |
   +-&lt; Physical Volume C68C0FB4-B85F-4376-B9C4-BC744520A5FA
   |   ----------------------------------------------------
   |   Index:    0
   |   Disk:     disk99s2
   |   Status:   Online
   |   Size:     6000831135744 B (6.0 TB)
   |
   +-&lt; Physical Volume D0E04199-9402-46EF-8481-98C8BADEF8FE
   |   ----------------------------------------------------
   |   Index:    1
   |   Disk:     disk101s2
   |   Status:   Online
   |   Size:     2000054960128 B (2.0 TB)
   |
   +-&gt; Logical Volume Family 33A1CC8F-A3AF-4B5D-8EA0-379F4C9D6D0E
       ----------------------------------------------------------
       Encryption Type:         AES-XTS
       Encryption Status:       Unlocked
       Conversion Status:       Complete
       High Level Queries:      Fully Secure
       |                        Passphrase Required
       |                        Accepts New Users
       |                        Has Visible Users
       |                        Has Volume Key
       |
       +-&gt; Logical Volume 10DE6E37-6701-46F8-8E1F-42759C42B5A0
           ---------------------------------------------------
           Disk:                  disk100
           Status:                Online
           Size (Total):          6000471965696 B (6.0 TB)
           Revertible:            No
           LV Name:               Macintosh HD
           Volume Name:           Macintosh HD
           Content Hint:          Apple_HFS</pre>



<p>Note however that the logical volume &#8211; your target volume; your would-be Fusion Drive &#8211; is still listed as being the same, older size. &nbsp;It&#8217;s not yet making full use of the additional disk you added &#8211; that&#8217;s the next &amp; last step.</p>



<h3 class="wp-block-heading">6. &nbsp;Resize your target volume to make use of the new space from your additional disk.</h3>



<p>Since Fusion Drives split data across their constituent disks (as opposed to mirroring it, as you might do in a RAID setup for improved reliability), their size is the sum of all the disks you put into them. &nbsp;You can in fact have multiple volumes on a single Fusion Drive, splitting up its total space amongst them however you wish, and having them share all the underlying disks on a kind of first-come-first-served basis. &nbsp;However, the details of that scenario are left as an exercise for the reader &#8211; here we&#8217;ll only look at the simple, most common example where you just have a single volume on your Fusion Drive.</p>



<blockquote class="wp-block-quote is-layout-flow wp-block-quote-is-layout-flow">
<p><code>diskutil cs resizeVolume &lt;target volume's UUID> 0g</code></p>
</blockquote>



<p>The target volume&#8217;s UUID was listed in the output of step 2, from the <code>diskutil cs list</code> command.</p>



<p>When you run the resizeVolume command, it will typically take a little while &#8211; it does some sanity checking &amp; validation of the target volume first, to make sure it&#8217;s in good health to begin with, and then the resize operation itself can take a few moments. &nbsp;You should see output similar to:</p>



<pre class="wp-block-preformatted">The Core Storage Logical Volume UUID is 10DE6E37-6701-46F8-8E1F-42759C42B5A0
Started CoreStorage operation
Verifying file system
Volume was successfully unmounted
Performing fsck_hfs -fn -x /dev/rdisk100
Checking Journaled HFS Plus volume
Checking extents overflow file
Checking catalog file
Checking multi-linked files
Checking catalog hierarchy
Checking extended attributes file
Checking volume bitmap
Checking volume information
The volume Macintosh HD appears to be OK
File system check exit code is 0
Restoring the original state found as mounted
Growing Logical Volume
Resizing Core Storage Logical Volume structures
Resized Core Storage Logical Volume to 8,000,242,745,344 bytes
Growing file system
Finished CoreStorage operation</pre>



<p>You&#8217;re now done! &nbsp;Your target volume is now a Fusion Drive, ready for use, and should reflect its new, enlarged size in the Finder and elsewhere.</p>



<p>Note: &nbsp;the new total size shown for your Fusion Drive will be slightly less than the sum of the underlying disks &#8211; usually by a few hundred megabytes. &nbsp;This is presumably because some space is reserved for Core Storage metadata and other essential system data.</p>
]]></content:encoded>
					
					<wfw:commentRss>https://wadetregaskis.com/creating-a-fusion-drive-from-an-existing-disk-without-erasing-it/feed/</wfw:commentRss>
			<slash:comments>0</slash:comments>
		
		
			<media:content url="https://wadetregaskis.com/wp-content/uploads/2018/07/Apple-Fusion-Drive-presentation-2048x1098.avif" medium="image" />
<post-id xmlns="com-wordpress:feed-additions:1">4160</post-id>	</item>
		<item>
		<title>Silent data corruption</title>
		<link>https://wadetregaskis.com/silent-data-corruption/</link>
					<comments>https://wadetregaskis.com/silent-data-corruption/#respond</comments>
		
		<dc:creator><![CDATA[]]></dc:creator>
		<pubDate>Wed, 06 Jul 2016 05:23:36 +0000</pubDate>
				<category><![CDATA[Ramblings]]></category>
		<category><![CDATA[APFS]]></category>
		<category><![CDATA[Bugs!]]></category>
		<category><![CDATA[lies]]></category>
		<category><![CDATA[Sad]]></category>
		<category><![CDATA[Snafu]]></category>
		<category><![CDATA[Time Machine]]></category>
		<guid isPermaLink="false">https://blog.wadetregaskis.com/?p=3652</guid>

					<description><![CDATA[Alternate title:  Apple&#8217;s file system engineers are sadly naive. I was quite disappointed to see that APFS isn&#8217;t even trying to provide data integrity.  Data integrity is kind of step 0 of any file system, and checksums or use of ECC is pretty much standard in modern &#38; leading-edge file systems.  APFS doesn&#8217;t want to&#8230; <a class="read-more-link" href="https://wadetregaskis.com/silent-data-corruption/" data-wpel-link="internal">Read more</a>]]></description>
										<content:encoded><![CDATA[<p>Alternate title:  Apple&#8217;s file system engineers are sadly naive.</p>
<p>I was quite disappointed to see that APFS isn&#8217;t even trying to provide data integrity.  Data integrity is kind of step 0 of any file system, and checksums or use of ECC is pretty much standard in modern &amp; leading-edge file systems.  APFS doesn&#8217;t want to be one of those, it seems.</p>
<p>Case in point why this matters:</p>
<p style="padding-left: 30px;">I have a bunch of old backup drives, because drives are cheap and until recently I could just buy a new one once the current one filled, instead of ever deleting a backup.  Periodically I go back through these old backup drives and do some basic integrity checks (S.M.A.R.T. bad block scans, file system checks, etc).</p>
<p style="padding-left: 30px;">I <em>also</em> run a comparison of key data between those backups and the current versions on my computer, for files which generally <em>shouldn&#8217;t</em> change nor disappear &#8211; e.g. photos, videos, key documents, etc.</p>
<p style="padding-left: 30px;">And today I found that at least half a dozen valuable personal videos (and a few photos) were corrupt, in the versions on my computer.  Luckily, the versions in the ancient backups were still good, so I could replace the corrupt ones.</p>
<p style="padding-left: 30px;">This corruption was completely silent, until my &#8216;paranoid&#8217; and time-consuming checks discovered it.</p>
<p>It&#8217;s far from the first time.  A failing drive years back corrupted a huge portion of my music library &#8211; silently, as far as the file system &amp; OS were concerned.  Periodically I&#8217;ve discovered photos (of which I have huge numbers &#8211; the majority of my data) which have become corrupt at some indeterminate point.  And I&#8217;ve of course had file system [metadata] corruption occur many times, sometimes requiring complete erasure of the disk, and recovery or rebuilds from backup (a few times I&#8217;ve had to use data recovery software, where backups weren&#8217;t available).</p>
<p>Most, if not all, of these issues would have been discovered by even the most trivial file integrity protections, in the file system.</p>
<p>The notion that modern disks somehow magically protect against all silent data corruption is abject poppycock.  They&#8217;re <em>more</em> likely to suffer from it than older disks &#8211; a byproduct of higher densities and market demand for cheaper, crappier storage products.</p>
<p>And the implicit assertion that Apple&#8217;s file system driver, and kernel overall, are somehow completely free of bugs… is just batshit crazy.</p>
<p><strong>Addendum</strong></p>
<p>Since Apple aren&#8217;t interested in protecting anyone&#8217;s valuable personal data, I&#8217;m on the look-out for other options.  Manual use of <em>shasum</em> is one, for now, but a more streamlined and fool-proof system would be better.  Alas, none seems to exist[1. There is <a href="https://github.com/laktak/chkbit" data-wpel-link="external" target="_blank" rel="external noopener">chkbit</a>, but it relies on MD5… <em>probably</em> acceptable for this use case, but needless in the face of <em>decades</em> of better hash algorithms.  And it&#8217;s written in JavaScript.  Ew.].  Yet.</p>
]]></content:encoded>
					
					<wfw:commentRss>https://wadetregaskis.com/silent-data-corruption/feed/</wfw:commentRss>
			<slash:comments>0</slash:comments>
		
		
		<post-id xmlns="com-wordpress:feed-additions:1">3652</post-id>	</item>
	</channel>
</rss>
