The art and science of OS X system upgrades

Since 10.2 (or was it 10.1?), we Mac users have had to make a choice when it comes to major ("dot release") OS X releases: how will we upgrade? In its current incarnation, the OS X installer offers three options:

• Upgrade Install: The easiest option for users, this simply patches the necessary bits of the system and bundled applications to migrate from the current OS to the new release.
• Archive and Install: The installer moves the entire current system, Users folders and all, into a Previous System folder, and then installs a new copy from scratch. You can optionally migrate over your Users folder to ease the transition.
• Erase and Install: The "wipe the slate clean" approach. Your hard drive is erased, a new system is installed, and you start over from scratch.

Which method to use is seemingly a matter of great debate. Apple has a good overview available, too, with a brief description of each option.

Having just recently started (note that it's not yet completed) migrating my primary boot drive to 10.4, read on for my thoughts on upgrade strategies, the strengths and weaknesses of each of these methods, and which I prefer (and why).

General Upgrade Advice

Before I discuss the particulars of upgrading an OS X installation, here's some general advice that I always follow when undertaking a major upgrade like this. Note that some of these steps can be quite time consuming, but since there's no going backwards from an upgrade, I feel they're worth the time. There's also a level of overkill in here, but my data is my livelihood (in both my job and my website 'hobby'), so the redundancy helps me breathe easier during the upgrade.

1. Run Disk Utility's Repair Permissions. I'm not one of those who thinks this is a magic solution to all problems, and I don't usually run it very often. But I always make a point to run it before starting a major upgrade, just to make sure things are as Apple intended.
2. Make a boot disk backup. Not your entire hard drive, just clone the system itself onto a bootable disk. That way, you'll have an old version of the system, just in case things go south. I have a spare 40GB partition set aside just for the old version of OS X, and I clone to it before a major upgrade. I've been using Carbon Copy Cloner (CCC) to make my clones. Note that CCC hasn't been updated yet for Tiger, so if you're trying to clone a 10.4 disk, you'll need another solution. SuperDuper has been recommended to me, though I haven't tried it yet.
3. Test your boot disk backup. After creating a bootable backup, set it as the startup disk and make sure it works.
4. Back up your critical files. Note that this doesn't necessarily mean your entire hard drive. In my case, I don't back up my applications, nor do I bother with my Downloads folder or my Installers folder (with the exception of a "No longer available" folder of old installers I keep around). I focus on my data, as that's the stuff that's not replaceable if I were to have a crash. I use a 200GB FireWire drive for these backups, and it's only connected and powered on when I'm writing to or reading from it. Other than that, it's off and not connected to anything. I use a free program called Silverkeeper to create backups of these files. It lets me easily set what to back up and when, and seems to work well. You could, of course, just copy them in the Finder if they're all user-level items.
5. Back up custom Unix-level stuff. I'm not a big Unix power user. Still, there are a few things on my machine that I'd hate to lose. My Apache configuration file, for one. My local copies of my sites for another. And the MySQL databases that drive those sites. So I take some time to back these up separately, even though they're on the cloned drive, too. I use the same FireWire disk and Silverkeeper to do these backups as well. If I'm really feeling paranoid, I'll also use MySQL to export my databases to SQL files, and back those up.

Which Upgrade Method to Use

With the preparatory work out of the way, it's time to upgrade. But which method should you use? I have my favorite, of course, but first, let's look at the strengths and weaknesses of each method.

• Upgrade Install: This is by far the easiest method, and it's the one that's recommended for most OS X users. I don't actually dispute that advice -- it's what I tell my mother and friends to use on their Macs, for instance. But it's not for everyone, myself included. Why not? Let's start with the pros and cons, and then I'll explain my thinking.

  Pros	Cons
  Very easy to do	Leaves all third-party extensions installed
  Lowest chance of data loss	Modified applications may not be updated
  Upgrade that requires the least pre-work	System stability may be an issue post-upgrade

  OK, so why don't I run upgrade installs? If you're the type who tweaks OS X (perhaps by applying many of the hints from MacOSXHints.com), you may have issues. As a real-world example, I tried (for the first time ever) an Upgrade install on my PowerBook. When I was done, things seemed OK at first. Then I tried using the Calculator. It worked OK in basic mode, but the different View modes were shot -- all I got was a big blank metal window. After some digging, I figured out that it was because I had applied this hint to enable other Calculator modes. To get Calculator working again, I had to go copy a "clean" version from my G5's 10.4 partition.

  The other downside to an upgrade is that if you like running all sorts of system add-ons, you may wind up with an instable system when all is said and done. Although the upgrade on the PowerBook seems good at first, I soon began having troubles -- the Finder would quit, I occasionally couldn't launch Safari, the SystemUIServer (menubar) restarted spontaneously, etc. To what do I attribute these issues? I'm not sure, but I suspect one of the many extensions that I have in the menubar may have been at fault. GeekTool, MenuMeters, Salling Clicker, Butler, and a couple others I don't recall all lived up there. So even though the OS had been upgraded to 10.4, I hadn't yet checked the extensions for compatibility.

  In the end, the PowerBook wasn't really usable after the upgrade. Despite my disabling the extensions, trashing preferences, etc., I couldn't get rid of all the problems. So I decided to start over fresh, and ran a Erase and Install (more on that later).

• Archive and Install: This is, at least in theory, just like doing an Erase and Install. When done, you'll have your new system and a folder with (most of) your old system, allowing you to somewhat easily retrieve pieces you may need. The old system isn't bootable, due to changes the installer makes to it during the install process -- as one example, the top-level /Library/Application Support folder is moved into the new system. Here are the pros and cons of this method as I see them.

  Pros	Cons
  Third-party system extensions are not copied over	Takes the most disk space
  Automatically creates a backup	Manual work may be required to clean up after the install
  Installed third-party applications remain in place	Takes a long time to run

  For the first time ever with 10.4, I tried an Archive and Install (have a sense now of which method I prefer?). I chose to try it on my G5 at home, reasoning that the time saved over an Erase and Install would be worth it. In the end, though, I haven't saved any time, and I don't think I'll use this method again. Note that this is my experience, and not necessarily a recommendation.

  For me, the troubles started when I told the installer not to copy over my Users folder. My rationale for this is that I didn't want to bring over all my prefs and everything else in the Library -- that's where I install my menubar extensions, for instance, and I didn't want them brought over. So after the installer finished, I created a new "me" (ah, if it were only that easy in reality), and then started bringing over those certain prefs and extensions that I knew worked in Tiger.

  What I failed to take into account, I think, was that certain applications also put their stuff in the user's Library, causing various issues. Motion, for instance, failed to launch. Despite my best efforts with Spotlight and moving folders, I couldn't get it to load. So I pulled out the installer and reinstalled. Motion now launched, but would crash out after the splash screen. What I found odd was that Motion was still version 1.0.1, even though my CD is only version 1.0. So I then manually trashed everything about Motion that I could find, including the application itself, and ran the installer again. This time, Software Update came up and offered the Motion update, and the program ran fine after that. I now have to investigate the rest of my apps -- most will work fine, I'm sure, but some will require some futzing. So in the end, this method isn't going to save me any time over my usual preferred method.

• Erase and Install: Often times called a "clean install," this is my preferred upgrade method. It's also, by far, the most work, so it's not for everyone. When you use the erase and install option, you get exactly what's described: an erased hard drive with a brand-new, factory-fresh OS X installation. Here's what I see as the pros and cons of this approach.

  Pros	Cons
  Finished product matches factory-fresh machines	Requires a lot of pre-install work
  Every application is brand spankin' new	Has the greatest potential for loss of data through stupidity
  It's the quickest install, after the setup work	Requires a lot of post-install work

  Given the infrequency with which major OS X updates are released (roughly yearly, though the pace will be slowing now that Tiger is out, or so Apple has stated), I find the amount of work needed for a clean install to be bearable, given the benefits. However, there is the possibility of doing something really stupid to cause data loss, as you are moving gigabytes of data back and forth. One little mistake of dragging "from" and "to" in the wrong order, and you could find your data gone -- multiple backups are a good thing.

  I like the erase and install option as it gives me a good chance to clear out the cruft that accumulates between major releases. There is something about my machine, however, that makes this process a bit simpler for me: the vast majority of my data files and applications live on separate partitions. My Documents folder holds only my web- and magazine-related stuff; everything else is on a Data partition. Video work has its own partition, so there's nothing in Movies. Music lives on its own MP3 partition. There are pros and cons to partitioning, of course, but ease of major upgrades is one of the biggest pros from my seat.

  After the bad experience with the PowerBook's upgrade install, I took it home for some major surgery. I plugged in the FireWire drive, backed up everything, then did an Erase and Install. After copying my data back over (but not my User environment), reinstalling some applications (and simply copying some others), the machine was ready to go. It is, however, a time-consuming process when you consider the pre-work and post-work required to bring things back to a usable state. Reinstall every third party application, every font, every system extension, etc. This method is not for those who are pressed for time!

  Since the clean install, the PowerBook has been rock-solid stable and I've had no issues with it at all. I may wind up doing this on the desktop machine as well, if I have too much trouble with the applications and stuff brought over in the Application Support folder. Only time will tell...

Conclusion and Recommendations

I appreciate that Apple offers three different installation options -- judging by the new poll on the hints site, it seems that all three methods are getting use (it's early in the poll process; I just posted it an hour or so ago). Based on my experiences, and reading about others, here's when I think each method should be used:

• Upgrade Install: If your haven't modified your stock applications and you don't have a bunch of customized Unix stuff installed, this method should work fine. For most "typcial" Mac users (note -- I don't think most people who read hints fall into that category!), this is the upgrade method of choice. While you don't need to implement every step of my paranoid pre-upgrade ritual, it's vitally important that you back up key data files before starting -- you never know what's going to happen during an upgrade.
• Archive and Install: If you'd like the benefits of a clean install without all of the work, and have the drive space, this is the best option. Similarly, if you have a bunch of Unix stuff that you want to keep around, this is a good choice, as you'll be able to find it in the Old System folder. I would recommend letting OS X bring over the Users' settings as well, and then just cleaning out the unneeded stuff. My method of doing this the other way around doesn't seem to be worth the effort. Again, back up first!
• Erase and Install: If you've got the time, the drive space, and the motivation, I really do think this is the best option. It forces you to take a look at everything you've got running on your machine, and decide just how much of that stuff to reinstall. It does take time, and you want to be very careful that you don't overwrite backup files when moving the data back onto the clean machine, so there are downsides.

This entire piece is, of course, my opinion based on my experiences. I welcome your thoughts, and hope that one point made it through most clearly: regardless of how you choose to upgrade, please back up your important data beforehand!