My Lion “Clean Install” Saga

Updated! Scroll down for the latest.

I upgraded my 2011 Macbook Pro and my 2006 Mac Pro to Lion on Day 1. The MBP has had no problems at all. The Mac Pro, on the other hand, was acting a little squirrelly. For example, after installing Xcode 4.1, iTunes could no longer see my iPhone or iPad when I attached them. Looking in Console, I saw errors about trying to start usbmuxd, but not being able to find it. This is because /System/Library/PrivateFrameworks/MobileDevice.framework had been gutted. Only a few files were left, none of which were the files in question. I could tell when they went missing from looking at the Time Machine backups, and it was the Xcode 4.1 installer’s fault. Xcode 4.1 also crashed on startup, every time. I had also been having some strange goings-on between my AppleTV and iTunes. All in all, I figured maybe a clean install was warranted. After all, I have never done one. I bought this machine new in 2006, and it came with Tiger. I then upgraded to Leopard, Snow Leopard and now Lion.

So I made a bootable Lion disc this morning, booted from it, repartitioned my hard drive and then started the install. It took a while. Finally, it asked me if I wanted to transfer data from another machine or from a Time Machine backup. I selected Time Machine, picked the backup to restore from, and then unselected the stuff I didn’t want to come over. I noticed that ~/Library was not shown, so I assumed that meant it would not be restored. I was wrong. Once I got into the system, I could tell that ~/Library had been restored. That meant that a good deal of the cruft I was hoping to cleanse was now back. So I started the whole process over again.

This time, when it asked if I wanted to transfer/migrate/restore, I said “no.” I ended up at a clean desktop, with no customizations, just like you would expect. I started restoring things from Time Machine, starting with my iTunes library. I have over 230GB of stuff in iTunes, so when it was finished, and I tried to open iTunes, I was a bit shocked when it said it couldn’t open the iTunes folder because it was on a locked volume. Clearly it wasn’t, but something was up. I ended up finding a command that seemed to fix this, but I also noticed that the gid on some of the files was wrong. So I typed

sudo chown -R jgibson:staff .

and shortly thereafter, iTunes was happy. But then I tried to restore my Documents folder. It got through a few of them, and then died with a similar message. Upon closer inspection, I could see part of the problem: the group id (gid) of all of my files in the backup was 501, but my current gid was staff (20). The group 501 doesn’t exist on this system (that’s a relic of a previous OSX version), so I figured I could just chgrp the files and all would be fine.

Except it wasn’t. As root, I was able to change most of the files and directories to have the proper gid, but some just responded with “operation not permitted.” That’s it. No further explanation about why or what to do about it. This was probably the crux of my problem, but I couldn’t fix it, even as root. I scoured the web, looking for answers, and found several, like ensuring the uchg flag is not set. I did so, but the problem persisted.

Next I decided to create a group that had 501 as its gid and add myself to it. Once I did this, the files in the backup volume now showed the new group’s name instead of 501. I thought I was home free. I was wrong.

I tried to change the gid of the files, but couldn’t. I figured I would need to logout and back in, since group changes usually require that on a Unix-y system. I did so. And then I noticed something else. None of the changes I had made to preferences or the dock had persisted. I use a Microsoft keyboard, so I had swapped the values of the Command and Option keys; they had reverted. I had removed Photo Booth from the Dock; it was back. I had changed the settings for Terminal and chosen “use settings as defaults;” the defaults had defaulted to their defaults.  Something else was definitely going on. After spending a few hours trying to find anything that looked possible, I gave up. I decided to start yet another install, and pay close attention to see if there’s something I missed. Perhaps when I first tried to restore from Time Machine, I ended up bringing across some bizarro setting that caused the whole thing. I don’t know. I’m frustrated and concerned and really tired of messing with this stupid thing today. This is not how I planned to spend my Saturday.

Dear Steve, this is not cool. I have no idea what’s going on here. Why the f@*k should I not be able to restore from Time Machine and have it set permissions properly? Why the f@*k should I see messages telling me what happened, but not why or what I might do about it? Why the f@*k should iTunes think it was on a locked disk when it wasn’t?

I will keep updating this post as my latest install progresses…

23:45 Update: I did a third clean install and everything seems to be OK. At first my preferences were not being saved, just like before, but after a couple of logouts/reboots, it occurred to me that the installation DVD was still in the drive. I took it out and rebooted, and now things were working as expected. I can’t prove it, but it seems to me that the system was booting from the DVD, even though it didn’t appear to be. Or just the presence of the install DVD caused the OS to mount the main drive “differently.” I can’t say, but removing the DVD seems to have fixed the problem.

Once I realized that preferences and settings were getting saved and survived logouts/reboots, I started restoring from Time Machine. Rather than trying to do several directories at a time (Documents, Pictures, Music, Movies, etc.) I did them one at a time, verifying things once each restore was complete. I got none of the “operation not permitted” or “can’t copy because you can’t read the files” errors, which was a nice change.

After restoring my Music folder, I updated iTunes to the latest version and ran it. It came up fine, found my library and everything was fine.

Since I’m running up against my draconian Comcast bandwidth cap, I tried to copy things from my MBP where possible, rather than download them. I was able to copy several gigabytes of stuff over, which was nice. I was a bit worried that I wasn’t going to be able to copy the “Install Xcode” package, because when I tried, I did get the “you can’t read this” error. I had set everything to what I thought was proper, but still no go. So I tarred the whole package up and copied that. I untarred it on the Mac Pro, then from Terminal, I went into the package to inspect. Instead of a gid of “staff” it was “admin.” I did a chgrp on all the files in the package, and then ran it. It installed without a hitch and at the end, Xcode fired up just like it should.

The real test was did it destroy the MobileDevice.framework directory like it had before. Covering my eyes, I navigated to the directory and took a peek: everything was there! So then I plugged in my iPad and Xcode saw it, iPhoto saw it and iTunes saw it. The trifecta, w00t w00t!

I did end up having to reinstall iWork ’09 from DVD, and I had to reinstall iPhoto from the app store, which meant a 700MB download, but that was the only one.

So, if you decide to do a clean install for Lion, make certain your backups are good and once it seems to have finished installing, and dumps you to the Finder, don’t believe it! Eject the DVD and reboot.