Drive failure? Yagotta try Knoppix first

Drive failure? Yagotta try Knoppix first

Summary: There's nothing like spending the day indoors during New England's first hot sunny Saturday in 2006.  But indoors is where I spent it trying to get some valuable data off what, for all intents and purposes, is a failed hard drive.

TOPICS: Hardware

There's nothing like spending the day indoors during New England's first hot sunny Saturday in 2006.  But indoors is where I spent it trying to get some valuable data off what, for all intents and purposes, is a failed hard drive.  Corrupted sectors from what I can tell at this point.  It's a 2.5 inch Hitachi drive that came in my Thinkpad T42 -- a system that just came back after having a its completely failed LCD repaired.  There's nothing like getting your system back from the repair shop only to have to send it back.

Among the nearly 100 comments I've gotten so far, many ZDNet readers offered sage words of advice.  But there was criticism too.  Apparently, a power user like me should have known better than to not have my hard drive backed up.  But the truth is that I did have a pretty recent back up. It was just some files that I had created in the last couple of weeks -- notes from recent interviews with Microsoft and Google that I took in notepad.exe, some photos of the Moto Q that I edited with Photoshop (the copy of which is on the bad hard drive so, it wasn't as simple as retaking the photo since I had no photo editor on my other computers), and a couple audio files associated with my pending podcasts of AMD and Navio executives talking about their companies.  I guess I could let those files go.  But what if I could get them back?

A ZDNet reader that goes by the name of Yagotta B. Kidding wasn't kidding when he mentioned that Knoppix (a distro of Linux) was his preferred way for dealing with such nightmares.  Knoppix? At first, it sounded a little too "ixish" to me.  I run Linux here in my lab for Web, database, file, and print serving,  but I'm by no means proficient when it comes to working with hard drives.  But as the day wore on and things got worse and I grew more desperate, Yagotta convinced me to spare the few spins of the hard drive that may have been left and, via email, he sent me the link to download a bootable CD image of Knoppix.  The idea is to plug a big honkin' USB drive into one of the USB ports (got one... a 150 gigger), then to boot the notebook from the Knoppix CD, mount the hard drive, mount the USB-based drive, and copy files from one to the other.  My hard drive was definitely down.  But maybe it wasn't out. If the system files are the ones that are corrupted and the machine can't run off its own hard drive but could run otherwise, maybe it can run off the Knoppix CD and treat the hard drive as a slave.

So, I downloaded the ISO file and burned a CD using the 24x burner that came in the AMD Turion-based Acer Ferrari that I have here, popped the CD into the Thinkpad, powered up and voila.  Not only was I was quite shocked at how well Knoppix recognized the Thinkpad's hardware (wireless networking, display, Trackpoint, etc.), I was shocked when it effortlessly opened up the Thinkpad's hard drive.  Now I know why those all those stolen or lost notebooks are such a problem.  Originally, I thought that password protecting those systems offered at least some degree of security -- especially since they are using the the supposedly-more-secure-than-FAT NTFS filesystem.  But there they were.  All my files. Security? What security. for Knoppix, reading a supposedly secure an NTFS filesystem was like cutting soft butter with a steak knife.  Writing to an NTFS-based drive, on the other hand (as I learned), is a slightly different story. 

There I was, all queued up and ready to go.  I had opened Knoppix windows to the Thinkpad's hard drive and the 150 GB USB drive.  All I had to do now was drag one folder (my XP user folder) to the USB hard drive and I was done.  Right? Well, not quite.  As the CD off of which Knoppix was running whirred to life, the first message I got was that I couldn't copy files to a write-protected destination.  Write-protected? My USB drive? I never heard of such a thing.  To be sure, I unplugged the USB cable from the Thinkpad, plugged it into the Acer Ferrari, opened it up and tried copying something to it.  No problem. As far as XP was concerned, write-protection was not an issue with the USB drive.  So, I plugged the USB drive back into the Thinkpad and tried again. Boom. Same error message. 

To make a long story short, when Knoppix (at least my CD-bootable image of Knoppix) mounts hard drives, it mounts them as read-only by default and you have to switch their read/write mode before you can copy anything to them.  To do this, you right click on the icon for the drive whose mode needs to be switched, select "ACTIONS" from the menu that pops up, and then picked "Change read/write mode."   

This is the point where I started to vent at Microsoft.  It felt so good to know that I was just a click away from getting my data back. But the wind was taken out of my sails when another message came up telling me that the USB drive was formatted for NTFS and that if I attempted to copy anything to it with Knoppix, not only might it not work, I could corrupt the drive.  This is also the same USB drive that has my most recent backup.  This was bad news.  With one drive on the blink, I couldn't risk corrupting my only backup. Knoppix's error message provided one other clue.  It suggested using an NTFS driver called "captive-ntfs" and it said the driver was accessible via the utilities menu.  Why was I venting at Microsoft? Apparently, NTFS isn't exactly an open file system. In other words, Linux developers have had a bear of a time writing a reliable NTFS driver that allows Linux to write to NTFS. Reading from NTFS has been no problem. But writing to it is apparently a different story.  So, a quick note to Bill Gates before he departs Microsoft: how about opening up NTFS? I'm sure Microsoft has what  it thinks are perfectly legitimate reasons for not opening it up.  But, at the end of the day, if it causes customers who want to use your software to have these sorts of problems, then you can see why they might decide not to use your software. 

At this point, I'm wondering if, in the future, I'm better off setting up non-Microsoft partitions on my computers' hard drives, storing all my user data there, and just pointing Windows at that user directory instead of the one that XP prefers to use. Windows can work pretty well with most non-Windows filesystems.   I'm just not sure if it's possible to point to a non-NTFS volume for all user data that Windows keeps (not just my data files, but other stuff stored in the user directories like cookies, caches, etc).  But I can see how, if it was, it might have been one headache avoided.  

I'm not going to bother going into the details of why captive-ntfs may increase the odds of successful writes to an NTFS volume. First, I don't know the answer. Second, it was irrelevant in my case.  As it turns out, even though the error dialog said I could access the captive-ntfs driver from Knoppix's menus, it was nowhere to be found.  Googling "captive-ntfs" turned up several hits of people reporting the same problem along with other hits indicating that captive-ntfs didn't always work.  I'm not sure what it would've taken to get captive-ntfs working.  Maybe another download.  Perhaps of a captive-ntfs binary or, of completely different version of Knoppix (the one I downloaded was 4.something).  I wasn't going to spend the rest of my Saturday figuring it out.  It was time for Plan B.

After thinking about it a bit, I realized that I could take the good backup that was already on the USB drive and copy it to the Ferrari system.  Then, I could wipe out the USB drive and repartition it to be a FAT32 drive instead of NTFS.  Knoppix apparently has no problems working with partitions based on the old FAT32 technology. Even better? I happened to have a copy of Symantec's Norton Partition Magic on the Ferrari. Serendipity at last! The gods may have been smiling upon me after all.  So, first, I plugged the USB drive into the Ferrari system and copied the 16 GB worth of backup data to the hard drive in the Ferrari. Then, as an extra precaution, before wiping the USB drive clean, I made another back up of the data I just copied to the Ferrari by burning it in five chunks to five 4.7 GB DVD+R DVDs (the Ferrari's CD drive is also a DVD burner).  Next, using Partition Magic, I deleted the old partition on the USB drive, repartitioned it as a FAT32 volume, and formatted it. Would plan B work? I'd know soon enough.

I moved the USB drive's cable from the Ferrari to the Thinkpad, single-left-clicked the icon for the USB drive on the Knoppix desktop (a gesture that simultaneously mounts the drive and opens a window to it).  And there it was, 150 GBs of wide-open FAT32 drivespace.  Then, I right-clicked on the USB drive's icon, went to the ACTIONS menu and switched its read/write status.  This time, there were no funky filesystem errors.  It just worked.  For good measure, I right clicked on the USB drive's icon again, picked the Properties option from the resulting menu, reset anything that looked as though it might remotely stand in the way of writing to the drive (there are a few settings here worth looking at).  Then, as a small test, I dragged a small data folder from the Thinkpad's hard drive to the USB drive.  It worked.  Goosebumps.

But, before attempting the big copy (instead of individually dragging files and folders, I was just going to drag my top level user folder over), I thought of deleting a bunch of large files (particularly a lot of old audio files that were used to produce my podcasts) in order to reduce the amount of stress that I'd be putting on the crippled drive. But, I forgot that deleting files involves writing some bits of data as well.  So, why would that be a problem? Because of the two drives connected to my system where Knoppix might want to write "delete information" to -- the CD from which Knoppix was running, and the hard drive that contained the data I wanted to delete -- Knoppix could not write to either. The CD is read-only and the hard drive is an NTFS partition (the kind that Knoppix doesn't exactly agree with when it comes to writing).  Perhaps there was a way to delete those files. I wasn't going to spend one minute longer trying to figure it out.  My choices were to copy my entire user folder, or, spend hours selecting and deselecting files before copying them.  Hours I didn't have. I decided to go for the whole kitchen sink.

By the end of the day, close to 15.5 GB finished transferring to the USB drive. The copying stopped on some obviously corrupted files. I'll live. None of them were mission critical and most of them were ones I had backed up already. All things considered, it wasn't a bad Saturday.  Thanks to Knoppix.  Oh, and the Thinkpad? Just for kicks, I tried booting it to Windows XP's safe mode (with networking) again and for some reason, it not only worked, LSASS didn't crash it.  Not sure why.  Maybe all that copying re-aligned something.  So, as long as it was running, I decided to let Windows scan the drive and correct any errors it found.  It took close a day before it was done.  But, now that it's finished, Windows is running perfectly and there isn't so much as a grind, bop, or peep coming out of the hard drive. Risky business? Probably.  But, if I bring a perfectly working system in for repair, what do I tell the repair guy? (I'm glad I have a photo).

Topic: Hardware

Kick off your day with ZDNet's daily email newsletter. It's the freshest tech news and opinion, served hot. Get it.


Log in or register to join the discussion
  • You're better off keeping that USB drive FAT32

    because as your story indicates, it's not as if NTFS is giving you any additional security anyway. Even if you plug it into another WinXP machine all you have to do is claim ownership of the files and you have full control over them.

    If you need security you'll want something along the lines of PGP, or better yet get a USB drive that has encryption software on it already, preferably one that works on Windows, OS X, and Linux (they do exist).
    Michael Kelly
    • Not switching any time soon...

      I have no plans to switch this USB drive back to NTFS. Not after this fiasco. Thanks.

  • David, you should try using Linux for all of your real work, Windows only

    for testing. If you use rsync, you just need a Unix/Linux server somewhere in the cloud, and you can do daily or even 5 times a day backups. rsync is a very smart program that only copies the changed files and also can use ssh for the transport for security.

    Here is an example command:

    rsync -ave ssh /home/berlind

    Come on Dave, give it a spin, rsync was written by a very smart person. Do you know who?

    You also need to try out gedit for a plain text editor, it is head and shoulders above notepad. It has tabs, and even has multi-language spell checking.

    Oh, you can install rsync under Windows using cygwin.
    • Long term plans

      My plans, long term, are to wipe this system clean, run a stripped down version of Linux.. stripped down to the point that it can run VMware. Then, have some Linux-based VMs and some Windows-based VMs that can run on top of it. There are certain applications that I either (a) can't do without or (b) are the corporately supported applications (in case something goes wrong).

      My problem is time to do it. Also, for the VM scenario, I need a bigger hard drive (might as well get one now, eh?) and much more memory than I have. So, some $$$ there too.

      • The nice thing about Linux + VMWare

        is that when you consider VMWare Server is now free (at least the beta is, and that beta is just as stable as any production grade VMWare I've seen) and any Linux distro can be considered a "server", there's no financial investment necessary, other than obtaining legal Windows software (I don't think you can move an OEM copy over to a VM). I've even heard that VMWare Server will remain free (as in beer) after the beta phase is over, but even if it isn't if you download it now and set up your VMs you can always use VMware Player from that point on.
        Michael Kelly
        • Another nice thing about Linux + VMWare

          If you have a Windows virtual machine on a dual-boot linux/Windows machine, you can use the NTFS support in your Windows virtual machine to mount and safely read/write the native NTFS filesystem from linux. In conjuction with samba, this can allow you to move data
        • Another nice thing about Linux + VMWare

          If you have a Windows virtual machine on a dual-boot linux/Windows machine, you can use the NTFS support in your Windows virtual machine to mount and safely read/write the native NTFS filesystem from linux. In conjuction with samba, this can allow you to safely move data to and from NTFS and linux filesystems.
      • Job Security!

        If all you tech writers switch to Linux, where will you get all the true-life fiascos that you write about so well? The stories just won't have the same zing if there's no real chance of data loss. It's like finding out the car driving off the cliff can only drop 6 feet before hitting a safety net!
  • Just curious, but how "protected" was your NTFS?

    In your blog you mentioned how you were surprised that Knoppix was able to just read your data.

    Did you actually "encrypt" the drive or files? I believe that NTFS encryption is strong and not readable in Linux. Or were you referring to some other sort of password protection, such as an account login, and their 'private' user data?
    • Question

      I ask because I do not know, but if your system is borked and you move a NTFS encrypted drive to another WinXP machine, will you be able to recover your files?

      I mean it would be nice if you could recover an encrypted NTFS drive in Linux too (with the proper credentials of course) but I could understand if MS did not allow this. But if the encryption scheme only allows the original system access to these files then there's a gaping hole in this scheme.
      Michael Kelly
      • Pretty sure you cannot even copy...

        My understanding of the NTFS encryption is that it is tied to a fairly unique (I don't know how many random bits) user ID hash that is local to that installation and user creation.

        In other words, if you re-install Windows, it generates a new userID and your data can not be read.

        Linux/Unix doesn't encrypt a user's /home/ files, either. So any 'root' level user can get in and read data.

        It is too easy to lock yourself out of your own files with encryption, making it a poor tool for the casual user.
        • But you could...

          Conceivably use a tool like "dd" from linux to create an exact copy of the device and restore that back to the original machine. Somebody should try this...I don't have any Windows machines anymore.

          Linux by default does not encrypt filesystems but there are a number of options to allow this, both at the kernel level and in user space.
    • Not encrypted

      I did not use encryption. I think a lot of people are under the belief that by having userids and passwords on their computers, that their data is protected. That's not the case.

      • It's protected on the network

        at least as well as can be reasonably expected in a Windows environment. And that's a good reason to use NTFS on a desktop or notebook. But once someone has the physical disk in their possession all security is gone.
        Michael Kelly
      • Bingo

        You mounted your drive in Knoppix and trusted Knoppix to bypass passwords and sharing status. No, your drive isn't secure unless it's encrypted. Especially from the operating system that "owns" the drive, which Knoppix did in your scenario.

        What NTFS does far better than FAT is avoid software file system corruptions. It's not perfect (obviously) but it is a much safer file system than FAT or FAT32.

        I would still recommend replacing that hard drive. The corruption came from [i]somewhere[/i]. I've heard a lot of horror stories about Hitachi drives in Thinkpads since IBM sold their hard drive division.
        diane wilson
        • The Passwords

          Are handled at the OS level;- not at the data level on the drive. This is a real real real big problem. Any OS that can mount the drive, it doesn't have to be Knoppix, and copy the data is going to bypass any OS level password.

          The answer is to either;

          1. Not store data on the local machine... not likely to happen.

          2. Encrypt the data on the Hard Drive. Again this is not bullet proof.
          Edward Meyers
  • Entire 'Docs & Settings' Relocated


    You can move the entire "Documents and Settings" folder during the initial install of XP using an unattended install floppy. This puts ALL user profiles where you choose. I use a separate partition on the same HD. I used to use FAT32 for this data partition, but I preferred the security and stability of NTFS. (By security, I mean from inadvertent corruption via its journaling capabilities, not security of preventing file access.)

    FWIW, if you're not afraid of the Registry, you can move D&S to another drive on an existing XP install; however, it requires Bart PE and the patience to make 100+ registry edits by hand. I've done it successfully, but I don't recommend it. There are a few keys that are particularly tricky to find.
    • Seems like a perfect opportunity...

      for a utility. Why not move it to some other sort of volume that has some of the reliability attributes of NTFS, that Windows can connect to, but that's more open. Perhaps ZFS? I don't know. Maybe someone else does.

      • It crossed my mind, but

        I am not a programmer, so I couldn't do it. In any event, the driver would have to be guaranteed to load before XP ever needed to "touch" the D&S folder, or I would expect a blue screen. I suppose it is doable, if an enterprising programmer wanted to try!

        As for me, I use Ghost to create monthly full images of my data partition and daily incremental images. These images are stored on a second hard drive and I periodically archive them to DVDs, some of which are kept "off site".

        Working in IT, I know the time and effort that can be expended trying to recover data that wasn't backed up. So, I've made data backup part of my normal regimen.
      • Maybe MacDrive?

        HFS+ is native to OS X, but Linux also has full kernel support. The only thing missing is native Windows support, but MacDrive gives you read/write support in WinXP. MacDrive seems to be more seemless, and can be had for $50 or so.

        Now I don't know if you'd want to put your D&S partition on HFS+ (or if that's even a reasonable possiblity), but it would certainly make for a decent alternative filesystem for that backup USB drive, one guaranteed to work for all three operating systems.

        Though with knoppix you may need to "modprobe hfsplus" to get the driver running. Or maybe not, it may load automatically. I haven't tried it to be honest.
        Michael Kelly