How to migrate almost half a million messages to Office 365 (without tears)

How to migrate almost half a million messages to Office 365 (without tears)

Summary: Our own David Gewirtz moved 20 gigabytes (about half a million email messages) using an online service that made the whole process much easier than you'd think. Here's the story.

SHARE:
20

Back in March, I described my move from my regional Exchange hosting provider to Office 365. I explained why I chose Office 365 over Google Apps, my experiences with Microsoft's support, and even my ongoing dismay at the dismal quality of hold music customers are subjected to by cloud service providers. Here's the entire DIY-IT project page on that project.

But what I didn't do was tell you how I moved 20 some-odd gigabytes of email between the two Exchange providers. That's what this article is about.

Here was the scope of the problem. There were only two mailboxes: mine and my wife's. On the surface, that might seem like an easy migration. But we'd used our old hosting provider for over a decade and we get a lot of messages (and I get an absolutely mind-boggling amount of mail daily).

We'd also gotten into the habit of using our email as the "database of record" for everything we've done, from online purchases to the details of business relationships. As a result, we file everything in our various email folders, and delete little besides spam.

And we had all that, going back to 2002. All told, we had 480,000+ messages to move, along with all their associated attachments.

This is a non-trivial task. One way would be to go into Outlook, establish a connection with both services, and copy all the messages over, folder-by-folder. I've done this before, on a much smaller scale, and even on a much smaller scale, it's incredibly error-prone, Outlook doesn't play along with any degree of enthusiasm, and there are both transfer failures and corrupted messages.

It's ugly. I needed a better way.

And that's when I stumbled on YippeeMove. Yeah, the name alone almost put me off, but I needed a workable solution. And, believe it or not, that solution cost me $30 — $15 each for my wife's mailbox and mine.

I'll tell you this, the folks over at YippeeMove earned that thirty bucks, not even because they moved all my messages, but because I was an absolute Nervous Nellie during the transport process and I probably bugged them 20 times during the week of the move.

Let's talk about the details first. They can move from most email providers to most other email providers. The key is that both providers need to have an IMAP interface available.

What you do (and this, too, was a bit nerve-wracking) is you go to the YippeeMove Web site, log in, and enter your login details for both your source and destination email accounts. You enter in a few more details (no more than you'd need to connect to your email on a smartphone), and then kick off the job.

This is not a fast process. It took a little over a week for the messages to get from provider to provider. It is, however, a hands-off process. Once you click the "go" button, that's all you have to do. Take a week's vacation, and when you come back, your mail will have migrated over.

Of course, I couldn't leave it there. I had to check with them every few hours. Whenever I saw a slow-down in the message transfer rate, I had to ask them what was wrong. When I saw everything stop for a day, I bugged them incessantly, before eventually realizing I'd changed the password on the destination account in the middle of the transfer.

There are also security implications. You're giving you email account credentials to an unknown third-party company. On top of that, you're letting them see every email message you may have. Hmm... it strikes me that YippeeMove might be an ideal front for the NSA, Chinese hackers or the Russian mob.

I was, indeed, worried about this obvious security issue. I did change my login credentials on both sides as soon as the transfer was completed, but the fact remains that they did get to touch (and who knows what else?) all my historical email, going back a decade.

But I made a decision here. The decision was that I could either recruit some help, or I'd lose my historical email, because I never would have had the time to move 480K+ email messages on my own, by hand. I decided it was more important to retain the functionality of that archive, and — besides — my main email store didn't have all that much that's particularly confidential (my government email accounts are in completely separate systems).

It was a risk, it took a leap of faith, but it worked out fine. I was a nervous wreck during the entire transfer week, but now I look back on it and smile. The fact is, for all of thirty bucks, the service offered by YippeeMove (wacky name and security concerns notwithstanding) is a heck of a deal and worked out just fine.

Oh, and in the three months since the move was completed, I've seen absolutely no evidence that any of the data I moved was compromised in any way. The company seems completely to be just what it says it is — an email migration machine.

If you have to move a massive pile of email, YippeeMove is probably the best thirty bucks you'll ever spend.

Topics: Cloud, Microsoft, Networking

About

David Gewirtz, Distinguished Lecturer at CBS Interactive, is an author, U.S. policy advisor, and computer scientist. He is featured in the History Channel special The President's Book of Secrets and is a member of the National Press Club.

Kick off your day with ZDNet's daily email newsletter. It's the freshest tech news and opinion, served hot. Get it.

Talkback

20 comments
Log in or register to join the discussion
  • question is not how..

    ...but why.

    and answer does not exist.

    make article "how to finally move to thunderbird". now that would be usefull.
    ljenux
    • So Thunderbird is a server application?

      Learn something new each day, I was under the impression is was simply client software.
      thekman58
  • Office 365 Migration

    Did you even try the Office 365 Migration? It works great. It could be simpler but it works. The biggest challenge I've had so far is with the initial connection. You need to have a valid SSL cert to make it work.

    The diagnostics are on another Microsoft site but if your going to an account that includes support Microsoft will walk you through it.
    jdb5
    • Nope.

      Never tried it.
      David Gewirtz
  • Maybe I missed it, but...

    It sounds like you could just create your new office 365 profile in outlook and then import the entire PST file from your old account into it. Probably would have taken you all of 3 minutes.
    mrefuman
    • Not really.

      It doesn't work that way. It would be nice if it did, but nothing that size works in five minutes.
      David Gewirtz
    • IMAP

      YippieMove.com requires IMAP. They do not migrate via POP3. (PST)
      Alice Is
  • Just had a 365 guided tour yesterday by a vendor ....

    ...we're considering moving our email system to 365 not to mention some of the other features (office, lync, sharepoint, etc). Very impressive. I've dabbled with google apps here and there and it really comes down to this: Enterprise level PROFESSIONAL solutions (MS) vs. hobby/home ones (Google).

    Anyone trying to make a comparison between the two is essentially either (1) not familiar with enterprise needs and/or (2) only "plays" with their computer, i.e. using it as a Nintendo substitute.

    Anyone still buying MS office licenses: You're C R A Z Y. Jump on 365. Soon. Don't wait.

    Google apps and the other 12 "competitors" - they only compete as long as a customer doesn't see 365. Then there is no competition anymore. It really is THAT much better, that much more mature, that different.

    It's like going back and forth between Paint and Photoshop, or Pacman and Warcraft. There is no comparison. I really don't get what some of these other snippy comments are above - obviously written by 12 year olds that don't have 300 screaming, whiny users to contend with every day.
    rock06r
    • It seems

      You are still under the PR haze.. it will fade away, eventually. Let's hope you will stay just as satisfied with the product. But then, "nobody got fired for buying IBM". :-)

      Or, those 300 screaming, whiny users will just eat you alive....
      danbi
      • It seems

        You are still under the PR haze of Google, which tells me you're one of those people that stay Google as long as you don't see 365.

        Hopefully your 300 users don't see it either, otherwise you just may be that guy that got fired for buying Google... ;)
        William Farrel
    • Nope. Just say no to the cloud.

      I cant stress enough how bad an idea it is to just let ones self gradually be sucked into the whole cloud scheme.

      Slowly but surely, step by step...
      Cayble
  • On privacy

    I wonder why you were so worried about the privacy of your e-mail so much.. since both your old and your new e-mail storage was at some third party's facilities, and well known at that, why would you were so much worried that another third party would 'touch' it? It is as if you have your "private" data in Facebook and are too much worried that Google might eventually see it, if you migrate...
    danbi
    • part two

      Well, someone has to fix this "anti-spam" filter. Or we will just stop posting...

      The well known bit is important. It is guaranteed, that if someone wants to tap on your "private" data, will first seek to tap the "well known" services, for example gmail, outlook, yahoo. Then, most e-mail service providers do not support encrypted e-mail transport (SSL or STARTTLS) -- this makes it trivial to intercept all your communication while in transit. To my knowledge, this has been happening the US for as long as there was public Internet over there.

      While I have well over twice your amount of e-mail (and for over twice the period), I have come to the conclusion few years ago, that nothing beats running your own server to store your own e-mail. 50GB is not so much storage and you might even host it somewhere, if you don't want to deal with the "hassle" of supporting the hardware. But, to each their own.

      But in any case, I seem to miss one piece -- where is your e-mail backup? What will happen, if the Office 365 service suffers an outage and you lose all your e-mail? Their T&C certainly make you agree to such outcome.
      danbi
      • Privacy...

        I knew both companies at each end. I didn't know YippeeMove. The hosting company was recommended a decade ago by one of my most trusted colleagues and I've worked with them for years, way before privacy became such an issue. And I know the parameters of Microsoft's privacy and feel comfortable with them within that context.

        The company moving the data I didn't know, so I was somewhat more concerned. They did a good job, though and I've seen no evidence of any issues since.
        David Gewirtz
      • I have never used office 365 exchange

        But as far as I know it is just Exchange in the cloud, meaning you can use outlook to connect to it, outlook by default runs in cached mode, which means it downloads the emails to the local computer. Outlook 2010 and earlier download everything, Outlook 2013 downloads the last 12 months (which you can change to either less or more up until all months).

        It downloads it to a local ost file, which could easily be copied (in outlook) to a pst file if another backup is needed. This is Exchange we are talking about, with DAG and multiple copies of databases in multiple locations, there is no way one would ever have to worry about loosing data.
        sjaak327
        • Have you ever tried to copy 500K messages in PSTs?

          Been there, done that. Created the web site. It is an incredibly slow and tedious hand-task. Try it.
          David Gewirtz
          • 500k

            Is nothing. You do know that there are (free) tools that negate the manual process ?
            sjaak327
  • My question here

    Would be, did your old provider refuse to execute a powershell oneliner and give you the two resulting pst files ?
    sjaak327
  • Is your PC running slow???? You can get amazing speed increase....

    by cutting your email message archive by two orders of magnitude....
    Just have to wonder who in the world needs to retain ecen 2,000 old messages.... mail junkie????
    Willnott
  • So, um, claiming to be concerned about SECURITY...

    you still put half a mil mails into potentially public access???? Really???
    Willnott