My experiments with multi-boot selection with UEFI boot manager
Summary: How I configured grub as the default bootloader on a UEFI Boot systems
After the two previous posts about installing openSuSE 12.3 and Fedora 18 on my sub-notebooks with UEFI BIOS and Windows 8, my intention was to continue with the same theme a third time and write about Ubuntu.
However, I have decided not to do that quite yet, first because I'm getting pretty bored with it, and second because Ubuntu 13.04 is supposed to be released at the end of this month, so I think I will wait until we are closer to the release date before writing that.
Instead, I am going to pick up a thread that I mentioned in each of the previous UEFI Boot posts and I have promised to get back to "real soon now".
This one is going to be somewhat more technical, and will have a lot less pretty pictures included, so you might want to get a strong cup of coffee before starting, if you are determined to stay awake until the end. For the impatient readers, I will start with a summary:
- The UEFI boot systems I have always boot the file bootmgfw.efi from the EFI Boot partition, come hell or high water.
- All of my attempts to change the configuration to get it to boot something else have eventually failed.
- The UEFI-compatible Linux systems I have installed include the grub2-efi boot loader. Getting the computer to boot this by default is the trick.
- The simplest way to permanently achieve this is to copy the Linux bootloader files to the default bootloader directory, and then rename the Linux bootloader to bootmgfw.efi
That's it, in a nutshell. All the gory details follow, bit I will say this as clearly as possible first. Mucking about with the boot process on your computer can be dangerous. If it all goes pear-shaped you may have to reload from scratch.
The details and procedures which I present here are as complete and accurate as I could make them, and work on my HP and Acer systems, but there may be significant differences in UEFI implementation, so don't be surprised if things look different or work differently on your computer.
Oh, and by the way, based on the information I have read about UEFI boot, Linux, and Samsung computers, even I would not try this on one of those, period.
So here come the details of my experiments.
If you have a UEFI Boot system, and you followed the details of either or both of my preceding posts, when you rebooted your system after installing Linux you probably got a surprise - no sign of Linux. Just the same old ordinary Windows 8 boot.
No Grub boot menu, no operating system selection, no Linux boot. Just Windows.
There are two related "problems" at work here - first, the whole theory of booting has changed with UEFI boot, and second the current state of UEFI Boot Managers is very poor (in my opinion, and based on my experience with HP and Acer UEFI systems).
I will explain each of those problems in more detail separately.
The boot process has been radically changed with UEFI boot. As I understand it, the process is now broken into two pieces, one is the "Boot Manager", which finds, presents and controls what can be booted, and the other is the "Boot Loader", which actually does the booting (load and run) of whatever is selected by the Boot Manager.
By the way, if I have this completely wrong, please feel free to correct me in the comments - I'm sure there will be plenty of "corrections" anyway, many of which will themselves be completely wrong, nothing new in that.
So when the computer is initially powered on, the Boot Manager looks around and decides what candidates for booting are available, and which one has first priority. Unless it is interrupted (by you pressing the boot selection key) it will then try to pass control to whatever boot loader it decides comes first.
The Boot Manager which came on my HP and Acer systems is pretty grim, to be kind.
It is difficult to configure, it is prone to tossing whatever configuration changes you might make and going back to its original default configuration, its on-screen presentation looks like something that came out of the original War Games movie from 1983, and well, it is just generally not very pleasant. The best that can be said for it is that it gets the job done, more or less.
If you take the time to find and read the documentation for the UEFI boot system in general, and the Boot Manager in particular, it sounds like it could be a lot better than that. It could have a Graphical User Interface, it could present wonderful classy menus of boot candidates, it could have a very spiffy configuration center, and so on.
However, the ones I have experience with are really good at exactly one thing - starting the Windows Boot Loader. Well, two things... they are also really good at throwing away everything I try to do to configure them, and returning to the first thing they are really good at.
I wrote not long ago about the rEFInd boot loader, which does a lot of the things which I just mentioned, but it has to be installed as a separate package after installing Linux, and it is not easy to get working with Secure Boot enabled. My intention here is to show how I get dual-boot/multi-boot working with a minimum effort, using only what gets installed with a standard Linux distribution. So I will not go into any more detail here about rEFInd.
The Windows Boot Loader is also supposed to be capable of booting multiple operating systems. I have tried to get it to do this with Linux installed in addition to Windows, and I have failed miserably. I have read everything I can find on the Internet about configuring it, and I have tried to use my knowledge from configuring the boot loaders in Windows XP, Vista and Windows 7.
I have tried using BCDedit and easyBCD, and I have never managed to get it to do anything other than boot Windows 8. I have gotten it to present a graphical "selection" menu with pretty buttons to click for Windows 8 and whatever else I am trying to add, but none of the buttons other than Windows 8 ever actually works.
Perhaps I am just too dense to understand it, and if anyone would like to enlighten me, with a complete and functioning example, please feel free to do so. Oh, make that a complete and functioning example which includes booting Linux, because being able to "multi-boot" several different versions of Windows is not interesting to me, and doesn't seem to be particularly difficult...
So, in my case there are only two ways to boot Linux - either interrupt the Boot Manager, by pressing whatever the Boot Selection key is on your computer (ESC, F9, F12 or some such), or install some other Boot Loader and convince the Boot Manager to start that by default instead of the Windows Boot Loader. If you take the first course, and interrupt the Boot Manager, you get something like this:
This screen shot was taken from one of the system where I have openSuSE and Fedora both installed in addition to Windows 8. I can move up and down with the arrow keys, and then press return when I am on the one I want to boot. I wrote something very similar to this in a shell script in about 1982, and I am shocked and disappointed to find it here... If only I had thought to patent it at the time, I could be making a fortune today! Or not...
Well, whatever, it is what it is, and I find it to be unpleasant and inflexibie, so I need to figure out how to replace it with something I like better. To help me do that, I will use the Linux efibootmgr utility to list and edit the boot manager configuration. Here is the list from one of my systems:
The first few lines are what I am interested in, it is a list which corresponds to what the Boot Manager presented in the boot selection list. I am not going to spend a lot of time on things that I have learned the hard way don't work here, I will just say that there are a lot of options for the efibootmgr program which allow you to add and delete items and change the boot order (use the -? option to list them all), but most everything I did, which appeared to work when I listed them again, ended up being thrown away either during the next reboot, or the next time Windows was booted.
So the only really important bit of information here is the path of the Windows Boot Loader, which is the object that the UEFI Boot Manager insists on booting by default - "\EFI\Microsoft\Boot\bootmgfw.efi". I'm hoping that if I replace that file with the boot loader of my choice, I will be able to trick the Boot Manager into doing what I want.
The paths used by the Boot Manager are relative to the EFI Boot Partition, which is /dev/sda2 on both of my systems, and which is mounted on /boot/efi under Linux - the actual partition may change, but the mount point will always be the same. Linux installations which are UEFI Boot compatible will create their own directory to contain their boot loader binary and configuration files, so on my system I have /boot/efi/EFI/opensuse and /boot/efi/EFI/fedora. Their contents are:
The important things there for my purpose are the grub EFI binaries, which you can boot directly if you don't have Secure Boot enabled, and the shim EFI binaries, which are what you have to boot if you do have Secure Boot enabled. Assuming that I want to boot the openSuSE version of grub by default, what I do next is copy the contents of this directory to /boot/efi/EFI/Microsoft/Boot/. Please note that I said copy, not move, because you don't want to destroy your existing boot configuration, so if things go wrong you can always recover by using the boot select key. Also, you don't absolutely have to copy everything, but it doesn't hurt to do so; if you have gotten this far and are still awake, you should be able to figure out which bits you don't need without too much difficulty, at least by trial-and-error if nothing else.
Once I have the openSuSE boot files in the default boot directory, all I have to do is rename things so that the boot manager runs grub rather than the Windows Boot Loader. I try to be careful and leave myself a way to recover from misunderstandings and mistakes, so I first rename the file for the Windows Boot Loader from bootmgfw.efi to something like bootmgfw.ms.
Then I rename shim.efi to bootmgfw.efi and I am done. The shim file will boot whether Secure Boot is enabled or not, but if you know that you are going to have Secure Boot disabled, and you want to save one step and simplify the boot process a bit, you can rename grubx64.efi to bootmgfw.efi instead.
But be aware, if you do this, and then later enable Secure Boot, the next time you try to boot the Boot Manager will notice that something is wrong.
What it does seems to depend on the specific system on which it happens; it can range from just using a fall-back boot image which will start Windows 8, or dropping into the boot selection menu, or restoring a copy of the original boot files, or it can even go so far as to run what seems to be a very large and very complex Windows recovery procedure in order to get back to the factory boot configuration.
When you reboot after making these changes, you should get the openSuSE grub boot loader, which will give you a simple graphic menu where you can choose to boot either openSuSE or Windows 8. Hooray!
There are just a few loose ends that I would like to clean up.
For the vast majority of people, who are trying to set up a simple dual-boot configuration with Windows and Linux, this procedure will work equally well using either openSuSE or Fedora. In my case I want to set up a multi-boot configuration with several different Linux distributions, so I always use openSuSE as the base because I have found that its grub is more flexible, and at least with Secure Boot disabled it is able to boot other EFI images (such as Fedora and Ubuntu) and even non-EFI "traditional" boot distributions such as Linux Mint.
Please remember, mucking about with the boot configuration is dangerous. Make sure that you have complete backup and/or recovery media and procedures on hand before you try this.
As I have mentioned a couple of times, there seem to be differences between vendors in the details of the implementation, checking and automatic "repair/recovery" procedures, so don't expect that I have covered all the possibilities here, and don't be surprised if at some point you try to boot and rather than starting grub your computer starts showing that blasted rotating circle of dots, indicating that Windows is doing something whether you want it to or not.
If that happens, what I would do is not try to stop it, just let it finish, and you should be able to go back and "pick up the pieces" or just start over again.
Kick off your day with ZDNet's daily email newsletter. It's the freshest tech news and opinion, served hot. Get it.
Talkback
Please explain, for a simpleton (and lazy one, at that)...
Let's say I find an outstanding deal on a new laptop, except that it has, of course, win8 installed. What's wrong with doing things the old-fashioned way: "nuke"ing win8 and turning the machine into a solitary-booting Linux machine. At the risk of repeating myself, I obviously don't care if I NEVER see win8.
Keep up the great work, and
Warmest regards...
Nothing wrong with that
Thanks for reading and commenting.
jw
It also has to be noted...
I know you don't want Win8 but for those who do...
Thanks for another great article
Thanks for fighting your way through the FUD for us Jamie, keep up the good work.
toddbottom3...quit using the same old PLOY over and over again
Maybe its you who dosen't fully understand the whole problem.....have you thought of it that way?
You and Loverock Davidson & Owllllnet1 always seem to have some sort of problems or negative attitudes about anything Linux.....care to share why?........ oh maybe you can compile a answer between the three of you................
I think in this case...
For example, they keep referring to it as 'Microsoft's' Secure Boot when in fact, it's a standard feature of UEFI which was created by a collection of companies starting with Intel. Microsoft isn't even the first company to use Secure Boot (Apple is).
Why this is an issue now is that Microsoft decided (for better or worse) that they wanted to use Secure Boot on new OEM Win8 systems and Secure Boot needs a key that's registered with the OEM's UEFI installation. Once you turn on Secure Boot, the system will only allow booting into systems with a registered key. Since Linux isn't a corporation - no one wanted to spend the time and effort to get a key - and even if they did, the odds of getting OEMs to include it is low since there's little financial motivation for them to do this. So they decided to beg Microsoft to let them piggy back on Microsoft's key - which Microsoft actually did!
The FUD that's being spread is the notion that if you buy a Win8 laptop you *can't* run Linux on it. That's patently false. You can run anything you want on it - you just can't leave Secure Boot turned on (unless you have one of the shims installed). Once you turn Secure Boot off, you can do pretty much anything you want. If you want to install someone really weird or old, many Win8 devices actually include a "Legacy" mode that essentially simulates the old BIOS.
At the time, Microsoft made it clear - any laptop that wants to be certified for Windows 8 *must* include at least the ability to disable Secure Boot. The fact that so many Linux people were suggesting that laptop makers would just go ahead and not allow disabling Secure Boot is the core FUD. It simply wasn't true and was speculation assuming the worst possible case as the likely one - even when Microsoft said that wasn't going to be the case.
Finally, even Windows 8 can run out of Secure Boot. The copy that comes with the laptop will most likely be an OEM version, and so will require Secure Boot, but retail copies don't. So the only real criticism that's fair is that people buying Win8 laptops will have to buy another copy of Windows 8 if they want both Win8 and Linux in dual boot (at least, if they want to do it the simple way).
The comment is backhanded
Several Important Issues
toddbottom3 and others are right, there has been a lot of FUD about this, and I find that disappointing. It is new, and it is radically different, and good, clear, simple documentation about it is not exactly plentiful or easy to find. It is out there, but you have to look fairly hard for it. I got some very good pointers from Adam Williamson (and not for the first time, thanks Adam), and there have been several other extremely useful comments posted to my blog, I appreciate them all, and learn from many of them. But in my opinion, there has been far too much doom and gloom written about Secure Boot and UEFI - or, as I like to say, there has been far too much heat and not enough light.
zafrod's question is a good one, and TheWerewolf's answer is correct, but perhaps goes a bit too far, or omits an intermediate solution. If Secure Boot and keys is your problem, you can dual boot by disabling only Secure Boot, but still using UEFI boot, you don't have to go all the way back to Legacy Boot. Adam has also said that using Legacy Boot rather than UEFI actually misses out on some of the other features and advantages of UEFI. It is undoubtedly true, though, that going back to Legacy Boot really makes it possible to install any other operating system that you could on a non-UEFI system, including various Linux distributions which don't support UEFI Boot yet.
Finally - and the reason I am writing this comment - in my opinion, based on my experience, GPT disk partitioning is simultaneously as big of a win and as big of a problem as UEFI and Secure Boot, and can be much more insidious (less obvious). You can very happily try to install a Linux distribution which does not really understand GPT partitioning and not get any errors, then when you try to reboot you discover that it got the partitions wrong, and it overwrote one or two of your existing partitions by mistake. THAT is bad news - whereas UEFI and/or Secure Boot problems generally result in either installation failure or boot failure, but no other damage to your existing system.
Thanks to all for reading and commenting.
jw
toddbottom3: "Thanks for another great article"
"there has been a concerted effort in the Linux community to spread the FUD and blame every multi-boot issue they face on Secure Boot."
Please tell us all exactly what Linux is. Is it the kernel? The operating system, including the Linux kernel? Is Android Linux? And one caveat: Commercial GNU/Linux vendors Red Hat, SuSE (and its parent, Attachmate) and Canonical have all taken a no nonsense approach to secure boot and use a key generated on their behalf by Microsoft. Also note that this includes their FOSS non-commercial distros Fedora, OpenSuSE and Ubuntu (and its derivatives), respectively.
j.a.watson, since you generally agreed with toddbottom3's comment, you can feel free to respond to these questions as well (or not). And since your blog falls under "Jamie's Mostly Linux Stuff", it really wouldn't hurt for you to tell us what you mean by "Linux". And from your blog's home page:
"Various thoughts and adventures, including but not limited to Linux, assorted bits of hardware new and old, and occasionally Windows XP/Vista/7."
You might want to add Windows 8.
Easier way to configure GRUB
http://www.youtube.com/watch?v=eAnlhkbMang&feature=share&list=PLk2sjg_-F-McRbCBoVRkP1sYMbmDf6zJM
If you run Boot-Repair under Ubuntu, GRUB will be configured to work with Windows 8 & Ubuntu. This doesn't disable secure boot, doesn't require hand-editing of GRUB config files and doesn't need any manual file renaming. Just keep 'ubuntu' as the default entry in UEFI Boot Manager to launch GRUB.
The specifics about Boot-Repair are here, in case you want to skip to the end. http://youtu.be/_cEwj8bBBC4?t=3m35s
Not in The REAL World
The presenter in these videos made no reference to this - he just said "I installed Windows first, then booted it and it worked, then I installed Ubuntu and made its grub the default boot loader". Well, I did that too - in fact, Fedora, openSuSE and Ubuntu all did that when I installed them, but the next time I booted, or at least the next time I booted Windows 8, it happily rewrote the UEFI boot configuration to restore it to the Windows boot loader. If I could figure out how to stop it from doing that, I assure that you I would post it here and I would most certainly NOT be writing instructions which included copying and renaming several obscure files.
As for Boot-Repair, what it does was by far the least of my problems. In fact, if simple dual-boot with Linux and Windows 8 is your objective, both openSuSE and Fedora 18 get the configuration for that right when they are installed, it is absolutely not necessary to edit the grub configuration files or run any kind of boot-repair utility. It is only because I want to multi-boot several different Linux installations and Windows 8 that I need to manually edit the grub configuration files.
Thanks for reading and commenting.
jw
Fun with workarounds
I believe the "rewrite the bootloader" issue is one of two problems ...
1. Windows 8 changing the UEFI boot variable so the Windows boot loader is the top entry. That can be fixed by changing the order in Boot Manager. I haven't seen that happen very often, but it could happen after a Windows Update. [Note: Windows Update could also restore the .efi files you copied over in the workaround]
2. The vendor (HP, Acer, etc.) has some BIOS mechanism to restore the boot order. This isn't a great option for someone trying to use their own OS. In this case, the workaround you described avoids that problem by tricking the recovery widget into launching grub/shim instead of bootmgfw.efi (Windows loader).
Thanks for putting so much focus on UEFI & Linux. I think you're helping people understand how things actually work versus how the rumor mill assumes it works.
Thanks and Congratulations
I believe that I have seen both of the situations you mentioned, although the second is the more frequent and more aggressive of the two. My HP Pavilion is absolutely rabid about checking both the UEFI boot configuration AND the contents of the EFI boot partition. To this end, it actually has a complete copy of the \EFI\Microsoft\Boot directory squirreled away under \EFI\HP, and when it notices certain changes in the EFI boot disk, it seems to simply wipe the current contents and copy it from the HP directory again.
On the other hand, at least once I have seen the Acer go through something which was pretty obviously the Windows "Recovery" procedure when I made a change that caused a boot failure. The dreaded rotating dots came up for a long time, then the even more dreaded 1% complete... 2%complete... and when it was finally done, it looked like it had completely rebuilt the EFI boot partition.
It seems to me that the UEFI Forum and some of the key members have done a pretty good job of providing information about how UEFI Boot and Secure Boot "should" work, but the big problem now is that most system vendors provide very little or no information about how their specific information "really" works. The most extreme example of this is Samsung, of course, but my experiences with HP and Acer show this as well. I seems like they have put so much effort into making sure that it doesn't get "broken" that they have made it very difficult to make any changes. To make matters worse, a lot of the things they do "behind the scenes" ends up looking like magic to the user - or simply looking like Linux doesn't work with UEFI Secure Boot.
Thanks for reading and commenting.
jw
I have the same problem as 'zafrod'...
I would like to try out different distros. My 'downloads' page has several "xxxxxx.iso" distro files which I'd like to load on my laptop, but I can't seem to find A program to convert these different versions to a bootable USB, so I can load them onto my no-DVD laptop.
What is your "go-to" program for converting a distro file to USB, once you've downloaded it?
________________________________________________________
@Another View; @Restricted_access:
How about cutting toddbottom3 some slack. I've read and re-read his comment several times, and can find nothing to fault him for. Now, if you're TRYING, you might accuse him of being a conspiracy theorist because of his statement
"... Unfortunately, there has been a concerted effort in the Linux community to spread the FUD and blame every multi-boot issue they face on Secure Boot."
Then again, place yourself in my and zafrod's position today, as opposed to nine months ago: then--no problems. As JAW would say, "Everything was right with the world." Today? What do you think MY answer is to someone who asks ME why they can't simply dual-boot a Linux distro just like in the old days. In pretty much the same words, it'll closely mirror TB3's comments.
We're all on this spaceship together. None of us will get off alive. Remember that, please.
Warmest regards to all...
Convert ISO to USB
- The "simplest" case is if it is a "hybrid ISO" image, and you already have a running Linux system. You can then just "dd" the ISO file onto the raw device of the USB stick. Quite a few distributions are using this format now, including openSuSE, Fedora, Debian and Linux Mint, and certainly others. If you don't already have a Linux system to use to do the "dd", things get a bit more difficult. There are various Windows programs available which can be used to achieve the same thing, but I am not familiar enough with Windows to name any of them.
- The next simplest case is if the ISO image is compatible with the wonderful "unetbootin" utility. There are both Linux and Windows version of this; it is available from sourceforge.net.
- Some distributions require that you use their own "USB creator" utility. The best known of these is Ubuntu, I have never managed to create a bootable Ubuntu USB drive with anything other than Ubuntu's own "Startup Disk Creator". I have seen notes from others saying that this or that utility works with Ubuntu, but I have never actually found and used one myself.
- Some distributions just can't be converted to a bootable USB stick, at least not in any way that I have found. If I am absolutely determined to use one of those, I dig out my dusty old USB DVD/CD-RW drive and burn it to a disk.
To answer the original question, my favorite method is "dd", but that is because I am a hard core Linux/Unix person, and I have been using "dd" for all sorts of things for over 30 years now. (who remembers using conv=ebcdic?). For distributions which don't work with "dd", my next favorite is unetbootin, that way my go-to converter for a long time before hybrid ISO format caught on.
Thanks for reading and commenting.
jw
why not just have seperate hard disks with a power button on each?
or just reef out the sata cable as needed...
this UEFI thingy means I cannot get into bios on a gigabyte mobo, when a gigabyte gpu is installed...
unless I invite rooters in by disabling secure boot
crrraaaaazy maaan, craaaaazy