Secrets of BIOS, Grub, and Triple Boot Servers


Have you read: Rodger’s Very Simple Dual Boot Method?

If you haven’t already, do check it out. One advantage: no Master Boot Record (MBR) modifications! Another advantage: it allows all disks and operating systems the option to be completely independent of each other. You can remove any disk, insert it into another machine, and it will boot perfectly fine. The disks are no longer married to each other. But mainly, this post won’t make any sense to you, otherwise.

A few months ago, I used this method with only two operating systems: Redhat 5.5 and Windows 7. Then recently, in the same server, I installed Redhat 6.4 onto a third disk. But suddenly I had an awful time getting all three to work.

Symptoms:

Redhat 6.4 booted Redhat 5.5 perfectly. But when I booted Windows from the Redhat 6.4 grub.conf file, I got the error:

File:  \BOOT\BCD
Status:  0xc0000001
Info:  An error occurred while attempting to read the boot configuration file

Redhat 5.5 booted Windows 7 perfectly. But when I tried to boot Redhat 6.4 from the Redhat 5.5 grub.conf file, I got the error:

Error 2: Bad file or directory type

So, I could only boot two of the three operating systems at one time.  To make things work, I’d have to be changing sata cables. That really defeated the purpose of a triple boot.

This was really mysterious, and no one on the newsgroups had any idea how to fix it. Mere mortals would have given up and implemented some kind of work around. 🙂 But here is how I ultimately overcome the errors and got all the three disks to boot cleanly. 

Trusted Boot:

When I first started getting errors, I noticed that the entry for Redhat 6.4 looked different from the existing entry for Redhat 5.5. Turns out that it was using Trusted Boot technology.

Trusted Boot is used to make servers more secure. It requires the package, tboot.
The grub.conf entry for tboot will look different from before, as you can see from this example on the Fedora website:

title Fedora (2.6.38-0.rc5.git1.1.fc15.x86_64)
root (hd0,0)
kernel /boot/vmlinuz-2.6.38-0.rc5.git1.1.fc15.x86_64 …
initrd /boot/initramfs-2

title Fedora w/ tboot (2.6.38-0.rc5.git1.1.fc15.x86_64)
root (hd0,0)
kernel /boot/tboot.gz logging=vga,serial,memory
module /boot/vmlinuz-2.6.38-0.rc5.git1.1.fc15.x86_64 …
module /boot/initramfs-2.6.38

Note that the traditional grub.conf entry had the commands:
root
kernel
initrd

But using tboot, the commands are:
root
kernel
module
module

As I investigated, I noticed that tboot.gz is actually a compressed file. I figured that tboot might be the cause of the problem. After all, tboot was on Redhat 6.4, but not 5.5. So, I completely reinstalled Redhat 6.4, making sure not to install tboot, and got a traditional grub.conf file. See this link to see where tboot is found during the install process. Unfortunately, reinstalling without tboot did not solve the problem.  😦

Clearly Identify Your Drives:

Hint: If you create different numbers of partitions on your hard drives, it will make identifying drives -much- easier. If you also use different brand names of hard drives, it will also be much easier to identify them in the BIOS. Currently, I have one Hitachi and two different models of Western Digital drives. And, each disk had a different number of partitions: 3, 4, and more. So, drive identification was rather easy. Had all drives been the same brand and model, with the same partitions and sizes, it would have probably been impossible to distinguish which drive had which operating system.

Setting The Boot Order:

Given that it was the latest technology, I had a sense that Redhat 6.4 would need to be the disk with grub.conf. So in the BIOS of the machine, I set the boot order of the disks to be:

Redhat  6.4
Redhat  5.5
Windows 7

Original Redhat 6.4 grub.conf:

default=0
timeout=5
splashimage=(hd0,1)/grub/splash.xpm.gz
hiddenmenu

title Redhat 6.4 Kernel (2.6.32-358.el6.x86_64)
root (hd0,1)
kernel /vmlinuz-2.6.32-358.el6.x86_64 ro root=UUID=d624839b-32e7-4e8a-8454-5a837cecd947 rd_NO_LUKS rd_NO_LVM LANG=en_US.UTF-8 rd_NO_MD SYSFONT=latarcyrheb-sun16 crashkernel=auto KEYBOARDTYPE=pc KEYTABLE=us rd_NO_DM rhgb quiet numa=off
initrd /initramfs-2.6.32-358.el6.x86_64.img

title Redhat 5.5 Enterprise Linux-base (2.6.18-194.el5)
root (hd1,0)
kernel /vmlinuz-2.6.18-194.el5 ro root=LABEL=/ rhgb quiet
initrd /initrd-2.6.18-194.el5.img

title Windows 7
rootnoverify (hd2,0)
makeactive
chainloader +1
boot

This booted Redhat 5.5 easily enough. But when Windows was chosen, a Windows (not grub) error resulted:

File:  \BOOT\BCD
Status:  0xc0000001
Info:  An error occurred while attempting to read the boot configuration file

So, the booting process had moved beyond grub, and into Windows, then crashed. Why? And more importantly, what was the solution?

Useful Grub Commands:

While researching the solution, I read a lot of webpages, and discovered a lot of very useful grub commands. Like a lot of Linux shells, once inside grub>, you don’t have a lot of help, with really clear and simple examples. No man pages either. Here are some useful navigation and commands. Other useful links are below.

When the grub selections come up, click the up arrow to stop the automatic boot. From there you can press the keys:

c: to get the command line and get to the grub> scripting prompt.

e: to edit each section of grub, say, Redhat 6.4. Once inside the section, you can press e again to edit the actual command line. Inside, there are other commands to edit the script, o/O to open/add lines, d to delete lines.

b: to boot the machine by running the grub commands

ESC:  To get back to the original menu, press ESC multiple times. This even works if the grub script errored out and stopped. Very useful to avoid another long reboot process.

In grub, your hard drives are designated as: (hd0). Or, (hd1), (hd2), etc. The first hard drive is indicated by zero, not one.

A very useful grub command is: geometry.

grub> geometry (hd2)
geometry (hd2)
drive 0x82: C/H/S = 48641/255/63, The number of sectors = 781422768, /dev/sdc
Partition num: 0,  Filesystem type unknown, partition type 0x82
Partition num: 1,  Filesystem type is ext2fs, partition type 0x83
Partition num: 2,  Filesystem type is fat, partition type 0xc
Partition num: 3,  Filesystem type is ext2fs, partition type 0x83

This will list all the partitions for a particular hard drive.

Another useful command is the tab key, which can fill in possible choices. For instance, once you are in the grub> command prompt, type

grub>root (hd

and then hit the <tab> key. This will automatically show the possibilities available. In my case:
hd0, hd1, hd2

If you choose one, and type:
root (hd1,
and hit the tab key, all the possible partitions will then be displayed.

The boot record will be on a particular partition of one hard drive. This needs to be identified in this format: (hd0,0). (hd0,0) would indicate the first hard drive, and first partition.

Over a number of days, I spent a lot of time running grub commands, trying different options, modifying the BIOS, and grub.conf. But all I got was more errors. There were many boring multi-minute reboots. I posted questions online, but got no solutions.

Other errors that I got in this trial and error process were:

Error 13: Invalid or unsupported file format.

Error 15: File not found

Window error: Bootloader is missing. Press control-alt-delete to restart

Lots Of Ways To Identify a Drive:

Interestingly, there are a lot of ways that a drive is identified. In my BIOS alone (Phoenix BIOS, Asus BIOS 1006), a hard drive is identified by multiple identifiers. Simultaneously, one drive can be identified by:
IDE3
Sata Port 2
Fixed Disk 0
as well as: Hard Disk, although once this info flashes through the initial screen, I can’t find it through the BIOS menus again.

Does anyone know if these are mutually exclusive? Or not? If so, please leave a comment.

Then in the BIOS, there also is the Boot Order.

Once past the BIOS, in grub, you identify the first drive and partition by hd0,0.

After booting, Redhat then identifies the first drive and partition by /dev/sda1.

Got all that?

Originally, the BIOS showed the disks connected as:

Redhat 5.5:  Fixed disk 0, Sata Port 2, IDE 3
Windows:     Fixed disk 1, Sata Port 3, IDE 4
Redhat 6.4:  Fixed disk 2, Sata Port 4, IDE 5

And the boot order was set to:
Redhat 6.4
Redhat 5.5
Windows 7

Solution – Big Picture:

Change the physical sata cable to hard drive mappings in the server itself.

Change the boot order in the BIOS.

Change grub.conf to reflect the correct hard drives.

After a number of days of struggle, this solution came to me as I was waking up.

I changed the sata cable to disk mapping in the BIOS to be:

Windows:     Sata Port 2
Redhat 5.5:  Sata Port 3
Redhat 6.4:  Sata Port 4

I had a sense that Windows always wants to be number one, so I set it to be the first drive the BIOS sees.

Then, to boot Redhat 6.4 first, I changed the boot order to be:

Redhat 6.4
Windows
Redhat 5.5

And, this configuration cleanly booted all three! Yeah! Initially, I was able to test each without rebooting and changing grub.conf, by modifying the root (hd?,?) command in the grub> shell.

New grub.conf:

When they all worked, I changed the Redhat 6.4 grub.conf file to:

default=0
timeout=8
splashimage=(hd0,1)/grub/splash.xpm.gz
hiddenmenu
title Redhat 6.4 Kernel (2.6.32-358.el6.x86_64)
root (hd0,1)
kernel /vmlinuz-2.6.32-358.el6.x86_64 ro root=UUID=d624839b-32e7-4e8a-8454-5a837cecd947 rd_NO_LUKS rd_NO_LVM LANG=en_US.UTF-8 rd_NO_MD SYSFONT=latarcyrheb-sun16 crashkernel=auto  KEYBOARDTYPE=pc KEYTABLE=us rd_NO_DM rhgb quiet numa=off
initrd /initramfs-2.6.32-358.el6.x86_64.img

title Windows 7
rootnoverify (hd1,0)
makeactive
chainloader +1
boot

title Redhat 5.5 Enterprise Linux-base (2.6.18-194.el5)
root (hd2,0)
kernel /vmlinuz-2.6.18-194.el5 ro root=LABEL=/ rhgb quiet
initrd /initrd-2.6.18-194.el5.img

Analysis:

I’m not 100% sure why this particular configuration works. However, note that the cable to disk mappings in the BIOS are:

Windows:     Sata Port 2
Redhat 5.5:  Sata Port 3
Redhat 6.4:  Sata Port 4

In the BIOS, the boot order is set to:

Redhat 6.4
Windows
Redhat 5.5

When found by grub (and reflected in grub.conf), the order is the same as the boot order:

Redhat 6.4    hd0
Windows        hd1
Redhat 5.5    hd2

Post Boot Configuration:

What’s interesting to note is the order of disks after the machine is booted, as indicated by fdisk.

/dev/sda is: Redhat 5.5
/dev/sdb is: Windows
/dev/sdc is: Redhat 6.4

This is actually the reverse of what the boot order was set to. Hmm.

If anyone can give a simple explanation of all this, please let me know. In particular, what’s the rule of thumb to set the cables? After a week of struggle, at this point, my feeling is, if it works, don’t fix it!

Thoughts:

Three different technology layers: Physical, BIOS, grub boot loader.

Just to get to the fourth technology layer, the operating system.

No wonder this was so difficult.

Rants:

Unfortunately, this install turned out to be a really long trial and error process over a number of days. I expected this to take a half hour! It really bothers me when simple business is so difficult! It’s such a waste of my time, and energy.

It also bothers me that I’m not trying to learn grub. I’m trying to install other software, so that I can use it. Search my blog to see more struggles with installations I’ve done.

You have to hand it to Microsoft, when you install their software, it’s rare to struggle with the installation. You usually don’t have to search and read dozens of webpages, just to figure out how to install it,  do lots of trial and error, and dozens of reboots.  You just install it, reboot, and get on with your life. Good one!

Although, as I think of it though, perhaps I’m the pioneer, going where no one has gone before. I don’t know anyone else doing my Very Simple Dual Boot Method, where not one MBR is modified. And, certainly I don’t know anyone who is doing it as a triple boot and wrote about it. So, perhaps it all comes with being a pioneer.

Useful Links:

Igor’s GRUB bootloader – Full tutorial
http://www.dedoimedo.com/computers/grub.html

Grub command line interface (CLI), and keyboard shortcuts:
http://members.iinet.net/~herman546/p20/GRUB2%20CLI%20Mode%20Commands.html

Grub: Advanced Boot loader Options
http://www.linuxtopia.org/online_books/installation_guides/redhat_fedora_core_5_installation_guide/sn-bootloader-advanced.html

Grub Command List:
http://www.linux-sxs.org/administration/grub-cmd.html

Grub Command Reference:
http://www.linuxselfhelp.com/gnu/grub/html_chapter/grub_12.html

Using the grub command, map, to boot Windows
Interesting, even though I didn’t use map
http://www.linux-sxs.org/administration/grubxp.html

BIOS Boot partition
https://en.wikipedia.org/wiki/BIOS_Boot_partition

If this has been useful, please “Like”.

Advertisements

Leave a Reply

Please log in using one of these methods to post your comment:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: