RedHat Linux V.8 Disk Mirror



7/2/2003 This page gets a lot of hits, so we tried to clean it up a bit.

Our main purpose of the mirror is to backup the main disk, but not as a hot standby. The mirror will be removed after synchronized and put in a secure place. The only major changes will be to the data, which can be backed up by CD. So, a monthly mirror re-sync is envisioned. The other goal is have the mirrored drive simply installed in another PC, then let it find new hardware and a new server is up and running.

Disk mirroring was accomplished - with much pain! So, the process we used follows: The drives for mirrors chosen are a Master IDE of 20 GB and a Slave IDE of 13GB. The mirroring for Linux is a mirroring of partitions, not drives. So, the Master Boot Record (MBR) is not part of the mirror, therefore the mirrored drive is not inherently bootable. The process of making a mirror destroys all present data on both drives. So boot from Linux CD Disk1 and use FDISK and Disk Druid (no program exists on disk by that name, a Redhat phenom) , which can be easier than using " mkraid " command.

When you get to the point of choosing between FDISK or Disk Druid, choose FDISK (don't worry, you will use both!) FDISK is easier to sort out the partitions. You may want to get familiar with FDISK at the shell prompt before this whole process. It was hard to see the FDISK screen in the Linux Setup.

There are two commands for disk partitions /sbin/fdisk and /sbin/parted. Neither one seems complete for setting a disk mirror. Parted almost does but uses disk sizes instead of cylinders - unwieldy. To set up partition sizes, you need to work backward starting from the swap size (swap = 2X RAM [256MB RAM in this case] ). The root (/ ) sectors = boot (/boot ) + 1, to max cylinders - swap +1. The number of swap and boot cylinders was taken from the original default Linux setup, read with FDISK.

So for example, the 13GB (limit for both drives) had 1665 Cylinders. From a previous Linux install, the /boot partition had 13 cylinders (approx. 100MB) and the swap had 66 cylinders (approx. 500MB), so the / directory (root) was set from 14 to 1599 cylinders.

The partitions on both drives were setup, using fdisk, before doing Disk Druid, as follows (hda = 20GB, hdb = 13GB):

Disk /dev/hda: 255 heads, 63 sectors, 2431 cylinders (20 GB Western Digital)

Units = cylinders of 16065 * 512 bytes

Partition table (* = boot partition):

Device Boot Start End Blocks Id System

/dev/hda1 * 1 13 104391 fd Linux raid autodetect

/dev/hda2 14 1599 12739545 fd Linux raid autodetect

/dev/hda3 1600 1665 530145 fd Linux raid autodetect


Disk /dev/hdb: 255 heads, 63 sectors, 1665 cylinders (13 GB Matrox)

Units = cylinders of 16065 * 512 bytes

Device Boot Start End Blocks Id System

/dev/hdb1 * 1 13 104391 fd Linux raid autodetect

/dev/hdb2 14 1599 12739545 fd Linux raid autodetect

/dev/hdb3 1600 1665 530145 fd Linux raid autodetect



All partitions are set with ID = fd - Linux raid autodetect.

Drive size in bytes = cylinders X 16065 X 512

hda = 19,995,655,680 bytes

hda1 (/boot - 13 cylinders) = 106,928,640 bytes

hda2 (/ - 1586 cylinders) = 13,045,294,080 bytes

hda3 (swap - 66 cylinders) = 542,868,480 bytes

So, 6,300,564,480 bytes on the 20GB are unused. The whole Maxtor 13GB drive is used. Both /boot and swap sizes are apparently generous. You can have 4 partitions, but many volumes.



Another example with a Western Digital 40GB mirroring the above 20 GB disk.

The cylinders on the 20GB disk did not match the 40 GB. To have the 40GB hdb drive mirror the hda drive (will not work vice versa) the size of the partitions (blocks) of hdb needed to be greater than or equal to hda.

The number of cylinders equating to number of blocks need to be calulated for each partition. 13 cylinders (hda1) on the 20 GB require 208 cylinders (hdb1) on the 40 GB for equal or greater block size of the same partition.


Disk /dev/hdb: 77545 cylinders - 39082680 Approx Blocks
Units = cylinders of 1008 * 512 bytes

Device Boot Start End Blocks Id System
/dev/hdb1 * 1 208 104832 fd Linux raid autodetect
/dev/hdb2 209 25486 12739608 fd Linux raid autodetect
/dev/hdb3 25487 26539 530208 fd Linux raid autodetect
/dev/hdb4 26540 77545 25707024 83 Linux

Since the 40 GB did not use RedHat CD Disk 1, files systems were made manually (mkfs.ext3 /dev/hdb1,2,4 and mkswap /dev/hdb3). In order to get the partition of /dev/hda4 recognized by the file system, mkdir /data was used as the mount point and /etc/fstab was edited by copying the entry for /boot /dev/md0 and editing for /data /dev/hba4 , where appropriate. Re-boot or do: mount -t ext3 /dev/hda4 /data to make the partition available.

Using FDISK -ul /dev/hda shows the file system on each partition - as PARTED does.

# /sbin/parted /dev/hdb > parted.txt (then P enter, Q enter, to write to parted.txt)

GNU Parted 1.4.24

Using /dev/hdb

Information: The operating system thinks the geometry on /dev/hdb is 1665/255/63.

Therefore, cylinder 1024 ends at 8032.499M.

(parted) p

Disk geometry for /dev/hdb: 0.000-13063.640 megabytes

Disk label type: msdos

Minor Start End Type Filesystem Flags

1 0.031 101.975 primary ext3 boot, raid

2 101.975 12542.937 primary ext3 raid

3 12542.937 13060.656 primary linux-swap raid

After the partitions are set (don't forget to set the boot partition, and write the data) hit the back button and choose Disk Druid.

The pain comes from not paying close attention to the settings in Disk Druid. We wound up re-loading Linux with only the packages wanted, several times. There is no Disk Druid settings memory for RAID configurations, as there is for FDISK partition data. There looks like there is Disk Druid settings memory, but watch out for the mount point being dropped (e.g. /boot, /) and the RAID number (RAID1 for mirror). So, check all of the settings.

In Disk Druid, each partition is set as Linux raid and after the RAID button is pressed and partitions selected to mirror ( ex. /dev/hda1 & /dev/hdb1), the file system is selected (ex. format as ext3 or swap). Remember to pick RAID1 and the mount point ([/boot] and [/] [no mount point for swap partition])!

After Linux was installed, #cat /proc/mdstat showed that md1 & md0 were not mirrored ([2/1] [U_] - (they are in the below example).

Personalities : [raid1]

read_ahead 1024 sectors

md2 : active raid1 hda3[0] hdb3[1]

530048 blocks [2/2] [UU]

md1 : active raid1 hdb2[1] hda2[0]

12739456 blocks [2/2] [UU]

md0 : active raid1 hdb1[1] hda1[0]

104320 blocks [2/2] [UU]

unused devices: <none>

A look at the System Log showed that there was a " superblock update time inconsistency " with hda1 & hdb1; hda2 & hdb2. So, hdb1 & hdb2 were kicked out of the mirror.

The fix is to manually put the drive kicked in the mirror: # /sbin/raidhotadd /dev/md1 /dev/hdb2 .... etc.

All the RAID info is hard to piece together - trial & error on the internet.

There were a couple of ways found to get a mirrored drive bootable, from the internet. Could not get LILO bootable to work except by doing the following (GRUB is below):

With the IDE drives on different buses (hda is the bootable), at the shell prompt type: " dd if=/dev/hda of=/dev/hdc " hit Enter key and wait for approx. 1 minute, then do Ctrl + C to stop the process. The two drives have different disk parameters (less cylinders on the 13 GB). One minute is enough time to copy the MBR, and then some.

Possible data corruption could have occurred on the mirrored drive, so then re-synched the mirror partitions: " /sbin/raidhotadd /dev/md0 /dev/hdc1 " and so on for all the partitions. The boot on the 13GB worked. If both drives are the same, you can just let the dd command finish (a few hours?) and you will have a bootable mirror.

If you have a problem getting the system to boot-off and load a good mirror instead of the damaged disk (superblock being newer - the kernel kept picking the damaged disk). Run fdisk and change the type ID on the bad disk to 83 - hdc1(/boot), 83 - hdc2 (/ ), 82 - hdc3 (swap), for the 3 partitions on the bad disk. Re-boot with the good disk as primary master (hda), then with the good disk loaded change the bad disk partitions back to ID type fd - Linux raid auto-detect. Then raidhotadd the partitions of the bad disk to the good.


To Make a bootable GRUB disk (make the mirror first [need /boot directory], type what is shown right of "#" and grub> ) :

# grub --batch
Probing devices to guess BIOS drives. This may take a long time.


GRUB version 0.93 (640K lower / 3072K upper memory)

[ Minimal BASH-like line editing is supported. For the first word, TAB
lists possible command completions. Anywhere else TAB lists the possible
completions of a device/filename.]
grub> root (hd0,0)
root (hd0,0)
Filesystem type is ext2fs, partition type 0xfd
grub> setup (hd0)
setup (hd0)
Checking if " /boot/grub/stage1 " exists... no
Checking if " /grub/stage1 " exists... yes
Checking if " /grub/stage2 " exists... yes
Checking if " /grub/e2fs_stage1_5 " exists... yes
Running " embed /grub/e2fs_stage1_5 (hd0) "... 16 sectors are embedded.
succeeded
Running " install /grub/stage1 (hd0) (hd0)1+16 p (hd0,0)/grub/stage2
/grub/grub.conf "... succeeded
Done.


If you want to make /dev/hdb bootable, substitute root (hd1,0) and setup (hd1) in the above.

If you have a bad mirror due to a software problem (ie. deleted file or such) you now have two bad disks. Hopefully, you still have a third good disk - albeit dated! Putting a good bootable disk as primary master on the IDE cable doesn't assure that it will be booted by GRUB. In order to boot the good disk with the GRUB loader you need to select "a" on the blue RedHat boot screen.

You will then see something like : ro root=/dev/md1

You need to change that to the root of the good disk (example) : ro root=/dev/hda2

After bootup, at the terminal prompt do #cat /proc/mdstat and you might see that /dev/hdb is the primary mirror - the bad disk. You need to change the partition types (fdisk) of /dev/hdb to 83 for Linux or 82 Linux swap. Re-boot and the good disk will be booted and should be shown by #cat /proc/mdstat to be /dev/hda. Change the partition types of /dev/hdb back to "fd" for Linux raid auto detect. Then do /sbin/raidhotadd /dev/md0 /dev/hdb1 for all the partitions.


Brought to you by - Droit Technologies, Inc. www.droittech.com PO Box 322 Carlstadt, NJ 07072