Installing OpenBSD 64Bit on a Software Raid 1 (Mirroring)

Credits

This very good article is written by Marcus Redivo.
http://www.eclectica.ca/howto/openbsd-software-raid-howto.php

I added a warning to stress one point (see red font) which took me 5 hours to find the error, the rest is the orginal tutorial, have fun.

Installing OpenBSD 64Bit on a Software Raid 1 (Mirroring)

This document describes how to set up RAID mirroring on OpenBSD with the RAIDframe driver built into the kernel.

Overview

This procedure assumes OpenBSD 4.5, the amd64 architecture (should work with any other architecture too), and two 320GB sata disks; wd0 and wd1. The steps involved are:

* Install OpenBSD on wd0
* Compile a kernel that supports RAID
* Reboot into RAID-enabled kernel
* Configure RAID partitions on wd1 as half of a broken mirror
* Copy all files onto the RAID partitions
* Reboot into broken mirror
* Reallocate wd0, hot-add it, and reconstruct the broken mirror
* Reboot into complete RAID environment

OpenBSD will not boot from a RAIDframe RAID set at present, so wd0a and wd1a will be set aside as partitions to boot from. Kernel RAID autoconfiguration will prepare the RAID set at boot time, and mount the RAID partitions as per /etc/fstab.

Disk Preparation

I like to start with completely clean disks. If they have been used before, I wipe them before doing anything else, using dd(1). Boot from the OpenBSD distribution CD to get a shell, and execute:

dd if=/dev/zero of=/dev/rwd0c bs=1024000 &
dd if=/dev/zero of=/dev/rwd1c bs=1024000 &

This will take a while. To keep busy, you can set up the Master Boot Record and the BSD disklabel on each disk.

wd0 will get a temporary installation of OpenBSD, but we will only keep the first partition when we are done. We will use this installation to configure and compile a kernel with RAID support. The following sizes will work:

Part   Size  FS Type  Purpose
----  -----  -------  -------------------------------
 a:    512M  4.2BSD   /  (becomes boot partition)
 c:   -----  unused   Entire drive
 d:   1024M  4.2BSD   /usr
 e:    512M  4.2BSD   /tmp
 f:   1024M  4.2BSD   /var
 g:    512M  4.2BSD   /home

On wd1, we need two partitions; a boot partition, and a partition to contain the RAID set:

Part   Size  FS Type  Purpose
----  -----  -------  -------------------------------
 a:    512M  4.2BSD   Boot partition
 c:   -----  unused   Entire drive
 d:       *  RAID     Everything except boot kernel

It’s important to set the FS Type to RAID for the wd1d Partition, otherwise the System won’t boot !
If later during boot you get the following error, check the FS Type with “disklabel wd1″ and set it to RAID, if this not the case

Kernelized RAIDframe activated
softraid0 at root
root on wd1a swap on wd1b dump on wd1b
warning: /dev/console does not exist
init: not found
panic: no init
Stopped at         Debugger + 0x5 .... and so on

Use the following commands to implement this:

fdisk -i wd0
disklabel -E wd0
...

fdisk -i wd1
disklabel -E wd1
...

Reboot from media, and follow the normal installation procedure to install OpenBSD onto wd0. If you followed the steps above, all partitions will already be set up; simply quit disklabel(8) and move on to assigning partitions to their mount points. If you started here, use the instructions above while in disklabel.

Make a RAID-Capable Kernel

To build a kernel, we need the source code, and a configuration. Source code is on the CD:

mount /dev/cd0a /mnt
cd /usr/src
tar xzpf /mnt/src.tar.gz
cd sys/arch/i386/conf

If you don’t have the cd available you can get it also per FTP
for name enter “anonymous”, for password just press ENTER

cd /usr/src
ftp mirror.roothell.org
Connected to mirror.roothell.org.
220 blog.roothell.org FTP server ready.
Name (mirror.roothell.org:user123): anonymous
331 Guest login ok, send your email address as password.
Password:
230 Guest login ok, access restrictions apply.
Remote system type is UNIX.
Using binary mode to transfer files.
ftp> get /pub/OpenBSD/4.5/src.tar.gz ./src.tar.gz
ftp> get /pub/OpenBSD/4.5/sys.tar.gz ./sys.tar.gz

tar xzvvf src.tar.gz
tar xzvvf sys.tar.gz

The configuration is the same as GENERIC, with two additions. Create a file called GENERIC.RAID in /usr/src/sys/arch/amd64/conf with the following contents:

---Begin---
#
#	GENERIC.RAID - Add kernelized RAIDframe driver
#

include "arch/amd64/conf/GENERIC"

option		RAID_AUTOCONFIG	# Automatically configure RAID at boot

pseudo-device	raid		4	# RAIDframe disk driver

----End----

The following steps will build and install the new kernel:

config GENERIC.RAID
cd ../compile/GENERIC.RAID
make depend && make
mv /bsd /bsd.original
cp bsd /bsd
chmod 644 /bsd

Now reboot into the RAID-enabled kernel.

Second Disk Setup – Make A Broken Mirror

To prepare the second disk, we need to accomplish the following:

* Make it bootable
* Create a RAID configuration
* Apply the configuration to the disk
* Create a disklabel for the RAID device
* Create filesystems in the RAID device partitions
* Make the RAID device autoconfigurable
* Copy our installation onto the new partitions
* Update fstab to refer to the RAID device
* Reboot into the RAID environment

Make the second disk (wd1) bootable:

newfs /dev/rwd1a
mount /dev/wd1a /mnt
cp /bsd /mnt/bsd
cp /usr/mdec/boot /mnt/boot
/usr/mdec/installboot -v /mnt/boot /usr/mdec/biosboot wd1

Create /etc/raid0.conf, which should look like this:

START array
# rows (must be 1), columns, spare disks
1 2 0

START disks
/dev/wd2d
/dev/wd1d

START layout
# sectPerSU SUsPerParityUnit SUsPerReconUnit RAID_level
128 1 1 1

START queue
# queue mode, outstanding request count
fifo 100

Notice that under disks, a (non-existent) wd2 is specified. When the RAID set is created, this serves as a placeholder, and will appear as failed. Once we are ready, we will hot-add wd0 as a spare and reconstruct the mirror on it. Once that is done, this configuration file will be updated to reflect the actual disk assignments.

Now that you have this configuration file, implement it with:

raidctl -C /etc/raid0.conf raid0
raidctl -I 20050900 raid0
raidctl -iv raid0

The argument to -I is a unique identifier that permits RAIDframe to determine which disks belong to which RAID sets; I used the year and month, plus ’00′ to indicate the first RAID set.

It is time to define the partitions on the RAID device. For my purposes, I used the following:

Part   Size  FS Type  Purpose
----  -----  -------  -------------------------------
 a:    512M  RAID     /
 b:   1024M  swap     swap
 c:   -----  unused   Entire drive
 d:      5G  RAID     /usr
 e:      5G  RAID     /tmp
 f:     42G  RAID     /var
 g:       *  RAID     /home

Again, we use disklabel(8) to put this into effect. Once that is done, create filesystems in the partitions that need them, and make the RAID set auto-configurable as a root filesystem:

disklabel -E raid0
...

newfs /dev/rraid0a
newfs /dev/rraid0d
newfs /dev/rraid0e
newfs /dev/rraid0f
newfs /dev/rraid0g
raidctl -A root raid0

Copy the data from the initial installation over to the RAID partitions, making certain to preserve permissions:

mount /dev/raid0a /mnt
(cd /; tar -Xcpf - .) | (cd /mnt; tar -xpf -)
tar: Ustar cannot archive a socket ./dev/log
umount /mnt

mount /dev/raid0d /mnt
(cd /usr; tar -cpf - .) | (cd /mnt; tar -xpf -)
umount /mnt

mount /dev/raid0f /mnt
(cd /var; tar -cpf - .) | (cd /mnt; tar -xpf -)
tar: Ustar cannot archive a socket ./cron/tabs/.sock
tar: Ustar cannot archive a socket ./empty/dev/log
umount /mnt

mount /dev/raid0g /mnt
(cd /home; tar -cpf - .) | (cd /mnt; tar -xpf -)
umount /mnt

(You can ignore the errors; the sockets will be recreated.)

Update /etc/fstab on the broken mirror to point to the raid0 partitions instead of the old wd0 partitions, and reboot:

mount /dev/raid0a /mnt
sed 's/wd0/raid0/g' /mnt/etc/fstab > /mnt/etc/fstab.tmp
mv /mnt/etc/fstab.tmp /mnt/etc/fstab
umount /mnt
reboot

boot> boot wd1a:/bsd

Complete the Mirror Pair

The last step is to integrate the first disk into the mirror pair. We need to update the wd0 disklabel to match wd1, which is easily done by copying:

disklabel wd1 >disklabel.wd1
disklabel -R wd0 disklabel.wd1

There is no need to put a disklabel into the RAID partition, as this will be taken care of when the mirror is reconstructed.

To reconstruct the array, hot add wd0 as a new spare, and then start reconstruction. When reconstructions is complete, rebuild the parity.

Note: Reconstruction and parity rewrite can take a long time. Do not reboot while this is in progress, or you may lose everything and need to start over.

raidctl -a /dev/wd0d raid0
raidctl -vF component0 raid0
raidctl -vP raid0

One last step remains; the RAID configuration file must be updated to remove the bogus wd2 entry and replace it with an entry for wd0:

sed 's/wd2/wd0/g' /etc/raid0.conf > /etc/raid0.conf.tmp
mv /etc/raid0.conf.tmp /etc/raid0.conf

Reboot, and check the RAID status:

reboot

...

raidctl -s raid0
raid0 Components:
           /dev/wd0d: optimal
           /dev/wd1d: optimal
No spares.
Parity status: clean
Reconstruction is 100% complete.
Parity Re-write is 100% complete.
Copyback is 100% complete.