Last update: March 31 2006
I had to install a couple of HP ProLiant servers delivered with Marvell SATA MV88SX6041 controllers. The card supports up to 4 SATA disks that can be configured as independent disks, as RAID-0 or as RAID-1. I use them as independent disks and let Linux software RAID make them work as RAID-5 or RAID-1.
There is a proprietary driver available here.. As I could not find any installation procedure, I'm writing this HOWTO.
This procedure is written for FC3, but works also for FC4 (just use the appropriate kernel versions). For FC4 use the patched driver available here. Precompiled modules for some of the standard FC3 and FC4 kernels are available here.
For FC4 kernel compilation, see also Brian Zoller's mini-HOWTO.
The installation is not complex but is time consuming, as we need to reboot several times and copy stuff from diskettes (yes! diskettes refuse to die! :-). Basically, these are the steps we will follow:
To compile the kernel you need a working FC3 system with the development environment (gcc and friends). Either you connect an IDE disk in the same machine you have the SATA card and install FC3 on it, or you use a separate machine, it doesn't matter too much as far as you can compile the driver somewhere.
Download the driver from here. Create a work directory (for instance MVSATA in your home directory) and unzip the file with:
unzip -d MVSATA mvsata340.zip
Install the kernel sources corresponding to the FC3 ISO images (kernel-2.6.9-1.667.src.rpm). Either you get this file from the sources CD or you download it from one of the Fedora mirrors. Once you have it, install it with the command (as root):
rpm -ivh kernel-2.6.9-1.667.src.rpm
Go to the directory /usr/src/redhat/SPECS and edit the file kernel-2.6.spec. On top of it, you will see the lines:
%define buildup 1
%define buildsmp 1
Set "buildsmp" to 0 (we need only the UP kernel for the time being).
A little bit lower you will see the line:
%define release %(R="$Revision: 1.667 $"; RR="${R##: }"; echo ${RR%%?})%{rhbsys}
Remove the "%{rhbsys}" at the end, so that the kernel signature is the same as the original installation kernel.
Launch the kernel compilation with:
rpmbuild -bc --target=i586 kernel-2.6.spec
If your CPU is not Intel, choose the target so that it is in accordance with the kernel used by the FC3 installer on your hardware. If you don't know which kernel is used by the installer, boot from CD as if you were to start the installation, and when it tells you to load the drivers manually because it doesn't find any disk, press Ctrl-Alt-F2. This will switch the terminal to a shell prompt where you can type "uname -a". The output tells you which is the architecture. In my case the installer was using "i586".
With FC3 2.6.11 kernels we don't need to fully compile the kernel, we are interested only in the config and include files. Therefore, as soon as you see the first compilation warnings (in sked.c) quit the compilation with Ctrl-C. Unfortunately with FC4 and FC3 2.6.12 you need a full compilation to obtain the Module.symvers file.
Go to /usr/src and do:
cd /usr/src
ln -s
redhat/BUILD/kernel-2.6.9/linux-2.6.9 linux-2.6
Finally, go to MVDATA directory where you unzipped the driver and compile it with (as normal user or root):
cd LinuxIAL
bash build.sh
The driver should compile flawlessly leaving a "mv_sata.ko" module ready to use with the 2.6.9-1.667 kernel. Copy this file to a diskette that you can use on the machine with the SATA controller to get the driver. To copy it to a diskette do:
mcopy mv_sata.ko a:
Label the diskette as "i586".
If you don’t have a diskette, you will need to use a FTP or sFTP server on a separate machine. It is possible to use USB memory keys, but this shifts the names of the SCSI disks (sda gets used by the USB device) what complicates the installation.
I put here a few precompiled drivers for Fedora Core 3/4. I only put the ones I used or that were contributed by other enthusiasts. As there are many combinations of architecture, kernel version and UP/SMP, feel free to contribute with your own modules (be sure they work before posting them! :-)
The installation procedure runs a i586 kernel so that it can work with (relatively) old CPUs. But normally your machine will be a i686 or a i686smp. This means the installed kernel is NOT the same as the installation kernel. Therefore, we need to repeat the previous procedure to compile the driver for the i686 architecture. As it is recommended to upgrade the original FC3 kernel anyway, we will compile the driver only for the i686 even if the computer is SMP, so that we can boot from SATA and upgrade FC3 later. The i686 kernel compilation is done with:
cd /usr/src/redhat/SPECS
rpmbuild -bc
--target=i686 kernel-2.6.spec
As before, you don't need to wait for a full compilation. Abort it with Ctrl-C when initialization phase is done. Then recompile the Marvell SATA driver again as you did before (bash build.sh), and copy the resulting module to another diskette, this time labeled "i686".
Now it is time to install FC3 in the machine with the SATA controller. Boot from the installation CD as usual. If you plan to format part or all of your disks with Reiser FS, type the following line in the boot prompt:
linux reiserfs
In a few moments you will be asked to choose the drivers manually as it cannot find any disk. Don't touch anything here and switch to the second console with Ctrl-Alt-F2.
On the second console you will get a shell prompt. From here we will load the i586 driver we compiled before. If you copied the module to a diskette, you can load it with the commands:
mkdir /mnt/fd
mount /dev/fd0
/mnt/fd
modprobe /mnt/fd/mv_sata.ko
umount /dev/fd0
dmesg
The last line will show all the kernel messages since boot. If everything went OK, at the end of the dmesg output you should see the Marvell SATA driver initialization displaying the brand and size of each of your SATA disks.
The most common problems at this stage are version missmatch between the driver and the running kernel because the spec file was not edited correctly when the module was compiled. In this case the driver refuses to load and shows you its version versus the current kernel version (arch = i585 and uni-processor). Normally this gives you the clue about where the error is, and you can fix the spec file and do again the kernel config + driver compilation.
If the driver loaded OK, you can come back to the installation screen with Ctrl-Alt-F1 and press "Done". This will resume the installation procedure.
A few screens later you will be asked if you want to partition the disk automatically or with Disk Druid. This is up to you, but if you want to use the disks in RAID choose Disk Druid.
Personally I'm a fdisk fan, so at this point I go back to the shell screen with Ctrl-Alt-F2 and type:
fdisk /dev/sda
Linux cannot boot from LVM or RAID. This is why I created a 100MB partition at the beginning of each disk. Only the one in the first disk will be used, the rest are there so that all disks have equal partition sizes.
The swap space is more efficient if it is spread in several disks. Therefore I created a 1GB partition for swap (type 82) just after the 100MB partition. I left the rest of the disk for the software RAID (type fd). That is, I got 3 partitions on /dev/sda:
1 500MB type 83
(Linux native)
2
1GB type 82 (Linux swap)
3 rest type
fd (Linux soft RAID)
Repeat the same partition scheme on the other disks. Take in account that your total swap space will be the sum of the swap partition size you have created in each disk. It is recommended that it is about twice the size of you RAM (although with today's memory sizes it seems it is time to revise this concept!
Once the partitioning of all disks is ready, go back to the installation screen with Ctrl-Alt-F7 and enter Disk Druid.
Here you will see a pretty display of your partitions and disks. In my case, I selected /dev/sda1 and pressed "Edit". Then chose the "/boot" mounting point and the "ext3" format.
If you want to use your disks in RAID, choose "Create RAID", format "LVM". Don't forget to select the RAID mode (default is 0, I normally use 5). Then create your "partitions" (logical volumes) by pressing "LVM" and "Add" inside the new window.
Just as an example, I used RAID-5 + LVM with the following partition sizes (I had 3x72GB SATA disks):
Once you are satisfied with your partitionning, proceed with the rest of the installation as usual. Eventually, you will be asked to reboot.
Don't reboot yet! :-)
Now FC3 is installed in the SATA disks, but we need to tell the kernel to load the driver at boot time.
If you went too fast and rebooted already, boot from CD using the boot arguments "linux rescue" at the boot prompt. Once again you will be asked to load the drivers manually. As before, do Ctrl-Alt-F2 and load the SATA driver in the kernel with:
mkdir /mnt/fd
mount /dev/fd0
/mnt/fd
modprobe /mnt/fd/mv_sata.ko
unmount /dev/fd0
dmesg
Then go back to console 1 with Ctrl-Alt-F1 and resume the boot of the rescue mode. It will automatically mount all your partitions under /mnt/sysimage
If you have not rebooted, just do Ctrl-Alt-F2 to go to the shell console.
Now we need to mount the i686 diskette, the one with the driver compiled for the i686 architecture, and copy the driver to an appropriate place. This is done with the commands:
mkdir /mnt/fd (only if you
have
rebooted)
mount /dev/fd0 /mnt/fd
cp /mnt/fd/mv_sata.ko
/mnt/sysimage/lib/modules/2.6.9-1.667/kernel/drivers/scsi/
umount
/dev/fd0
To make the driver available during the boot, we need to update the boot ramdisk image (initrd).
First, we need to update the /etc/modprobe.conf file to tell 'mkinitrd' that 'mv_sata' is the driver of our SCSI host adapter ('mv_sata' is a kind of plugin for the generic Linux SCSI drivers, as many other disk drivers like usb_storage or IOmega). This is done with the following commands:
chroot /mnt/sysimage
echo "alias
scsi_hostadapter mv_sata" >> /etc/modprobe.conf
depmod
-ae -F /boot/System.map-2.6.9-1.667 2.6.9-1.667
mkinitrd -v -f
/boot/initrd-2.6.9-1.667.img 2.6.9-1.667
The "depmod" will update the modules dependency table for the installed kernel. The extra line in 'modprobe.conf' is needed to include the proper SCSI drivers in the initrd image. The "mkinitrd" command will generate the ram disk image to be used at boot time. It contains all the drivers and programs needed to mount the root partition.
Finally, you can exit, sync, and reboot (with Ctrl-D).
During reboot, the GRUB screen should appear normally. If you have a SMP machine, choose the UP kernel, as it is the only one that has the mv_sata module.
Now that you could boot from SATA, you normally want to upgrade your kernel. The first thing is to choose the version you want to update to. Once you are decided, download the corresponding kernel sources from one of the Fedora mirrors (under "updates/SRPMS"). You can work on the newly installed machine or in the one you used to compile the drivers before. In any case, you need the new kernel sources and the driver sources.
Install the kernel sources with:
rpm -ivh kernel-2.6.xxxx.rpm
Go to /usr/src, remove the link "linux-2.6" and create it again now pointing to the newly downloaded sources. Don't forget this step!, otherwise the module will compile with the 2.6.9 kernel (it happened to me several times!).
Before launching the new kernel rpmbuild, edit the spec file. Choose "buildup" or "buildsmp" depending on your hardware (uni or multi processor). Don't forget to remove the "%{rhbsys}" at the end of the "release" parameter, otherwise the kernel signatures will not match.
Run "rpmbuild" as you did before, and stop the compilation with Ctrl-C once the config phase is over.
Finally, go back to the MVSATA sources and compile again with the command:
bash build.sh
You can verify that the compiled module version corresponds with the kernel you want to use by doing:
strings mv_sata.ko | grep magic
Now you can install the new kernel in your SATA box. This is the standard FC3 kernel update procedure. For instance, for a multi-processor machine:
yum update kernel-smp
Before using yum you may want to import the official Fedora repositories GPG keys with:
cd
/usr/share/doc/fedora-release-3
rpm
--import RPM-GPG-KEY RPM-GPG-KEY-fedora RPM-GPG-KEY-fedora-rawhide
RPM-GPG-KEY-rawhide
Of course, the newly installed kernel version should be in accordance with the kernel you used to compile the driver.
Once the kernel has been installed, copy the mv_sata.ko module somewhere under /lib/modules/2.6.xxxx/ as you did at the end of the installation phase, and repeat the depmod and mkinitrd commands as we did before (now with the new kernel version number, of course). You don't need to touch /etc/modprobe.conf again as it has already all what is needed. This is:
depmod -ae -F
/boot/System.map-2.6.xx-yy
2.6.xx-yy
mkinitrd -v -f /boot/initrd-2.6.xx-yy.img 2.6.xx-yy
If everything went OK, now you should be able to boot with your latest kernel from SATA!
Enjoy!
Carlos Vidal (carlos@tarkus.se)