Documentation for Lustre filesystem. Very much in progress.

Installation

The following pages cover various steps in installation of a Lustre filesystem similar to the AOC installation. General order of steps are

• Lustre Overview LustreOverview
• Configure hardware LustreHardware
• Install Operating systems (documented elsewhere, stock NRAO RHEL5.5 installs)
  • We install Everything including 32bit RPMs but excluding conflicts using something like this kickstart file.
  • Note: It's best to destroy any existing RAID groups on the controllers prior to first time OS installation, otherwise the installer kernel and runtime kernel differ on whether the OS disk is /dev/sda and RAID groups are /dev/sd{b-e} or whether the RAID groups are /dev/sd{a-d} and the OS disk is /dev/sde. Destroying RAID groups ensures the first case. If you don't want to destroy the RAID groups you'll need to modify the grub.conf the first time the system boots.
• Configure 3ware and Raid groups 3WareInstall
• Install and configure Infiniband InfinibandInstall
• Install and configure 10Gb Lustre10G
• Perform raw tests RawPerformanceTests
• Install Lustre server RPM LustreSoftware
• Configure Lustre filesystems LustreFilesystemCreation
• Configure Clients LustreClientInstall
• Install monitoring tools LustreMonitoringSystems
• Tuning Paramaters LustreTuning
• Notes from LustreTraining
• Further notes

Testing

• Performance tests PerformanceTests

LARD Installation Notes

• LARD installation, Part 1
• LARD installation, Part 2
  • Test Results: mkfs with 64K chunk size
  • Test Results: mkfs with 128K chunk
  • Test Results: mkfs with 256K chunk size
  • Test Results: zfs experiment
• LARD installation, Part 3

NAASC Lustre 2.x Upgrade Activities

• Upgrade Path
• NaascToasterNotes
• July 15-17 Activities (on staff wiki)

Maintenance

Scripts

• 3w-blink: Script to blink a specific disk on a 3ware card.

• OST-distribute-1.sh: Script to re-balance lustre after adding a new OST.

-- JamesRobnett - 2011-07-11

• ALMA-Pipeline-Cluster-specification.pdf: Original ALMA purchase specification document.