LARD Project Notes, Part 2
1.1 Look at your devices
[root@lard-oss-2 ~]# lsscsi
[0:0:8:0] enclosu QUANTA JB9 SIM 0 1030 -
[0:2:0:0] disk AVAGO MR9380-8e 4.60 /dev/sdc
[0:2:1:0] disk AVAGO MR9380-8e 4.60 /dev/sdd
[0:2:2:0] disk AVAGO MR9380-8e 4.60 /dev/sde
[1:0:47:0] enclosu QUANTA JB9 SIM 0 1030 -
[1:2:0:0] disk AVAGO MR9380-8e 4.60 /dev/sdf
[1:2:1:0] disk AVAGO MR9380-8e 4.60 /dev/sdg
[1:2:2:0] disk AVAGO MR9380-8e 4.60 /dev/sdh
[8:0:0:0] disk ATA INTEL SSDSC2BB08 G201 /dev/sda
[8:0:1:0] disk ATA INTEL SSDSC2BB08 G201 /dev/sdb
1.2 Make some partitions (smaller than 16 TB each) on your device of choice
parted /dev/sdc
(parted) mklabel gpt
(parted) mkpart sdc1 0% 25%
(parted) mkpart sdc2 25% 50%
(parted) mkpart sdc3 50% 75%
(parted) mkpart sdc4 75% 100%
(parted) quit
cat /proc/partitions
1.3 Set blocksize on all devices you want to test
blockdev --setra 16384 /dev/sdc
1.4 Prepare testing packages
1.4.1 xdd (for raw disk tests)
- copy of xdd source code lives over on the NM filer. Scp it over (as yourself, not root) from NM's filer (ssh.aoc.nrao.edu:/home/src/xdd/xdd65.013007.tgz)
- compile it as follows
mkdir /tmp/root
cd /tmp/root
scp <you>@ssh.aoc.nrao.edu:/home/src/xdd/xdd65.013007.tgz .
tar xfvz xdd65.013007.tgz
cd xdd65.013007
make -f linux.makefile xdd
cd bin
1.4.2 iozone (for filesystem tests)
1.4.2.1 Make ext4 file system (if testing on a file system)
- Your stride and stripe-width need to correspond to the strip of the array when it was created
- Stride is the number of 4K blocks in a chunk:
- If it's 128K then the stride is 32 (128K / 4K)
- If it's 256K then stride is 64 (256K / 4K).
- The stripe width is then stride * # data disks (not spares) so it will be either 224 (32*7) or 448 (64*7).
1.4.2.2 Handy table
Strip size on array |
mkfs command |
64 K |
mkfs.ext4 -j -m0 -E stride=16 -E stripe-width=112 /dev/sdX |
128 K |
mkfs.ext4 -j -m0 -E stride=32 -E stripe-width=224 /dev/sdX |
256 K |
mkfs.ext4 -j -m0 -E stride=64 -E stripe-width=448 /dev/sdX |
1.4.2.3 Get iozone
mkdir /tmp/root
cd /tmp/root
scp <you>@ssh.aoc.nrao.edu:/home/src/iozone/iozone3_434.tar .
tar xfv iozone3_434.tar
cd iozone3_434/src/current/
make linux
1.4.2.4 Preparation
blockdev --setra 16384 /dev/sdX # if you didn't do this before
mkfs.ext4 -j -m0 <pick from the table above>
mkdir -p /mnt/sdX
mount /dev/sdX /mnt/sdX
done
[root@lard-oss-2 bin]# ./xdd.linux -op read -targets 1 /dev/sdc -queuedepth 1 -blocksize 512 -reqsize 2048 -mbytes 131072 -passes 2 -verbose
3.1 Run iozone
/tmp/root/iozone3_434/src/current/iozone -ec -t 1 -r 1M -s 128g -+n -i 0 -i 1
3.2 'Chunk' size 64K Results (also includes xdd results)
- Chunk size refers to the strip size set at the time the RAID arrays are initialized
- see here
3.3 'Chunk' size 128K Results
- Chunk size refers to the strip size set at the time the RAID arrays are initialized
- see here
3.4 'Chunk' size 256K Results
- Chunk size refers to the strip size set at the time the RAID arrays are initialized
- see here
Proceed to Part 3