LARD Project Notes, Part 2

  • PART TWO INCLUDES:
    • Performance Testing

1 Performance Testing

Applies to
OSSes

1.1 Look at your devices

[root@lard-oss-2 ~]# lsscsi
[0:0:8:0]    enclosu QUANTA   JB9 SIM 0        1030  -       
[0:2:0:0]    disk    AVAGO    MR9380-8e        4.60  /dev/sdc 
[0:2:1:0]    disk    AVAGO    MR9380-8e        4.60  /dev/sdd 
[0:2:2:0]    disk    AVAGO    MR9380-8e        4.60  /dev/sde 
[1:0:47:0]   enclosu QUANTA   JB9 SIM 0        1030  -       
[1:2:0:0]    disk    AVAGO    MR9380-8e        4.60  /dev/sdf 
[1:2:1:0]    disk    AVAGO    MR9380-8e        4.60  /dev/sdg 
[1:2:2:0]    disk    AVAGO    MR9380-8e        4.60  /dev/sdh 
[8:0:0:0]    disk    ATA      INTEL SSDSC2BB08 G201  /dev/sda 
[8:0:1:0]    disk    ATA      INTEL SSDSC2BB08 G201  /dev/sdb 

1.2 Make some partitions (smaller than 16 TB each) on your device of choice

parted /dev/sdc
(parted) mklabel gpt
(parted) mkpart sdc1 0% 25%
(parted) mkpart sdc2 25% 50%
(parted) mkpart sdc3 50% 75%
(parted) mkpart sdc4 75% 100%
(parted) quit

cat /proc/partitions

1.3 Set blocksize on all devices you want to test

blockdev --setra 16384 /dev/sdc

1.4 Prepare testing packages

1.4.1 xdd (for raw disk tests)

  • copy of xdd source code lives over on the NM filer. Scp it over (as yourself, not root) from NM's filer (ssh.aoc.nrao.edu:/home/src/xdd/xdd65.013007.tgz)
  • compile it as follows
mkdir /tmp/root
cd /tmp/root
scp <you>@ssh.aoc.nrao.edu:/home/src/xdd/xdd65.013007.tgz .
tar xfvz xdd65.013007.tgz
cd xdd65.013007
make -f linux.makefile xdd
cd bin

1.4.2 iozone (for filesystem tests)

1.4.2.1 Make ext4 file system (if testing on a file system)

  • Your stride and stripe-width need to correspond to the strip of the array when it was created
  • Stride is the number of 4K blocks in a chunk:
    • If it's 128K then the stride is 32 (128K / 4K)
    • If it's 256K then stride is 64 (256K / 4K).
  • The stripe width is then stride * # data disks (not spares) so it will be either 224 (32*7) or 448 (64*7).

1.4.2.2 Handy table

Strip size on array mkfs command
64 K mkfs.ext4 -j -m0 -E stride=16 -E stripe-width=112 /dev/sdX
128 K mkfs.ext4 -j -m0 -E stride=32 -E stripe-width=224 /dev/sdX
256 K mkfs.ext4 -j -m0 -E stride=64 -E stripe-width=448 /dev/sdX

1.4.2.3 Get iozone

mkdir /tmp/root
cd /tmp/root
scp <you>@ssh.aoc.nrao.edu:/home/src/iozone/iozone3_434.tar .
tar xfv iozone3_434.tar
cd iozone3_434/src/current/
make linux

1.4.2.4 Preparation

  blockdev --setra 16384 /dev/sdX  # if you didn't do this before
  mkfs.ext4 -j -m0 <pick from the table above>
  mkdir -p /mnt/sdX
  mount /dev/sdX /mnt/sdX
done

2 XDD Performance Test Results (64K chunk size)

[root@lard-oss-2 bin]#   ./xdd.linux -op read -targets 1 /dev/sdc -queuedepth 1 -blocksize 512 -reqsize 2048 -mbytes 131072 -passes 2 -verbose

3 Iozone Performance Test Results

3.1 Run iozone

/tmp/root/iozone3_434/src/current/iozone -ec -t 1 -r 1M -s 128g -+n -i 0 -i 1

3.2 'Chunk' size 64K Results (also includes xdd results)

  • Chunk size refers to the strip size set at the time the RAID arrays are initialized
  • see here

3.3 'Chunk' size 128K Results

  • Chunk size refers to the strip size set at the time the RAID arrays are initialized
  • see here

3.4 'Chunk' size 256K Results

  • Chunk size refers to the strip size set at the time the RAID arrays are initialized
  • see here

Proceed to Part 3

This topic: HPC > WebHome > LustreFilesystem > LardLustrePerformMain
Topic revision: 2016-04-20, JessicaOtey
This site is powered by FoswikiCopyright © by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding NRAO Public Wiki? Send feedback