HPC Meeting: 2017/03/01

Participants

• NRAO:
  • David Halstead
  • Jessica Ottey
  • Patrick Murphy
  • Mike Hatz
  • James Robnett
• JAO:
  • Maxs Simmonds
  • Jose Parra
  • Alejandro Barrientos
  • Bernardo Mallet

Agenda

• Upgrade CV Lustre to 2.5.5
  • CV updated Lustre to 2.5.5 weekend (25/26 FEB 2017)
  • This was due to Lustre 2.5.3 not allocating OSTs in a round-robin fashion for new files (some OSTs were not getting used)
  • Confirmed as a bug in Lustre 2.5.3
  • Wolfang Baudler circulates a test script to test this
  • Manual draining process still in use to deactivate an OST
  • Maxs Simmonds didn't see this on 2.5.5 release notes (is this confirmed somewhere else?)
• Infiniband optimizations
  • MDS/OSS on same physical infiniband switch
  • Use 3 ports to cascade switches
  • Mike Hatz would send a diagram of layout (sent already)
• Use same Lustre version everywhere?
  • We should strive not to just have same version everywhere, but rather focus on what works for each application/site workload
  • If Lustre is stable, consensus is don't upgrade
  • Pipeline just needs a stable POSIX file system
  • Each site should check their own setup (versions/hardware) and learn from other experiences
• Indication to focus on automatization of cluster, and how pipeline is executed (big chances for improvement)

Action Items

• JAO: test Lustre 2.5.3 allocation bug on JAOPOST Lustre
• JAO: check infiniband IS6030 switch firmware versions, would ask NRAO for firmware if needed