Casa Parallelization meeting minutes

Monday July 2nd, Room 317, 3:30pm MDT

Attendees: Bryan, Jeff, James, Kumar, Jim

Targets/tasks

• Complete or nearly complete test script James
• Begin completion of pclean vs clean parametrization Kumar
• Identify large number channel data set Bryan
• SSH key management, consider fork() for localhost Jeff to send to ESO

Discussion/Agenda

• Benchmark update
  • Still awaiting hardware from Supermicro
  • Currently running standard pipeline against 3c147 data set
• Status of HPC effort in Garching: Parallel Framework including partitioning Kern
• Pcube testing report Kern/Golap
  • Incorrect numeric results
  • Differences between clean and pclean parameters

Deferred Items

• Memory requirements in imaging
• Describe and implement engine vs threaded gridder determination as a function of memory
  • Need dynamic image partitioning scheme for 2, 3, 4 or more threads.
  • (Total memory - some delta ) / memory per engine = number of engines
  • int (Total cores / number engines) = number of threads per engine
• Model column data tests James
  • Pending completion of test script
  • James to get with Kumar in late June
• Memory resident subtable tests James late June
  • Pending completion of test script
  • Check that it affects number of FDs as well
• Spectral imaging tests James
  • Understand channel chunk choice effect on performance James
  • Pending completion of test script

CASA HPC Initiatives for the 3.5 Cycle

• Tentative list from last meeting
  • Reduce in memory requirements during imaging (maybe removed)
  • Parallelize multi-field imaging
  • Parallelize MFS terms > 1 in the gridding step
  • Parallelize Cube imaging scatter gather
  • Expand threaded gridder case to >4 threads
  • Implement core affinity for threaded gridder case (first out)
  • Parallel framework (partitioning, parallel go) work in ESO
  • Filler to write a MMS in parallel

Deferred Topics

ARDG Targets