OODT Coordination Meeting

  • Socorro, NM 2013 September 25,26 2013

Agenda

Day 1 - 9/25
  • 02:00pm - Introductions
  • 02:30pm - Current progress in Apache OODT - Mattmann
  • 03:00pm - Discussion
  • 03:30pm - NRAO deployments of Apache OODT for ALMA and EVLA
    • ALMA Kern/Davis
    • EVLA Butler/Harland
  • 04:00pm - Q&A for OODT by NRAO - NRAO doers
  • 04:30pm - Adjourn
  • 05:00pm - Drinks/Dinner/Social
Day 2 - 9/26
  • 09:00am - MeerKAT/KAT-7 replicated archive
    • Could we use OODT to build a replicated archive in the US
  • 10:00am - Scientific Visualization Toolkit for Astronomy Planning - Kern
  • 10:30am - JPL BigData other activities (Climate; DARPA XDATA)
  • 11:00am - Specific Planning for JPL & NRAO collaborations
    • Visualization - Kern/Mattmann
    • Archive Access Toolv2 - Butler/Mattmann
    • ALMA specific needs - Kern/Davis
    • EVLA specific needs - Butler/Harland
  • 11:30am - Upcoming meetings or workshops/papers/proposals that we can/should plan - All
  • 12:00pm - Lunch
  • 01:00pm - Adjourn

ALMA OODT Experience (Lindsey Davis, personal impressions)
  • focus
    • workflow manager
  • work period
    • December 2012 - January 2013, September 2013, 2-3 weeks off and on
  • get it working (with the help of Keith Cummings !)
  • make an ALMA namespace friendly project
    • alma-pipe-oodt project https://safe.nrao.edu/wiki/pub/Software/CASA/Sept2013OODTMeeting/alma-pipe-oodt.txt
      • maven: cloned maven setup, edited some path definitions
      • project: constructed skeleton project with simpler directory structure, cloned skeleton workflow
    • experience
      • maven: for me a distraction, complex, difficult for novice to understand and tune
      • workflow manager: workflow interfaces, policy definitions, easy to understand
      • libraries: large number of jar files, most not needed, large overlap with ALMA
  • ALMA Common Software (ACS) integration
  • Connection to ALMA pipeline infrastructure code
    • steps
      • generated artificial project (pipeine processing request etc), no external metadata or archive connection required
      • integrated OODT code with existing pipeline infrastructure, up to but not including running the CASA script
      • experimented passing an object between OODT tasks, used Base64 encoding assuming serializable object
    • experience
      • documentation minimal
  • Issues
    • OODT distribution mechanism
      • tarballs ?, tarballs with jar and source files ?, building selected jars from source ?
    • configuring OODT in third party development environment
      • environment: environment variables, java properties, loggers, hardcoded paths
      • runtime environment: process, logging, and repository directories
    • passing meta data between tasks
      • passing objects: any experience with this
    • starting, stopping, queuing executions
    • documentation
      • ?
Partial EVLA OODT Experience (David Harland, blog of attempt to create "Hello, World" app)

-- JeffKern - 2013-09-24
Topic attachments
I Attachment Action Size Date Who Comment
PIPELINE-Science-lib.txttxt PIPELINE-Science-lib.txt manage 241 bytes 2013-09-24 - 12:37 LindseyDavis PIPELINE-Science-lib
PIPELINE-Science-src.txttxt PIPELINE-Science-src.txt manage 2 K 2013-09-24 - 12:27 LindseyDavis PIPELINE-Science-src directory
alma-pipe-oodt.txttxt alma-pipe-oodt.txt manage 6 K 2013-09-24 - 12:06 LindseyDavis alma-pipe-oodt project
sss-pipe-oodt-clean.txttxt sss-pipe-oodt-clean.txt manage 6 K 2013-09-24 - 12:00 LindseyDavis sss-pipe-oodt-clean project
wmgrEXT wmgr manage 4 K 2013-09-26 - 09:33 LindseyDavis workflow manager script
wmgr-clientEXT wmgr-client manage 1 K 2013-09-26 - 09:34 LindseyDavis workflow manager client
Topic revision: r5 - 2013-09-26, JeffKern
This site is powered by FoswikiCopyright © by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding NRAO Public Wiki? Send feedback