Archive policy meeting 5/16/11 (ML, Felix, Maurizio, Masao, John H.) 1) Notification of PIs: State of ObsUnitSet is maintained in the database (see following links for project lifecycles: http://almasw.hq.eso.org/almasw/bin/view/HLA/ObsProjectLifeCycleJpg http://almasw.hq.eso.org/almasw/bin/view/HLA/ObsUnitSetLifeCycleJpg http://almasw.hq.eso.org/almasw/bin/view/HLA/SchedBlockLifeCycleJpg ) Problem is that data transfer is asynchronous - metadata replicated via Oracle, bulk data transferred differently. This is a problem both for PI notification (to let them know that data is available for download) and for triggering the pipeline at SCO. New piece of software is required - a "Data Tracker" - to query metadata and poll the bulk store to see which files have been delivered. Upon successful polling this software will change a state in the database and (if run at the ARCs) trigger an email to the PI. Data tracker run at SCO sets state "ready for pipeline" (maybe also emails pipeline operator). Data tracker run at ARCs sets state "ready for download" in SCO archive and notifies PI by email. This software falls under the purview of Archive rather than ObsOps. 2) Submission of requirements to CDR 9. Lars is responsible for presenting "requirements" to CDR 9 (where requirements include reprioritization, additional scope, and implementation details, besides new capabilities). Should be 1-2 powerpoint pages per subsystem (actual meeting is in July, so time to fill out details). Also need to cross-check against existing spreadsheet of requirements. Identified several areas: (Felix to draft slides) Request Handler Data Packer Data tags News and events DB Plone 4 version of the SP (Me to draft slides) Science archive DB and VO (ping Doug..) Data tracker Pipeline product ingestion into archive (esp cubes) Shiftlog ingestion as PDF FS/ML to circulate draft slides to SciOps by Wednesday 3) AOB - Masao raised question of software testing. Lars and Mark Rawlings are responsible for Ops and CSV software, respectively. Integrated and end-to-end tests both carried out and planned.