JAO Support Group Meeting, Thursday July 21, 2011, 10:00-11:00 ET (1400-1500UT)

Meeting Logisics

  • Time: 10:00 - 11:00 ET = 1400 - 1500 UT
  • Todays location: CV-ER209
  • 2nd Thursdays Location:
    • Room: CV-ER331
    • Video Hub: 192.33.117.19
    • Domestic Call In: 434-293-7109
    • Int'l Call In: 434-293-6691
  • 4th Thursdays Location:
    • Room: CV-ER209
    • Video Hub: 192.33.117.17
    • Toll-free: 866 560 2652
    • Toll: 517 308 7468
    • Passcode: 2652628

Open Action Items

  1. Action on JH: Write up implementation plan for evolution of Tiger Team effort to as follows: Tiger Team -> reduction of test datasets -> reduction scripts for SV data -> Staff training on SV data -> SV guides for users -> evaluate pipeline performance on SV data -> pipeline hueristics

Agenda

  • Attendees: JH, HL, AW, EF, RI, CB, Itziar. Phone: SC, NM
  • Apologies: Todd (@SMA)
  • Minutes from last meeting are posted at https://safe.nrao.edu/wiki/bin/view/ALMA/JAOsupportMinutes2011Jul12
  • Report from Chile (Nuria or Stuartt?) Nearly back to 14 ant. 17 next week. R8.0 still not debugged. Cleaning up R8.0.3.
  • New Activities:
    • Any news from OT Developers Meeting in Garching that Harvey attended? Productive. No OT upgrade for R8.0, but R8.1.1 will have greatly improved SB generation. Some discussion on "doppler setting" vs. "doppler tracking" - use the former.
    • Pipeline - ppt presentation from Remy
    • Informal and very much unofficial discussion of possible "Plan B"'s for data reduction of PI data during Cycle 0: see "Additional Material" below
    • Phase2 training - will be topic of next weeks meeting
  • On-going Activities:
    • Pencil schedule for CSV support: NAASC2Chile (Nuria in Chile till 8/6; Kartik going down for Panel meetings, staying for 1 turno after)
    • Technical Assessments & TA workshop/pan-ARC lovefest:
      • Participants: AW, HL, TH, CB, SC, SS, JdF, [JH at SciOpsIPT that week]
      • Each TA assigned ~40 proposals. Accept/reject assignments by July 20; Triage results by July 26; enter assessments & identify problematic proposals to discuss at workshop week of July 26 (can start earlier); workshop Aug 1-5. Final reports entered by COB Aug 5.
      • TA Guidelines (v2.1, revised July 14, 2011 14UT)
      • Instructions for TA process (restricted access)
      • Workshop Details (restricted access)
      • Calibrator Source Survey:
  • Next Meeting: July 28

Additional Material:

Cycle 0 Data Reduction: current Plan A (as interpreted by J. Hibbard based on SciOps meetings, implementation plans & reviews from last year (if this plan has changed, I am not aware of it):
  • All PI data is reduced in Chile by members of DSO/CSV
    • ARC Staff on deployment to Chile expected to contribute to this effort
  • Data is processed through QA2, which is defined as looking at the pipeline products & evaluating whether observations met users specified science goals
  • No data leaves Chile until it has successfully passed QA2
  • After QA2, DSO tars data and products up and ships to ARCs for transmission to PIs. Images, scripts and QA parameters are ingested into Archive & mirrored
  • ARCs work with PIs for any subsequent data (re)processing needs, filing QA3 reports as necessary
  • Concerns: Does DSO have sufficient staff/expertise to do all this data reduction, using just that fraction of ARC effort associated with CSV/AoD support?

Possible Plan B1 (retains the concept that no data leaves Chile before QA2 is performed & products ingested into Archive)
  • DSO/CSV team takes data to stage that I will call "QA1.5": that the project appears to have all the necessary data to produce good images, but w/o an explicit evaluation of whether or not it meets the requested goals
  • ARC data reduction experts travel to to SCO to work side by side with JAO staff, reducing PI data from their region (thru QA2) and helping train up JAO staff
  • After QA2, reduction team tars data and products up and ships to ARCs for transmission to PIs. Images, scripts and QA parameters are ingested into Archive & mirrored
  • ARCs work with PIs for any subsequent data (re)processing needs (reduce effort since already familiar w/data), filing QA3 reports as necessary (also reduced effort)
  • I would hope this is a few ~2 week trips (not turnos) by 2-4 staff per region during the first 3-4 mo of cycle 0 observing (before our big community outreach push in Feb-Mar)
  • CON: burden on ARC staff; must do less in other areas. No way to involve expert PIs. Traveling overhead. PRO: by end, JAO staff should have sufficient expertise to do this w/o ARC support

Possible Plan B2 (violates concept that no data leaves Chile before QA2 is performed & products ingested into Archive)
  • DSO/CSV team takes data to stage that I will call "QA1.5": that the project appears to have all the necessary data to produce good images, but w/o an explicit evaluation of whether or not it meets the requested goals
  • These data are transmitted to ARC of PI
  • ARC staff reduce data through QA2 stage (ARCs could involve expert PIs willing to travel to ARC; identify these during ph2)
  • After QA2, ARCs tar data and products up and ships to JAO and to PIs
  • When received back at JAO, Images, scripts and QA parameters are ingested into Archive & mirrored
  • ARCs work with PIs for any subsequent data (re)processing needs (reduce effort since already familiar w/data), filing QA3 reports as necessary (also reduced effort)
  • CON: burden on ARC staff; must do less in other areas. JAO would gain experience only by reducing data from non-Executive regions? JAO would probably want to continue this model until pipeline finished. If data do NOT pass QA2, need to communicate back to JAO, get more data, do it again (same as other cases, but more latency). PRO: less disruptive for ARC staff not to travel; can involve expert PIs

Discussion
  • Which model is more promising? There was a strong consensus feeling that Plan B1 is the best plan; but only if the JAO commits to the following criteria:
    • Define a small persistent team devoted to this and available (not different people each time; not gone half the time on turno/off time)
    • We work one-on-one with specific JAO personnel (i.e. not just on our own; we can't have our people travel down there and not have people to work with. We will not just go there to reduce data - if thats what they want, then we need to move to Plan B2)
    • Computing and workspace resources must be adequate
  • What is the required effort? Who could we mobilize and at the expense of what? We felt we could probably find 3-4 people who would be willing to spend ~2weeks at a time ~3 times over the initial 4mo period. Might be able to use some of the EVLA matrixed staff. These people may not be available for CDEs or extensive work on documentation upgrades; may need to use postdocs to provide more efforts in those areas.

-- JohnHibbard - 2011-07-20

Topic attachments
I Attachment Action Size Date Who Comment
pipeline_struct.20110721.pdfpdf pipeline_struct.20110721.pdf manage 1 MB 2011-07-26 - 09:29 RemyIndebetouw remy's slides on pipeline
This topic: ALMA > WebHome > NAASC > JAOSupportGroup > JAOsupportMinutes2011Jul21
Topic revision: 2011-07-26, RemyIndebetouw
This site is powered by FoswikiCopyright © by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding NRAO Public Wiki? Send feedback