KFPA Pipeline Overview

Pipelines

  • Imperative for large focal plane arrays (high data rate).
  • More than just a handy quick-look tool.
  • Defaults should be sensible to avoid the need to reprocess the data.
    • Defaults may be context specific (vary with use case).
    • Reprocessing is painful.
  • Not a data processing environment (i.e. not interactive like GBTIDL or AIPS).
  • Likely to be useful in processing data from other receivers with more leisure data rates.
  • Other telescopes routines provide roughly-calibrated data to their users - most institutions consider this the starting point of data reduction.

Goals of the Prototype Pipeline (KFPA Critical Design Review - Jan. 2009)

  • Support KFPA commissioning
  • Explore new processing tools/techniques not yet widely available in GB (vector calibration, statistical data flagging and editing, visualization, parallel processing).
  • Prototype an automated pipeline - add necessary meta data to capture user intent
  • Prototype tools necessary to support larger focal plane arrays (e.g. parallel computing)
  • Based on prototype tools, estimate costs associated with delivering a pipeline and necessary computing hardware to handle the expected data rates for a larger focal plane array.

Out-of-scope items (deferred) (Critical Design Review)

  • complicate calibration schemes (e.g. "basketweaving" or it's FPA equivalent)
  • cross-correlation
  • continuum

"The initial observing modes should be kept simple yet effective - even at the cost of reduced flexibility for observers." - GBT KFPA Critical Design Review Final Report.

This topic: KPAF > TWikiUsers > BobGarwood > KFPAPipelineReviewOverview
Topic revision: 2010-06-16, BobGarwood
This site is powered by FoswikiCopyright © by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding NRAO Public Wiki? Send feedback