Algorithm Development List

This version is preparatory for development during Q4 of 2009.

Priorities are (per StevenMyers et al.):
  1. Essential
  2. Highly desirable
  3. Desirable, but not critical

Status is (per FrazerOwen):

  1. Algorithm is selected for active development
  2. Algorithm developers devise, test, and document the algorithm. Staff astronomers assist with testing
  3. Programmers code algorithm in standard package.
  4. Staff astronomers commission code.
  5. Astronomy community uses the algorithm for research

Algorithm Development Table

Subject Instrument Date required Priority Status Developer(s)
ALMA/Rau automatic editing   2009 Q4 1 d EVLA Testers
uv-data automatic editing EVLA 2010 Q4 1 d BillCotton, LeoniaKogan
Automated RFI excision   2010 Q3 1 b BillCotton, LeoniaKogan
Wide-band, narrow-field imaging of isolated sources EVLA 2009 Q4 1 d SanjayBhatnagar, EVLA Testers
Wide-band, narrow-field imaging of confused sources EVLA 2010 Q1 1 b SanjayBhatnagar
Wide-band, wider-field imaging EVLA 2010 Q3     SanjayBhatnagar
Full sensitivity full beam wide-band imaging EVLA 2013 Q1      
First order narrow-band, wide-field polarization calibration and imaging EVLA 2010 Q3      
More sophisticated approaches to bandpass calibration          
Generalized pixel representation with (orthogonal) functional dependence on frequency          
Full Stokes wide-band narrow-field imaging EVLA 2012 Q4      
Full Stokes wide-band wide-field imaging EVLA        
Wide-field spectral polarimetry imaging          
Pointing/primary beam corrections EVLA 2011 Q1 1 b SanjayBhatnagar
High dynamic range imaging, wide-band EVLA 2014      
High dynamic range imaging, direction-dependent calibration EVLA 2015      
Maximum Entropy, other deconvolution algorithm improvements EVLA 2011 Q1      
Mosaicking EVLA 2011 Q1      
Corrections for ionospheric effects EVLA 2012 Q1 1   BillCotton, HuibIntema
Rotation Measure Imaging EVLA 2010 Q2 1 c LeoniaKogan
3D Source Cataloging EVLA 2012 Q1      
3D Visualization EVLA 2012 Q4      
Specialized algorithms, e.g. recombination line stacking EVLA 2012 Q4      
Weak feature detection in large spectral cubes   2012 Q1      
Spectral dynamic range improvement for faint absorption detections          
Utility to analyze spectral line data cubes containing multiple chemical species with multiple transitions ALMA        
Improved techniques for estimating a time series in phase/group delay          
Enhanced multifacet CLEAN         BillCotton

Algorithm Implementation List

Algorithm Package Date required Implementer Reference Author
Auto-boxing for Clean CASA 2009 Q4 AmyKimball Automatic CLEAN windowing BillCotton
Auto-boxing for Clean in AIPS EricGreisen
Auto-centering of point sources onto a pixel, and within a facet       Image Pixelization and Dynamic Range BillCotton, JuanUson
Baseline-dependent averaging       Effects of Baseline Dependent Time Averaging of UV Data BillCotton

Algorithm Development Descriptions

Subject Brief discussion
ALMA/Rau automatic editing  
uv-data automatic editing  
Automated RFI excision Impementation and extension of RFI excision algorithms.
Example: A new approach which identifies and excises RFI in interferometric data using the fringe-stop pattern introduced by the correlator. This post-correlation, offline scheme has been implemented for GMRT data. This concept is of general use and may be applied to excise RFI in data from any interferometer. RamanaAthreya
Wide-band, narrow-field imaging of isolated sources This corresponds to wide-band imaging of area covering a relatively narrow field of view around the center of the antenna PB at the highest frequency. In this, errors due the frequency scaling of the PB and its rotation on the sky with Parallactic Angle (time) are minimized.
The MS-MFS algorithm (Rau, 2009), which can be configured to the simpler case of Sault-Wieringa MFS (Sault & Wieringa, 1994) algorithm where appropriate, is sufficient for this. In some (many?) cases, post deconvolution correction for wide-band PB will be sufficient. SanjayBhatnagar
Wide-band, narrow-field imaging of confused sources This algorithm is an extension of Rau's MS-MFS. The difference is that before running Rau's routine we make a narrow-band image cube and clean it. Then outside of some radius R, specified by the user, we subtract all the clean components from the uv data in each narrow-band. This way we remove all the strong sources far out in each narrow-band FOV and can run Rau's algorithm on what is left inside of radius R. For example, R might be the FWHM at the highest frequency. Using this approach we should be able to image perhaps a quite large field but we need to test the limits of this algorithm. It seems very likely it will improve the "narrow-field" imaging and perhaps allow rather wide-field to be imaged. SanjayBhatnagar
Wide-band, wider-field imaging For frequency spread of >10% imaging beyond approximately inner 10% of the PB, residual errors due the frequency and time dependence of the PB become significant. Errors due to frequency scaling of the PB and time dependent errors due to antenna pointing errors and rotation of the PB with Parallactic Angle (PA) are maximum at the half-power point of the PB1. Therefore, for wide-band continuum imaging of fields wider than the inner 10% of the sensitivity pattern, integration of the MS-MFS algorithms with the A-Projection algorithm is required. SanjayBhatnagar
Full sensitivity, full beam wide-band imaging Full sensitivity imaging, particularly at lower frequencies will require image deconvolution out to at least the first side lobe and correcting for time variable antenna pointing errors. SanjayBhatnagar
Wide-field, image-plane, instrumental polarization correction In order to correct the full-field, instrumental polarization to fist order, this algorithm will 1) calculate the mean full-field instrumental polarization from a model averaged over the observed Parallactic angles for Q and U, then 2) multiply this correction by the observed total intensity image and 3) subtract the resulting correction image from the observed, uncorrected Q and U images. SanjayBhatnagar
Faraday Rotation Synthesis Imaging The algorithm of Brentjens & de Bruyn, (2005) is implemented using observed Q and U cubes and the resulting 3D imaged cleaned to produce an estimate of Faraday depth in each pixel. LeoniaKogan
More sophisticated approaches to bandpass calibration Currently this is mostly done with estimating the antenna/datapath specific complex gain on a channel-by-channel basis from observations of a very strong source. The actual function with frequency is much better behaved and some sort of functional representation may allow using weaker sources and/or give better results. Also, time variable.
The instrumental (spurious) polarized response to unpolarized emission is a function of frequency and needs a better way of characterizing than broadband averages. Can/should this be incorporated into bandpass calibration?
An efficient way of parameterizing (and modeling if possible) the antenna power pattern as a function of frequency. An efficient way of parameterizing (and modeling if possible) the antenna polarized response as a function of frequency. BillCotton
Generalized pixel representation with (orthogonal) functional dependence on frequency What is the appropriate representation of a pixel in a broadband continuum image? Some representation of the spectrum (spectral index, curvature...) is necessary. Expansion of ln(flux) in powers of ln(nu) is traditional for overall spectra but this is not well adapted to imaging. An orthogonal basis set could be of great benefit in imaging. There needs to be a single parameter that can be used as a proxy for brightness in the traditional representation.
The linearly polarized emission (Q + jU) in a pixel is even more complex than the spectrum of a pixel as the "polarization angle" is also a function of frequency. A single Faraday screen imparts a ramp in polarization angle proportional to lambda^2 but a complex emission/transmission case can have a more complicated behavior. Some paramaterized way of representating this behavior per pixel is needed. It would be best if this were related to the solution to the pixel Stokes I spectrum problem. BillCotton
Full Stokes wide-band narrow-field imaging Full-Stokes imaging close to the center of the PB, where direction dependence of the PB polarization is relatively weak. This will involve extending the MS-MFS algorithm to the full-Stokes case. SanjayBhatnagar
Full Stokes wide-band wide-field imaging As in the case of Stokes-I imaging, this is the final goal and will involve integration of algorithms to correct for direction and time dependent polarization effects with MS-MFS algorithm. This will probably also require incorporating Faraday Rotation Synthesis. SanjayBhatnagar
Wide-field spectral polarimetry imaging  
Pointing/primary beam corrections  
High dynamic range imaging, wide-band In addition to the corrections for the direction dependent gains varying with time and frequency, high dynamic range imaging will also require algorithms for better modeling of complex emission as well as accounting for pixel quantization errors for compact emission. Conventional deconvolution algorithms (CLEAN and its variants and MEM and its variants) model the sky in the pixel bases. This fundamentally ignores coupling between image pixels in images with extended emission and the fact that even compact emissions may not be centered on an image pixel. Both these problems lead to residual deconvolution errors which limit the highest achievable imaging dynamic range. Several algorithms which model the sky in a scale-sensitive basis have been recently developed. Most promising of them are the Asp-Clean and the MS-Clean algorithms. Note that in principle some approaches to multi-scale deconvolution can also correct for the pixelation errors (Voronkov & Wieringa, 2004). An alternate approach based on mutli-facet imaging to correct for pixelation errors also exists (Cotton & Uson, 2008). While the MS-Clean algorithm exists in CASA, Asp-Clean algorithm has not yet been ported from AIPS++.
This involves extending the multi-scale deconvolution techniques for scale-sensitive modeling of the emission along the frequency axis as well. Run-time efficiency of the available algorithms for large data volumes will be the crucial parameter to determine the optimal algorithm to use. SanjayBhatnagar
High dynamic range imaging, direction-dependent calibration Achieving high imaging dynamic range depends on calibration and correction for direction dependent errors. Since it is nearly impossible to measure these direction and time varying gains at the required accuracy and time resolution, algorithms to solve for these effects will be required. SanjayBhatnagar
Maximum Entropy, other deconvolution algorithm improvements  
Mosaicking  
Corrections for ionospheric effects  
Rotation Measure Imaging  
3D Source Cataloging  
3D Visualization  
Specialized algorithms, e.g. recombination line stacking  
Weak feature detection in large spectral cubes  
Spectral dynamic range improvement for faint absorption detections  
Utility to analyze spectral line data cubes containing multiple chemical species with multiple transitions  
Improved techniques for estimating a time series in phase/group delay This is a critical component of phase (astrometric) calibration. The current technique is to estimate these independently in a series of "solution intervals" which may later be smoothed. Direct estimation of a time series functional form could force continuity and possibly other physical constraints which could improve the quality of the representation at high SNR and allow using lower SNR data for calibration. BillCotton
Enhanced multifacet CLEAN It is possible to align the facets such that all the pixels are on a common grid, i.e, the pixels themselves are aligned. This allows multiple facets to be used in a common minor cycle CLEAN. The dirty beams of adjacent facets are close enough that the dirty beam of one will allow approximate subtraction of the response of a source in one facet from adjacent facets. Thus, all facets with significant emission can be CLEANed in a single major cycle; this allows a major reduction in the number of major cycles (I/O) and more opportunities for parallelization.
The primary benefit of this should be a major performance increase. The potential payoff is large enough that it's worth a fair effort. I've started looking into it but haven't gotten far enough that I'm either really confident it will work (are the increased(?) W distortions managable?) or that I can predict the total amount of effort. BillCotton

-- GarethHunt - 2009-08-21
Topic revision: r21 - 2009-12-03, FrazerOwen
This site is powered by FoswikiCopyright © by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding NRAO Public Wiki? Send feedback