Further Development and Debugging of ALMASIMMOS

CASA Modification Request 11C108, November 2007



1. Introduction

The casa simulation package is almasimmos. It relies on sm

2. Background

Most previous ALMA simulations were made with the GILDAS ALMA simulator, memos 386 Pety 2001, 387 Pety 2001, 398 Pety 2001, 488 Tsutsimi 2004. While capable, the CASA simulator needs to go beyond the GILDAS one in several ways, notably the GILDAS simulator creates "calibrated" data by assuming that a constant percentage of the phase noise for example can be calibrated out. It is important to test whether those assumptions about calibration are true in practice with the system as implemented.

Simulation capability was once a construction deliverable as part of the OT, but this has been descoped. The most accurate description is probably as follows, although it may not have percolated to the official SSR requirements yet:

"3.1-R11 The observing tool shall be able to estimate S/N in a "calculator" using basic observation parameters (band, channel width, source flux, size scale)." Priority: 1"

"3.8.1 General Simulation Requirements Capability of simulating all ALMA modes of observation is needed for planning observations, and comparison of data with models (in data reduction, e.g. in order to test data reduction procedures and their reliability). Note that this simulator will not be directly integrated with the Observing Tool, which will have instead a simple exposure time calculator (3.1-R11). Various levels of complexity and speed of execution will be necessary. "

This effectively leaves simulation in the hands of the offline package CASA, and has the secondary effect of changing the priority order for the various functions of a simulator described in the next section.

3. Requirements

The ALMA simulation package is needed in order to simulate ALMA or EVLA data with various instrumental imperfections. At present, the package is working at a minimal state and needs more work. Prioritized uses are:

  1. Accurately generate observations for refining configurations, observing, calibration, and imaging strategies. It should be possible to generate
    • combinations of ALMA, ACA/interferometric, and ACA/total power data.
    • sequential observations with science target(s), calibrator(s) and WVR measurements Presuming that these investigations are carried out using CASA, this will also provide important CASA testing. I don't know what the requirements are for the EVLA and the widar correlator; someone else will have to modify all this for EVLA -- RemyIndebetouw
  2. Generate synthesis images of astronomical objects (existing data or models). Allow astronomers to better understand what properties, sensitivities, and dynamical ranges are possible. Making the simulator package accessible to external users from non-synthesis backgrounds will require more careful and detailed user interface and documentation than for internal/project use.
  3. Generate data with as accurate format (ASDM) as possible for testing the pipeline. In particular, metadata about the array state would need to be generated consistently with synthetic visibilities. In the near future, the CASA simulator will generate a measurement set. The pipeline requirement could be accomplished if an MS to ASDM converter is written, for example by the team that has written the ASDM filler.

4. Design

capabilities within each section are arranged in roughly prioritized order

Model Image Specification
  1. fits with WCS
    1. document the required header tags and restrictions
    2. allow images with different common flux and SB units, with automatic scaling for almasimmos
  2. fits w/o WCS ignorecoords parameter
    1. specify output image size, pixel scale, and peak or total flux level
    2. reproject and scale input image to produce output model image
    3. specify output orientation and WCS other than RA---SIN;CROTA2=0.

Parameter Specification
  1. python input parameter list
    it was proposed in 2/2007 that there be a meta-task that generates parameters for the simulator
    I think it makes more sense to implement what we can with "hidden parameter lists" until we run into an issue that requires meta-programming.
    We still should have a "verifier" step that runs either when a parameter is changed (as currently parameters out of range turn red), or at least as the first step when the task is run, halting if parameters are nonsensical. the verifier could return simple RMS noise and synthesized beam, and could take the same inputs as the full simulator, so someone could run the verifier separately to get beam and noise before running the simulator. In the sort term we will implement whatever is possible with the "hidden parameter" and existing parameter checking framework in python tasks, and leave a separate task for the future.
  2. take observation parameters from "empty/template" ASDM (could contain simulated metadata, but only blank visibilities)
    1. use existing ASDM filler to make blank MS? is metadata preserved?
    2. SB to empty ASDM task, not necessarily built in CASA but preferably callable from casapy

Blank/Empty MS Creation
  • apparently correct for ALMA 12m continuum in sm.observe()
  • not tested recently for spectral line
  • single dish?

Noiseless Visibilities
  • apparently correct for continuum in sm.predict()
  • not tested recently for spectral line
  • single dish?
  • 12+7m X-corrs?

Noise

The general strategy currently envisioned, of applying noise terms where possible via cal tables, has the advantage of simplicity and making it possible to apply cal tables to synthetic data, but the disadvantage of not providing as independent of a test of the calibration code in CASA.

Input especially requested for priorities in this section

  1. generate delay(t), Tsys(t), Tant(t) table, first with a python prototype, later c++ if required
    1. thermal noise
    2. atmospheric phase screen
    3. atmospheric emission+absorption
    4. Rx gain drift
    • this module should remain separate enough that someone can plugin their favorite phase screen model, or generate an ionospheric model for EVLA, etc.
  2. convert the simple table above to a proper cal table and apply Moellenbruck
  3. generate properly correlated tables for source, cal, WVR Moellenbruck
  4. generate polarization cal table (leakage and instrumental pol)
  5. generate pointing error tables
    • there are questions about whether the current cal table structure is sufficiently general (position-dependent) to do pointing errors fully, so we may use Sanjay's developement task for this
  6. link pointing errors to phase screen self-consistently e.g. with input parameter of wind speed
  7. generate B table(s)

Imaging/Deconvolution
  • in principle this simply calls im.clean
  1. homogeneous array some issues under investigation by RobReid
  2. optionally invert & deconvolve the noise-free image as well as the noisy one.
  3. simultaneous deconvolution of two heterogeneous arrays (but not X-corrs)
  4. real deconvolution of a heterogenous array Golap?

Fidelity RobReid has at least prototype python for all of these as of 1/08
  1. image plane fidelity image
  2. uv plane fidelity image
  3. scalar fidelities

5. Deployment Checklist

  • 15 Feb: better documentation, served through the simulation wiki SimulatorCookbook In particular make sure that fits file input is clear and correct.
  • 1 Apr:
    1. test spectral line mode and combinations with ACA and update plan accordingly
    2. make detailed plan and begin prototype for noiseless single dish simulation
    3. generate generic tables for simple noise terms (phase and amp with time)
    4. ? convert generic table to cal table and apply (requires Moellenbruck effort)
    5. prototype of new inputs with parameter expansion and checking

6. Test Plan

6.1 Internal Testing

6.2 Sponsor Testing

Much of the testing up to now has been done by Rob Reid, Crystal Brogan and Steve Myers; the development by Kumar Golap. Rob should take over the lead of testing.

6.3 Integration/Regression Tests

It would be useful to have a regression test which starts out with simulated data.

6.4 Testing for Scientific Validity

The scientific validity of the simulator can be tested by determining from the calibration and imaging. Since the image and the additional instrumental errors are known inputs to the simulator, the processing of the data should reveal the input parameters to a high degree of accuracy.


Signatures

APPROVED: I acknowledge that my request is fully contained in this MR, and if the CASA development group delivers exactly what I specified, I will be happy.

ACCEPTED: I acknowledge that I have validated the completed code according to the acceptance tests, and I am happy with the results.

Written - - - - -
Checked - - - - -
Approved by Scientific Sponsor - - - - -
Accepted/Delivered by Sponsor - - - - -

Symbols:
  • Use %X% if MR is not complete (will display ALERT!)
  • Use %Y% if MR iscomplete (will display DONE)


Discussion Area

-- NicoleRadziwill - 30 Oct 2007

-- EdFomalont - 06 Nov 2007
Topic revision: r5 - 2008-01-24, RemyIndebetouw
This site is powered by FoswikiCopyright © by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding NRAO Public Wiki? Send feedback