Imaging questions

Q1: Is there a way to track the total number of clean iterations since clean started? The guide recommends cleaning for X iterations, but I'm finding it difficult to keep track of how many iterations I've gone through, especially when I change the number of iterations in the green tool bar.
  • A: This depends on the mode with different modes reporting differently to the logger. Improvements are planned to the CLEAN logging in future CASA releases so that there will be consistent logging modes.

Q2: After self cal, my CO3-2 image changed off-source RMS from .013 to .010, and the on-source maximum changed from .89 to 1.1. This doesn't look like a substantial change. The guide indicates that I should see a substatial increase in the flux density. I'm wondering what I did wrong? I deleted my original image and cleaned again with more attention to the mask. I cleaned deeper into the image in an attempt to recover more flux. Still the numbers above are what I get.
  • A: (General point, not necessarily direct answer) Suggestion to be careful with measurement of "off-source" statistics. Considerations are: away from the source but close enough so that the statistics are representative and not skewed by the primary beam correction. Also, when considering spectral line cubes the "off-source" statistics in line-free channels and in channels of bright emission may be very different. In a dynamic range limited image the RMS in the line-bearing channels will be much higher than in line-free channels.

Q3: [ Spectral Line Imaging ] I would like to hear a clear explanation of what Doppler shifting ALMA does and doesn't do both during the observing and during the data reduction. Found the discussion to short for me.
  • A: Covered during previous session.

Q4: When using 'clean' why isn't the box labeled 'all polarizations' checked in the viewer?!?
  • A: The SV data were all considered without polarization and imaged combining polarizations.

Q5: "For illustrative purposes we first make a dirty image to see if there is emission and what the exact beam size is. It should be on the order of 1" but this will vary a bit according to the uv-coverage in the actual data. We will start with a cell size of 0.2" to oversample the beam by a factor of 5." What is meant by "oversample"?
  • A: Any spatial frequencies with a wavelength less than twice the pixel size will be aliased, i.e. for good imaging there needs to be at least two samples per wavelength, and this sets the maximum recommended pixel size. The factor of 5 quoted above is reasonable, but it is not exactly the amount of oversampling - 1 pixel per beam would be undersampling. Since the PSF includes spatial frequencies with wavelengths smaller than the beam's FWHM, at least a few pixels per beam are needed. Imaging quality generally improves as the number of pixels per beam is increased, but that also increases the computational load.

Q6: Where are the PSF and imagermodes documented? CLEAN's help is super terse on these and having a good grasp of these seems important. If there's time during the imaging part of the training, I'd appreciate some explanation of the key imagermode and psfmode choices in CLEAN.
  • A: See Steve's talk. CASA cookbook contains lower-order explanation.

Q7: Is there any ability for CLEAN to make theoretical noise maps now that Tsys tables are in place? This might simplify the comparison in the guides where one checks the noise vs. expectations (especially for mosaics) - comes up in all the imaging guides. They'd also be useful for image analysis. Just curious if this is in there or qualifies as a "wish list" item.
  • A: Potentially add to list of desirables.

Q8: It'd be good to get the lowdown on the pointing tables... if we're deleting these I assume it means that we are not applying corrections for pointing? This doesn't appear obvious to me in any of the calibration plots (e.g., with EVLA you sometimes see amplitude gain drift down with time then reset with the pointing when there are issues). Are these being used? What's their status? What's the issue that made us delete the tables. Is it just CLEAN imaging mosaics garbling things?
  • A: Only mosaicking (and fixplanets) use the pointing tables right now. Currently modifications to the pointing table made to attempt to support on-the-fly mosaicking are not working. Ignoring the table may be automatic in a future release.

Q9: Why not just delete the entire POINTING table rather than leave a table with no rows? * A: The POINTING table is required by the MS definition. Deleting it would leave a gap where some code could expect it to be there. (Some of the code can cope with its absence, but it is safer to leave a 0 row stub.)

Q10: cellsize. Is 5 pixels/beam really necessary? The rule of thumb we used in the old VLA days was 3 pixels/beam. Why is there a need to oversample the beam to such a great extent? * A: See Q5. Heavily oversampling was argued to handle point sources better by avoiding the case of a point source centered on a pixel edge with pixels large compared to beam. NRAO and/or AIPS memo from years ago. Notes & suggestions: your cell size should be informed by the actual u-v coverage / beam (after flagging). A useful best practice is to make a dirty image first (niter=0).

Q11: Is there an easier way to estimate the total integration time for the target source, rather than looking at the plot of amplitude vs time? Also, since we have real tsys information and all of the relevant parameters, it should be possible to derive the estimated theoretical noise within CASA.
  • A: See above and the CSV analysis_scripts directory contains some functionality to this effect which operates on ASDMs (readscan.py) - this may be added to CASA later. A comparison of readscan.py and listobs outputs would be useful.

Q12: What is the easiest way to CLEAN a single channel deeper, after using the same iteration for all of the channels once? (Do we need to deselect the CLEAN boxes for all of the other channels (that are finished) and run again?)
  • A: The easiest approach would be to run once to the common, shallower threshold. Remove the mask, start interactive clean (no clean box in interactive clean indeed means no CLEAN), place clean boxes only in the desired channels, then proceed.

Q13: What is the difference between uvcontsub (ngc3256) and uvcontsub2 (antennae)?
  • A: This has been covered somewhat above. Key difference is that uvcontsub2 can fit spanning multiple spws.

Q14: Antennae: Synthesized beam of the mosaic is the average beam of all pointings? If so, sparse and non-uniform uv coverage may become a problem for large mosaics, as the beam characteristics may be different for different pointing centers?
  • A: Synthesized beam is though to be an average over all pointings.

Q15: Is there an easy way to get a quantitative measure of the phase and amplitude scatter (e.g. RMS)? this will help us decide whether the wvr and tsys calibrations have worked.
  • A: Discussed during calibration.

Q16: uvcontsub2 creates scratch columns in the output MS. Is that necessary? If it assumes that the input data are calibrated, it could store the continuum-subtracted visibilities simply in the data column of the output MS.
  • A: Yes.

Q17: trying to automate the final cleans proved difficult. The result I achieved in interactive cleaning seemed better. It seems to depend strongly on the mask used. But this mask is highly subjective!
  • A: A likely driver here is that if the "threshold" parameter is set too low, CLEAN will diverge and not fill the model column. When running CLEAN non-interactively, it is important to set a sensible threshold. This may have driven this divergence. Note, too, that if CLEAN fails then the model column will not be filled by the the 'calready' option and subsequent selfcals will fail. Check for this if your selfcal implodes.

Q18: How to decide what clean box region? not happy about choice of clean box - it looks like the extent of the clean box determines the size of the emission region (in TWHya CO, for example). is this just a dynamic range issue, and will it be "fixed"with more antennas? How to decide when to stop cleaning?
  • A: This will get better with improved u-v coverage and S/N in future cycles.

Q19: what is csclean? clean method: briggs, clark, hogbom? what are the advantages/disadvantages of each?
  • A: Discussed during S. Myer's talk.

Q20: what does calready do?
  • A: Calready instructs CLEAN to fill the model column of the measurement sets being CLEANed. It is necessary to prepare a data set for selfcal. If you do NOT want to selfcal and your measurement set lacks a model data column you might want to set this to False. But probably you want true. If you set calready=False, you could in theory use the model image to fill the model column (CLEAN should probably this, FT or SETJY alternative paths).

Q21: Why is rms of map 3x theory? In TWHya I managed to get the continuum rms down to 0.98mJy. twiki gets 1.6mJy (after self-cal). time calculator gets 0.3mJy!!!
  • A: ...

Q22: Why not define masks based on the integrated data (i.e, one or just a few planes) and then use them for all instead of having to define them on the fly?
  • A: This is fine. Note however that CLEAN is not robust to changes in the line parameter. Note that the viewer can be used to help guide definition of a mask using overlays. It's important to clean these things out.

Q23: What is the reason why one of the polarizations shows systematic flat-topped bandpasses after Tsys corrections?
  • A: ... directed to offline followup.

Q24: Clear data outliers are still visible after calibration. Is this flagging included in the "effects seen after imaging" part?
  • A: Yep.

Q25: Has the correction of phases in bandpass for ~1wrap been carefully checked?
  • A: Yes?

Q26: I am still a bit confused on why are we doing a short gaincal for the calibrators to scratch some photons but are happy to work with quite less reliable solutions for the science target. I thknk that this just really states the fact that for the source we cannot actually do better because of the telescope nodding cycles, but it is obviously ignoring the fact that we have the WVR corrections already applied (?). Is this implying that the WVR corrections are NOT working???
  • A: Part of this discussed previous day as part of gaincal. The selfcal portion of this question is directed to that discussion.

Q27: Under what circumstances would we need to apply a different calibration strategy? (e.g. How is the calibration strategy changed when we observe with longer baselines, or at higher frequencies?)
  • A: The sketched scheme in TW Hydra should be fairly general. Bandpass calibration may become significantly more difficult at high frequency as sources become resolved and you being to resolve out structure. Pushing to more extreme regimes require the inclusion of better models.

Q28: The CasaGuide says data "were taken in six different datasets". What is a dataset, in this context? A contiguous observation? The output of executing a SB one time? Something else? How does a dataset relate to the output of a SB?
  • A: In this context a data set was the generated from a single run of a scheduling block.

Q29: Can you describe the observing sequence as described in the "ScanIntent"? In the workshop, it would be useful to go through the sequence of the observation. As one specific example, describe what is going on at the beginning of the observation of the NGC3256 field with the WVR setting. In general, the WVR correction is presented as a black box.
  • A: Discussed yesterday.

Q30: During cleaning, the casaguide discusses the residuals inside the mask and outside it. I believe these assessments are intended to be made at this point just by eyeballing the images. True? Or can we calculate the rms inside and outside the mask, somehow?
  • A: Image statistics during interactive clean would be very desirable. Wish list item.

Q31: In the logger, why does "clean" repeatedly report that it has used "0 iterations" to apprach a threshhold of X. Is an "iteration" defined by the removal of a single clean component from the image, for each channel?
  • A: This is CLEAN reporting on attempting to clean a channel where the signal is already below the target threshold. It indicates that CLEAN has checked the channel and decided not to use it.

Q32: It can be instructive to inspect clean components. How can we do that?
  • A: See talk by S. Myers. Clean components are not explicitly saved. They are added together to form the model column and saved in the model image.

Q33: Starting in the section on CO(3-2) imaging, it's not clear what is meant by the terms "constant frequency channels" and "constant velocity channels". Please explain.
  • A: Discussed during S. Myers talk. Constant frequency channels refers to data obtained on an even frequency gridding. CVEL and CLEAN often grid the data to a new spacing with even velocity (which implies stretches, often mild, in frequency.).

Q34: When cleaning data, the terminal shows the percentage readout several times, like so: 0%....10....20....30....40....50....60....70....80....90....100%. Why are there several such readouts, and what is it iterating over, here?
  • A: This indicates gridding and PSF creation step ... subsequent countdowns may vary by mode.

Q35: What Fixplanets is actually doing to the data? Is it only fixing the coordinates by looking the date in an ephemeris file? or is it doing something more sophisticated?
  • A: (described in help) fixplanets determines the midpoint in the unflagged observation time for that object then goes to the pointing table for the refant to get the ephemeris (this is just where this is accessible) then uses this to enter the correct information in FIELD and SOURCE table. It then re-calculates the u,v,w coordinates. So you need to choose a refant that has good pointing (not one that's flagged).

Q36: Is it possible to subtract the continuum from an other spectral window ( in frequency mode) that the one we study (if the line is too broad, or if their is a strong pseudo continuum)
  • A: Discussed earlier.

Q37: How to decide which is real or artifact to set mask to clean? is it better to use small ones and more, or to define per channel? what is best to avoid the negative regions created after cleaning?
  • A: Ideally one can do this per channel. Quicker still one can just collapse and mask all channels based on the moment0.

Q38: What does "Note for wide bandwidth data it is never a good idea to use the continuum estimate generated by uv-continuum subtraction as your continuum data" mean?
  • A: The continuum estimate from the uvcontsub ... the channel averaging loses information on behavior vs. frequency, which prevents high dynamic range imaging and tosses out information on the spectral index.

Q39: NGC3256: you will get better performance if you used imsize=120??? (5*(2**2)*3) or even imsize=180 (5*(2**2)*(3**2)) following cookbook recommendations and taking into account the x1.2 padding that clean does to your imsize underneath my timing was:
  • 100x100 = 3m57s
  • 120x120 = 3m05s
  • 180x180 = 3m13s

  • A: CLEAN pads your image under the hood by factor of 2. They

Q40: Continuum imaging of Antennae: too bad half of the data are thrown away (one of the two spectral windows) before the continuum image. I suppose this is done to speed up the reduction, but it would be good to comment on this in the guide.
  • A: Can be improved in future. There's no strong driver for this.

Q41: In the interactive clean I tried using the circle tool (a small one) and it drew a very strange shape - whazzup with that?
  • A: Circle tool is broken.

Q42: We a priori flag a lot of data, e.g. the autocorrelation data. Im sure people will wonder when and if they will ever use these data either in their analysis or calibration. Perhaps it would be worthwhile to explain why these data are taken at some level. The same can be true for all the spectral windows especially when we start "splitting" because the spectral windows get renumbered. So if you do not start the guide from the beginning an you see that spw=0 is the WVR data, that may be misleading. Perhaps we can make it a convention to keep the same numbering scheme for the windows? Just a thought...
  • A: Discussed yesterday, we keep the autocorrelation at the moment for diagnostic purposes and future improvement might attempt science with them.

Q43: Regarding removal of continuum from spectral line data, there may be a substantial amount of spectral line emission. Is there a recommended method for proceeding when it becomes difficult to single out channels that are bereft of spectral line emission (thinking toward the future here)?
  • A: Area of future improvement.
Topic revision: r10 - 2011-09-09, AdamLeroy
This site is powered by FoswikiCopyright © by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding NRAO Public Wiki? Send feedback