Questions on calibration

OBSERVING STRATEGY

Q1: How common is it to observe a secondary phase calibrator; when should that decision be made?
  • A: There are no clear cut answers to this question. This depends on science goals and your knowledge of the sufficient brightness of a nearby calibrator. In the case of TW HYa, the secondary calibrator was chosen partly to check for proper motion being properly handled in data reduction. In general, this will be used for high precision astrometry and is used to measure phase variations. It can also be used to hedge your bets between a weak/nearby source and a bright/far-away calibrator. Applying the phase from one calibrator to the other can be used to determine the uncertainty in the position of your science source, which depends on phase transfer as well as the uncertainty in the primary calibrator position.

As an aside, self-calibration on your science target does not necessarily destroy your positional accuracy.

Q2: I think the case for claiming that Titan as a flux calibrator is a bad choice is well justified here, but this is the same calibrator used for ALL SV data with casaguides available! Isn't is more fair to say that it's only when strong lines are in the spectral window that this matters? And if so, what frequencies do we need to worry about? It seems like this is something that is not well established yet. Is there a plan to survey the flux calibrators over the entire bands to better inform people on this in the future?
  • A: Titan was, at the time of SV, one of the only sources available for flux calibration. One must take care that there be multiple spectral windows to make sure that at least one is not corrupted by broad spectral lines. This is important for other planets, moons, and quasars in the galactic plane. Care must be taken during data reduction, especially because ALMA is so sensitive!

As an aside, is there a plan to do spectral sweeps of flux calibrators? Not to our knowledge.

Q3: Titan was close to Saturn during these observations. Doesn't this violate QA0 guidelines? What precautions will be taken to ensure this does not occur with user data?
  • A: It has been our intent to define observing modes that provide constraints in the software that prevent this from happening, but we're on there yet. For now, it is up to the AOD to check this, and it is more of an issue at longer wavelengths.

Q4: Why do we need to apply the phase (on int time) gaincal at all? why is WVR not correcting for this on a ~10s timescale?
  • A: Part2 - The WVR takes data on 1s intervals, so longer integration times degrade data quality (with the benefit of reducing data size). Real life testing of the WVR has not be consistently strong, so we are not yet sure that the WVR can (on its own) remove the 10s level variations.

Q5: Antennae: On the face of it, it is surprising to me that 3c279, ~19 degrees away, is a good phase calibrator for this source. What is introducing the phase errors that this calibrator will correct? On what timescale does the atmospheric phase vary? Is that entirely corrected by WVR? Are there any types of observations that would require rapid measurement of phase as we do in "phase referencing" observations with the VLA or VLBA?
  • A: We have not yet created a complete grid of possible calibration sources, so in some cases the one might be stuck with a phase calibrator at this distance. We hope to know the fluxes of more possible calibrators in recent measurements, so that we can find a better database of weaker (but still strong enough) and closer calibrators. It was suggested that pre-observations be made to check calibrator fluxes.

REDUCTION STRATEGY

Q6: The pointing/atmosphere files are not flagged, as was done in the AntennaeBand7 reduction. Is this important?
  • A: In some guides these files are flagged, but not all. All Tsys measurements are made in TDM, all pointing data are (supposed) taken in TDM. In the NGC3256 data, all the data were taken in TDM, so the pointing and Tsys data to be flagged. In TW Hya, the science data were in FDM, so just by dealing with the FDM, the pointing and Tsys data were removed. In the case of NGC3256, for some reason some of the pointing and or Tsys data were taken in FDM, so the data had to be flagged based on intents.

Q7: What is the proper way to parse the scan intent output by listobs? It is cryptic.
  • A: It is possible that the next release of CASA will put scan intent as an input into listobs. There are already was in AnalysisUtils to let you do this now.

Q8: What is the difference between FldId and SrcId? They appear to be identical in the NGC3256 Band 3 guide.
  • A: There is a bug in data capture that manifests itself when there is both FDM and TDM, with IDs assigned as the sources are taken. Sometimes one could see the same target with multiple numbers. CASA used FldId almost exclusively, and after splitting out the science data, these numbers are correct. When in doubt, use field names (not numbers) because they are always unique. SrcId is used in mosaics, but that's it (the source table is even optional).

Q9: How much time averaging is it safe to apply to (i) compact array, and (ii) extended array observations. Should this depend on the science goals?
  • A: Anita Richards has been investigating this:
ALMA timesmear.pngALMA0.95bwsmear.pngALMA spindex smear.png

The plots above show the time and frequency binning that can be tolerated. Time smearing acts as a decorrelation and supresses the peak in the outer parts of your image, so it is important for sources not at the phase center. We are working on a tool to help decide what can be tolerated.

Q10: "For the second day, the baseline DV10-PM03 is corrupted." How? I don't see any problem with this baseline. Later: Okay, after calibration the problem with this baseline became apparent. How did I miss it eariler...?
  • A: It is common, especially as higher frequencies where sources are weaker, to miss these problems before calibration. After calibration, with higher signal-to-noise, it is easier to find these sorts of errors.

As an aside, it was asked whether it is possible that the first pass through of flagging could be done automatically, but Crystal would not yet trust such an algorithm.

Q11: "However, the depth of the absorption feature is exaggerated due to the way ALMA normalizes the cross-correlation spectra by the auto-correlation spectra prior to storage." How does normalizing the cross-correlation spectra by the auto-correlation spectra exaggerate the strength of the absorption feature?
  • A: This normalization is done for the benefit of telcal, to get rid of some bandpass shape. The single dish sees this line in emission, but the interferometric observations see this in absorption, so normalizing by the single dish (auto-correlation) exaggerates this atmospheric line. It could be useful to be able to un-normalize data, especially for this interested in observing near atmospheric water lines.

Q12: When warned about concatenating data not taken close in time, how long is close in time? What needs to be considered besides antennas being moved?
  • A: It was a mistake in the guides to concatenate data sets. In general, it is not safe to concatenate data sets. For example, in bandpass calibration one can not always track variations from scan to scan. Instead, we combine across scans. If data from different days or times are concatenated first, then the bandpass may be different in the two data sets. You are probably OK to concatenate after splitting off target data, just before imaging, though maybe when many antennas have moved it is still not a good idea. Perhaps this problem comes from moving antennas, but more investigation is coming. One could use multiple ms files in the call to clean for imaging (we think it even works for mfs imaging).

Q13: Reference antenna is DV07 in self-calibration, but DV04 was used at the calibration stage. DV07 is the antenna with delay problem. Is this OK?
  • A: It is better to use a consistent reference antenna throughout.

Q14: How do you chose a reference antenna? Q14: How do you choose a reference antenna? * A:

Q15: Antennae: In the output of listobs, why are the number of visibilities (i.e. nVis and Int) slightly different for different pointings of the mosaic?
  • A: The number of visibilities is different because one specifies the total observing time, but the slew time varies and counts against the observing time. The integration time changes, but we don't know why.

Q16: Will ALMA data come to user with autocorrelations flagged at some point in the future? Does the autocorrelation data have any science use?
  • A: Autocorrelations can be used to undo the normalization, at some point in the future. Autocorrelations can also be used to diagnose correlator problems. Autocorrelations can be used for single dish observations (in the future), to find RFI, or for calculating Tsys.

Q17: Compared to the AntennaeBand7 casaguide, In the NGC3256 Band 3 guide the WVR and Tsys calibrations are applied in two steps here, rather than in one. Is there any benefit to doing so? E.g., can split after the WVR tables are applied and remove spw's that are no longer needed? Also, the "gainfield" and "field" parameters in applycal are not set to the same value for the WVR corrections (but they are for the Tsys correction), as recommended in the AntennaeBand7 guide. How important is it to do this?
  • A: The main benefit of separating the steps would be to see the effect of the two calibrations independently. The benefit of combining the two steps is to save disk space and time. Note that you need to split after each step, if separating Tsys and WVR calibration.

Q18: In this guide, the WVR/Tsys calibrations are applied before the initial flagging (e.g. autocorrelations, shadowed data). Is there a benefit to doing this in one order or another?
  • A: No, the order is not important. In Cycle 0, we will pre-apply Tsys and WVR before PIs get their data.

Q19: Do we really need to do so many splits for this dataset?
  • A: The calls to split in the TW HYa and Antenna are all necessary to guard against things going wrong. You definitely need to split after Tsys and WVR, and you definitely need to split before imaging (to make self-calibration smooth). Calibration always looks at the data column. This is also safer because data can be corrupted during the imaging stage with a control-c.

Q20: It could be useful to have information about the weather condition during observations. Is this planned?
  • A: See in analysisUtils: au.plotWeather('X5d8.ms')
X5d8.ms.weather.png

Q21: Why do we restore the spectral flagging we did on the calibrators ?
  • A: This was only done for educational purposes. It is important to unflag before applycal, or else the previously flagged data will not be calibrated.

Q22: Antennae: Missing Phase Calibrator Observations: I don't think we should recommend to always flag all data that is not bracketed by gain calibrator observations. If the phases are very well behaved, it is perfectly fine to extrapolate.
  • A: It is sometimes OK to extrapolate, but make a note to check the data later. You can always try it both ways and see which works better.

SHADOWING

Q23: For datasets with a lot of shadowing, is there a guide for how much the diameter can be safely relaxed?
  • A: Possible answer is in the plots:
EVLA: EVLA: ALMA: ALMA: What does it look like when you have shadowed data? What you see is large variations in amplitude (typically on the high side). You can use uvdisk to look for possible shadowing.

If there is an antenna in the array that is not being used, is shadowing going to be handled correctly? No, CASA doesn't know about the antenna, and right now there is no solution for this problem.

Note that shadowing is a purely geometric effect, and does not really depend on the primary beam size or shape.

WVR

Q24: How are wvr observations made? What's different about wvr scans as opposed to regular target scans?
  • A:

Q25: "We then once again split out the CORRECTED data column, this time only retaining spectral windows 1,3,5,7. This will get rid of the extraneous spectral windows, including the channel averaged spectral windows and spw 0, which is the one that contained the WVR data." Why was it necessary to carry along the extraneous spectral windows and the WVR data the first time split was run?
  • A: Before you run wvrgcal, do not split the data. The same goes for Tsys. It is recommended that you do both calibration stages before splitting off the science data.

Q26: How do we check (and know) if the wvr data are correct? I plotted it and it looks like most of them have 1.
  • A: Part 1 - It is hard to say if it is correct on its own, but you can check what it does to your data to see if it improves coherence. It is not that useful to look at the wvr cal table, but that's a total power observation, and what you need is a cross-correlation correction. Part 2 - Perhaps you are plotting amp vs time, and not phase? The phase solution is the important bit of wvr, and the amplitude solution is going to be 1 by definition. Only use an antenna with a good wvr solution for a reference antenna.

DELAY

Q27: In the implementation of the delay correction, the parameter set has 8 values. How are the parameters for spw 0,2,4,6 chosen when there are no curves with which to estimate the delay? In the next step, cals are only applied to spw 1,3,5,7 not the windows 0,2,4,6. I realize this means that what you put in the above step doesn't really matter, but then why not just put in delays where you know them? What would happen if you tried to apply cals to all the spws?
  • A:

Q28: While I understand how the delay is measured and applied here, I do not understand where the 3.05 or 1.10 numbers come from and what they exactly apply to. And what is meant by a "K-type" delay table -- are there other type of delay tables that one needs to check etc?
  • A:

Q29: We corrected delays here before going ahead with bandpass calibration. But how bad could the delays be and we can try to take them out using bandpass calibration only? (I heard people have done this in AIPS even when there is wrapping). What is considered good and bad practice along these lines?
  • A: The bandpass would likely have fixed the delay in this case, because the correction was relatively small.

Q30: In gencal, the parameters are listed in the order [spw1 XX, spw1 YY, spw3 XX, spw3 YY, spw5 XX, spw 5 YY, spw7 XX, spw7 YY] rather than [spw1 XX, spw3 XX, spw5 XX, spw7 XX, spw1 YY, spw3 YY, spw5 YY, spw7 YY]. The CASA help for gencal says "When specifying a long list of calibration parameter values, these should be ordered first (fastest) by pol (if pol!=''), then by antenna (if antenna!=''), and finally (slowest) by spw (if spw!='')." From reading that, I would have thought to go with the second ordering, rather than the first (the one that's used in the guide). Do you have a good way to remember which one is correct?
  • A: This will be answered offline.
for i in range(3): # loop over the first three ms's
	name=basename[i]
	os.system('rm -rf cal-'+name+'_del.K')
	gencal(vis=name+'.ms', caltable='cal-'+name+'_del.K',
	caltype='sbd', antenna='DV07', pol='X,Y', spw='1,3,5,7',
        parameter=[1.00, 1.10, -3.0, -3.0, -3.05, -3.05, -3.05, -3.05])

Q31: Let's say I've run gencal but then realized I've screwed up the inputs. What do I do? Do I have to totally start my reduction from scratch is there a way to undo my mistake so I don't essentially lose everything I've already done to this point?
  • A: If this is a question about gencal, you can just remove the table and run gencal again. You will have to redo calibration that depended on the gencal table. If this is an applycal question, then this will be answered later on.

TSYS

Q32: This is the only casaguide which says that it is important to check whether there are Tsys measurements for each field. Is this something that we generally should worry about?
  • A: These data sets were among the first to have Tsys measurements. It is important to check which sources have Tsys measurements and which do it, and then choose which measurements to apply to which sources. You want a measurement close to your target, in both elevation and time. This is controlled via the gainfield parameter.

Q33: "Upon examination of these plots, we see that for uid___A002_X1d54a1_X174.ms, there is a outlying feature in spw=7, antenna DV04. This feature occurs during scans 5 and 9, so we flag those data with the following command."
  1. How is the scan of the feature identified? I've plotted the Tsys table for spw 7 and iterated over time. I don't see any Tsys measurements for scans 5 or 9. The feature appears in scan 8.
    • A:
  2. Why do we flag data when there is a "feature" in the Tsys measurements, rather than keeping the data and calibrating it with the Tsys measurements? I would assume a feature in the calibration table is compensating for a feature the raw data... Is this a bad assumption?
    • A: It is fine to apply Tsys and then flag data that have problems exacerbated by a Tsys measurement that is suspect. You can look at your data and your Tsys measurements to see if you trust the Tsys value. It is OK to use data that have been corrected by a high Tsys, since the weights of the data will be correspondingly adjusted by the inverse of the Tsys. If the antenna is not behaving well in general (not just Tsys), just flag it.

Q34: "currently Tsys data can only be taken in the wide (TDM) correlator mode with 128 channels per spw, these correspond to spws=1, 3, 5, 7 above. The frequencies of these TDM spw were carefully chosen so the the narrowband FDM spws fall within them. What are TDM and FDM acronyms for?
  • A: Time Domain Modulation (coarse spectral resolution) and Frequency Domain Modulation (high spectral resolution).

Q35: Antennae: says "The Tsys values in Figure 2 look reliable, with typical values ~150 K except for some large values of Tsys at ~300 and 400 K for DV04. We will flag the data for that antenna later. " We flag the DV04 related data for uid___A002_X1ff7b0_Xb, but it looks like the Tsys is generally higher (by ~ x2) in all data for DV04. Should we flag all data related to this antenna?
  • A:

Q36: why does increase in flux at end of SPW3 not get removed by Tsys calibration?
  • A: The normalization from autocorrelations prevents this from getting corrected by Tsys.

Q37: what are the units of amplitude after Tsys correction?
  • A: Units should be on a Jansky scale, if all has gone well.

ANTENNA POSITIONS

Q38: In the Antennae CASA guides, we applied antpos tables to improve the antenna positions. Could you discuss with us diagnosing problems with datasets, such as poorly known antenna positions or any other issues you think we could potentially see? What things do we need to be on the look out for?
  • A: It is hard to know when new antenna solutions have been calculated. By the time ES starts, the positions should be good to 0.2 mm. You will notice a residual error when applying your phase correction to your source (especially on the secondary calibrator). Self-calibration can correct these antenna position errors.

Q39: when to use corrected antenna pad positions, ie how much does an incorrect pad position affect the results (and how is it seen)? how to measure & check they are correct? this is not done in the guides.
  • A:

Q40: Is it possible to check the accuracy of the antennas pointing and tracking? Or we just check the elevation and azimuth in function of time.
  • A: Who knows?

FIXPLANETS

Q41: fixplanets was not done in this guide, but rather, this step was done in the pre-calibration, so that the MS's that the user downloads in the tgz file already have the correct position for Titan. I believe in this case, this is okay, since every step involving the bandpass and gain calibrations is carried out separately on each MS. However, in the NGC3256 guide, this step is not done until after the data are concatenated, which happens after the WVR/Tsys tables are applied. It is not clear to me when this task needs to be performed, in relation to other tasks.
  • A: It is now clear that you should fix the coordinates early in the calibration.

Q42: "It is important to apply the fix after concatenation of the data, or Titan will be at slightly different positions in all three datasets" why?
  • A: Otherwise your image will be smeared. The u,v,w coordinates will be wrong if you have the wrong position for your source. In Cycle 0, in a single dataset the position will be correct for the start of that data set. You will still do to correct the position to combine data between data sets if you want to image the source.

BANDPASS

Q43: Why set fillgaps = 17 for bandpass? 17 seems arbitrary. In some other guide, fillgaps=1 was used.
  • A:

Q44: Is there a way to make a time-varying bandpass solution, for instance by not combining across scans?
  • A: Yes, but the default is to combine across scans, so you have to change the default.

Q45: When bandpass calibration is done, why are phase-only solutions rather than amp+phase solutions determined?
  • A: This is in reference to the gain solution before the bandpass solution. The phase solution removes decorrelations that would hurt your bandpass solution. Because the bandpass is fairly flat and the WVR seems to work well, you can probably get away with skipping the early phase solution. Ed points out that he sometimes uses an amp+phase solution before bandpass calibration.

Q46: In the gain calibration step: bandpass is applied on the fly with a interp=linear (default) option, but the bandpass calibrator is observed only once for each dataset. We should explicitly specify interp = 'nearest'?
  • A: It is safe to explicitly set your interpolation scheme. If you have a single bandpass solution, your interpolation scheme doesn't matter.

Q47: Antennae: PM01 shows large ripples in the amp vs. freq plot of the bandpass. shall we flag? phase appears to be OK.
  • A: If the antenna really has the ripple, then it is fine to calibrate it out. Then apply the calibration and look to see if it improves your data on another source. If the ripple changes with time, just flag that antenna.

Q48: the bandpass is noisy - especially in FDM as the resolution is small. is is not better to smooth the bandpass before applying it?
  • A: In many cases it is a good idea to smooth the bandpass. It should be fine to smooth across several channels, but some people suggest smoothing your data that same amount. Some people use bpoly for this, but that requires fine-tuning your optimal polynomial order.

Q49: Gaincal for short times: How many channels should be averaged to get good snr, but without introducing phase vs freq variations? Does it depend on the number of channels? or the particular data set?
  • A: See above.

Q50: In the bandpass calibration for TWHydrae, it is noted that significant variations in the phase across some scans as well as smaller but significant variations across a scan. So, the method chosen to handle this is to come up with an independent bandpass for each scan instead of combining scans for bandpass.
  • First, what is defined by significant? Is this example an upper limit? Or is there a general rule of thumb in which a user can decide that they can combine scans for the bandpass calibration versus needing to take a more individual approach?
    • A:
  • Is there always benefit from just applying an independent bandpass for each scan or can this cause problems in some cases?
    • A:

Q51: Antennae: The first applycal takes very long to finish (more than 5 hours on our machine). Since we later split out only one spw anyway, it might make sense to do this split right at the beginning. This will speed up the applycal later.
  • A: This is likely an indication of something going wrong on your computer, perhaps a lack of memory or someone else is also using your computer. You probably need 16 GB of data for this data set. Shared disk arrays can also slow you down.

Q52: If you have a not-so-strong bandpass calibrator (or partly resolved), then applying it might add significant noise to all sources. Aside from using bpoly, is it possible to smooth the spectrum, then solve for bandpass, then apply the solution to unsmoothed data? This could be useful in FDM mode, since the features in the bandpass are typically many channels wide. The smoothcal task seems to only smooth over time.
  • A:

Q53: Do we have to worry about the intrinsic spectrum of 3C279 using it as a bandpass calibrator?
  • A: Yes. We have models for the spectral index for planets (in effect) but not for the quasars.

GAIN CALIBRATION

Q54: Several of us had questions about using the intphase gaincal on the phase calibrator to guard against decorrelation on shorter times then using the scanphase gaincal on the target object, which may suffer decorrelation especially if the phase cal does and especially if the source is too weak for selfcal 1 - This is always the correct thing to do for your phase calibrator, but the other way can be wrong. WVR corrections may make this unnecessary, but still correct. Make sure to use the integration time only when you have enough signal to noise in an integration time. You can check this by looking at the solutions for both polarizations. 2 - For our target, we use a scan solution, but we still have the large phase variations seen in the phase calibrator. These can be corrected via a self-calibration on your target. If you can not self-calibrate, then it is less clear what to do, because you want to know how de-correlated your science data are, but you also want to calibrate your science target as well as possible. If your data are really poor, it doesn't matter much what you do, your science target will still be unlikely to be detected.

Q55: During the amplitude calibration you apply a decorrelation correction but we don't in NGC3256. We talked about this at the original train-the-trainers and I think I am on board that you should always use your approach. Because both the primary (flux/Titan) calibrator and the secondary (phase) calibrator will have internal decorrelation, it seems to me that doing this or not doing this doesn't bias you in a particular direction - it just makes you more accurate if you do correct for decorrelation. So I'm on board, though I don't know the magnitude of the correction in general (it's a couple 0.1s of a Jy out of 10 on 3c279 to do this or not do this in the Antennae Band 7). How critical do you view this step to be? Aside from just best-practice.
  • A:

Q56: We selfcal in each case in the CASA guides. If you don't selfcal and you're looking at a point source, you will get less flux due to decorrelation. It's entirely possible that we're going to get projects like this (chasing some 5 sigma point source at redshift crazy or something). If you see it and you can't selfcal then your flux will be biased low by phase scatter right? What's the remedy here? You don't want to report fluxes biased low? Is there a remedy to correct for this? You could measure the phase scatter on the calibrators and then attempt a statistical correction, but this seems complicated... a secondary calibrator seems like a good option but this isn't standard practice, right? If you had one would you believe the correction factor you derived using it?
  • A: Steve M. suggests a tool to figure out the decorrelation's effect on your amplitude.

Q57: What exactly does the term "on the fly" mean in these contexts?
  • A:

Q58: Is there an easy way to plot and check phase closure?
  • A:

Q59: At Gain calibration, should we use interp = 'nearest' to be consistent with the applycal command to be used later, rather than using the default 'linear'?
  • A:

Q60: Looking at phase vs. time (eg after initial cal, or after self-cal), it's not really clear what a "discrepant" solution is here. there are still apparently phase jumps for different SBs. and for different targets. how do you judge this quality, and what is an acceptable phase change between integrations?
  • A: If you know you can self-cal, then only egregious phase jumps need to be considered. If you can not self-cal, you need to be more careful, which may mean flagging the offending data.

Q61: TWHYA: Why are so many different steps in generating the gaincal being used?. In the end, we are using self-cal and also at that stage it clearly shows that there has been an error e=in the process (a clear outlier for a VERY significant amount in phase)...
  • A:

Q62: In previous tutorials for getting the bandpass solution we use minblperant=4, but here we use minblperant = 3 in spite that we have more antennas. Why do we use 3 here instead of 4? How this parameter affects the quality of the solution?
  • A: Crystal prefers a value of 4, you can choose as you wish. 4 is the actual minimum needed for an amplitude solution, 3 for phase only.

Q63: In case of clear phase jump, can we introduce a "break" i.e. determine two independent solutions?
  • A:

Q64: Why do we solve only for amplitude (calmode=a) instead of calmode=ap ? The guide says: "even though we allow phase to vary, because most phase variations are removed by the "intphase" solution." So what is the difference by doing calmode=ap, as it is done in other guides?
  • A: calmode=ap can be used to look for the residual phase, but just an amplitude solution is OK.

Q65: After doing the last round of flagging for NGC3256, only amp vs time and phase vs frequency are shown. Yet the phase vs time plot shows a lot of scatter in a few individual integrations (attached plot shows one example between 24:13:20 and the next tickmark). Was this overlooked, or are such scatterings in phase acceptable?
  • A: If you think that this would affect the calibration solution, then we need to correct it. Otherwise, let it be.

Q66: [ Gain Calibration: Error: not an image at 'http://casaguides.nrao.edu/index.php' ] I haven't figured out why the gain amplitudes are lower for Titan than for the secondary calibrator. I had expected them to be the same but perhaps the measurements imply that the Titan set-up is different. If that i the case, how does CASA know how to deal with this situation when performing the rest of the calibration steps?
  • A:

FLUXSCALE/SETJY

Q67: [ Gain Calibration ] Not sure what is meant by the following line 'Unlike the gain calibration steps, this is not an incremental table'. I am assuming it has something to do with which of the aspects of calibration can be done iteratively and which make permanent changes but I am lost on how this is being done in practice and how I whether it should be obvious to me which are which.
  • A: For fluxscale, you feed it the amplitude table and adjust the amplitudes in that table, and then apply just the fluxscale table (which has the amplitude table already incorporated into it).

Q68: I don't understand using fluxscale to transfer the flux scaling from Titan to 3C279. Why not transfer the flux scaling from Titan to the Antennae sources? What's the advantage of using the secondary calibrator as a middle-man?
  • A: This allows you to apply the time-varying portion of the amplitude solution to your science target, which is not included in your Titan solution.

Q69: For fluxscale, since we flagged Titan: does it automatically use nearest for interpolating? How does it know what Titan observations to use in transferring to 1037?
  • A: This can be set in the gaintable and gainfield.

Q70: In fluxscale -- how is the SNR calculated? Peak flux / rms in some area - where and how is the rms calculated?
  • A: We'll try to find out from Geoerge.

Q71: In the TW Hya band 7 imaging guide, it is noted that "the spw do not have the same amplitude. Unfortunately, the red spw (highest frequency) should be higher than the green one (lowest frequency) because optically thin dust goes as nu^4, and optically thick as nu^2. This inconsistency is due to imperfect amplitude/absolute flux calibration." I'm not sure what calibration error we're seeing here.
  • A: This may be due to problems with fluxscale, but we're not sure.

Q72: If you know the spectral slope of the SED of your calibrator (or self-cal) source can you force the amplitude calibration to reproduce that spectral index within the sub-bands rather than just between them? I would think this is an issue for 2GHz sub-bands (continuum/low-res spectra) if you are trying to squeeze out the maximum image fidelity. Again, might be a CASA development issue.
  • A: Is the scalebychan parameter in setjy what you are looking for? When scalebychan is True and a standard is being used (as opposed to a user-supplied value), the flux density will be calculated for each channel. If a user-supplied value is being used, the spectral index can also be given with spix (in recent CASA versions).

Q72B: bandpass correction of the spectral index; fine if your flux and bandpass calibrator are the same but otherwise, multiple runs of bandpass are required - I don't think our tutorials illustrate this.
  • A: We don't know yet.

Q73: After fluxscale, what is reasonable for the amplitude scatter as a function of uvdist? 10% is reasonable as plotted?
  • A: It depends on the signal to noise for your calibrator.

Q74: Will the situation with the ephemeris objects coordinates not being stored propely in the ASDMs improve with new ACS releases?
  • A:

Q75: Why is a fluxscale done PRIOR to doing an applycal to the data?
  • A: It would be useless to do it afterwards, since only applycal changes the data.

APPLYCAL

Q76: applycal is called with gaintable set to both the gain calibration and bandpass calibration tables. When the gain calibration table was created, the bandpass table was applied "on the fly". What does "on the fly" mean? It sounds like the data is corrected, a new calibration table is created based on the corrected data, and then the corrected data is thrown away. Is this correct?
  • A: This indicates that the phase solution will be used in a transitory way, just to find the bandpass solution, and this has not changed your data in a way that persists. You need to apply these tables using applycal for the changes to stick.

Q77: In terms of running applycal, please go in to more detail about what is going in to the gainfield parameter (e.g. what is the difference between gainfield=['0','0','0'] and gainfield=['0','3,4','3,4'] - which calibrations are being applied where).
  • A: For the first table, apply the calibration from field 0. For the 2nd and 3rd tables, apply the calibration from fields 3 AND 4. The order of the gaintable and gainfield parameters have to be the same, and the number entries in gaintable have to be equal to the number of entries in gainfield.
for field in field_names_north:
       applycal(vis=asdm+".ms", spw='1',field=field, gainfield=["",field,field],
           interp='nearest', gaintable=[asdm+".antpos",asdm+".tsys.cal.fdm",asdm+'.wvr.cal'],
           flagbackup=F)

Q78: How is it that setting field=field and gainfield=["",field,field] makes it possible that the Tsys and WVR calibrations are only applied to the source which they are measured? I found this confusing.
  • A: See above.

Q79: I don't understand how the parameter "gainfield" works in applycal. From the examples, it seems kinda random, and the reference manual is entirely unclear.
  • A:

Q80: 'In the call to applycal, we will specify interpolation="nearest". This is important for the WVR corrections, ...' How important is important? Does this mean one should always use interpolation="nearest" when applying WVR corrections to ALMA data. If so, it would be very useful to say that.
  • A: It's probably better to use nearest.

Q81: Oops I've run applycal but I am all thumbs today and got an input wrong. How do I undo (unapply) that? Is it as simple as deleting the gain table?
  • A: Data can be flagged during applycal, and there is a parameter in applycal to create a backup flag table (this can be slow). It can be faster to backup flags before applycal and not use the flag backup option in applycal. After fixing flags, you run applycal again (correctly) which then overwrites your incorrect calibration. However, your data weights have been modified incorrectly, and this can't be recovered.

Q82: At applycal for target, should we only use the primary calibrator if we want to really compare the phase transfer to the secondary phase calibrator?
  • A:

Q83: in applycal, interp=nearest is in time, distance on the sky, or what? better to take a running mean?
  • A: Just time. In the future maybe interpolation in elevation will be possible.

Q84: When the secondary calibrator was corrected using its own corrections looked better, then the calibration from the primary calibrator for the target will show the same issues, right? In this case we will self-calibrate, so should be fine, but what if the target source cannot be used for self-cal?
  • A: See above.

ADDITIONAL

Q85: At Early Science, it seems like it will be standard to observe the science target in one correlator mode and the phase calibrator (etc) in another, making sure to observe a strong source with both modes to check for any phase offset. Could you outline with us extra steps in the data analysis this setup will/could introduce?
  • A:

Q86: Could you tell us any tips you have for calibrating data in which the line of interest is in a region of very bad atmospheric transmission. How do we optimise the calibration in these instances?
  • A:

Q87: If you have an unexpectedly faint gain calibrator, or narrow spws, can you use combine='spw' to improve S/N on the time variation (after individual bandpass calibration) and then apply to the individual spws? Can you scale the solution by frequency (to account for the scaling of phase)?
  • A:

Topic attachments
I Attachment Action Size Date Who Comment
ALMA0.95bwsmear.pngpng ALMA0.95bwsmear.png manage 92 K 2011-09-07 - 18:47 CrystalBrogan  
ALMA_spindex_smear.pngpng ALMA_spindex_smear.png manage 78 K 2011-09-07 - 18:47 CrystalBrogan  
ALMA_timesmear.pngpng ALMA_timesmear.png manage 68 K 2011-09-07 - 18:46 CrystalBrogan  
X5d8.ms.weather.pngpng X5d8.ms.weather.png manage 84 K 2011-09-08 - 09:46 ToddHunter  
shadow.pngpng shadow.png manage 38 K 2011-09-08 - 12:44 ToddHunter EVLA shadowing
shadowalma.pngpng shadowalma.png manage 37 K 2011-09-08 - 12:44 ToddHunter  
uid___A002_X1c99f2_X1ce.ms.weather.pngpng uid___A002_X1c99f2_X1ce.ms.weather.png manage 96 K 2011-09-07 - 18:44 CrystalBrogan  
This topic: ALMA > WebHome > AlmaSci > AlmaTiger > DataReducWorkshop > ALMACalQuestions
Topic revision: 2011-09-09, RobReid
This site is powered by FoswikiCopyright © by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding NRAO Public Wiki? Send feedback