ALCHEMI Data Reduction Group: Flux Scaling and Normalization

Status Updates

• NOTE: Questions are in red.
• See the ALCHEMI DR Meeting 2018-06-26 and later DR meeting minutes for detailed reports on flux scaling for the ACA data.
  • Status and Reports
    • Scaling-factors from band-edge overlaps and those from statcont should be compared for consistency.
    • [Seb]: Flux alignment using StatCont implemented for ACA data. Correction factors comparable to the edge-channel alignment from Kazushi. Small (<5% for B4, up to 20-30% in B7)
      • I am still fighting with spws/tunings at the extreme B7 band edge.
      • Data has been spatially smoothed to 15, 10, and 8 arcsec for B4,6,7.
      • We can average spw in the same sidebands, if necessary. From LSB to USB we do see some differences sometimes (error in the spectral index of the flux calibrator ???)
      • There are clear systematics from some tunings.
      • Line crowding seems not a big issue for StatCont.
      • Next step: astrometry check.
  • Issues
    • Do we want to apply the scales to visibilities and then image everything again? Or do we want to rescale all the image cubes and image products?
    • Are we going to determine relative-SG scales separately for 7m and 12m, and apply them separately to 7m and 12m data before making 12m+7m images?
    • [Kazushi]: Need to decide if we want to apply the final uniform amplitude scaling to the MS or to images.
      • [JGM]: Seems cleaner to apply final scaling to MSs.
      • [Kazushi and Seb]: When you finish your amplitude scaling tests come up with a recommendation for amplitude scaling process.
• 2018-09-14:
  • [Seb]: Below is a comparison between flux scaling coefficients from:
    • Overlapping channel method: This method involves solving the matrix of the rij=a_i/a_j where rij is the measured amplitude ratio between tunings i and j, and a_i the scaling factors with the constraints mean(a_i)=1). See Kazushi's input in the Wiki for more details. Coefficients are those from Kazushi's July 31 report. Maybe updated values might change a few.
    • Continuum level alignment: This method runs statcont on the cubes to separate line and continuum emission. For a given band, continuum fluxes in a given region for all spws are plotted and fitted with a power law. The scaling coefficients are calculated to "force" the level to match that of the fitted power law. Assumptions are that the continuum emission follows a power law and that the continuum level derived by statcont is correct. Coefficients are spw-based, but could be averaged per tuning.
    • Coefficients have been obtained on data smoothed to 15, 10, and 8 arcsec, for B4,6,7 respectively, for both methods.
    • From this plot, I find the agreement more than satisfying and it is actually very good to derive the corrections from two independent methods !!! Strong evidence that flux corrections are needed/robust.
    • As a consistency check, that would allow us to check that the continuum level is "correctly" derived by statcont (ie that the spread of coefficients for a given tuning is not due to line contamination in the statcont continuum level), should compare to Kazushi's independent analysis.
  • [Kazushi]: Below are my updated scaling factors in a Python dictionary. The raw amplitudes should be divided by these numbers. Clean thresholds were deeper and more uniformly controlled this time, but these factors don’t deviate much from my old ones. I haven’t derived 12m parameters and they are set to unity.

{'B4a': {'7m': 0.995, '12m': 1},
 'B4b': {'7m': 1.002, '12m': 1},
 'B4c': {'7m': 1.014, '12m': 1},
 'B4d': {'7m': 0.982, '12m': 1},
 'B4e': {'7m': 0.993, '12m': 1},
 'B4f': {'7m': 1.021, '12m': 1},
 'B4g': {'7m': 0.993, '12m': 1},
 'B6a': {'7m': 0.958, '12m': 1},
 'B6b': {'7m': 1.011, '12m': 1},
 'B6c': {'7m': 1.014, '12m': 1},
 'B6d': {'7m': 1.011, '12m': 1},
 'B6e': {'7m': 1.006, '12m': 1},
 'B6f': {'7m': 0.906, '12m': 1},
 'B6g': {'7m': 1.059, '12m': 1},
 'B6h': {'7m': 0.964, '12m': 1},
 'B6i': {'7m': 1.000, '12m': 1},
 'B6j': {'7m': 1.072, '12m': 1},
 'B7a': {'7m': 1.009, '12m': 1},
 'B7b': {'7m': 1.002, '12m': 1},
 'B7c': {'7m': 0.976, '12m': 1},
 'B7d': {'7m': 1.242, '12m': 1},
 'B7e': {'7m': 0.983, '12m': 1},
 'B7f': {'7m': 0.986, '12m': 1},
 'B7g': {'7m': 0.979, '12m': 1},
 'B7h': {'7m': 0.824, '12m': 1},
 'B7i': {'7m': 1.026, '12m': 1},
 'B7j': {'7m': 0.969, '12m': 1},
 'B7k': {'7m': 0.973, '12m': 1},
 'B7l': {'7m': 1.031, '12m': 1},
 'B7m': {'7m': 1.035, '12m': 1},
 'B7n': {'7m': 1.012, '12m': 1},
 'B7o': {'7m': 0.925, '12m': 1},
 'B7p': {'7m': 1.029, '12m': 1}}

• • • It is in principle possible to derive SPW-based scaling factors because all SPWs have overlaps. Before trying that, I’d like to be convinced that such scaling is really necessary. I’m mot saying SPW-based variation does not exist. It may. (It’s young ALMA after all.) But I’m cautious because it is rather awkward to introduce more than 100 adjustable parameters to our calibration. The need for SG-based scaling was evident by looking at spectra, so I didn’t hesitate to introduce 30+ SG-based parameters in our calibration. SPW-based scaling is more subtle but needs more parameters. As I explained before, one can check the need for SPW-based scaling with the help of SPW-based flux densities measured by the pipeline for our gain calibrators. Fit them with a power-law (with two free parameters, spectral index and flux at the reference freq.), and look at the residuals. If the residuals are more than a few % of the flux densities in the SPWs, then, indeed, we'll need SPW-based flux rescaling. So, Seb, why don’t you check this for a few tunings where you found the largest variation of scaling factors among SPWs in the same SGs? Although a little complicated by the presence of multiple EBs, you can compare the pattern of your scaling-factor variation among SPWs and that in the gain calibrator flux densities. If they agree with each other then that would be a strong motivation to introduce ~100 new parameters (or revisit the pipeline calibration for its improvement.) I’ll then bump up my priority to derive SPW-based scaling factors.
  • [Sergio]: One of the things i like the the least about the continuum approach is the assumption of the power law. One of the potentially nice results in such line survey may be the shape of the continuum, right?
  • [Seb]: About the continuum emission fitted by a power law, yes we have to assume a model. Note that I separate each band (B4,6,7) and each has indeed a very different spectral index:
    • B4: 0.13 ± 0.02
    • B6: 2.11 ± 0.15
    • B7: 3.76 ± 0.17
    • One other problem is that the fit accommodate for the average, hence we could get a bias to slightly lower flux, or we could iterate once, removing the low-flux outliers (?). But we are talking about peanut at this level, as you say.
  • [Sergio]: I see, a power law per band makes more sense.
  • [Jeff]: I am sorry for being behind in this discussion, but I am having trouble seeing how a power law index per band makes sense. I believe that the continuum emission is from a combination of dust and free-free across all 5 bands (including B5 here), which should be a combination of a rather flat (~0.1) and rather steep (~3) power law. Maybe that is what Seb has derived?
  • [Seb]: Kazushi, I can certainly make a check of the gain calibrator data. However, if there is an error in the spectral index of the flux calibrator, it will propagate and add up to the spectral index of the gain cal (which I do not know). So what I will find for the gain cal is just if it indeed shows a beautiful power law (but in this case I still cannot disentangle if it is correct, could be completely wrong), or if it is crap (ie all spw not well aligned, within the limit of their small delta_freq/freq). For the moment, we might just play with average of the scaling factors.
    • For the channel overlap method, there is also a bias toward lower flux level, since there is no way, if I understand well, to select out "tunings with low level", right?
    • In the first publication "presentation of the ACA data", I guess it would be good to present those two flux scaling methods, with a small paragraph for each. What do we take at the end? the average of both methods?
    • I can do the check of the power law for the gain calibrator, after forcing the scaling factors that I derived for each tuning/ each spw. Then see if the gain calibrator is nicely showing a power law or a chaotic scatter of flux points. Maybe that's what you meant Kazushi?
    • Kazushi: Do you assign weights (rms of the residuals?) to the different tunings (or per spws), and use them for deriving the flux scaling factors from overlapping channels? I haven't implemented this in my statcont alignment method yet, maybe I should.
• 2018-09-20:
  • [Seb]: Here the first results for flux scaling factors of 12m-array data for B4 and B6 , image smoothed to 1.5 arcsec. Scaling factors seem reasonable, except there is a problem for the spectral baseline for the spw containing CO 2-1. Also, failing two spws of B6c, which didn't work out in the smoothing (original beam too large?) ... Have to investigate.

# B4 
Fit for spectral index of the continuum power law:   0.092164605 ± 0.041482733
MULTIPLICATIVE FACTORS for B4 (data so far at hand)
----------------
TUNING  FREQ FACTOR
1 150900 0.9994
1 138700 0.9889
1 149000 1.0038
1 136900 0.9852
2 142400 1.0193
2 154500 1.0260
2 152700 1.0336
2 140600 1.0253
3 156300 0.9695
3 158200 0.9683
3 146000 0.9774
3 144200 0.9903
4 149700 0.9992
4 161800 1.0160
4 160000 0.9977
4 147900 1.0063

and for B6 main array tunings:
# B6
Fit for 1:   1.694308385 ± 0.092073501
MULTIPLICATIVE FACTORS
----------------
TUNING  FREQ FACTOR
1 227000 0.9989
1 228600 0.9692
1 212000 0.9718
1 213600 0.9813
2 217000 0.9712
2 232000 0.9889
2 230300 1.3525
2 215300 0.9526
3 233700 0.9464
3 235300 0.9545
4 238700 1.0243
4 237000 0.9979
4 223700 0.9688
4 222000 0.9712
5 242100 1.0306
5 227100 1.0087
5 240400 1.0190
5 225400 1.0167
6 258900 1.0088
6 260600 1.0078
6 245600 1.0141
6 243900 1.0293
7 264000 0.9756
7 262300 0.9832
7 247300 0.9834
7 249000 0.9789
8 267300 0.9893
8 265700 0.9766
8 252300 0.9906
8 250700 1.0214
9 270700 1.0043
9 269000 0.9856
9 254000 1.0068
9 255700 0.9635
10 274000 1.0051
10 272400 1.0044
10 259000 1.0320
10 257400 1.0142