Testing of September 2013 heuristics

*Note: If the link to the analyzemscal result is broken, this project has not been run through the new pipeline yet.* Please list your uid and amc command or terminal file at AnalyzeMSCalFailures

# Project ARC SCOPS Obs. Date Band Type (spw BW MHz) #Antennas
(low gain)
Flux cal
(antennas)
OUS ASDM=EID (linked to analyzemscal results) Status,
version
Pipeline reducer Manual reducer, refant Notes Suspect pointing Flagging
1
2011.0.00010
NA
129
24-Jan-2012
6
FDM (1875)
17 (0)
Titan (unresolved), 107 arcsec from Saturn, pipeline/manual=0.998
uid://A002/X327408/X217
pipeline success; runtime = 2:43:24; runtime w/o imaging = 2:10:03; AMC v1.1343
Anand
Ed, DV05
 
none
 
2
2011.0.00142
NA
167
16-Jun-2012
6
TDM
22
Neptune (11/22) ra,dec=(0,0), pipeline/manual=0.938
uid://A002/X327408/X313
done, v1.268 pipeline run time=1h 19m (w/out imaging=1h 7m)
Nathan
Brenda, DV04
 
none
 
3
2011.0.00150
NA
119
16-Nov-2012
7
FDM (1875)
28 (3)
Callisto (16/28), 596 arcsec from Jupiter, pipeline/manual=0.857, intent=AMPLI
uid://A002/X327408/X3d
pipeline success; runtime = 2:33:30; w/o imaging = 1:58:36; AMC v1.403
Nathan
Todd, DV19
wvrflags DV13, not warranted
none
 
4
2011.0.00175
NA
166
09-May-2012
7
TDM
16 (2)
Titan (13/16), 195 arcsec from Saturn, pipeline/manual=0.994
uid://A002/X327408/X276
done, v1.241, pipeline run time=1h 17m (without imaging=1h 0m)
Dongchan
Schnee/Wootten,DV12
 
none
WVR: DV15
5
2011.0.00243
NA
243
16-Jun-2012
6
TDM
21
Uranus (8/21) radec=(0,0), pipeline/manual=0.995
uid://A002/X3ff4ae/X38
done, v1.262, pipeline run time=1h 01m (without imaging=0h 51m)
Dongchan
Mark Lacy, DV04
 
none
 
6
2011.0.00348
NA
238
18-Dec-2012
3
FDM (1875)
28 (1)
Ceres (unresolved)
uid://A002/X391d0b/X280
pipeline success; runtime = 3:52:43; w/o imaging = 3:05:02; AMC v1.404
Nathan
Crystal, DV04
 
DV01, DV23, DV24
 
7
2011.0.00348
NA
238
16-Dec-2012
3
FDM (1875)
25 (0)
Ceres (unresolved), pipeline/manual=1.114
uid://A002/X391d0b/X280

pipeline crashed with "KeyError: 'fluxd'" in hif_GcorFluxscale.

pipeline finished with one error (No Spw ID(s) matched specifications).




Nathan
Crystal, DV04
spw15 Tsys derivative
DV23, DV24
 
8
2011.0.00348
NA
238
05-May-2012
3
FDM (1875)
16 (0)
J085448+200647, pipeline=manual=0.115
uid://A002/X391d0b/X280
pipeline success; amc success
Anand
Crystal, DV12
 
PM02
 
9
2011.0.00397
NA
49
28-Aug-2012
7
TDM
27 (2)
Titan (unresolved), 164 arcsec from Saturn, pipeline/manual=1.000
uid://A001/X74/X231

done, v1.244

rerun OK 9/19/2013, no amc

runtime = 2:24:02, w/o imaging = 1:51:50

pipeline success

amc error (ValueError: 'J1256-057' is not in list) at fluxscale command

Paul
Carol, DV04
 
none
 
10
2011.0.00476
NA
203
09-May-2012
3
FDM (1875)
16 (0)
Titan, 194 arcsec from Saturn,
pipeline/manual=1.046
uid://A002/X391d0b/X183

uid___A002_X3fdb4f_X78c, Todd's version with Tsys spike flagging: uid___A002_X3fdb4f_X78c

weblog

pipeline done, v1.259, pipeline run time=2h 10m (without imaging=1h 18m)

pipeline success

amc failed with ValueError: 'J1256-057,' is not in list.

Dongchan
Adam, DV12
file permission error on manual reduction directory.
none
WVR: DV15
11
2011.0.00476
NA
203
28-Jul-2012
3
FDM (1875)
25 (1)
Titan, 152.6" from Saturn,
pipeline/manual=1.042
uid://A002/X391d0b/X183
pipeline done, v1.251, pipeline run time=3h 37m (without imaging=1h 52m)
Dongchan
Adam, DV04
file permission error on manual reduction directory.
 
 
12
2011.0.00526
NA
5
16-Dec-2012
7
FDM (1875)
21
Titan (unresolved)
115.6" from Saturn,
pipeline/manual=1.047
uid://A001/X74/X186
Failed Sep release, r27438; failed with with r27480... (brackets in field names); confirmed to fail with r27480... (brackets in field names)
Anand
Crystal, DV04
 
DV01, DV23, DV24
 
13
2011.0.00539
NA
114
13-Jul-2012
7
TDM
18
Callisto (10/18),
450 arcsec from Jupiter,
pipeline/manual=0.845
uid://A002/X391d0b/X16d

done, v1.267

rerun OK, 9/19/2013, no amc

runtime = 1:08:26

w/o imaging = 0:59:36

Paul
John Hibbard, DV04
 
none
 
14
2011.0.00652
NA
189
09-May-2012
3
2 TDM
16 (0)
Mars (0/16)
ra,dec=(0,0)
pipeline/manual=1.025
uid://A002/X3216af/Xa
r27438 success; amc success v1.1522
Anand
Jim Braatz, DV12
 
none
 
15
2011.0.00747
NA
83
28-Aug-2012
9
TDM
27
Neptune
ra,dec=(0,0)
pipeline/manual=38.85
uid://A001/X74/X130

uid___A002_X4b58a4_X31,

weblog

Todd's run with tsysflagchans metric=median: uid___A002_X4b58a4_X31

done, v1.268 pipeline run time=2h 6m (w/out imaging=1h 41m)
Nathan
Todd, DV04
 
none
 
16
2011.0.00958
NA
51
14-Aug-2012
7
TDM
22 (1)
Neptune
ra,dec=(0,0)
pipeline/manual=0.943
uid://A002/X314ea1/X231
pipeline success w/ r27480; amc success
Anand
Todd, DV04
 
none
 
17
2011.0.00958
NA
51
13-Aug-2012
7
TDM
25 (2)
Neptune
radec=(0,0)
uid://A002/X314ea1/X233
success runtime w/imaging:1hr15min
Sarah
Todd, DV04
 
 
 
18
2011.0.00978 = 00399
NA
246
20-May-2012
7
FDM (58)
16 (0)
Titan,
71.8 arcsec from Saturn
pipeline/manual=1.449
uid://A002/X327408/X34f
done. v1.257 runtime = 2:31:40 w/o imaging = 0:32:28
Paul
Adam, DV12
 
none
 
19
2011.0.00978 = 00399
NA
246
20-Nov-2012
7
FDM (58)
25 (2)
Ceres
pipeline/manual=1.119
uid://A002/X327408/X34f

done,v1.348

(Runtime:

Total=4hr35m

No Im=3h37m)

Sarah
Adam, DV04
 
none
 
20
2011.0.00511
NA
112
27-Jul-2012
7
FDM (937)
23
Callisto,
315 arcsec from Jupiter
pipeline/manual=0.867
uid://A002/X391d0b/X101
done,no AMC 1hr 45m w/I 1hr 23m w/o I
Sarah
Nuria, DV04
 
DV02, PM02
 
21
2011.0.00150
NA
119
16-Nov-2012
7
FDM (1875)
28 (2)
Callisto,
599 arcsec from Jupiter
pipeline/manual=0.856
uid://A002/X327408/X37
pipeline success; runtime = 3:39:13, w/o imaging = 2:38:41; AMC v1.408
Nathan
Nuria, DV19
Tsys bandpass shape issue (CSV-1807), manual reduction used the phase cal as bandpass. Pipeline results flagging affected tsys: uid___A002_X54d35d_X664
none
 
22
2011.0.00150
NA
119
17-Nov-2012
6
FDM (1875)
27
Callisto,
477 arcsec from Jupiter, pipeline/manual=0.984
uid://A002/X327408/X3b
done, v1.297
Todd
Scott, DV19
 
none
 
23
2011.0.00511
NA
112
18-Jun-2012
7
FDM (937)
19
Callisto,
314 arcsec from Jupiter
uid://A002/X391d0b/X101

done, no amc

(Runtime:

Total=2h 3m

No Im=1h 40m)

Sarah
Nuria (Dongchan re-reduced), DV04
archive broken (execblock_status=broken)
 
 
24
2011.0.00206.S
NA
198
27-Mar-2012
6
TDM
15 (4)
Titan,
77 arcsec from Saturn
uid://A002/X391d0b/X1e
done, no amc 2hr23m w/I 2hr8m w/oI
Sarah
Amy
 
 
 
25
2011.0.00319.S
EU
266
17-Jun-2012
6 (mosaic)
FDM (1875)
19
Mars,
ra,dec=0,0
uid://A001/X74/X23c
sharps2_limit=0.1: uid___A002_X435875_X204
sharps2_limit=0.3: uid___A002_X435875_X204
done, v1.297
Todd
 
DV15 has birdies
 
 
26
9999.4.00013.CSV
JAO
118 flux track
27-Jan-2012
3
TDM
17
Mars,
ra,dec=0,0
uid://A002/X327408/X1c0
done, v1.317
Todd
 
 
 
 
27
2011.0.00053.CSV
JAO
130 flux track
06-Jun-2012
7
TDM
18
Neptune,
ra,dec=0,0
uid://A002/X3ff4ae/X7c

pipeline failed on gaincal intint BANDPASS: ParseException: Expected W:(0123...) (at char 0), (line:1, col:1);

pipeline failed on importasdm and crashed on wvrgcalflag (parse error)

Todd
 
J1924 has wrong coordinates; tried to use J1700-261 as next brightest
 
 
28
2012.1.00912.S
NA
336
24-Jan-2013
3
TDM
32
 
uid://A002/X5a9a13/X528
done, v1.343
Todd
 
tsysspwmap fails with tsysChanTol=1
 
WVR: DV23

If any antennas were flagged by hif_wvrgcalflag, please include them in the Notes column. After analyzemscal runs successfully, please include the version number in the Status column.

# Project ARC SCOPS Obs. Date Band Type (narrowest spw) #Antennas
(low gain)
Flux cal OUSSorted ascending ASDM=EID (linked to analyzsemscal results) Status Pipeline reducer Manual reducer Notes Suspect pointing Flagging
19
phase V test dataset
NA
 
 
3
3TDM,1FDM (117)
34
Titan
 
 
 
 
 
 
 
6
2011.0.00054.CSV
JAO
 
16-apr-2013
3
TDM
24
Titan, 156 arcsec from Saturn, intent=AMPLI,AMPLI
uid://A002/X576a5e/X1e
 
 
Nuria
Calsurvey run manual results
 
 
7
2011.0.00054.CSV
JAO
 
16-apr-2013
7
TDM
24
Titan, 156 arcsec from Saturn, intent=AMPLI,AMPLI
uid://A002/X576a5e/X1e
 
 
Nuria, DA51
Calsurvey run manual results. Previous pipeline results (v 1.197) using DV01 as reference.
 
 
23
2012.1.00423.S
 
 
27-Jan-2013
3
4FDM (1875)
30
J0132-169
uid://A002/X5a9a13/X114
v1.297
Todd
 
platforming
 
 
2
2012.1.00989.S
NA
364
25-Jan-2013
6
4TDM
29
J0334-401, intent=AMPLI
uid://A002/X5a9a13/X141
success, v1.310
Hunter
n/a
 
 
 
5
2012.1.00762.S
EA
348
28-Jan-2013
3 (ACA)
FDM (1000)
9 (1)
J1107-448, intent=AMPLI
uid://A002/X5a9a13/X68b
success if only one asdm is used in PPR, JIRA:CAS-5939, v1.412
Sarah
n/a
 
 
 
3
2012.1.00096.CSV
JAO
31-Jan-2013
3
1TDM,3FDM (234)
30 (1)
J1107-448, intent=AMPLI
uid://A002/X5a9a13/X6ca

success, v1.297

pipeline success with one error (Maximum uv distance is zero),

amc finished with 2 SEVERE warnings (wvr_before does not exist, and No match found for state expression: FLUX)

Hunter
Hunter
2/3 of antennas are flagged for Tsys on phase cal. Good test of tmf1=0.666
 
 
1
2012.1.00185.S
NA
356
15-Mar-2013
3
2TDM,2FDM (1875)
34 (2)
Titan, 153 arcsec from Saturn, intent=AMPLI
uid://A002/X5a9a13/X726
success, v1.297
Hunter
n/a
 
 
 
17
2012.1.00139.S
EA
 
01-Jun-2013
7
4TDM
30
3, Titan plus 2 quasars
uid://A002/X5a9a13/Xf1
success, v1.297
Todd
 
Titan flagged on 19/30 antennas for Tsys, 96% overall
 
 
18
2012.1.00139.S
EA
 
01-Jun-2013
7
4TDM
30
3, Titan plus 2 quasars
uid://A002/X5a9a13/Xf5
success, v1.297
Todd
 
Titan flagged on 19/30 antenans for Tsys
 
 
20
2012.1.00635.S
NA
 
18-Jun-2013
6,7
1FDM,3TDM in band 6 (1875)
1FDM,3TDM in band 7 (1875)
27
J1733-130
uid://A002/X5d7935/X2fb
uid___A002_X67f61e_X687
pipeline finished incompletely at hif_wvrgcalflag.
Todd
 
pipeline fails at wvrgcalflag with gaincal::Calibrater::setapply(type, applypar) Cube::resize() - attempt to form n on-Cube Check inputs and try again.
 
 
9
0000.0.00099.CSV
JAO
 
16-Apr-2013
3
3FDM,1TDM (117)
25
Titan,
163 arcsec from Saturn, intent=AMPLI,FLUX
uid://A002/X5d9e5c/X1ed
Success, v1.297
Scott, then Todd
Scott
250ps delay issues! Needs tsysChanTol=1
 
 
8
0000.0.00099.CSV
JAO
 
22-Apr-2013
3 (ACA)
FDM (125)
9
Titan,
91.5 arcsec from Saturn
pipeline/manual=1.449, intent=AMPLI,FLUX
uid://A002/X5d9e5c/X1f4
pipeline w/o I:1hr19m with I:1hr29,v1.413
Sarah
Scott, CM02
7m array
none
 
4
2012.1.00118.CSV
JAO
13-Apr-2013
3
1TDM,3FDM (234)
25 (2)
Titan, 65 arcsec from Saturn, intent=AMPLI,FLUX
uid://A002/X5d9e5c/X3cb

success, v1.297

pipeline success, amc success v1.412

Hunter
Hunter
See CSV-2757, lots of examples of low amplitude at beginning of scans
 
 
11
0000.0.00083.CSV
JAO
 
11-May-2013
3 (ACA)
FDM (1992)
8 (mosaic)
none!
uid://A002/X5d9e5c/X64

Done, v1.308. But need to re-run with intentgroups=['*ATMOSPHERE*']. Final hif_tsysflag bombs with: IndexError: tuple index out of range;

pipeline finished with one error (AttributeError: 'Qa2Result' object has no attribute 'vis')

Scott
 
Tsys spike, good one to test new heuristics on
 
 
10
2011.0.00054.CSV
JAO
 
16-apr-2013
7 (ACA)
TDM
10
Titan,
165 arcsec from Saturn, intent=AMPLI,AMPLI
uid://A002/X5d9e5c/Xa1
 
 
Nuria
Calsurvey run manual results
 
 
14
0000.0.00089.CSV
JAO
 
26-Mar-2013
3
3FDM,1TDM (117)
23
3c279
uid://A002/X5d9e5c/Xdd

success, v1.297

pipeline success, amc failed v1.412 (ValueError: invalid literal for int() with base 10: 'UNSPECIFIED')

Todd
 
spike in Tsys
 
 
13
2012.1.00229.S
NA
 
03-June-2013
7 (pointing with 6)
1FDM,3TDM (468)
35 (13)
J2232+117
uid://A002/X628157/Xd1
success, v1.247
Scott
 
WVR on DV23 (antenna 31) has high scatter
 
 
21
2012.1.00127.CSV
JAO
 
05-Jun-2013
3
TDM, some with 32 and 64 channels
42
none
uid://A002/X638463/X2e
 
 
 
 
 
 
 
22
phase V test dataset
NA
 
01-Jun-2013
7
4FDM (937)
30
J2232+117
uid://A002/X639a2a/X2a
done, v1.297
Tony
 
platforming on DV16
 
 
12
2012.1.00124.CSV
JAO
 
17-May-2013
6
FDM (117)
34
Titan
uid://A002/X639a2a/X66
 
Sarah
 
 
 
 
15
0000.0.00125.CSV
DDT
 
08-Jun-2013
3
4TDM
32
Neptune
uid://A002/X6610eb/X21
uid___A002_X668bd2_X6f1
intentgroups=*ATMOSPHERE*:

pipeline success

amc success v1.412

Todd
Al, DV05
not all high Tsys values flagged, Neptune has 2 field IDs
 
 
16
0000.0.00125.CSV
DDT
 
08-Jun-2013
3
4TDM
32
Neptune
uid://A002/X6610eb/X21

success, v1.297

pipeline success, amc success v1.412

Todd
Al, DV05
DA44 spw13 Tsys has ripples on one edge, but not flagged
 
 

-- ToddHunter - 2013-07-30
Topic revision: r81 - 2013-12-31, SarahWood
This site is powered by FoswikiCopyright © by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding NRAO Public Wiki? Send feedback