Meeting Information

  • 1600 UT (1000 MDT)
  • AOC 317; IP: 146.88.16.8; phone: +1 575-835-7370

Agenda/Minutes

Key: BOLD: Minutes/notes from last week; BLUE: Notes from this week.

News

  • Ongoing correlator work during Brent's visit; took largest data set to date (382 GB for RSRO project).
  • Staff Travel:
    • Krauss: 23 Jul - 27 Jul
    • McMullin: 26 Jul - 30 Jul
    • van Moorsel: 26 Jul - 30 Jul
  • Visitors:
    • Brent Carlson: 5 July- 16 July
  • Key Dates:
    • PASEO Meeting: 15-16 July; Note, meeting is open for those interested; agenda is posted in link.

Critical Commissioning review/reports

  • Overall status: Several updates to CBE software; believed to have resolved CBE node failure problem (Robnett/Rupen). Note that the CBE node failure has not occurred since the fix of 03 Jul. Since last week, many 'fixes' to the correlator hardware/firmware from Brent. Overall, ready for low frequency OSRO/RSRO, modest high frequency OSRO; high frequency RSRO is blocked by ongoing investigations into lost/muddled data.
  • Highest priority development areas; readjusted based on resolution of several outstanding OSRO blockers.:
    • 1) Commissioning tests/system/correlator stability (particularly focused on using Brent's time). Focus on connectivity issues (most salient result will be ability to route signals to arbitrary BlB - would enable the OST!); also synchronization issues. Report from Brent/Michael/Ken.
    • 2) RSRO science observing. Really driven by weather issues; currently, many outstanding issues for high frequency RSRO, same issues seen in OSRO but exacerbated for RSRO. We need to get in RSRO SBs now!
    • 3) 3-bit sampler testing; Full test plan needs to be completed by 20 August to maintain January timeline for production equipment. Must outline further tests soon to meet this schedule.
    • 4) OSRO science observing issues/observations, includes switched power implementation. Outstanding problems: Missing scans; data mislabeling; integer zeros; lost fringes/incoherence. reference pointing recovery.
    • 5) Demo science observing.

How is this handled illustration.
  • At this meeting we review the priorities as above, including requests for test time for groups. SBs should be submitted (don't wait to do this).
  • There is a daily meeting (generally between 1600-1800) with Joe, Michael, Ken, Joan, (Rick, Vivek), reviewing the state of the system, tests and science plans for the night. A schedule is developed and sent to the operator.

System Verification/Hardware Commissioning:

Testing and characterization of the array based on the design goals.

Targets

Area Target Priority JIRA Deliver Status
High Frequency Reference pointing quantitative analysis 0 ECSV-16/ECDATA-91 2010Q3 Issues uncovered - see report below
Calibration Phase transfer 0 ECDATA-271 2010Q3 Needs better weather for real conclusions.
Calibration Switched power gain correction 0 - 2010Q3 pending SDM implementation for further work
Calibration RFI Sweep for all bands 2 - 2010Q3 Pending wide band widths
Calibration Develop set of frequency accuracy tests (X band is complete) 2 - 2010Q4 Perley:
Calibration Delay clunking 2 - 2010Q4 Rupen/Pokorny:

Reports

  • Antenna: Perley
    • Monitoring update (Morris)
      • See plots attached. See different behavior between CX and CK (drifting seen on CX); X is the failsafe position; indications of scans following this not changing as expected (switch issue). Agreed to a set of CX/CK tests each Wednesday following maintenance for monitoring.
1. In the K scans, power varies slowly across all IFs.  The magnitude and rate of these is different for different antennas.  I expect that these are real variations in received power, at the receiver.
2. In the C-band scans, there is a spread in power across the IFs, as much as 3-5 dB.  This is due primarily to the lookup table values for the BD T304 attenuator settings.
Rob Long is turning my list into action items for maintenance day.
    • syscoll update (Sowinski) Still not well understood; TelCal not doing anything with Q band data (no antsol, no ptg sol; BB: Intents set? KS: Yes).
  • Receiver: Perley
    • EA12 Q band Rx installed and working (Sowinski).
    • Ku rxs coming (2) next week. Pushed along - 21st now. Another S band due at the same time. First wide band X by the end of the month. Kas coming for 11 and 26 this week.
    • RP noted that we are identifying which Rxs have the quad hybrid; compare performance.
  • Array: Perley
      • Wideband X-band Cyg A (Perley): 4 sub-bands identically zero; beyond those data were very good (gains/bandpass); illustrates the promise when we're not changing bands/configurations.
      • Reference pointing; OSRO/RSRO reference pointing (Perez/Sowinski): good news is that the OSRO and RSRO mode reference pointing agree; bad news is that for RSRO pointing, there is a larger fraction of lost data/solutions; overall problem with the solutions being smaller, approximately a 20% effect due to fitting the amplitude linearly rather than matching the beam shape (BB: Noted we shouldn't be using analytic functions but true beam knowledge); good reason to believe this had been working based on comparison between pointing runs; under-correcting by some amount; being fixed.
      • Phase transfer discussion (Dhawan) - do at end of meeting.. Ran out of will for many due to the time. Vivek presented plots and table summaries from ECATA-271; bottom line was that the results/limitations were consistent with the atmosphere but due to the weather being less than ideal, this is not conclusive for the suggested issues at higher frequencies. Better data was taken last night that will be reviewed.
  • Correlator: Rupen
    • Status: Brent in town pushing on connectivity (ensure that signals are propagating properly from the antennas to the correlator (StB->XBB->etc). Brent/Kerry have been hitting roughly a rack a day; seeing issues with bad cables (30 miles of cable in the system) and bent pins; also discovered issues with the FPGAs; being fixed. Kerry is getting good training on the work so we're taking advantage of the knowledge-transfer opportunity. Synchronization losses are also being looked at (during configuration changes); improvements here but still seeing some errors - contacting Dave Forte on this. Some longer term issues being addressed (using the synthesizer to probe wobbles and frequency response).Hoping these areas will be complete by the end of the week; negotiating with Brent on his next visit.
    • Succinct list of issues: missing data; Michael reviewed all data in July and sees several different scenarios leading to missing scans; there are 4 scans that indicate mislabeled scans; both point to the CBE (on hold during Brent's visit); still some cases of incoherence particularly following reference pointing. Discussion on whether we should work to adapt programs to single configuration based on the quality of Rick's results and the above problems. RP noted that pointing up to K band had been okay; expect a few tenths of an arcminute in pointing errors when running open loop. Bryan noted that the test version of the OPT should be tried out for pointing (new scripts).

Observing Band Checkout

Encompasses principally core operation at those frequencies followed by detailed calibration strategies, flagging.

No updates; noted that plans should be getting cobbled together as switched power is looming. RP asked about the status. Rich (MCAF) returns this week and will work on getting the information to the SDM. Need Sonja to do some work to write information from the StB for MCAF to see (on vacation currently). So data in SDM could be available by early August (if Sonja remains out of contact for the whole period though something could likely be tested in another week).RP asked when it would be available for post-processing; longer period needing discussion; Eric predicted a year for CASA since there have been no changes to the UVFITS output; not expecting it to be this bad.

Targets

Reports

  • Low Frequency (L/S/C): Momjian
    • Status: Operating; users should include one of the canonical flux calibrators in SB.
    • Upcoming:
      • Need switched power
        • Develop set of observations to check Tcal/Tsys/SEFD across bands
    • Links: ECSV-38

  • Mid Frequency (X/Ku): Dhawan
    • Status: Operating; users should include one of the canonical flux calibrators in SB.
    • Upcoming:
      • Need switched power
        • Develop set of observations to check Tcal/Tsys/SEFD across bands
    • Links

  • High Frequency (K/Ka/Q): Sjouwerman/Claussen
    • Status:
      • Operating; users should include one of the canonical flux calibrators in SB
      • Reference pointing in C or X
      • No observing at Ka < 32 GHz; No longer blocked; SSS/OPT code is consistent with limitations in hardware.
    • Upcoming:
      • Need switched power
        • Develop set of observations to check Tcal/Tsys/SEFD across bands
    • Links: ECSV-30

Observing Mode Support:

Testing and development within broad observing modes distinguished by their science goals (continuum, spectral line, polarization).

Targets

Area Target Priority JIRA Deliver Status
Spectral Line Bandpass stability 1 ECSV-35 2010Q3  
Spectral Line Doppler tracking 1 ECSV-32 2010Q3  
Spectral Line Spectral line setups; calibration procedures recommendations (OSRO/RSRO) 1 ECSV-31 2010Q3  
Polarization On-axis leakage calibration 1 ECSV-25 2010Q3  
Polarization R-L phase calibration 1 ECSV-25 2010Q3  
Polarization Observer Recommendations (OSRO/RSRO) 2 ECSV-25 2010Q3  
Polarization Beam squint calibration 2 ECSV-25 2010Q3  
Polarization Primary beam (off-axis) calibration 2 ECSV-25 2010Q3  
Polarization Polarization calibration service observation plan 3 ECSV-25 2010Q4  
Demo Science SBs for approved EPO projects 1 ECSV-57 2010Q3 Started for a subset

Reports

  • Spectral Line Modes (Ott/Goss): TSPE
    • Testing:
      • Doppler setting/Doppler tracking discussion. Discussion going on in e-mail. We believe that Doppler setting is okay from OPT to AIPS; there is a discussion about what level of channel oversampling is needed. Juergen will design some tests for this week. Doppler tracking on scan boundaries will be discussed within the group; ALMA is not offering this but hasn't ruled it out either. More from the group later.
      • Spectral Line general commissioning tests (diff spw widths, diff frequencies, bandpasses, etc): ECSV-92 * Bandpass stability: ECSV-35: More date coming soon; asked to revise source to take it out of the day time competition with development/construction activities.
    • Links: ECSV-19
  • Polarization Modes (Myers): TPOL
    • Status: Report below
    • Testing
      • Low frequency check outs completed; derived fractional polarization for known sources agree with predictions.
        • C band report (Marvil). ECSV-69 shows the recent results from the 3 hour C band; overall things look good and quite stable; report will be posted into JIRA.
      • Multi-band leakage/pos. angle calibration report pending.
      • Post-processing (AIPS): AH1004 data checkout (still needed)
    • Links:

  • Demo Science (Chandler/Condon): TDEM
    • Status: Begun; list requires prioritization; limited remaining time in D-config; Will try for IRC+10216 this week as possible.
    • Testing
      • Cyg A (Perley): SBs ready; Issues as noted above; will need repeat.
      • Crab Nebula (Frail): SBs submitted; 1 hour trial report (Frail)
    • Links: https://staff.nrao.edu/wiki/bin/view/EVLA/DemoScience

  • Advanced Planetary: TPLA
    • Testing
      • Cassini observations obtained.; Under review by Bryan.
    • Status:
      • Executor: Ready for testing (Butler).
      • SDM: Ephemerides/motion not supported.
      • OPT: webtest has standard planets; should be tested.
  • Advanced: Solar: TSOL
    • Status: Planning/prototype testing ongoing; working with L, Ku (special Rx setups in these two bands). RP noted that Tim Bastien will be out this week; following PASEO will work on solar mode testing.
  • Advanced: Pulsar (Brisken): TPUL
  • Advanced: VLBI (Walker/Brisken): TVLB
  • Advanced: Astrometry (Mioduszewski)

Observing Support:

Testing and development of the end-to-end capability of the array to support observations to the community (Documentation, OPT, Scheduling, Instrument models, Calibrator survey/models DB, Rapid response observing, demo science, post-processing).

Targets

Area Target Priority JIRA Deliver StatusSorted ascending
Software CBE averaging/decimation (time/freq) 1 - 2010Q3; vci averaging - honors other-than-1s dumps.  
Software/Calibration Recommended data path for weights in calibration 1 - 2010Q3  
Flux Calibration SBs for Flux calibrator models 1 ECSV-43 2010Q3 Needed for D config (2 freqs/band); 5 h of observations ready
Software CASA development cycle priorities 0 -   Spreadsheet developed; time allocations needed for P1s

Reports

  • Meta summary
    • Multi-config SBs in regular use; note 'overhead' in configuration changes; Must add 30 seconds to each configuration change - Gustaaf's group will check projects for this until relaxed, in particular, reference pointing must expand to 3 minutes..

  • Software Systems (Overlord: Butler):
    • Versioning Page: https://staff.nrao.edu/wiki/bin/view/EVLA/EVLASoftwareVersioning
    • *CBE store - new lustre file system (larger than 2 TB has not yet been installed).*Maybe next week for this.
    • OPT (Lorant): New version discussion this week; would like to get wideband defaults added to the system. Existing issues with wideband scripts sorted out (Harland)
    • Scheduling (Joan): OST in partial use; first 'fixed' observing tonight.
    • Executor (Barry):
    • Corr (CBE/CM; Michael): Outstanding issues with missing scans, etc are linked to the CBE; Martin will get back on the top of the priority heap next week after Brent departs.
    • Archive (John): Still minor issues with project lock/unlock states; security hole is being plugged (less said the better).
    • Post-processing
      • CASA (Nick):
        • Finalized plan draft sent to developers - not clear time allocations. Needs plan of work; Nick not available. flagdata interface being worked on; will make a final recommendation to CASA; Josh developed a task to review weather information - working to add this to CASA.
      • AIPS (Eric): OBIT SDM Filler (either AIPS dataset or FITS binary table format); will find out the details; working on fitting spectral indices.
    • ASDM
      • SDM total power/switched power table write-up (Rupen).
      • Flag table write-up (Myers/McMullin).Draft in and being reviewed; will make changes and percolate more broadly.
      • SDM document (with at least table descriptions/contents/units)Needed by the end of the month. (Rupen).
      • Priority 3: Ephemeris table resolution (Butler)
      • Bryan raised the issue of changes to the filler for more precise UVW calculations; will contact Joe on this.

  • Flux Calibrator Models (Amy/Rick): TCAL
    • Report: TCAL0001_sb1576498_1.55377.02314818287, TCAL0001.sb1575783.eb1620753.55378.450005439816. 5 hours of observations taken (low frequencies). Postpone report to next week.
    • Links: ECSV-43

  • RSRO/ECSO: TRSR
    • Status: RSRO Readiness (Cycle1) Master List
      • Need RSRO SBs in now! Please note the following:
        • Need 2x2 minute OSRO scans (for AIPS might want at a different frequency to ensure they're in a different SPW).
        • No scans longer than 90s
        • Set it up using the extended OSRO configurations
        • Other nominal guides for OSRO (e.g., ensure wrap is set for first frew scans).
        • Much confusion on the above; will try to write this more clearly; Action for Frazer to argue the case for why 90s scans matter; Joe must argue why 90s is really a limitation given some empirical data that we can handle more than this.
    • RSRO Science Observations (Red=recent observations requiring report; Blue=active; Black=pending):
      • AB1345 (TRSR0004): TRSR0001_sb1620012_1_002.55380.22739263889 (Rupen). Data look good; will proceed with project.
      • AC992: AC992_sb1426692_1_000.55380.88644908565 (Chomiuk). Looks good.
      • AB1346 (TRSR0005): AB1346.55382.19686146991 (Brogan/Hunter). Report available in: http://bugs.aoc.nrao.edu/browse/RSRO-3; many problems; data not viable.
      • AS1015 (TRSR0010): AS1015_sb1626830_1.55380.92802219908 (Chomiuk). Report available in: http://bugs.aoc.nrao.edu/browse/ECDATA-267; data look good; time logged from RSRO project.
      • AV318: Converted to OSRO and completed.
      • AM1014 (TRSR0009)
      • AT374 (TRSR0011); Russ is here. Will develop test run ASAP.
      • AK726 (TRSR0007)
      • AA330 (TRSR0003)
      • AL746 (TRSR0008): TRSR0008_Ka.55382.59489054398, TRSR0008_C.55382.62113342593; Report posted at: ECDATA-268; issues with C band RFI; missing spw; B/D not so good (didn't use set-and-remember for B/D).
      • AC982 (TRSR0002): AC982_sb1537991_1.55388.49498728009 (382 GB of love).Being reviewed.;
      • AH1006 (TRSR0006)
    • Resident Schedule: Posted on home page of twiki now!
      • 01 Jan - 16 May: Adam Deller (AC982: RSRO-4): 300A
      • 31 Jan - 31 Jul: Laura Chomiuk (AS1015/AK726: RSRO-11/RSRO-6): 200C
      • 04 Feb - 01 Jun: Jacqueline van Gorkom (AV318: RSRO-13): 200D
      • 25 Feb - 01 Jun: James Miller-Jones (AM1014: RSRO-9): 200E
      • 02 Mar - 07 Jun: Manuel Aravena (AA330: RSRO-1): 300G
      • 16 Mar - 08 Apr: Dave Green (AB1345: RSRO-2): 300G
      • 20 Apr - 30 Jun: Crystal Brogan (AB1346: RSRO-3): 200B
      • 20 Apr - 30 Jun: Todd Hunter (AB1346: RSRO-3): 200B
      • 30 Apr - 15 Jul: Joseph Lazio (AC982: RSRO-4): 266
      • 07 Jun - 16 Sep: Laura Perez (AC982: RSRO-4): 200E
      • 28 Jun - 05 Oct: Volker Heesen (AH1006: RSRO-5): 200G
      • 05 Jul - 26 Oct: Russ Taylor (AT374: RSRO-12): 301; Shannon Jaeger is also here.
      • 21 Jul - 23 Sep: Dave Green (AB1345: RSRO-2): 300G
      • 15 Sep - 15 Dec: Adam Leroy (AL746: RSRO-7): 200C
      • 01 Oct - 01 Jan: Hendrik Linz (AC982: RSRO-4): 200C
      • 01 Jan - 01 Apr: Jane Greaves (AC982: RSRO-4): 300G
      • 02 Jan - 02 Apr: Stuart Corder (AC982: RSRO-4): 300G
    • Current Residents
      • Laura Chomiuk (POC: Miller Goss; area(s): Correlator checkout; Spectral line commissioning; OSRO data validation
        • This week: Work on AS1015 data set.
      • Laura Perez (POC: Claire Chandler); area(s): High frequency observing
        • This week: Work on OSRO/RSRO reference pointing run, AC982 data.
      • Joe Lazio (POC: Claire Chandler): Low frequency observing.
        • This week: Work on AC982 data; pack.
      • Volker Heesen (POC: ):
        • This week: Work on wideband polarization test data set.
    • Past (and future residents)
      • Crystal Brogan (POC: Claire Chandler); area(s): High frequency observing; demo science
      • Todd Hunter (POC: Claire Chandler; area(S): science support software interfaces for users and high frequency observing; band/polarization checkout.
      • Manuel Aravena (POC: Chris Carilli); area(s): high frequency observing
      • Adam Deller (POC: Michael Rupen); area(s): phased array system design and implementation
      • Dave Green (POC: Sanjay Bhatnagar): area(s): algorithm tests for wideband imaging (currently away).
      • James Miller-Jones (POC: Claire Chandler; area(s): demo science use cases
      • Jacqueline van Gorkom (POC: Miller Goss; area(s): Correlator checkout; spectral line commissioning)

  • Science Support Team: Adding assignments for targeted data reviews
    • McMullin: Scheduling/Operations/On-call; CSV test data reviews; PASEO Prep; FLAG table (with Steve); updates to CSV planning.*
    • Myers: Technical reviews; PASEO work; look at multi-band polarization SBs (with Josh).*
    • Ott: Doppler tests; M51 SBs; RSRO SBs; CASA planning.
    • Greisen: fitting curved spectra to calibrator sources.
    • Dhawan: technical reviews; data reviews from targeted tests.
    • Kogan: RM application development; UV RFI rejection algorithms (EVLA Memo 146)*
    • Rupen: PASEO work; work with Brent on correlator connectivity issues; outstanding OSRO/RSRO blockers; support operations.*
    • Krauss: CASA documentation; pipeline scripting examples.

Advanced Science Support:

These are the testing/development areas needed to fulfill the full capabilities of the EVLA along with additional observing modes for other branches of the community.

  • Algorithms (Owen/Bhatnagar)
    • Status: Frazer: using M87 dataset testing wideband algorithms. Seeing issues with phase transfer; smoothly varying function by antenna; needs to be explored. (under calibration transfer). Discuss next week.
    • Links: https://staff.nrao.edu/wiki/bin/view/EVLA/Algorithms

  • Astrometry (Mioduszewski)
    • Status: Wait for A array(?)

  • HPC
    • Status: Minutes from last meeting.
The Postprocessing Hardware group met on July 2 at 1pm.

Attending in Socorro: Elias, Golap, Greisen, Owen, Robnett, van Moorsel in Charlottesville:  Cotton,Halstead, Hatz, Hunt, Lacy


  Kumar reported that had gotten the continuum simulation to run on the cluster and has circulated his primary results to this group. James argued that all the results can be understood in terms of disk head contention and the speed of the current network, which will be improved in the EVLA cluster so that the issues Kumar sees should not be a problem. Kumar will pass scripts to James next week so that James can run a set of tests himself to determine the final parameters of the EVLA cluster to be purchased.  It looks like the final parameters can be specified in time to allow a purchase by the end of July. Hopefully James can report his recommendations at the next meeting, which will be a major milestone.

  However, an important question is whether we should make the purchase then given the delays in the WIDAR commissioning. It would be good to be able to get on with the commissioning of the cluster. That means working out the procedures for submitting and queuing jobs for the cluster and then running test cases. James suggests this might be done by using a subset of the prototyping cluster and other existing hardware. If this is possible, we might defer the final cluster purchase in order to get more for our dollars; otherwise we might want to go ahead and purchase the cluster soon so we can get started.

  We further discussed the future of this meeting. The group wants to continue the twice-a-month format. However, once the EVLA cluster parameters have been settled, possibly at the next meeting, move the focus to more observatory-wide issues of HPC. Another meeting should be set up for local issues relating to the EVLA cluster.

  Because of the PASEO meeting on July 16, we will hold our next meeting on July 23rd.
    • Links:

AOB

  • Phase transfer discussion (Dhawan)

Misc Actions

  • [McMullin]: Review/recover metrics for Priority 0 items.
  • [Perley/McMullin]: SPW/IF/BB lexicon and mapper
  • List of known problems (can be used by OSRO commissioning team (van Moorsel)
      • Med Priority [Rupen]: X, Ku band subbands are inverted (IF 1 is labelled as IF4, IF2 as 3, IF3 as 2, IF 4 as 1). Noted that this had been tested but there were enough changes to warrant a re-do.
      • Med Priority [TBD]: Phase offsets within scans (phase jump at scan boundaries). R&L phase offset on some scans - probably L302 LO problem, needs tracking down. MR: Seen on EA07, EA25 (both RR and LL)
      • Med Priority [TBD]: Added. Phase change when frequency changes (needs to be tested).
      • Med Priority [TBD]: Anti-aliasing does not behave as advertised.
      • Low Priority [TBD]: First scan often bad.
      • Low Priority [TBD]: Phase (and amp) wobbles; long history.

-- JosephMcMullin - 07 Jul 2010
Topic revision: r4 - 2010-07-13, jmcmulli
This site is powered by FoswikiCopyright © by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding NRAO Public Wiki? Send feedback