USNOVLBASupportMeetingJuly2020 < NM/Electronics

You are here: NRAO Public Wiki>NM/Electronics Web>USNOVLBASupport>USNOVLBASupportMeetingJuly2020 (2020-07-16, JimOgle) Edit wiki text Edit Attach Print version

Date: July 16, 2020 (delayed one week to Polycom problems)

Time: 11:00 am Mountain, 1:00 pm Eastern
Phone: 434-817-6468 ext 228111# (Polycom still not working, will try using Microsoft Teams instead...that didn't work either so trying a conferencing system at (203)-418-3123 6479449#...which not everyone was able to join due to a limit on participants.)

NRAO: Jim, Walter, Justin, Mark
USNO: Chris, Nicole, Andy, Philip, Stewart, Matt, Dave

VLBA Observations
- Daily Observation Report (N-series)
  - Four reobservations in June; three probably triggered by bad RFI at PT, one due to RDBE glitching at PT. The latter observation was also redone with SC instead of MK due to scheduling conflicts.
  - A few observations had cable cal problems at PT, cause unknown.
  - The N-series isn't getting as good data quality as the F-series so maybe change the setups? If change N also change O. The new P-series has been using the F-series setups on the HN-MK baseline.
  - S-band filters will probably be pulled off of receivers in the near future, might affect quality - this would not apply to PT and MK because they don't have S-band filters.
  - August request asks to reduce duration from 1.5 hours to 1 hour and changes the window for observing time. The goUSNO script will need to be updated for the shorter duration.
- Other Intensive Observations
  - O-series test on HN-OV baseline
    - Will run simultaneous with the N-series observations but will not interrupt other observing.
    - Possibly expand to include another baseline like SC-BR or SC-LA (avoiding S-band RFI at BR.)
    - Possible oscillation/tracking issue at HN recently detected, suspect an elevation encoder issue.
      - Only seen at low temperatures, hasn't been noticed since January.
    - August request asks to reduce duration from 1.5 hours to 1 hour and changes the window for observing time. The goUSNO script will need to be updated for the shorter duration.
    - Study of O-series doesn't show any movement of OV from earthquakes in the vicinity.
  - F-series test on six of the VLBA stations
    - August request changes the window for observing time.
  - P-series intensive observations roughly every other day on the HN-MK baseline
    - Project code for these observations will be PYYDOY.
    - Using the F-series setups rather than the ones used by the N and O series but duration is 1.5 hours like the N and O series rather than the 1 hour of the F series.
    - Internet problems at the HN station that lasted for four days prevented two instances of the P-series from being run in June.
  - "Holiday" intensive observations at MK
    - Will be substituting MK for Kokee on federal holidays in 2020 such as 7/3/20.
    - Project code for these observations will be IYYDOY.
  - MK-WZ backup observation series
    - Project code for this observation will be QYYDOY.
    - Involves a new post-observing procedure for operators to copy the data to correlator in Japan. This was retested in early June and is working after a configuration change was undone on the correlator side (blocked access).
    - Being briefly restarted in late June and early July due to the Ishioka station being unavailable.
      - Dates are 6/20, 6/21, 6/27, 6/28, 7/4, 7/5, 7/11, 7/12.
      - There was a problem with exper_name in one of the June vex files being wrong which confused the VLBA system and prevented a file from running. Editing the file to use the correct exper_name fixed the problem.
- Post-Monthly Report Review
  - No backlog from May that needed to be caught up in June.
  - Sowders replacement observation used less than their requested 8 hours.
  - A new observation of similar type to the Sowders replacement (but 6cm instead of 13cm) had requested 16 hours a month since April but has used none of it.
  - 'UG003Y' was in the request for June but PI told data analysts that the project finished in May. USNO was going to get in touch with David Gordon to confirm. No UG003 was observed in June. The July request calls for one as well.
- Pre-monthly request
  - As noted above, August request asks that the N-series and O-series be reduced in duration from 1.5 hours to 1 hour and the window for observation changes. The F-series also has its window changed.
  - August request asks that the UD009 observation be scheduled back-to-back with a new 24-hour observation (no project code assigned yet).
  - There isn't a UG003 in the August request. Might be a mistake? Project is expected to continue but might be with a different PI and project code.
Open issues
- USNO emails to ut1 list are not being forwarded along to other USNO people.
  - Mail is being blocked at the Dept. of Defense level. A possible fix is being tested.
- HN-OV baseline is more precise than any of the others USNO has data for, not sure why. Plus with all the recent problems showing up at PT maybe when fiber available to HN switch the N-series to them and PT-MK to the O-series?
  - HN fiber probably won't be in place until August or September, the work was expected to begin in May but hasn't yet.
    - HN fiber speed expected to be 2.5 Gbps.
- LA fiber may be at 10 Gbps when it's installed; this might impact consideration towards future use of it in baselines.
  - Was told last month that it was on schedule for deployment in late July but there's been no update since then.
- BR was added to the F-series for testing of additional baselines. S-band RFI might be a problem though, similar to what PT was recently experiencing but worse. Mitigation work on BR's S-band RFI probably won't happen any time soon given current situation.
  - It looks like only one S-band channel had to be dropped so the RFI may not be that bad with the F-series setups (which are different from the N-series setups).
  - BR looks pretty good.
- USNO needs to renew support contract for MK fiber.
  - This contract is being moved into the larger support contract (see below), will need to coordinate on handoff.
  - Will end up doing a short support contract of a couple months and then a more regular one to start after that.
- Networking Support Contract: Contract will be renewed.
  - Working on renewal, might involving changing some dates on it to avoid bunching all USNO contracts up at the same time of year. VLBA might all be bundled together at a different date. Moving forward.
- Correlator Refresh Contract
  - Lustre filesystem and second set of nodes were assembled and tested in Socorro in June and then disassembled for shipping to USNO.
  - The new "BeeGFS" file system has been assembled and configured and tested in Socorro.
  - The second set of nodes cannot be installed until USNO is off of their old correlator first, waiting for confirmation from USNO that they have done so.
    - Have started parallel testing of old and new correlator. Seeing some differences, tracking down what might be causing them.
  - Install was on hold due to COVID-19 work and travel restrictions, a remote installation will most likely be done instead.
    - The old Lustre filesystem can't be updated and is viewed as a security concern so there's pressure to get this done.
    - BeeGFS file system hardware arrived and has been installed. Installation of second set of nodes not complete yet.
    - Old correlator and Lustre nodes about ready to be shut down, been running on new stuff for about a week, waiting on some networking setups to get finalized for final switchover.
  - Extension through 11/30 for the hardware refresh has been approved.
- A follow-on to the correlator support contract is being worked on. Walter will coordinate with USNO for NRAO's side. Has been resubmitted and is moving forward.
- USNO is moving to new ingest data server and IP address is not working even though it's on NRAO's whitelist. USNO reports attempts to transfer data to the .179 address does not work. Looks like problem might be firewalls at USNO end.
  - Waiting on hardware and software updates, ETA end of July.
- Transition to Mark6 recording system at VLBA is done, including 4 Gbps capability.
  - Tests for transitioning to the higher data rate, perhaps with C-band, will be considered. (Though C-band has ionosphere problems.)
    - Mark Claussen will provide some 4 Gbps setups for USNO to look at for the tests. Hasn't happened yet.
  - Might be some USNO funding for more Mark6 disk drives/packs. Currently VLBA has sufficient supply of Mark6 disk packs though as 4 Gbps ramps up having more packs will be helpful.
    - USNO has sent twenty older disk packs of 32 TB capacity to VLBA for use.
- New IVS session code naming structure (long-term issue)
  - There is likely to be a change to format of the IVS "session code", but maybe not until 2021. As the USNO works to make the VLBA intensives look like IVS sessions we want to discuss what impacts such a change might have on this effort. This particularly relates to the 'exper_name' in the .vex files.
  - Intensives are all now using the xYYDOY naming format (ie, N20001, O20001, etc)
  - New IVS codes will be similar to existing codes but perhaps slightly longer, for example "NYYYYDOYNN". USNO & Goddard have discussion about it, length will be more than six characters. NRAO has already used some seven character length codes (such as "N20014B") without it causing any problems.
- VLBA able to participate in CONT2020 (Nov-Dec 2020). Won't be a full 15-day session, maybe a "mini-CONT" of around five days. (Long-term issue)
  - Looks like not even a mini-CONT this year.
  - There's interest in having VLBA observe along with a VGOS session in "mixed-mode". But there's apparently some support problems with latest version of 'DifX' correlator software and VGOS, more technical discussions need to be held to determine what could be done. Some discussion has happened, may not be as big of a problem as initially thought.
- Peculiar delays (long term issue) - VLBA has a delay compared to Kokee that perhaps should be checked out. Maybe put some VLBA stations into a KK observation as tagalongs so comparisons can be made. Walter reached out to some people, says it looks like it would be a bigger effort than anticipated and may have to wait until things return to normal.
- Next meeting August 13.

-- JimOgle - 2020-07-07

Topic revision: r5 - 2020-07-16, JimOgle

NM/Electronics

Copyright © by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding NRAO Public Wiki? Send feedback