Date: January 16, 2026
Time: 10am Mountain, Noon Eastern.
Meeting will be held over Zoom. Details in email.

NRAO: Jim Ogle, Justin Linford, Heather Marshall
USNO: Bruce Thornton, Chris Dieck, David Gordon, Chris DiLullo, Nathan Secrest, Andy Sargent

  • Station Update - all ten stations operational. MK weather station still not communicating though and BR continues to have trouble with its subreflector positioning.

  • Intensive Observations
    • P-series runs Tuesdays and Thursdays around the time of I-series intensives, Saturdays and Sundays after Q-series intensives.
    • NZ-Series runs approximately every three days.
      • BR cable/phase cal problem. Comes and goes, leading theory is thermal expansion and contraction of cables.
      • SC also has a cable cal problem but is slightly different in effect and doesn't seem to be due to thermal issues like at BR.
    • Q-series intensive observations at MK run on weekends.
      • GSI now supports using etc/etd for file transfers but attempts to install it on the DBS at MK has so far failed to work so we continue to use tsunami.
    • I-series "Holiday" intensive observations at MK, substituting for Kokee on federal holidays.
      • Kokee will be down for some signal change and receiver upgrades, currently scheduled for March 2026. MK will substitute for it during that time.
        • Downtime for KK has moved to April.
  • Post-Monthly Report Review for December 2025
    • One P-series observation did not run due to mechanical issues with the FRM at HN.
    • UF007 requested 24 hours but only used 19.85 hours.
  • Request for January 2026
    • UD020 incremented to UD021, UG006 incremented to UG007
    • UG007 has two sessions instead of the usual one.
    • UF007 initially not in requested, added back in this week. I believe it should have a time charge of 2 but the revised request still has it at a 1.
  • Pre-Monthly Request for February 2026
    • GC040 will substitute for UD021 in February.
    • UF007 added back in after initially being left out.
  • Correlator Statuses
    • V2 correlator status:
      • Contract extension expected for 5 months, to 6/22/26.
      • Tuesday 5-7pm (Eastern Time) Maintenance Window weekly patches continue, with weekly reporting to USNO at the conclusion of each patch.
        • Window may be revised to facilitate training of USNO staff during ET business hours.
      • Troubleshooting as requested to maintain functionality.
      • 5 spare storage drives remain available at USNO.
      • In December, three new USNO staff were designated for training on maintenance; however, USNO IT staff access to production system is blocked by USNO internal firewall(?) or other issue - helpdesk ticket is open. Access to wiki and V2 test bed at DSOC are in place for trainees; sponsored accounts in progress.
    • V3 correlator status:
      • Milestone 3 complete - All switches & servers ordered.
      • Switches, KVM, and first storage systems have arrived in DC, more to come
      • Test bed assembly is underway in New Mexico
      • First installation visit planned at USNO for week of February 2-6.
      • Issues in DoD contract database are blocking processing of invoices. NAVSUP working the issue (we think/hope)

  • Open issues
    • Expiration of the V2 support contract
      • Expecting contract extended from January to June
        • COOP tests to resume using the V2 for duration of the extension?
          • Dates needed.
      • COOP tests and uses will eventually need to migrate to the VLBA correlator, probably using the e-transfer system.
        • Currently e-transfer does not have set procedures for downloading from VLBA stations to Socorro.
        • Hardware upgrade of the VLBA correlator could involve a second head node devoted to USNO needs.
        • Procedures for handling use of the VLBA correlator by USNO will need to be developed.
        • COOP likely to be renamed to BACKUP.
    • Data ingest of IVS data to NRAO/Socorro in support of USNO V2 correlator ("backup correlator") continuity of operations (COOP):
      • We will do a monthly COOP test of the V2 correlator by correlating a WZ-KK intensive. .
        • Due to original ending of the support contract for the V2 the last scheduled COOP test was held on January 7. To resume for the duration of the contract extension?
        • COOP used January 6 and 7 to do correlation of intensives due to power outages at USNO.
        • See above about moving COOP/BACKUP to using the VLBA correlator.
      • Further use of the V2 correlator in the processing of P-series and NZ-series observations, possibly also types of observations too, had been proposed.
        • With ending of the V2 support contract this discussion will move to using the VLBA correlator. (move to above section)
      • USNO would like to be able to check automatically if the V2 correlator is up or down but are unable to do so.
        • The switch to using VLBA correlator instead of V2 would eliminate the need for this.
        • Checking on the V2 is no longer much of a priority due to the end of the V2 support contract. (can drop from agenda)
    • DBS software changes
      • xcuberec program which records the data on the DBS will be replaced by sendit. This will cause a change in the naming structure of data files and the PFB (Mark5B) data format will no longer be supported, instead it will be converted on the fly to VDIF format.
      • Initial testing of sendit has begun by running it in parallel with xcuberec at MK. Once the software is running at HN we will be ready to do some comparison testing when USNO is able to do a double correlation.
        • Is USNO ready to do double correlation testing? P2026013 was successfully recorded by both xcuberec and sendit.
        • Will email Bruce with location of the sendit data for P2026013.
    • VLBA e-transfer system
      • RDV observations having the non-VLBA stations e-transferred.
      • Anthony Sowinski PoC for NRAO.
      • KK having issues getting etc/etd to transfer to WACO and VLBA. But they can transfer to Haystack okay?
    • Is it possible to reduce the latency on 24-hr geodetic USNO-only sessions, with a goal of a latency between the completion of observation to analyzed result under 7 days?
      • E-transfer not practical at this time so expedited shipping of disk packs is needed.
        • Now that e-transfer is working, should be consider trying it with UD011?
          • Under consideration, would require some coordination around observing and bandwidth concerns especially with the slower stations like SC. Also no procedures in place yet for e-transferring data from VLBA stations to DSOC.
      • Might be limited to just attempting it for one of the two UD011 observations per month.
      • Record of time to correlation:
        • first year of observations (UD011 AA - AR) average time was 9 days. (Specifics can be found in minutes of previous meetings.)
        • second year of observations (UD011 AS - BQ) average time was 12 days counting all observations, just counting the one observation a month we put a rush on the disk pack shipping the average time was 9 days.
        • third year of observations (UD011 BR - CO) average time was 15 days counting all observations, just counting the one per month with rush shipping the average was 13 days.
        • fourth year: UD011 CP - 12 days, CQ - 15 days, CR - 13 days, CS - 13 days, CT - 29 days (problems with correlator updates at DSOC slowed things down), CU - 16 days, CV - 22 days, CW - 20 days
    • Current DiFX release (2.9)
      • One issue involving VGOS data in HOPS package still being worked on, a workaround is in place. A new version of HOPS is expected to resolve the issue, still in development.
    • VEX2
      • Vex2 standard will cover missing tuning information that SKED and VieSked++ aren't able to include.
        • This information will still be required but VEX2 will have a space for it so manual editing won't be needed.
        • Things are okay right now with standard USNO setups and observations, could become an issue if USNO starts using non-standard setups in new observations.
        • USNO has been sent vex2 samples, will have them send NRAO a file to test and make sure it works.
          • NRAO has received samples, no test file yet.
        • Question from Ed H at Goddard about status of vex2, and any shortcomings that might push an incremental VEX2 release? USNO has one thing they'll bring up.
          • Some progression in IVS meetings, but issues remain to be addressed particularly with old files and if conversion will be necessary. Another meeting at end of January.
      • NRAO is using VEX2 now.
        • Sched 12 is out, NRAO has switched to using vex2 internally.
        • Vex 1.5 will remain supported but once VNDA is installed and running in 2026 NRAO will probably end supporting v1.5 after a year.
          • Should start a test phase on USNO-provided vex files with vex2 now rather than later.
            • Problems reported by USNO with fourfit and vex2, there's a workaround.
            • vex2 not supported by the mark4/HOPS data path? This and fourfit might be the same issue?
            • for the immediate future the vex 1.5 files can be used to avoid the fourfit and mark4 issues.
    • Installation of new GNSS/GPS receivers at VLBA sites.
      • Receivers have been installed at all sites but are not in use yet.
        • All stations have been seen to transmit data but it's not usable by NRAO yet.
        • Also waiting on firmware updates to address using them for timing purposes.
          • Done, but not all stations are working identically so some issues are still being tracked down.
          • Nine of the ten have valid time offset code running. One more update before end of calendar year hope to have all ten running. Would like to have six months of running data from the sites before they declare it fully operational. VLBA would be able to start own analysis during that time.
    • Mixed mode correlation
      • 2025 RDV sessions will have at least one VGOS station in mixed mode, NRAO needs assistance in correlating them. Maybe contact Haystack or Bonn?
      • Have tried mixed-mode correlation starting with RV169 but no success as of yet.
        • Data analysts report some progress but still not successful so still working on it.
        • Think we'll just be giving up on having VGOS stations in RDVs, not getting good support from Haystack and WACO on figuring out how to make it work.
    • USNO asked about getting CASA to read pulse cal data the way AIPS does, it would help with their imaging efforts on weak astrometry targets at higher frequencies.
      • A CASA developer has been assigned and work has started on it.
      • Hoping to have a test version available in early December.
        • Test version available, being worked with, won't make it into the next release of CASA but hopefully will be in the next one (April 2026).

    • Are the monthly feedback reports from USNO still useful? Chris Dieck leaving current position so someone else at USNO would need to take over. Typically if something bad comes up it gets brought to NRAO's attention right away not in the feedback report and USNO has its own tracking system for problems. Does NRAO get any value from the reports? (Jim does look at them.) Also, are they required by contract? It was back in the 2012 contract but is it still in current contracts?

  • Long term issues

    • USNO reports detecting a drift in MK baselines beginning mid-2023 and still happening.
      • Approximately 0.1 microsecond per day drift, works out to about 0.25 cm/year.
      • It's seen in the NZ series but not P-series, so maybe a correlation or analysis issue rather than with MK itself.
    • Realtime correlation of intensives
      • Fast network connections at HN and MK make it theoretically possible but neither NRAO nor USNO are set up at the correlator end to do it yet.
      • NRAO could attempt this but there will be some limitations - VLBA stations only, limited to observations that record at 512 Mbits/sec or less and NRAO would need help from USNO in determining sessions, creating files and looking at the output.
      • An additional consideration is that while these observations would still record to Mark6 disk packs, they would not record data to the DBS units so would not be easily downloadable should the realtime correlation not work.
      • USNO will wait for the v3 correlator before they can start considering realtime correlation.
    • .Peculiar delays - VLBA has a delay compared to Kokee that perhaps should be checked out, maybe put some VLBA stations into a KK observation as tagalongs?
      • Walter reached out to some people, says it looks like it would be a bigger effort than anticipated.
        • Possibility of adding KK to an RDV observation instead raised..
          • KK will be in the April 2024 RDV observation so could get data from that.
            • RV165 observed April 24. Has been correlated.
            • After USNO (or NASA?) does analysis a copy of the clock offsets can be provided to VLBA for comparisons.
              • Analysis finished, but didn't include VLBA stations so still need to do that and get the info to NRAO.
            • Kokee does have a different maser compared to last time VLBA correlated with them, might have an effect?
        • VLBA thinks it will have to wait until after all ten stations have received new GPS units (and they're all in use).
          • Prioritization of this and other lingering issues should be considered once we can start looking at this again. Maybe mid to late 2026?
        • Moving to VNDA will also change VLBA delays slightly, so it'll probably have to wait until after that's in place as well.
          • The RV165 clock offset data would be useful for a before and after comparison, but it would not be used to adjust the VLBA offset. A second observation like RV165 would be needed so we could make the comparison.
          • VNDA timescale looks to be lining up with the v3 correlator so we'll just wait for both of those to be completed.

  • Fiber Status at each site:
    • 600 Mbps minimum at each site
    • 1 Gbps at PT, KP, OV, NL. FD
    • 5 Gbps at HN
    • 10 Gbps at LA,BR, MK
    • 600 Mbps at SC (download up to 400 Mbps) - possibility it could be going up to 1 Gbps
  • The practical link capacity due to overhead and other traffic is at most 85% of the reported link speed
Next meeting February 13

-- JimOgle - 2026-01-08
Topic revision: r11 - 2026-03-13, JimOgle
This site is powered by FoswikiCopyright © by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding NRAO Public Wiki? Send feedback