• P-series runs Tuesdays and Thursdays around the time of I-series intensives, Saturdays and Sundays after Q-series intensives.
• NZ-Series runs approximately every three days.
  
  • BR cable/phase cal problem. Comes and goes, leading theory is thermal expansion and contraction of cables.
  • SC also has a cable cal problem but is slightly different in effect and doesn't seem to be due to thermal issues like at BR.
• Q-series intensive observations at MK run on weekends.
  • GSI now supports using etc/etd for file transfers but attempts to install the software on the DBS at MK has so far failed to work so we continue to use tsunami.
    • latest info is that one of the provided port addresses at GSI is not working for NRAO and prevents data transfer.
• I-series "Holiday" intensive observations at MK, substituting for Kokee on federal holidays.
  
  • Kokee will be down for some signal change and receiver upgrades, now scheduled to start August 19. MK will substitute for it during that time.
  • Could be down for 4-6 months? Work might only take one month but checkout and testing could take longer.

• UD021A did not run in January, backlog to February. Ran February 6.
• GC040 substituted for UD021.
• UF007 was added back into the request after initially being left out.
• UG007C did not run in February, backlog to March (ran March 5).
• Ten NZ-series sessions were requested, nine ran. February being a short month a 3-day cadence would only allow for 9 sessions.
• Sixteen P-series were requested, twelve ran. Three were canceled due to bad weather, fourth canceled due to ACU problems at MK.
• UF007 requested 24 hours, only used 18.6 hours.

• US009 was not in initial request for March, three hours added.

• The Q-series start times entry has an extraneous entry from March in it.

• V2 correlator status:
  • Contract extension expected for 5 months, to 6/22/26.
    • February's visit to USNO DC included discussions regarding V2 maintenance transitions and plans for backup operations correlations on VLBA correlator.
  • V2 Production system maintenance window is 1-3pm ET / 11a-1p MT on Tuesdays, within normal business hours for DC team. However, note that during kernel patch weeks, the outage window extends by 2-4.5 hours.
  • 4 spare storage drives remain available at USNO.
• V3 correlator status:
  
  • Milestone bills paid, thank you!
  • RMF Audit documentation is nearly complete, paused temporarily in favor of NM Test system commissioning and testing, toward authorization for Dell to proceed with full DC server fabrication. All switches, KVMs, and storage JBODs have been delivered to DC, remaining computing equipment completion pending NM test results.

• COOP tests and uses will need to migrate to the VLBA correlator, probably using the e-transfer system, after the V2 support contract extension ends.
  • Currently e-transfer does not have set procedures for downloading from VLBA stations to Socorro.
  • Hardware upgrade of the VLBA correlator could involve a second head node devoted to USNO needs.
  • Procedures for handling use of the VLBA correlator by USNO will need to be developed.
  • COOP likely to be renamed to BACKUP.
  • Additional use of VLBA correlator for processing of P-series and NZ-series observations also part of the discussion.
    • Use of VLBA correlator for processing of a P-series was tested by Walter with P2026043 (ran February 12).
      • Has been correlated and sent on to USNO for analysis.
      • There was a clock offset from the VLBA correlator that USNO normally takes out so they'll send information about doing that back to NRAO so the two can be properly compared. Walter also wonders if sendit vs xcuberec might introduce a difference at the USNO end.
        • Hardware concerns for doing the correlation at VLBA? No not really for things like the P-series. VGOS stations though record 4x the amount of data (8 Gbps) so that could cause slower correlation in correlating those on VLBA.
• Data ingest of IVS data to NRAO/Socorro in support of USNO V2 correlator ("backup correlator") continuity of operations (COOP):
  • We will do a monthly COOP test of the V2 correlator by correlating a WZ-KK intensive.
    • See above about moving COOP/BACKUP to using the VLBA correlator in the future.
    • February test ran on the 19th.
    • March test scheduled for the 19th.
      • COOP exercised on March 9
        • Request asked for II-series and V-series downloads in addition to I-series but USNO correlator internet was restored before that was needed.
        • II has been downloaded once before using the procedures for the I downloads.
        • V has never been done before and involves connecting to different servers. Attempts to use tsunami to download the Kokee data failed. Never got around to attempting it with Kokee before USNO correlator internet connection was restored.
        • Noted that using the V2 is slow due to needing to data transfer from gygax to the correlator, the V-series would be even worse due to the amount of data. Is probably not worth trying to do a V-series on the V2, hold off on that for using the regular VLBA correlator as the backup.
• DBS software changes
  • xcuberec program which records the data on the DBS will be replaced by sendit. This will cause a change in the naming structure of data files and the PFB (Mark5B) data format will no longer be supported, instead it will be converted on the fly to VDIF format.
  • Initial testing of sendit has begun by running it in parallel with xcuberec at MK. Once the software is running at HN we will be ready to do some comparison testing when USNO is able to do a double correlation.
    • Is USNO ready to do double correlation testing? P2026013 was successfully recorded by both xcuberec and sendit.
    • Emailed Bruce with location of the sendit data for P2026013.
      • Has it been correlated? Not yet, but data was downloaded. Should get correlated in February.
        • Hasn't been done yet.
• VLBA e-transfer system
  • RDV observations having the non-VLBA stations e-transferred.
  • Anthony Sowinski PoC for NRAO.
  • KK having issues getting etc/etd to transfer to WACO and VLBA. But they can transfer to Haystack okay?
• Is it possible to reduce the latency on 24-hr geodetic USNO-only sessions, with a goal of a latency between the completion of observation to analyzed result under 7 days?
  • E-transfer not practical at this time so expedited shipping of disk packs is needed.
    • Now that e-transfer is working, should be consider trying it with UD011?
      • Under consideration, would require some coordination around observing and bandwidth concerns especially with the slower stations like SC. Also no procedures in place yet for e-transferring data from VLBA stations to DSOC.
  • Might be limited to just attempting it for one of the two UD011 observations per month.
  • Record of time to correlation:
    
    • first year of observations (UD011 AA - AR) average time was 9 days. (Specifics can be found in minutes of previous meetings.)
    • second year of observations (UD011 AS - BQ) average time was 12 days counting all observations, just counting the one observation a month we put a rush on the disk pack shipping the average time was 9 days.
    • third year of observations (UD011 BR - CO) average time was 15 days counting all observations, just counting the one per month with rush shipping the average was 13 days.
    • fourth year: UD011 CP - 12 days, CQ - 15 days, CR - 13 days, CS - 13 days, CT - 29 days (problems with correlator updates at DSOC slowed things down), CU - 16 days, CV - 22 days, CW - 20 days, CX - 19 days, CY - 16 days, CZ - 13 days, DA - 15 days, DB - observed March 9
• Current DiFX release (2.9)
  • One issue involving VGOS data in HOPS package still being worked on, a workaround is in place. A new version of HOPS is expected to resolve the issue, still in development. Might be ready for testing in the spring.
• VEX2
  • Vex2 standard will cover missing tuning information that SKED and VieSked++ aren't able to include.
    • This information will still be required but VEX2 will have a space for it so manual editing won't be needed.
    • Things are okay right now with standard USNO setups and observations, could become an issue if USNO starts using non-standard setups in new observations.
    • USNO has been sent vex2 samples, will have them send NRAO a file to test and make sure it works.
      • NRAO has received samples, no test file yet.
        • Fell off the radar, Phillip will look into it from their side again. Walter says not time-critical yet since RDBE's will remain in place through end of CY26 so vex1.5 support will remain until at least then but once VNDA takes over completely that's likely to go away.
    • Question from Ed H at Goddard about status of vex2, and any shortcomings that might push an incremental VEX2 release? USNO has one thing they'll bring up.
      • Some progression in IVS meetings, but issues remain to be addressed particularly with old files and if conversion will be necessary.
        • No recent IVS meetings so no updates. Most remaining issues are due to outdated catalogs so this is likely to be a long-term issue maybe not worth tracking here.
  • NRAO is using VEX2 now.
    
    • Sched 12 is out, NRAO has switched to using vex2 internally.
    • Vex 1.5 will remain supported but once VNDA is installed and running in 2026 NRAO will probably end supporting v1.5 after a year.
      
      • Should start a test phase on USNO-provided vex files with vex2 now rather than later.
        • Problems reported by USNO with fourfit and vex2, there's a workaround.
        • vex2 not supported by the mark4/HOPS data path? This and fourfit might be the same issue?
        • for the immediate future the vex 1.5 files can be used to avoid the fourfit and mark4 issues.
• Installation of new GNSS/GPS receivers at VLBA sites.
  
  • Receivers have been installed at all sites but are not in use yet.
    • All stations have been seen to transmit data but it's not usable by NRAO yet.
    • Also waiting on firmware updates to address using them for timing purposes.
      • Done, but not all stations are working identically so some issues are still being tracked down.
      • Nine of the ten have valid time offset code running. One more update before end of calendar year hope to have all ten running. Would like to have six months of running data from the sites before they declare it fully operational (so mid 2026?). VLBA would be able to start own analysis during that time.
      • A couple months of data has been looked at by NRAO, all ten sites now providing data but there are still some issues that need to be discussed with the providers.
• Mixed mode correlation
  • Have tried mixed-mode correlation on VLBA correlator starting with RV169 but with no success.
    • Giving up on having VGOS stations in RDVs, not about to figure out how to make it work on VLBA correlator.
      • USNO asked about the backlog of RDV correlations that were being held up by the mixed mode attempts that are no longer needed.
        • NRAO analysts will prioritize getting caught up on those.
    • Probably need to try again with a small data set that USNO correlates first to show it works and then send the data to VLBA to try correlating it.
    • Might be due to differing versions of DiFX being involved, most VGOS requires older version (v2.5.4) that VLBA doesn't use anymore? Even USNO has some trouble correlating it (bit-flipping, complex data, etc).
• USNO asked about getting CASA to read pulse cal data the way AIPS does, it would help with their imaging efforts on weak astrometry targets at higher frequencies.
  • A CASA developer has been assigned and work has started on it.
  • Hoping to have a test version available in early December.
    • Test version available, being worked with, expected to be in the April CASA release.
• Are the monthly feedback reports from USNO still useful?
  • Chris Dieck leaving current position so someone else at USNO would need to take over.
  • Typically if something bad comes up it gets brought to NRAO's attention right away not in the feedback report and USNO has its own tracking system for problems.
  • Does NRAO get any value from the reports? (Jim does look at them.)
  • Also, are they required by contract? It was back in the 2012 contract but is it still in current contracts?
    • Walter think that if anywhere it would be in the old implementation plan, not the MOA, so probably isn't required.
• USNO is considering adopting a DOI (Digital Object Identifier) policy on their processed data (IDI FITS files, MARK4 files, etc), wonder if NRAO might be interested or able to do the same? They've begun discussions with the DOE who provide DOI services to the government. Walter will check with NRAO's archive team and see how practical it might be, maybe there's already some form of DOI in place?

Long term issues

• Realtime correlation of intensives
  • Fast network connections at HN and MK make it theoretically possible but neither NRAO nor USNO are set up at the correlator end to do it yet.
  • NRAO could attempt this but there will be some limitations - VLBA stations only, limited to observations that record at 512 Mbits/sec or less and NRAO would need help from USNO in determining sessions, creating files and looking at the output.
  • An additional consideration is that while these observations would still record to Mark6 disk packs, they would not record data to the DBS units so would not be easily downloadable should the realtime correlation not work.
  • USNO will wait for the v3 correlator before they can start considering realtime correlation.
• .Peculiar delays - VLBA has a delay compared to Kokee that perhaps should be checked out, maybe put some VLBA stations into a KK observation as tagalongs?
  • Walter reached out to some people, says it looks like it would be a bigger effort than anticipated.
    
    • Possibility of adding KK to an RDV observation instead raised..
      • KK will be in the April 2024 RDV observation so could get data from that.
        • RV165 observed April 24. Has been correlated.
        • After USNO (or NASA?) does analysis a copy of the clock offsets can be provided to VLBA for comparisons.
          
          • Analysis finished, but didn't include VLBA stations so still need to do that and get the info to NRAO.
        • Kokee does have a different maser compared to last time VLBA correlated with them, might have an effect?
    • VLBA thinks it will have to wait until after all ten stations have received new GPS units (and they're all in use).
      • Prioritization of this and other lingering issues should be considered once we can start looking at this again. Maybe mid to late 2026?
    • Moving to VNDA will also change VLBA delays slightly, so it'll probably have to wait until after that's in place as well.
      
      • The RV165 clock offset data would be useful for a before and after comparison, but it would not be used to adjust the VLBA offset. A second observation like RV165 would be needed so we could make the comparison.
      • VNDA timescale looks to be lining up with the v3 correlator so we'll just wait for both of those to be completed.

• 600 Mbps minimum at each site
• 1 Gbps at PT, KP, OV, NL. FD
• 5 Gbps at HN
• 10 Gbps at LA,BR, MK
• 600 Mbps at SC (download up to 400 Mbps) - possibility it could be going up to 1 Gbps