• P-series runs Tuesdays and Thursdays around the time of I-series intensives, Saturdays and Sundays after Q-series intensives.
• NZ-Series runs approximately every three days.
  
  • BR cable/phase cal problem. Comes and goes, leading theory is thermal expansion and contraction of cables.
  • SC also has a cable cal problem but is slightly different in effect and doesn't seem to be due to thermal issues like at BR.
  • BR experienced an FRM failure mid-July which resulted in first no fringes being seen in the NZ observations and then the antenna being left out of the NZ's for the rest of the month. It also developed vertex room HVAC issues and equipment had to be shut down to avoid damage from overheating.
• Q-series intensive observations at MK run on weekends.
• I-series "Holiday" intensive observations at MK, substituting for Kokee on federal holidays.
  
  • There will be an upcoming period in the future where Kokee will be down, some signal change and receiver upgrades, but no dates yet. March 2026 currently the target.

• Two I-series observations were added late when Kokee needed to do more fiber work.
• An extra NZ session was observed.
• UF007 went slightly over their requested time.

• MK will not be available for a couple P-series observations in August due to subreflector painting work. Currently scheduled for the week of August 18.
• US008 was tentatively scheduled for August but has been pushed back to September. This reduced the requested hours from 211 to 188.

• Looks fairly normal.

• V2 correlator status:
  • Tuesday 5-7pm (Eastern Time) Maintenance Window weekly patches continue, with weekly reporting to USNO at the conclusion of each patch.
  • Troubleshooting as requested to maintain functionality.
  • USNO and NRAO received Authorization for the NGN/V2 network as of July 22, 2025.
  • NRAO is investigating a few issues since late April/early May - power supply faults (test) and a kernel on one MD node (production). The latter issue was flagged in a STIG scan by USNO in early June; a coordinated repair is planned for August 12 (having been postponed from July) with participation from both USNO stafff on site (connect KVM in advance, performing an image backup to external drive) and NRAO stafff remotely (modifying the kernel, testing.) To ensure similarity between the test and production systems, normal maintenance for this day will be postponed to the following Thursday (8/14), if USNO approves, or to the next weekly scheduled NRAO maintenance window on August 19, in compliance with the 2-week minimum cycle.
  • NRAO will begin training a representative of USNO/N6 on ongoing V2 maintenance, scope for which USNO intends to internalize in CY26, in the next month
• V3 correlator status:
  
  • NRAO completed Milestone 3, Software Testing in Socorro, on the prototype hardware; however, value of this milestone is to be redefined, and incorporated into new proposal with new milestones.
  • USNO delivered an Updated PWS to NRAO; however, additional changes in requirements and deployment process were briefed by USNO to NRAO in late June. A no-cost Extension to the contract, extending PoP to end of August, was executed. NRAO has solicited quotes and prepared an updated proposal according to most recent discussions and understanding of final requirements. These updates were briefed to Technical COR in early Aug. NRAO submitted this proposal to NAVSUP on August 7.

• USNO reports detecting a drift in MK baselines beginning mid-2023 and still happening.
  • Approximately 0.1 microsecond per day drift, works out to about 0.25 cm/year.
  • There was a major maintenance visit to MK in May 2023 during which the dichroic panel and antenna panels damaged by an ice storm in 2022 were replaced. Only other major work done in 2022 at MK was the replacement of an elevation motor in April.
  • It's seen in the Z/NZ series but not P-series? So maybe a correlation issue rather than with MK itself. Possibly seen in Q-series as well. Quick look at GPS data doesn't indicate a physical movement of the station.
  • Some kind of analysis issue is the current thinking.
• VLBA e-transfer system
  
  • RDV observations having non-VLBA stations e-transferred.
  • Direct transfers have worked, bypassing need for some stations to go through Haystack. Efforts continuing to add more stations.
  • Bonn has successfully pulled GMVA data from DSOC.
  • Anthony Sowinski PoC for NRAO.
• Data ingest of IVS data to NRAO/Socorro in support of USNO V2 correlator ("backup correlator") continuity of operations (COOP):
  • We will do a monthly COOP test of the V2 correlator by correlating a WZ-KK intensive. .
    
    • July test was on July 17. Due to additional fiber work at Kokee this was be done with MK-WZ.
    • August test was on August 7, KK-WZ.
  • MOA actually was that the VLBA correlator would be used for the COOP not the V2. Once eTransfer is in place that could be done.
    • e-transfer is in use now, time to revisit this topic?
      • Different DiFX versions are in use, some prep work would be necessary. A larger discussion is needed.
  • Further use of the V2 correlator in the processing of P-series and Z-series observations has been proposed.
    
    • Would need to use e-transfer.
    • An eventual goal would be to see a demonstration of these being processed without the involvement of any DC-based USNO personnel.
    • Another goal would be processing other intensives and bigger projects such as a 24-hour R4 observation.
      • etc/etd will be the only transfer software initially supported by NRAO e-transfer system.
        • For intensives, Wetzell has switched to using etc/etd, Kokee change was delayed due to fiber outage.
        • Kokee likely to be a lot of work involved in making the switch due to an older OS being used there.
    • Test correlations have been done on both P and Z series observations with the V2 correlator.
      • Existing storage space in the DSOC /datacopy areas is insufficient to hold data from a P series and a Z series at the same time.
        • More space will be available once e-transfer is in place.
          • Currently we are still using the datacopy areas for transfers from VLBA stations, procedures for using e-transfer area instead need to be developed.
      • Data transfer rates were exceptionally slow, probably due to using rsync because tsunami and etc/etd were not available to USNO users on gygax.
        • Will need to get USNO access to the new transfer software before further testing can be done.
        • Moving to eTransfer would also get the procedures off gygax which isn't really suitable for it.
• USNO would like to be able to check automatically if the V2 correlator is up or down but are unable to do so.
  • Will probably need to be handled via the e-transfer system due to internet security concerns.
  • If we switch to using VLBA correlator instead of v2 as per the MOA it would eliminate the need for this.
• Is it possible to reduce the latency on 24-hr geodetic USNO-only sessions, with a goal of a latency between the completion of observation to analyzed result under 7 days?
  • E-transfer not practical at this time so expedited shipping of disk packs is needed.
    • Now that e-transfer is working, should be consider trying it with UD011?
      • NRAO will put it on target list, would need to coordinate the download so as not to interfere with any realtime correlation observations.
      • Will be discussed in an NRAO e-transfer meeting July 30.
        • Under consideration.
  • Might be limited to just attempting it for one of the two UD011 observations per month.
  • Record of time to correlation:
    
    • first year of observations (UD011 AA - AR) average time was 9 days. (Specifics can be found in minutes of previous meetings.)
    • second year of observations (UD011 AS - BQ) average time was 12 days counting all observations, just counting the one observation a month we put a rush on the disk pack shipping the average time was 9 days.
    • third year: UD011 BR - 18 days, UD011 BS - 8 days, UD011 BT - 21 days, UD011 BU - 13 days, BV - 27 days; BW - 15 days; BX - 23 days; BY - 13 days; BZ - 20 days, CA - 17 days, CB - 13 days, CC - 17 days, CD - 17 days, CE - 10 days, CF - 13 days, CG - 14 days, CH - 13 days, CI - 17 days, CJ - 14 days, CK - 14 days, CL - 11 days, CM - observed July 28, awaiting correlation
• VNDA test with UD011 happening today. There was a holdup in getting the VNDA stations to record RDBE and VNDA data simultaneously. USNO will check on fringing but will not do any correlation.
• Next DiFX release (2.9) - Slow development continues.
  
  • testing underway., NRAO testing hopefully finished in 1-2 weeks. Key developers meeting at the end of August, release possible in early September.
    
    • One issue involving VGOS data in HOPS package still being worked on, a workaround is in place. A new version of HOPS is expected to resolve the issue, TBD.
• VEX2
  • Vex2 standard will cover missing tuning information that SKED and VieSked++ aren't able to include.
    • This information will still be required but VEX2 will have a space for it so manual editing won't be needed.
    • USNO has been sent vex2 samples, will have them send NRAO a file to test and make sure it works.
      • NRAO has received samples, no test file yet.
  • Things are okay right now with standard USNO setups and observations, could become an issue if USNO starts using non-standard setups in new observations.
  • Some testing of VEX2 has begun at NRAO.
    • Sched 12 is out, NRAO has switched to using vex2 internally.
      • Vex 1.5 will remain supported but once VNDA is installed and running in 2026 NRAO will probably end supporting v1.5 after a year.
      • Should start a test phase on USNO-provided vex files with vex2 now rather than later.
        • Problems reported by USNO with fourfit and vex2.
        • vex2 not supported by the mark4/HOPS data path? This and fourfit might be the same issue?
        • for the immediate future the vex 1.5 files can be used to avoid the fourfit and mark4 issues.
• Installation of new GPS receivers at VLBA sites.
  
  • Receivers have been installed at all sites but are not in use yet.
    • All stations except LA have been seen to transmit data but it's not usable by NRAO yet.
    • Some sites are experiencing overheating problems in the new GPS receivers.
      • Resolved, fan speed issue.
    • Also waiting on firmware updates to address using them for timing purposes.
      • Supposed to have been done, not certain.
• Mixed mode correlation
  • 2025 RDV sessions will have at least one VGOS station in mixed mode, USNO needs assistance NRAO in correlating them. Maybe contact Haystack or Bonn?
  • Have tried mixed-mode correlation starting with RV169 but no success as of yet.
  • Update from analysts?
• USNO asked about getting CASA to read pulse cal data the way AIPS does, it would help with their imaging efforts on weak astrometry targets at higher frequencies.
  • This is on the CASA developer's target list but low priority. Move to LT
• VLBA planned to start upgrading its correlator from RHEL7 to RHEL8 in January 2025.
  
  • USNO still needs to use DiFX 2.7.
  • Upgrade progress - Mark6's need some hardware updates before VLBA playback units can be upgraded to RHEL8.
    
    • First testing of new hardware on one Mark6 unit was successful, hardware for remaining upgrades has been ordered.
    • Hardware has arrived, upgrading of Mark6 units has started.
      • Slight delay arose, some of the new ordered hardware proved incompatible and will need to be exchanged.
      • Replacement hardware has arrived, upgrade of first two playback units is in progress.
      • Delays experienced due to needing DifX 2.9 to support two expansion chassis' per unit.

Long term issues

• Realtime correlation of intensives
  • Fast network connections at HN and MK make it theoretically possible but neither NRAO nor USNO are set up at the correlator end to do it yet.
  • NRAO could attempt this but there will be some limitations - VLBA stations only, limited to observations that record at 512 Mbits/sec or less and NRAO would need help from USNO in determining sessions, creating files and looking at the output.
  • An additional consideration is that while these observations would still record to Mark6 disk packs they would not record data to the DBS units so would not be easily downloadable should the realtime correlation not work.
  • USNO would have to wait for the v3 correlator before they could start planning for realtime correlation.
• DBS software changes
  • New software on the DBS units will ultimately cause a change in the naming structure of data files and PFB (Mark5B) data format will no longer be supported, instead it will be converted on the fly to VDIF format.
  • Software will need testing before deployment and first iteration of the updates may not change naming structure of data files or include the PFB->VDIF conversion.
  • Low priority, won't happen until after v3 correlator.
• New IVS session code naming structure change for intensives.
  
  • Problems encountered with the longer project codes:
    
    • Longer project codes not supported by all IVS stations so not all intensives will change to a longer code.
    • DiFX did have some project code length limitations, a change went into DiFX 2.8 .
      
      • USNO not using DiFX 2.8 yet. They can correlate with 2.8 but fringing doesn't work.
    • NRAO's Mark6 disk pack shipping/tracking program Packtrack has some problems with lengthy project codes. This would only affect any intensives which get correlated in Socorro off of disk packs, but currently there are none.
• .Peculiar delays - VLBA has a delay compared to Kokee that perhaps should be checked out, maybe put some VLBA stations into a KK observation as tagalongs?
  • Walter reached out to some people, says it looks like it would be a bigger effort than anticipated.
    
    • Possibility of adding KK to an RDV observation instead raised..
      • KK will be in the April 2024 RDV observation so could get data from that.
        • RV165 observed April 24. Has been correlated.
        • After USNO (or NASA?) does analysis a copy of the clock offsets can be provided to VLBA for comparisons.
          
          • Analysis finished, but didn't include VLBA stations so still need to do that and get the info to NRAO.
        • Kokee does have a different maser compared to last time VLBA correlated with them, might have an effect?
    • VLBA thinks it will have to wait until after all ten stations have received new GPS units (and they're all in use).
      • Prioritization of this and other lingering issues should be considered once we can start looking at this again.
    • Moving to VNDA will also change VLBA delays slightly, so it'll probably have to wait until after that's in place as well.
      
      • The RV165 clock offset data would be useful for a before and after comparison, but it would not be used to adjust the VLBA offset. A second observation like RV165 would be needed so we could make the comparison.

• 600 Mbps minimum at each site
• 1 Gbps at PT, KP, OV, NL. FD
• 5 Gbps at HN
• 10 Gbps at LA,BR, MK
• 600 Mbps at SC (download up to 400 Mbps) - possibility it could be going up to 1 Gbps