This document covers the suggested plan for moving from VEX1 to VEX2.


A key point in understanding the transition plan is that VEX2 is not backward compatible with VEX1. Normally we strive for backward compatibility. In this case, we feel that not requiring backward compatible has the advantage for software, but maybe more particularly for humans, of not having to consider that there might be more than one way to do things in a VEX2 file. It also allows us to remove obsolete features from the specification.

One reason we have followed this approach is that once VEX2 features have been implemented, we believe it should be relatively easy to convert existing usage of discontinued VEX1 features to the corresponding VEX2 features. While this means that strict backward compatible is not possible, VEX1 files will still be usable as a variant for at least some period of time, as described for the Transition Period below.

We encourage parsers to be made both VEX1 and VEX2 tolerant so that a given user can read either format with one parser. In this approach, users of parsers are encouraged to consider using if/then/else type programming structures (or allowing and looking for both options) in order to avoid having to keep two different versions of software up to date.

The summary of the VEX1 features removed for VEX2 is:

  • electronics_rack_* parameters
  • record_transport_* parameters
  • tape_* parameters
  • record_density parameter
  • number_drives parameter
  • fanin_def parameter
  • $ROLL block
  • $HEAD_POS block
  • $PASS_ORDER block

Other VEX2 changes for existing systems include:

  • Mark 5B uses $BITSTREAMS instead of $TRACKS
  • tape_control parameter replaced with record_control parameter
  • tape_motion parameter replaced with record_method parameter
  • sample_rate parameter removed from $FREQ and put into $TRACKS for Mark IV recording
  • obsolete equipment removed: Mark3A, S2, VLBA, VLBAG
  • IF connections now use "connection" parameter in $DAS
  • Roll and Pass are no longer used

The actual transition plan has two major steps:

  1. Test Period
  2. Transition Period

These are described in detail below. It should be noted that before and through-out the Test Period, we plan to be responsive to requests for changes in VEX2 both due to typographic errors in the documentation and problems with the proposal. We will also respond to requests from the community for additional features due to pressing needs. It may be necessary to defer some changes until VEX 2.1 if the issues cannot be resolved easily. We expect that the VEX2.1 update will be incremental and require significantly less time to specify than VEX2.0 has.

We chose to defer implementing several items that had already been requested by the community for the sake of speeding up the process of getting VEX2 finished and/or because our feeling was that they needed more development. However, if there is a pressing need for any of the these, we can consider them again during the Test Period subject to the caveat above.

Test Period

This period allows a test implementation of VEX2 to be created as a separate eco-system to verify that it works and get a critical mass of software ready for the Transition Period. The steps are:

  1. Implementation. The sub-steps here may be done in parallel to the extent appropriate:
    • One (or both) of SCHED/SKED implement an option (forked source, command line switch, or other method) to produce VEX2.
    • One (or both) of FS/VLBA stations implement processing of VEX2 as an option (can be selected by version number in file). It is particularly important that this option be thoroughly verified since an error at a station loses data.
    • One (or both) of DiFX/SFXC correlator develop VEX2 reading as an option. Development of this option may be more tolerant of errors since experiments can be re-correlated, but operational limitations may not allow that.
  2. Testing. Schedule writers/stations/correlators cooperate to iron-out issues, culminating in (successful) test observations that cover different writer/station/correlator combinations. There are a minimum of six cases to consider. Testing HOPS might require additional cases.
  3. Finalization. Once this phase ends, the VEX2 specification would be finalized released beyond the test community.

Please note:

  • The process would start with the listed small universe of writers/stations/correlators to get the ball rolling, but others (vie_sched, ALMA, DSN, etc) are of course welcome to join in and are encouraged to do so.
  • Developers need to be aware that they will need to maintain both VEX1 and VEX2 support during the Transition Period, which may go on for an extended period. If developers choose to use two separate sets of software, it may be necessary to make parallel changes in both versions to keep them up to date during the Transition Period.
  • It is a goal of the Test Period to have a large enough set of testers to test VEX2 fully and allow detection and correction of significant shortcomings.
  • Another goal of the Test Period is to bring the initial testers as close to having a working implementation as possible to help make the Transition Period as short as feasible.

Transition Period

This period will see VEX2 starting to be used in the field, with more and more use of VEX2 over time until VEX1 is no longer needed. The key points are:
  • Users who have not implemented VEX2 support during the Test Period will need to do so.
  • VEX writers will put out both VEX1 and VEX2 schedule files. The naming convention for the new VEX2 files is TBD, but .vex2 has been suggested.
  • Stations and correlators can pick-up the schedule version that is appropriate for them.
  • If an experiment requires VEX2 capabilities for at least some stations, it will be necessary for at least those stations and the target correlator to be able to handle the VEX2 format.
  • The Transition Period should be kept as short as possible.
  • It may be necessary to encourage some stragglers to start using VEX2 so we can end the transition.

Once all VEX writers/stations/correlators can handle VEX2, we can consider the transition complete and allow VEX1 support to end. Of course users can continue to support VEX1 internally for their own purposes if they choose.
Topic revision: r3 - 2021-01-07, EdHimwich
This site is powered by FoswikiCopyright © by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding NRAO Public Wiki? Send feedback