Data Description XML Schema

The new method of specifying data descriptions is via XML files -- one file per manager. This document outlines the XML format, and tag attributes. Please note that the DDL XML functionality is only supported under linux. Therefore a limited few systems will continue to use DDL's. The updates have been made such that this is opaque to application programs. (With only two VxWorks and one Solaris Manager left in the system this should be short-lived.)

General Changes

New $YGOR_TELESCOPE/xml Directories

The new DDL XML files are located in a tree located under the $YGOR_TELESCOPE/xml directory, using the form:
  • $YGOR_TELESCOPE/xml/DeviceName/ManagerName.xml

So for example, the weather station which has { Weather, Weather1, Weather2, Weather3 } sub-devices will have the XML files:
  • $YGOR_TELESCOPE/xml/Weather/Weather.xml
  • $YGOR_TELESCOPE/xml/Weather/Weather1.xml
  • $YGOR_TELESCOPE/xml/Weather/Weather2.xml
  • $YGOR_TELESCOPE/xml/Weather/Weather3.xml

Hierarchical Code vs. Flattened XML

Previously data descriptions were performed by writing code. The for nested complex types the code was hierarchical. Internally the Data Description code would 'flatten' the resulting hierarchy. The new XML form does not allow hierarchical data definitions. The Sampler or Parameter must be described by listing the 'flattened' or 'unrolled' fields, as they would appear in memory. Sounds harder than it really is. A quick example illustrating the difference is given below.

To describe the complex data type:
// Example of nested type:
class PVA 
{
    double p;
    double v;
    double a;
};
class Fooob
{
    PVA Az;
    PVA El;
    double time;
};

Using the 'old-style' DDL code for complex type above:
    pva = new DataDescriptionBranch("PVA", "pos, vel, acl");
    pva->addField("pos", "Position", BasicType::Double, BasicUnit::none);
    pva->addField("vel", "Velocity", BasicType::Double, BasicUnit::none);
    pva->addField("acl", "Acceleration", BasicType::Double, BasicUnit::none);
    DataDescriptionBranch *foo;
    foo = new DataDescriptionBranch("Foob", "struct of two pva");
    foo->addField("Az", pva);
    foo->addField("El", pva);
    foo->addField("seconds", "time in seconds", BasicType::double, BasicUnit::Seconds);
    dd = new DataDescription("bar", "A complex parameter example",
                             MyManagerId::barId,
                             foo);
    addDescriptor(dd);

The same data structure definition using XML would look like this:
    <!-- New style definition of complex type, using XML: -->
    <Parameter name="bar" id="15" doc="A complex parameter example">
        <Field name="bar,Az,pos" type="double" units="none" doc="Position" />
        <Field name="bar,Az,vel" type="double" units="none" doc="Velocity" />
        <Field name="bar,Az,acl" type="double" units="none" doc="Acceleration" />
        <Field name="bar,El,pos" type="double" units="none" doc="Position" />
        <Field name="bar,El,vel" type="double" units="none" doc="Velocity" />
        <Field name="bar,El,acl" type="double" units="none" doc="Acceleration" />
        <Field name="bar,time"   type="double" units="Seconds" doc="time in seconds" />
    </Parameter>

  • Note, just as with code-style definitions, the XML field definition order is important.

New Feature: Shared Definitions

One issue in the conversion was a number of DDL codes which performed things like retrieving receiver name and backend names from the ScanCoordinator.conf file. A second and similar issue were the use of SKY_PATH and SWITCHING_SIGNALS macros in the DDL code. Both of these are examples of Parameter definitions which are 'global' or shared by more than a single device.

In order to address the issue of needing to update each-and-every XML file (e.g. when a receiver is added), a new shared feature has been added via the 'externaldef' attribute of Parameter or Sampler tags. In a nutshell, the attribute indicates to the parser Hey, go find the Field definitions of this Parameter in another XML file, specified by the attribute value.

For example, the receiver Parameter in MyManager's XML might appear like this:
    <Parameter name="receiver" id=222 doc="My receiver parameter" externaldef="ScanCoordinator/ScanCoordinator.xml">
    </Parameter>

Note the lack of any <Field> tags, since these are fetched from the file specified by externaldef. The external file is specified relative to the $YGOR_TELESCOPE/xml directory. This has all been tested successfully, however, some other minor limitations apply:
  • Only one level of external referencing is allowed (i.e. ScanCoordinator.xml's receiver Parameter cannot also have an externaldef tag).
  • The Parameter and name attribute must be on the same line in the file.
  • The local Parameter name, id and doc attributes are used, not the referenced ones (i.e. only Fields are imported).
  • If the Parameter is a dynamic array, this must tag must be included locally, it will not be imported from the external definition.
  • There must be distinct Parameter start and end tags on different lines e.g:
      <Parameter>
      </Parameter>
      NOT the compact form: <Parameter/>
  • Commented out Parameter or Sampler tags near an import target may confuse the import parsing mechanism.

Glish Interface Not Supported

For historic reasons, the Glish interface is written such that XML interface is not supported. However, the changes have been made so that if DDL files are available, the interface can still use those files in lew of XML. Since we are dropping Glish support, this has not been addressed.

DDL XML File Outline:

A typical device file will have the following general outline:

   <Device>
      <Manager>
         <Parameters>
            <Parameter>
               <Field />
            </Parameter>
         </Parameters>
         <Samplers>
            <Sampler>
               <Field />
            </Sampler>
         </Samplers>
      </Manager>
   </Device>

The Device Tag <Device>

The <Device> tag is the 'root' of the XML and may only occur once in the file. It has the following attributes:
Attribute Type Required? Description
id integer yes The major device number of the manager
name string yes The name of the device (e.g. Antenna)

The Manager Tag <Manager>

The <Manager> tag may only occur once in the file. It has the following attributes:
Attribute Type Required? Description
id integer yes The number of the manager (also known as the subdevice number)
name string yes The name of the manager (e.g. Antenna)

The Parameters Tag <Parameters>

The <Parameters> tag may only occur once in the file. It has no attributes, and serves only to group Parameters.

The Parameter Tag <Parameter>

The <Parameter> tag has the following attributes:
Attribute Type Required? Description
id integer yes The Parameter id number
name string yes The name of the parameter (e.g. "asap")
doc string yes A general description of the parameter
dynamic_array bool optional Indicates the presence of a dynamic array type (variable length array)
externaldef string optional An alternate source for the Parameter definition. See 'Shared Definitions' above.

The Field Tag <Field>

The <Field> tag has the following attributes:
Attribute Type Required? Description
type type_string yes The field type string.
name string yes The name of the parameter (e.g. "asap")
doc string yes A general description of the parameter
count integer optional defaults to non-array value
values string optional Enumeration string
precision integer optional Field precision value, defaults to 0
accuracy integer optional Field accuracy value, defaults to 0
units unit_string optional Field units, defaults to "none"
FITSname string optional Alternate column name for FITS files (Deprecated feature?)

  • Where a 'type_string' is one of the type names below:
    • "char",
    • "byte",
    • "short",
    • "int",
    • "long",
    • "float",
    • "double",
    • "longlong",
    • "longDouble",
    • "unsigned short",
    • "unsigned long",
    • "enum",
    • "Bool",
    • "String",
  • Where a 'unit_string' is one of the unit names specified by in ygor/libraries/DataDesc/DataTypes.cc (list too long to include here).

The Samplers Tag <Samplers>

The <Samplers> tag may only occur once in the file. It has no attributes, and serves only to group Samplers.

The Sampler Tag <Sampler>

The <Sampler> tag has the following attributes:
Attribute Type Required? Description
id integer yes The Sampler id number (not including the base id. e.g: 0,1,... not 8902992, 8902993)
name string yes The name of the Sampler (e.g. "ccuData")
doc string yes A general description of the Sampler
externaldef string optional An alternate source for the Sampler definition. See notes below.

Generating XML Files From DDL Files

In order to reduce the impact of changing over to the XML format, a conversion tool is available (showDeviceInfo).

To create a XML file from a DDL library:
  • First verify the directory $YGOR_TELESCOPE/xml/_device_ does not exist. (Otherwise it will attempt to load an XML file.)
  • run the command:
    • showDeviceInfo -xml device manager > /tmp/manager.xml
    • Repeat command above for all the managers in device
    • mkdir $YGOR_TELESCOPE/xml/device
    • mv /tmp/manager.xml $YGOR_TELESCOPE/xml/device

Which DDL Libraries Need to be Maintained

As stated previously, XML encoding is currently only supported on linux. This means that no linux DDL's are required, and can be omitted[1]. However, Solaris based managers which act as clients still need DDL libs for the servers they connect to. For example the Solaris based SpectralProcessor and VxWorks based LO1A/LO1B all connect to the IFManager, therefore Solaris and VxWorks libIFManager.so/.lo files are still required. The SpectralProcessor Solaris DDL file is probably also required for the SpectralProcessor to connect to its children.
  • [1] - Assuming Glish support is not required.

-- JoeBrandt - 2011-12-31
Topic revision: r3 - 2012-01-02, JoeBrandt
This site is powered by FoswikiCopyright © by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding NRAO Public Wiki? Send feedback