Purpose

Past efforts for implementation of a tasking environment have ended in functional limitations and technology lock-in. Given modern tasking environments and future environments, boundary classes should be complete implementation of functionality without any direct coupling to a tasking environment. With interface description and support information contained in an unbound representation, binding to a multitude of environments can be generated through simple translators.

Limitations

A consequence of this late binding is users will not have the ability to make modifications to tools / tasks without recompilation of the underlying source code. For example, a user wanting to tweak the way a tool / task performs its operation would need to modify the C++ code for its implementation, recompile (and possibly regenerate bindings).

Technologies

The main tools utilized in the processing of our tasking environment in order to create our bindings are listed in the following sections below.

Xerces

Performs XML parsing. Elaborate (e.g., how is it provided to the developer, how and when is it used).

Xalan

Compiles XSL files into translators. (e.g., how is it provided to the developer, how and when is it used).

CCMTools

Generates local and remote bindings.(e.g., how is it provided to the developer, how and when is it used).

Tasking Abstraction

A tasking abstraction layer will be created within the casa package to allow for a complete general representation of boundary classes.
--++ XML file representation In order to represent a boundary class without binding it to a tasking environment, XML is used to describe the classes interface complete with signature, default values and constraints for interface parameters, and descriptive help information for each of the classes exposed services.

The XML file contains specific tags and attributes to offer a language for representing its interface. These are as follows:
               
<tool name="">...</tool>
               <method name="">...</method>
               <param name="" type="">...</param>
               <value></value>
               <returns type="">...</returns>
               <description>...</description>

C++ tool/task interface

Boundary classes are contained within a casa library that offers a a service or facility. Boundary classes are a self contained, complete representation of some functionality that is being exposed to an external client. Generally, this class is a fascade pattern composed of many casa library classes hidden from the client. In the case of a task, these internal classes are actually boundary classes (tools) utilized to perform some complex script-like operation. %ENDCOOR%

XSL helper generators

Two utilities have been created

Help file converter for XML

To simplify the developers task, a utility has been created to extract information from the DO help files and create an XML file for a boundary class (tool / task) called help2xml. This utility is an XSL translator that takes as input a DO help file and generates as output a boundary class XML file.

C++ tool/task interface stub generator

Once a XML file has been created, this information can be used along with the utilities cxml2header and cxml2stubs to generate the boundary class C++ code. These utilities are also XSL translators taking as input a boundary class XML file and generating both a C++ header file and source file stub. The header file contains an include file for any additional methods and attributes the user wants to include within the boundary class that should not be exposed to external clients.

Binding Generation

The binding of boundary classes is achieved through generators which process a description of the public interface to create the necessary binding, along with logic for defaulting and range checking, to the desired tasking environment.

Functional Details

CCMTools reads the IDL, creates an attributed parse tree, and then traverses the parse tree calling functions defined in a generator implementation class to do the work of creating the bindings. So in general, creating a new generator involves creating the java code with functions that are called for the various IDL constructs, e.g. component, interface, etc. There is significant common functionality which is implemented in the base class, and complete examples from other generators, i.e. local binding and remote CORBA binding.

(1) create a python generator class which extends CppGenerator. Probably, the easiest way to do this is to copy and rename an existing generator which is closest to the binding target, e.g. the CORBA generator should be close for the ACS target.

(2) create initial template files for each of the IDL constructs, or at least the primary ones, i.e. component and interface. If an existing generator implementation was copied in step (1), all of the template files can be copied to a new templates directory which can be named whatever you like. The template directory is determined by a parameter to the CppGenerator constructor which identifies the language, e.g. "CppPython".

(3) tie in the new generator implementation file into the main class (ConsoleCodeGenerator). The binding which is being created is specified on the command line, e.g. c++local, c++remote, c++python. The generator for the target language (the new implementation class created above) is tied in when ConsoleCodeGenerator.createTemplateHandler is called. It is within createTemplateHandler is where we tie in the creation of our new binding generator.

(4) run the generator with the new language binding parameter and check the output. Initially, it will look identical to the generated output for the language we copied it from.

(5) adjust the template files (which we copied) to change the output. The macros are indicated via '%(MACRO_NAME)s', and the string that is substituted is filled in within java. The place where the substitution is happening can be found by searching for the macro name both within the java generator class and the templates directory.

Adjusting the output involves iteration over (4) and (5). Most of the template expansion can be achieved by changing virtual functions within our generator class which correspond to the IDL constructs, e.g. 'data_MComponentDef(...), data_MStructDef(...), etc. However, it is also easy to directly load and fill a template. This is done by retrieveing the template via "template_manager.getTemplate(...)", creating a Map which maps macro names to strings, and then substituting the macros with a call to 'substituteVariables(map)' on the 'Template' object returned by 'getTemplate(...)'. Finally, the 'writeOutput(...)' function within our generator class handles writing the substituted template into output files. By convention, the templates for matched output files (.h/.cc) are collected within a single file and split an output based upon a '<<<<<<<SPLIT>>>>>>>' line. Our generator class controls its own output, though.

Python Binding Generator

(1) evaluate the output of the partially complete python generator that came with ccmtools

(2) study the python binding interface

(3) implement the "homefinder" interface which is used to locate containers (aka homes). This can be implemented outside of the code generator because only one of these exist in the system. This allowed testing the python interface library alone.

(4) having stubbed out the homefinder interface, the python binding generator was adjusted to create a component home. This is as simple of an example as possible. It has only one function, 'create()', which returns a new component

(5) with the experience of generating the component home, the python templates were adjusted to invoke the local component interface which can already be generated. This involved creating conversion functions for the standard C++ types, e.g. std::vector, to/from python types. Creating new python types involved creating & filling a structure with all of the function tables necessary to describe all of the member functions and attributes of the type.

As for time distribution, about 20% of the time was spent learning to use Java & Eclipse, with about 50% going toward each understanding python's API and implementing the strategy for tying the component into python, and the remaining 30% was spent understanding how to make use of the attributed parse tree and properly expand the templates. There is, however, some overlap between the latter two.

Tool Binding

A tool is an encapsulation of an service offered to the user. The DO is a representation of the public interface (boundary class) to a tools functionality, the signature of which consists of mostly simple AIPS++ types (see section on type mappings). Tools are bound to tasking environments through binding generators (one for each environment).

XSL translators

The generators are xsl files which are processed through Xerces/Xalan (publicly licensed and distributed). Currently, we support binding to ACS, IDL, and Python.

  • Latex documentation: casa2tex - generate latex files from XML
  • ACS IDL binding casa2acsidl - ACS based IDL file
  • ACS Component binding casa2acsch - C++ impl header file casa2acscpp - C++ impl source file
  • Python binding casa2acspy - python wrapper with defaults
  • IDL3 binding: casa2idl3 - IDL3 generator
  • CCMTools connector: casa2ccm - ccm tools library connector
  • RDO connector casa2rdo - rdo connector

Task Binding

A task is an encapsulation of multiple calls to existing DOs within the casa sourcetree. A task writer simply implements a CASAtask within the library for which it is interfacing to (in the case that RDOs from multiple libraries are being combined, this would be implemented in either the highest layer library or in a support library called casatask - TBD). The task writer has absolutely no exposure to any binding specific code. The generator creates this binding.

Parameter Handling

Type/Range Checking

Simple type and range checking is done at the binding layer (RDO and above) while context dependent checking is done at the library layer (DO and below).

Interface Type Mappings

Native Types

Native types can be further divided into three groups: primitive types, aggregate types, and container types.

Primitive Types

Basic types supported directly by C++ and mappable to storage within the operating system.

Aggregate Types

Managed heterogeneous collections of types. The main types are String, Complex, DComplex, and Record.

Container Types

Traversable homogeneous collections of types. These include Vector<T> and Array<T>.

Indirect Types

Indirect types are not directly supported by the underlying system, but must be mapped to supported primitive types managed by a wrapper layer. For instance, a Quantity is represented by a String internally.

Non-DO Component Types

Not all components offer a facility to the user. Some simply maintain state that is utilized by other DO components. This state is thus a complex type represented by an aggregation of simple types combined with operations.

Type Mapping Summary

TYPE STAGE XML IDL CORBA C++ CMPT CASA library Python
double   double double double double Double PyFloat
int   int long long int Int PyInt
long   long long long long long long long Int64 PyLong
string   string string char * std::string String PyString
bool   bool boolean bool bool Bool PyBool
complex   complex struct complex complex std::complex<double> DComplex PyComplex
quantity   quantity any any record Quantity PyDict
measure   measure any any record Measure PyDict
record   record any any record Record PyDict
double[]   DoubleVec sequence<double> DoubleVecSeq std::vector<double> Vector<Double> PyList
int[]   IntVec sequence<long> IntVecSeq std::vector<long> Vector<Int> PyList
long[]   LongVec sequence<long long> LongVecSeq std::vector<long long> Vector<Int64> PyList
string[]   StringVec sequence<string> StringVecSeq std::vector<std::string> Vector<String> PyList
bool[]   BoolVec sequence<boolean> BoolVecSeq std::vector<bool> Vector<Bool> PyList
complex[]   ComplexVec sequence<complex> ComplexVecSeq std::vector<std::complex<double>> Vector<DComplex> PyList
Object (e.g., Table)   Object Object Object* Object* Object* PyObject

-- DarrellSchiebel - 15 Sep 2005

IDL record representation

We have chosen to make a basic C++ STL/STD based record available as a primary type which will be eventually be passed through CORBA as an "any" (an untyped block of data). The code generation system will handle the to/from conversion when CORBA support is required, but for our current system (direct binding to Python), very little packaging/ unpackaging will be required. Our record class will be minimal and will deviate as little as possible from the STL library (which has no support for heterogeneous Maps)

XML Layout of tool/task interface

A description may be found at CASAToolLayout.
Topic revision: r1 - 2010-01-21, RemyIndebetouw
This site is powered by FoswikiCopyright © by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding NRAO Public Wiki? Send feedback