EDIT ONLY WITH "Edit wiki text"

Binary Release Creation (with make-app script)

Binary releases of CASA are currently provided on both OS X and RedHat. On both of these platforms, the release is created with a script called make-app which can be found at: This page describes how this script works, what pieces it uses to create the release, the command line flags it accepts, and references related scripts.

Description of Operation

This script supports creation of a binary distribution on two very different operating systems by using templates which layout the application. These templates contain subdirectories that correspond to the subdirectories in the application. The template also contains broken symbolic links which indicate files or directories to be pulled in to create the application. The OS X template and Linux template are available from the CASA subversion repository.

The symbolic links within the template provide the pieces of the distribution which the script replaces with the directories or files that the symbolic link references. The commands and references that are embedded in these symbolic links are intended to balance flexibility and specificity. So that different platforms can be supported by the same script despite the variances inherent among the various distribution formats. Here is an example you might see if you checked out one of the templates from subversion:
-bash$ ls -l msuvbin
lrwxrwxrwx 1 user group 20 Dec  9 09:55 msuvbin -> @@ARCH@@/bin/msuvbin
-bash$ 
This indicates that the make-app script should copy a file (or directory) called msuvbin from the bin directory beneath architecture directory within the CASA build workspace to the location of this symbolic link. The link itself could be called something different from the make of the file or directory contained in the link target, but in this case, both the thing to be copied and the place where it is being copied are the same. The goal of this approach is to include the key binaries that make up the distribution as this sort of symbolic link command, and then fill the rest of the distribution by analyzing the explicitly included binaries with ldd (linux) or otool (osx) and pull in the dependencies which these tools indicate. Some care must be exercised, though, because not every dependency can be pulled into the distribution. Some system files are either very portable from the perspective of the APIs that CASA uses and some should not be copied from system to system. Some, like encryption libraries, should not be exported. For this reason, make-app tries to limit the files actually pulled into the distribution.

This section describes the various dangling symbolic links that the make-app script supports.

There are three location tags that are used in the symbolic links in the distribution templates. These are @@ARCH@@ (the workspace architecture directory), @@WS@@ (the workspace root directory), and @@PREFIX@@ (one of the prefix directories, in the GNU autoconf sense, where third-party packages are found). These location tags (referred to as loc in the table below) are used with subdirectory paths (as shown above), with a few functions, or with a search option. These are described in the table below:

Function Description Example
SVN(@@URL@@) checkout subversion URL into symlink name data -> SVN(@@https://svn.cv.nrao.edu/svn/casa-data/distro@@)
@@loc@@/path/to/source copy file or directory from loc tag location VERSION -> @@WS@@/code/VERSION
UNION(@@loc@@/path/one @@loc@@/path/two) copy the contents of multiple directories into symlink as a directory python2.7 -> UNION(@@ARCH@@/python/2.7\ @@PREFIX@@/lib/python2.7)
@@loc@@[the_name] search for a file or directory called the_name beneath loc tag location codecs -> @@PREFIX@@[codecs]

Packaging CASA

With these basic dangling symlink "functions" and the make-app script, it is possible to describe CASA's deployment application for both OS X and Linux. The distribution templates for these to operating systems can be found at in the following locations:

Operating System URL
OS X https://svn.cv.nrao.edu/svn/casa/development_tools/packaging/template/osx/CASA.app
Linux https://svn.cv.nrao.edu/svn/casa/development_tools/packaging/template/linux/casa-distro

For linux, make-app creates an output directory which is ready to be tar'ed up to create the final distribution. For OS X, another script, make-dmg is used to generate the disk image that is distributed to users.

Command Line Options

The make-app script supports a few command line options. Here are some examples that were recently used to create distributions:

Operating System Example
Linux make-app arch=linux_64b version=4.6.0 out=/tmp type=DEV ws=trunk
OS X make-app arch=darwin out=/tmp ws=trunk

However, there are several other command line options that can be used. The following table presents the command line options and parameters which the script accepts.

Command Line Option Description
ws=/path/to/workspace supply the path to the CASA workspace that should be used to create the distribution
arch=architecture-subdir subdirectory beneath workspace which contains the build-products from the CASA build
out=/path/to/output/dir create the release in a subdirectory beneath /path/to/output/dir
url=subversion-url override the subversion URL that is used as a template with subversion-url
-q run quietly
-v run verbosely
-ni don't prompt for whether to continue or not
-ignore-missing-dep continue even if a necessary dependency cannot be found

Details

This script has been used for creating the last few releases for OS X. Each of these releases were made relocatable by changing the fully qualified paths within constituent binaries to relative paths based upon @loader_path. The OS X utility called install_name_tool allows this substitution to be done. The @loader_path suffix causes the OS X loader to find the libraries relative to the binary currently being loaded. This solution avoids using environment variables like DYLD_LIBRARY_PATH to find the dependencies. These loader environment variables are problematic mainly because the affect the load-time behavior of all binaries that are run. For an application like CASA, this may include binaries that are not part of the CASA distribution.

The same problems with environment variable modification to load-time behavior with linux, except the environment variable changes to LD_LIBRARY_PATH. However, up until recently the Linux distributions of CASA have relied on LD_LIBRARY_PATH and LD_PRELOAD (generally even worse than LD_LIBRARY_PATH). Another discussion of this issue can be found at: Unfortunately, the problem is worse for run-of-the-mill Linux than for OpenSolaris. OpenSolaris reserves space at the beginning of binaries which allows the runpath to be set even if a runpath was specified at compile time. With Linux in general, the only runpath space that can be utilized after compile time is the path that was specified at compile time. This is an unfortunate situation, but it is one that we can work with by ensuring that all libraries are resolved by libraries and executables which we control. We use the chrpath Linux utility to control the relative resolution of our Linux binaries using the $ORIGIN loader variable, which is the Linux analog of the OS X @loader_path loader variable.

-- DarrellSchiebel - 2015-12-08
Topic revision: r4 - 2017-01-20, DarrellSchiebel
This site is powered by FoswikiCopyright © by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding NRAO Public Wiki? Send feedback