EDIT ONLY WITH "Edit wiki text"
Binary Release Creation (with make-app
script)
Binary releases of CASA are currently provided on both OS X and RedHat. On both of these platforms, the release is created with a script called
make-app
which can be found at:
This page describes how this script works, what pieces it uses to create the release, the command line flags it accepts, and references related scripts.
Description of Operation
This script supports creation of a binary distribution on two very different operating systems by using templates which layout the application. These templates contain subdirectories that correspond to the subdirectories in the application. The template also contains
broken symbolic links which indicate files or directories to be pulled in to create the application. The
OS X template and
Linux template are available from the CASA subversion repository.
The symbolic links within the template provide the pieces of the distribution which the script replaces with the directories or files that the symbolic link references. The commands and references that are embedded in these symbolic links are intended to balance flexibility and specificity. So that different platforms can be supported by the same script despite the variances inherent among the various distribution formats. Here is an example you might see if you checked out one of the templates from subversion:
-bash$ ls -l msuvbin
lrwxrwxrwx 1 user group 20 Dec 9 09:55 msuvbin -> @@ARCH@@/bin/msuvbin
-bash$
This indicates that the
make-app
script should copy a file (or directory) called
msuvbin
from the
bin
directory beneath architecture directory within the CASA build workspace to the location of this symbolic link. The link itself could be called something different from the make of the file or directory contained in the link target, but in this case, both the thing to be copied and the place where it is being copied are the same. The goal of this approach is to include the key binaries that make up the distribution as this sort of symbolic link command, and then fill the rest of the distribution by analyzing the explicitly included binaries with
ldd
(linux) or
otool
(osx) and pull in the dependencies which these tools indicate. Some care must be exercised, though, because not every dependency can be pulled into the distribution. Some system files are either very portable from the perspective of the APIs that CASA uses and some should not be copied from system to system. Some, like encryption libraries, should not be exported. For this reason,
make-app
tries to limit the files actually pulled into the distribution.
Dangling Symbolic Links
This section describes the various dangling symbolic links that the
make-app
script supports.
There are three location tags that are used in the symbolic links in the distribution templates. These are
@@ARCH@@
(the workspace architecture directory),
@@WS@@
(the workspace root directory), and
@@PREFIX@@
(one of the prefix directories, in the GNU autoconf sense, where third-party packages are found). These location tags (referred to as
loc
in the table below) are used with subdirectory paths (as shown above), with a few functions, or with a search option. These are described in the table below:
Function |
Description |
Example |
@@loc@@[the_name] |
search for a file or directory called the_name beneath loc tag location |
codecs -> @@PREFIX@@[codecs] |
SVN(@@URL@@) |
checkout subversion URL into symlink name |
data -> SVN(@@https://svn.cv.nrao.edu/svn/casa-data/distro@@) |
UNION(@@loc@@/path/one @@loc@@/path/two) |
copy the contents of multiple directories into symlink as a directory |
python2.7 -> UNION(@@ARCH@@/python/2.7\ @@PREFIX@@/lib/python2.7) |
@@loc@@/path/to/source |
copy file or directory from loc tag location |
VERSION -> @@WS@@/code/VERSION |
Packaging CASA
With these basic dangling symlink "functions" and the
make-app
script, it is possible to describe CASA's deployment application for both OS X and Linux. The distribution templates for these to operating systems can be found at in the following locations:
For linux,
make-app
creates an output directory which is ready to be tar'ed up to create the final distribution. For OS X, another script,
make-dmg
is used to generate the disk image that is distributed to users.
Command Line Options
The
make-app
script supports a few command line options. Here are some examples that were recently used to create distributions:
Operating System |
Example |
Linux |
make-app arch=linux_64b version=4.6.0 out=/tmp type=DEV ws=trunk |
OS X |
make-app arch=darwin out=/tmp ws=trunk |
However, there are several other command line options that can be used. The following table presents the command line options and parameters which the script accepts.
Command Line Option |
Description |
ws=/path/to/workspace |
supply the path to the CASA workspace that should be used to create the distribution |
arch=architecture-subdir |
subdirectory beneath workspace which contains the build-products from the CASA build |
out=/path/to/output/dir |
create the release in a subdirectory beneath /path/to/output/dir |
url=subversion-url |
override the subversion URL that is used as a template with subversion-url |
-q |
run quietly |
-v |
run verbosely |
-ni |
don't prompt for whether to continue or not |
-ignore-missing-dep |
continue even if a necessary dependency cannot be found |
Details
This script has been used for creating the last few releases for OS X. Each of these releases were made relocatable by changing the fully qualified paths within constituent binaries to relative paths based upon
@loader_path
. The OS X utility called
install_name_tool
allows this substitution to be done. The
@loader_path
suffix causes the OS X loader to find the libraries relative to the binary currently being loaded. This solution avoids using environment variables like
DYLD_LIBRARY_PATH
to find the dependencies. These loader environment variables are problematic mainly because the affect the load-time behavior of all binaries that are run. For an application like CASA, this may include binaries that are not part of the CASA distribution.
The same problems with environment variable modification to load-time behavior with linux, except the environment variable changes to
LD_LIBRARY_PATH
. However, up until recently the Linux distributions of CASA have relied on
LD_LIBRARY_PATH
and
LD_PRELOAD
(generally even
worse than
LD_LIBRARY_PATH
). Another discussion of this issue can be found at:
Unfortunately, the problem is worse for run-of-the-mill Linux than for OpenSolaris. OpenSolaris reserves space at the beginning of binaries which allows the runpath to be set
even if a runpath was specified at compile time. With Linux in general, the only runpath space that can be utilized after compile time is the path that was specified
at compile time. This is an unfortunate situation, but it is one that we can work with by ensuring that all libraries are resolved by libraries and executables which we control. We use the
chrpath
Linux utility to control the relative resolution of our Linux binaries using the
$ORIGIN
loader variable, which is the Linux analog of the OS X
@loader_path
loader variable.
--
DarrellSchiebel - 2015-12-08