Installing DiFX

Quick help

To get usage instructions:

difxbuild -h

To get in-line documentation printed to your terminal:

difxbuild -d

Introduction

This DiFX installation guide is based on the python program difxbuild (which is linked from the bottom of this page).This program allows installation and management of multiple DiFX versions, each on multiple platforms, and associate setup scripts.First some terminology: A version is a official numbered release of DiFX or an unofficial intermediate version drawn from the subversion repository.Examples are the recent DiFX 2.1 release and the head of the development, called trunk.Each version described here has a name.The name for these two versions are DIFX-2.1 and DIFX-DEVEL respectively.An architecture (or arch for short) is a computer type, usually defined by the instruction set of the CPU.Currently difxbuild explicitly knows about two architectures: x86_64 for 64-bit Intel CPUs and i386 for 32-bit Intel CPUs.The architecture of your machine can be determined on the command line with uname -p .Finally a platform is a particular configuration defined both by the architecture and the software installed on it.For example, if multiple Mark5 units with different software development kit (SDK) versions are being used, each would be a different platform .For each DiFX installation there is a primary, or default, platform simply identified by the name of the version .Each additional platform is assigned an additional name (which could be SDK8 and SDK9 for the Mark5 situation) and is identified by concatenating the version name, a hyphen, and the additional name (e.g., DIFX-2.1-SDK8 ).

The installation process has a number of steps, including bootstrapping, source acquisition and autotooling, building (separately on each platform ).Any of these steps can be repeated if needed, however, in many cases it does not make sense to repeat a single step out of order.

Each place where a command is to be typed into the computer, it is displayed after a %

Assumptions

• It is assumed that before the DiFX installation is started that a fully usable Linux operating system is already running and a few bits of software are installed. These include the Intel Integrated Performance Primitives (IPP) library and PGPLOT.

• It is assumed that all machines running DiFX cross mount the same filesystem, that the local name of the cross-mounted filesystem is the same on each machine, and that all parts of the DiFX installation will reside on such a partition.

• It is assumed that during source acquisition steps the machine on which difxbuild is being invoked has access to the internet (specifically http and svn).

Installation Part 1

Part 1 of installation deals with aspects of the installation that are specific to one version but all architectures and platforms .For each new version of DiFX, these steps will be repeated.

Bootstrapping

The bootstrapping step can start from a pristine computer (as long as the above assumptions are met) and will generate a DiFX environment from a rather simple input file.This bootstrap input file consists of a few lines of ASCII text that describe in a minimal manner the intended parameters of the DiFX installation.A complete description of such a file can be found at FIXME.

# difxbuild bootstrap file for trunk version of DiFX

#-------------------------------------
# here are version-specific parameters
#-------------------------------------

# version of DiFX installed by this file
version = DIFX-DEVEL

# the label that will be attached to data
label = <label for working copy>

# svn username environment variable
env SVN_USER=<svn username for working copy>

#--------------------------------------------------------
# below here, all installed versions should look the same
#--------------------------------------------------------

# identify which node should run the core process
headnode = node-1

# top level directory for all DiFX software
difxbase = /home/usno/difx

# location of installed Intel Integrated Performance Primitives
ipproot = /home/usno/intel

# define mark5 alternate architecture
altplatform1     = SDK9
altplatform1arch = i686
altplatform1test = [[ x`pkg-config --modversion streamstor` > "x9.0" ]]
altplatform1host = mark5fx-usno-1

# MPI network selection: restrict which network devices are used
mca = btl_tcp_if_include=p2p1

This file will be called trunk.bootstrap. Note a similar file for DiFX version 2.1 could be made by simply changing version. The user will need to add to the bootstrap file a label for the working copy, and the user will need to add a SVN_USER environment variable containing the svn username to pull the working copy.

Bootstrapping is executed as follows:

% difxbuild -v bootstrap trunk.bootstrap

If this completes successfully, you will be told to source the new setup file:

% . /home/usno/difx/DIFX_DEVEL/setup_difx

Check out source code from subversion

A single command will cause the built-in selection of components to be downloaded from the ATNF subversion repository:

% difxbuild -v svn all

The all parameter here, and in later commands, refers to all components.To see which components this would apply to:

% difxbuild -v list

If the all is excluded, the component corresponding to your current working directory (which would be none at this point) would be selected.Alternately, a list of components can be selected.Each component's source will be put in a separate subdirectory of $DIFX_SRC .

Configure the source trees for out-of-tree building

This step runs the "autotools" on the selected components.To achieve the purpose of supporting multiple platforms , all building is performed out of the source directories, so this step stops short of running configure itself.

% difxbuild -v autotool all

Set this version of DiFX as the default version

If you want this version of DiFX to be the default:

% difxbuild -v default

Note that this step can be performed at any time.Changing to a different default version is done by sourcing the setup_difx script for that version and running this command.

Installation Part 2

Part 2 of the installation deals with installations of architecture dependent code that can work across different versions (and platforms as long as the they are of the same architecture ).Sharing these bits of code across different versions requires that the base directory, as specified in the bootstrapping stage, are the same for each version .Repeat these steps for each architecture by logging into a representative machine of each architecture , sourcing the appropriate setup_difx file, and continuing... Note that spice and pgplot are almost certainly not needed for any of the alternate platforms.

Installing OpenMPI

Most Linux operating systems come with some version of OpenMPI these days, but most won't work for DiFX installations with multiple architectures as a particular configure-time parameter ( --enable-heterogenerous ) is usually not set.To download and install the latest stable version of OpenMPI:

% difxbuild -v openmpi

Installing NASA's spice library (optional)

If you want to support correlation of moving objects, you will need to install the spice library as follows:

% difxbuild -v spice

Installing Caltech's pgplot library (optional)

If you want to build the "sniffer" plotting tools or hops, you need to install the pgplot plotting library:

% difxbuild -v pgplot

Installation Part 3

The 3rd part of installation must be done once for each platform (and always separately for each version , there are no shortcuts here!)This is the actual source code building step.For the non-primary platforms, simply log onto one of the machines representing that platform and be sure to source the appropriate setup_difx file and then proceed.

Build it!

This part is simple, but may take a few minutes:

% difxbuild -v build all

Updating

If for some reason the source code changed and a new compilation across all platforms is desired:

% difxbuild -v pbuild all

where pbuild is short for "parallel build".The build process will simultaneously be started on the local machine and one representative member (as specified in the bootstrap file)of each alternate platform .Like for other commands, all can be excluded if only the current directory needs to be rebuilt, or a specific list of components can be specified.

Installation Part 4

The final step of installation is configuring the account of the user that will run difx. It is assumed here that this account is called oper , that bash is the shell used for this account, and the account for installation was difxmgr . The only remaining steps are to ensure the environment is correctly configured by copying some files from the difxmgr account.

% cp -r ~difxmgr/.openmpi/ ~/

% echo ". /home/usno/difx/bin/setup_difx" > ~/.profile

% ln -s ~/.profile ~/.bashrc

At this point once oper logs back in the account should be ready to run.

-- WalterBrisken - 2012-05-15

• difx building script: difxbuild