Compiling casapy from source in Ubuntu

Note:

Compiling casapy from source is usually only necessary after making modifications to the C++ code. Most users should use the ready-made .debs.

Precise Pangolin Peculiarities Placeholder

They resisted using the obvious "penguin", but went with something that sounds similar.

Oneiric Ocelot Onerousness:

  1. Oneiric comes with boost 1.46, which is incompatible with casapy's alma libraries. Downgrade to 1.42.
  2. Wherever build commands call for python2.6, substitute python2.7 (optional?)
  3. (Not optional): ccmtools will need to be rebuilt to use the right python version:
    1. cd to your ccmtools directory.
    2. rm $CASAARCH/lib/libccmtools*
    3. rm -rf ccmtools cpp-environment java-environment
    4. svn up
    5. Edit GNUMakefile (if necessary):
# Old
PYTHONVER := $(shell python -V 2>&1 | perl -pe "s|^.*?(\d+\.\d+)\.\d+\$$|\$$1|")
# New
PYTHONVER := $(shell python -V 2>&1 | perl -pe "s|^.*?(\d+\.\d+)\..+\$$|\$$1|")

and right after ifeq "$(os)" "linux":
# Old
$(C++) -shared -Wl,-soname,$(notdir $@) -o $@ $(filter %.o,$^)
# New
$(C++) -shared -Wl,-soname,$(notdir $@) -o $@ $(filter %.o,$^) -L$(PYTHONLIBD) -lpython$(PYTHONVER)

    1. Run the sudo make command, and ls -l /usr/local/lib/libccmtools_local.so* to make sure the libs were updated.

Natty Narwhal News:

(or less alliteratively, things you need to do with an existing build after updating Ubuntu to 11.04.)

  1. "make[2]: * No rule to make target `/usr/lib/libm.so'": A fairly common cmake problem with 11.04, the solution is rm CMakeCache.txt in both your code and casacore build directories. Yes, I've switched to using cmake for casacore, which is probably a bigger reason to update the notes.
  2. sudo ln -s /usr/lib/x86_64-linux-gnu/dbus-1.0/include/dbus/dbus-arch-deps.h /usr/include/dbus-1.0/dbus/dbus-arch-deps.h More elegant hacks/explanations welcome.
  3. Make sure casacore is >= 15121 and code is >= 15132.
  4. Rebuild both casacore and code.

Disclaimer

These are the notes I kept and refined while installing casapy on
  1. a 32 bit home computer, which did not begin in a "vanilla" state matching any official Ubuntu release. There were many false starts (and lots of help from Jonas and Darrell), and an upgrade of Ubuntu from 9.10 to 10.4 beta in the middle, and then
  2. a 64 bit i3 laptop, using a comparatively fresh install of Ubuntu 10.4rc.

I make no guarantees about listing all the dependencies here, or only the minimum required set, but any corrections are welcome.

Also, I frequently used sudo. Installing everything as a regular user is a different challenge.

Claimer

I used already-installed packages where possible, then available .debs, and then compilation from source. Whenever possible I avoided using alien to convert the supplied casapy .rpms into .debs, or use quasi-duplicate versions of standard software in /usr/lib/casapy. The big one is python - I use the python 2.6 that came with Ubuntu, not the python 2.5 normally included with casapy.

Works with 10.10 too. You may have to svn up, though.

Install dependencies

Adjust version numbers as necessary, but be aware that newer versions aren't always compatible, esp. for xerces (use 2, not 3) and boost. I use libboost-all-dev here, which resolves to the distribution's currently recommended version. For Ubuntu 10.4-prerelease that was 1.40. Speaking of libboost-all-dev, it is a rather large metapackage, and if you wish you could try figuring out which of the component packages CASA really needs (most, I think).
sudo apt-get install ant antlr libantlr-dev libantlr-java pgplot5 libxml2-dev \
libcfitsio3-dev libdbus-1-dev libdbus-c++-dev tix tix-dev libxerces-c2-dev qt4-dev-tools \
libqwt5-qt4-dev libjdom1-java libjfreechart-java libblas-dev scons cmake \
libhdf5-serial-dev openjdk-6-jdk flex liblapack-dev libfftw3-dev libreadline-dev \
libboost-all-dev python-foolscap xvfb subversion bison gfortran \
python-matplotlib ipython python-nose libxslt1-dev

(Thanks to Stephen Bourke for catching a few packages that I had missed because they were already installed on my computers.)

Preliminaries

# Where you want to keep your casapy, casacore, wcslib, etc. code and builds...
export CASAROOT=$HOME/casa/active  # Just in case (unlikely) you eventually want to build something besides active.
mkdir -p $CASAROOT

# Don't use ARCH - there might still be scripts lying around that pick it up.
MY_ARCH=linux_64b   # linux_gnu for 32 bits.
export CASAPATH="$CASAROOT $MY_ARCH whatever whocares"
export CASAARCH=$CASAROOT/$MY_ARCH 

# How many parallel make threads do you want to try?
NMAKE=3

WCSLib


Install wcslib from source. The default install location (/usr/local) is fine as long as you don't mind sudoing the install, but you probably want to untar and build the code in your CASA build directory, so make one now:
cd $CASAROOT
wget ftp://ftp.atnf.csiro.au/pub/software/wcslib/wcslib.tar.bz2   # The WCSLib homepage is beautiful, but it's hard to find the actual download link.
tar xfj wcslib.tar.bz2
cd wcslib-4.4.4          # Your version may of course vary.

64 bit only:

Edit makedefs and add -fPIC to CFLAGS and FFLAGS. See this Gentoo page, section 3.g for a discussion of this little difference between 32 and 64 bits.

Both architectures:

make

# Recommended.  If nothing else it's trippy.
make check

# For the paranoid: make -n install
sudo make install
sudo ln -s /usr/local/include/wcslib-4.3 /usr/local/include/wcslib

CCMTools:

Using the NRAO version is mandatory (and easier!) since Darrell has modified it:
cd $CASAROOT
svn checkout https://svn.cv.nrao.edu/svn/ccmtools/trunk ccmtools
cd ccmtools

# Since separating install_* from compilation looks like a bit of a pain, check that Darrell hasn't snuck anything in there ;-)
make -n INSTDIR=/usr/local DESTROOT=/usr/local PYTHONINC=/usr/include/python2.6

# Now the real thing:
sudo make -j $NMAKE INSTDIR=/usr/local DESTROOT=/usr/local PYTHONINC=/usr/include/python2.6

# Did it work?
ls -l /usr/local/lib/libccmtools_local.*
# You should see something like:
# lrwxrwxrwx 1 root root     26 2010-04-20 21:59 /usr/local/lib/libccmtools_local.so -> libccmtools_local.so.0.5.5
# -rwxr-xr-x 1 root root 359727 2010-04-20 21:59 /usr/local/lib/libccmtools_local.so.0.5.5

AATM

Use a newer repository than what's listed here, and avoid having to modify the code!
cd $CASAROOT
svn checkout https://svn.cv.nrao.edu/svn/casa/active/aatm aatm
cd aatm

# NOTE OPTIONS!
./configure --with-boost-program-options=boost_program_options-mt

make
sudo make install

RPFITS.h (dialect of FITS for ATCA data)

cd $CASAROOT
wget ftp://ftp.atnf.csiro.au/pub/software/rpfits/rpfits-2.19.tar.gz
tar xfz rpfits-2.19.tar.gz
cd rpfits
cd linux # (or linux64)

Edit GNUMakefile: comment out the top set of flags, from FC to RANLIB, since gfortran will be used instead of g77. Uncomment the lower FC and FFLAGS. Leave LDFLAGS commented! Comment out the -Wimplicit to suppress a compiler warning. You will probably still see some warnings about dummy variables.

64 bit only:

Add -fPIC to CFLAGS and FFLAGS. (Same reason as for WCSLib.)

Both architectures:

make -j $NMAKE
sudo make install 

Minimum data repository

cd $CASAROOT
mkdir -p ./data
cd data
svn checkout https://svn.cv.nrao.edu/svn/casa-data/distro data

casacore

cd ..
svn checkout https://svn.cv.nrao.edu/svn/casa/active/casacore core
cd core

# Note the cfitsio includes are in /usr/include, not /usr/include/cfitsio as in RHEL5.

# Pick one:
# 32 bits
scons -j $NMAKE --prefix=$CASAARCH --extra-cppflags="-O2 -g -DCASA_USECASAPATH -DCASACORE_NEEDS_RETHROW" --libdir=$CASAARCH/lib --data-dir=$CASAROOT/data --cfitsio-libdir=/usr/lib --cfitsio-incdir=/usr/include --extra-cflags="-O -fno-omit-frame-pointer" --extra-fflags="-O  -fno-omit-frame-pointer" --extra-libs="-lfftw3f_threads -lfftw3_threads -lfftw3f -lfftw3 -lgomp" --enable-shared --disable-static && scons install |tee ../build.log

# 64 bits
#scons -j $NMAKE --prefix=$CASAARCH --extra-cppflags="-O2 -g -DCASA_USECASAPATH -DCASACORE_NEEDS_RETHROW -DAIPS64_B" --libdir=$CASAARCH/lib --data-dir=$CASAROOT/data --cfitsio-libdir=/usr/lib --cfitsio-incdir=/usr/include --extra-cflags="-O -fno-omit-frame-pointer" --extra-fflags="-O  -fno-omit-frame-pointer" --extra-libs="-lfftw3f_threads -lfftw3_threads -lfftw3f -lfftw3 -lgomp" --enable-shared --disable-static  && scons install |tee ../build.log

# Now that your options are established, in future runs you can just do "scons install (|tee ../build.log)".
cd ..

export PATH=$PATH:$CASAARCH/bin

Note:

I have not actually used HDF5 with casacore, because of the zoo of HDF5 and MPI choices in Ubuntu (and Debian too, I think). The apt-get line above settled for the serial version of HDF5, which does not produce any complaints from scons and is the right choice if you do not want a multithreaded HDF5. If you manage to find the right combination of HDF5 and MPI packages, especially ones that work with a cmake build of casacore (not covered here), please let me know. I ended up uninstalling HDF5.

Peter Teuben (acting on a suggestion from Honglin Ye), found that commenting out the include (FindHDF5) line in $CASAROOT/casacore/CMakeLists.txt (and/or CMakeLists.txt.gcc44) stops cmake from looking for HDF5. Which brings us to our next overdue update...

Compiling casacore with cmake instead of scons

I think most people (including me) use cmake instead of scons for compiling casacore now. scons still (last I checked) works with casacore, but I can at least show you what I use (on a 64 bit machine):
cd $CASAROOT
mkdir -p builds/casacore/dbg  # First time only
cd builds/casacore/dbg
cmake -DCMAKE_INSTALL_PREFIX=$CASAARCH -DCBLAS_ROOT_DIR=/usr -DCLAPACK_ROOT_DIR=/usr -DCFITSIO_ROOT_DIR=/usr -DHDF5_ROOT_DIR=/opt -DFFTW3_ROOT_DIR=/usr -DDATA_DIR=$CASAROOT/data -DCMAKE_CPP_FLAGS="-O3 -g -DCASA_USECASAPATH -DCASACORE_NEEDS_RETHROW -DAIPS64_B" ../../../core
make -j3 && make install

Note that I do not have HDF5 installed in /opt, /usr, or anywhere else on this computer (see above).

The googlecode version of casacore has a page for compiling casacore with cmake. Note that with this method the build directory is separate from the source directory (which is good). Unlike code/ (below), a make install is necessary because code/ needs the casacore headers and libraries to be installed before it can be compiled.

casa

Help it find xerces:

xerces.jar seems to be called xercesImpl.jar in Ubuntu. Either
  • change xerces.jar to xercesImpl.jar in the CLASSPATH of /usr/local/bin/ccmtools (it's a bash script), or
  • sudo ln -s /usr/share/java/xercesImpl.jar /usr/share/java/xerces.jar

Finally the main event:

Use the standard active repo here too:
svn checkout https://svn.cv.nrao.edu/svn/casa/active/code code

# Does cmake not look in /usr/local?!
sudo ln -s /usr/local/include/wcslib-4.3 /usr/include/wcslib
sudo ln -s /usr/lib/libqwt-qt4.so /usr/local/lib/libqwt.so

# (Slightly experimental) Let's build outside code/ so that the build directories stay off svn's radar.
mkdir builds
cd builds

# You can have multiple build directories with multiple compiler settings (i.e. optimized, debug, profiling),
# but the libraries and executables all get installed in $CASAARCH, overwriting whatever was there before.
mkdir debug
cd debug

cmake -DCMAKE_BUILD_TYPE=Debug $CASAROOT/code
make -j $NMAKE

cd $CASAROOT
. ./casainit.sh

rehash # necessary?
which casapy  # Does it show what you just built?

Make and cd to a test directory, and start up casapy!

At present you need to supply the path to pymodules, i.e.:

casapy --path /usr/lib/pymodules/python2.6

(The logger font will probably be in italics. Darrell thinks it's a pgplot configuration issue, but it's not urgent enough for us to have looked into.)

-- RobReid - 2010-04-11
Topic revision: r26 - 2011-10-26, RobReid
This site is powered by FoswikiCopyright © by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding NRAO Public Wiki? Send feedback