How to Build CASA 3rd Party Packages on OS X with MacPorts

Original process by Darrell Schiebel. Documentation, testing, and tweaking by Scott Rankin.

Status: BROKEN

  • Out of sync with current requirements

This page includes instructions for version 12 of the CASA 3rd party packages built using this procedure, which is being tested now.



To Do

  1. Clean up
    • This was rewritten in a rush to get something working. It needs cleanup to be easier to duplicate.

  1. Verify almawvr source is current.

  1. Re-chunk packages to install
    1. General tools - cmake, etc.
    2. 3rd party packages required by casacore
    3. 3rd party packages required by code
    4. 3rd party packages required by gcwrap
    5. 3rd party packages required by asap

  1. Move CASA MacPorts tree out of /opt/local
    • As is, this procedure requires removing NRAO Managed Software in /opt/local.

  1. Remove requirement to build as root.

Done

  1. Put n-*-macports.sh scripts in change control.

Limitations

THIS PROCESS MUST BE RUN END-TO-END.

It does not yet support incremental updates.

  1. This process will break NRAO Managed Software if it is installed.
  2. CMake searches /opt/local for required packages. CASA fails to build correctly if NRAO Managed Software is installed.
    • Possible solutions
      1. Explicitly set CMAKE_FIND_ROOT_PATH to include the entire desired search path on the CMake command line.
        1. the developer's install directory
        2. the CASA 3rd party directory
      2. After testing, add this to all CMakeLists.txt files.
  3. This procedure requires access to an admin account on the OS X system being setup so you can run "sudo su -".
  4. This procedure installs several packages with different versions than those used on EL5. See https://safe.nrao.edu/wiki/bin/view/Software/CASA/CASADevelopmentToolsAndLibraries#CASA_4.2.0. This may cause CASA to behave subtly differently than when building on EL5 or EL6.
  5. This procedure builds several versions of GCC we will not use. This wastes time building code we will not use, and may confuse developers.
  6. This procedure will build CASA using GCC 4.4 from MacPorts. This may cause CASA to behave subtly differently than when building CASA with Apple compilers.

We will remove these limitations in the future as time allows.

Removed Limitations:
  1. This procedure depends on a CASA developer's private Subversion ports repository.
  2. This procedure requires an NRAO UNIX account.
    • Solution: move the patch that had to be copied via scp from Darrell's user account to ports/graphics/pgplot/files, where it had to be copied to. This patch will now be included when checking out the full set of ports.

Preparation

  1. Choose a user to own the installed packages.
    • In this page I use the variable $CASA_ADMIN to name this user.
    • Due to limitations in the qt4 port, CASA_ADMIN must be root, because the port installs files in /Library.

  1. Check for and remove the following files:
    • /Library/LaunchAgents/org.freedesktop.dbus-session.plist
    • /Library/LaunchDaemons/org.freedesktop.dbus-system.plist
    • If present, these files will break the installation of the qt4-mac port.

  1. If /opt/local exists,
    1. Move it (for example, to /opt/X-local, to allow later restoration) or delete it.
    2. Remove /opt/local from all paths.

Install Packages

Xcode

MacPorts requires Xcode. If you develop on OS X 10.8.5 or later, download Xcode from the Apple App Store. Otherwise, search Downloads for Apple Developers for the latest version of Xcode for your version of OS X.

  • OS X 10.7 -> 4.6.3
  • OS X 10.8 -> 5.0.2

After installing Xcode, install the Command Line Tools.

  • On OS X 10.7, 10.9, Launch the Xcode app, then select the menu options Xcode -> Preferences. From the Preferences dialog, select Downloads. On the Downloads tab select Components. On that tab click the Install button to the right of the label "Command Line Tools".
  • On OS X 10.9, run

xcode-select --install

Setup 3rd Party Package Build Environment

As $CASA_ADMIN:

As of this writing, CASA_ADMIN must be root.

In the following,
  • 3rd party packages will be installed in /opt/local/, as this is what MacPorts developers expect. I will try to move this to a versioned directory under /opt/casa/3rd-party/ later.

Setup Shell Environment

Ensure the following is at the end of $CASA_ADMIN ~/.profile

#
# For CASA 3rd party packages.
export CASA_3RD_PARTY_BUILD_SCRIPTS_DIR="$HOME/macports-install-scripts"
export CASA_3RD_PARTY_VERSION="12"
export CASA_ADMIN="$USER"
export CASA_DIR="/opt/local"
#
# Temporary, until I can get MacPorts to build and install in $CASA_DIR/3rd-party/CASA_3RD_PARTY_VERSION.
export CASA_3RD_PARTY_DIR="$CASA_DIR"

export CASA_3RD_PARTY_BUILD_LOG_DIR="${CASA_3RD_PARTY_DIR}/build-logs-${CASA_3RD_PARTY_VERSION}"
mkdir -p "$CASA_3RD_PARTY_BUILD_LOG_DIR"

export PATH="$CASA_3RD_PARTY_DIR/bin:$CASA_3RD_PARTY_BUILD_SCRIPTS_DIR:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/bin:/usr/X11/bin"
umask 0022

Then source ~/.profile

Get Current Scripts

Next, check out scripts that will be used to configure and build MacPorts, and to install packages using MacPorts. The following will checkout the scripts from Subversion into ~/macports-install-scripts, or update a checked out copy if it exists, then add these scripts to the PATH.

if test -e "$CASA_3RD_PARTY_BUILD_SCRIPTS_DIR" && ! test -d "$CASA_3RD_PARTY_BUILD_SCRIPTS_DIR"; then
    echo "ERROR: $CASA_3RD_PARTY_BUILD_SCRIPTS_DIR is not a directory."
    echo "ERROR: Please move $CASA_3RD_PARTY_BUILD_SCRIPTS_DIR and try again."
elif test -d "$CASA_3RD_PARTY_BUILD_SCRIPTS_DIR" && ! test -d "$CASA_3RD_PARTY_BUILD_SCRIPTS_DIR/.svn"; then
    echo "ERROR: $CASA_3RD_PARTY_BUILD_SCRIPTS_DIR does not contain a Subversion repository."
    echo "ERROR: Please move $CASA_3RD_PARTY_BUILD_SCRIPTS_DIR and try again."
else 
     if test -d "$CASA_3RD_PARTY_BUILD_SCRIPTS_DIR/.svn"; then
         cd "$CASA_3RD_PARTY_BUILD_SCRIPTS_DIR" && svn update
    else
        svn co https://svn.cv.nrao.edu/svn/casa-3rd-party/trunk/MacPorts/install-scripts "$CASA_3RD_PARTY_BUILD_SCRIPTS_DIR"
    fi
    chmod u+x $CASA_3RD_PARTY_BUILD_SCRIPTS_DIR/*.sh
    export PATH="$CASA_3RD_PARTY_BUILD_SCRIPTS_DIR:$PATH"
fi

Build and Install 3rd Party Packages

End to End

If you have not build 3rd party packages on this computer before, you can run just one scripts to build and install everything.

build-casa-3rd-party-packages.sh

It will exit at the first failure. If you see any output from this script other than

Building and installing CASA 3rd party packages with (mostly) MacPorts.
[some progress chatter...]
Done building and installing CASA 3rd party packages with (mostly) MacPorts.

see $CASA_3RD_PARTY_BUILD_LOG_DIR/build-casa-3rd-party-packages.log.

Being Rewritten.

Run
1-build-macports.sh

It will exit at the first failure. If you see any output from this script other than

1-build-macports.sh done.

Check $CASA_3RD_PARTY_BUILD_LOG_DIR/1-build-macports.log for errors.

Configiure MacPorts

Run
2-configure-macports.sh

It will exit at the first failure. If you see any output from this script other than

2-configure-macports.sh done.

Check $CASA_3RD_PARTY_BUILD_LOG_DIR/2-configure-macports.log for errors.

Build 3rd Party Packages

Required by casacore and code

Run
3-build-core-macports.sh

It will exit at the first failure. If you see any output from this script other than

3-build-core-macports.sh done.

Check $CASA_3RD_PARTY_BUILD_LOG_DIR/3-build-core-macports.log for errors.

Required by gcwrap and asap

Run
4-build-python-macports.sh

It will exit at the first failure. If you see any output from this script other than

4-build-python-macports.sh done.

Check $CASA_3RD_PARTY_BUILD_LOG_DIR/4-build-python-macports.log for errors.

Check build results.

Run the command "port installed". You should see:

cbt-d11-3:logs root# port installed | wc -l 
     153
cbt-d11-3:logs root# port installed
The following ports are currently installed:
  aatm @0.50_3 (active)
  almawvr @1.1.1_1 (active)
  autoconf @2.69_2 (active)
  automake @1.14.1_0 (active)
  bison @2.7.1_0 (active)
  boost @1.43.0_0+python27 (active)
  boost-jam @3.1.18_0 (active)
  bzip2 @1.0.6_0 (active)
  cairo @1.12.16_2+x11 (active)
  cctools @845_1+llvm33 (active)
  cctools-headers @845_0 (active)
  cfitsio @3.340_0 (active)
  cloog @0.18.0_0 (active)
  cmake @2.8.11.2_0 (active)
  curl @7.35.0_0+ssl (active)
  curl-ca-bundle @7.35.0_0 (active)
  db46 @4.6.21_9 (active)
  db_select @0.1_2 (active)
  dbus @1.6.12_0+startupitem (active)
  dbus-cpp @0.9.0_1 (active)
  dbus-glib @0.100.2_0 (active)
  dbus-python27 @0.84.0_0 (active)
  docbook-xml @5.0_2 (active)
  docbook-xml-4.1.2 @5.0_1 (active)
  docbook-xml-4.2 @5.0_1 (active)
  docbook-xml-4.3 @5.0_1 (active)
  docbook-xml-4.4 @5.0_1 (active)
  docbook-xml-4.5 @5.0_1 (active)
  docbook-xml-5.0 @5.0_1 (active)
  docbook-xsl @1.78.1_0 (active)
  doxygen @1.8.5_3 (active)
  dyld-headers @239.3_0 (active)
  expat @2.1.0_0 (active)
  fftw-3 @3.3.3_6 (active)
  fftw-3-single @3.3.3_6 (active)
  flex @2.5.37_1 (active)
  fontconfig @2.11.0_2 (active)
  fop @1.1_0 (active)
  freetype @2.5.2_1 (active)
  gawk @4.1.0_0 (active)
  gcc46 @4.6.4_3 (active)
  gcc_select @0.1_8 (active)
  gdbm @1.11_0 (active)
  gettext @0.18.3.2_0 (active)
  glib2 @2.38.2_0 (active)
  glpk @4.48_0 (active)
  gmp @5.1.2_0 (active)
  gobject-introspection @1.38.0_1 (active)
  gperf @3.0.4_2 (active)
  gsed @4.2.2_0 (active)
  gsl @1.16_2 (active)
  gtk-doc @1.20_0+python27 (active)
  gzip @1.6_0 (active)
  hdf5-18 @1.8.12_1+cxx (active)
  ipython_select @0.3_1 (active)
  isl @0.12_0 (active)
  itstool @2.0.2_0+python27 (active)
  jpeg @9_1 (active)
  lcms @1.19_5 (active)
  ld64 @136_2+llvm33 (active)
  libedit @20121213-3.0_0 (active)
  libffi @3.0.13_0 (active)
  libgcc @4.8.2_0 (active)
  libiconv @1.14_0 (active)
  libidn @1.26_0 (active)
  libmng @1.0.10_3 (active)
  libmpc @1.0.2_0 (active)
  libpixman @0.32.4_0 (active)
  libpng @1.6.9_0 (active)
  libtool @2.4.2_3 (active)
  libunwind-headers @35.1_1 (active)
  libxml2 @2.9.1_0 (active)
  llvm-3.3 @3.3_1 (active)
  llvm_select @0.2_0 (active)
  m4 @1.4.16_0 (active)
  mpfr @3.1.1-p2_0 (active)
  ncurses @5.9_2 (active)
  netcdf @4.3.0_3+dap+netcdf4 (active)
  nosetests_select @0.1_0 (active)
  openjade @1.3.2_7 (active)
  opensp @1.5.2_2 (active)
  openssl @1.0.1f_0 (active)
  pcre @8.33_1 (active)
  perl5 @5.12.4_0+perl5_16 (active)
  perl5.16 @5.16.1_3 (active)
  pgplot @5.3.1_1+gcc46 (active)
  pkgconfig @0.28_0 (active)
  ppl @1.0_2 (active)
  py27-altgraph @0.11_0 (active)
  py27-backports @1.0_0 (active)
  py27-backports-ssl_match_hostname @3.4.0.2_0 (active)
  py27-cairo @1.10.0_3+x11 (active)
  py27-dateutil @1.5_1 (active)
  py27-foolscap @0.5.1_1 (active)
  py27-gobject @2.28.6_3 (active)
  py27-ipython @0.10.2_0+parallel+scientific (active)
  py27-libxml2 @2.9.1_0 (active)
  py27-macholib @1.6_0 (active)
  py27-matplotlib @1.3.1_4+tkinter (active)
  py27-modulegraph @0.11_0 (active)
  py27-nose @1.3.0_0 (active)
  py27-numpy @1.8.0_2 (active)
  py27-parsing @2.0.1_0 (active)
  py27-py2app @0.8_0 (active)
  py27-pyobjc @2.5.1_0 (active)
  py27-pyobjc-cocoa @2.5.1_0 (active)
  py27-readline @6.2.4.1_0 (active)
  py27-scientific @2.9.2_2 (active)
  py27-setuptools @2.2_0 (active)
  py27-six @1.5.2_0 (active)
  py27-tkinter @2.7.6_0 (active)
  py27-tornado @3.2_0 (active)
  py27-twisted @12.0.0_0 (active)
  py27-tz @2013d_0 (active)
  py27-zopeinterface @3.8.0_1 (active)
  python27 @2.7.6_0 (active)
  python_select @0.3_3 (active)
  qt4-mac @4.8.5_1 (active)
  qwt52 @5.2.3_0+qt4 (active)
  readline @6.2.000_0 (active)
  rpfits @2.20_0+gcc46 (active)
  sqlite3 @3.8.3.1_0 (active)
  swig @2.0.10_0 (active)
  swig-python @2.0.10_0 (active)
  tcl @8.6.1_0+corefoundation+threads (active)
  texinfo @5.2_0 (active)
  tiff @4.0.3_2 (active)
  tk @8.6.1_1+quartz (active)
  wcslib @4.17_1+gcc46 (active)
  xercesc @2.8.0_3 (active)
  xmlcatmgr @2.2_1 (active)
  xorg-bigreqsproto @1.1.2_0 (active)
  xorg-inputproto @2.3_0 (active)
  xorg-kbproto @1.0.6_0 (active)
  xorg-libpthread-stubs @0.3_0 (active)
  xorg-libX11 @1.6.2_0 (active)
  xorg-libXau @1.0.8_0 (active)
  xorg-libxcb @1.9.1_0+python27 (active)
  xorg-libXdmcp @1.1.1_0 (active)
  xorg-libXext @1.3.2_0 (active)
  xorg-renderproto @0.11.1_0 (active)
  xorg-util-macros @1.17.1_0 (active)
  xorg-xcb-proto @1.8_0+python27 (active)
  xorg-xcb-util @0.3.9_0 (active)
  xorg-xcmiscproto @1.2.2_0 (active)
  xorg-xextproto @7.2.99.901_0 (active)
  xorg-xf86bigfontproto @1.2.0_0 (active)
  xorg-xproto @7.0.25_0 (active)
  xorg-xtrans @1.3.2_0 (active)
  xrender @0.9.8_0 (active)
  xz @5.0.5_0 (active)
  zlib @1.2.8_0 (active)

Save Results

Save 3rd party package build results so other developers don't have to go through this pain.

Todo: formalize distribution.

darwinVersion=$(uname -r | awk -F "." '{ print $1 }')
packageVersion="$CASA_3RD_PARTY_VERSION" # Todo: automate tracking.
cd /opt
gnutar cfpvz "/opt/casa-3rd-party-$packageVersion-darwin$darwinVersion-x86_64.tar.gz" local

Test Results

Test results are now in JIRA. See CAS-5903

-- ScottRankin - 2014-02-26
Topic revision: r71 - 2014-05-01, ScottRankin
This site is powered by FoswikiCopyright © by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding NRAO Public Wiki? Send feedback