HERA Librarian software

Instructions on configuring the hardware can be found at https://staff.nrao.edu/wiki/bin/view/CIS/Documentation/Herastore01


Installation

The first few steps are run as root, the rest are run as the here user heramgr


As Root

Make accounts for testing

echo "heramgr:x:101:101:test hera mgr:/export/home/herastore01/users/heramgr:/bin/bash" >> /etc/passwd
echo 'heramgr:<HASH>:17263:0:99999:7:::' >> /etc/shadow
mkdir -p /export/home/herastore01/users/heramgr
chown heramgr /export/home/herastore01/users/heramgr
chown heramgr /export/hera/herastore01-1
chown heramgr /export/hera/herastore01-2
chown heramgr /export/hera/herastore01-3
chown heramgr /export/hera/herastore01-4

Install Librarian

mkdir -p /opt/services
cd /export/hera/herastore01-1
git clone https://github.com/HERA-Team/librarian.git
chown -R heramgr /export/hera/herastore01-1/librarian
mkdir /export/hera/herastore01-1/aipy
chown -R heramgr /export/hera/herastore01-1/aipy

Configure SSH Host Keys

Create ssh host keys so one can ssh to localhost without passwd challenge
echo "`hostname`" >> /etc/ssh/shosts.equiv
echo "`hostname`.`dnsdomainname`" >> /etc/ssh/shosts.equiv
echo "heralib" >> /etc/ssh/shosts.equiv
echo "heralib.`dnsdomainname`" >> /etc/ssh/shosts.equiv
echo "HostbasedAuthentication yes" >> /etc/ssh/sshd_config
echo "HostbasedAuthentication yes" >> /etc/ssh/ssh_config
echo "EnableSSHKeysign        yes" >> /etc/ssh/ssh_config
echo "`hostname`.`dnsdomainname` " >> /etc/ssh/ssh_known_hosts
cat /etc/ssh/ssh_host_rsa_key.pub >> /etc/ssh/ssh_known_hosts
/etc/init.d/sshd restart

Make librarian-local

mkdir -p /export/hera/herastore01-1/librarian-local
chown -R heramgr /export/hera/herastore01-1/librarian-local

Initialize Postgresql

Make directory for postgres data
mkdir -p /export/hera/herastore01-1/librarian-local/pgsql/data
chown -R postgres /export/hera/herastore01-1/librarian-local/pgsql
Create sysconfig file to set location of postgres data
echo "PGDATA=/export/hera/herastore01-1/librarian-local/pgsql/data" >> /etc/sysconfig/pgsql/postgresql
echo "PGLOG=/export/hera/herastore01-1/librarian-local/pgsql/pgstartup.log" >> /etc/sysconfig/pgsql/postgresql
Start postgres and create user
chkconfig --level 345 postgresql on
/etc/init.d/postgresql initdb
/etc/init.d/postgresql start
su -c "createuser -d heramgr" postgres

Install Startup Scripts

Save the attached nrao-hera-librarian to /export/hera/herastore01-1/librarian-local

Save the attached hera-librarian to /export/hera/herastore01-1/librarian-local

ln -s /export/hera/herastore01-1/librarian-local/nrao-hera-librarian /etc/init.d
chkconfig --add nrao-hera-librarian


As Heramgr

Setup virtuaenv

virtualenv /export/hera/herastore01-1/librarian-local/python-env
source /export/hera/herastore01-1/librarian-local/python-env/bin/activate

Configure librarian

cd /export/hera/herastore01-1/librarian
python setup.py install
Save the attached server-config.json to /export/hera/herastore01-1/librarian/server

Install dependancies

I was unable to install basemap (a.k.a. matplotlib-basemap).
pip install flask-sqlalchemy numpy astropy tornado alembic pyephem pyfits matplotlib psycopg2

Install aipy

cd /export/hera/herastore01-1
git clone https://github.com/HERA-Team/aipy.git
cd aipy
python setup.py install

Create Database Schema

createdb librarian
cd /export/hera/herastore01-1/librarian/server
alembic upgrade head

Start Server

cd /export/hera/herastore01-1/librarian/server
./runserver.py   

Connect

http://heralib.aoc.nrao.edu:21106


To Do/Issues

  • I was unable to install basemap (a.k.a. matplotlib-basemap). Does it matter?
  • Ask helpdesk to allow port 21106 from the Internet. Using 21106 instead of 80 provides a bit of security-through-obscurity and allows the software stack to run as a non-root user.
  • Ask helpdesk to allow ssh from 196.24.41.0/24 (south africa) so data can rsync. What account is it going to use?
  • These instructions currently don't have a password for postgresql access. Do we want one?
  • What are sources in the server-config.json file?
  • How do I set up multiple stores? The software seems to just add whatever is last in the json file.


References

  • the README.md files in /opt/services/librarian and /opt/services/librarian/server


Topic revision: r2 - 2017-04-25, KScottRowe
This site is powered by FoswikiCopyright © by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding NRAO Public Wiki? Send feedback