AIPS Scripts

MANDATORY ::: Make your directory names (dgal) the same as you want your file names to have (up to 6 characters). It is easiest if you set dgal and gal equal to the same 6(?) characters in the setup scripts...

This tarball contains the follwing:

bpfix.pl
ps2jpg.sc
README.TXT
runme.sc
UP2low.sh
send_window.pl
REDUCT2.001
REDUCT1.001
PREIM.001
READIN.001
PLOTS.001
REMOVE.001
OUT2FITS.001
COMBINER.001 / COMBINA.001
N4163BJ1.08K
D181BJ1.08U

The scripts execute in the following order: <Setup file>, READIN, REDUCT1/2, PLOTS, PREIM, COMBINER, REMOVE, OUT2FITS

ALL of them except for the two setup scripts (N4163BJ1.08K and D181BJ1.08U) need to be edited to suit your machine. N4163BJ1.08K and D181BJ1.08U are setup script examples for both the single FREQID and mulitple FREQID cases, respectively.

What the scripts do

READIN--
This is a script that reads in the selected data if required. This reads in and creates the following files: *.CH0, *.LINE, and *.LINCOP. This script requires files containing the flag tables to be preloaded and named accordingly (gal-conf-obs.tab). It also assumes you have run CALRD and loaded the appropriate models.

REDUCT1/2--
This script does the calibration of the data from VLANT through CLCAL and all the usual steps in between.

PLOTS--
Script to output the requested plots. RECOMMENDED::Run this AFTER you are "sure" your calibration is good. But just in case, it deletes old versions first, so re-running causes no harm.

List of output plots:
bandpass table
cl table (phase and amplitude)
amp vs time (phase calibrator) (calibrated and uncalibrated)
phase vs time (phase calibrator) (calibrated and uncalibrated)
real vs imaginary (phase calibrator) (calibrated and uncalibrated)
spectrum (galaxy and phase calibrator) (full uv coverage and inner 5 kilolambda) (phase and real)

PREIM--
This run file does the following tasks: SPLIT, UVLSF, CVEL, and IMAGR for any individual observation. The IMAGR output is just a Natural weighted (robust = 5) dirty (no cleaning) image cube used for checking for bad channels.

You have to enter 8 numbers when it asks for the line free channels... enter 0's if you dont have any more numbers to enter. i.e. if you want the ranges 1-40 and 50-55 when prompted, enter: 1 40 50 55 0 0 0 0

COMBINER--
This procedure is the finale in our AIPS processing. This will combine four observation files (the .CVEL files) into one UV dataset, with the caveat that it will only work if you have one B-J1, one C-J1, one D-J1, and one D-J2 observation files. If you have anything else, you must torturously go through these steps by hand...pity. This procedure uses DBCON to combine the files, UVPLT to make UV coverage plots of the final data set, and then makes two over-sized dirty image cubes with IMAGR (one robust = 5 (Natural weighting) and one robust = 0.5). It will then use SUBIM to cut out the required imsize from the oversized cubes. It does this because we are not allowed to image in AIPS and must export the files to image in Miriad. However, in order to image properly we need to have a beam twice as large as the image....hence the SUBIM task. After all this the final dirty image cube and dirty beam will be output to disk and be ready to use in Miriad. Joy!

BTW...this may take a Myr to run...

COMBINA--
Same as COMBINER except you have to manually enter the AIPS catalog IDs (up to 8) that you want to combine.

REMOVE--
This procedure automatically deletes the unneeded aips files. Run this after you have completed the calibration and combination of the data and are ready to run OUT2FITS. The procedure gives you the option of running OUT2FITS as well.

OUT2FITS--
Self explanatory, really. Outputs all files in your AIPS catalog to a fits file. You should either remove all of the unnecessary files manually or run REMOVE before running this script.

Setting up the scripts:

1) Make sure that the setup scripts are saved into the AIPS/RUN directory with the proper extension (all caps).

- To figure out what extension you need type "ehex <user id #>" in AIPS.

- It should spit back to you a three character (mix of numbers and letters) sequence.

- If it only spits back a two character sequence, then the first character is a 0 (zero).

- For example, ehex 308 spits back "97 " for galaxy NGC4163 (B array our observation),

so the setup script should be saved as N4163BJ1.097

- The non setup scripts (i.e. READIN.*, REDUCT1/2.*, PLOTS.*, etc.) should be copied to the 31DEC08/RUN directory and have a .001 extension

2) Also make sure that the auxillary perl and shell scripts are saved in a directory that is

defined in the scripts themselves

Starting up the scripts:

1) Type "restore 0" into AIPS (thats a zero not an o)

2) Type "compress"

These two tasks reset the variables that AIPS may have stored into memory. AIPS does not have a large memory buffer so it is convenient if not necessary to type these each time you recompile any of the scripts.

3) Now compiles the scripts in this order by typing:

run vlbautil

run reduct1

run readin

run n4163bj1


I find it easier that once you have read in the data, to comment out the READIN call in the setup file so you do not have to type "run readin" anymore for that file...same goes for REDUCT1/2. You will always have to type "run vlbautil" first though...

4) To run the script just type:

n4163bj1


After you have completed the REDUCT1/2 scripts and are happy with the outputs, then you can move onto the run files: PLOTS, PREIM, COMBINER, and OUT2FITS

5) If you need to update the setup script you must recompile it in aips by typing "run <file>",

but you should do the whole restore 0 and compress steps and re-compile all of the files in the

order listed above.

PLOTS Script Readme File

The file name extensions legend is as follows:

p = phase

v = versus

r = real

t = time

s = spectrum

g = galaxy

c = calibrated

u = uncalibrated

i = imaginary or inner 5 kilolambda (it should be obvious which one...)

f = full uv range

Examples:

rvipc = real vs imaginary phase (calibrator) calibrated

psgf = phase spectrum galaxy full uv range

-- StevenWarren - 2009-02-17

If you want an script that combines only specific CVEL files you can use COMBINA (A for Adrienne). It will prompt you for the catalog numbers of the CVEL files to DBCON together. -- AdrienneStilp - 2009-06-02

Miriad Scripts

Download this tarball and put it somewhere in your (python) path:

The tarball here contains three files:
image.py the script you run
immodules.py supporting functions for image.py
example.param example parameter file with all available parameters commented out
jvm.sh shell script to run supermongo and make a spectrum plot. This must be in your $red/../SCRIPTS directory
jvm.pro idl script to created a jvm-corrected cube. Must be in your idl path somewhere.
readme.txt readme file
miriad2fits.py python script that writes out all miriad files to fits format

In the example.param file (you should rename it for each galaxy - so n4190.bcd50.isubim.1.fits gets n4190.r5.param for example), the galaxy name is the base name before ".isubim.1.fits". So in that case, n4190.bcd50.isubim.1.fits has a base name of "n4190.bcd50". This is what you set galaxy equal to in the parameter file.

To run the script, you need a version of Python on your system. On our linux system at Washington, we have Python 2.5.2. I can't guarantee that the script will work with anything less than that. I don't know about Python 2.6, but some of the things I use are deprecated in the newer Python versions (apparently, according to the documentation).

The packages that I import are os, sys, math, numpy, pylab, csv.

The syntax for running the script is:

python image.py <parameter file> --restart=<keyword>

Available keywords for restarting are:
readin, clean, smooth, mask, reclean, moments, jvm, cd, spect
. If you do restart, the script will re-make all the files starting at that step.

The files made by each step are:
readin *.map
*.beam
clean *.clean
*.restor
*.cc.restor
*.resid.restor
*.restor.linmos
smooth *.restor.convol
*.restor.convol.hanning
mask *.mask
*.region
reclean *.mclean
*.mrestor
*.cc.mrestor
*.resid.mrestor
*.mrestor.linmos
moments *.m0
*.m1
*.m1.mask
*.m2.mask
*.m2
*.m-2
jvm *.spect.ps
*.spect.txt
cd *.m0.cd
spect *.pyspect.txt
*.pyspect.eps

There are some other files made in moments, but they're useless. I should probably even just delete them in that step. I'll consider this. -- AdrienneStilp - 2009-03-31

Tar Script

This script assumes all of the files are in one folder. If they are not you must edit the script to suit your needs or put all of the files into one folder. The final output is: <galaxy>.all.tar.gz and .setup.tar.gz. The .all. file contains all of the data, plots, setup files, .tab files, etc. for a galaxy (excluding raw data). The .setup. file contains just the AIPS and miriad setup scripts and .tab AIPS files for easy reproduction of what is in the .all. file.

To run this script you must make it an executable file by typing:

chmod +x taritup.sh

Then to run the script type:

taritup.sh <galaxy> <ehex #>

The ehex number is just the extension you added to the end of the AIPS run files for that galaxy.

Hint:: I find it easiest to alias taritup.sh in my .bashrc file so that I can execute the script from any folder

Here is a list of the tar files it produces and what is contained in them:

tar file sub-tar file file
<galaxy>.all.tar.gz    
  <galaxy>.aips.plots.tar.gz * -b-*.jpg
* -c-*.jpg
* -d-*.jpg
* -bcd-*.jpg
  <galaxy>.aips.fits.tar.gz * subim*
* lincop*
* basfit*
* newch0*
* tab*
* dbcon*
*<ehex #>
  <galaxy>.aips.log.tar.gz *combiner.txt
*plots.txt
*preim.txt
*readin.txt
*out2fits.txt
*reduct.txt
  <galaxy>.mir.05.tar.gz * 05*.fits
* 05*param
* 05*spect*
* 05*log
* 05*region
  <galaxy>.mir.50.tar.gz * 50*.fits
* 50*param
* 50*spect*
* 50*log
* 50*region
<galaxy>.setup.tar.gz   * tab*
*param
*<ehex #>

Topic attachments
I Attachment Action Size Date Who Comment
COMBINA.001001 COMBINA.001 manage 10 K 2009-06-03 - 17:22 UnknownUser changed output name to -bcd
aipsscripts.tar.gzgz aipsscripts.tar.gz manage 21 K 2009-04-08 - 11:03 UnknownUser newest version
aipsscriptsv2.tar.gzgz aipsscriptsv2.tar.gz manage 21 K 2009-04-08 - 11:11 UnknownUser  
aipsscriptsv3.tar.gzgz aipsscriptsv3.tar.gz manage 21 K 2009-04-09 - 11:37 UnknownUser  
ascriptv10.tar.gzgz ascriptv10.tar.gz manage 21 K 2009-04-30 - 17:13 UnknownUser  
ascriptv11.tar.gzgz ascriptv11.tar.gz manage 21 K 2009-05-07 - 09:53 UnknownUser  
ascriptv12.tar.gzgz ascriptv12.tar.gz manage 21 K 2009-05-07 - 14:34 UnknownUser  
ascriptv13.tar.gzgz ascriptv13.tar.gz manage 22 K 2010-04-16 - 12:39 UnknownUser  
ascriptv14.tar.gzgz ascriptv14.tar.gz manage 23 K 2011-01-21 - 13:31 UnknownUser  
ascriptv4.tar.gzgz ascriptv4.tar.gz manage 21 K 2009-04-13 - 15:37 UnknownUser  
ascriptv5.tar.gzgz ascriptv5.tar.gz manage 21 K 2009-04-14 - 14:54 UnknownUser  
ascriptv6.tar.gzgz ascriptv6.tar.gz manage 21 K 2009-04-16 - 21:33 UnknownUser  
ascriptv7.tar.gzgz ascriptv7.tar.gz manage 21 K 2009-04-21 - 11:48 UnknownUser  
ascriptv8.tar.gzgz ascriptv8.tar.gz manage 21 K 2009-04-22 - 16:25 UnknownUser  
ascriptv9.tar.gzgz ascriptv9.tar.gz manage 21 K 2009-04-23 - 14:35 UnknownUser  
imagingScript.tartar imagingScript.tar manage 30 K 2009-04-01 - 16:03 UnknownUser imaging script v1.1
imagingScript.v1.1.tartar imagingScript.v1.1.tar manage 30 K 2009-04-07 - 18:35 UnknownUser  
imagingScript.v1.2.tartar imagingScript.v1.2.tar manage 30 K 2009-04-27 - 12:29 UnknownUser  
imagingScript.v1.3.tartar imagingScript.v1.3.tar manage 30 K 2009-04-29 - 20:03 UnknownUser  
imagingScript.v1.4.tartar imagingScript.v1.4.tar manage 30 K 2009-05-04 - 13:16 UnknownUser fixed bug in createdFiles to enable proper restarts
imagingScript.v1.5.tartar imagingScript.v1.5.tar manage 30 K 2009-05-04 - 18:58 UnknownUser fixed bug in getNoise
imagingScript.v1.6.tartar imagingScript.v1.6.tar manage 30 K 2009-05-05 - 13:36 UnknownUser added support for 4 lf channel ranges
imagingScript.v2.0.tartar imagingScript.v2.0.tar manage 30 K 2009-05-05 - 19:25 UnknownUser moment map making feature!
imagingScript.v2.1.tartar imagingScript.v2.1.tar manage 40 K 2009-05-17 - 21:59 UnknownUser added jvm, fixed some bugs
imagingScript.v2.10.tartar imagingScript.v2.10.tar manage 60 K 2010-03-18 - 16:19 UnknownUser miriad imaging script v2.10 - fixed column density map bug
imagingScript.v2.11.tartar imagingScript.v2.11.tar manage 60 K 2010-04-13 - 12:13 UnknownUser mom2 maps masked at same level as mom1
imagingScript.v2.12.tartar imagingScript.v2.12.tar manage 60 K 2010-07-27 - 15:46 UnknownUser  
imagingScript.v2.13.tartar imagingScript.v2.13.tar manage 60 K 2010-07-27 - 16:46 UnknownUser fixed getbeamsize bug
imagingScript.v2.14.tartar imagingScript.v2.14.tar manage 60 K 2010-07-28 - 19:55 UnknownUser fixed spect so that yaxis labels are correct
imagingScript.v2.15.tartar imagingScript.v2.15.tar manage 60 K 2011-01-21 - 19:12 UnknownUser added new way of making spectra (spectrum.sh) and script no longer pops up noise plots
imagingScript.v2.2.tartar imagingScript.v2.2.tar manage 40 K 2009-05-18 - 14:09 UnknownUser added check for denom = 0 in jvm
imagingScript.v2.3.tartar imagingScript.v2.3.tar manage 50 K 2009-06-04 - 18:26 UnknownUser changed number of iterations in clean to 50000 (from 5000). added "restart=recover" option. [June 4, 2009]
imagingScript.v2.4.tartar imagingScript.v2.4.tar manage 50 K 2009-06-09 - 17:23 UnknownUser fixed typo in getBeamsize
imagingScript.v2.5.tartar imagingScript.v2.5.tar manage 50 K 2009-06-24 - 14:04 UnknownUser added jvm-corrected cube, ability to specify number of iterations, moment maps created from jvm-corrected cube
imagingScript.v2.6.tartar imagingScript.v2.6.tar manage 50 K 2009-06-24 - 16:08 UnknownUser minor fix
imagingScript.v2.7.tartar imagingScript.v2.7.tar manage 50 K 2009-06-24 - 16:16 UnknownUser  
imagingScript.v2.8.tartar imagingScript.v2.8.tar manage 50 K 2009-06-26 - 16:54 UnknownUser  
imagingScript.v2.9.tartar imagingScript.v2.9.tar manage 60 K 2009-12-15 - 13:19 UnknownUser added restart=cd (column density)
scripts.tar.gzgz scripts.tar.gz manage 21 K 2009-04-08 - 11:01 UnknownUser newest version
taritup.shsh taritup.sh manage 3 K 2009-06-09 - 15:31 UnknownUser  
This topic: Main > TWikiUsers > JuergenOtt > VlaAngst > DataReduction > ScriptInfo
Topic revision: 2011-01-21, AdrienneStilp
This site is powered by FoswikiCopyright © by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding NRAO Public Wiki? Send feedback