Instructions for Extragalactic HI Observing at GBT JHibbard, Aug2006. Updated Jan 2008 (pipeline scripts)

1) Preobserving: get setup file and schedule blocks.

Example files (see attachments at end): Catalogue file for cleo scheduler. Not used by the system at all (optional). reads in catalogues; defines global variables (e.g. hoff), sets up receivers. This file is read in by all other scheduling blocks (s.b.'s) Catalogue of source positions, velocities Catalogue of calibrator sources positions, velocities. Includes both flux calibrators and spectral line calibrators. observing s.b. for sources. Move to source, balance, on/off for some amount of time. observing s.b. for flux calibrator observing s.b. for spectral line calibrator

2) Preobserving: validating schedule blocks

Log into computer that can run astrid (e.g. thalassa, despina, thales, euler, fourier, etc. all at Start up astrid:

Select the option to work offline.
Click on "Edit" tab. Select your project from the drop-down menu. If you have already saved any schedule blocks for this project, the will show up in the left column. Click on the one you want to edit. If you have the files on disk, then click "Import from file", then click on the validate button. When it validates, click "Save to Database". It is now available for use when observing.

Note: you do not validate your catalogue files by reading them in. Instead, you read in a s.b that reads the catalogue and validate that.

3) Optional: Observing remotely.

!!NOTE!! GBT remote observing instructions changed May 2008. I have not propagated these changes into my instructions. I quote from the email sent by Ron Maddalena. See the referenced link.

Change in Remote Observing Instructions

We have recently changed the method that observers should use for remote
observing.  Observers should now use to access the
control room computers.  We are also seeing a rash of observers not
knowing that they need to explicitly kill their VNC servers at the end
of their observing run.  Multiple servers from previous observers puts
excessive strain on our computer resources and noticeably interferes
with the work of the current observers.  

I suggest that all remote observers read for updated
instructions.  Observers should pay special attention to the section on
how to close their VNC session.  By the end of June 2008, VNC will no
longer work with the old instructions and when using

R. Maddalena

(see GbtRemoteObserving)
  • Must have account on an nrao machine to do this.
  • Initially:
    • NOT NEEDED FOR LINUX SYSTEMS?: Install the VPN software on your machine.
    • Install the VNC software on your machine (e.g. chicken of the VNC).
      • Before running vnc the first time:
        despina%mkdir .vnc 
        despina%cp ~jbraatz/.vnc/xstartup  ~/.vnc/.
  • ~20min before your scheduled set-up time:
    • Call the GBT operator (304-456-2346) and let them know you will be observing remotely (you should have ok'd this with Carl Bignell before), which receiver you'll be using.
    • NOT NECESSARY?: Initiate VPN client on your home machine. Connect to e.g.
    • Start up a vnc desktop:
      • ssh into a greenbank computer (e.g. thalassa, despina, thales, euler, fourier, leibniz, planck, all at
        (instructions from GBT page said to type this:
        ssh -N -C -L
        but we didn't need to do this).
      • start up the vnc server:
        despina%vncserver -geometry 1280x1024 -depth 8
        (for Mac 15" running at 1280x854, then
        despina%vncserver -geometry 1200x800 -depth 8
        seems to work better).
      • The first time you do this you will set a password. Write that someplace safe.
      • Make note of the desktop number that it reports.
      • On your home machine:
        • For Linux:
          superglide%vncviewer [machine]
          where [machine] is the machine in gb that you logged into and # is the desktop number that was reported above.
        • For Mac:
          • double click on vnc
          • give hostname (, password and desktop no.
        • This will pop-up a display that shows a desktop on the machine you logged into in greenbank.
    • Within that desktop open an xterm and start-up your cleo tools (more on this below)
    • If you want, you can start up astrid and run it offline to check out your schedule blocks.
  • At your scheduled start-up time:
    • Within the vnc desktop, open an xterm and log into one of the machines that controls the GBT (e.g. titania):
      despina%ssh titania
    • Call the GBT operator (304-456-2346) and ask to be put into the gateway for whatever machine you logged into (e.g., titania).
    • Start up astrid from the xterm on titania:
  • Follow instruction below. Except - only ssh titania for running astrid online. If you want to run astrid offline, or run cleo, use a normal xterm from the machine you logged into (e.g., despina). So...see the next few items. Run those from the vnc desktop.
  • To disconnect: within any xterm in your gb desktop type:
    depina%vncserver -kill :#
    where # is the desktop number

4) Observing:

Go to observer room 30min before your run. Ask Operator to put your receiver in. Start up astrid on observing console, but do not go online till they tell you you can.

Meanwhile, in another window start cleo in another window.

If observing remotely: launch=>utilities/tools=>Talk and Draw

%cleo scheduler
In skyview: 
- select "real time"
- Catalog...
  => Add/select/deselect catalog
     =>Add user catalog
        navigate to your catalog file, double click on it, and click "ok".
  To see your sources, you may have to change the entries in the
  "Intensity" field and press return.

You use the scheduler to decide what to observe, but should be looking at the status window during your observations - keep an eye on the balance, make sure no untoward messages appear, see how long until the next scan.

By now its probably time to take control of the telescope, so go back to the astrid window and take the telescope online with control.

Go to the "Edit" tab. Highlight your first scheduling block (should include an autopeakfocus followed by a ~60sec observation of your flux calibrator - see in attachments). Click on Validate. Save to database. Give it a new name if need be.

Now click on the "Run" tab. Select the scheduling block and press "submit".

For autopeakfocus, you can see the scans by clicking on the "Data Display -1" Tab.

To go back to the Edit/Run pages, click on Observation Managment.

You are observing!! Keep tabs on things with the cleo "status" display. Of particular interest will be the levels for the spectrometer after doing a balance on a source. These should both be around 1.3-1.5 and similar to each other.

4.1) Notes on Observing Sequence and using attached sb's

In general, I'll first submit the file, wait to see the first spectra come up to make sure the spectrometer is working fine, then submit (after first uncommenting the spectral line calibrator I want to observe). This shows me that the system is fully functional (and will provide a calibration check when I reduce the data). Then I'll repeatedly edit (to uncomment the desired source) and submit for the rest of the night (in the attachments I have included as the example of a source schedule block. There is also an example of a 3x3 mapping schedule block). Its a good check to go back at least once to a calibrator - either, or So here is a possible sequence

  • uncomment desired calibrator and AutoPeakFocus command in
  • submit to queue
  • uncomment desired spectral calibrator in (but keep AutoPeak commented out)
  • submit
  • uncomment desired source in (but keep AutoPeak commented out)
  • submit
  • repeat last two steps for about half the night
  • uncomment desired calibrator in [and/or] and submit
  • go back for rest of your sources

Whether and how often to go back to a calibration source, and whether to calibrate on a radio galaxy or a spectral line calibrator depends on what you are trying to do. Include at least one observation of a spectral line calibrator as a test on your calibration. Include more if you want to understand your systematics better.

At L-band I am definitely not worried about the focus after doing it at the start of the night, unless the operator tells me I should. If the observer before you was using L-band, you may not even have to both with focus. But its cheap so do it. I am also generally not worried about pointing (AutoPeak) after I do it at the start of the run, so often will not comment out the AutoPeak functions in the attached and schedule blocks. However, if you have a largish galaxy (including spectral line calibrators), you may want to AutoPeak near it to make sure you are centered - I have seen spectra taken of the same source that show differences in the shape of the line profile that must be due to the receding/approaching side falling in a different region of the beam.

You may also find an apparent absorption profile at a different velocity than your target source. This is probably due to an emission source in the "Off" position. If the profile is not bothering your target spectra, you can leave it as is and get a "free" HI observation of another source. But if you need the real estate, uncomment out the other "OnOff" command to use a different offset (off=ra_offset instead of off=dec_offset).

5) Looking at your data with gbtidl:

(also see example scripts in attachments)

Open an xterm.

ssh to another gb machine (so you don't interfere with the observing machine).

cd to the directory with any idl programs you wish to use (you might want to create a directory for each night you observe and cd there, e.g. mkdir aug11, cd aug11)

Start up gbtidl:

;; basic idl commands for gbtidl
;; JHibbard  Aug2006

;; to work online

;; to work with a specific sdfits dataset

;; see what is in your database

;; get a scan and display it (only shows YY pol by default)

;; spcifically display YY, then XX

;; display a single integration of a single polarization of a scan


;; save the displayed plot to a file (can also do with "print" button
;; on display

;; save data from last manipulated plot to an ascii file

;; display all ints, one after the other

for i=0,29 do begin $
getps,22,plnum=0,int=i & endfor

;; fit a baseline of order 5

getps, 25
boxcar, 20, /decimate
nregion,[50,300,500,800]  ;; could have used "setregion" to set interactively
baseline, nfit=5

;; get statistics for whatever is plotted:

;; accumlate consequent scans from 60-73, boxcar smooth by 20 channels
;; and display. this requires the file to be in your directory or 
;; you will need to specify the path. Note: Instead of specifying nregion, you 
;; could use "setregion" to set it interactively.

.compile <path/>
sumscans, 60, 73, 20

;;; Reducing 9-pt map:

getsigref, 19, 22

;; where 19 is your on-source pointing and ref is your reference pointing. 
;; Do each of these, and accumulate those you want to accumulate. 

;; To get a single scan:

getscan, 19


;; A more complicated example for summing scans w/o RFI:
;; Sum scans [8,10,12,26,28,30]. But scan 25/26 has RFI in ints=20-22, 
;; so individually sum the int outside of that range.

for i=long(8),12,2 do begin $
    getps,i,plnum=0 & accum & $
    getps,i,plnum=1 & accum  & endfor
;;only accumulate ints w/o RFI for scan 25/26
for j=long(0),19 do begin $
    getps,i,plnum=0,int=j & accum & $
    getps,i,plnum=1,int=j & accum & endfor
for j=long(23),29 do begin $
    getps,i,plnum=0,int=j & accum & $
    getps,i,plnum=1,int=j & accum & endfor
for i=long(28),30,2 do begin $
    getps,i,plnum=0 & accum & $
    getps,i,plnum=1 & accum  & endfor

;;This will accumulate XX,YY separately:

for k=0,1 do begin 
   for i=24,29,2 do begin $
       getps,i,plnum=k & accum,k  & endfor 
   for j=long(0),19 do begin $
       getps,i,plnum=k,int=j & accum,k & endfor
   for j=long(23),29 do begin $
       getps,i,plnum=k,int=j & accum,k & endfor
   for i=long(28),30,2 do begin $
       getps,i,plnum=k & accum,k  & endfor
   if k eq 0 then XXspec=dcextract(!g.s[0])
   if k eq 1 then YYspec=dcextract(!g.s[0])

5a) GBTIDL position-switched pipeline

I have written some GBTIDL scripts to automate reduction of on/off HI data reductions. The main files are (one for each night of observing) and The latter calls the following:,,, and Its also useful to use the macro

To use these, do the following:

  • Edit to identify scans with RFI, calibrator scans, source scans, and input/output data. Use rough values for default_region - you will refine these later
  • start up gbtidl and execute the setup file via: "@gbt_setup". This reduces all the data specified in gbt_setup.
  • use to look at each source/calibrator separately to determine best region:
    • GBTIDL ->i=1 & GBTreduce1, source[i] & chan
  • Enter these regions into and run again!

For more than one night, I use the same keepfile, so that all the data are archived together.

6) Flagging:

Copied from GBTIDLBlankFlagIntro

  flagrec, record, bchan=bchan, echan=echan, chans=chans, chanwidth=chanwidth,
        idstring=idstring, /keep

One uses idstring to associate with a rule an identifying string that is typically a reminder of the reason for the flag.


The following example shows how to flag a channel range for a small number of scans and integrations. Note that either the scan parameter or scanrange keyword is required but both can not be used at the same time. Otherwise, if a parameter is not specified, "all" is assumed. So in this example, all polarizations are flagged:

  flag, [18,19,20], int=[1,3], bchan=512, echan=514, idstring="RFI"

Equivalently, using the scanrange keyword:
  flag, scanrange=[18,20], int=[1,3], bchan=512, echan=514, idstring="RFI"
The next example shows how to flag all channels for a given integration in one scan:
  flag, 15, int=3, idstring="spectrometer glitch"
The next example shows how one could flag all data for the given three scans:
  flag, [101,105,107]
The next example shows how one might flag a record in a processed data file (a "keep" file):
  flagrec, 15, idstring="Glitch", /keep
The next example shows how mone might flag different channel ranges for a record:
   flagrec, 16, bchan=[0,100], echan=[10,110], idstring="Two RFI Spikes"
The next example flags the same channel ranges:
   flagrec, 16, chans=[5,105], chanwidth=11, idstring="Two RFI Spikes"
Flagging a few more scans, not in a nice sequence:
   flag, [40,42,44,47,48,50,56], int=[1,3], bchan=512, echan=514, idstring="More RFI"

-- JohnHibbard - 22 Aug 2006

Topic attachments
I Attachment Action Size Date Who Comment
2PIGG.catcat manage 4 K 2006-08-23 - 09:25 JohnHibbard source catalogue file for astrid (read in by setup file)
2PIGG.sbsb manage 2 K 2009-03-13 - 14:48 JohnHibbard example xgal HI source schedule block for astrid
9pmap2PIGG.sbsb manage 2 K 2009-03-13 - 14:48 JohnHibbard example GBT xgal HI source schedule block for astrid - 3x3 map manage 1 K 2009-03-13 - 14:50 JohnHibbard setup file for GBT HI extragalactic observations with ACS
GBTreduce.propro manage 8 K 2008-01-23 - 17:26 JohnHibbard master file for reducing an entire GBT HI on/off dataset
GBTreduce1.propro manage 583 bytes 2008-01-23 - 17:27 JohnHibbard utility file to reduce data from one source/calib
accumscans.propro manage 568 bytes 2008-01-23 - 17:27 JohnHibbard accumulate and average scans given by an array
avepointmap.propro manage 1 K 2006-08-23 - 09:41 JohnHibbard gbtidl macro to reduce each pointing of a NxNxn map
calsources.catcat manage 1 K 2009-03-13 - 14:31 JohnHibbard flux and spectral line calibrator catalogue file for astrid (read in by setup file)
gbt_setup.propro manage 4 K 2008-01-23 - 17:30 JohnHibbard setup parameters for use in
peakcal.sbsb manage 721 bytes 2009-03-13 - 14:49 JohnHibbard example GBT xgal HI flux calibrator schedule block for astrid
peakspcal.sbsb manage 774 bytes 2009-03-13 - 14:49 JohnHibbard example GBT xgal HI sp.line calibrator schedule block for astrid
scheduler.catcat manage 11 K 2006-08-23 - 09:22 JohnHibbard example catalogue file for cleo scheduler
spcalflux.propro manage 10 K 2009-03-13 - 14:21 JohnHibbard get fluxes etc for some spectral line calibrators
sumscans.propro manage 485 bytes 2006-08-23 - 09:31 JohnHibbard gbtidl macro to sum both pols, range of scans, smooth, fit baseline
sumscansPol.propro manage 474 bytes 2006-08-23 - 09:40 JohnHibbard gbtidl macro to sum single pol, range of scans, smooth, fit baseline
write_spectra.propro manage 1004 bytes 2008-01-23 - 17:29 JohnHibbard make postscript and ascii files for displayed spectra
This topic: Main > TWikiUsers > JohnHibbard > GBTextragalHI
Topic revision: 2009-03-13, JohnHibbard
This site is powered by FoswikiCopyright © by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding NRAO Public Wiki? Send feedback