Observing Instructions for VEGAS Pulsar Modes

VEGAS Pulsar Modes will be offered for shared risk observing starting with the 17A semester. Only the 800 and 1500 MHz bandwidth modes (using both coherent and incoherent dedispersion) will be available at this time. The low bandwidth (200 and 100 MHz) incoherent and coherent dedispersion modes are still being developed and commissioned. Dual backend operation with GUPPI is supported for for those modes which are shared between VEGAS and GUPPI. We have endeavored to make observations with VEGAS similar to GUPPI from the user perspective in order to simplify the transition to VEGAS.

These instructions will be updated as final commissioning is completed. Observers should consult this wiki before their observations to ensure that they are familiar with the current observing practices.

If you have any questions about VEGAS pulsar modes, please contact Ryan Lynch (rlynch@nraonospam.example.edu)

Overview of VEGAS Pulsar Modes

The VErsatile Green Bank Astronomical Spectrometer, or VEGAS, is a flexible digital backend built around CASPER ROACH2 FPGAs and GPU/CPU high performance computers. VEGAS was originally commissioned for GBT spectral line observations and is currently being deployed for pulsar observations. VEGAS pulsar modes will offer all the same capabilities as GUPPI but with greater instantaneous bandwidth. VEGAS consists of eight independent spectrometer "banks" (consisting of a ROACH2 + HPC) that can each process a bandwidth of up to 1.5 GHz. Filters in the GBT IF chain limit the usuable bandwidth to 1.25 GHz per spectrometer. 800 MHz bandwidth modes are also available, and 100 and 200 MHz modes are currently under development. VEGAS pulsar modes support coherent and incoherent dedispersion in online-folding and pulsar search modes (as well as calibration modes).

In incoherent dedispersion modes each VEGAS bank (i.e. ROACH2 + HPC) can be centered at an independent rest frequency. If each spectrometer band is spaced contiguously this offers up to 8x1.25 GHz = 10 GHz of useable instantaneous bandwidth, though not all receivers actually offer this much bandwidth.

In coherent dedispersion mode, a single ROACH2 sends data to all eight VEGAS HPCs, where the coherent dedispersion is accomplished. This limits the useable instantaneous bandwidth in coherent dedispersion modes to 1.25 GHz or less.

Configuring Astrid for VEGAS Pulsar Modes

VEGAS pulsar modes can be selected by specifying the necessary keywords in an Astrid configuration block. We've made every attempt to use similar keywords for GUPPI and VEGAS. Below are some example configuration blocks for different VEGAS observing modes.

VEGAS Single-Bank Operation

In sinlge-bank mode the VEGAS configuration block is almost identical to GUPPI, except we use vegas.* keywords instead of guppi.*. However, VEGAS has no equivalent to guppi.datadisk.
config_vpm_single= """
obstype = 'Pulsar'
# 'receiver' can be any GBT receiver except MUSTANG. Here, we use Rcvr1_2, aka L-Band
receiver = 'Rcvr1_2'
restfreq = 1500.0
nwin = 1
pol = 'Linear'
backend = 'VEGAS'
bandwidth = 1500.0
# tint is highly flexible but subject to data rate limits. The true tint will be rounded to the nearest value acc_len * nchan / bw
# where acc_len is an integer that controls the hardware accumulation length
tint = 40.96e-6
deltafreq = 0.0
swmode = 'tp_nocal' # Choose 'tp' for calibration, 'tp_nocal' for pulsar observation
swtype = 'none'
swper = 0.04
swfreq = 0
noisecal = 'off' # Choose 'lo' for calibration, 'off' for pulsar observation
vlow = 0.0
vhigh = 0.0
vframe = 'topo'
vdef = 'Radio'
# The following keywords are VEGAS specific
vegas.obsmode = 'search' # search, cal, fold, coherent_search, coherent_cal, or coherent_fold
vegas.polnmode = 'total_intensity' # Can be full_stokes or total_intensity for search mode. All other modes require full_stokes
vegas.numchan = 2048 # Power of 2 between 64 and 8192. Data loss may occur in coherent modes for >= 4096 channels.
vegas.scale = 100.0
vegas.outbits = 8
# These parameters are only used in cal, fold, coherent_cal, or coherent_fold modes
vegas.fold_bins = 256
vegas.fold_dumptime = 10.0
vegas.fold_parfile = '/home/gpu/tzpar/B1937+21.par'
# vegas.dm only used in coherent_search mode
vegas.dm = 0.0
"""

VEGAS Multi-Bank Operation

In dual-bank mode it is easiest to configure each spectrometer bank with a different rest frequency using the restfreq dictionary format. The eight banks are labeled A-H. More information can be found in the GBT Observer's Guide. A simple example is provided below that uses two banks (A and B) covering 3 GHz of bandwidth at C-Band. Note that the dopplertrackfreq keyword is required when using the restfreq dictonary.
config_vpm_multi= """
obstype = 'Pulsar'
# 'receiver' can be any GBT receiver except MUSTANG. Here, we use Rcvr4_6, aka C-Band
receiver = 'Rcvr4_6'
restfreq = [{'bank': 'A', 'restfreq': 4850.0}, {'bank': 'B', 'restfreq': 6350.0}] # Use the restfreq dictionary format to configure multiple banks
dopplertrackfreq = 5600.0 # Required even without doppler tracking
nwin = 2 # Must match number of entries in restfreq dictionary
pol = 'Linear'
backend = 'VEGAS'
bandwidth = 1500.0 # Bandwidth per spectrometer bank. In this case the total bandwidth is larger.
# tint is highly flexible but subject to data rate limits. The true tint will be rounded to the nearest value acc_len * nchan / bw
# where acc_len is an integer that controls the hardware accumulation length
tint = 40.96e-6
deltafreq = 0.0
swmode = 'tp_nocal' # Choose 'tp' for calibration, 'tp_nocal' for pulsar observation
swtype = 'none'
swper = 0.04
swfreq = 0
noisecal = 'off' # Choose 'lo' for calibration, 'off' for pulsar observation
vlow = 0.0
vhigh = 0.0
vframe = 'topo'
vdef = 'Radio'
# The following keywords are VEGAS specific
vegas.obsmode = 'search' # search, cal, fold. Coherent modes are not currently supported for dual-bank operation.
vegas.polnmode = 'total_intensity' # Can be full_stokes or total_intensity for search mode. All other modes require full_stokes
vegas.numchan = 2048 # Power of 2 between 64 and 8192
vegas.scale = 100.0
vegas.outbits = 8
# These parameters are only used in cal, fold
vegas.fold_bins = 256
vegas.fold_dumptime = 10.0
vegas.fold_parfile = '/home/gpu/tzpar/B1937+21.par'
"""

VEGAS/GUPPI Dual Backend Operation

When operating in dual backend mode, any vegas.* or guppi.* keywords will be passed to both backends and do not need to be specified twice unless you want to configure VEGAS and GUPPI to use different values or there is no equivalent keyword for the other backend. For example, 'vegas.numchan = 512' will configure VEGAS and GUPPI to use 512 channels, so you do not need to also specify guppi.numchan.

config_vpm_guppi= """
obstype   = 'Pulsar'
# 'receiver' can be any GBT receiver except MUSTANG.  Here, we use Rcvr1_2, aka L-Band
receiver  = 'Rcvr1_2'
restfreq  = 1500.0
nwin = 1
pol = 'Linear'
# Choose dual backend operation by specifying VEGAS/GUPPI.  GUPPI/VEGAS also works.
backend   = 'VEGAS/GUPPI'
# GUPPI offers a maximum bandwidth of 800 MHz.  Low bandwidth modes of VEGAS are still being comissioned
bandwidth = 800.0
# tint is highly flexible but subject to data rate limits. The true tint will be rounded to the nearest value acc_len * nchan / bw 
# where acc_len is an integer that controls the hardware accumulation length
tint = 10.24e-6
deltafreq = 0.0
swmode = 'tp_nocal' # Choose 'tp' for calibration, 'tp_nocal' for pulsar observation
swtype = 'none'
swper  = 0.04
swfreq = 0
noisecal = 'off' # Choose 'lo' for calibration, 'off' for pulsar observation
vlow = 0.0
vhigh = 0.0
vframe = 'topo'
vdef = 'Radio'
# The following keywords are VEGAS/GUPPI specific
vegas.obsmode = 'coherent_fold' # search, cal, fold, coherent_search, coherent_cal, or coherent_fold
vegas.polnmode = 'full_stokes' # Can be full_stokes or total_intensity for search mode. All other modes require full_stokes
vegas.numchan = 512 # GUPPI has a maximum nchan of 2048 (or 4096 for the Fast4K mode)
vegas.scale = 2.0
vegas.outbits = 8
# These parameters are only used in cal, fold, coherent_cal, or coherent_fold modes
vegas.fold_bins = 2048
vegas.fold_dumptime = 10.0
vegas.fold_parfile = '/home/gpu/tzpar/B1937+21.par'
# vegas.dm only used in coherent_search mode
vegas.dm = 0.0
# These keywords will be different from those for VEGAS and should be specified independently for GUPPI
guppi.scale = 0.8
# Note that there is no equivalent to guppi.datadisk for VEGAS
guppi.datadisk = 'data1' 
"""

Monitoring Observations

VEGAS is already integrated into CLEO and so does not require as many stand-alone programs to monitor observations. Detailed information can be found in the GBT Observer's Guide. Briefly, the VEGAS CLEO screen provides information on the current VEGAS configuration and state of the software monitors. The VEGAS power monitor screen shows real time input power levels (which should be around -20 dBm) and histograms of the output values from the ADCs (analogous to guppi_adc_hist). These histograms should have a roughly Gaussian shape with a FWHM of about 20 counts or so. A screen shot of the VEGAS power monitor is show below for a well balanced system.

vegas_power_monitor.png

Shared memory can be viewed using vegas_status (similar to guppi_status)

In incoherent modes, a live plot of the pass-band can be viewed using vegas_monitor (similar to guppi_monitor).

In coherent fold modes, a data can be viewed in real time by visiting http://www.gb.nrao.edu/vpm/ (similar to http://www.gb.nrao.edu/guppi/)

In all coherent modes, the state of each VEGAS HPC can be viewed using vegas_hpc_status (similar to guppi_gpu_status).

Accessing and Processing Your Data

VEGAS pulsar modes write data in the PSRFITS standard. These can be processed using your pulsar software data package of choice. Data are written to the main GBT data storage lustre filesystem. Whereas GUPPI data were written using a directory structure like /<dataDisk>/<username>/<projectID>/<dateString> (e.g. /data1/rlynch/AGBT16A_470/20160618), VEGAS uses the standard directory structure for GBT spectral line data. Thus, data can be found in /lustre/gbtdata/<projectID>_<sessionID>/<bankName> (e.g. /lustre/gbtdata/AGBT16A_470_01/A). Session IDs are specified in Astrid. Output file names follow the same format as GUPPI, but with VEGAS as the backend name (e.g. vegas_57557_B1937+21_0001_0001.fits).

Rather than writing data to disks on the VEGAS HPCs, coherent dedispersion data is written to bank-specific sub-directories in the main directory tree. In fold-mode, these data can then be combined to cover the full bandwidth using the PSRCHIVE command psradd. Since the data are not spread across different gpu nodes, get_data and combine_psrfits are no longer required. However, a convenience script similar to combine_psrfits called vpmMergeFold.py is available.

A script to combine coherent search mode data and incoherent search mode data taken with multiple banks is under development.

-- RyanLynch - 2016-11-28
Topic attachments
I Attachment Action Size Date Who Comment
vegas_power_monitor.pngpng vegas_power_monitor.png manage 80.5 K 2016-06-23 - 17:31 RyanLynch VEGAS power monitor screen shot
Topic revision: r6 - 2016-11-28, RyanLynch
 

This site is powered by FoswikiCopyright © by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding NRAO Public Wiki? Send feedback