West Virginia University Ultimate Pulsar Processing Instrument (WUPPI)
The WUPPI is a clone of GUPPI. It is a hardware which was developed from reconfigurable off-the-shelf hardware platforms (IBOB's and BEE2) and software tools that allow deployment of astronomical signal processing systems. One of the main applications is to develop a FPGA based hardware and software for Pulsar observations using the 43m. The main difference between WUPPI and WUPPI would be the hardware they used and the data acquisition machine WUPPI uses (Tofu). The hardware used in this setup was 2 IBOB's and a BEE2.
The goal of this project is to develop a clone of WUPPI which can act as a pulsar machine for observations using the 43m telescope.
WUPPI User's Guide
What is WUPPI?
The West Virginia University Ultimate Pulsar Processing Instrument is a backend hardware and has a custom FPGA. WUPPI is implemented using the custom "Gateware" that is compiled and loaded into the 7 Xilinx FPGA's.
WUPPI's specifications are:
- Full Stokes parameters available
- Frequency and time down sampling
User defined parameters
When using WUPPI, there are quite a few parameters that you can change to affect the way that WUPPI takes data. The vast majority of these parameters are passed to WUPPI (or the data acquisition software) by defining or setting FITS-header-like keyword/value pairs in a shared memory on "Tofu", the data acquisition machine.
WUPPI software user's guide
Below are the instructions to get things up and running. This documentation is from Shilpa Bollineni and Glen Langston. But, getting WUPPI running was possible with the great help from many people like Jason Ray, Randy McCullough
, Paul Demorest, Ron DuPlain
and Scott Ransom.
To work with WUPPI we will need several xterms, all connected to "tofu".
A - Try communicating with bee2 via telnet
1. Open a xterm window and connect to tofu. Login as yourself, and ping bee2 with the command:
[user@tofu ~]$ ping 169.254.128.20
PING 169.254.128.20 (169.254.128.20) 56(84) bytes of data.
64 bytes from 169.254.128.20: icmp_seq=1 ttl=64 time=0.682 ms
64 bytes from 169.254.128.20: icmp_seq=2 ttl=64 time=0.309 ms
64 bytes from 169.254.128.20: icmp_seq=3 ttl=64 time=0.307 ms
64 bytes from 169.254.128.20: icmp_seq=4 ttl=64 time=0.307 ms
-- 169.254.128.20 ping statistics --
4 packets transmitted, 4 received, 0% packet loss, time 3000ms
rtt min/avg/max/mdev = 0.307/0.401/0.682/0.162 ms
This implies that the communication with the bee2 is good.
Then ssh into bee2 using the following commands:
[user@tofu ~]$ ssh root @ 169.254.128.20
Last login: Thu Aug 20 15:31:39 2009 from tofu.gb.nrao.edu
This indicates that we are able to talk to the bee2.
B - Open the xterm/console window to use for the wuppi software
1. Open a xterm window and connect to tofu. Login as yourself, and source the wuppi startup file (which will give you access to the WUPPI commands).
user@tofu ~]$ source /opt/64bit/wuppi/wuppi.csh
You should do that in each xterm that you will be using. Source wuppi.csh or wuppi.bash as appropriate for your login shell.
2. Start up the command line interpreter:
[user@tofu ~]$ wuppi
Welcome to the NRAO WUPPI interpreter and command prompt.
3. To check which design files are loaded in the hardware, use the unload () command.
If you get the following response:
This implies all these files are loaded into the FPGA's now.
If you are get the response
then you will need to load all the files using the commands:
note: 1. The .bof files that we have to load into the BEE2 are located in the folder /home/gbt1/casper/jray/bit_files/Guppi_Production_Files.
2. We don't manually load, unload or set the parameters in the designs any more. We have python scripts embedded in the wuppi interpreter which will do those functions for us.
The python script that has these functions is wuppi.py and it is located at:
The functions we have in the wuppi interpreter are :
wuppiStart() --- This function is used to load the bof files in the BEE2.
init() --- To initialize the different parameters in the design.
setParams() --- This function is used to set any parameters in the design.
Now the data should be streaming...
5. Go to part C below to start the wuppi monitor software
6. To exit out of this, press CTRL+D, then type exit
C - Open the xterm window to start the 10gig data monitor
1. Open up a couple more xterm windows on tofu.
2. In each of them source the "wuppi" environment stuff:
[user@tofu ~]$ source /opt/64bit/wuppi/wuppi.csh
Note that that command should be used before you try to do anything with WUPPI.
3. Now we're going to make sure that we can communicate with WUPPI via shared memory. In one of the windows type:
[user@tofu ~]$ guppi_set_params packets=1SFA nchans=4096 bw=800 freq=400
That will put new data in the shared memory that is used by the WUPPI software. You can view that by running the following in another xterm. If you just give the "guppi_set_params" it will show you a help screen.
[user@tofu ~]$ WuppiStatus
4. Now we are going to stream bogus data through the system so that we can check levels etc.
[user@tofu ~]$ test_net_thread
and in the window running the "wuppi" command line interpreter, type:
In the wuppiStatus window you should see values changing now, indicating that data is flowing.
5. In yet another xterm, run "wuppi_monitor".The system bw is 800 MHz but the frequency range of 0 to 250 MHz was eliminated using filters.
Example guppi_monitor output for 43m telescope . Red = average, Blue = min/max.
6. To exit out of this, select the plot window, and hit the q key. Then over in the putty window, type exit.
C - Look at histogram of input levels
1. In the WUPPI command interpreter (see part A), type "arm()" to refresh the data in the BRAMS. ( NOTE: Do not do this if someone is currently recording data to disk!).
2. Type "plot_adc_hist()" to show the histogram (see sample plot below). Close the plot window when done. plot_adc_hist() also reports suggested changes to the input attenuation for proper signal level.
Example Histogram of input sample levels to WUPPI, with the data from the 43 telescopes.
D - Changing the WUPPI sampling rate.
By default, WUPPI is set to integrate 16 raw samples in the 4096 channel default FPGA code. We'll call that number acc_len (you'll see later). To see how that translates to the sample time (dt) you will get, the formula is (in sec): dt = acc_len x nchan / BW. So for the nominal mode: dt = 16 x 4096 / 800e6 = 4.096e-05, or ~21us.
We can change acc_len by running the command 'set_acc_len' which will set the WUPPI hardware parameter 'BEE2/FPGA2/ACC_LENGTH' to the value you state minus one in hex. So don't be confused by the value returned by the set_acc_len command.
For example, to see what that parameter is set at you can do:
In this case, 'f' is hex for 15, so acc_len is 15+1 = 16.
You will almost certainly need to change the accumulation length for the low-BW modes. The current lowest value allowable for 'BEE2/FPGA2/ACC_LENGTH' is 5 (for acc_len = 6) 800MHz-BW mode, and 3 (for acc_len = 4) for the narrow-band (100MHz and 200MHz) modes.
Here is an example of setting the accumulation length to the fastest dumping rate for an 800MHz mode (for acc_len = 6) which gives 15.36 us sampling (note, though, that we can't write data or even fold data at that rate -- folding data rate limits are around 400MB/s and search-mode data rate limits are around 200MB/s).
set BEE2/FPGA2/ACC_LENGTH to 5 ... True
If you use set_acc_len(4) for the 200MHz mode, your sampling rate will be 40.96 us. For the 100MHz mode it will be 81.92 us. Those are currently the fastest sampling rates we can get for those modes (using 4096 channels).
For search observations, it is often useful to get an even number of samples per second so that different scans can be exactly pieced together (scans start on integer seconds). For 800MHz BW mode, acc_len=25 gives 64 us sampling and therefore exactly 15625 samples per second. For the narrow band modes, unfortunately there are no reasonable values of acc_len that give an integer number of samples per second.
Also note that you must tell wuppi_set_params (during data taking) that you are using a non-standard accumulation length. Just set the command-line parameter "--acc_len=6" for the above examples (remember that it is the value for 'BEE2/FPGA2/ACC_LENGTH' plus 1).
Make sure to check all the values in WuppiStatus before you start taking data if you are worried!
Here is the WUPPIDataTakingGuide.
To change the destination address for the high speed data, the following command resets the destination
wuppi> verbose_set('BEE2/FPGA2/DEST_IP', 'c0a8030a')
In this case, the IP address is 192.168.3.10 == 'c0a8030a'
WUPPI guide for different frequencies (200MHz, 400MHz)
Appendix: Trouble Shooting the communications link
The following steps are not usually necessary, but are helpful in trouble shooting the communications link between WUPPI and the Linux data taking computer
This command receives packets from the WUPPI 10 GbE switch and computes the drop rates.
/users/pdemores/local64/bin/udp_recv -a -e -p 50000 -s 8224 192.168.3.8
The configuration of WUPPI depends on communications using a demon. This demon will occasionally hang, and
must also be restarted after each computer power cycle. The user must be logged on as root to execute these commands.
To start the demon, type:
If the demon stops, it can be restarted using the following command:
- /home/pulsar64/bin/python /home/pulsar64/bin/supervisord
The command will generate two messages: wuppi: stopped and wuppi: started
These commands require a configuration file be placed in the /etc directory or the local directory where the command is executed.
On Cicada2 and Tofu the file is located at:
-- GlenLangston - 2010-11-17
-- Main.Shilpa Bollineni - 18 Aug 2009
- sudo /home/pulsar64/bin/supervisorctl restart wuppi