Location and regular date and time of meeting - agreed weekly on Mondays; GB-137 / Soc-280; 2-3pm ET
BLOSS, MARTIN ALAN
FORD, JOHN M
MCCULLOUGH, RANDY L
VAN TONDER, VEREESE
Randy, Jason, Ray, Marty, Ryan, Justin, Richard
BOF file status updates
Switching signal builds
c1500x4096, i1500x2048, and i0800x2048 all seem to behave in the same manner as the non-SS builds
c0800x4096 mode scans fail to initialize
NETSTAT appears to remain in "Waiting" for too long, and Manager aborts because the hpc process is taking too long to start
This is because packets are not flowing
Ryan to double check addresses
Can also try it in DIBAS
Recompiled GPU transpose code
BLI found using packet-sockets solves this problem and found them highly efficient. That code is done and available.
Will need to integrate this into current code, but other modes have higher priority
Ryan to contact Dave McMahon about these changes to see what needs to be done to incorporate them into our "release" (or pre-release versions)
guppi_daq_server version comparisons
Is this the SS version?
This has been tested and performs identically to the previous build, i.e. scans run but with dropped packets.
Scans failing during long integrations
Ray built a guppi_daq version that uses the correct variable type in shared memory. This seems to have solved the problem, though the mechanism is still not entirely understood.
Used successfully during Breakthrough Listen tests (though with some packet loss in the RAW mode)
Ryan talked with Andrew Siemion about analysis of their pulsar data. This will be done in the near future.
Randy built a version with SS support
Ryan tested but there problems with bandpass
Low channels appeared to completely saturate, despite raw counts being within an acceptable dynamic range
Higher channels seemed OK
Randy and Jason are looking into the problem with this build
Fix for nchan calculation in coherent modes
verified - ready to roll into other modes
Bandpass issues in i0800x0064, 0128, 8192
Spent some time testing 64 channel mode
Injected a digital tone at 100 MHz, which always shows up at the correct frequency
Tone shows up in the correct bins, but generates spurs if injected at Fs/8 or nearby
Spurs move with the test tone and decrease with amplitude if you move away from Fs/8
May be problem with MMCM or OGP calibration
A suggestion was made to build a BOF with digital noise source to test
tough for 8192, others should be ok
test w/ 64 chl mode since testing already done by Jason
work needs to prioritized against other work
Jason worked with VEGAS hardware and saw same issues as on srbs
On srbs, spurrs were showing up when injecting a tone at Fs/4
DIBAS modes have clean bandpass
On VEGAS with AP, spectrum still looked poor, with spurs showing up
Result of discussions with Paul regarding OGP/MMCM calibration
Duplicated signal in i0800x8192
Low bandwidth modes
Need to decide on how to proceed with FFTs being done either on FPGA or by GPU. This is a hardware vs software approach.
If bandwidth is low enough all processing could be done on the GPU instead of the FPGA, but is there any advantage one way or the other?
More precision in the GPU
Could use one HPC to process full 200 MHz, otherwise need to do FFT on the FPGA
GPU code would have to do filterbank, not just coherent dedispersion
If done on the GPU, very minor changes to the BOF files, but need CUDA development (personnel is a problem)
If it is done on the FPGA, new BOFs have to be built, not clear how this would be done
In coherent dedispersion, time series would be fed into HPC, which would do a PFB/FFT, corner turn, FFT, complex multiply, FFT, accumulate
Need to add the PFB/FFT and corner turn, the rest is part of the coherent dedispersion algorithm
In incoherent mode, just do the PFB/FFT and then accumulate
Roach output would be the same as the spectral line modes, so we would have to merge the PFB/FFT, corner turn, and coherent dedispersion blocks, but these all exist
Another option is to use DSPSR, which can already do everything, but would use shared memory, which is going away
Ryan did further tests to investigate bandpass discrepancies
VEGAS bandpass does not reflect the S-Band receiver response the way GUPPI does
Issue seen when configuring only with VEGAS, and for older BOF file and newer SS-capable BOF file (both in i0800x2048 channel mode)
Spectral line mode 3 is similar to what we expect based on GUPPI
Same problem seen at L-Band
Injected tone recovered with GUPPI but frequency was wrong in VEGAS
Ryan and Justin tracked this down to an incorrect specification of IF3 in config tool and unexpected python behavior that could cause config tool to apply the LBW frequency shift in pulsar modes. These have been corrected but not tested.
Astrid reports that VEGAS fails to balance, although levels in the VEGAS power monitor seem normal
This is probably because Astrid cannot balance the requantization gain
This the equivalent of the guppi.scale / vegas.scale parameter. Can we balance this automatically in pulsar modes?
LBW gain so that dBFS is equivalent in and out of FPGA
There is an MR that describes balancing for spectral line modes
In spectral line mode this is handled during Balance
Are spectral line snap blocks in pulsar BOF files?
Astrid issued a warning about a discrepancy in the requested and actual IF rack power levels, though this did not cause a failure
An observer already expressed interest in configuring GUPPI and VPM in slightly different modes. We were not planning to support this but if we did, how much effort would it be?
Justin thinks this would be fairly easy
If we allow different scale parameters then we will have to support something like this anyway
Backend-specific parameters would probably be easy to set independently but higher level parameters like integration time would be harder to set independently
GBT Status shared memory parameters in PSRFITS headers
Switching signal generation
Cal successfully controlled by VEGAS
Actual swper still not matching requested swper
Justin to work on it some more this week
Incidentally, in dual backend mode DCR was automatically chosen as switching signal master
Justin has all threads running for all observing modes, and FITS files are being written
Need to check that FITS files are being written properly; Ryan to help with this
Richard suggests automating this by comparing FITS files with standards that are known to be correct
Justin working on VEGAS Manager to use Matrix instead of older guppi_daq
Justin hopes to have this done within 1--2 weeks.
Looking into VEGAS vs GUPPI bandpass differences
Filters did not match drawings and do not fit circuit board
Need a new circuit board, then we have all the necessary parts
VEGAS/DIBAS-style backend for FAST
Work progressing as independent project
Any overlapping work to note at this time?
Proposed language concerning the effects of FAST on GBT pulsar observations. This is for inclusion in the FAST approval documents.
Effect on GBT pulsar science:
The overall effect of the FAST project on GBT pulsar observations should be completely mitigated by continued use of the GUPPI pulsar backend for current and planned timing observations. The beginning of parallel operation needed to verify timing solutions of VEGAS Pulsar and GUPPI match will be delayed up to three months if the VEGAS Pulsar work is completely terminated to free up the FAST project resources. If the shutdown is more targeted, it remains possible that the high-priority VEGAS Pulsar modes may be able to start close to the original schedule, but the medium and low priority modes will likely be delayed. This is not a problem as long as the GUPPI system continues to be reliable.
Multicasting for spectral line and pulsar modes
Modifications to ROACH configuration and net task must bind to a multicast group instead of a specific host
All the 10-gig addresses on the ROACH would be multicast groups, and config will be told which to bind to
Some changes to net task code (in VEGAS HPC and guppi_daq code) to initialize sockets