NRAO Algorithm Research and Development Group (ARDG)

Welcome to the Algorithm Research and Development Group (ARDG) page on NRAO Wiki.

---

The Algorithms Research and Development Group (ARDG) was re-constructed in 2018 under the NRAO's Data Management and Software (DMS) division. The group is tasked with developing and executing a long-term road-map for algorithms R&D at NRAO, and distributing yearly progress report to the community.

Group Members

The total ARDG effort is about 2 FTE contribution from the functional time of the following staff members:

Sanjay Bhatnagar	Scientist	Group Lead
Mingyu (Genie) Hsieh	Software Engg-IV
Felipe Madsen	Software Engg
Preshanth Jagannathan	Assoc. Scientist
Members At-large
Brian Kirk	NRAO Reber Fellow/PhD Student		Reports to thesis supervisor (U. Rau)
Hendrik Mueller	Jansky Post-doctoral fellow		NRAO Scientific Services and Research
Srikrishna Sekhar	Assistant Scientist		The CASA Group/NRAO Scientific Services and Research

Activities

• From Cells to Galaxies initiative (C2G) : This is initiative to bring together researchers from the medical imaging and radio astronomical imaging communities together for joint research, development and collaborations. This is an exciting, for both communities, activity developing by the year. Here is a short summary of the story so far.
  1. We organized a meeting at the NRAO, Charlottesville, VA office in 2019 to gauge the interest in both communities, and develop plans for the future. The conclusion was that there is more than sufficient interest in both communities for a meeting.
  2. An inaugural meeting was organized in Santa Fe, New Mexico in 2020. And then the COVID-19 lockdown hit!
  3. During the lockdown, to keep the momentum going, we organized an online lecture series in which Dr. Neb Duric, University of Rochester Medical School, Dr. Daniel Sodickson, NYU (for MRI) and Dr. Urvashi Rau, NRAO (for RA) gave the introductory talks. There were about a dozen other talks.
  4. We finally met in-person, but at St. Paul, MN in Sept. 2022. It was a workshop style meeting, were we also invited a few researchers from the EU. By many measures, that meeting was a success with significant interest in (a) organizing the next meeting in about 2 years, (b) expanding the participation from both communities.
  5. Planning is currently in progress for the next C2G meeting targetted for 2025.

• The LibRA project : The primary goal of the LibRA project is to directly expose algorithms used in Radio Astronomy (RA) for image reconstruction from interferometric telescopes. The primary target users are research groups (R&D groups at observatories, at university departments) and individual researchers (faculty, students, post-docs) who may benefit from a software system with production-quality implementation of the core algorithms which are also easy to use, deploy locally and modify as necessary. Therefore, a strong requirement driving this work is to keep the interface simple, the software stack shallow and the software dependency graph small. This repository contains only the algorithmically-necessary code, and a build system to compile it into a library of algorithms. Such a library can be directly used as a third-party library by others in the RA community. Interfaces are provided to access the algorithms from C++ and Python, or as an end-user via standalone applications to conveniently configure and execute the algorithms from a Linux shell. The low-level algorithms exposed by these interfaces are factorized to be used as components in a higher-level generalized Algorithm Architecture.

• Deep Imaging at a National Scale : Using the The LibRA project as a path-finder for ngVLA-scale processing, we deployed the computationally intensive components of the Algorithm Architecture (paper in preparation) on a scale about 10x larger (O(100) GPUs) than what has been attempted so far at NRAO (O(10) GPUs) to investigate the operational and computational challenges of distributed computing at this scale. For this, in collaboration with the Center for High Throughput Computing (CHTC, Univ. of Wisconsin-Madison, WI), we used a nation-wide network of computers in the Open Science Pool (OSPool), U.S. National Science Foundation's Pathways for Advancing Throughput computing (PATh), San Diego Supercomputer Center (SDSC) at the Univ. of California San Diego and the National Research Platform (NRP). This produced the deepest image ever made with the VLA, and in the RA community world-wide of the Hubble Ultra-Deep Field (HUDF) achieving a noise floor of 1 microJy/beam.

ARDG Memos

Reference	Author(s)	Date
ngVLA Computing Memo No. 7	S. Bhatnagar, F. Madsen and J. Robnett	May 09, 2022
Same as ngVLA Comp. Memo No.7 with minor corrections in Figs. 1&2
ngVLA Draft Memo - Recommended Developments for Applying (W)Asp	M. Hsieh, S. Bhatnagar, R. Hiriart, M. Pokorny	Mar. 2022
Wide-band implementation of the Asp-Clean algorithm	M. Hsieh, U Rau & S. Bhatnagar	Mar. 2022
The design of the AWProject framework	S. Bhatnagar	Feb. 2022
High Performance Gridding	M. Pokorny	Aug. 31, 2021
Size-of-Computing Estimates for ngVLA Synthesis Imaging	S. Bhatnagar, R. Hiriart and M. Pokorny	Aug. 10, 2021
Full-Mueller Imaging with ALMA: ALMA Study Project report	S. Bhatnagar et al.	Feb. 2021
Narrow-band implementation of the Asp-Clean algorithm	M. Hsieh & S. Bhatnagar	Jan. 2021
VLASS Memo #15: VLASS Imaging Project: Proposed workflow	J. Tobin, J. Marvil, J. Ott, and S. Bhatnagar	May, 2020
Heterogeneous Pointing Corrections in AW-Projection	P. Jagannathan, S.Bhatnagar	Oct 2019
Building a model for antenna aperture illumination pattern	S.Sekhar, P.Jagannathan, B.Kirk, S.Bhatnagar and R.A.Taylor	Feb 2019
Wideband Mosaic Imaging for VLASS	U. Rau & S.Bhatnagar	Sep 2018

Talk Slides

Reference	Author(s)	Date
Synthesis Imaging with CASA	P. Jagannathan	Mar. 24, 2022
A to Z Solver Methodology	P. Jagannathan	Jan. 5, 2022
A Systems approach to Imaging and Calibration (a talk in the RCI Seminar Series)	S. Bhatnagar	Feb. 12, 2021
Algorithms R&D Update: Retrospective 2019 + Proposed plan 2020	S. Bhatnagar	Jan. 22, 2020
Understanding and using the AWProjection algorithm	S. Bhatnagar	Sept. 7, 2018

ARD Discussion Topics (dates and speakers TBD)

TOPIC	DESCRIPTION	DATE
Foundations of radio interferometric observations	Details of the measurement process in theory and in practice	MM/DD/YY
Demystification of post-processing: Understanding RA calibration and indirect imaging	Simplified description of the theoretical framework for calibration and imaging	MM/DD/YY
Quantization in the measurement chain	Understanding the effects of the inevitable quantization at various stages of the measurement, calibration & imaging process	MM/DD/YY
User-level understanding the algorithmic framework implementation in CASA	Mapping the theory to the data processing operations most users are familiar with	MM/DD/YY
Understanding connections between algorithms, imaging performance and computing load	Impact on run-time of the algorithmic choices, estimating what's needed when	MM/DD/YY
Deconvolution through the ages	A review and comparison of deconvolution algorithm formulations over the past 20yrs.	MM/DD/YY
Spectral Line Deconvolution	State of the art in line search algorithms and the need for joint deconvolution algorithms	MM/DD/YY
RFI mitigation	ngVLA study and other ideas to pursue at the VLA. Review of what other observatories do	MM/DD/YY
Faraday Rotation Synthesis	Review of methods and what would be useful to adopt here	MM/DD/YY
Wide-field Imaging	Review of methods such as AW-Projection, Different types of DD-cals, Using spherical geometry for W-term, etc	MM/DD/YY
Single Dish and Interferometer Combinations	State of the art and outstanding problems/requirements	MM/DD/YY
Generic HPC solver frameworks	Moving beyond our standard chi-square minimization and major/minor cycles for next gen algorithms	MM/DD/YY
Primary Beam Models	CASA's models, Zernicke models for ALMA/VLA/MeerKAT, Review of other approaches, guidelines for what a good model is	MM/DD/YY
Full Mueller Imaging	Review of methods, explanation of Full Mueller Imaging vs what is usually described as Full-Stokes Imaging	MM/DD/YY
Machine Learning Applications	Do we have use cases where Machine Learning and AI will halp ?	MM/DD/YY
Pipeline Heuristics	Invite SRDP/ALMA/VLA pipeline folk to describe top heuristics they depend on plus unsolved algorithmic problems they face	MM/DD/YY
Simulations in CASA	Going beyond simobserve.	MM/DD/YY

Relevant Scientific Publications

Software Packages

The ARDG staff conducts majority of their R&D using the CASA package.

• wide-band implementation of of the Asp-Clean algorithm: WAsp.pdf

• Memo for ngVLA-38: Recommended Developments Necessary for Applying WAsp to ngVLA: ngvla_wasp.pdf