casaviewer

July 2010, Draft 003

Nick Elias, Darrell Schiebel

Introduction

The casaviewer is an important component of CASA because it provides the user with a direct interface to his/her data. As a matter of fact, users will give significant weight to the casaviewer when deciding whether CASA is a good overall interferometric imaging package or not. With the creation and maintenance of this document, we are essentially ready to go for casaviewer development.
History

The first casaviewer used by aips++ was called aipsview, and it was developed primarily by Harold Ravlin at NCSA from 1995 to 2003. This casaviewer was based on Motif and was largely independent from the aips++ code base. Some people working on CASA (aips++ renamed) still use this casaviewer because it has features that have not yet been reproduced in the current casaviewer, e.g., particularly the ability to have multiple images which have linked cursor tracking (moving the cursor in one reveals the same world coordinate position in the other image).

aipsview was thought to be insufficiently programmable and Motif was thought to have a limited future. As a result, in 1998 development was started on a new display toolkit. This toolkit allowed developers to insert and remove data, create customized display solutions, and control the display environment at a low level. The scriptable end of this toolkit was based on Glish while the GUI was based upon Tk widgets (including the TCL-less Rivet library). Unfortunately, it was difficult for typical users to make use of the flexibility and power of this system, and the messaging requirements on the Glish bus for the low-level widget interaction resulted in a sluggish system. The death knell for this toolkit came in 2004 when the CASA project (at NRAO) decided to switch from Glish to Python to bring itself in line with ALMA, its primary funding source, as well as other software projects within the astronomy community.

A period of some experimentation followed during which time a rework of the display toolkit idea was tried in Python + Tk. However, in 2005 work began on the current casaviewer. Design elements can be traced to the original aipsview application, and the low-level display library source code upon which the Glish toolkit was based is still used in the current casaviewer. In principle, it should be possible to resurrect the features of aipsview into casaviewer. All that is required is a plan, design, and time.
Development Concern

While the existing casaviewer implementation works pretty well, some features are cumbersome to use and need improvement. For example, GUI widgets that are valid for images may not be necessarily valid for MSes. Also, there are a number of outstanding casaviewer tickets in the JIRA system corresponding to non-existing required features and annoying bugs.

The casaviewer’s capabilities are somewhat limited when compared to other packages, but that does not necessarily mean that we should rush to incorporate new features without sufficient planning. Given our constraints in manpower and expertise, we must carefully determine what/why/how/when features should be included, otherwise we may end up with another depreciated application like plotxy. In this document, we attempt to annotate and clarify all issues so that we can formulate a coherent long-term plan for casaviewer development.

Apart from real-time programming, creating effective GUIs is one of the most difficult jobs in software engineering. It requires 1) an understanding of the tasks required by the user and how the user will interact with these tasks, 2) GUI development skills, 3) ergonomic planning, and 4) even a bit of artistic capability. Item 1) can be derived from a list of requirements (which already exists for the CASA casaviewer). We all agree that requirements are important. The code, however, should flow from a maintained detailed design not requirements, which has not been developed and maintained for the casaviewer.

Given the previous history of plotxy, Nick Elias began to have concerns about the casaviewer. He decided that it was time to investigate its existing design, modify the design if necessary, create a plan, and create JIRA tickets based on that plan. Apart from the JIRA tickets, all of these steps must involve both the developers and science advisors. Darrell has already done the research for the existing design, and that appears in Sections 1.0-4.0 of this document.

I should also point out another wrinkle that caused me to pause and reflect. I received an email from Dirk Petry stating that ESO had hired a developer who was planning to investigate using the casaviewer for optical/IR spectral-line imaging, modify the code as necessary, and possibly merge it back to our version. In principle, I am very much in favor of finding new customers for CASA (e.g., I have already CLEANed and viewed optical interferometer data from NPOI), but this episode renewed my conviction that we must plan casaviewer development more carefully.

image00.png

Qt Toolkit and DBus

The Qt toolkit is a cross-platform (Linux, OS X, Windows) widget toolkit. It provides a very nice GUI development platform, and a version of it is available under the Lesser GNU Public License. The Qt layer built upon the original display library has been developed since 2005. The casaviewer application can be run directly by the user or it can act as a server for display panels. In the case where the user starts casaviewer, they are initially presented with a display panel along with a dialog for opening CASA images. The display panel provides options for opening other images, looking at the channels of the image cube, zooming, etc. The user can set a number of options for each loaded dataset (e.g. colormap, scaling, units, labeling etc.). There are a few tools which allow for different views of the data (e.g. spectral profile) in addition to zooming and region selection from the display panel dialog.

When casaviewer is started within casapy (CASA's python environment), it acts as a server waiting for connections from DBus clients, typically casapy. DBus is a desktop (single host) messaging system which allows casapy to control the casaviewer, e.g., opening display panels, loading images, zooming, etc. DBus supports scripting from any language that includes DBus support, such as Python, C++, and Qt. Many other languages provide DBus bindings as well.

DBus provides a proxy/server architecture. In the case of Python, introspection is used to automatically create proxies. Qt can automatically create DBus bindings which export public signals and slots. The standard generated C++ binding is very close to the DBus C library. The interface is specified in XML and DBus C++ bindings can be generated for either the proxy or the server.

Design of the Qt Layer

The main class is called QtViewer. This class is a factory which produces QtDisplayPanelGUI objects. The QtDisplayPanelGUI corresponds to the "Display Panel" GUI shown above, and it is what users typically think of when they are talking about the casaviewer. The QtDisplayPanelGui contains a number of elements. The primary element is the central widget that is used as a drawing surface. At the lowest level, this area of the panel is the QtPixelCanvas widget. This is the point where the generic "display library" meets the Qt library (also windowing system independent) and actual drawing occurs using Qt primitives. The QtPixelCanvas is derived from PixelCanvas which provides the low level drawing surface within the display library. These two widgets are "low level" in the sense that they only deal with drawing in pixel coordinates. Above this is layered a drawing surface that understand world coordinates (WorldCanvas). The WorldCanvas is derived from a variety of event handler classes and option classes. These classes implement the display library functionality that handles the notification of events triggered by the world canvas. Because this notification mechanics are in the GUI-independent display library, it does not make use of Qt's signal and slot mechanism. However, the classes within the Qt layer do make use of the signal/slot tools.

image01.png

Another element of casaviewer functionality that is accomplished at the world canvas level is axis labeling. This is accomplished in the casaviewer by using PGPLOT along with a wcslib plugin for PGPLOT called PGSBOX. This implementation path is the result of aips++'s historical use of PGPLOT along with the choice of using the tool at hand instead of reimplementing that portion of PGPLOT which was required.

This implementation, however, is interesting for a number of reasons: 1) only a small portion of PGPLOT's functionality is used; 2) PGPLOT is written in Fortran; and 3) wcslib is used for drawing the axes instead of the world canvas and the CASA coordinate system. That said, PGPLOT does a good job of drawing the axes and it (literally) draws the letters of the labels (which has benefits for hard copy/screen correspondence). Because reimplementing this functionality is non-trival, careful consideration is required before attempting to remove PGPLOT from the casaviewer. Axis drawing is accomplished by WCCSNLAxisLabeller, which is a derived class. This class, along with PCVGBuffer, accomplishes the drawing of the axes in the casaviewer. PCVGBuffer is the way PGPLOT drawing commands are transferred to the QtPixelCanvas.

image02.png

Most of the GUI components of the casaviewer are created with programmer-created classes rather than with Qt's designer application. The primary exception is the small "tape deck" GUI found below the canvas window.

Display Library

Development on the "display library" was started in 1998. It was designed as the GUI foundation for image visualization. It is designed on the principle that most of the coding effort would be expended on a core of functionality which was independent of the windowing system and GUI widget set. This would allow developers to write a relatively thin layer at the top (buttons, menus, etc.) and the bottom (QtPixelCanvas) leaving the bulk of the code portable.

The primary class in the display library is the DisplayData class. It is the base class for a tree of classes designed for representing the data which can be drawn on the WorldCanvas. Each display data object knows how to draw itself onto the world canvas. There are a number of types of data which may be displayed.

Astronomical coordinates are used to indicate where drawing should occur in the WorldCanvas. These coordinates are converted to the corresponding pixel canvas and the various display data are drawn onto the underlying QtPixelCanvas widget which is treated within the display library as a PixelCanvas (i.e. the base class).

There are also around 30 other classes within the display library that perform other operations in world coordinate space. These classes handle zooming, scaling, resampling, cursors and selection, event handling, and coordinate conversion. These classes are clearly essential, but they are integrated into the functions of how data is displayed on the canvas and how the user interacts with the data that is displayed.

DBus Integration and Scripting

The casaviewer DBus module uses the adaptor pattern. It is a layer (really just one class, QtDBusViewerProxy) which interacts with both the display library and the Qt layer. The only dependency of the Qt layer on the DBus adaptor is that QtViewer creates and destroys the QtDBusViewerProxy object.

image03.png

The DBus adaptor sets up the connection to the DBus session bus (casapy starts its own session bus) and accepts commands from DBus processes which connect to casaviewer's DBus adaptor. The interface defined by the DBus adaptor is independent of the casaviewer. Thus, it could be replaced in the future with a new implementation of this interface without disrupting any of casaviewer's client DBus applications.

Projected Manpower for All CASA Visualization

Darrell Schiebel and Honglin Ye have been the recent primary developers for the casaviewer. They typically spend ~ 0.25-0.5 FTE on it. Honglin may be spending less time on it in the near future because of HPC needs. George Moellenbrock spent ~ 0.25-0.5 FTE over the last year adding additional features to casaplotms. In addition, David Mehringer was asked to add functionality to the casaplotms task interface.

Given the other demands on the aforementioned developers, I do not anticipate increasing their levels of effort in visualization unless project priorities change. Fortunately, Daren Wilson has just been hired to fill the visualization position, so will be able to slowly increase our rate of visualization development as he is integrated within the team. He will spend ~ 0.5 FTE on casaviewer and 0.5 FTE on casaplotms.

Possible Strategies for Adding Features to the casaviewer

There are many possible strategies for adding new features to the casaviewer, so they should be organized before consideration. All options should be considered, even radical ones. Even options that are not chosen often lead to good ideas for the final option. Our primary goals are to determine how we can best meet science requirements and produce a robust and maintainable piece of software. We should not assume that our existing strategy is the optimum one.

image04.jpg

In the following figure, we display a tree diagram. The strategies can be divided into three groups: red, green, and purple. If one of the two red boxes is chosen, no others can be chosen. On the other hand, if neither red box is chosen, one green and one purple must be chosen. NB: All strategies will require significant designing, planning, and scheduling.

The first red box is labeled “viewer,” which corresponds to continuing casaviewer development (the present strategy). The second red box is labeled “viewer + use/adapt another package.” The other package corresponds to image display software written by a non-CASA developer or group of developers. In other words, casaviewer development continues slowly and another package acts as a stopgap for features that do not yet exist in CASA.

The first green box is labeled “imview,” which corresponds to continuing development with the image part of casaviewer called imview. Some development for this strategy has already been implemented, namely regularizing the tool/task interface. The second green box is labeled “imview + use/adapt another package.” The other package is the same as the image display software in the second red box. The third green box contains “Adapt another package,” which means completely abandoning the existing casaviewer in favor of another image package (a radical and highly controversial strategy).

The first purple box is labeled “msview,” which corresponds to continuing development with the MS part of casaviewer called msview. Some development for this strategy has already been implemented. The second purple box is labeled “plotms,” which means that the MS display capabilities of casaviewer have been migrated to casaplotms and the casaviewer will handle only images.

In the following table, we elaborate on the advantages and disadvantages for each box in the tree diagram.
Box Advantages Disadvantages
viewer
  • the code is ours
  • Even if we hire an additional visualization FTE, there could be a lot of remaining development depending on the number of desired features.viewer + use/adapt another package
viewer + use/adapt another package
  • Use: Even if we do not hire another visualization FTE, users will have more tools to display data. casaviewer development becomes less stressful.
  • Adapt: Support for the other package may be infrequent or non-existent, which means that we could adopt the “ASAP model” (maintain and modify the code ourselves).
  • We must insure that the other package runs on platforms supported by CASA.
  • The other package will only deal with images, not MSes.
  • We must insure that the other package can read CASA image formats (FITS and/or native).
  • Use: Support for the other package may be infrequent on non-existent.
  • Adapt: The other package must be integrated into CASA so that it can interact (via dbus) with CLEAN and image analysis tasks.
imview
  • The imview CLI and GUI will be simplified.
  • Existing image features will remain and act as the basis for additional development.
  • Even if we hire an additional visualization FTE, there could be a lot of remaining development depending on the number of desired features.
imview + use/adapt another package
  • The imview CLI and GUI will be simplified.
  • Use: Even if we do not hire another visualization FTE, users will have more tools to display images. Imview development becomes less stressful.
  • Adapt: Support for the other package may be infrequent on non-existent, which means that we could adopt the “ASAP model” (maintain and modify the code ourselves).
  • We must insure that the other package runs on platforms supported by CASA.
  • Use: Support for the other package may be infrequent or non-existent.
  • We must insure that the other package can read CASA image formats (FITS and/or native).
Adopt another package
  • The other package will have many, if not all, of the features that we need for image visualization.
  • Support for the other package may be infrequent on non-existent, which means that we must adopt the “ASAP model” (maintain and modify the code ourselves).
  • The other package must be integrated into CASA so that it can interact (via dbus) with CLEAN and image analysis tasks.
  • We discard man-years of effort on a task that works pretty well.
msview
  • The msview CLI and GUI will be simplified.
  • Existing MS features will remain and act as the basis for additional development.
  • Even if we hire an additional visualization FTE, there could be a lot of remaining development depending on the number of desired features.
plotms
  • casaplotms is designed specifically to look at MSes. All MS visualization will be located in a single task.
  • Many of the features desired for msview already exist in casaplotms: A mature and expandable CLI, caching, averaging, multipanel (soon), iteration (soon), etc.
  • Without careful planning, we can turn casaplotms into another plotxy.
  • casaplotms does not have a 2D TVFLAG-like plug-in (which mimicks the behavior of casaviewer). Adding this behavior will require a significant amount of work.

Strategy Decisions

Casaviewer is a large and complex package. It is based upon our best ideas from 10 years ago, primary among these is the development of a generic infrastructure. In many instances, casaviewer code goes beyond this strategy by leaving hooks for future development which have never been implemented. The casaviewer has embraced the event driven paradigm common to development of interactive applications and applied it to other classes one step removed from the actual GUI widgets. It often overloads the function invocation operator to handle the "switch" between several different types of operations. It is not thread safe and would likely require significant work to make it a multi-threaded application.

Each of these features, in isolation, is not bad or in some cases actually positive, but in combination they make it difficult to debug, extend, or enhance the casaviewer. Therefore, new additions to the casaviewer have largely been implemented as largely stand-alone applications (though within the same process space) implemented in Qt, e.g., the spectral profile tool. It would take a large development effort, however, to replace the casaviewer. The casaviewer has most of the functionality users desire, and simple adjustments can be implemented in a straightforward manner.

Based on Darrell's investigation, which is summarized in Sections 1.0-4.0, we have determined that the low-level code is reasonably well designed and written. This fact, combined with the advantages and disadvantages enumerated in Section 5 and in the previous two paragraphs, leads us to the following conclusions:

  1. We should continue with existing casaviewer development.
  2. We should split the casaviewer into imview and msview.
  3. We should inform users that if they need visualization features that are not yet available in casaviewer, they should run the exportfits task to translate CASA images to FITS images and load the latter into their favorite visualization software (e.g., ds9 or kvis).
  4. Amazingly, kvis is purported to support direct visualization of aips++ images. Jürgen Ott and Nick Elias tried to load a CASA image (which should be ~ the same as an aips++ image) without success. We suspect that the CASA image format may have diverged slightly from the aips++ format. Jürgen will write an e-mail to the creator of kvis to ask if he would update the capability. We are not optimistic (since the creator now works at Google), but it's worth a single e-mail to ask.

Interfaces to Other Tasks

In this section, we describe how casaviewer interfaces to other tasks to return specific types of information. Six areas have been identified now, but others may be added in the future.
Interactive CLEAN

casaviewer is presently used for interactive CLEAN. Extra widgets allow users to change the the number of iterations, adding/removing boxes, continue, and run to the end. There may be ergonomic issues, e.g., adding/removing boxes for a single plane or multiple planes (important for the large number of planes expected), but they will be addressed in the future.
Spectral Viewer

A basic spectral viewer GUI already exists. It can create the spectral profile of a single pixel or the sum of multiple pixels. It can overlay or difference two spectral profiles. It provides many other options, such as plotting in world or pixel coordinates, radio/optical velocity reference frames, and frequency. There are also a few buttons for saving results. New ergonomics and bug fixes, if required, must be discussed before scheduling and implementation.
Spectral-Line Fitting

There is enough room in the spectral viewer GUI to add more features, such as spectral-line fitting. Gaussian profiles are the norm. But others, such as the Lorentzian, could be added. Fitting a single line requires no input from the user other than pressing a button. Fitting multiple lines, however, requires the user to identify the peaks, so the GUI must include that capability. Dave Mehringer is working on the CLI version of a spectral fitter. The toplevel task is called specfit. Spectral-line fitting from the GUI should call the C++ code for the spectral fitter to avoid code duplication. The results of the fits should be written to the terminal and the logger (and logger files), the latter acting as a decent permanent record that can be cut and pasted into publications.
Spatial Statistics and Fitting

The present casaviewer already employs a spatial statistics calculator. To avoid cluttering the main GUI with a spatial fitter and to be consistent with the spectral viewer/fitter tool, we believe that a separate GUI (or GUIs?) should be provided for these jobs. After the GUI is launched, a box can be drawn around the desired region for statistics, which include the number of points, sum, mean, rms, standard deviation, minimum, and maximum. If multiple channel and/or Stokes planes of a cube are specified in the GUI, all of these values should be provided for each plane. Fitting can be performed in a similar manner. After selecting a box, fitting a single Gaussian (or Lorentzian, or ...) happens automatically. For fitting multiple Gaussians, however, the user should specify the peaks on the image. For image cubes, there is an ambiguity because peaks may occur in different places (or not occur at all) in each plane. This ambiguity must be discussed and solved.

Dave Mehringer has been working on CLI versions of spatial statistics calculator and spatial fitter. The toplevel tasks are called imstat and imfit, respectively. The GUI should call the relevant C++ code to do these jobs to avoid code duplication. The results of the fits should be written to the terminal and the logger (and logger files), the latter acting as a decent permanent record that can be cut and pasted into publications.
Position-Velocity Diagram

Position-Velocity (P-V) diagrams are used to study the rotation curves of galaxies. It should be launched from a separate GUI. There are two ways to specify the region: 1) enter numbers into the GUI; or 2) draw the box, which can be stretched and rotated. The orientation collapse axis must be specified as well. Assuming that there is a single emission line species (most likely HI), the velocity must be fit automatically for each spatial pixel before collapsing. Dave Mehringer has been working on the collapse function in the image analysis tool. It is possible that this function will need to be modified to handle collapsing small regions. To avoid duplicating code, the GUI should be linked directly to the C++ code.
Spatial/Spectral Slices

This subject is quite complicated, so we will discuss it in the future.

Open casaviewer JIRA Tickets

The summary spreadsheet containing the open casaviewer JIRA tickets is stored in Google docs. It will be maintained by the casaviewer technical lead. The list order should be roughly the same as the completion order. JIRA is a good tool, but we need a spreadsheet in order to clearly see the big picture as well as checking the statuses, brief comments, estimate completion times, and priorities of all tickets. JIRA has limited capabilities in these regards. We do not anticipate that too much time will be required for maintaining the spreadsheet.

Unfinished Requirements

We have gone through the ALMA and EVLA visualization requirements to determine which items are: finished, unfinished with a JIRA ticket, and unfinished without a JIRA ticket. We list the latter two types here. We have done an initial prioritization that is subject to change. We have not yet reconciled this list with the JIRA ticket spreadsheet. These tables will be maintained by the casaviewer technical lead.
Generic (images and visibilities)
Issue Details JIRA Ticket(s) Priority
Blinking between two planes in the same cube Two channels, two Stokes parameters, two correlations, etc.   High
Save publication quality plots and images Need to determine what this means for different outputs CAS-1773 High
Support of FITS, PDF, PNG, GIF, JPEG The capability appears in the print GUI, but it is not very ergonomic   High
Axes of multipanels only along outer panels Should be relatively easy to do. CAS-1872 High
Blinking between two planes in different cubes Two channels, two Stokes parameters, two correlations, etc.   Medium
Setting the relative transfer functions of two images A split screen would be required.   Medium
Export information from pixel or region to the logger or ASCII file Honglin tells me that this task is not trivial.   Medium
User-defined kernel spectral Hanning smoothing From task interface, not on the fly. These should be done in other tasks first, making this a lower priority. CAS-1866 Low
Fixed kernel spectral Hanning smoothing (-0.25, 0.5, -0.25) From task interface, not on the fly. These should be done in other tasks first, making this a lower priority. CAS-1866 Low
Boxcar spectral averaging From task interface, not on the fly. These should be done in other tasks first, making this a lower priority. CAS-1866 Low
Boxcar spectral smoothing From task interface, not on the fly. These should be done in other tasks first, making this a lower priority. CAS-1866 Low
Rendered and wireframe 3D surfaces Honglin has a simple prototype 3D viewer for images.   Low
HSV and CMYK raster images In the past, journals required these formats, but we believe that they do not do so anymore.   Low
Setting of relative transfer functions of planes We must come up with an ergonomic way of doing this task in a no tedious manner.   Low
User selection of animation order Assuming that the selection comes from the GUI, we must insure that we consistently use the same selection format used elsewhere in CASA   Low
Extra information axes encodable on standard plots At present, we tend to support mostly EVLA and AMLA   Low
Boxcar time smoothing From task interface, not on the fly. CAS-1897 Low
Boxcar time averaging From task interface, not on the fly. CAS-1897 Low
Choice of logarithmic amplitude scales There is something similar available, but it is not true logarithm.   Low
Control of line width and style for XY plots (spectra) Probably buttons on the spectral viewer.   Low
Import/Export of color maps and/or functions Need format of file and function parameters.   Low
User editable/definable color maps or functions Need GUI configuration.   Low
Display and plotting parameters savable and reloadable This mostly works already. Need to do final check.   Low
Pseudo-multicolor overlays (RGB intensity scales) For images with the same axes, sizes, and orientations.   Low
Images

Issue Details JIRA Ticket(s) Priority
Region histogram How to implement via GUI? CAS-2063 Medium
1D slice sent to spectral viewer E.g., used for PV diagrams. NB: We already do 2D. Just need to determine how to implement this in the GUI. CAS-1869 Medium
Choose different coordinate and frequency/velocity units From task interface or GUI?   Medium
Plot spectra on a pseudo-grid corresponding to location on image Thumbnails corresponding to small regions on the plot specified as M rows by N columns. Each thumbnail goes to correct location of pseudo-grid.   Medium
Local AZ/EL coordinate grid Do not rotate image, rotate grid.   Low
Overlay multiple grids Cannot rotate image since each grid may have a different orientation.   Low
Overlay markers from standard files Control parameters for symbol, size, position, and test. Positions should be in pixels, relative pixels, relative coordinates, or world coordinates.   Low
Layering datasets Can be interactively colormapped, switched on/off, registered/unregistered, and overlaid with controllable transparency.   Low
Overlay functional fits How and where to specify functions?   Low
Overlay data with different coordinate systems Choose the coordinate system and size can be chosen and the data transformed appropriately before visualization   Low
Shift, rotate, and scale images interactively How and where to specify this functionality?   Low
Holographic data If the beam shape is already in CASA image format, done   Low
MSes
Issue Details JIRA Ticket(s) Priority
Choose different time and velocity/frequency units From task interface or GUI?   Medium
Plot real time axis Now, only row numbers are plotted.   Medium
Plot uv distance axis Now, baselines are sorted to ~ uv distance.   Medium
Plot frequency/velocity axis Must sort out how to calculate values in different frames. Must somehow specify frame.   Low
Plot u or v axes Not yet implemented.   Low
Plot azimuth and elevation axes Not yet implemented.   Low
Plot hour angle axis Not yet implemented.   Low
Plot parallactic angle axis Not yet implemented.   Low
Plot different flag types Must wait until EVLA and ALMA sort this out.   Low
Plot delay and delay rate Not yet implemented.   Low
Plot weights Wait until weights are sorted out EVLA and ALMA.   Low
Plot Tant and/or Tsys Must wait until EVLA and ALMA sort this out.   Low
Selection by name for antennas, correlations, etc. Already done from the task interface, must do for GUI.   Low
Interpolate/extrapolate any data type onto visibility or calibration solution point. Need much more information.   Low

Plan

Boilerplate

Darrell, Daren Wilson, and Honglin are the members of the casaviewer team. Darrell is the technical lead. Apart from Nick, he has the final say in team management. Honglin may be called upon to do more HPC work, so he may not contribute as much to the casaviewer effort in the near term. Daren is our recent hire, so he will be spending the first few months ramping up his expertise on casaplotms. Over the long term, he will be spending ~ 0.5 FTE on casaviewer and ~ 0.5 FTE on casaplotms.

Since plans change as we learn new things, weekly or biweekly casaviewer technical meetings will be required. Nick recommends that Darrell organize and lead a meeting on Friday afternoon at 1400 MT (1600 ET), if required. This time is already reserved for CASA technical meetings and it has hardly been used during the last year or so.

The "plan" is essentially the spreadsheet of open casaviewer JIRA tickets. The should be handled ~ top to bottom, but issue type and short-/long- term plans should be taken into account. Nick has created the first version of this document with approximate priorities. They can be changed at any time. Darrell is responsible for maintaining the spreadsheet. When a new casaviewer ticket is submitted, he and Nick will decide its relative priority, then Darrell will insert it with a red background to alert management that it is new ticket that may need reprioritization by management.
Development Cycle 3.1 (and Possibly Part of Development Cycle 3.2)

At the top of the spreadsheet, we have put the most serious bugs. We believe that they can be solved while we deal with CAS-2243, CAS-2244, CAS-2246, and possibly CAS-2248. These JIRA tickets represent our major effort to split casaviewer into msview and imview.

The splitting work involves:

  1. Finalizing the designs for the task interfaces of imview and msview (CAS-2244, CAS-2246; CAS-2248 may be a dupe depending on how this work proceeds and whether Daren has time during the 3.1 development cycle). The msview interface should be similar to casaplotms. We need input and direction for CSSC. We may not be able to implement all input parameters immediately, but it would be good to have a list so that we know what to do. We create subtickets, if needed.
  2. Refactoring the ergonomics of the GUIs (also CAS-2244, CAS-2246). The existing GUI is mostly OK, although it could use a little reorganization (and eventually better help files!). Since we are going to split the GUIs for images and MSes, the refactoring will happen anyway. We need drawings of what the GUIs and subGUIs look like as well as the behavior of each widget. These drawings must take into account existing and future functionality. In other words, existing functionality should be immediately present in the new GUIs and hooks should be available for new functionality (from the JIRA ticket spreadsheet and Section 9). The hooks may be as simple as leaving spaces for new buttons or having a drop-down menu that can be extended. Therefore, care should be taken to group widgets in a coherent manner.

The other JIRA tickets in the spreadsheet and the desired features listed in Section 9 may or may not be tackled during development cycles 3.1 and 3.2. They will be handled in order unless we determine that a different order would be more advantageous or we receive directives from management.

ESO Field Spectrometers

ESO needs a good three-dimensional (one plane at a time) visualization tool for the field spectrometers. Pascal Ballester and Dirk Petry have decided that casaviewer is the best choice. They have already written a requirements document, and they will come up with a plan which will be coordinated with CASA management. Two major requirements are being able to handle wavelengths (in place of frequencies or velocities) and the incorporation of extendable plugins. There is already a JIRA ticket which is a subset of the plugin requirement.

Martin Kümmel has been hired by ESO to perform this work. We would like Martin to act as another CASA developer, with his own personal build and check-in privileges. He will be subject to the same rules that Nick has set for all CASA developers, and we will ask him to help with testing (Section 11). The only major procedural question is how development should proceed. There are two possibilities: 1) Martin works on the imview task with Darrell, Honglin, and Daren; or 2) Martin works on an temporary parallel task (esoview?) which will eventually be merged with imview. We will decide with path to take in the very near future.

Testing

Apart from creating hardcopy plots using the task interface with the nogui parameter, casaviewer cannot be tested with the CASA unit and regression testing systems. We have a "script" on CASAIndex which contains a sequence of commands that a tester executes. Some parts of the "script" are related to plotxy, and they will disappear when plotxy is removed from CASA. Laura Chomiuk has created a list of suggested additional "script" tests for casaviewer which should be implemented in the future.

The two most important aspects of testing are regularly maintaining and performing the tests on most supported platforms. Regularity insures less stress and less stress insures a better product. Maintenance is required because we will be continuously adding new features to the casaviewer, and it should be driven by completed JIRA tickets. If at all possible, Nick would like to avoid assigning these activities to CASA developers, not only because it reduces their precious development time but because good software engineering practice dictates that detailed testing should be done by independent testers (this statement does not relieve the developers of their duty to perform preliminary tests). We will depend on ad hoc testing by users (bugs and improvements will magically appear via the help and JIRA systems), but we need organized and focused testing before releasing versions to the community. Therefore, we will solicit help from EVLA and the ALMA/NAASC for maintaining and performing the tests.

Ideally, maintaining and performing tests should be done before the stable branch is updated. This schedule is clearly too demanding since the testers will have other duties. The best compromise would be to schedule these tasks during the month-long feature freeze immediately before every release. Review of casaviewer JIRA tickets for maintaining the test scripts would be the first order of business, followed by the actual testing itself. If possible, the testers should insure that the unit and regression tests exercise casaviewer from the task interface.

We may receive help for casaviewer testing. Since incorporating ESO requirements into casaviewer will require a lot of testing and the existing CASA developers do not have sufficient time to do it, I will ask Martin to perform the testing before stable branches are created (there will still be other testing from external users) and maintain the testing "scripts." He will create and maintain unit or regression tests with the nogui parameter as well. Asking Martin to take on casaviewer testing also has the advantages of him learning more about casaviewer and how it fits into CASA. We will discuss tall of hese issues when Martin begins development.

Ongoing Work

Separation of casaviewer into imview and msview

This task has been performed and the ticket (CAS-2234) resolved.
Task interface for imview

This task has been performed and the ticket (CAS-2246) resolved.

Future

Radio astronomers are used to viewing image cubes one plane at a time. This near-3D visualization will allow us to complete other pressing tasks before full-3D visualization. Honglin has created a working 3D viewer prototype. It has not been tested with large datasets. Eric Feigelson (Penn State) has submitted an NSF proposal for finding features in 3D space, which will be important for large datasets.

Darrell supports implementation of new semi-independent pieces, like the spectral profile tool, in a way that will allow them to be used from a possible future reimplementation of the casaviewer. When it is useful to have these independent GUIs in another process, control and communication should be supported via DBus. We should also consider starting a lower-priority visualization research project which would investigate implementing a unified set of visualization tools that would support all of our data exploration needs, e.g., 1D plots of spectra, 2D plots of rasters, and 3D plots and cubes. This toolkit should target responsiveness, correctness, and where appropriate, parallelization (OpenMP, threads, and GPUs). The goal should be useful tools rather than reimplementing the functionality already provided by the casaviewer. Instead of reimplementing or expanding the display library, this set of tools should be implemented directly in Qt or Qwt or other Qt based tools (as we have already done with tools which supplement casaviewer functionality). Qt is already a cross-platform library which is well implemented and well supported, so we should use it.

Performance Tests from Laura Chomiuk -- 2010 May 01

Here are some times it took to load various data sets into viewer. They were all run on the LINUX machine zaurak. I'd like to find a ~20 GB dataset, as it looks like some non-linearity sets in between 7GB and 40GB, but I haven't found one yet.
Dataset SizeSorted descending Load time
Leo Data-- complete data set 920 MB ms ~6 seconds to load into viewer.
Leo Data-- science source split off 338 MB ms ~3 seconds to load into the viewer.
AS1013 run 2 (AS1013_sb1215545_1.55285.42343756944) 13 GB sdm / 40 GB ms 29 minutes to load into viewer.
WicLoop1 Data 7.0 GB ms 2 minutes to load the data.
AC934 Data (old VLA) 3.2 GB ms 1 minute to load the data.
WicLoop1 Data-- one source split off 2.8 GB ms 45 seconds to load the data.
AC934 Data (old VLA)---science source split off 2.3 GB ms 44 seconds to load the data.

Every time I needed to 'load MS vis. data' (e.g., selecting only a subset of SPWs to be displayed, the first time i changed 'data maximum), it takes ~3 minutes to load. Typically ~30s to save an edit. Some saves are shorter (more like 5s). Switching between 'visibility components' (what pixel brightness represents) takes ~30--45 seconds: Amplitude (~30s), Phase (~45s), Real (~20s), Imaginary (~20s). It takes no time to switch between Amplitude, Amp Diff, Amp RMS. On the bright side---it takes no time to switch between plotting baseline vs. time (TVFLG style) and plotting channel vs. time (SPFLG style). It also takes no time to switch between correlations or spws.

-- DarrellSchiebel - 2012-03-30

Topic attachments
I Attachment Action Size Date Who Comment
image00.pngpng image00.png manage 126 K 2012-03-30 - 14:35 DarrellSchiebel CASA Viewer Display Panel
image01.pngpng image01.png manage 10 K 2012-03-30 - 14:36 DarrellSchiebel Dependency Diagram
image02.pngpng image02.png manage 5 K 2012-03-30 - 14:37 DarrellSchiebel Dependency Diagram
image03.pngpng image03.png manage 5 K 2012-03-30 - 14:38 DarrellSchiebel CASA Viewer Architecture
image04.jpgjpg image04.jpg manage 37 K 2012-03-30 - 14:38 DarrellSchiebel Decision Tree
This topic: Software > WebHome > LegacyCASA > ObtainingCASA > CasaIndex > CasaViewer > CasaViewer_2010_Overview
Topic revision: 2012-03-30, DarrellSchiebel
This site is powered by FoswikiCopyright © by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding NRAO Public Wiki? Send feedback