Splatalogue


This is the Splatalogue Web

This page is maintained by Anthony Remijan and provides information on current development program of Splatalogue (available at www.splatalogue.net ).

Documentation

Notes from Paul Frieden's visit - Nov 21-22, 2011

Objective:

To discuss further development of the Splatalogue front page and interfacing the Splatalogue database to allow displays of existing spectral line data and to make simulated spectra based on user defined inputs and the spectroscopic parameters contained in Splatalogue.

Milestones:

  1. NRAO will take control of the maintenance and development of the existing, primary Splatalogue homepage written in php by the end of Q1 FY12. Dongchan will take over control of the Splatalogue website and PFrieden will continue to provide offsite help when and where appropriate. This is a NO COSTservice.

    • This was complete as of Q1 FY12. Furthermore, we are now offering new options to navigate the spectral lines available via Splatalogue. The user community has suggested a simpler, more efficient way of searching for and obtaining the more common spectral line features from the radio to submillimeter wavelength.
      • The Splatalogue search page has several new and quick search features including:
        1. The Quick Picker: Located on the far left. Popular species are included. Click on your favorite, hit search and the results will pop up. You can also limit the frequency by entering in your preferred range in GHz.
        2. Search Bar: Located in the center of the page. Type in the name (or in some cases, the formula) of your favorite molecule and all species with that molecule name, including isotopologues of that species, will be displayed. Again, you can limit the frequency displayed by entering in your preferred range in GHz.
        3. Telescope Band Search: Located at the center of the page. This feature allows users to search molecules by telescope bands of the GBT, Jansky VLA, and ALMA. Instead of limiting your search by typing in a specific frequency range, you can choose your favorite telescope band of interest. NOTE: the current version only allows searching one band at a time!
        4. Astronomical Filters: Located on the far right. This option allows you to limit your search to the species currently known within certain astronomical environments. Also available is the "Top 20 list" which is the same as in the ALMA OT. When selected, the Top 20 species will be displayed. You can also choose your own desired frequency range here as well to limit the output.
    • However, you can always return to the original Splatalogue homepage but clicking on the "Advanced" tab from the basic page. This view will become the new Splatalogue home page.
  2. DataSplat Development is now its own section posted below...
  3. SLiSE Development is now its own section posted below...
  4. We will do a full rebuild of the Splatalogue homepage from php into JAVA for a) ease of future enhancement and interactivity with the delivered data and b) enhanced security of the service. PFrieden will create a servlet back-end code in JAVA to build a database wrapper. This servlet will later pass results to a front-end presentation layer. Dongchan in the short term will continue the enhancements and maintenance of the php front page (see #1) but will eventually migrate the php to utilize PFrieden's servlet results. The new JAVA enabled Splatalogue frontpage will be delivered to NRAO no later than the end of Q2 FY13 at a COST OF: .

List of Enhancements to existing Front Page/JIRA ticket/Responsibility (Lead/Help1/Help2/Help3, etc...):

  • Develop a new "basic search" page based off the existing Splatalogue quick picker. This page will contain the Splatalogue quick picker, a table of the "most popular" and molecular source, a "Google-like" search bar that will enable string searches./SPLT - 23/(Dongchan/Remijan)
  • From the existing Splat homepage which will now be considered the "Advanced Search" - add a "Google-like" search box above the molecule list that will "jump to molecule./SPLT - ???/(Dongchan/Remijan)
  • Add a feature to the mass calculator that will "jump to" the first molecule in that mass range./SPLT - 9/(Dongchan/Pulliam)
  • Dump the metadata displayed in the pop-up window to a file./SPLT - 20/(Dongchan)
  • Use "receivers" table to enable colors on the main splat return page - customize for GBT, EVLA, IRAM, etc.../SPLT - ???/(Frieden/Dongchan)
  • Develop a new "RFI" table and determine the relevant columns for that table./SPLT - 33/(Frieden/Pulliam)
  • Add new columns to "main" to enable searches by transitions or QNs. Also needed for -VAO-SSLDM (e.g. WEEDS package)./SPLT - 40/(Pulliam/Dongchan)
  • Sort columns in main frame (e.g) E, Aij, Sij)./SPLT - 26/(Frieden)
  • Add indexes to tables in database for ease of searching./SPLT - ???/(Frieden/Pulliam)
  • Display sources of U-lines./SPLT - ???/(Dongchan/Remijan)
  • Display data of last update from CDMS/JPL off main table./SPLT - ???/(Dongchan/Remijan/Hunter)
  • Make GHz default search criteria./SPLT - 36/(Dongchan/Remijan)
  • IOffer enhanced search/display criteria including wavelengths./SPLT - 35/(Dongchan/Remijan)I
  • Save parameters from last search using browser cookies./SPLT- ???/(Dongchan/Sharp)
  • Work on development of m.splatalogue.net./SPLT - 37/(Kent/Remijan)
  • IEdit quick picker display to only return orderedfreq./SPLT - 38/(Dongchan/Remijan)I
  • Setup new test server area for Splatalogue development at webtest.cv.nrao.edu./SPLT - 34/(Dongchan/PMurphy)
  • Setup a "splat" repository on the gitdev server so Dongchan and PFrieden can check in the code for the Splatalogue homepage./SPLT - 24/(Witz/Murphy/Dongchan/Frieden)
  • In-source the splatalogue.net domain name to NRAO./SPLT - 39/(Remijan/Malone/Markwick)
  • IDevelop the DataSplat webservice./SPLT - 29/(Remijan/Kent/Frieden)I
  • On splatalogue basic page, typing "carbon", "oxygen", or "nitrogen" displays atomic carbon, atomic oxygen, or atomic nitrogen and all respective isotopologues./SPLT - 30/(Dongchan/Pulliam/Remijan)
  • Implement a pull down menu on the splatalogue basic page to allow the user to select specific telescope bands frequency searches./SPLT - 31/(Dongchan/Pulliam/Remijan)

DataSplat Development

  1. We will develop a new Python module called DataSplat that will allow users to interact with the Splatalogue database and enable the generation of simulated spectrum of molecules using user defined input parameters including temperature, column density, source size, beam size, line width, and filling factor. The initial requirements for DataSplatinclude:
    • Talk with Splatalogue/Query Requisite Parameters
    • Take user parameter input as XML table using VAO protocols with defined set of parameters
    • Perform data conformation from Splatalogue and User inputs
    • Return XML table with intensity, frequency and "Delta frequency" (linewidth in frequency units)
    • Extendable - use with a variety of packages and platforms
    • Make all Splatalogue inputs "transparent" - (I don't remember what this means)

A requirements document will be written to explain each step of DataSplat but a graphical representation of the workflow is shown below:

DataSplatFlow.png

This Python module will take user inputs as an XML document, perform the appropriate operations and then return an XML document, possibly using VOTables. As such, it can easily be interfaced with many reduction packages including the CASA viewer through a socket interface.

Work on the development of DataSplat by PFrieden will take place starting Q2 FY12 and will be delivered to NRAO no later than the end of Q3 FY12 at a COST OF: . PFrieden will work with DTody and BKent for input/insight into the VAO table protocols and will work with RPulliam and Remijan for scientific input and interface with the Splatalogue database.

  • DataSplat Status from B. Kent. The code is currently being tested. I'm using git for version control and hosting it here: https://bitbucket.org/brkent/datasplat.
  • Pull requests can be made via:
    git clone https://bitbucket.org/brkent/datasplat.git 
    .
  • We will keep the splatalogue SQLite database within SVN. Checkout requests can be made via:
     svn co https://svn.cv.nrao.edu/view/splatalogue/ 
  • DataSplat has three modes of operation. 1) Command line (see python datasplatclass.py -h) with various input and output switches. 2) Socket mode for listening on a given port (future use with CASA or Slise) and 3) Interactive mode for use at an IPython prompt. 4) As a CASA task
  • Current developments include adding calculations for an interferometer (DONE), using the CASA offline version of splatalogue(DONE), writing unit tests, and completing documentation.

SLiSE Development

  • In addition to the DataSplat service, PFrieden will also edit the existing JFreeChart interface of SLiSE to conform to the new DataSplat web service. This will allow us to put actual spectral line data off the Splatalogue homepage from personal archives and will allow users to import their own data for display and line simulations. The new SLiSE interface will be delivered to NRAO no later than the end of Q3 FY12 at a COST OF: .
  • The following list out lines the requirements of SLiSE in no priority order. Prioritization can come after formal discussion with developers.
  • Paul's original response is in red. Follow up to Paul's response is in blue.
  • Anticipated SLiSE Plan of work:
    CropperCapture98.png
  • Data Input/Output
    1. Needs to be able to interface with the Splatalogue database
    2. Needs to interface with the "Remijan" archived line lists from direct mysql queries. I thought these were coming through a web service... (I think here what I really meant: The user should be able to upload telescope data they have so that it may be made public for other users to have access too. Not to mean that users can store any and all files they want. It was my understanding that this was originially a goal of SLiSE (in terms of outside users providing their observations to be made available for all.)
    3. Must be able to pass user defined XML entries to DataSplat; read XML output from DataSplatandbeable to display the results on the plot. This uses XML internally, but I don't think the XML would ever be visible to the user. (Correct, the XMl is used for easy passing of information, not for the user to fumble with.)
      • must be able to edit the XML entries easily and repass to DataSplat
      • must be able to pass multiple XML entries for each molecular species
      • must be able to plot the results of each returned result from each species from DataSplatandplotindifferentcolors
        • if possible, make a "composite" spectrum of all the calculated individual results (perhaps DataSplat can do that and pass to SLiSE?) The web service will only be returning "sticks", so this will have to be generated client side. Correct, the composite spectrum should be generated in the app such as SliSe.
    4. Option to save the displayed data for export into a plotting package for a publication quality plot
    5. Need to be able to print the displayed plot
    6. Need to be able to output all data to an ascii file (including the simulated spectra): intensity, velocity and frequency should all be included at a minimum. We'll have to talk about how to format this. There could be multiple data series, so we'll have to either output multiple columns or multiple files. For me, I think multiple files would be best to distinguish between simulated spectra and observations; unless we label the columns.
    7. Need to be able to import own data either temporarily or for permanent storage. See below for questions about permanent storage. Claified below.
  • Display
    1. Need to change the x-axis between frequency (MHz and GHz) and velocity (km/s) with appropriate reference frame on the plot
    2. Need to be able to change the y-axis intensity scale units from K, mK, J, μJ on the plot
    3. Plot needs to be interactive with zooming in and out with the mouse
    4. Need to be able to zoom in on regions as well as zoom back out within the plot
    5. Need to have an option to clear all markers on the plot
    6. Need to be able to overlay a stick spectra of all known molecules We need to discuss how these are going to work. All of this data is going to need to come from datasplat or some other source, so we'll need to define where it comes from. Right now Brian Kent is doing it with matplotlib - pretty simple. The idea is that the datasplat python
    7. Need to be able to overlay a stick spectra of molecules specific to certain regions of space (comet, hot cores, etc.) As above (as above as well)
    8. Need to be able to overlay a stick spectrum of a specific molecule As above (as above as well)
    9. Need to be able to overlay several data files (in different colors) if desired (for telescope and laboratory comparisons)
    10. Option - ability to add plot annotations I'm not sure how this will interact with JFreeChart. JFreeChart is a PHP library. Since Paul will code in Java he will likely not use this.
  • GUI / Manipulation Controls
    1. Need to have options to apply marks and lines to the plot
      • option to click on the plot and add annotations
      • option to click and place a vertical line with coordinates printed
      • option to click and just place a marker (x) with coordinates
    2. Need to be able to choose frequency range want displayed. Or choose a center frequency with a certain ± bandwidth.
    3. Need to be able to print the displayed plot
    4. Need to be able to fit a gaussian or multiple gaussians to the spectral features We need to be careful with terminology here...Tony has said that we specifically are not fitting. Do not necessarily mean "fitting" here. Would like to be able to overlay a gaussian to a selected spectral line and have parameters such as peak intensity and center frequency/velocity returned as well as line width.
    5. Would be nice to be able to apply hanning or boxcar smoothing to the data
    6. Option to save displayed plot with all markers present as a .ps or .pdf in a publication quality plot
    7. For permanent storage, need a separate section in the upload where the user can type in information about their data. When the particular data is selected after upload this information will be displayed. Example of what will be included:
      • data type (telescope, laboratory)
      • frequency coverage
      • instrument (telescope or lab instrument)
      • spectral resolution if known
      • source velocity
      • ?
      • How do you see permanent storage working? This question come up earlier too. I think that users would provide their own data files in most cases, and so they can handle storing them. Users shouldn't ever have to upload their data to the server to display it. That doesn't preclude online data sources, but they aren't required. (I think here what I really meant: The user should be able to upload telescope data they have so that it may be made public for other users to have access too. Not to mean that users can store any and all files they want. It was my understanding that this was originially a goal of SLiSE (in terms of outside users providing their observations to be made available for all.)
    8. Would be nice if the source velocity of the data could be changed (?) I'm not sure what this means. smile (The doppler effect. I would like to be able to shift the frequency freely by a certain factor based on a new velocity of my choice as compared to a reference velocity that was set when the observations were obtained ).
    9. Would be nice if a simple baseline could be fitted and then subtracted out (?)
    10. Need an option to determine the noise level of the spectrum

Linelists web service

  • A simple RESTful type web service has been created with four PHP classes to facilitate the generation of XML file from the linelists.source and linelists.source_data tables.
  • The service URL can be found at http://www.cv.nrao.edu/php-internal/bkent/linelists/linelists.php?sourceid=14 with the main parameter right now being the 'sourceid'.
  • This service can be called, for example, by a simple client application. Brian has written one in python as a small CASA task. It calls the service, parses the XML file, and plots the spectrum. Currently the results are limited to about 4000 channels for testing purposes. The CASA task and XML are attached below. Pop the task and XML file into a directory, start CASA, and run os.system('buildmytasks') followed by execfile('mytasks.py').
  • Example linelists task for CASA (remove .txt as added by the wiki)
  • Linelists XML file required for creating a CASA task.

Topic attachments
I Attachment Action Size Date Who Comment
CropperCapture98.pngpng CropperCapture98.png manage 102 K 2012-07-16 - 14:19 AnthonyRemijan  
DataSplatFlow.pngpng DataSplatFlow.png manage 65 K 2011-11-23 - 08:36 AnthonyRemijan  
GBT_Molecule_Upgrades_FINAL.pdfpdf GBT_Molecule_Upgrades_FINAL.pdf manage 60 K 2013-01-28 - 11:37 AnthonyRemijan  
SplatQuickUserGuide_v2.1.pdfpdf SplatQuickUserGuide_v2.1.pdf manage 1 MB 2012-04-16 - 14:33 BrianKent Splatalogue quick user guide
SplatalogueMemo_v2.1.docdoc SplatalogueMemo_v2.1.doc manage 1 MB 2012-04-16 - 14:33 BrianKent Splatalogue Memo
linelists.xmlxml linelists.xml manage 1 K 2012-12-10 - 11:15 BrianKent Linelists XML file required for creating a CASA task.
sliseworkplan.odsods sliseworkplan.ods manage 29 K 2012-10-10 - 11:16 UnknownUser Anticipated SLiSE plan of work
task_linelists.py.txttxt task_linelists.py.txt manage 1 K 2012-12-10 - 11:23 BrianKent Example linelists CASA task
Topic revision: r42 - 2014-03-11, AnthonyRemijan
This site is powered by FoswikiCopyright © by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding NRAO Public Wiki? Send feedback