Notes from ADASS XXI

General Notes

• Lots of whining about too much data, but we're starting to see emergence of DM applications.
• LOFAR Scheduling: spoke with the single developer for the LOFAR scheduling system
  • no weather; scheduling weekly sciences goals; operator schedule changes; service observing; no carry over (same for EVLA).
• Sharing DSS with EVLA: descussed sharing the scheduling/scoring algorithms developed for DSS with EVLA; perhaps even sharing antioch; Brian would like to discuss this some more; win for GB I think
• Everyone is trying to unify immplementation; building tools and frameworks for sharing systems; porting legacy systems to current architectures; generalize codes to support different instruments and data formats; etc. Would it be better to document algorithms and include test cases that can be used to validate the implementation?
• Many generalized tools already exist for data mining; especially in R.

Posters

Functional programming for real world applications: monitoring and control of the UniBoard using Erlang

• Perhaps useful for testing VEGAS? Developer willing to share code

Tuesday (Session 5) GPUs

Benchmarking the CRBLASTER Computational Framework on a350-MHz 49-core MAESTRO Development Board

Unleashing the Power of Distributed CPU/GPU Architectures: Massive Astronomical Data Analysis and Visualization case study

• One core dedicated to network communications; another for scheduling using a custom message queue
• Communication between compute nodes through shared memory.
• Limited by the communication overhead; needs to be improved.

AAOGlimpse - Fun with OpenGL and FITS files

• A FITS image display program 2D -> 3D
• Uses wcslib for WCS support
• Goes from 2D to 3D by stacking the 2D.
• Good for looking closely at your data.
• Status: OS X executable. http://asterisk.apod.com/viewtopic.php?f=35&t=25585
• The guys next to me really liked the talk. ROTF

GPUs and Python: A Recipe for Lightning-Fast Data Pipelines

• Uses PyCUDA.
• Modified existing Python pipeline (FATBOY) to use GPUs.
• PyCUDA: Driver class handles memory; execute code on GPUs through function calls from Pythons. Takes code string as argument.
• Order of magnitude speed up on 1-D and 2-D median calculations. Relative to numpy.
• Wants to do the same for spectroscopy algorithms -> Super-FATBOY.

Tuesday (Session 6) VO

The IVOA Architecture

• International Virtual Observatory Alliance (IVOA)

Tuesday (Session 7)

Modeling Physical Quantities and Measurement Sets

• Way over my head, I was not alone.

The challenges of new observing/operating modes at ground based optical observatories

• In the past, observers come to visit the telescope.

Tuesday (Session 8)

The ALMA Observing Tool, experiences from Cycle 0

• 9:1 over subscription rate
• server crashed 45 minutes from the proposal deadline
• never under estimate the imagination of the users
• Feb 2012 call for cycle 1

Wednesday (Session 9)

Knowledge discovery workflows in the exploration of complex astronomical data sets

• Automatic transit detection using decision trees
• Extraction of optical candidate quasars with clustering
• The Weak Gated Expert (WGE) neural networks
• Clustering-Labels-Scores Pattern Spotter (CLaSPS)
  • Applied to any unsupervised cluster
  • Can be used to find ACNs.
• tools: Python, STILS, R (Knowledge Discovery algorithms)
• future: IVOA, KDD IG, VAO science priority

VisIVO, a library and integrated tool for large astrophysical dataset exploration

• Visualisations on the web
• users upload data to the tool (large data?)
• underlying library available

Wednesday (Session 10)

R (Feigeison, Eric Penn State)

• R/CRAN took off in 2005
• www.r-project.org
• C-like language, IDL
• Difficult to find what you are looking for because there are so many packages
• Neural network packages
• Even more packages in CRAN
  • Bayesian, classification ...
• find packages: cran.r-project.org/web/views
• not as powerful for imaging
• R ingests subroutines from other languages
• C, Fortran, Python, and Ruby can ingest R functions
• CRAN package devoted to data streaming
• R Summer School in stats for astronomy
• Book: Modern Statistical Methods for Astronomy with R
• Publications on Astrostatistics at asaip.psu.edu

Data Mining on Ice

• Ice Cube detector (neutrino detector)
• 2600 attributes
• DM environment RapidMiner (Java)
• Jaccard index!
• Used Random Forest (supervised) for classifiers
  • out performed Naive Bayes

Wednesday (Session 11) Long-Term Preservation of Analysis Capabilities

Long Term Preservation of Data Analysis Software at the NASA/IPAC Infrared Science Archive

• Retain institutional memory
• Regression testing; store benchmarks results
• Upgrade when possible; when resources allow
• Leverage connections and multiple projects

Space Missions: Long term preservation of IDL-based softwares using GDL

• GDL is a free clone of IDL; fully syntax compliant with IDL 7
• Offers a regression test suite
• Written in C++; interfaces with Python
• Author has been using IDL for 20 years
• Looking for contributions.

Challenges in Code Discoverability

• Astronomy Software CL ASCL version 2
• Catalog of all the astronomy software that is available
• Widen the scope? Repository? Part of the IVOA?

Wednesday (Session 12) Long-Term Preservation of Analysis Capabilities

IRAF: Lessons for Project Longevity (NOAO)

• Negative comments from twitter
• 1981 - Present; from Vax systems to iPad (sort of)
• Get your design right from the beginning
• Replacement Challenges: community projects -> dedicated SW -> toolboxes
• aggregate rather than design
• concentrate on glue code and migrating systems to new architectures

A Future Astronomical Software Environment (FASE) Poster 109

• Why a common evn? Lists various reason why not. Everyone must agree.
• Requirements
• An architecture/framework for unifying software systems
• Prototype available online
• Argued for a common API

Astronomical Data Processing Using SciQL, an SQL Based Query Language for Array Data

• monetdb - open-source database system designed for data mining
• Why not RDBMS? SQL is difficult, said scientist. Its too slow
• SciQL - an array query lang. based on MSSQL:2003. 2011 VLDB award!
• create array structures rather than tables
• built in statistical operations on arrays; integrated with a query language
• demonstrated a LOFAR use case

Thursday (Session 13)

Linking to Data – Effect on Citation Rates in Astronomy

• Making it easier for literature to reference data.
• roughly 20% more citations with data links than those without.
• Paper: Sharing detailed research data is associated with increased citation rate A.
• Data sharing is a multi-discipline discussion
• Possible to link to astronomical software database.

Thursday (Session 14)

SAMP: Application Messaging for Desktop and Web Applications

• Enables cross site access to applications
• www.ivoa.net/samp
• Provide SAMP message with a URL.

Displaying digital deep sky images

• Problem: dark background in optical images
• Limited dynamic range: must fit ~90dB on 8 bits; "Stretch functions"
• Apply gamma correction to clean up images and for quality control
• Calibrate your display: right LCD (IPS or MVA/PVA); digital connection cable; use gamma calibration chart
• STIFF: converts FITS images to TIFF; astromatic.net
  • works with hugh images (24GB); additional luminance gamma correction; etc...

HELIO: A Heliospheric Virtual Observatory

• Solar and planetary VO.
• Speciallizes in solar system events such as solar flares

The ADS All-Sky Survey (CfA)

• Extracting data embedded in papers and make it available in viewers; tables, graphs, images, etc...
• Astrotagging: connecting a paper from an image
• astroreference: object coordinates, orientation and pisxel scale; article timestamp
• astrotagg using source name and wavelength in the caption.
• facilitates community annotations
• planing to build a database that will go back 100 years

WorldWide Telescope in (Real) Research, and in Education

• Jim Gray (& Alex Szalay) paper in 2004 on WWT
• After Jim's disapearance Bill Gates gave Jonathan Fay and Curtis Wong (MS research) permission to work on WWT as a tribute to Jim. * www.worldwidetelescope.org ; Data coverage tool
• "Seamless Astronomy" tools: span literature and data
• WWT Ambassadors: WWT for interactive learning; ambassadors are astronomers trained in WWT who make tours in WWT; some go to schools to teach kids; wwtamabassors.org "Cooler than 'Call of Duty'!", school kid.
• 100,000 of participants