Notes from ADASS XXI

General Notes

  • Lots of whining about too much data, but we're starting to see emergence of DM applications.
  • LOFAR Scheduling: spoke with the single developer for the LOFAR scheduling system
    • no weather; scheduling weekly sciences goals; operator schedule changes; service observing; no carry over (same for EVLA).
  • Sharing DSS with EVLA: descussed sharing the scheduling/scoring algorithms developed for DSS with EVLA; perhaps even sharing antioch; Brian would like to discuss this some more; win for GB I think
  • Everyone is trying to unify immplementation; building tools and frameworks for sharing systems; porting legacy systems to current architectures; generalize codes to support different instruments and data formats; etc. Would it be better to document algorithms and include test cases that can be used to validate the implementation?
  • Many generalized tools already exist for data mining; especially in R.


Functional programming for real world applications: monitoring and control of the UniBoard using Erlang

  • Perhaps useful for testing VEGAS? Developer willing to share code

Tuesday (Session 5) GPUs

Benchmarking the CRBLASTER Computational Framework on a350-MHz 49-core MAESTRO Development Board

Unleashing the Power of Distributed CPU/GPU Architectures: Massive Astronomical Data Analysis and Visualization case study

  • One core dedicated to network communications; another for scheduling using a custom message queue
  • Communication between compute nodes through shared memory.
  • Limited by the communication overhead; needs to be improved.

AAOGlimpse - Fun with OpenGL and FITS files

  • A FITS image display program 2D -> 3D
  • Uses wcslib for WCS support
  • Goes from 2D to 3D by stacking the 2D.
  • Good for looking closely at your data.
  • Status: OS X executable.
  • The guys next to me really liked the talk. ROTF

GPUs and Python: A Recipe for Lightning-Fast Data Pipelines

  • Uses PyCUDA.
  • Modified existing Python pipeline (FATBOY) to use GPUs.
  • PyCUDA: Driver class handles memory; execute code on GPUs through function calls from Pythons. Takes code string as argument.
  • Order of magnitude speed up on 1-D and 2-D median calculations. Relative to numpy.
  • Wants to do the same for spectroscopy algorithms -> Super-FATBOY.

Tuesday (Session 6) VO

The IVOA Architecture

  • International Virtual Observatory Alliance (IVOA)

Tuesday (Session 7)

Modeling Physical Quantities and Measurement Sets

  • Way over my head, I was not alone.

The challenges of new observing/operating modes at ground based optical observatories

  • In the past, observers come to visit the telescope.

Tuesday (Session 8)

The ALMA Observing Tool, experiences from Cycle 0

  • 9:1 over subscription rate
  • server crashed 45 minutes from the proposal deadline
  • never under estimate the imagination of the users
  • Feb 2012 call for cycle 1

Wednesday (Session 9)

Knowledge discovery workflows in the exploration of complex astronomical data sets

  • Automatic transit detection using decision trees
  • Extraction of optical candidate quasars with clustering
  • The Weak Gated Expert (WGE) neural networks
  • Clustering-Labels-Scores Pattern Spotter (CLaSPS)
    • Applied to any unsupervised cluster
    • Can be used to find ACNs.
  • tools: Python, STILS, R (Knowledge Discovery algorithms)
  • future: IVOA, KDD IG, VAO science priority

VisIVO, a library and integrated tool for large astrophysical dataset exploration

  • Visualisations on the web
  • users upload data to the tool (large data?)
  • underlying library available

Wednesday (Session 10)

R (Feigeison, Eric Penn State)

  • R/CRAN took off in 2005
  • C-like language, IDL
  • Difficult to find what you are looking for because there are so many packages
  • Neural network packages
  • Even more packages in CRAN
    • Bayesian, classification ...
  • find packages:
  • not as powerful for imaging
  • R ingests subroutines from other languages
  • C, Fortran, Python, and Ruby can ingest R functions
  • CRAN package devoted to data streaming
  • R Summer School in stats for astronomy
  • Book: Modern Statistical Methods for Astronomy with R
  • Publications on Astrostatistics at

Data Mining on Ice

  • Ice Cube detector (neutrino detector)
  • 2600 attributes
  • DM environment RapidMiner (Java)
  • Jaccard index!
  • Used Random Forest (supervised) for classifiers
    • out performed Naive Bayes

Wednesday (Session 11) Long-Term Preservation of Analysis Capabilities

Long Term Preservation of Data Analysis Software at the NASA/IPAC Infrared Science Archive

  • Retain institutional memory
  • Regression testing; store benchmarks results
  • Upgrade when possible; when resources allow
  • Leverage connections and multiple projects

Space Missions: Long term preservation of IDL-based softwares using GDL

  • GDL is a free clone of IDL; fully syntax compliant with IDL 7
  • Offers a regression test suite
  • Written in C++; interfaces with Python
  • Author has been using IDL for 20 years
  • Looking for contributions.

Challenges in Code Discoverability

  • Astronomy Software CL ASCL version 2
  • Catalog of all the astronomy software that is available
  • Widen the scope? Repository? Part of the IVOA?

Wednesday (Session 12) Long-Term Preservation of Analysis Capabilities

IRAF: Lessons for Project Longevity (NOAO)

  • Negative comments from twitter
  • 1981 - Present; from Vax systems to iPad (sort of)
  • Get your design right from the beginning
  • Replacement Challenges: community projects -> dedicated SW -> toolboxes
  • aggregate rather than design
  • concentrate on glue code and migrating systems to new architectures

A Future Astronomical Software Environment (FASE) Poster 109

  • Why a common evn? Lists various reason why not. Everyone must agree.
  • Requirements
  • An architecture/framework for unifying software systems
  • Prototype available online
  • Argued for a common API

Astronomical Data Processing Using SciQL, an SQL Based Query Language for Array Data

  • monetdb - open-source database system designed for data mining
  • Why not RDBMS? SQL is difficult, said scientist. Its too slow
  • SciQL - an array query lang. based on MSSQL:2003. 2011 VLDB award!
  • create array structures rather than tables
  • built in statistical operations on arrays; integrated with a query language
  • demonstrated a LOFAR use case

Thursday (Session 13)

Linking to Data – Effect on Citation Rates in Astronomy

  • Making it easier for literature to reference data.
  • roughly 20% more citations with data links than those without.
  • Paper: Sharing detailed research data is associated with increased citation rate A.
  • Data sharing is a multi-discipline discussion
  • Possible to link to astronomical software database.

Thursday (Session 14)

SAMP: Application Messaging for Desktop and Web Applications

  • Enables cross site access to applications
  • Provide SAMP message with a URL.

Displaying digital deep sky images

  • Problem: dark background in optical images
  • Limited dynamic range: must fit ~90dB on 8 bits; "Stretch functions"
  • Apply gamma correction to clean up images and for quality control
  • Calibrate your display: right LCD (IPS or MVA/PVA); digital connection cable; use gamma calibration chart
  • STIFF: converts FITS images to TIFF;
    • works with hugh images (24GB); additional luminance gamma correction; etc...

HELIO: A Heliospheric Virtual Observatory

  • Solar and planetary VO.
  • Speciallizes in solar system events such as solar flares

The ADS All-Sky Survey (CfA)

  • Extracting data embedded in papers and make it available in viewers; tables, graphs, images, etc...
  • Astrotagging: connecting a paper from an image
  • astroreference: object coordinates, orientation and pisxel scale; article timestamp
  • astrotagg using source name and wavelength in the caption.
  • facilitates community annotations
  • planing to build a database that will go back 100 years

WorldWide Telescope in (Real) Research, and in Education

  • Jim Gray (& Alex Szalay) paper in 2004 on WWT
  • After Jim's disapearance Bill Gates gave Jonathan Fay and Curtis Wong (MS research) permission to work on WWT as a tribute to Jim. * ; Data coverage tool
  • "Seamless Astronomy" tools: span literature and data
  • WWT Ambassadors: WWT for interactive learning; ambassadors are astronomers trained in WWT who make tours in WWT; some go to schools to teach kids; "Cooler than 'Call of Duty'!", school kid.
  • 100,000 of participants
Topic revision: r2 - 2011-11-13, MikeMcCarty
This site is powered by FoswikiCopyright © by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding NRAO Public Wiki? Send feedback