• Proposal 547: Neuroscope: Neural Machine Intelligence Tools for Discovery and Interpretation in Complex ALMA Data

  • Reviewer 5

Grade: 1

NeuroScope: Neural Machine Intelligence Tools for Discovery and Interpretation in Complex ALMA Data

1. Alignment with NA ALMA Partnership strategic goals;

This study is well-aligned with the ALMA NA strategic goals in that it aims to enhance the exploitation of ALMA data by implementing new modes of data analysis, beyond the common practices in radio astronomy. This will enable archival data to be better exploited to their fullest potential, in addition to new data.

2. Strength of the scientific case for the proposed ALMA upgrade concept; Comment on the relevance to the ‘ALMA 2030’ development documents.

Such software is important now and will only become more important in the future with the possibility of expanded bandwidths.

3. Quality of the upgrade conceptual design;

The conceptual design is of very high quality. The authors have already demonstrated proof-of-concept on a current ALMA dataset and there is a clear path to the use of these data on more complex data sets that the PI and scientific leads have access to. This is an excellent example of how tools developed for other fields can be employed in the analysis of complex radio astronomy data without reinventing the wheel.

4. Readiness for production in the context of the ALMA Development Plan (the aim is to support a range of upgrades including both those which can be implemented rapidly and those requiring longer-term research and development);

The authors already have the working prototype available and can begin to study its utility immediately. The software does not appear to be ready for implementation into the common user platforms, (i.e. CASA), but they plan to make the MATLAB protoype available to the community.

5. Strength of the consortium organization (if applicable);

Rice is a great organization to carry out this task with a strong astrophysics group and this is a cross-discipline study within the university.

6. Qualifications of the key personnel of the Study;

The key personnel are well-qualified to carry out this study. Andrea Isella is an expert in radio astronomy data from ALMA and other observatories. The PI Erzsebet Merenyi is the developer of the NeuroScope tool used initially for other NASA missions.

7. Technical expertise, past experience (also in series production, if relevant) and technical facilities in the Institutes taking part in the Study;

Isella has the radio astronomy expertise needed for such a project and Merenyi has the technical expertise with the NeuroScope tools in order to make this project a success.

8. Assessment of the level of risk inherent in the design;

The authors did not mention the risk associated with the tool not offering a superior method for analysis when applied to more complex data. This is an inherent risk as there are other esoteric methods that have been put forth over the years and have yet to see wide adoption. However, this is the purpose of the development program to help these ideas be tested to see if they are feasible for wide adoption.

9. Strength of the Scientific Team supporting the Study;

See comments above, team is excellent and well-qualified.

10. Level of support guaranteed by the Institutes;

Rice is contributing about 1/3 of the cost of the project and having an commitment from the institute at this level is very helpful for achiveing this project.

11. Budgeted cost of the Study;

The budget for this project is higher that others, but the complexity and software development needed for the tool to optimize if for radio astronomy may be greater than some of the other development studies that have a more narrowscope. Overall the level of commitment seems reasonable.

  • Reviewer 7

Grade: 2

Title: Neuroscope: Neural Machine Intelligence Tools for Discovery and Interpretation in Complex ALMA Data

1. Alignment with NA ALMA Partnership strategic goals; This proposal aligns with the strategic goals ● 1. Improve and extend technical capability by developing and implementing new software and computing technologies that improve and extend capabilities for scientific research (1.2) ● 5. Strengthen the North American Radio Astronomy community by training a PhD student who could contribute to the field in the future ( 5.2)

2. Strength of the scientific case for the proposed ALMA upgrade concept; Comment on the relevance to the “ALMA 2030” development documents. The specific science cases to be studied are a protoplanetary disk and a PDR. If the technique successfully allows discovery of results that would not have been easily learned by other means, then it could have wide applicability given that ALMA has and will continue to observe many PPDs. The ability to analyze data from many telescopes in a uniform and simultaneous way could be very enlightening. Though not immediately relevant to ALMA 2030, the proposal has potential relevance to Roadmap goal 1. Improvements to the ALMA Archive: enabling gains in usability and impact for the observatory. If the study shows wide applicability of SOMs for ALMA data, one could imagine them being generated automatically and stored in the Archive as part of the pipeline.

3. Quality of the upgrade conceptual design; This proposal is much improved since an earlier version was submitted to a previous Study call. The science is better motivated, the details of how NeuroScope/SOMs work is better explained, and they have compared the technique to existing standard analyses. They have picked two high profile example ALMA data sets that they are already familiar with since Isella is on the observing team for both. A “traditional astronomy” paper on the PPD data is in press. The tasks are reasonably described and there is a large FTE effort. One issue not addressed is how the technique might fare in the presence of imaging artifacts or non-Gaussian noise. It is asserted that they don’t need to know the RMS noise level, but are there underlying assumptions about the characteristics of the noise?

4. Readiness for production in the context of the ALMA Development Plan (the aim is to support a range of upgrades including both those which can be implemented rapidly and those requiring longer-term research and development); The study would produce example working code in MATLAB and C/C++. It would not be immediately ready for production; the authors suggest they would proposal a follow-on Development Proposal to create the production version.

5. Strength of the consortium organization (if applicable); NA

6. Qualifications of the key personnel of the Study; PI and co-I are highly qualified in their respective domains, both scientifically and technically, and will supervise a PhD student.

7. Technical expertise, past experience (also in series production, if relevant) and technical facilities in the Institutes taking part in the Study; Team has relevant technical expertise and experience to accomplish the proposed work.

8. Assessment of the level of risk inherent in the design; Risk is fairly low. They will either show their technique works well or not. Either way is a result.

9. Strength of the Scientific Team supporting the Study; See #6.

10. Level of support guaranteed by the Institutes; PI and co-I are each contributed in-kind time of ~2 FTE-months each.

11. Budgeted cost of the Study; Costs are reasonable, though is it unusual to ask for computers? In-kind contribution of ~88K is good.

  • Reviewer 10

Grade: 3.5

Title: Neural Machine Intelligence Tools for Discovery and Interpretation in Complex ALMA Data (PI: Merenyi)

Summary: The authors propose to apply and extend prior tools (NeuroScope) developed for feature extraction and interpretation of hyperspectral images and functional MRI images using neural machine learning to rich ALMA image cubes. Specifically their technical approach uses both supervised and unsupervised learning, under the broad area of self-organizing maps (SOMS).

1. Alignment with NA ALMA Partnership strategic goals;

This proposal aligns most closely with the NA ALMA Partnership strategic goals: • (1.2) Improve and extend technical capability by exploring, developing, and implementing new software and computing technologies that improve and extend capabilities for scientific research.

2. Strength of the scientific case for the proposed ALMA upgrade concept; Comment on the relevance to the ‘ALMA 2030’ development documents.

As described under archive upgrades in the ‘ALMA 2030’ document, tools are needed for the analysis of very rich ALMA image cubes. This proposal, though focused on the characterization of proto-planetary disks in the first instance, has broad applicability to this problem. As a neural network approach it is inherently algorithmically robust.

3. Quality of the upgrade conceptual design;

This conceptual design builds on an extensive prior body of work using closely-related implementations of this neural network algorithm on datasets in other domains, as described above. A strength of the proposal is that it uses a particularly robust approach to characterizing similar clustered features in an ALMA data cube; here the focus is primarily on kinematic properties. Unsupervised neural networks offer distinct advantages over interpreting moment maps or fitting explicit kinematic models. The self-organizing maps, their connection structure, and visualization offers the potential for rapid new visual insights into data cubes.

The proposal is less convincing in the discussion of supervised classification or automatic alerts and characterization of cluster maps against kinematic models. Projection angles, incomplete coverage, and multiple kinematic components are known to make this a particularly challenging task.

4. Readiness for production in the context of the ALMA Development Plan (the aim is to support a range of upgrades including both those which can be implemented rapidly and those requiring longer-term research and development);

The authors have developed mature tools already and can proceed with the work plan immediately.

5. Strength of the consortium organization (if applicable);

Single organization.

6. Qualifications of the key personnel of the Study;

The key personnel are highly qualified with a demonstrated publication record in this area.

7. Technical expertise, past experience (also in series production, if relevant) and technical facilities in the Institutes taking part in the Study;

The institution has significant experience in this technique. The collaboration would be strengthened by addition of personnel from ALMA/NRAO.

8. Assessment of the level of risk inherent in the design;

There is little risk in the design. As noted above, supervised learning and automated kinematic model alerts seem ambitious but other WBS items are not high rish.

9. Strength of the Scientific Team supporting the Study;

The team could be strengthened by addition of someone in ALMA or CASA.

10. Level of support guaranteed by the Institutes;

Significant in-kind contribution.

11. Budgeted cost of the Study;

The budget appears to be reasonable and well-matched to the core scope of work.

  • Reviewer 11

Grade: 6.0

Title: NeuroScope: Neural Machine Intelligence Tools for Discovery and Interpretation in Complex ALMA Data - 547 - Merenyi

1. Alignment with NA ALMA Partnership strategic goals;

Interesting, if fairly vague proposal, in reasonably strong alignment with ALMA Partnership strategic goals. Study would explore implementation of propers' existing NeuroScope suite of adaptive learning classifier and discovery tools to two ALMA datasets, as a test case. The code is based upon construction of self-organizing maps (SOMs), and finding structures therein. The code has been implemented for brain MRI imaging (for example). Proposal spends most of its time describing SOM data manifolds, and less on what new information can be gleamed from ALMA data beyond, e.g., moment masking techniques, hyperspectral signal coherence, etc.

2. Strength of the scientific case for the proposed ALMA upgrade concept; Comment on the relevance to the "ALMA 2030" development documents.

The proposal claims direct, explicit relevance to the 'usability' goal of the ALMA development roadmap. I'm mostly in agreement, though hesitant given the somewhat vague/speculative nature of the proposal. In general I think this would certainly be an interesting study, but it seems less urgent relative to other more directly applicable software-centered studies.

3. Quality of the upgrade conceptual design;

While the proposal shows an application of NeuroScope to ALMA data in Fig. 2, it would have been helpful had it more explicitly quantified/described what new is learned by its application. I realize that this was only a first step, and am certainly interested in what the deliverables of the study might offer, but the proposal leaves the value of those deliverables fairly vague. Software deliverable would be in the form of MATLAB demos, which seems tangential to the Pythonic ecosystem into which the community has been moving.

4. Readiness for production in the context of the ALMA Development Plan (the aim is to support a range of upgrades including both those which can be implemented rapidly and those requiring longer-term research and development);

This can be developed alongside the range of long- and short-term upgrades implemented as part of this program. Software is easy!

5. Strength of the consortium organization (if applicable);

N/A

6. Qualifications of the key personnel of the Study;

The senior personnel seem clearly qualified, given their CVs. The graduate student (1 FTE) will need training, according to the project plan.

7. Technical expertise, past experience (also in series production, if relevant) and technical facilities in the Institutes taking part in the Study;

See above.

8. Assessment of the level of risk inherent in the design;

The proposal is vague enough to leave worries in this reviewers' mind as to the overall risk. The main deliverable is simply a MATLAB Demo of the software, so

9. Strength of the Scientific Team supporting the Study;

The scientific team are widely regarded experts in neural nets. I have no doubt they are capable of undertaking the proposed work.

10. Level of support guaranteed by the Institutes;

N/A

11. Budgeted cost of the Study;

The total cost ($270K) seems high relative to the proposed deliverable, which effectively amounts to a MATLAB demo.

-- AlWootten - 2017-07-19
Topic revision: r1 - 2017-07-19, AlWootten
This site is powered by FoswikiCopyright © by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding NRAO Public Wiki? Send feedback