• Proposal 544: High Bandwidth Correlator Study

• Reviewer 1

Grade: 4.0

Title: High Bandwidth Correlator Study

Investigate use of McGill and Dunlap software correlators for ALMA.

really terse proposal with few details for such a large funding request.

Good team and useful concept but specifics are really hard to judge with the small amount of information provided by the proposers.

1. Alignment with NA ALMA Partnership strategic goals;

Comment.

The proposal is well aligned with the strategic goal of larger bandwidths for ALMA, potentially pointing the way to 128 MHz system.

2. Strength of the scientific case for the proposed ALMA upgrade concept; Comment on the relevance to the ‘ALMA 2030’ development documents.

Comment.

Increasing the output bandwidth to 128 GHz bandwidth instantaneous has clear impact on a number of valuable scientific investigations and could significantly increase ALMA efficiency. I do note that this is not discussed in detail within the proposal but there is mention of a potential 33% reduction in observing time, 8x faster observing capability for a variety of continuum applications and flexible beamforming.

3. Quality of the upgrade conceptual design; Comment:

The proposal appears to present a very nice concept for studying an option to expand the ALMA bandwidth although the details themselves are very terse.

4. Readiness for production in the context of the ALMA Development Plan (the aim is to support a range of upgrades including both those which can be implemented rapidly and those requiring longer-term research and development);

Comment

The result of this study will be primarily a costing plan to implement the wideband correlator. Costs will include both capital for construction as well as operations costs. The hardware this is based on is well-understood but there are development efforts needed and this is most likely a longer-term program to reach the stage of a new wideband correlator.

5. Strength of the consortium organization (if applicable);

Comment.

6. Qualifications of the key personnel of the Study;

Comment.

The key personnel involved are critical players in the McGill/Dunlap correlator development and appear well qualified to undertake the proposed study.

7. Technical expertise, past experience (also in series production, if relevant) and technical facilities in the Institutes taking part in the Study;

Comment.

The team appears to have all technical expertise needed to apply knowledge of the McGill/Dunlap correlator to the development study for the wideband ALMA correlator.

8. Assessment of the level of risk inherent in the design;

Comment.

The risk assessment as provided in the proposal is low to moderate and each of the three listed items has a remediation plan in place that seems reasonable to reduce the risk.

9. Strength of the Scientific Team supporting the Study;

Comment.

This is an excellent team both technically and scientifically for the proposed study.

10. Level of support guaranteed by the Institutes;

Comment.

In-Kind contributions from McGill are listed at the 45% level for this program.

• Reviewer 3

Title: Proposal 544, Pen, High Bandwidth Correlator Study, Ue-Li Pen

Grade: 2.5 (0.1 to 9.9, 0.1 is best)

REVIEW

---

This a proposal to study how a Chime like correlator could be modified to become an ALMA correlator, using GPU's and FPGA's for ALMA.

In my opinion, the next ALMA correlator should be built using as much off the shelf components (COTS) as possible, and use industry standard protocols and industry standard boards and connectors (eg: 40 GBit/sec ethernet or 100 Gbit/sec ethernet, ADC boards that use an FMC standard connector, off the shelf WDM industry standard 40 and 100 Gbit fiber transceivers, commercially available FPGA boards, commercially available GPU boards, and commercial 100Gbit/sec switches).

Using industry standard protocols and hardware is cheaper (China makes all of this), and makes the correlator upgradeable, scalable (relatively easy to add more bandwidth, more antennas), fault tolerent, and easier for people to maintain and add features (millions of people understand how to program FPGAs and GPUs - you don't need specialized correlator engineers who might retire and put correlator upgrades or repairs in jeopardy).

This proposal is a step in this direction.

But there are some weaknesses in the proposal: There is very little technical information in this proposal. Here are a few weaknesses:

0) Chime is not so easily adapted to ALMA: Chime is a low bandwidth large N correlator. Alma is a high bandwidth small N correlator (64 antennas). Large N correlators have a lot of computation per bit/sec (computation goes as N^2). Small N correlators have small computation, but the I/O problem is much more severe. The new powerful GPU's are not well suited to small N correlators because they don't have very high input data rates. The proposers don't address this problem.

1) The proposers don't know much about high bandwidth ADC's - they say they will look into them. The Chime ADC digitizes 400 MHz bandwidth. That's too small for ALMA (too many mixers and downconverters). The proposers say they want to use an 8 bit ADC - but there is only one high bandwidth 8 bit ADC system available and it's very expensive, power hungry, bulky, and hard to use (the proposers don't mention this).

1. , 4 or 6 bit ADC's are much better suited for ALMA.

2) They don't discuss how they will implement data transmission. They don't have expertise in this, but they have a great team, and can learn quickly, and there are COTS solutions.

3) The chime correlator uses AMD GPU's. Nvidia is faster and better these days at correlation -- that's an easy problem to fix.

4) I think this proposal is similar, but probably not as well thought out, to a proposal led by Jonathan Weintroub to research high bandwidth ADC's, FPGA's, and GPU's for a next generation ALMA correlator. But i haven't read the Weintroub et al proposal, so i can't say for sure which proposal is better.

---

1. Alignment with NA ALMA Partnership strategic goals: good

2. Strength of the scientific case for the proposed ALMA upgrade concept: good

3. Quality of the upgrade conceptual design: fair

4. Readiness for production in the context of the ALMA Development Plan: Fair

5. Qualifications of the key personnel of the Study: excellent

7. Technical expertise, past experience: excellent

8. Assessment of the level of risk inherent in the design: low risk

9. Strength of the Scientific Team supporting the Study;

not a science proposal - this is engineering.

10. Level of support guaranteed by the Institutes: excellent, about $200K will be brought in by dunlap.

11. Budgeted cost of the Study: OK

• Reviewer 4

Grade: 4.5

Title:High Bandwidth Correlator Study

1. Alignment with NA ALMA Partnership strategic goals;

Proposal is well aligned with ALMA strategic goals

2. Strength of the scientific case for the proposed ALMA upgrade concept; Comment on the relevance to the ‘ALMA 2030’ development documents.

Proposal is for studying the modification of the Dunlap and McGill correlator system for a wide bandwidth correlator for ALMA- 16 GHz to 128 GHz upgrade. Although the team is well experienced in development of software correlators, this proposal would have been stronger if more details were provided.

3. Quality of the upgrade conceptual design;

One of the weakness of this proposal is a lack of discussion of other existing technologies for wide bandwidth correlators. No comparative study is presented. Also in section 3.3, not clear if phasing of subset of antennas can be done for VLBI with the proposed correlator. References are not listed at the end (Only arXiv preprint numbers- in line in text).

4. Readiness for production in the context of the ALMA Development Plan (the aim is to support a range of upgrades including both those which can be implemented rapidly and those requiring longer-term research and development);

One of the missing details is about high speed ADCs. The proposed study’s main deliverable is a costing plan to implement the new correlator for ALMA; both the input (ADC etc), and output (data processing system) interfaces are undefined.

5. Strength of the consortium organization (if applicable);

Collaboration with McGill University and University of Toronto. Team at both institutes are well qualified and with proven track record in this field.

6. Qualifications of the key personnel of the Study;

See above.

7. Technical expertise, past experience (also in series production, if relevant) and technical facilities in the Institutes taking part in the Study;

Team has sufficient and proven expertise in the field of software correlators.

8. Assessment of the level of risk inherent in the design;

Primary risk appears to be regarding availability of the next generation of hardware (ADCs and GPUs) required for this project.

9. Strength of the Scientific Team supporting the Study;

See above. PI’s CV is not included, and only some of co-I’s.

10. Level of support guaranteed by the Institutes;

Project is well supported by PI’s institute.

11. Budgeted cost of the Study;

Total cost of $690,000 over two years, Cost seems high, but it is a relatively large team of 5 investigators including PI. Total material cost is about $100,000.

---

• Reviewer 6

Grade: 6

Title: High Bandwidth Correlator Study

1. Alignment with NA ALMA Partnership strategic goals;

Increasing ALMA's analysis bandwidth is a strategic goal for ALMA.

2. Strength of the scientific case for the proposed ALMA upgrade concept; Comment on the relevance to the ‘ALMA 2030’ development documents.

Increasing ALMA's analysis bandwidth is a strategic goal within the ALMA 2030. This proposal addresses a key part of that capability.

3. Quality of the upgrade conceptual design;

The proposal covers a study of adapting the author's relatively narrowband system for many antennas to an ALMA system, which would have an order of magnitude more instantaneous bandwidth for each of an order of magnitude fewer antennas. While many of the software aspects can be scaled, the proposal does not properly acknowledge the differences and difficulties in an ALMA system's much higher speed digitization, high-fidelity digital transmission, and timing constraints. Using cutting-edge digitizers is not a scalable and straightforward extension of technology and techniques for samplers with <1 GHz clocks. The proposal's lack of discussion of this key aspect is a substantial weakness.

4. Readiness for production in the context of the ALMA Development Plan (the aim is to support a range of upgrades including both those which can be implemented rapidly and those requiring longer-term research and development);

This study is step toward long-term upgrades.

5. Strength of the consortium organization (if applicable);

N/A

6. Qualifications of the key personnel of the Study;

Highly qualified to study correlator architectures.

7. Technical expertise, past experience (also in series production, if relevant) and technical facilities in the Institutes taking part in the Study;

Past experience is very strong, but in systems with order-of-magnitude differences in requirements.

8. Assessment of the level of risk inherent in the design;

Risk is low for coming to conclusions in the digital signal processing aspects, but high for an overall system in which the samplers are critical components. Identifying a practical sampler system is a key element in the overall concept.

9. Strength of the Scientific Team supporting the Study;

Straightforward technology proposal, scientific justification lies in ALMA 2030 report and elsewhere.

10. Level of support guaranteed by the Institutes;

Proposal includes text indicating in-kind cost share with McGill-Dunlap correlator development.

11. Budgeted cost of the Study;

Probably reasonable, but hard to say without a more detailed statement of work.

-- AlWootten - 2017-07-18