ALMA Holography Problems and Solutions


TIP Last Update: JeffMangum - 11 Jan 2007
Table of Contents:


The following presents some of the problems (and their solutions) encountered during ATF production holography integration November 2006.

Sofware Problems

  • M&C Software (2006/11/02): The process of checking and debugging hardware and software is severely disabled by the interference introduced by the high-level GUI-based monitor and control software. It is clear that an approach which is inconsistent with known hardware and software integration practices has been adopted. A more practical and efficient software development approach is a "bottom-up" development scheme whereby the low-level software (hardware communication, basic antenna control, etc.) are developed first. This allows for the integration and debugging of these integral hardware and software systems before the high-level software is integrated. This also accelerates the integration of high-level software as its performance can be characterized using a known foundation (the previously integrated low-level system).
  • No Way to Stop an SB (2006/11/02): This is a serious time sink given that most holography maps fail. Can't stop the SB, so have to kill everything and restart.
  • No Low-Level Interface Software (2006/11/02): Found out quite early on that no software tools for simply dumping data from the DSP or doing simple hardware checks were available (even though it was in our requirements list...see below). This is apparently a byproduct of the misguided emphasis on top-level software. Will need to develop these software tools over the next few days. Fortunately, some low-level control and monitoring functions for the holography hardware have been developed. (2006/11/04): Ralph Marson and Jeff Kern fix this problem. Can now dump data independent of data capture.
  • Clic Not Installed (2006/11/03): The clic software necessary for archive extraction, conversion, and analysis of holography maps was only installed last night. RM did some simple tests of this software (extracted and converted a test holography map). (2006/11/04): Scott Rankin builds new version of the kitchen sink which includes clic. Unfortunately, have not succeeded in using it to process a map.
  • Frail M&C Environment (2006/11/04): While trying to simply dump data from the DSP using a python script encountered many instances where the M&C environment crashed. Usually manifests itself as one of the CONTROL/ACC containers dieing, but also on a few occasions the CONTROL/ABM001 container. Also see the well-named "other exception" failure.
  • Cannot Have Multiple M&C Access Points to Holography Receiver (2006/11/05): When running our low-level data dumping python script using the M&C environment while also running the real-time LabView monitoring software encountered DSP data dropouts (visible on LabView display). This was a problem encountered in 2003 with prototype holography system. Solution is to run only one M&C access program at a time. (2006/11/05): After discussions with Ralph Marson believe that this may only be a problem for the LabView interface display. Currently does not seem to interfere with M&C interface or data acquisition.
  • No Focus Control (2006/11/05): Apparently there is no way to check the focus by commanding it to (X,Y,Z) positions. Note, though, that we would need to loosen some cabling to make it mechanically safe to move the focus.


Hardware Problems


Transmitter

  • The relative frequency stabilities of Tx and Rx need improving. Antonio has this in hand.
  • The amplitude stability of the transmitter needs improving. I think it would be preferable to monitor the output and have some simple closed loop stabilization, rather than running everything open loop into the X6 multiplier. We've been running the Agilent output flat out, which has helped amplitude stability a lot, but the output amplitude still varies, in particular by several dB day/night.
  • Reliability. Before Christmas we had a transmitter failure, which may have been a mechanical relay problem in the tx box, or it may have been something in the control electronics. Robert R. fixed this by bypasssing the relay and all control electronics, and some of the flimsy connectors, within the tx box. I think this should be the way it is left. There is no need to control the tx on and off remotely, since the Agilent synthesizer output can do the same thing, and having a mechanical relay in circuit at the top of the tower can only hurt reliability.
  • The Labview monitoring of the tx current needs to be fixed, preferably by correcting the tx hardware. At the moment, you have to take the indicated value, subtract 0.333 amps, and then multiply by 2 find what the real current being drawn by the tx is. 0.33 amps indicated means zero current, and 1.33 amps indicated means 2 amps are really being drawn. This is misleading, hinders (in fact hindered) diagnostics, and should be fixed.
  • We need accurate documentation. I still have not seen an accurate schematic for the tx in use at the ATF, only a later schematic that Kirk sent us corresponding to the later model of tx. This was better than nothing, but we really should have documentation that corresponds to the hardware actually in use.
  • Absorber. We have a sheet of absorber with about 10 dB transmission loss in front of the transmitter feed, as well as absorber covering the entire face of the tx box. This helps, but could be better engineered (with an angled face for example). It might be better engineering this in the lab there than using our somewhat makeshift arrangement at the top of the tower. I'll talk to you separately about this.


Receiver

  • The relative frequency stabilities of Tx and Rx need improving. Antonio has this in hand.
  • The amplitude stability of the reference receiver need improving. We've noted as much as 5 or 6 dB change in relative gain, over 2 hours, between the reference rx and the main signal rx, when looking at boresight. This change is probably related to temperature changes, as it happens most often in the evening between sunset and midnight.


DSP

  • DSP Hangups (2006/11/04): JGM, DTE, and JK encountered some serious problems with the DSP during today's tests. After seeing these failures and iterating on their solutions, we have come to characterize the as follows:
    • Symmetric Failure: In this failure mode the DSP produces:
      1. Garbage data
      2. An untouched most significant bit (i.e. it is unset)
      3. Total power amplitudes between -1.0 and 1.0
      4. The workaround for this problem is to do TWO DSP resets.
    • Asymmetric Failure: In this failure mode the DSP produces:
      1. Garbage data
      2. Most significant bit set more or less randomly
      3. Total power amplitudes between 0.0 and 1.0
      4. The workaround for this problem is to power cycle the TE distributor (CRD?) in the receiver cabin with the smart plug strip.


Topic revision: r3 - 2007-01-11, JeffMangum
This site is powered by FoswikiCopyright © by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding NRAO Public Wiki? Send feedback