FrontEndOperationManual (FEND-40.00.00.00-173-A-MAN)

Table of Content

1 Acronyms

ALMA
Atacama Large Millimeter Array
AMB
ALMA Monitor and control Bus
AMBSI1
AMB Standard Interface 1
CAN
Controller Area Network
CPDS
Cartridge Power Distribution System
FE
Front End
FEMC
Front End Monitor and Control
M&C
Monitor and Control
OWB
One Wire Bus
SSI
Synchronous Serial Interface

2 Applicable Documents

Ref Title Number
[AD1]
Interface Control Document between Front End and Control Software
[AD2]
Front End Embedded Software Description
[AD3]
Front End Engineering Control Software User Manual
[AD4]
Front End Control Software Procedures and Algorithms
[AD5]
FEMC Module Internal Wiring
[AD6]
FEMC Module - Serial Mux FPGA Code
[AD7]
Front End Monitor and Control Module: Assembly Procedure
[AD8]
FEMC FPGA Firmware Update Procedure
[AD9]
ALMA Front End - Shipping, Handling and Storage Procedure
[AD10]
FRONT END SUBSYSTEM MAINTENANCE MANUAL
[AD11]
ALMA Frontend Cryogenics and Vacuum Handling Procedures

3 Referenced Documents

Ref Title Number
[RD1]
ALMA FE Assembly - Preliminary Acceptance On-site (PAS) Plan and Procedures
[RD2]
CENTER TWO CENTER THREE Vacuum Gauge Controller manual

4 Firmware Releases

Ref Title Number
[RF1]
FEMC Module - Arcom Pegasus Software
[RF2]
FEMC Module - AMBSI1 Software
[RF3]
FEMC Module - Serial Mux FPGA Code

5 Prerequisites

This manual assumes that the user is familiar with the applicable and referenced documents listed above.

6 Hardware Description

This section provides a brief description of the FEMC hardware and its functionality.

6.1 Gate Valve Driver

The gate valve driver (GVD) is situated in a small plastic enclosure installed behind the gate valve. Its main scope is to control and monitor the gate valve state and provide an interface with the FEMC via the Cryo M&C hardware.

The GVD hardware has undergone several revisions. The FEMC firmware has masked most of the changes but some of them might become apparent under certain conditions. Under no circumstances this poses and issue for hardware or personnel. The rest of this section will explain these differences.

Before Rev.2.5.0 the any monitor and control action related to the gate valve was blocked if the backing pump was disabled. Any attempt to monitor or control the gate valve was blocked by the FEMC code and the user notified.

With Rev.2.5.0 (current release) the behavior was changed to allow monitoring of the state of the gate valve but not control. This allowed the user to verify the state of the valve before turning on the backing pump thus allowing control of the valve.

The GVD hardware was then changed to Rev.D. This hardware does not allow the monitoring of the state of the gate valve before the backing pump is enabled. A modification to the FEMC firmware was put in place to deal with this hardware modification and will be available starting with Rev.2.6.0.

Eventually all the front ends will be equipped with GVD Rev.D and FEMC firmware Rev.2.6.0 but, until that moment, the following tables show what return monitored values are expected during operation with mixed hardware and firmware revisions. The differences are highlighted in italic.

OLD Gate Valve Driver Hardware (Rev.C and earlier)
 
 
 
Firmware Rev. 2.4.0 (or earlier) 2.5.0 (Current) 2.6.0 (or later)
Backing Pump ON
BLOCKED
STATE
STATE
Backing Pump OFF
BLOCKED
STATE
ERROR

NEW Gate Valve Driver Hardware (Rev.D)
 
 
 
Firmware Rev. 2.4.0 (or earlier) 2.5.0 (Current) 2.6.0 (or later)
Backing Pump ON
BLOCKED
STATE
STATE
Backing Pump OFF
BLOCKED
ERROR
UNKNOWN

Where (see [AD1] for more information):
  • BLOCKED means that the hardware cannot be monitored or controlled
  • STATE represents the actual state of the gate valve
  • ERROR represents a returned error state
  • UNKNOWN represents a returned unknown state

There is only one change in the control behavior: while the valve is moving, further control messages will be ignored until the valve has reached the new position unless the GVD is in over-current state than a procedure is required to restore normal functionality. This is described in the troubleshooting section of this manual.

6.2 Solenoid Valve interlock

A solenoid valve is placed between the turbo pump and the backing pump to keep the vacuum integrity.

The solenoid valve interlock provides a safety feature to prevent a cryostat from being exposed to dangerous pressure conditions.

The solenoid valve can only be operated if the following conditions are true:
  • The DC voltage is applied to the Front End
  • The AC voltage is applied to the Front End
  • The command to turn on the backing pump was issued to the cryo M&C system

Also, it is essential that the vacuum gauge controller is configured with the following limits:
Relay SP1 SP2 SP3
Low
20
10^-2
10
High
30
3*10^-2
30

(values are in mbar)

When evaluating the table, remember that there is hysteresis in the gauge controller limits:
  • when the pressure is decreasing, SPx is triggered when the LOW limit is passed
  • when the pressure is rising, SPx is triggered when the HIGH limit is passed

Once these are verified, the following truth table applies:
0 0 0 0 0 0 0 1 E
0 0 0 0 0 1 x x E
Vacuum Gauge Controller
ITR90
ITR90
TTR90
Turbo Drive
 
Gate Valve
 
Solenoid Valve
Error SP1 SP2 SP3 Error Up-to-speed Open Closed Enabled/Disabled
1
x
x
x
x
x
x
x
D
 
1
x
x
x
x
 
x
D
0 1 x x x x 1 0 E
 
 
1
1
x
x
x
x
D
 
 
1
 
1
x
x
x
D
0 0 1 0 0 x x x E
 
 
 
1
x
x
x
x
D
 
 
 
 
1
x
x
x
D
 
 
 
 
 
 
x
 
D

Legend:
  • D means that the solenoid valve is closed
  • E means that the solenoid valve can be remotely controlled
  • x means don't care

This logic derived from this table is shown in the following schematic (Interlock logic). This is used to generate the programming file for the Xilinx CPLD used to implement the interlock.

Interlock verification procedure

The procedure to test the interlock system is to recreate every scenario in the truth table given above and make sure that the interlock behaves as it is supposed to and stop the opening of the solenoid valve when it presents a risk for the equipment.
  • Case 1: If the vacuum gauge controller has an error, the solenoid valve shall not open because we can not read the pressure or rely on the pressure reading.
Test 1: It is difficult to create an error state of the vacuum gauge controller; it is easier to simulate an error by disconnecting the cable going to the controller and by shorting the pins that are in contact when the controller is in an error state. The interlock must prevent any opening of the solenoid valve.

  • Case 2: The pressure inside the cryostat is above 30mbar and the gate valve is closed. The interlock system does not know the exact pressure inside the Dewar, but we do know that the differential pressure at opening for the gate valve should be ≤30mbar. According to the manufacturer operating requirements, the gate valve can not be safely operated and therefore the solenoid valve must remain closed.
Test 2: The gate valve is closed and the pressure inside the cryostat is above 30mbar, the interlock must stop the command to open the solenoid valve regardless of the status of the other vacuum pieces of equipment.

  • Case 3: Enable Solenoid Valve. The pressure inside the cryostat is above 30mbar and the gate valve is not closed. It is acceptable to open the solenoid valve; the cryostat has a bad vacuum already and is not going to be much worse if the solenoid valve is open.
Test 3: If the gate valve is open, the solenoid valve will open as long as the pressure inside the cryostat is above the 30mbar. IF the gate valve was closed, it has to be opened first. The safest way to open the gate valve is to bring both sides of the valve to atmospheric pressure by letting air in. When the cryostat is at atmospheric pressure, when the gate valve is open the solenoid valve opens and closes as soon as the gate valve is closed.

  • Case 4: The pressure inside the cryostat is between 30mbar and 1E-2mbar and the pressure inside the vacuum line is above 10mbar. The backing pump does not work, the solenoid valve is closed
Test 4: The cryostat has been pump down below 30mbar using an external pump and the on board backing pump ON/OFF switch is on the OFF state. The pump will not start. The pressure in the vacuum line is above 10mbar, the solenoid valve will not open.

  • Case 5: The pressure inside the cryostat is between 30mbar and 1E-2mbar and the backing pump is working but the turbo pump doesnÂ’t work. The solenoid valve is closed.
  • Test 5: The turbo pump is in error state, this can be simulated by disconnecting the cable to the turbo controller (remote X1 connector) and by connecting the pins that are connected when an error is present. When the pins are connected the solenoid valve must remain closed.

  • Case 6: Enable Solenoid Valve. The pressure inside the cryostat is between 30mbar and 1E-2mbar, the backing pump is working and the turbo pump is also working because the turbo drive has no error. It is safe to open the solenoid valve regardless of the position of the gate valve.
Test 6: An external turbo pump connected to the auxiliary port of the cryostat is required to bring the vacuum within the 1E-2mbar range. The interlock will allow you to open the solenoid valve as long as both pumps are running. This test should be done with the gate valve open and closed.

  • Case 7: The pressure inside the cryostat is below 1E-2mbar but the backing pump is not working. The solenoid valve is closed.
Test 7: An external turbo pump connected to the auxiliary port of the cryostat is required to bring the vacuum below 1E-2mbar. The backing pump must be turned OFF and the pressure inside the vacuum port above 10mbar. The solenoid valve will fail to open.

  • Case 8: The pressure inside the cryostat is below 1E-2mbar the backing pump is working but the turbo has failed. The solenoid valve is closed.
Test 8: An external turbo pump connected to the auxiliary port of the cryostat is required to bring the vacuum below 1E-2mbar. The backing pump is ON but you have to simulate a failure of the turbo pump as in the test 5 by disconnecting the cable to the controller (remote X1 connector) and connect the pins that are connected when the turbo is in error state. When the pins are connected, the solenoid valve is closed and will fail to open.

  • Case 9: The pressure inside the cryostat is below 1E-2mbar, the backing pump is working and the turbo is ON but it is not up to speed. The gate valve is open or in between position and the solenoid is closed to avoid any back streaming into the cryostat.
Test 9: An external turbo pump connected to the auxiliary port of the cryostat is required to bring the vacuum below 1E-2mbar. The gate valve is open and the turbo pump is accelerating. As soon as the turbo pump is up to speed the solenoid valve will open. If the turbo pump is stopped the solenoid valve will close and if the pump is started again the solenoid valve will open as soon as it is back up to speed.

  • Case 10: Enable Solenoid Valve. The pressure inside the cryostat is below 1E-2mbar, the backing pump is working and the turbo is ON and accelerating. The gate valve is closed and the solenoid valve is enabled because the turbo pump needs the backing pump to be able to reach full speed if the turbo pump is started with a pressure above a few mbar.
Test 10: An external turbo pump connected to the auxiliary port of the cryostat is required to bring the vacuum below 1E-2mbar. The gate valve is closed and the backing pump running, as soon as the turbo pump starts, the solenoid valve can be opened. If the gate valve is open, the solenoid valve will close and if you close the gate valve again the solenoid vale will open.

  • Case 11: Enable Solenoid Valve. The pressure inside the cryostat is below 1E-2mbar the backing pump is working and the turbo is ON and is up to speed. It is safe to open the solenoid valve regardless of the gate valve position.
Test 11: This test follows the test 10, only that one has to wait long enough for the turbo pump to get up to speed then if the gate valve is open or closed, the solenoid valve will stay open.

6.3 Front End Monitor and Control (FEMC)

The FEMC provides an interface between the Front End (FE) hardware and the ALMA Monitor and control Bus (AMB).This is achieved using 3 main boards:
  • AMB Standard Interface 1 (AMBSI1)
  • ARCOM Pegasus
  • Serial Multiplexer Board (Serial Mux)

6.3.1 AMBSI1

The AMBSI1 is a standard interface that has to be used to interface to the AMB, as mandated by the computing IPT.The board is based on a Infineon C167 (STMicroelectronics ST10) chip.In the FEMC this board is simply working as an interface and the firmware is kept as slim as possible.

6.3.2 ARCOM Pegasus

This is the heart of the FEMC module. This boards receives the Controller Area Network (CAN) from the AMBSI1 and performs a series of operation depending on the nature of the message.There are two kind of messages:* Control. These messages are used to change the state of the FE. During operation, when a control message is received, the FEMC module checks the request to make sure that it will cause no harm to the FE and sends it through the Serial Mux to the addressed FE hardware.* Monitor. These message are used to query the state of the FE. During operation, when a monitor message is receive, the FEMC will query the addressed hardware and check the returned value against a set of limits and safety rules. If any of these are triggered, then the FEMC will act to prevent damage to the FE. In any case, the message is packaged with extra troubleshooting information and returned to the AMBSI1 to be sent back on the AMB.

6.3.3 Serial Mux

The FEMC system is build in a star configuration with the FEMC module at the center. Each remote M&C hardware is connected directly to the FEMC module using a 10 MHz Synchronous Serial Interface (SSI) bus. The Serial Mux board performs this low level communication with the remote M&C hardware.

7 Setting up

7.1 Interim Power Supply

7.1.1 Thermostat settings

The interim power supply provides a safety interlock to prevent overheating. This interlock has to be properly set to assure the correct behavior. The settings vary depending of the altitude.The following table shows typical settings.*The values for 3000m and 5000m were obtained from experimental testing. Although measurements were made to evaluate the correct temperature, the speed of the forced air of the cooling system and the orientation of the device might have an impact on the accuracy of the actual setting. These values are only indicative.*

Location Altitude Thermostat Setting
Integration Centers
~ sea level (0m)
45 deg C
OSF
~ 3000m
35 deg C
AOS
~ 5000m
25 deg C

7.2 Vacuum System

7.2.1 Vacuum Gauge Controller

It is essential for the correct operation of the vacuum system that the vacuum gauge controller is configured with the following limits:
Relay SP1 SP2 SP3
Low
20
10^-2
5
High
30
3*10^-2
30

(values are in mbar)

8 Status verification

...

9 Initial Conditions

The following assumptions are made for the listed systems.

9.1 Monitor and Control system

  • The computer is equipped with the proper hardware
  • The hardware is properly connected to the receiver
  • The control software is installed and communication has been established with the front end

9.2 Cryogenic system

  • The components (outdoor unit, indoor unit, cryohead) of the cryogenic system are properly connected to the Front End and in operating conditions.
  • Block diagram of the cryostat components:
    cryostat-components.PNG

9.3 Power supply system

  • The power supply system and the 230V AC are correctly connected to the Front End through the bulkhead
  • Power is turned on, within operating parameters and available to the Front End

10 Procedures

10.1 Verify Initial Conditions

Verify that the Front End is ready to be operated.

10.1.1 Power Supply

Verify that the Power Supply is correctly connected to the Front End and turned on. Verify that all the voltages on the Power Supply are turned on and operating within normal parameters.

10.1.2 230V AC Power

Verify that the 230V AC Power is available and correctly connected to the Front End.

10.1.3 Control Software

Verify that the control computer is correctly connected and that the control software is up and running.

10.2 Get to or restore operating vacuum conditions

Once the Front End conditions have been verified and the Front End is set to the initial conditions, the procedure to bring the Front End to operating conditions can be initiated. If the Front End is installed on the antenna then the following procedure can be used to restore the operating conditions.

Note: An important limitation is that the differential pressure across the gate valve cannot be greated than 30mbar by manufacturer specfications. In the current design is not possible to directly monitor the pressure on both sides of the gate valve so, in certain conditions, it is necessary to operate the pumps and valves in a correct order to avoid bringing the Front End in a configuration where the use of the outside roughing pump is required.

Pressures (mbar)
 
Order of operations
 
 
 
 
Port Dewar Backing Pump Solenoid Valve Turbo Pump Gate Valve Comments
X
Ambient
1
3
4
2
This procedure can be applied ater the front end has been sitting at ambient pressure for a long time (to be quantified). This will ensure that the pressure differential across the gate valve is small.
X
<30
1
2
3
4
The order of operation is critical to minimize air leakage in the dewar.
X
>30 and < Ambient
N/A
N/A
N/A
N/A
In this condition it is not possible to guarantee that the pressure across the gate valve is <30mbar. This condition can present itself after a loss of power in teh middle of a pump-down procedure or during warm up if the front end has been operating for a long time. In this condition, an external pump should be used to bring the dewar pressure to <30mbar at which point one of the previous procedure can be used to pump down the dewar to operating conditions.

Where:
  • X is 'Don't care'
  • N/A is 'Not Allowed'

10.2.1 Pumping down

The first step is to pump down the Front End from its current pressure to a pressure where is safe to initiate the cool-down.

10.2.1.1 Initial condition

The initial condition should set to the following:
  • Gate Valve: CLOSED
  • Solenoid Valve: CLOSED
  • Turbo Pump: OFF
  • Backing Pump: OFF
  • Vacuum Gauge Enable: ON
These conditions apply for ALL the pumping down procedures listed in this section.

10.2.1.2 Cryostat at atmospheric pressure and room temperature

The following procedure is valid when the cryostat is at atmospheric pressure and room temperature. This state can be verified using the control software:
  • All the temperatures listed under the Cryostat Temperatures section have to be at the local room temperature.
  • The pressure listed under Cryostat has to be at the local atmospheric pressure.

10.2.1.2.1 External roughing pump available

This is the procedure in case an external roughing pump is available.

10.2.1.2.1.1 Set up external roughing pump
  • Make sure the auxiliary valve is completely closed.
  • Attach the external roughing pump to the auxiliary valve.

10.2.1.2.1.2 Start rough pumping
  • Turn on the external roughing pump.
  • Slowly open the auxiliary valve to expose the cryostat to the external roughing pump.

10.2.1.2.1.3 Start on-board pumping system
  • Wait for the cryostat pressure to go below 3 mbar. If this is not achieved within 2 hours then check the troubleshooting section.
  • Start backing pump and wait for the port pressure to go below 3 mbar. If this is not achieved within 1 minute then check the troubleshooting section.
  • Open the solenoid valve and wait for the port pressure to go below 3 mbar. If this is not achieved within 1 minute then check the troubleshooting section.
  • Start the turbo pump and wait for it to get up to speed. If this is not achieved within 5 minutes or if the turbo pump returns error, then check the troubleshooting section.
  • Open the gate valve. This will expose the on-board pumping system to the cryostat.

10.2.1.2.1.4 Stop rough pumping
  • Close the auxiliary valve to isolate the cryostat from the external roughing pump
  • Turn off the external roughing pump.
  • The external roughing pump can now be disconnected from the cryostat.

10.2.1.2.2 External roughing pump NOT available

This is the procedure in case an external roughing pump is not available.

10.2.1.2.2.1 Start on-board pumping system
  • Start backing pump and wait for the port pressure to go below 3 mbar. If this is not achieved within 1 minute then check the troubleshooting section.
  • Open the gate valve.
  • Open the solenoid valve.
  • Wait for the cryostat pressure to go below 4 mbar. If this is not achieved within 6 hours the check the troubleshooting section.
  • Start the turbo pump and wait for it to get up to speed. If this is not achieved within 10 minutes or if the turbo pump returns error, then check the troubleshooting section.

10.2.1.3 Restore vacuum during normal operation

During normal operation the inevitable leaks in the cryostat will let gases into the vacuum chamber. Many of these gases will freeze over the coldest part of the cryostat increasing the thermal load causing up to the point where the cryogenic system will not be able to maintain the temperatures of the different stages within the specified parameters.

10.2.1.3.1 Front End installed in the antenna

This procedure describes how to restore vacuum when the Front End is installed in the antenna.

10.2.1.3.1.1 Stop the cooling
  • Turn off the compressor.
  • Wait for:
    • The cryostat temperatures of the 4 K and 15 K stages to reach 80 K
    • OR the cryostat pressure to reach 10 mbar.

10.2.1.3.1.2 Restore vacuum

When either of the previous condition is met, then the cryostat is ready to be pumped again. Since the cryostat is in the antenna cabin, then the External roughing pump NOT available should be used to restore the operating pressure.

10.2.1.3.2 Front End in the lab

This procedure describes how to restore vacuum when the Front End is installed in the lab.

10.2.1.3.2.1 Stop the cooling
  • Turn off the compressor.
  • Wait for:
    • The cryostat temperatures of the 4 K and 15 K stages to reach 80 K
    • OR the cryostat pressure to reach 10 mbar.

10.2.1.3.2.2 Restore vacuum

When either of the previous condition is met, then the cryostat is ready to be pumped again. Since the cryostat is in the lab, then either the External roughing pump NOT available or the External roughing pump available can be used to restore the operating pressure.

10.3 Cooling down

This procedure describes how to cool down the Front End once the operating pressure is reached/restored using the procedures described in Pumping down.

10.3.1 Verify that the operating pressure has been reached

The operating pressure can be considered reached when the cryostat pressure is below 1.0x10-4 mbar. If this is not achieved in 2 days then check the troubleshooting section.

10.3.2 Start cooling

Once the operating pressure has been reached, the cooling procedure can be initiated by turning on the compressor. This can be accomplished in two way depending on the availability of the compressor monitor and control system.

10.3.2.1 Compressor monitor and control available

To be written

10.3.2.2 Compressor monitor and control unavailable

To turn on the compressor, the main controls are available on the front panel of the indoor control unit.
  • Verify that the cryohead switch (located to the top of the front panel) is set to (Denis?)
  • Turn on the compressor by toggling the main switch (located at the bottom of the front panel). If the compressor does not turn on after performing the listed operations then check the troubleshooting section.

10.3.3 Cryopumping

At this point the receiver is cooling down while being pumped. When both the 4K and 15K stages reach a temperature below 30K the system is in cryopumping mode and the Pumping down procedure can be stopped since is no longer efficient. If this condition is not achieved in 3 days, check the troubleshooting section.

The Front End Control software (engineering version for Windows, as well as ALMA software) should be placed into its "Cryo-Pumping" mode.This runs a process which monitors the 4K and 15K stage temperatures and then automatically closes valves and stops pumps when both pass below 30 K.In the engineering control software, this mode is enabled via a button labeled "Cryo Pumping" on the user interface screen.

10.3.3.1 Stop the on-board pumping system

This is the procedure to follow in order to stop the on-board pumping system:
  • Close the gate valve
  • Close the solenoid valve
  • Turn off the turbo pump
  • Turn off the backing pump

10.3.4 Operating condition

The Front End can be considered to be in operating condition when:
  • The 4K stage temperature is below 4K
  • The 15K stage temperature is (Denis?)
  • The 110K stage temperature is (Denis?)

Cool-down plot

The Front End Control software (engineering version for Windows) produces a text file named for example like "ThermalLog-2012-03-13-162648.txt" which includes sensor data for all front end temperatures and pressures plus the state of the valves and pumps.Normally one row is recorded every 10 seconds.Given that the software is operating during the cool-down interval, the contents of this file may be formatted into a cool-down plot to show the performance over time.[Add example plot]

10.4 Warming up

Procedure for warming up to ambient temperature and atmospheric pressure:
  • Turn off the compressor. Note that this is functionally equivalent to power failure.
  • Allow all three cryostat stages to warm above 80K.
  • Inject dry sufficient nitrogen to bring the cryostat pressure up to 20 mbar
  • Wait for the cryostat stages to reach ambient temperature
  • Inject dry nitrogen or air to bring the pressure up to atmospheric.

Warm-up plot

The Front End Control software (engineering version for Windows) produces a text file named for example like "ThermalLog-2012-03-13-162648.txt" which includes sensor data for all front end temperatures and pressures plus the state of the valves and pumps.Normally one row is recorded every 10 seconds.Given that the software is operating during the warm-up interval, the contents of this file may be formatted into a warm-up plot to show the performance over time.[Add example plot]

10.5 Cryostat hold time measurement and plot

One of the measurements routinely performed by the FEICs to check the performance of the cryostat is the Hold Time measurement.

This shall be measured with the completed receiver (i.e. with all the available bands installed). The FE Engineering Control Software includes an option to log all temperature and pressure sensors to a text file. Instructions for enabling and using this option are given in the Software's user manual[RDxx].

Procedure

The pressure and temperature inside the cryostat will be monitored after the cryostat has been operating at base temperature for at least 2 hours.
  1. (for convenience) Stop all LabView software and then start up just FRONT END CONTROL.This step is not strictly required but it will force it to create a new ThermalLog file.
  2. Wait a minute, verify that the cryostat temperatures and pressures are updating on the screen.
  3. Turn off the compressor ( Sumitomo Cryocooler (Switch OFF indoor unit)).
  4. Wait 30 minutes.
  5. Turn back on the compressor (Switch ON indoor unit).
  6. After 6 hours, collect the ThermalLog file from the test system computer.
  7. Plot the data and calculate the recovery time.

Estimate of accuracy and repeatability

Pressures can be measured to 10-8 mbar, temperatures can be measured to 0.5 K on all of the stages. Warm-ups during testing of the receiver will be compared to obtain the repeatability.

Example of results

Hold Time.png

According to [ADxx FE Tech Specs], the temperature drift on the stages may be as follows:

4 K: less than 2 mK / minute

15 K: less than 50 mK / minute

110 K: less than 100 mK / minute

In the chart above, the temperature drifts at the 6½ hour mark are, respectively 1 mK, 5 mK, 10 mK per minute. The cryostat is considered to be in specification, even though the 110 K shield temperature has not stabilized at the 6½ hour mark.

11 Operating the Front End

11.1 Latencies

This sections reports a list of known latencies in the Front End organized by control/monitor: how long does a controlling software have to wait after issuing this control/monitor point?

The following describes the variable used in the latencies table and their possible values.

Variable Value Description
[Ca]
0->9
This indicates the addressed band. The value is calculated using band number - 1. For example, Ca=5 refers to band6.
[Po]
0, 1
Cartridge polarization
[Sb]
0, 1
Cartridge sideband

Name Assembly Latency Blocking Description
SET_POWER_DISTRIBUTION_MODULE[Ca]_ENABLE
CPDS
5ms
Yes
This is the time required for the powered cartridge to stabilize and be initialized. Starting with Rev.2.6.0 of the FEMC firmware, this latency is not blocking anymore. There will still be an hardware latency due to the powering up of the CCA and WCA monitor and control hardware but the FEMC will keep being responsive to messages.
SET_IF_SWITCH_CHANNEL[Po][Sb]_ATTENUATION
IF Switch
100 ms
No
Max attenuator setting time from FEND-40.08.01.01-008-A-SPE 2009-01-26 Released
SET_IF_SWITCH_CARTRIDGE
IF Switch
100 ms
No
Max switching time from FEND-40.08.01.01-008-A-SPE 2009-01-26 Released
SET_LPR_OPT_SWITCH_PORT
LPR
500 ms max 100 ms goal
No
From ALMA-56.05.00.00-023-A-SPE 2009-03-13 Draft. In the same document, the switching duty cycle has a minimum specified at 10s.
SET_LPR_EDFA_MODULATION_INPUT_VALUE
LPR
50ms (Power)
No
This was suggested in a verbal communication by the LPR designer. There is no direct mention of this specification in ALMA-56.05.00.00-023-A-SPE 2009-09-29.
SET_LPR_EDFA_MODULATION_INPUT_VALUE
LPR
30s (Phase)
No
This was suggested in a verbal communication by the LPR designer. The main reason for this delay is due to long term drift caused by the variation in optical power in the system. There is no direct mention of this specification in ALMA-56.05.00.00-023-A-SPE 2009-09-29.
SET_CARTRIDGE[Ca]_LO_YTO_COARSE_TUNE
WCA
investigate
No
The WCA tech specs documents do not state the settling time.
SET_CARTRIDGE[Ca]_POL[Po]_SB[Sb]_SIS_VOLTAGE
ColdCart
10ms (for -12mV to +12mV swing)
No
The bias module tech specs do not state a settling time. FEND-40.04.02.00-005-D-SPE 2009-06-04 Released
SET_CARTRIDGE[Ca]_POL[Po]_SB[Sb]_SIS_MAGNET_CURRENT
ColdCart
investigate?
No
The bias module specs do not state a settling time. In practice this should be slewed stepwise from zero up to the desired value to avoid trapping flux, esp. for band 9 but possibly also for 7&8. FE P&A document needs update.

The Blockingcolumn identifies if this latency blocks or not the acknowledgment of incoming messages by the FEMC. If a latency is blocking, the FEMC is not going to acknowledge any incoming message during the latency time. If it is not, then the addressed hardware requires the latency time to stabilize but messages to the FEMC will be acknowledge during the latency time.

11.2 Safety

This section deals with equipment safety when operating the front end. These are requirements that have to be met in order to assure the safety of the Front End and its sub assemblies.

11.2.1 Front End subsystem safety

This describes safety issues related to every single individual Front End subsystem.In this section is assumed that the other subsystems interfacing to the analyzed one are operating within the parameters specified in the applicable interface documents.

11.2.1.1 Cryostat and Vacuum

  • The gate valve should not be operated if the differential pressure between the two sides is above 30mbar.

The pumping down sequence described in this document should always be followed to prevent this from occurring.

As an example, the following sequence will put the cryostat in a condition where the gate valve should not be opened:
  • the cryostat is at ambient pressure, the valves are closed and the pumps are off
  • the on-board backing pump is turned on
  • the solenoid valve is opened (at this point the differential pressure between the cryostat and the pumping system is to high to open the gate valve). The only solution is then to attach the external roughing pump and pump down the cryostat.

There is nothing preventing the operator from issuing the command to open the gate valve, so this situation should be avoided.

11.2.1.2 CPDS - Cartridge Power Distribution System

11.2.1.2.1 Monitor and Control safety
  • No more than 3 cartridges should be turned on at the same time.
    This is checked by the FEMC software which will prevent more than 3 cartridges to be turned on at any time when in operational or maintenance mode. If in troubleshooting mode, there is no restriction to the number of cartridges that can be powered at any given time.
    For the same reason, at startup, the FEMC software will unconditionally shut down all the cartridges to assure alignment between the software and hardware status.

11.2.1.3 CC - Cold Cartridge

11.2.1.3.1 Good practices

When operated at room temperature, it is good practice not to leave a Cold Cartridge powered for extended periods of time. Even more so if the cartridge is installed in a evacuated dewar.

11.2.1.3.2 Monitor and Control safety
  • The mixer heater should not be turned on for longer than specified by the cartridge manufacturer.
    Starting with Rev.D of the BIAS module, a circuit has been added to the hardware to assure that the heater will be automatically shut off after 1 second. Previous version of BIAS module don't have this protection. For system implementing these earlier bias module, version of the FEMC software are available that will prevent the usage of the mixer heaters for bands that require this.

  • The SIS magnets should not be biased if the mixers temperature is above 30K.

11.2.2 Front End safety

This sections describes safety issued related to the whole Front End considered as a system. This sections assumes that the requirements listed in the previous section are met.In this section is assumed that the other subsystems interfacing to the Front End are are operating within the parameters specified in the applicable interface documents.

11.2.2.1 WCACC

This section describes potential safety issues related to the interface between WCA and CC.

11.2.2.1.1 Monitor and Control safety
  • Since the PAs retain their setting even after the WCA is powered off while the YIG doesn't, the drain and gate voltages of the PAs should be set to 0.0V (drain voltage first) every time a cartridge is powered up to prevent delivering too much power to the cold multiplier at the wrong frequency.
    This is performed by the FEMC. Starting from Rev. 2.1.0, whenever a band is powered up, the PA drain and gate voltage (in this order) are set to 0.0V.

  • If the temperature of the mixers in the CC is above 30K, no LO power should be delivered to the CC.

  • The input power to the cold multiplier should be modulated depending on the operating frequency. The values are WCA and CC dependent.

11.2.2.2 (CC+WCA)Cryostat

This section describes the potential safety issues related to the interface between the Bands (CC+WCA) and the cryostat. Where applicable, a description of a solution has been added.

11.2.2.2.1 Monitor and control safety
  • In operation mode if the temperature of the cryostat 4K is above 30K, no band should be powered.

  • In operation mode if a Band is powered and the temperatures of the cryostat 4K rises above 30K, that band should be turned off.

11.2.2.3 LPRWCA

This section describes the potential safety issues related to the interface between the LPR and the WCA.

11.2.2.3.1 Monitor and Control Safety
  • The EDFA photodetector power should be kept below 9 mW when optical power is present to prevent damage to the WCA photomixer.

  • If the optical power is not available, the LPR output power should be set to the minimum value.

11.2.3 System safety

This section describes safety issues that might affect the Front End equipment safety due to interaction with other ALMA subsystems.

11.2.3.1 Central LO Reference

This section describes potential issue raising from the interaction with the Central LO reference system.

* If the LO reference optical power is not available to the Front End, the readout of the optical power delivered to the WCA photomixer will look much lower than the one required for locking. This might lead to increase the amplification factor for the optical amplifier within the LPR to reach the required value. At this point, if the LO reference optical power is suddenly turned on, this will cause damage to both the WCA photomixer and the LPR.

12 Maintenance

12.1 Monitor and Control system

This section describes troubleshooting procedures for the Front End Monitor and Control System.

12.1.1 Connecting to the FEMC

12.1.1.1 CAN

...

12.1.1.2 Ethernet

...

12.1.1.3 RS-232

This section describes how to troubleshoot the Front End Monitor and Control System connecting directly to the module using the RS-232 connector on the front panel.

12.1.1.3.1 Required Equipment
  • a computer with a terminal software such as (hyperterminal, secureCRT or equivalent that support z-modem transfer) and a RS-232 serial port
  • a RS-232 null-modem cable

12.1.1.3.2 Terminal software setup

Ensure the terminal software has the following settings:
  • Baud rate: 115200
  • Data bits: 8
  • Stop bits: 1
  • Parity: None
  • Flow Control: None

12.1.1.3.3 Connection procedure

This procedure is the general procedure to follow to establish a connection to the FEMC using the RS-232 serial port.This procedure assumes that the FEMC module is installed inside a Front End.
  1. Turn off any activity on the CAN bus
  2. Turn off the auxiliary DC power supply and the 230V/50Hz AC power supply
  3. Open the side-panel on the side where the monitor and control sub rack is located
  4. Unlatch and pull the sub rack until the front of the FEMC module is visible
  5. Connect the RS-232 null modem cable from the RS-232 port of the FEMC to the control computer running the serial interface terminal software
  6. Start the terminal software with the above settings
  7. Turn on the auxiliary DC power supply and check the terminal software window. You should see the boot up sequence scrolling on the screen.
  8. Once the communication is established push ENTER a couple of time to verify that you can talk to the FEMC. This will also give you the console help if it hasn't been disabled via the relative CAN special control message.

12.1.1.4 The NED editor

The FEMC ROM-DOS comes with a rudimentary editor called NED (equivalent to the old EDIT for DOS). To edit a file issue the command:
   ned <filename> <Enter>

this will open the editor window with the file.

FEMC - 05 - NED.png

To move around the file, you can control the cursor using the following key-combination:
   CTRL+a  (left)
   CTRL+s  (down)
   CTRL+w  (up)
   CTRL+d  (right)
   Backspace (deletes previous character)

To navigate the menu press:
   ESC <desired letter>

For example:
   ESC f s (to save)
   ESC f x (to exit)

12.1.2 Configuration files

Some of the subsystems of the FE have configurations that vary from item to item. For example, the temperature sensors inside the cryostat are different from one cryostat to the next. In order for the monitor and control operations to be transparent with respect to scaling factors a calibration curves, the FEMC takes care of applying all necessary correction. This is achieved using a series of configuration (.INI) files during boot time. The configuration is loaded in memory and used during normal operation to correct the data.

Anytime the FEMC boots up, it will check the list of detected ESN against a locally stored list of known hardware. Both list will be made available to the ALMA control software. The latter will decide if an upgrade is required and if so will push the new configuration files to the FEMC using the provided ethernet connection. to be implemented

Until the previous mechanism is implemented or when a FE is being assembled at an integration center. These files will have to be modified manually using the NED editor as described in this manual.

The configuration files used for this purpose are the following:
  • CART[1-10].INI
  • WCA[1-10].INI
  • LPR.INI
  • CRYO.INI
  • FRONTEND.INI
  • ESNS.INI
If any of the hardware is changed, upgraded or modified, a new configuration file will be downloaded from the configuration database once during boot time.Other configuration files are generated by the FEMC code to support special features and will require editing to modify the behavior of the FEMC software.In the following sections each file will be described in detail.

12.1.2.1 CART[1-10].INI

This files contains the configuration information for the Cold Cartridge Assemblies (CCA).

12.1.2.1.1 Section: [INFO]
  • ESN = [8 bytes of Electronic Serial Number] this field contains the ESN of the CCA.
  • SN = "[String identifying the LPR Serial Number]" this contains the serial number of the CCA.

12.1.2.1.2 Section: [RESISTOR]
  • VALUE = [SIS current sense resistor value in ohm] this fields contains the value of the current sense resistor for the CCA. This value is used by the FEMC firmware to evaluate the SIS mixer current. As of now, the firmware assumes the resistor value to be the same for all the mixers in a given CCA.

12.1.2.1.3 Temperature sensors

Every temperature sensor can be marked as available (default) or not available. The FEMC firmware will read the provided information at startup and used them during operation.

12.1.2.1.3.1 Section: [4K_STAGE]
  • AVAILABLE = [0-1(Y-N/T-F)] this indicates if this sensor is available for monitoring. If this is set to 0 (N or F) any attempt to monitor this sensor will be rejected by the FEMC and an hardware range error will be returned.
  • OFFSET = [float] this contains the offset measured for this sensor. This offset is added to the measured temperature and the sum is returned at final temperature.

12.1.2.1.3.2 Section: [110K_STAGE]
  • AVAILABLE = [0-1(Y-N/T-F)] this indicates if this sensor is available for monitoring. If this is set to 0 (N or F) any attempt to monitor this sensor will be rejected by the FEMC and an hardware range error will be returned.
  • OFFSET = [float] this contains the offset measured for this sensor. This offset is added to the measured temperature and the sum is returned at final temperature.

12.1.2.1.3.3 Section: [POL0_MIXER]
  • AVAILABLE = [0-1(Y-N/T-F)] this indicates if this sensor is available for monitoring. If this is set to 0 (N or F) any attempt to monitor this sensor will be rejected by the FEMC and an hardware range error will be returned.
  • OFFSET = [float] this contains the offset measured for this sensor. This offset is added to the measured temperature and the sum is returned at final temperature.

12.1.2.1.3.4 Section: [SPARE]
  • AVAILABLE = [0-1(Y-N/T-F)] this indicates if this sensor is available for monitoring. If this is set to 0 (N or F) any attempt to monitor this sensor will be rejected by the FEMC and an hardware range error will be returned.
  • OFFSET = [float] this contains the offset measured for this sensor. This offset is added to the measured temperature and the sum is returned at final temperature.

12.1.2.1.3.5 Section: [15K_STAGE]
  • AVAILABLE = [0-1(Y-N/T-F)] this indicates if this sensor is available for monitoring. If this is set to 0 (N or F) any attempt to monitor this sensor will be rejected by the FEMC and an hardware range error will be returned.
  • OFFSET = [float] this contains the offset measured for this sensor. This offset is added to the measured temperature and the sum is returned at final temperature.

12.1.2.1.3.6 Section: [POL1_MIXER]
  • AVAILABLE = [0-1(Y-N/T-F)] this indicates if this sensor is available for monitoring. If this is set to 0 (N or F) any attempt to monitor this sensor will be rejected by the FEMC and an hardware range error will be returned.
  • OFFSET = [float] this contains the offset measured for this sensor. This offset is added to the measured temperature and the sum is returned at final temperature.

12.1.2.1.4 Hardware configuration

The FEMC firmware is designed to support a wide range of configurations for the CCA [AD2]. The cartridge model used by the FEMC consist in a "supersymmetric" cartridge including all the possible hardware that can be used.

The following table illustrate the base model for a cartridge in the FEMC firmware.
Supersymmetric Cartridge
 
 
 
 
 
Polarization 0 (P0)
 
Polarization 1 (P1)
 
 
SBand1 (S0)
SBand2 (S1)
SBand1 (S0)
SBand2 (S1)
SIS Mixer
1
1
1
1
SIS Magnets
1
1
1
1
LNA Stages
6
6
6
6
SIS Heaters
1
 
1
 
LNA LEDs
1
 
1
 
Schottky Mixers
1
 
1
 
Temperature Sensors
6
 
 
 

If necessary, the configuration file can be used to enable/disable monitor and control access to each individual subsystem according to the following description. Typically all the hardware is enable (default) and selection is made by addressing the correct relative CAN addresses (RCAs) while interacting with the hardware [AD1].

12.1.2.1.4.1 Section: [P0]
  • AVAILABLE = [0-1(Y-N/T-F)] this indicates if this subsystem is available. If this is set to 0 (N or F) any attempt to monitor this sensor will be rejected by the FEMC and an hardware range error will be returned.

12.1.2.1.4.2 Section: [P0_S0]
  • AVAILABLE = [0-1(Y-N/T-F)] this indicates if this subsystem is available. If this is set to 0 (N or F) any attempt to monitor this sensor will be rejected by the FEMC and an hardware range error will be returned.

12.1.2.1.4.3 Section: [P0_S0_SIS]
  • AVAILABLE = [0-1(Y-N/T-F)] this indicates if this subsystem is available. If this is set to 0 (N or F) any attempt to monitor this sensor will be rejected by the FEMC and an hardware range error will be returned.

12.1.2.1.4.4 Section: [P0_S0_SIS_MAG]
  • AVAILABLE = [0-1(Y-N/T-F)] this indicates if this subsystem is available. If this is set to 0 (N or F) any attempt to monitor this sensor will be rejected by the FEMC and an hardware range error will be returned.

12.1.2.1.4.5 Section: [P0_S0_LNA]
  • AVAILABLE = [0-1(Y-N/T-F)] this indicates if this subsystem is available. If this is set to 0 (N or F) any attempt to monitor this sensor will be rejected by the FEMC and an hardware range error will be returned.

12.1.2.1.4.6 Section: [P0_S0_LNA_S0]
  • AVAILABLE = [0-1(Y-N/T-F)] this indicates if this subsystem is available. If this is set to 0 (N or F) any attempt to monitor this sensor will be rejected by the FEMC and an hardware range error will be returned.

12.1.2.1.4.7 Section: [P0_S0_LNA_S1]
  • AVAILABLE = [0-1(Y-N/T-F)] this indicates if this subsystem is available. If this is set to 0 (N or F) any attempt to monitor this sensor will be rejected by the FEMC and an hardware range error will be returned.

12.1.2.1.4.8 Section: [P0_S0_LNA_S2]
  • AVAILABLE = [0-1(Y-N/T-F)] this indicates if this subsystem is available. If this is set to 0 (N or F) any attempt to monitor this sensor will be rejected by the FEMC and an hardware range error will be returned.

12.1.2.1.4.9 Section: [P0_S0_LNA_S3]
  • AVAILABLE = [0-1(Y-N/T-F)] this indicates if this subsystem is available. If this is set to 0 (N or F) any attempt to monitor this sensor will be rejected by the FEMC and an hardware range error will be returned.

12.1.2.1.4.10 Section: [P0_S0_LNA_S4]
  • AVAILABLE = [0-1(Y-N/T-F)] this indicates if this subsystem is available. If this is set to 0 (N or F) any attempt to monitor this sensor will be rejected by the FEMC and an hardware range error will be returned.

12.1.2.1.4.11 Section: [P0_S0_LNA_S5]
  • AVAILABLE = [0-1(Y-N/T-F)] this indicates if this subsystem is available. If this is set to 0 (N or F) any attempt to monitor this sensor will be rejected by the FEMC and an hardware range error will be returned.

12.1.2.1.4.12 Section: [P0_S1]
  • AVAILABLE = [0-1(Y-N/T-F)] this indicates if this subsystem is available. If this is set to 0 (N or F) any attempt to monitor this sensor will be rejected by the FEMC and an hardware range error will be returned.

12.1.2.1.4.13 Section: [P0_S1_SIS]
  • AVAILABLE = [0-1(Y-N/T-F)] this indicates if this subsystem is available. If this is set to 0 (N or F) any attempt to monitor this sensor will be rejected by the FEMC and an hardware range error will be returned.

12.1.2.1.4.14 Section: [P0_S1_SIS_MAG]
  • AVAILABLE = [0-1(Y-N/T-F)] this indicates if this subsystem is available. If this is set to 0 (N or F) any attempt to monitor this sensor will be rejected by the FEMC and an hardware range error will be returned.

12.1.2.1.4.15 Section: [P0_S1_LNA]
  • AVAILABLE = [0-1(Y-N/T-F)] this indicates if this subsystem is available. If this is set to 0 (N or F) any attempt to monitor this sensor will be rejected by the FEMC and an hardware range error will be returned.

12.1.2.1.4.16 Section: [P0_S1_LNA_S0]
  • AVAILABLE = [0-1(Y-N/T-F)] this indicates if this subsystem is available. If this is set to 0 (N or F) any attempt to monitor this sensor will be rejected by the FEMC and an hardware range error will be returned.

12.1.2.1.4.17 Section: [P0_S1_LNA_S1]
  • AVAILABLE = [0-1(Y-N/T-F)] this indicates if this subsystem is available. If this is set to 0 (N or F) any attempt to monitor this sensor will be rejected by the FEMC and an hardware range error will be returned.

12.1.2.1.4.18 Section: [P0_S1_LNA_S2]
  • AVAILABLE = [0-1(Y-N/T-F)] this indicates if this subsystem is available. If this is set to 0 (N or F) any attempt to monitor this sensor will be rejected by the FEMC and an hardware range error will be returned.

12.1.2.1.4.19 Section: [P0_S1_LNA_S3]
  • AVAILABLE = [0-1(Y-N/T-F)] this indicates if this subsystem is available. If this is set to 0 (N or F) any attempt to monitor this sensor will be rejected by the FEMC and an hardware range error will be returned.

12.1.2.1.4.20 Section: [P0_S1_LNA_S4]
  • AVAILABLE = [0-1(Y-N/T-F)] this indicates if this subsystem is available. If this is set to 0 (N or F) any attempt to monitor this sensor will be rejected by the FEMC and an hardware range error will be returned.

12.1.2.1.4.21 Section: [P0_S1_LNA_S5]
  • AVAILABLE = [0-1(Y-N/T-F)] this indicates if this subsystem is available. If this is set to 0 (N or F) any attempt to monitor this sensor will be rejected by the FEMC and an hardware range error will be returned.

12.1.2.1.4.22 Section: [P0_LNA_LED]
  • AVAILABLE = [0-1(Y-N/T-F)] this indicates if this subsystem is available. If this is set to 0 (N or F) any attempt to monitor this sensor will be rejected by the FEMC and an hardware range error will be returned.

12.1.2.1.4.23 Section: [P0_SIS_HEATER]
  • AVAILABLE = [0-1(Y-N/T-F)] this indicates if this subsystem is available. If this is set to 0 (N or F) any attempt to monitor this sensor will be rejected by the FEMC and an hardware range error will be returned.

12.1.2.1.4.24 Section: [P0_SCHOTTKY]
  • AVAILABLE = [0-1(Y-N/T-F)] this indicates if this subsystem is available. If this is set to 0 (N or F) any attempt to monitor this sensor will be rejected by the FEMC and an hardware range error will be returned.

12.1.2.1.4.25 Section: [P1]
  • AVAILABLE = [0-1(Y-N/T-F)] this indicates if this subsystem is available. If this is set to 0 (N or F) any attempt to monitor this sensor will be rejected by the FEMC and an hardware range error will be returned.

12.1.2.1.4.26 Section: [P1_S0]
  • AVAILABLE = [0-1(Y-N/T-F)] this indicates if this subsystem is available. If this is set to 0 (N or F) any attempt to monitor this sensor will be rejected by the FEMC and an hardware range error will be returned.

12.1.2.1.4.27 Section: [P1_S0_SIS]
  • AVAILABLE = [0-1(Y-N/T-F)] this indicates if this subsystem is available. If this is set to 0 (N or F) any attempt to monitor this sensor will be rejected by the FEMC and an hardware range error will be returned.

12.1.2.1.4.28 Section: [P1_S0_SIS_MAG]
  • AVAILABLE = [0-1(Y-N/T-F)] this indicates if this subsystem is available. If this is set to 0 (N or F) any attempt to monitor this sensor will be rejected by the FEMC and an hardware range error will be returned.

12.1.2.1.4.29 Section: [P1_S0_LNA]
  • AVAILABLE = [0-1(Y-N/T-F)] this indicates if this subsystem is available. If this is set to 0 (N or F) any attempt to monitor this sensor will be rejected by the FEMC and an hardware range error will be returned.

12.1.2.1.4.30 Section: [P1_S0_LNA_S0]
  • AVAILABLE = [0-1(Y-N/T-F)] this indicates if this subsystem is available. If this is set to 0 (N or F) any attempt to monitor this sensor will be rejected by the FEMC and an hardware range error will be returned.

12.1.2.1.4.31 Section: [P1_S0_LNA_S1]
  • AVAILABLE = [0-1(Y-N/T-F)] this indicates if this subsystem is available. If this is set to 0 (N or F) any attempt to monitor this sensor will be rejected by the FEMC and an hardware range error will be returned.

12.1.2.1.4.32 Section: [P1_S0_LNA_S2]
  • AVAILABLE = [0-1(Y-N/T-F)] this indicates if this subsystem is available. If this is set to 0 (N or F) any attempt to monitor this sensor will be rejected by the FEMC and an hardware range error will be returned.

12.1.2.1.4.33 Section: [P1_S0_LNA_S3]
  • AVAILABLE = [0-1(Y-N/T-F)] this indicates if this subsystem is available. If this is set to 0 (N or F) any attempt to monitor this sensor will be rejected by the FEMC and an hardware range error will be returned.

12.1.2.1.4.34 Section: [P1_S0_LNA_S4]
  • AVAILABLE = [0-1(Y-N/T-F)] this indicates if this subsystem is available. If this is set to 0 (N or F) any attempt to monitor this sensor will be rejected by the FEMC and an hardware range error will be returned.

12.1.2.1.4.35 Section: [P1_S0_LNA_S5]
  • AVAILABLE = [0-1(Y-N/T-F)] this indicates if this subsystem is available. If this is set to 0 (N or F) any attempt to monitor this sensor will be rejected by the FEMC and an hardware range error will be returned.

12.1.2.1.4.36 Section: [P1_S1]
  • AVAILABLE = [0-1(Y-N/T-F)] this indicates if this subsystem is available. If this is set to 0 (N or F) any attempt to monitor this sensor will be rejected by the FEMC and an hardware range error will be returned.

12.1.2.1.4.37 Section: [P1_S1_SIS]
  • AVAILABLE = [0-1(Y-N/T-F)] this indicates if this subsystem is available. If this is set to 0 (N or F) any attempt to monitor this sensor will be rejected by the FEMC and an hardware range error will be returned.

12.1.2.1.4.38 Section: [P1_S1_SIS_MAG]
  • AVAILABLE = [0-1(Y-N/T-F)] this indicates if this subsystem is available. If this is set to 0 (N or F) any attempt to monitor this sensor will be rejected by the FEMC and an hardware range error will be returned.

12.1.2.1.4.39 Section: [P1_S1_LNA]
  • AVAILABLE = [0-1(Y-N/T-F)] this indicates if this subsystem is available. If this is set to 0 (N or F) any attempt to monitor this sensor will be rejected by the FEMC and an hardware range error will be returned.

12.1.2.1.4.40 Section: [P1_S1_LNA_S0]
  • AVAILABLE = [0-1(Y-N/T-F)] this indicates if this subsystem is available. If this is set to 0 (N or F) any attempt to monitor this sensor will be rejected by the FEMC and an hardware range error will be returned.

12.1.2.1.4.41 Section: [P1_S1_LNA_S1]
  • AVAILABLE = [0-1(Y-N/T-F)] this indicates if this subsystem is available. If this is set to 0 (N or F) any attempt to monitor this sensor will be rejected by the FEMC and an hardware range error will be returned.

12.1.2.1.4.42 Section: [P1_S1_LNA_S2]
  • AVAILABLE = [0-1(Y-N/T-F)] this indicates if this subsystem is available. If this is set to 0 (N or F) any attempt to monitor this sensor will be rejected by the FEMC and an hardware range error will be returned.

12.1.2.1.4.43 Section: [P1_S1_LNA_S3]
  • AVAILABLE = [0-1(Y-N/T-F)] this indicates if this subsystem is available. If this is set to 0 (N or F) any attempt to monitor this sensor will be rejected by the FEMC and an hardware range error will be returned.

12.1.2.1.4.44 Section: [P1_S1_LNA_S4]
  • AVAILABLE = [0-1(Y-N/T-F)] this indicates if this subsystem is available. If this is set to 0 (N or F) any attempt to monitor this sensor will be rejected by the FEMC and an hardware range error will be returned.

12.1.2.1.4.45 Section: [P1_S1_LNA_S5]
  • AVAILABLE = [0-1(Y-N/T-F)] this indicates if this subsystem is available. If this is set to 0 (N or F) any attempt to monitor this sensor will be rejected by the FEMC and an hardware range error will be returned.

12.1.2.1.4.46 Section: [P1_LNA_LED]
  • AVAILABLE = [0-1(Y-N/T-F)] this indicates if this subsystem is available. If this is set to 0 (N or F) any attempt to monitor this sensor will be rejected by the FEMC and an hardware range error will be returned.

12.1.2.1.4.47 Section: [P1_SIS_HEATER]
  • AVAILABLE = [0-1(Y-N/T-F)] this indicates if this subsystem is available. If this is set to 0 (N or F) any attempt to monitor this sensor will be rejected by the FEMC and an hardware range error will be returned.

12.1.2.1.4.48 Section: [P1_SCHOTTKY]
  • AVAILABLE = [0-1(Y-N/T-F)] this indicates if this subsystem is available. If this is set to 0 (N or F) any attempt to monitor this sensor will be rejected by the FEMC and an hardware range error will be returned.

12.1.2.2 LPR.INI

This files contains the configuration information for the LO Photonic Receiver (LPR).

12.1.2.2.1 Section: [INFO]
  • ESN = [8 bytes of Electronic Serial Number] this field contains the ESN of the LPR.
  • SN = "[String identifying the LPR Serial Number]" this contains the serial number of the LPR.

12.1.2.2.2 Section: [EDFA_PD]
  • POWER_COEFF = [Power coefficient value] this is the coefficient used to calculate the current and power of the EDFA photodetector. The coefficient for each LPR is listed on a sticker attached to the LPR hardware. The default value entered in the delivered FEMCs is 28.0.

12.1.2.3 CRYO.INI

This files contains the coefficients of a sixth order polynomial needed to calculate the actual temperature of the TVO sensors installed in the cryostat.

The data necessary to fill up this file can be found on EDM. Under the CIN 40.03 there is a folder called "Cryostat Production" and under that folder the data delivered by the cryostat manufacturer for each cryostat.

Open the folder relative to the cryostat of interest (ex. "Cryostat #25"). In this folder there is typically a document called something like "Cryostat housekeeping wiring and thermometry 40.03.00.00-001-25".

This document contains two file: a PDF describing the location and name of the sensors and an excel file containing the coefficients for the calibration curves.

The data contained in these two files is used to complete the sections in the CRYO.INI configuration file.

NOTE: The 7 character limitation for the TVO_NO is in place for FEMC firmware rev.2.5.0 and lower. For rev.2.5.1 and higher, the new limit will be 31 characters.

12.1.2.3.1 Section: [INFO]
  • ESN = [8 bytes of Electronic Serial Number] this field contains the ESN of the cryostat.
  • SN = "[String identifying the cryostat Serial Number]" this contains the serial number of the cryostat.

12.1.2.3.2 Section: [CRYOCOOLER_4K] ("4K Cryocooler stage" on the data file)
  • TVO_NO = "[String identifying the TVO sensor number]" this string contains the TVO sensor number (it can be at most 7 characters long)
  • TVO_COEFFS = [Comma separated array of 7 coefficients starting with x^0] this array contains the coefficients necessary to evaluate the 6th order polynomial returning the temperature of this TVO sensor.

  • TVO_NO = "[String identifying the TVO sensor number]" this string contains the TVO sensor number (it can be at most 7 characters long)
  • TVO_COEFFS = [Comma separated array of 7 coefficients starting with x^0] this array contains the coefficients necessary to evaluate the 6th order polynomial returning the temperature of this TVO sensor.

  • TVO_NO = "[String identifying the TVO sensor number]" this string contains the TVO sensor number (it can be at most 7 characters long)
  • TVO_COEFFS = [Comma separated array of 7 coefficients starting with x^0] this array contains the coefficients necessary to evaluate the 6th order polynomial returning the temperature of this TVO sensor.

12.1.2.3.5 Section: [PLATE_4K_FAR_1] ("4K plate far side b" on the data file)
  • TVO_NO = "[String identifying the TVO sensor number]" this string contains the TVO sensor number (it can be at most 7 characters long)
  • TVO_COEFFS = [Comma separated array of 7 coefficients starting with x^0] this array contains the coefficients necessary to evaluate the 6th order polynomial returning the temperature of this TVO sensor.

12.1.2.3.6 Section: [PLATE_4K_FAR_2] ("4K plate far side a" on the data file)
  • TVO_NO = "[String identifying the TVO sensor number]" this string contains the TVO sensor number (it can be at most 7 characters long)
  • TVO_COEFFS = [Comma separated array of 7 coefficients starting with x^0] this array contains the coefficients necessary to evaluate the 6th order polynomial returning the temperature of this TVO sensor.

12.1.2.3.7 Section: [CRYOCOOLER_12K] ("15K shield top" on the data file)
  • TVO_NO = "[String identifying the TVO sensor number]" this string contains the TVO sensor number (it can be at most 7 characters long)
  • TVO_COEFFS = [Comma separated array of 7 coefficients starting with x^0] this array contains the coefficients necessary to evaluate the 6th order polynomial returning the temperature of this TVO sensor.

  • TVO_NO = "[String identifying the TVO sensor number]" this string contains the TVO sensor number (it can be at most 7 characters long)
  • TVO_COEFFS = [Comma separated array of 7 coefficients starting with x^0] this array contains the coefficients necessary to evaluate the 6th order polynomial returning the temperature of this TVO sensor.

12.1.2.3.9 Section: [PLATE_12K_FAR] ("15K plate far side" on the data file)
  • TVO_NO = "[String identifying the TVO sensor number]" this string contains the TVO sensor number (it can be at most 7 characters long)
  • TVO_COEFFS = [Comma separated array of 7 coefficients starting with x^0] this array contains the coefficients necessary to evaluate the 6th order polynomial returning the temperature of this TVO sensor.

12.1.2.3.10 Section: [PLATE_12K_SHIELD] ("15K shield top" on the data file)
  • TVO_NO = "[String identifying the TVO sensor number]" this string contains the TVO sensor number (it can be at most 7 characters long)
  • TVO_COEFFS = [Comma separated array of 7 coefficients starting with x^0] this array contains the coefficients necessary to evaluate the 6th order polynomial returning the temperature of this TVO sensor.

12.1.2.4 ESNS.INI

This files deals with the electronic serial numbers installed in the front end.Each front end subsystem requiring configuration is identified by an ESN. During boot time the FEMC collects the list of available ESNs and makes it available to the control software that can check the list against the data stored in the configuration database and after a series of handshakes and checks decide if it is necessary to update any of the configuration files in the FEMC module.

There is one section that can be edited in this file to enable/disable and modify the behavior of the ESNs simulator.

12.1.2.4.1 Section: [SIMULATOR]
  • USE = [Y/N] this will determine if the simulator is enabled or not. If it is enabled, then a certain number of ESNs will be simulated. If it is disabled, then the FEMC will look for available ESNs on the OWB.
  • SIMDEV = [1-35] this is the number of device to simulate
  • SIMBASE = [0-65535] this is a base number that is used in the generation of the simulated ESNs. If the number is left blank, and the simulator is enabled, it will be calculated once the first time the FEMC module is booted and used from that point on. It can be set to a particular value to identify the FEMC module. The format of the simulated number is as follows:
    • byte[7] -> 0
    • byte[6] -> MSB of SIMBASE
    • byte[5] -> LSB of SIMBASE
    • byte[4] -> MSB of number based on FEMC hardware
    • byte[3] -> next byte of number based on FEMC hardware
    • byte[2] -> next byte of number based on FEMC hardware
    • byte[1] -> LSB of number based on FEMC hardware
    • byte[0] -> progressive number from 0 to SIMDEV-1

12.1.2.5 FRONTEND.INI

This files is used by the FEMC for the following:* specify the name and CRC check of the configuration files for the front end subsystems* enable/disable bands

12.1.2.5.1 Section: [LPR]
  • FILE = this indicates the name of the file containing configuration information for the LPR M&C subsystem
  • CRC = [value] this is the expected CRC for the configuration file pointed at by FILE to be implemented
12.1.2.5.2 Section: [CRYO]
  • FILE = this indicates the name of the file containing configuration information for the cryostat M&C subsystem
  • CRC = [value] this is the expected CRC for the configuration file pointed at by FILE to be implemented
12.1.2.5.3 Section: [BAND] (where x is the band number)
  • AVAILABLE = [0-1(Y-N/T-F)] this indicates if band x should be considered available by the FEMC. If this is set to 0 (N or F) any attempt to communicate with this band will be rejected by the FEMC and, for monitor messages, an hardware blocked error will be returned.
12.1.2.5.4 Section: [CART] (where x is the band number)
  • FILE = this indicates the name of the file containing configuration information for band x cold cartridge M&C subsystem
  • CRC = [value] this is the expected CRC for the configuration file pointed at by FILE to be implemented
12.1.2.5.5 Section: [WCA] (where x is the band number)
  • FILE = this indicates the name of the file containing configuration information for band x warm cartridge M&C subsystem
  • CRC = [value] this is the expected CRC for the configuration file pointed at by FILE to be implemented

12.1.3 Software/Firmware/Configuration updates

12.1.3.1 ARCOM Pegasus

Since every new software release might have a slightly different update procedure, step by step instruction on how to update to that release are included with each release.

Please check the related [RF1] EDM page for more information.

The following is a general procedure that gives an idea on how the update is performed depending on the communication protocol used

12.1.3.1.1 RS232

This procedure can be used for any file, not just the main executable.

The first step to update the software using the RS232 interface is to connect to the module as explained in the previous section Connecting to the FEMC.

In the following example we are going to use HyperTerminal. Any communication software that meets the requirements outlined in section Connecting to the FEMC can be used.

Once the communication has been established, the screen should look something like the following:

FEMC - 01 - Connect.png

Once the connection is established, it is necessary to stop the software by pressing
   "q" <Enter>

This will bring you to the DOS prompt:

FEMC - 02 - Software Stopped.png

The directory structure will be similar to the following:

FEMC - 03 - Directory.png

The executable that is run when the module is powered is fe_mc.exe.

Before proceeding with the update turn on the DOS verification with:
   verify on <Enter>

this will force DOS to verify that data written to disk was recorded correctly.

It is a good procedure at this point to make a backup of the following files:
  • fe_mc.exe
  • *.ini
This can be accomplished by either creating a new directory with some meaningful name (e.g. current version number) using the:
   mkdir 02-01-02 <Enter>

The files can then be copied in the backup directory using the following commands:
   copy fe_mc.exe 02-01-02 <Enter>
   copy *.ini 02-01-02 <Enter>

Make sure that the directory is C:\ALMA>. This can be assured by issuing the following command:
   cd c:\alma

Download the update from EDM [RF1] (or any other file you might need to update). If available, please follow the instruction file attached to the release: those instruction have precedence over this general procedure.

The following procedure gives a guideline on how to update the main executable file. This procedure can be used to update any other file, for example the configuration (*.INI) files.

The executable should be renamed fe_mc.exe before the transfer.

The transfer will overwrite the existing files.

At this point you can choose the option Send File... under the Transfer menu. The protocol should be set to Zmodem with Crash Recovery. Browse to the location of the fe_mc.exe file and select it. Push the Send button. The transfer should start automatically.

Should the transfer fail, the an alternative procedure can be followed.

Verify that the system is still responsive by pressing Enter a few times. If this is not the case, power cycle the module and start from the beginning. (The latest version of the module will allow you to drop to the DOS prompt during the boot-up sequence).

Once you are back in the C:\ALMA>directory type the following:
   rz <Enter>

This will initialize the Zmodem reception.

At this point you can choose the option Send File... under the Transfer menu. The protocol should be set to Zmodem with Crash Recovery. Browse to the location of the fe_mc.exe file and select it. Push the Send button. The transfer should proceed automatically.

Repeat the working procedure for all the files that need to be transferred. Follow any additional procedure described in the update instruction attached to the distribution. HyperTerminal doesn't support multiple files transfer. The FEMC module supports that feature so if you are going to use a different terminal software, you can send more than one file at the time.

To check that the software was transferred correctly, the main executable can be launched:
   fe_mc.exe <Enter>

The software should go through the initialization and show something like this:

FEMC - 01 - Connect.png

The version can be verified by:
   i <Enter>

This should give you the version information:

FEMC - 04 - Version.png

To run the FEMC with the new software, power cycle the module.

To verify that configuration (*.INI) files have been transferred correctly you can open them in the NED editor (see section above).

12.1.3.2 AMBSI1

...

12.1.3.3 Serial Mux

This section will be expanded to include the information contained in [AD8].

13 Troubleshooting

13.1 Vacuum System

13.1.1 The gate valve is returning 0x04 (OVER_CURRENT)

When the gate valve returns over current it is most probably stuck where the over current occurred. To recover from an over current situation, it is necessary to issue a move to command in the direction opposite to the last that was issued.

If the last move to command is not known, it is possible to review the last issued control message by sending a monitor request to the control address. The FEMC firmware allows the control software to issue monitor requests addressed to control RCAs. These will return the last issued command to that address. In the case of the gate valve state, a monitor request sent to the 0x1C03C will return the payload of the last control message sent to that RCA.

Once the direction of motion originally commanded is know, it is necessary to send a control request to try move the gate valve in the opposite direction.
   Example:

   Control: 0x01    -> 0x1C03C // Open gate valve
   Monitor: 0xC03C  -> 0x04    // Gate valve found in over current condition
   Monitor: 0x1C03C -> 0x01    // If the last *move to* command is unknown, query the FEMC
   Control: 0x00    -> 0x1C03C // Issue the opposite motion command: close gate valve

After this series of command, a further monitor request on the monitor RCA 0xC03C should return either 0x02 (gate valve position unknown => the gate valve is moving) or 0x00 (gate valve closed). If 0x03 (error) or 0x04(over current) are returned, follow this procedure:
   Control: 0x00   -> 0x1C03C
   Wait 30 seconds
   Control: 0x01   -> 0x1C03C
   Wait 30 seconds
   Control: 0x00   -> 0x1C03C
   Wait 30 seconds
   Monitor: 0xC03C -> ?

If the result of the last operation is not 0x00, then the hardware should be inspected by a qualified technician and, if necessary, replaced.

13.1.2 The solenoid valve doesn't open or keeps opening and closing during pump-down

Verify that the pressure limits for the relays in the vacuum gauge controller are set according to the following table:

Relay SP1 SP2 SP3
Low
20
10^-2
10
High
30
3*10^-2
30

(values are in mbar)

The information on how to change the values can be found in Chapter 5 of the Leybold Center Two and Center Three manual [RD2].

13.1.3 Cannot control the solenoid valve in remote mode but the relays are clicking and the valve works in local mode.

Verify that the wiring behind the 24V switching power supply is correct and the 24V return (common, GND) is not connected to the sense terminal instead of the 24V return. If this is the case is possible that monitoring the voltage will give the correct answer (24V) but the power supply will not be able to sink enough current to operate correctly.

13.1.4 The pressure values look correct on the gauge controller but the values returned by the M&C systems are off

Verify that the sensors are connected properly to the back of the vacuum gauge controller.

Pressure1 is the cryostat pressure. Expected values are from ambient pressure all the way down to 10^-6 mbar.

Pressure2 is the port pressure. Expected value are from ambient pressure down to 2-3 mbar.

If pressure2 is indicating a pressure below 2-3 mbar, the wiring of the gauge controller is most probably wrong.

13.1.5 The pressure values and the PRT (110K) stage temperature values returned by the M&C system are ok but all the TVO (4K and 15K) temperature are off (~ 3 times bigger than expected).

First of all verify that the file CRYO.INI, stored in the FEMC module, has the correct coefficient for the TVO sensors installed in the cryostat.

If the coefficients are ok and the values are still wrong then update the FEMC software to rev.2.4.0 or newer. These revisions support both the new and the old hardware revisions of the Cryostat M&C boards. The newer revision has a different scaling factor for the TVO temperatures and FEMC software older than rev.2.4.0 will not support it.

13.2 Monitor and control system

13.2.1 A band is not responding to control messages and every monitor message directed to that band returns the "timed-out" status byte

Verify that the corresponding voltage regulator board is installed in the CPDS and that is properly working.

If the problem persists, verify the power and monitor and control cabling for that band including the correct installation of the mating plate at the base of the band.

13.2.2 A band is not responding to control messages and every monitor message directed to that band returns the "hardware blocked" status byte

There are two possible reasons for this behavior:
  • the command to power up the band has not been sent
  • the band is disabled in the FRONTEND.INI configuration file

If the command to power up the band has not been issued, the FEMC firmware will block every message directed to that band to avoid timeouts on the CAN bus and, for the monitor message, it will return just the status byte with the hardware blocked code [refer to the ICD for more information].

In case the command has been sent, then the band might be disabled in the FRONTEND.INI file.

To verify if the FRONTEND.INI file is configured correctly, refer to the monitor and control section under the maintenance chapter of this document. If the band is disabled in the FRONTEND.INI file, use the NED editor to modify the file and enable the band, then reboot the FEMC module. This will load the new configuration file.

13.2.3 The FEMC stop responding after updating the CRYO.INI file

First verify the version of the FEMC firmware.

For rev.2.5.0 or prior, verify that the length of the TVO_NO is less then 7 characters.

For rev.2.6.0 or later, verify that the length of the TVO_NO is less than 31 characters.
Topic attachments
I Attachment Action Size Date Who Comment
FEMC_-_01_-_Connect.pngpng FEMC_-_01_-_Connect.png manage 26 K 2009-05-18 - 17:48 UnknownUser FEMC - HyperTerminal - Connection
FEMC_-_02_-_Software_Stopped.pngpng FEMC_-_02_-_Software_Stopped.png manage 20 K 2009-05-18 - 17:56 UnknownUser FEMC - HyperTerminal - Software Stopped
FEMC_-_03_-_Directory.pngpng FEMC_-_03_-_Directory.png manage 24 K 2009-05-18 - 18:01 UnknownUser FEMC - HyperTerminal - Directory
FEMC_-_04_-_Version.pngpng FEMC_-_04_-_Version.png manage 26 K 2009-05-18 - 18:54 UnknownUser FEMC - HyperTerminal - Version
FEMC_-_05_-_NED.pngpng FEMC_-_05_-_NED.png manage 21 K 2009-05-18 - 18:57 UnknownUser FEMC - HyperTerminal - NED
Hold_Time.pngpng Hold_Time.png manage 15 K 2012-12-13 - 14:05 MorganMcLeod  
Interlock.pdfpdf Interlock.pdf manage 33 K 2010-03-19 - 11:07 UnknownUser Schematic for the solenoid valve interlock
cryostat-components.PNGPNG cryostat-components.PNG manage 14 K 2009-03-12 - 13:47 MorganMcLeod Block diagram of the cryostat components
Topic revision: r61 - 2013-03-15, RodrigoBrito
This site is powered by FoswikiCopyright © by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding NRAO Public Wiki? Send feedback