This shows you the differences between two versions of the page.
Both sides previous revision Previous revision Next revision | Previous revision Last revision Both sides next revision | ||
ds:projects:cryring:bpm [2020/11/14 21:01] rgeissler |
ds:projects:cryring:bpm [2023/02/09 11:02] rgeissler |
||
---|---|---|---|
Line 1: | Line 1: | ||
====== Cryring BPM Gateware Documentation ====== | ====== Cryring BPM Gateware Documentation ====== | ||
- | This documentation was automatically generated from Latex. | + | This documentation was automatically generated from the Markdown file '' |
- | Any changes should be applied to the Latex sources | + | Any changes should be applied to the Markdown file, which can be converted to DokuWiki format using the script '' |
- | The Latex sources | + | |
- | The documentation is also available as a PDF: {{: | + | The documentation is also available as a PDF: {{: |
+ | |||
+ | An up to date PDF version can be downloaded from the CI/CD section: https:// | ||
+ | |||
+ | Documentation that is common to multiple FPGA based projects can be found here: [[: | ||
+ | It is also available as a PDF: {{: | ||
===== Resources ===== | ===== Resources ===== | ||
- | All of the code of this project, the helper scripts | + | The code of this project and also the source of this documentation are under version control in a Git repository whose upstream is:\\ |
+ | https:// | ||
+ | The relevant branch is //master//. | ||
+ | |||
+ | Additional datasheets and papers are included as a Git submodule of the main Git repository. The upstream of the submodule is:\\ | ||
+ | https:// | ||
+ | The relevant branch is //master//. | ||
- | Additional datasheets and papers are included as a Git submodule of the main Git repository. The upstream of the submodule is: https:// | + | Common code is included as a Git submodule of the main Git repository. The upstream of the submodule is:\\ |
+ | https:// | ||
+ | The relevant branch is //master//. | ||
- | Installation scripts to set up a Gitlab runner for continuous integration including all necessary software to build the gateware can be found in a Git repository whose upstream is: https:// | + | Installation scripts to set up a Gitlab runner for continuous integration including all necessary software to build the gateware can be found in a Git repository whose upstream is:\\ |
+ | https:// | ||
+ | The relevant branch is //master//. | ||
- | ====== 1 Introduction | + | ===== 1 Introduction ===== |
This document describes the gateware (= FPGA firmware) implementation of the Beam Position Monitor (BPM) for the Cryring accelerator at GSI. The term Trajectory Measurement System (TMS) is also common for this system and is used as a synonym for BPM. The BPM measures the horizontal and vertical beam positions at nine places of the accelerator ring, resulting in 18 location results. | This document describes the gateware (= FPGA firmware) implementation of the Beam Position Monitor (BPM) for the Cryring accelerator at GSI. The term Trajectory Measurement System (TMS) is also common for this system and is used as a synonym for BPM. The BPM measures the horizontal and vertical beam positions at nine places of the accelerator ring, resulting in 18 location results. | ||
- | There had been a previous implementation by Piotr Miedzik, but since no documentation could be found besides a conference paper [[#15_references|[2]]], it was decided to reimplement the gateware. | + | There had been a previous implementation by Piotr Miedzik, but since no documentation could be found besides a conference paper [[#10_references|[2]]], it was decided to reimplement the gateware. |
- | ===== 1.1 Measurement principle | + | ==== 1.1 Measurement principle ==== |
At each of the 18 measurement spots two capacitor plates are used to detect the electrostatic induction of the passing by charged particle bunches. | At each of the 18 measurement spots two capacitor plates are used to detect the electrostatic induction of the passing by charged particle bunches. | ||
- | {{: | + | {{: |
- | **Figure 1.1:** Mechanical drawing of a single BPM. The two segments of the slotted tube are the capacitor plates. The box on the top is the amplifier. Image origin: [[#15_references|[3]]] | + | **Figure 1.1:** Mechanical drawing of a single BPM. The two segments of the slotted tube are the capacitor plates. The box on the top is the amplifier. Image origin: [[#10_references|[3]]] |
The 36 voltages of the capacitor plates are amplified and led via coaxial cables to a single evaluation point where the analog to digital conversion and the digital processing takes place. | The 36 voltages of the capacitor plates are amplified and led via coaxial cables to a single evaluation point where the analog to digital conversion and the digital processing takes place. | ||
- | {{: | + | {{: |
- | **Figure 1.2:** Cryring BPM system overview. This document only describes the implementation of the block labeled //125 MSa/s ADC DAQ System (36 ch.)//, excluding the software part. Image origin: [[#15_references|[3]]] | + | **Figure 1.2:** Cryring BPM system overview. This document only describes the implementation of the block labeled //125 MSa/s ADC DAQ System (36 ch.)//, excluding the software part. Image origin: [[#10_references|[3]]] |
- | The positions of the particle beam are calculated respectively from the voltage difference of two related capacitor plates using the algorithm described in [[# | + | The positions of the particle beam are calculated respectively from the voltage difference of two related capacitor plates using the algorithm described in chapter |
- | ===== 1.2 Processing hardware ===== | + | {{: |
- | Each of the 36 voltages coming from the amplifiers at the capacitor plates is sampled by a Renesas ISLA216P ADC at a sampling rate of 125 MHz with a resolution of 16 bits. Respectively four of the ADCs are placed on a single FMC board. Respectively two (ore only one for the last one) of the FMC boards are mounted on an AFC carrier board which is equipped with a Xilinx Artix XC7A200T FPGA for data processing [[# | + | **Figure 1.3:** Schematic |
- | {{: | + | The moving average filter |
- | **Figure 1.3:** An AFC carrier board with two mounted ADC FMC boards. | + | The Chebyshev filter |
- | The FPGA is a mid-range device providing the following resources [[#15_references|[5]]]: | + | {{: |
+ | |||
+ | **Figure 1.4:** Frequency response of the Chebyshev filter | ||
+ | |||
+ | ==== 1.2 Processing hardware ==== | ||
+ | |||
+ | Each of the 36 voltages coming from the amplifiers at the capacitor plates is sampled by a Renesas ISLA216P ADC at a sampling rate of 125 MHz with a resolution of 16 bits. Respectively four of the ADCs are placed on a single FMC board. Respectively two (ore only one for the last one) of the FMC boards are mounted on an AFC carrier board which is equipped with a Xilinx Artix XC7A200T FPGA for data processing [[# | ||
+ | |||
+ | {{: | ||
+ | |||
+ | **Figure 1.5:** An AFC carrier board with two mounted ADC FMC boards. The FPGA is located under the blue heat spreader. | ||
+ | |||
+ | The FPGA is a mid-range device providing the following resources [[#10_references|[5]]]: | ||
* Logic cells: 215,360 | * Logic cells: 215,360 | ||
Line 51: | Line 77: | ||
* Multipliers/ | * Multipliers/ | ||
- | The whole system uses five AFC carrier boards which are mounted in a MicroTCA crate together with a timing receiver and a CPU unit for post processing. Each of the five FPGAs is responsible for the processing of up to eight ADC data streams. The communication between the CPU unit and the FPGAs takes place via PCI Express over the so called backplane of the MicroTCA crate. | + | The whole system uses five AFC carrier boards which are mounted in a MicroTCA crate together with a timing receiver and a FEC for post processing. Each of the five FPGAs is responsible for the processing of up to eight ADC data streams. The communication between the FEC and the FPGAs takes place via PCI Express over the so called backplane of the MicroTCA crate. |
- | {{: | + | {{: |
- | **Figure 1.4:** MicroTCA crate with from left to right: power supply, MCH, CPU unit, timing receiver, 5 AFC boards with 9 mounted FMC ADC boards, second MCH | + | **Figure 1.6:** MicroTCA crate with from left to right: power supply, MCH, FEC, timing receiver, 5 AFC boards with 9 mounted FMC ADC boards, second MCH |
This document describes the gatewares of the FPGAs on the five AFC carrier boards. The gatewares are identical independent of the number of mounted FMC boards. | This document describes the gatewares of the FPGAs on the five AFC carrier boards. The gatewares are identical independent of the number of mounted FMC boards. | ||
- | ====== 2 BPM algorithm | + | ===== 2 BPM algorithm ===== |
The beam position is calculated from the measurement of the voltages of two corresponding plates: | The beam position is calculated from the measurement of the voltages of two corresponding plates: | ||
- | {{: | + | {{: |
with | with | ||
- | {{: | + | {{: |
and | and | ||
- | {{: | + | {{: |
- | where {{: | + | where {{: |
- | ===== 2.1 Capacitance correction | + | ==== 2.1 Capacitance correction ==== |
The capacitance of the two corresponding capacitor plates can differ from their nominal value so that one of the voltages has to be corrected by multiplying a correction factor: | The capacitance of the two corresponding capacitor plates can differ from their nominal value so that one of the voltages has to be corrected by multiplying a correction factor: | ||
- | {{: | + | {{: |
- | {{: | + | {{: |
- | The default value of {{: | + | The default value of {{: |
- | ===== 2.2 Least squares algorithm | + | ==== 2.2 Least squares algorithm ==== |
- | A linear least squares approach is used to reduce measurement errors. The choice of the algorithm is described in [[#15_references|[1]]]. The optimal approach would be an orthogonal least squares algorithm. Since the relative error of the difference signal {{: | + | A linear least squares approach is used to reduce measurement errors. The choice of the algorithm is described in [[#10_references|[1]]]. The optimal approach would be an orthogonal least squares algorithm. Since the relative error of the difference signal {{: |
- | {{: | + | {{: |
Minimizing | Minimizing | ||
- | {{: | + | {{: |
via partial differentiation | via partial differentiation | ||
- | {{: | + | {{: |
and | and | ||
- | {{: | + | {{: |
leads to | leads to | ||
- | {{: | + | {{: |
- | **Equation 2.1:** BPM algorithm | ||
- | ==== 2.2.1 Variance | + | **Equation 2.10:** BPM algorithm |
+ | |||
+ | === 2.2.1 Variance === | ||
The variance of the least squares algorithm result is calculated as follows: | The variance of the least squares algorithm result is calculated as follows: | ||
- | {{: | + | {{: |
- | ==== 2.2.2 Intensity | + | === 2.2.2 Intensity === |
The beam intensity is calculated as follows: | The beam intensity is calculated as follows: | ||
- | {{: | + | {{: |
- | ===== 2.3 Averaging | + | ==== 2.3 Averaging ==== |
- | For further reducing the data rate and reducing the measurement noise, the result of the least squares algorithm is averaged over an adjustable number of samples {{: | + | For further reducing the data rate and reducing the measurement noise, the result of the least squares algorithm is averaged over an adjustable number of samples {{: |
- | {{: | + | {{: |
- | ==== 2.3.1 Variance | + | === 2.3.1 Variance === |
The variance of the averaging result is calculated by averaging the variances and dividing by the averaging length: | The variance of the averaging result is calculated by averaging the variances and dividing by the averaging length: | ||
- | {{: | + | {{: |
- | ==== 2.3.2 Intensity | + | === 2.3.2 Intensity === |
The average intensity is calculated as follows: | The average intensity is calculated as follows: | ||
- | {{: | + | {{: |
- | ===== 2.4 Control signals | + | ==== 2.4 Control signals ==== |
- | ==== 2.4.1 Gate ==== | + | === 2.4.1 Gate === |
- | There is an input signal coming from a timing receiver that gates the calculation of the least squares algorithm. A calculation will start with the low to high transition of the gate signal and will be repeated continuously until a high to low transition is detected, after which the current calculation will still be completed. | + | There is an input signal coming from a timing receiver that gates the calculation of the least squares algorithm. By default, the gateware is configured to use the first MLVDS line as an input for the gate signal. A calculation will start with the low to high transition of the gate signal and will be repeated continuously until a high to low transition is detected, after which the current calculation will still be completed. |
- | ==== 2.4.2 RF pulse ==== | + | === 2.4.2 RF pulse === |
The RF pulse signal is intended to synchronize the calculation of the least squares algorithm with the frequency of the particle bunches. A possible previous calculation of the least squares algorithm will be finished and a new calculation will be started whenever a RF pulse is detected. | The RF pulse signal is intended to synchronize the calculation of the least squares algorithm with the frequency of the particle bunches. A possible previous calculation of the least squares algorithm will be finished and a new calculation will be started whenever a RF pulse is detected. | ||
- | ===== 2.5 Parameters | + | ==== 2.5 Parameters ==== |
- | ==== 2.5.1 Least squares algorithm calculation length | + | === 2.5.1 Least squares algorithm calculation length === |
This parameter defines the number of ADC samples that will be taken into account by the least squares algorithm if no RF pulses are present. The detection of a RF pulse will override this parameter. The overriding will only work as expected if the calculation length is set to a value that is longer than the period of the RF pulses. | This parameter defines the number of ADC samples that will be taken into account by the least squares algorithm if no RF pulses are present. The detection of a RF pulse will override this parameter. The overriding will only work as expected if the calculation length is set to a value that is longer than the period of the RF pulses. | ||
Line 157: | Line 184: | ||
The available range of values for the calculation length is 3 to 65536. | The available range of values for the calculation length is 3 to 65536. | ||
- | ==== 2.5.2 Averaging length | + | === 2.5.2 Averaging length === |
This parameter defines the number of least squares algorithm results that will be taken into account by the averaging algorithm. | This parameter defines the number of least squares algorithm results that will be taken into account by the averaging algorithm. | ||
Line 163: | Line 190: | ||
The available range of values for the averaging length is 1 to 1048576. | The available range of values for the averaging length is 1 to 1048576. | ||
- | ====== 3 Peripheral devices | + | ===== 3 Common FPGA based projects documentation ===== |
+ | |||
+ | This project incorporates the code from the // | ||
+ | |||
+ | ==== 3.1 Common monitoring and control features ==== | ||
+ | |||
+ | Documentation about the register bank, the architecture information storage and the observer can be found here:\\ | ||
+ | https:// | ||
+ | |||
+ | ==== 3.2 FPGA Observer ==== | ||
+ | |||
+ | There is a expert GUI that can be used together with multiple projects: | ||
+ | https:// | ||
+ | |||
+ | ==== 3.3 Build flow and simulation ==== | ||
+ | |||
+ | You can find instructions on how to build and simulate the gateware here:\\ | ||
+ | https:// | ||
+ | |||
+ | ==== 3.4 Helper scripts ==== | ||
+ | |||
+ | You can find usefull scripts here:\\ | ||
+ | https:// | ||
+ | |||
+ | ==== 3.5 Continuous integration environment ==== | ||
+ | |||
+ | Information about the continuous integration setup can be found here:\\ | ||
+ | https:// | ||
+ | |||
+ | ==== 3.6 Programming and hardware configuration ==== | ||
+ | |||
+ | You can find instructions on how to program the FPGA and configure other hardware here: | ||
+ | |||
+ | https:// | ||
+ | |||
+ | ===== 4 Peripheral devices ===== | ||
There are three different peripheral devices on each of the FMC ADC boards that have to be configured by the gateware. Since they have no persistent storage they have to configured after every power cycle: | There are three different peripheral devices on each of the FMC ADC boards that have to be configured by the gateware. Since they have no persistent storage they have to configured after every power cycle: | ||
Line 171: | Line 233: | ||
* ISLA216P ADC | * ISLA216P ADC | ||
- | ===== 3.1 Si571 programmable VCXO ===== | + | ==== 4.1 Si571 programmable VCXO ==== |
The Si571 programmable VCXO is connected via I2C using 0x49 as device address. Additionally, | The Si571 programmable VCXO is connected via I2C using 0x49 as device address. Additionally, | ||
Line 181: | Line 243: | ||
D09JW702+// | D09JW702+// | ||
- | The part properties can be decoded by providing the part number 571AJC000337 on a SiLabs web page [[#15_references|[6]]]: | + | The part properties can be decoded by providing the part number 571AJC000337 on a SiLabs web page [[#10_references|[6]]]: |
Product: Si571\\ | Product: Si571\\ | ||
Line 196: | Line 258: | ||
Operating Temp Range (C): -40 to +85 | Operating Temp Range (C): -40 to +85 | ||
- | A datasheet can be found on the SiLabs website [[#15_references|[7]]]. | + | A datasheet can be found on the SiLabs website [[#10_references|[7]]]. |
- | ==== 3.1.1 Programming the frequency | + | === 4.1.1 Programming the frequency === |
There are three adjustable parameters that define the output frequency: | There are three adjustable parameters that define the output frequency: | ||
- | {{: | + | {{: |
where | where | ||
- | * {{: | + | * {{: |
- | * {{: | + | * {{: |
- | * allowed values for {{: | + | * allowed values for {{: |
- | * allowed values for {{: | + | * allowed values for {{: |
- | The three parameters should be chosen in a way that {{: | + | The three parameters should be chosen in a way that {{: |
For a desired output frequency of 125 MHz the optimum values are: | For a desired output frequency of 125 MHz the optimum values are: | ||
- | * {{: | + | * {{: |
- | * {{: | + | * {{: |
- | * {{: | + | * {{: |
- | Since the uncorrected {{: | + | Since the uncorrected {{: |
- | {{: | + | {{: |
- | in order to get a more accurate result. {{: | + | in order to get a more accurate result. {{: |
The VCXO has a built-in configuration timeout of 10 ms. All I2C write operations from freezing to unfreezing the digitally controlled oscillator have to complete during this period to become active. | The VCXO has a built-in configuration timeout of 10 ms. All I2C write operations from freezing to unfreezing the digitally controlled oscillator have to complete during this period to become active. | ||
- | ==== 3.1.2 Configuration | + | === 4.1.2 Configuration === |
The following registers are read by the gateware for calculating the frequency correction: | The following registers are read by the gateware for calculating the frequency correction: | ||
- | ^ **address**^**description** | + | ^ |
| '' | | '' | ||
| '' | | '' | ||
Line 239: | Line 301: | ||
| '' | | '' | ||
- | **Table | + | |
+ | **Table | ||
The value of register 0x07 is only used to determine if the frequency has been programmed before, e.g. after a reloading of the bitstream of the FPGA without a power cycle of the FMC ADC board. The initial value of HSDIV is 4 and it is programmed to 5. Applying the frequency correction again would lead to a wrong result, since the RFREQ registers do not contain the factory defaults any more. | The value of register 0x07 is only used to determine if the frequency has been programmed before, e.g. after a reloading of the bitstream of the FPGA without a power cycle of the FMC ADC board. The initial value of HSDIV is 4 and it is programmed to 5. Applying the frequency correction again would lead to a wrong result, since the RFREQ registers do not contain the factory defaults any more. | ||
Line 245: | Line 308: | ||
The following registers are programmed by the gateware after the calculation of the frequency correction: | The following registers are programmed by the gateware after the calculation of the frequency correction: | ||
- | ^ **address**^ **value**^**description** ^ | + | ^ |
| '' | | '' | ||
| '' | | '' | ||
Line 256: | Line 319: | ||
| '' | | '' | ||
- | **Table 3.2:** Si571 registers programmed by the gateware | ||
- | ==== 3.1.3 List of devices ==== | + | **Table 4.2:** Si571 registers programmed by the gateware |
- | ^ **FMC version** | + | == Example configuration of some devices == |
- | | | + | |
- | | | + | |
- | | | + | |
- | | | + | |
- | The frequency is measured relatively to the processing clock of the AFC board. The values were measured with the PLL enabled. TODO: measure with disabled PLL. | + | ^ FMC version |
+ | | v2.3 | 155107 | ||
+ | | | ||
+ | | | ||
+ | | | ||
- | ===== 3.2 AD9510 PLL and clock distribution | + | The frequency was measured relatively to the processing clock of the AFC board. |
+ | |||
+ | ==== 4.2 AD9510 PLL and clock distribution ==== | ||
The Analog Devices AD9510 is connected via SPI. Writing to registers must be completed with a write to the register address 0x5A with the LSBit set in the write value (e.g. 0x01) to take effect. Multiple writes can precede the writing of register 0x5A, so that this needs to be done only once at the end of a write sequence. The maximum SPI clock frequency is 25 MHz. | The Analog Devices AD9510 is connected via SPI. Writing to registers must be completed with a write to the register address 0x5A with the LSBit set in the write value (e.g. 0x01) to take effect. Multiple writes can precede the writing of register 0x5A, so that this needs to be done only once at the end of a write sequence. The maximum SPI clock frequency is 25 MHz. | ||
Line 274: | Line 338: | ||
The phase frequency detector of the PLL, which compares the VCXO frequency to the reference frequency, has a maximum input frequency of 100 MHz. Higher frequencies have to be divided by the prescalers R (reference input) and N (VCXO input). A lock signal can be connected to a status pin, that is connected to a FPGA GPIO. | The phase frequency detector of the PLL, which compares the VCXO frequency to the reference frequency, has a maximum input frequency of 100 MHz. Higher frequencies have to be divided by the prescalers R (reference input) and N (VCXO input). A lock signal can be connected to a status pin, that is connected to a FPGA GPIO. | ||
- | A datasheet can be found on the Analog Devices website [[#15_references|[8]]]. | + | A datasheet can be found on the Analog Devices website [[#10_references|[8]]]. |
- | ==== 3.2.1 Configuration | + | === 4.2.1 Configuration === |
The gateware configures the AD9510 device to lock the VCXO frequency to a reference clock coming from the FPGA.\\ | The gateware configures the AD9510 device to lock the VCXO frequency to a reference clock coming from the FPGA.\\ | ||
The following registers are programmed by the gateware: | The following registers are programmed by the gateware: | ||
- | ^ **address**^ **value**^**description** | + | ^ |
| '' | | '' | ||
| '' | | '' | ||
Line 304: | Line 368: | ||
| '' | | '' | ||
- | **Table 3.3:** AD9510 registers programmed by the gateware | ||
- | ===== 3.3 ISLA216P ADC ===== | + | **Table 4.3:** AD9510 registers programmed by the gateware |
+ | |||
+ | ==== 4.3 ISLA216P ADC ==== | ||
The four ISLA216P ADCs are connected via SPI. The communication to each chip is enabled via an individual chip select line. The MOSI, MISO and CLK lines are shared between the four chips. Parallel configuration by driving all chip selects high at the same time works for the writing registers, but not for reading, since there would be multiple drivers on the MISO line. The maximum SPI clock frequency is given by the ADC sampling frequency divided by 16. At a sample frequency of 125 MHz this corresponds to a SPI clock frequency of 7.8125 MHz. | The four ISLA216P ADCs are connected via SPI. The communication to each chip is enabled via an individual chip select line. The MOSI, MISO and CLK lines are shared between the four chips. Parallel configuration by driving all chip selects high at the same time works for the writing registers, but not for reading, since there would be multiple drivers on the MISO line. The maximum SPI clock frequency is given by the ADC sampling frequency divided by 16. At a sample frequency of 125 MHz this corresponds to a SPI clock frequency of 7.8125 MHz. | ||
- | A datasheet can be found on the Renesas website [[#15_references|[9]]]. | + | A datasheet can be found on the Renesas website [[#10_references|[9]]]. |
The ADCs provide a configurable gain correction of +/- 4.2% and a configurable offset correction of +/- 138 LSBs. Since the gain correction and the offset correction are implemented digitally in the gateware, most of the configuration registers can be left at their default values. | The ADCs provide a configurable gain correction of +/- 4.2% and a configurable offset correction of +/- 138 LSBs. Since the gain correction and the offset correction are implemented digitally in the gateware, most of the configuration registers can be left at their default values. | ||
Line 316: | Line 381: | ||
The SPI interface in the gateware is implemented as a four wire interface, whereas the default setting of the ISLA216P SPI interface is a three wire mode. For being able to configure the ISLA216Ps interactively, | The SPI interface in the gateware is implemented as a four wire interface, whereas the default setting of the ISLA216P SPI interface is a three wire mode. For being able to configure the ISLA216Ps interactively, | ||
- | ==== 3.3.1 Configuration | + | === 4.3.1 Configuration === |
The following registers are programmed by the gateware: | The following registers are programmed by the gateware: | ||
- | ^ **address**^ **value**^**description** | + | ^ |
| '' | | '' | ||
- | **Table 3.4:** ISLA216P registers programmed by the gateware | ||
- | ====== | + | **Table |
- | ===== 4.1 Clocking | + | ===== 5 Gateware implementation ===== |
+ | |||
+ | ==== 5.1 Clocking ==== | ||
The gateware uses three primary clocks: | The gateware uses three primary clocks: | ||
Line 335: | Line 401: | ||
* FMC 1 ADC clock, 125 MHz | * FMC 1 ADC clock, 125 MHz | ||
- | ==== 4.1.1 PCIe reference clock ==== | + | === 5.1.1 PCIe reference clock === |
- | The PCIe reference clock comes from an output of an ADN4604 clock switch | + | The PCIe reference clock comes from an output of an ADN4604 clock switch |
// | // | ||
Line 347: | Line 413: | ||
The SDRAM interface IP core contains a MMCM which generates a 100 MHz clock named // | The SDRAM interface IP core contains a MMCM which generates a 100 MHz clock named // | ||
- | ==== 4.1.2 FMC ADC clocks | + | === 5.1.2 FMC ADC clocks === |
- | On each of the two FMC boards there is a Si571 programmable VCXO (see [[#31_si571_programmable_vcxo|chapter 3.1]]) which feeds the four ADCs. The frequency of the VCXO can be coupled to a reference clock coming from the FPGA (see [[#32_ad9510_pll_and_clock_distribution|chapter 3.2]]). The VCXO is programmed to a nominal output frequency of 125 MHz and coupled by the PLL with //clk_125// coming from the FPGA. Bringing the PLL to lock is quite demanding, so that with the current settings a stable lock can not be guaranteed. | + | On each of the two FMC boards there is a Si571 programmable VCXO (see chapter |
From each of the four ADCs of an FMC board an individual clock signal is led to the FPGA which is used for the deserialization in the IDDR primitives. For the further processing, only the clock signal from the first ADC of a FMC board is used since the clock frequencies of the four ADCs are identical. | From each of the four ADCs of an FMC board an individual clock signal is led to the FPGA which is used for the deserialization in the IDDR primitives. For the further processing, only the clock signal from the first ADC of a FMC board is used since the clock frequencies of the four ADCs are identical. | ||
Line 355: | Line 421: | ||
There are two clock domain crossing FIFOs in the gateware to synchronize the data from the ADCs to the main processing clock // | There are two clock domain crossing FIFOs in the gateware to synchronize the data from the ADCs to the main processing clock // | ||
- | ===== 4.2 Resets | + | ==== 5.2 Resets ==== |
- | ==== 4.2.1 PLL not in lock ==== | + | === 5.2.1 PLL not in lock === |
As long as the PLL in the MMCM producing the main processing clock //clk_125// is not yet in lock, the design is held in reset. After this the lock should be stable until the next power cycle. | As long as the PLL in the MMCM producing the main processing clock //clk_125// is not yet in lock, the design is held in reset. After this the lock should be stable until the next power cycle. | ||
- | ==== 4.2.2 Reset button | + | === 5.2.2 PCIe reset === |
+ | |||
+ | The design will be reset whenever the PCIe connection is re-initialized. This happens e.g. when the FEC is rebooted. | ||
+ | |||
+ | === 5.2.3 Reset button === | ||
There is a push button labeled //RST// at the center of the AFC front panel which is connected to the microcontroller for the MMC firmware. The OpenMMC firmware forwards a button press with a duration of at least two seconds to the FPGA pin AG26 as an active low signal to initiate a reset of the gateware. | There is a push button labeled //RST// at the center of the AFC front panel which is connected to the microcontroller for the MMC firmware. The OpenMMC firmware forwards a button press with a duration of at least two seconds to the FPGA pin AG26 as an active low signal to initiate a reset of the gateware. | ||
- | Resetting the FPGA leads to the loss of the PCIe connection. To re-enable the connection, the CPU unit has to be power cycled. | + | Resetting the FPGA leads to the loss of the PCIe connection. To re-enable the connection, the FEC has to be rebooted. |
- | ===== 4.3 Data flow diagram | + | ==== 5.3 Data flow diagram ==== |
- | {{: | + | {{: |
- | **Figure | + | **Figure |
- | Figure | + | Figure |
* processing clocks and clock domain crossings | * processing clocks and clock domain crossings | ||
Line 385: | Line 455: | ||
* data width conversions of AXI and AXI Stream connections | * data width conversions of AXI and AXI Stream connections | ||
- | ===== 4.4 Input delays | + | ==== 5.4 Input delays ==== |
The data inputs from the ADCs require a latency correction to compensate clock and routing delays. This is implemented via individually configurable input delay primitives for both the clock and for the data input pins. By increasing the input delay of either a clock or of the associated data inputs, the alignment can be corrected in both directions. | The data inputs from the ADCs require a latency correction to compensate clock and routing delays. This is implemented via individually configurable input delay primitives for both the clock and for the data input pins. By increasing the input delay of either a clock or of the associated data inputs, the alignment can be corrected in both directions. | ||
- | The input delays provide a 32 tap delay line with a configurable delay between 0 to 31 taps [[#15_references|[10]]]. Each tap corresponds to a delay of: | + | The input delays provide a 32 tap delay line with a configurable delay between 0 to 31 taps [[#10_references|[10]]]. Each tap corresponds to a delay of: |
- | {{: | + | {{: |
- | with {{: | + | with {{: |
- | ==== 4.4.1 Calculation of optimal delay values | + | === 5.4.1 Calculation of optimal delay values === |
The ADCs offers programmable user patterns that can be sent in place of the ADC samples to check the correct timing of the digital interface. For finding the optimum delay values, the following procedure is applied: | The ADCs offers programmable user patterns that can be sent in place of the ADC samples to check the correct timing of the digital interface. For finding the optimum delay values, the following procedure is applied: | ||
Line 403: | Line 473: | ||
* The optimum value is assumed to be the midpoint between these values. | * The optimum value is assumed to be the midpoint between these values. | ||
- | There is a configuration file '' | + | === 5.4.2 Chosen delay values === |
- | + | ||
- | ==== 4.4.2 Chosen delay values | + | |
For an ADC clock frequency of 125 MHz the optimal delay values are: | For an ADC clock frequency of 125 MHz the optimal delay values are: | ||
Line 416: | Line 484: | ||
* ADC data delay value: 0x00 | * ADC data delay value: 0x00 | ||
- | These values are programmed to the IDELAY primitives at start up. They can be changed via individual configuration registers (see [[#711_additional_configuration_registers|chapter | + | These values are programmed to the IDELAY primitives at start up. They can be changed via individual configuration registers (see chapter |
- | ===== 4.5 Clock domain crossings | + | ==== 5.5 Clock domain crossings ==== |
Even though the FMC clocks are coupled to the main processing clock by the FMC’s PLLs, they can jitter against the main processing clock or even run at a slightly different frequency if the PLLs unlock for any reason. | Even though the FMC clocks are coupled to the main processing clock by the FMC’s PLLs, they can jitter against the main processing clock or even run at a slightly different frequency if the PLLs unlock for any reason. | ||
Line 424: | Line 492: | ||
To prevent data corruption, two clock domain crossing FIFOs are used for the incoming ADC data, one for each FMC board. In the case of frequency deviations, there are two cases to differentiate: | To prevent data corruption, two clock domain crossing FIFOs are used for the incoming ADC data, one for each FMC board. In the case of frequency deviations, there are two cases to differentiate: | ||
- | | + | |
* one sample at a time will be discarded | * one sample at a time will be discarded | ||
* this happens synchronous for all four ADCs of a FMC board | * this happens synchronous for all four ADCs of a FMC board | ||
- | | + | |
* one sample at a time will be repeated | * one sample at a time will be repeated | ||
* this happens synchronous for all four ADCs of a FMC board | * this happens synchronous for all four ADCs of a FMC board | ||
Line 433: | Line 501: | ||
Due to the synchronous handling of the four ADCs on a FMC board, the discarding or repetition of samples should not have any measurable effect on the BPM results, since the two inputs to each BPM come from the same FMC board. | Due to the synchronous handling of the four ADCs on a FMC board, the discarding or repetition of samples should not have any measurable effect on the BPM results, since the two inputs to each BPM come from the same FMC board. | ||
- | ===== 4.6 BPM algorithm ===== | + | ==== 5.6 Gate signal |
+ | |||
+ | The gate signal is fed to the FPGA via one of the eight MLVDS lines on the AMC connector. The selection is made via a configuration register (see chapter [[# | ||
+ | |||
+ | ==== 5.7 RF signal | ||
+ | |||
+ | The pulses of the RF signal define the length of the linear regression in the BPM algorithm. The selection is made via a configuration register (see chapter [[# | ||
+ | |||
+ | ==== 5.8 BPM algorithm ==== | ||
+ | |||
+ | The proportionality factor {{: | ||
- | The proportionality factor | + | {{: |
- | {{: | ||
- | **Equation | + | **Equation |
- | The capacitance correction (see [[# | + | The capacitance correction (see chapter |
- | ==== 4.6.1 Pipeline steps performed every clock cycle ==== | + | === 5.8.1 Pipeline steps performed every clock cycle === |
- | The gain correction, the differences and sums of the incoming ADC data pairs {{: | + | The gain correction, the differences and sums of the incoming ADC data pairs {{: |
- | === Step 0: Capacitance correction | + | == Step 0: Capacitance correction == |
* offset and gain corrected ADC 0 data sample is strobed unchanged. Input: 17 bits signed, output 17 bits signed | * offset and gain corrected ADC 0 data sample is strobed unchanged. Input: 17 bits signed, output 17 bits signed | ||
- | * offset and gain corrected ADC 1 data sample is multiplied with a correction factor coming from a configuration register. Input: | + | * offset and gain corrected ADC 1 data sample is multiplied with a correction factor coming from a configuration register. Input: |
- | === Step 1: Calculation of sum and difference signals | + | == Step 1: Calculation of sum and difference signals == |
- | * {{: | + | * {{: |
- | * {{: | + | * {{: |
- | === Step 2: Calculation of products, sign extension, summation | + | == Step 2: Calculation of products, sign extension, summation == |
- | The maximum word length of the adders in the DSP48 blocks of the FPGA is 48 bits. When using these adders, the maximum summation length is limited by the word length of longest term {{: | + | The maximum word length of the adders in the DSP48 blocks of the FPGA is 48 bits. When using these adders, the maximum summation length is limited by the word length of longest term {{: |
- | * {{: | + | * {{: |
- | * {{: | + | * {{: |
- | * {{: | + | * {{: |
- | * {{: | + | * {{: |
- | * {{: | + | * {{: |
- | ==== 4.6.2 Pipeline steps performed with a reduced data rate ==== | + | === 5.8.2 Pipeline steps performed with a reduced data rate === |
- | The following pipeline steps are only performed once for every linear regression period. The length of the linear regression period is defined by the BPM linear regression length register (see [[#621_configuration_registers|chapter | + | The following pipeline steps are only performed once for every linear regression period. The length of the linear regression period is defined by the BPM linear regression length register (see chapter |
If a RF signal is present, the length is additionally controlled by the distances of the pulses of this signal. A new linear regression calculation will be started with every rising edge of the RF signal, while the post processing steps for the previous period will be started. | If a RF signal is present, the length is additionally controlled by the distances of the pulses of this signal. A new linear regression calculation will be started with every rising edge of the RF signal, while the post processing steps for the previous period will be started. | ||
- | === Step 3: Conversion to floating point === | + | == Step 3: Conversion to floating point == |
The DSP48 blocks in the FPGA can only handle multiplications up to 18 bits times 25 bits. For this reason, a conversion to a floating point format is performed. | The DSP48 blocks in the FPGA can only handle multiplications up to 18 bits times 25 bits. For this reason, a conversion to a floating point format is performed. | ||
Line 479: | Line 556: | ||
The floating point format is: | The floating point format is: | ||
- | * {{: | + | * {{: |
- | * {{: | + | * {{: |
- | which decodes to: {{: | + | which decodes to: {{: |
- | The sums {{: | + | The sums {{: |
- | A conversion is not necessary for {{: | + | A conversion is not necessary for {{: |
- | === Step 4: Calculation of the products of sums === | + | == Step 4: Calculation of the products of sums == |
- | The products {{: | + | The products {{: |
- | === Step 5: Shifting to align for subtraction and sign extensions | + | == Step 5: Shifting to align for subtraction and sign extensions == |
In general the results of step 4 will have different exponents, so that the mantissas have to be shifted to a common exponent before a subtraction can take place. | In general the results of step 4 will have different exponents, so that the mantissas have to be shifted to a common exponent before a subtraction can take place. | ||
Line 498: | Line 575: | ||
The mantissa of the float number with the smaller exponent is shifted by the difference of exponents digits to the right and the exponent is set to the larger exponent. Sign extensions by 1 bit take place to prevent over- and underflows by the subtraction. | The mantissa of the float number with the smaller exponent is shifted by the difference of exponents digits to the right and the exponent is set to the larger exponent. Sign extensions by 1 bit take place to prevent over- and underflows by the subtraction. | ||
- | === Step 6: Calculation of the subtractions in the numerator and the denominator | + | == Step 6: Calculation of the subtractions in the numerator and the denominator == |
Now that the operands have the same exponent, the subtractions can take place by subtracting the mantissas. | Now that the operands have the same exponent, the subtractions can take place by subtracting the mantissas. | ||
Line 504: | Line 581: | ||
The exponents of the results stay the same as that of the operands. | The exponents of the results stay the same as that of the operands. | ||
- | The results are: {{: | + | The results are: {{: |
- | === Step 7: Conversion of the mantissas to floating point === | + | == Step 7: Conversion of the mantissas to floating point == |
Due to the multiplication in step 4 and the sign extension in step 5 the mantissas have now a length of 37 bits, which is again too long for the final division. The mantissa is converted to the same floating point format as described in step 4. | Due to the multiplication in step 4 and the sign extension in step 5 the mantissas have now a length of 37 bits, which is again too long for the final division. The mantissa is converted to the same floating point format as described in step 4. | ||
Line 512: | Line 589: | ||
The results respectively have a mantissa of 18 bits and two exponents of 6 bits each which have to be united in the next step. | The results respectively have a mantissa of 18 bits and two exponents of 6 bits each which have to be united in the next step. | ||
- | === Step 8: Unification of exponents and start of division | + | == Step 8: Unification of exponents and start of division == |
Division is a costly operation in FPGAs. In this implementation it is performed by an IP core by Xilinx which is parametrized to 18 bits for both the divisor and the dividend. The result is 33 bits wide, of which 15 bits are fractional. | Division is a costly operation in FPGAs. In this implementation it is performed by an IP core by Xilinx which is parametrized to 18 bits for both the divisor and the dividend. The result is 33 bits wide, of which 15 bits are fractional. | ||
Line 520: | Line 597: | ||
The exponents generated in step 7 are united to the existing ones from step 6 by addition. | The exponents generated in step 7 are united to the existing ones from step 6 by addition. | ||
- | === Step 9 - 32: Waiting for the division to complete | + | == Step 9 - 32: Waiting for the division to complete == |
The results of step 8 are pipelined until the completion of the division. | The results of step 8 are pipelined until the completion of the division. | ||
- | === Step 33: Shifting and slicing the division result | + | == Step 33: Shifting and slicing the division result == |
The division result is shifted to the right by minus the exponent from step 8. After that, the lower 16 bits are sliced to form the result of the linear regression algorithm. | The division result is shifted to the right by minus the exponent from step 8. After that, the lower 16 bits are sliced to form the result of the linear regression algorithm. | ||
- | The result has to be interpreted as a relative position in the range {{: | + | The result has to be interpreted as a relative position in the range {{: |
- | Two signals are created for debugging purposes and are connected to the signal | + | Two signals are created for debugging purposes and are connected to the observer (see https://git.gsi.de/ |
* //result out of range// (1 bit): High if the absolute value of the numerator is greater than that of the denominator. This can happen if the phases of the two input signals are not aligned. In this case the result is set to the maximum or minimum value. | * //result out of range// (1 bit): High if the absolute value of the numerator is greater than that of the denominator. This can happen if the phases of the two input signals are not aligned. In this case the result is set to the maximum or minimum value. | ||
* //division by zero// (1 bit): Comes from the divider IP core and is high if the divisor is zero. This is very unlikely to happen. In this case the result is set to 0. | * //division by zero// (1 bit): Comes from the divider IP core and is high if the divisor is zero. This is very unlikely to happen. In this case the result is set to 0. | ||
- | === Limitations | + | == Limitations == |
Allowed values for the linear regression length are: 3, 4, 5, … , 4096 | Allowed values for the linear regression length are: 3, 4, 5, … , 4096 | ||
Line 543: | Line 620: | ||
The upper limit is caused by the maximum operand length of the adder in the DSP48 primitives in the FPGA. A higher limit would be implementable at the cost of an increased resource usage and two additional clock cycles of processing latency. | The upper limit is caused by the maximum operand length of the adder in the DSP48 primitives in the FPGA. A higher limit would be implementable at the cost of an increased resource usage and two additional clock cycles of processing latency. | ||
- | ===== 4.7 BPM averaging | + | ==== 5.9 BPM averaging ==== |
- | The result from the BPM algorithm is sign extended and added up until the desired number of samples is reached. Only powers of two are allowed for the averaging length. Allowing any desired number would require a general division operation at the end of the averaging process, whereas a division by a power of two can be implemented by a simple shift operation. This is why the configuration register ’log2 of BPM averaging length’ contains the dual logarithm of the averaging length (see [[#621_configuration_registers|chapter | + | The result from the BPM algorithm is sign extended and added up until the desired number of samples is reached. Only powers of two are allowed for the averaging length. Allowing any desired number would require a general division operation at the end of the averaging process, whereas a division by a power of two can be implemented by a simple shift operation. This is why the configuration register ’log2 of BPM averaging length’ contains the dual logarithm of the averaging length (see chapter |
- | The result is sliced to the same number of bits as the result from the BPM algorithm. It also has to be interpreted as a relative position in the range {{: | + | The result is sliced to the same number of bits as the result from the BPM algorithm. It also has to be interpreted as a relative position in the range {{: |
Available values for the averaging length are 1, 2, 4, … , 1,048,576. | Available values for the averaging length are 1, 2, 4, … , 1,048,576. | ||
Line 553: | Line 630: | ||
The upper limit is not caused by any implementation limitation, but was simply chosen because longer averaging lengths were not assumed to be useful. | The upper limit is not caused by any implementation limitation, but was simply chosen because longer averaging lengths were not assumed to be useful. | ||
- | ===== 4.8 AXI infrastructure | + | ==== 5.10 AXI infrastructure ==== |
The memory mapped data transfers inside the FPGA are handled via the AXI protocol using a star topology with a central interconnect. The common data width is 256 bits and the common clock is the main processing clock of 125 MHz. | The memory mapped data transfers inside the FPGA are handled via the AXI protocol using a star topology with a central interconnect. The common data width is 256 bits and the common clock is the main processing clock of 125 MHz. | ||
Line 560: | Line 637: | ||
* PCIe interface | * PCIe interface | ||
- | * scope 0 / observer | + | * scope 0 |
* scope 1 | * scope 1 | ||
* scope 2 | * scope 2 | ||
Line 577: | Line 654: | ||
Even though there is no need for the PCIe interface to write to the SDRAM, this access is enabled because otherwise the PCIe driver will crash in case of an erroneous write access to the SDRAM. | Even though there is no need for the PCIe interface to write to the SDRAM, this access is enabled because otherwise the PCIe driver will crash in case of an erroneous write access to the SDRAM. | ||
- | ===== 4.9 AXI Stream infrastructure | + | ==== 5.11 AXI Stream infrastructure ==== |
- | The scopes | + | The scopes internally use an AXI Stream bus to process the incoming data. The final data stream is converted to the AXI protocol. |
- | ==== 4.9.1 Scopes | + | === 5.11.1 Scopes === |
- | There are three so called scopes for interactively storing calculation results. For the storage format of the scope data see [[# | + | There are three so called scopes for interactively storing calculation results. For the storage format of the scope data see chapter |
- | === Scope 0: corrected ADC data === | + | == Scope 0: corrected ADC data == |
Since the frequency of the incoming ADC data samples is identical to the AXI clock, the data samples are parallized twice to allow flow controlled data processing. This also converts the 128 bits wide ADC data stream (8 * 16 bits) to the common AXI data width of 256 bits. | Since the frequency of the incoming ADC data samples is identical to the AXI clock, the data samples are parallized twice to allow flow controlled data processing. This also converts the 128 bits wide ADC data stream (8 * 16 bits) to the common AXI data width of 256 bits. | ||
Line 591: | Line 668: | ||
An IP core called //AXI data mover// manages the write access to the SDRAM. The block size of the AXI bus accesses is set to 4 MiBytes to allow a low protocol overhead. A block size of 256 bits (width of a single data word) would slow down the transmission in a way that the necessary data rate to store all incoming ADC data samples would not be reached. | An IP core called //AXI data mover// manages the write access to the SDRAM. The block size of the AXI bus accesses is set to 4 MiBytes to allow a low protocol overhead. A block size of 256 bits (width of a single data word) would slow down the transmission in a way that the necessary data rate to store all incoming ADC data samples would not be reached. | ||
- | === Scope 1: BPM results | + | == Scope 1: BPM results == |
The data rate of the BPM results is slow enough so that they do not have to be parallized before the transmission. | The data rate of the BPM results is slow enough so that they do not have to be parallized before the transmission. | ||
Line 597: | Line 674: | ||
For the same reason, the block size of the AXI bus accesses can be set to the width of a single data word which simplifies the transmission handling. | For the same reason, the block size of the AXI bus accesses can be set to the width of a single data word which simplifies the transmission handling. | ||
- | === Scope 1: BPM averaging results | + | == Scope 2: BPM averaging results == |
The data rate of the BPM averaging results is even slower than that of the BPM results, so that the same mechanism can be used. | The data rate of the BPM averaging results is even slower than that of the BPM results, so that the same mechanism can be used. | ||
- | ==== 4.9.2 Observer | + | ==== 5.12 Configuration of peripheral devices |
- | The observer uses the same infrastructure as scope 0, so that they cannot used at the same time. It additionally implements more complex observing features and a configurable two-state trigger. | + | The peripheral devices documented in chapter [[# |
- | The observer allows a visibility equivalent to an ILA (Integrated Logic Analyzer) without the need to connect a JTAG adapter. The input to the observer can be chosen from eight input vectors of 64 bits each. | + | === 5.12.1 SPI Interface === |
- | + | ||
- | The trigger can be set to each bit of each input vector or a combination of those and the two trigger stages can be used to detect signal transitions. | + | |
- | + | ||
- | ===== 4.10 Configuration of peripheral devices ===== | + | |
- | + | ||
- | The peripheral devices documented in [[# | + | |
- | + | ||
- | ==== 4.10.1 SPI Interface | + | |
There is an individual SPI interface for each of the two FMC boards. It is implemented as a four wire interface and connects to the four ADCs and to the PLL on the FMC ADC boards. The choice of the communication partner is implemented via indivial chip select lines. | There is an individual SPI interface for each of the two FMC boards. It is implemented as a four wire interface and connects to the four ADCs and to the PLL on the FMC ADC boards. The choice of the communication partner is implemented via indivial chip select lines. | ||
- | ==== 4.10.2 I2C Interface | + | === 5.12.2 I2C Interface === |
- | There is an individual I2C interface for each of the two FMC boards. It only connects to the VCXO. The VCXO’s transaction timeout has to be kept in mind when programming it interactively (see [[#31_si571_programmable_vcxo|chapter 3.1]]. | + | There is an individual I2C interface for each of the two FMC boards. It only connects to the VCXO. The VCXO’s transaction timeout has to be kept in mind when programming it interactively (see chapter |
- | ===== 4.11 PCIe Interface | + | ==== 5.13 PCIe Interface ==== |
This gateware uses the Xilinx IP core // | This gateware uses the Xilinx IP core // | ||
Line 631: | Line 700: | ||
The PCIe reference clock is routed to the FPGA via the MMC firmware and is configured to be driven by the 100 MHz //FCLKA// clock coming from the AMC connector. | The PCIe reference clock is routed to the FPGA via the MMC firmware and is configured to be driven by the 100 MHz //FCLKA// clock coming from the AMC connector. | ||
- | ===== 4.12 SDRAM interface | + | ==== 5.14 SDRAM interface ==== |
For the communication with the SDRAM, an IP core by Xilinx is used. The clock frequency of the SDRAM interface’s AXI bus is 100 MHz, so that an AXI clock converter is used to connect it to the rest of the AXI infrastructure which is clocked at 125 MHz. | For the communication with the SDRAM, an IP core by Xilinx is used. The clock frequency of the SDRAM interface’s AXI bus is 100 MHz, so that an AXI clock converter is used to connect it to the rest of the AXI infrastructure which is clocked at 125 MHz. | ||
- | ====== 5 Build flow and simulation ====== | + | ==== 5.15 Observer |
- | The build flow is designed and tested to be run on a Linux operation system. The bitstream generation should also work on a Windows installation, | + | This project incorporates the observer interface from the //FPGA_Common// Git submodule (see https://git.gsi.de/ |
- | ===== 5.1 Prerequisites ===== | + | The following signals are connected to the observer inputs: |
- | An installation | + | ^ value^input vector(64 bits) ^valid signal |
+ | | '' | ||
+ | | '' | ||
+ | | '' | ||
+ | | '' | ||
+ | | '' | ||
+ | | '' | ||
+ | | '' | ||
+ | | '' | ||
- | You can set your preferred Vivado version in the script '' | + | The rest of the multiplexer inputs are connected to zero. |
- | ===== 5.2 Build flow ===== | + | ===== 6 Gateware software interface |
- | ==== 5.2.1 Scripted build flow ==== | + | The communication between the gateware and the software takes place via a PCIe driver by Xilinx called XDMA. There is only one PCIe Bar in use in the gateware which maps the memory space to different physical memories on the AMC board. |
- | For a completely automatic script based build flow without using the Vivado GUI proceed as follows: | + | The following mapping is applied: |
- | | + | ^ |
- | | + | | '' |
+ | | '' | ||
+ | | '' | ||
+ | | '' | ||
- | A project will be generated in the folder '' | ||
- | ==== 5.2.2 Vivado GUI based build flow ==== | + | **Table 6.1:** Memory mapping |
- | === Via a Bash script === | + | The // |
+ | https:// | ||
- | * navigate to the root folder of the repository in a terminal | + | ==== 6.1 PCIe Driver ==== |
- | * type '' | + | |
- | This will open the Vivado GUI and set up a project, which can take some minutes. The project will be generated in the folder '' | + | Read and write accesses are mapped to virtual file accesses: |
- | === Via the Vivado GUI === | + | * '' |
+ | * '' | ||
- | If you intend to use the Vivado GUI itself to set up the project proceed as follows: | + | === 6.1.1 Reading from a register === |
- | * open Vivado | + | Example |
- | * use the TCL console | + | |
- | * type '' | + | |
- | ===== 5.3 Simulation ===== | + | < |
+ | uint32_t address | ||
+ | int fd = open("/ | ||
+ | lseek(fd, address, SEEK_SET); | ||
+ | uint64_t value; | ||
+ | read(fd, &value, sizeof(uint64_t)); | ||
+ | </ | ||
+ | === 6.1.2 Writing to a register | ||
- | ==== 5.3.1 Vivado GUI based simulation ==== | + | It is important to write the whole register width of 64 bits. If a register has less than 64 bits, the unused MSBs have to be written to any value. 32 bit write accesses will not have any effect. |
- | Prerequisite: an existing Vivado project see [[# | + | Example in C: |
- | Click on //Run Simulation// in the Vivado GUI. | + | < |
+ | uint64_t value = 42; | ||
+ | uint32_t address = 0x00000400; | ||
+ | int fd = open("/dev/xdma0_h2c_0", | ||
+ | lseek(fd, address, SEEK_SET); | ||
+ | write(fd, &value, sizeof(uint64_t)); | ||
+ | </code> | ||
+ | === 6.1.3 Reading of scope data === | ||
- | ==== 5.3.2 Scripted simulation ==== | + | Example in C: |
- | The scripted simulation checks that the simulation results match a predefined reference pattern. | + | < |
+ | uint32_t address = 0x80000000; | ||
+ | int fd = open("/ | ||
+ | lseek(fd, address, SEEK_SET); | ||
+ | char data[1024]; | ||
+ | read(fd, data, 1024); | ||
+ | </ | ||
+ | This example reads 1024 bytes of data from scope 0 to an array. For bigger data blocks, instead of using an array, you will probably prefer | ||
- | * navigate to the root folder of the repository in a terminal | + | ==== 6.2 Scope memory ==== |
- | * type '' | + | |
- | * you will find the output files of the simulation in the folder '' | + | |
- | ==== 5.3.3 Peripherals simulation models ==== | + | There are three scope memory regions of which the one for the corrected ADC data is the largest since it has the highest data rate. |
- | The toplevel simulation includes a Verilog simulation model from Micron, the manufacturer of the AFC’s SDRAM, which allows the simulation of the behavior of the external SDRAM. | + | ^ start address^ |
- | + | | '' | |
- | The SDRAM interface IP needs an initial calibration process which finishes after about 120 us. If the communication to the SDRAM is of interest the simulation time should be chosen to be longer than that. | + | | '' |
- | + | | '' | |
- | ====== 6 Gateware software interface ====== | + | |
- | + | ||
- | The communication between the gateware inside the FPGA and the software running on the CPU unit takes place via a PCIe driver by Xilinx called XDMA. There is only one PCIe Bar in use in the gateware which maps a coherent memory space of 0x80010000 bytes (= 2, | + | |
- | + | ||
- | The following mapping is applied: | + | |
- | + | ||
- | ^ **address**^ **size**^**memory type** | + | |
- | | 0x00000000| | + | |
- | | | + | |
- | | 0x80008000| '' | + | |
- | + | ||
- | **Table 6.1:** Memory mapping | + | |
- | + | ||
- | ===== 6.1 Scope memory ===== | + | |
- | + | ||
- | There are three scope memory regions of which the one for the corrected ADC data is the largest since it has the highest data rate. | + | |
- | ^ | ||
- | | '' | ||
- | | '' | ||
- | | '' | ||
**Table 6.2:** Scopes memory map | **Table 6.2:** Scopes memory map | ||
- | ==== 6.1.1 Scope 0: corrected ADC data ==== | + | === 6.2.1 Scope 0: corrected ADC data === |
The corrected ADC data is stored in the following format: | The corrected ADC data is stored in the following format: | ||
- | ^ **address**^ **bits**^**radix** | + | ^ |
- | | '' | + | | '' |
- | | '' | + | | '' |
- | | '' | + | | '' |
- | | '' | + | | '' |
- | | '' | + | | '' |
- | | '' | + | | '' |
- | | '' | + | | '' |
- | | '' | + | | '' |
- | | '' | + | | '' |
- | | | + | | |
**Table 6.3:** Corrected ADC data storage format | **Table 6.3:** Corrected ADC data storage format | ||
- | The corrected data is the result of two sequential operations on the raw ADC data: | + | The corrected data is the result of four sequential operations on the raw ADC data: |
- | | + | |
- | | + | |
+ | * configurable moving average filtering | ||
+ | * optional high pass filtering | ||
- | The correction summand | + | The correction summand, the correction factor |
- | The corrected ADC data scope memory can hold up to {{: | + | The corrected ADC data scope memory can hold up to {{: |
- | ==== 6.1.2 Scope 1: BPM result | + | === 6.2.2 Scope 1: BPM result === |
The BPM result is stored in the following format: | The BPM result is stored in the following format: | ||
- | ^ **address**^ **bits**^**radix** | + | ^ |
- | | '' | + | | '' |
- | | '' | + | | '' |
- | | '' | + | | '' |
- | | '' | + | | '' |
- | | '' | + | | '' |
- | | '' | + | | '' |
- | | '' | + | | '' |
- | | '' | + | | '' |
- | | '' | + | | '' |
- | | '' | + | | '' |
- | | '' | + | | '' |
- | | '' | + | | '' |
- | | '' | + | | '' |
- | | '' | + | | '' |
- | | '' | + | | '' |
- | | | + | | |
**Table 6.4:** BPM result storage format | **Table 6.4:** BPM result storage format | ||
- | The BPM result scope memory can hold up to {{: | + | The BPM result scope memory can hold up to {{: |
- | === BPM {0 - 3} result | + | == BPM {0 - 3} result == |
- | This value divided by {{: | + | This value divided by {{: |
- | === BPM {0 - 3} variance * N === | + | == BPM {0 - 3} variance * N == |
- | This value divided by {{: | + | This value divided by {{: |
The scaling with the linear regression length guarantees enough LSBs to evaluate. The variance itself quantized with 16 bits would otherwise often result in zero. | The scaling with the linear regression length guarantees enough LSBs to evaluate. The variance itself quantized with 16 bits would otherwise often result in zero. | ||
- | === BPM {0 - 3} intensity | + | == BPM {0 - 3} intensity == |
- | The intensity of the beam. A value of {{: | + | The intensity of the beam. A value of {{: |
- | ==== 6.1.3 Scope 2: BPM averaging result | + | === 6.2.3 Scope 2: BPM averaging result === |
The BPM averaging result is stored in the following format: | The BPM averaging result is stored in the following format: | ||
- | ^ **address**^ **bits**^**radix** | + | ^ |
- | | '' | + | | '' |
- | | '' | + | | '' |
- | | '' | + | | '' |
- | | '' | + | | '' |
- | | '' | + | | '' |
- | | '' | + | | '' |
- | | '' | + | | '' |
- | | '' | + | | '' |
- | | '' | + | | '' |
- | | '' | + | | '' |
- | | '' | + | | '' |
- | | '' | + | | '' |
- | | '' | + | | '' |
- | | '' | + | | '' |
- | | '' | + | | '' |
- | | | + | | |
**Table 6.5:** BPM averaging result storage format | **Table 6.5:** BPM averaging result storage format | ||
- | The BPM averaging result scope memory can hold up to {{: | + | The BPM averaging result scope memory can hold up to {{: |
- | === BPM {0 - 3} averaging result | + | == BPM {0 - 3} averaging result == |
- | This value divided by {{: | + | This value divided by {{: |
- | === BPM {0 - 3} averaging variance * N * N_avg === | + | == BPM {0 - 3} averaging variance * N * N_avg == |
- | This value divided by {{: | + | This value divided by {{: |
The scaling with the average linear regression length times the averaging length guarantees enough LSBs to evaluate. The variance itself quantized with 16 bits would otherwise nearly always result in zero. | The scaling with the average linear regression length times the averaging length guarantees enough LSBs to evaluate. The variance itself quantized with 16 bits would otherwise nearly always result in zero. | ||
- | === BPM {0 - 3} averaging intensity | + | == BPM {0 - 3} averaging intensity == |
- | The average intensity of the beam. A value of {{: | + | The average intensity of the beam. A value of {{: |
- | ===== 6.2 Register map ===== | + | ==== 6.3 Register map ==== |
- | ==== 6.2.1 Configuration | + | === 6.3.1 Status |
- | The following registers can be written | + | The following |
- | ^ **index**^ **address**^ **bits**^**radix** | + | ^ index^ |
- | | | + | | '' |
- | | | + | | '' |
- | | | + | | '' |
- | | | + | | '' |
- | | | + | | '' |
- | | | + | | '' |
- | | | + | | '' |
- | | | + | | '' |
- | | | + | | '' |
- | | | + | | '' |
- | | '' | + | | |
- | | '' | + | | |
- | | '' | + | | |
- | | '' | + | | |
- | | '' | + | | |
- | | '' | + | | |
- | | '' | + | | |
- | | '' | + | | |
- | | '' | + | | |
- | | '' | + | | |
- | | '' | + | | |
- | | '' | + | | |
- | | '' | + | | |
- | | '' | + | | |
- | | '' | + | | '' |
- | | '' | + | | '' |
- | | '' | + | | '' |
- | | '' | + | | |
- | | '' | + | | |
- | | '' | + | | |
- | | '' | + | | |
- | | '' | + | | |
- | | '' | + | | |
- | | '' | + | | '' |
- | | '' | + | | '' |
- | | '' | + | | '' |
- | + | | '' | |
- | **Table 6.6:** List of configuration registers | + | |
- | + | ||
- | === 0 - 7: ADC {0 - 7} offset correction summand === | + | |
- | + | ||
- | Correction summand for a possible offset deviation of the ADC. The offset correction precedes the gain correction. | + | |
- | + | ||
- | === 8 - 15: ADC {0 - 7} gain correction factor === | + | |
- | + | ||
- | Correction factor for a possible gain deviation of the ADC. The default value 0x8000 corresponds to a multiplication by 1. The possible correction range is {{: | + | |
- | + | ||
- | === 16 - 19: BPM {0 - 3} capacitance correction factor === | + | |
- | + | ||
- | The capacitances of the two corresponding capacitor plates of a single BPM can differ. Data 0 is fed unchanged into the BPM algorithm, while data 1 is multiplied by a correction factor. The default value 0x8000 corresponds to a multiplication by 1. The possible correction range is {{: | + | |
- | + | ||
- | === 20: BPM linear regression length - 1 === | + | |
- | + | ||
- | Number of samples over which the linear regression is calculated if no external RF pulse signal is present. This value is valid for all four BPMs. If an external RF pulse signal is present, the result of the linear regression will be output and a new calculation will be started on every rising edge of the RF pulse signal. For this to work, this register has to be set to a value that is longer than the interval between the RF pulses. | + | |
- | + | ||
- | Allowed values: 0x002 - 0xFFF | + | |
- | + | ||
- | The lower limit is determined by the throughput of the divider IP core of 1 in 3 clock cycles that is used for the final division of the BPM algorithm. | + | |
- | + | ||
- | === 21: Log2 of BPM averaging length === | + | |
- | + | ||
- | Dual logarithm of the number of linear regression results over which the averaging is calculated. This value is valid for all four BPMs. | + | |
- | + | ||
- | Allowed range: 0 .. 20. Higher values will be set to the maximum allowed value. This corresponds to an averaging length of 1, 2, 4, … , 1, | + | |
- | + | ||
- | === 22: Gate signal input select === | + | |
- | + | ||
- | ^ value^input | + | |
- | | 0 - 7|MLVDS line 0 - 7 on the backplane | + | |
- | | 8|FMC 0 //TRIG// input | | + | |
- | | 9|FMC 1 //TRIG// input | | + | |
- | + | ||
- | The gate signal input can be switched between one of the eight MLVDS lines on the backplane and the two MMCX connectors labeled //TRIG// on the FMC front panels. | + | |
- | + | ||
- | === 23: RF signal input select === | + | |
- | + | ||
- | ^ value^input | + | |
- | | 0 - 7|MLVDS line 0 - 7 on the backplane | + | |
- | | 8|FMC 0 //TRIG// input | | + | |
- | | 9|FMC 1 //TRIG// input | | + | |
- | + | ||
- | The RF signal input can be switched between one of the eight MLVDS lines on the backplane and the two MMCX connectors labeled //TRIG// on the FMC front panels. | + | |
- | + | ||
- | === 31: Partial reset === | + | |
- | + | ||
- | Writing a 1 to this register triggers a partial reset on the gateware, which affects the scopes and the BPM calculation modules. The PCIe interface, the SDRAM interface, the AXI infrastructure and the register bank will not be resetted. | + | |
- | + | ||
- | The reset will be automatically lifted so that the register does not have to be written to 0 after initiating a reset. | + | |
- | + | ||
- | === 32, 40, 48: Scope {0, 1, 2} capture length - 1 === | + | |
- | + | ||
- | The number | + | |
- | + | ||
- | Scope 0 can only handle even numbers of samples. Uneven numbers will be automatically handled as the next higher even number. For scopes 1 and 2, also uneven numbers are allowed. | + | |
- | + | ||
- | === 33, 41, 49: Scope {0, 1, 2} trigger mode === | + | |
- | + | ||
- | ^ value^trigger mode ^ | + | |
- | | | + | |
- | | 1|trigger on high state of gate signal | + | |
- | | 2, 3|trigger instantly after the trigger is armed, independent of the state of the gate signal | + | |
- | + | ||
- | === 34, 42, 50: Scope {0, 1, 2} arm trigger === | + | |
- | + | ||
- | Writing a 1 to this register will arm the trigger once. The register does not have to be reset to 0 before the next arm trigger, just write another 1 to it. If the corresponding register ’continuous trigger’ is set to 1, writing to this register does not have any effect. | + | |
- | + | ||
- | When in ’waiting for trigger’ state (see status register ’capture status’ in [[# | + | |
- | + | ||
- | === 43, 51: Scope {1, 2} capture mode === | + | |
- | + | ||
- | ^ value^capture mode ^ | + | |
- | | 0|capture until the number of samples defined by register {40, 48} are stored | + | |
- | | 1|the same, but cancel capturing when the gate signal goes low | | + | |
- | + | ||
- | A capture mode register is only available for scopes 1 and 2. Scope 0 (for corrected ADC data) always operates in capture mode 0. | + | |
- | + | ||
- | ==== 6.2.2 Status registers ==== | + | |
- | + | ||
- | The following status registers can be read by software: | + | |
- | ^ **index**^ | ||
- | | '' | ||
- | | '' | ||
- | | '' | ||
- | | '' | ||
- | | '' | ||
- | | '' | ||
- | | '' | ||
- | | '' | ||
- | | '' | ||
- | | '' | ||
- | | '' | ||
- | | '' | ||
- | | '' | ||
- | | '' | ||
- | | '' | ||
- | | '' | ||
- | | '' | ||
- | | '' | ||
- | | '' | ||
- | | '' | ||
- | | '' | ||
- | | '' | ||
- | | '' | ||
- | | '' | ||
- | | '' | ||
- | | '' | ||
- | | '' | ||
- | | '' | ||
- | | '' | ||
- | | '' | ||
- | | '' | ||
- | | '' | ||
- | | '' | ||
- | | '' | ||
- | **Table 6.7:** List of status registers | + | **Table 6.6:** List of status registers |
- | === 0 - 3: latest BPM {0 - 3} result | + | == 0 - 3: latest BPM {0 - 3} result == |
- | This value divided by {{: | + | This value divided by {{: |
- | === 4 - 7: latest BPM {0 - 3} variance * N === | + | == 4 - 7: latest BPM {0 - 3} variance * N == |
- | This value divided by {{: | + | This value divided by {{: |
The scaling with the linear regression length guarantees enough LSBs to evaluate. The variance itself quantized with 16 bits would otherwise often result in zero. | The scaling with the linear regression length guarantees enough LSBs to evaluate. The variance itself quantized with 16 bits would otherwise often result in zero. | ||
- | === 8 - 11: latest BPM {0 - 3} intensity | + | == 8 - 11: latest BPM {0 - 3} intensity == |
- | The intensity of the beam. A value of {{: | + | The intensity of the beam. A value of {{: |
- | === 12: effective linear regression length | + | == 12: effective linear regression length == |
The effective linear regression length is determined by the nominal value of the linear regression length configuration register and the frequency of the RF pulses. If no RF pulses are present, this register should hold the value of the linear regression length configuration register. | The effective linear regression length is determined by the nominal value of the linear regression length configuration register and the frequency of the RF pulses. If no RF pulses are present, this register should hold the value of the linear regression length configuration register. | ||
- | === 13: time from gate high transition | + | == 13: time from gate high transition == |
Counter value, driven by the 125 MHz main processing clock, starting from the high transition of the gate input signal. | Counter value, driven by the 125 MHz main processing clock, starting from the high transition of the gate input signal. | ||
- | === 16 - 19: latest BPM {0 - 3} averaging result | + | == 16 - 19: latest BPM {0 - 3} averaging result == |
- | This value divided by {{: | + | This value divided by {{: |
- | === 20 - 23: latest BPM {0 - 3} averaging variance * N * N_avg === | + | == 20 - 23: latest BPM {0 - 3} averaging variance * N * N_avg == |
- | This value divided by {{: | + | This value divided by {{: |
The scaling with the average linear regression length times the averaging length guarantees enough LSBs to evaluate. The variance itself quantized with 16 bits would otherwise nearly always result in zero. | The scaling with the average linear regression length times the averaging length guarantees enough LSBs to evaluate. The variance itself quantized with 16 bits would otherwise nearly always result in zero. | ||
- | === 24 - 27: latest BPM {0 - 3} averaging intensity | + | == 24 - 27: latest BPM {0 - 3} averaging intensity == |
- | The intensity of the beam. A value of {{: | + | The intensity of the beam. A value of {{: |
- | === 28: average effective linear regression length | + | == 28: average effective linear regression length == |
The average of the effective linear regression lengths of the BPM. | The average of the effective linear regression lengths of the BPM. | ||
- | === 32, 40, 48: Scope {0, 1, 2} capture status | + | == 32, 40, 48: Scope {0, 1, 2} capture status == |
- | ^ value^capture status | + | ^ value^capture status |
- | | 0|idle | + | | '' |
- | | 1|waiting for trigger | + | | '' |
- | | 2|capturing | + | | '' |
- | | 3|done | + | | '' |
The value 0 is only present before starting the trigger for the first time. After that, the effective idle state is 3. | The value 0 is only present before starting the trigger for the first time. After that, the effective idle state is 3. | ||
- | === 33, 41, 49: Scope {0, 1, 2} next write address | + | == 33, 41, 49: Scope {0, 1, 2} next write address == |
Address where the next data sample will be stored during the scope’s capturing process. | Address where the next data sample will be stored during the scope’s capturing process. | ||
- | === 127: FPGA serial number === | + | == 124: build timestamp |
- | The XDMA PCIe driver by Xilinx numbers | + | Time when the bitstream was created. This information |
- | ===== 6.3 Capturing procedure ===== | + | Format: |
- | ==== 6.3.1 Known number of samples ==== | + | ^ bits 31 - 27 ^ bits 26 - 23 ^ bits 22 - 17 |
+ | | day | ||
- | A typical procedure for capturing a predefinable number of samples starting from the rising edge of the gate signal is the following: | + | == 125: FPGA serial number == |
- | * write the number of samples minus 1 to the configuration | + | The XDMA PCIe driver by Xilinx numbers |
- | * write a 0 to the configuration register named ’trigger mode’ | + | |
- | * write a 0 (= default) | + | |
- | * write a 1 to the configuration register named ’arm trigger’ | + | |
- | * you can check the status register named ’capture status’ for the progress: 1: rising edge of gate signal not yet detected, 2: capturing is ongoing, 3: capturing completed | + | |
- | * you can check the current write address by polling the status register named ’next write address’ | + | |
- | ==== 6.3.2 Unknown number of samples ==== | + | == 126: module ID == |
- | BPM results are only calculated while the gate signal is high. If you want to capture a complete high period | + | The module ID can be used to identify |
+ | The module ID of the Cryring | ||
- | * write the maximum value 0x1FFFFFF to the configuration register named ’capture length - 1’ | + | The fields are defined |
- | * write a 0 to the configuration register named ’trigger mode’ | + | |
- | * write a 1 to the configuration register named ’capture mode’ | + | |
- | * write a 1 to the configuration register named ’arm trigger’ | + | |
- | * you can check the status register ’capture status’ | + | |
- | * the value of the status register named ’next write address’ will be static after completion and indicates how many samples have been captured | + | |
- | ====== 7 Extended | + | ^ bits 63 - 56 |
+ | | minor gateware | ||
- | Besides the interface documented in [[# | + | Here is an incomplete list of project IDs: |
- | While the productive interface is intended to be kept as downward compatible as possible, the extended interface may be subject to major changes during the development process. | + | ^ project ID^project name ^ |
+ | | '' | ||
+ | | '' | ||
+ | | '' | ||
+ | | '' | ||
+ | | '' | ||
+ | | '' | ||
- | ===== 7.1 Extended register map ===== | + | == 127: magic number |
- | ==== 7.1.1 Additional configuration registers ==== | + | The magic number can be used to determine if the gateware uses the expected register format.\\ |
+ | The value of this register is the same for all module IDs: '' | ||
- | The following additional | + | === 6.3.2 Configuration |
- | ^ | + | The following registers can be written by software: |
- | | '' | + | |
- | | '' | + | |
- | | '' | + | |
- | | '' | + | |
- | | '' | + | |
- | | '' | + | |
- | | '' | + | |
- | | '' | + | |
- | | '' | + | |
- | | '' | + | |
- | | '' | + | |
- | | '' | + | |
- | | '' | + | |
- | | '' | + | |
- | | '' | + | |
- | | '' | + | |
- | | '' | + | |
- | | '' | + | |
- | | '' | + | |
- | | '' | + | |
- | | '' | + | |
- | | '' | + | |
- | | '' | + | |
- | | '' | + | |
- | | '' | + | |
- | | '' | + | |
- | | '' | + | |
- | | '' | + | |
- | | '' | + | |
- | | '' | + | |
- | | '' | + | |
- | | '' | + | |
- | | '' | + | |
- | | '' | + | |
- | | '' | + | |
- | **Table | + | ^ index^ |
+ | | '' | ||
+ | | '' | ||
+ | | '' | ||
+ | | '' | ||
+ | | '' | ||
+ | | '' | ||
+ | | '' | ||
+ | | '' | ||
+ | | '' | ||
+ | | '' | ||
+ | | '' | ||
+ | | '' | ||
+ | | '' | ||
+ | | '' | ||
+ | | '' | ||
+ | | '' | ||
+ | | '' | ||
+ | | '' | ||
+ | | '' | ||
+ | | '' | ||
+ | | '' | ||
+ | | '' | ||
+ | | '' | ||
+ | | '' | ||
+ | | '' | ||
+ | | '' | ||
+ | | '' | ||
+ | | '' | ||
+ | | '' | ||
+ | | '' | ||
+ | | '' | ||
+ | | '' | ||
+ | | '' | ||
+ | | '' | ||
+ | | '' | ||
+ | | '' | ||
+ | | '' | ||
+ | | '' | ||
+ | | '' | ||
- | ^ **index**^ | ||
- | | '' | ||
- | | '' | ||
- | | '' | ||
- | | '' | ||
- | | '' | ||
- | | '' | ||
- | | '' | ||
- | | '' | ||
- | | '' | ||
- | | '' | ||
- | | '' | ||
- | | '' | ||
- | | '' | ||
- | | '' | ||
- | | '' | ||
- | | '' | ||
- | | '' | ||
- | | '' | ||
- | | '' | ||
- | | '' | ||
- | | '' | ||
- | | '' | ||
- | | '' | ||
- | | '' | ||
- | | '' | ||
- | | '' | ||
- | | '' | ||
- | | '' | ||
- | | '' | ||
- | | '' | ||
- | | '' | ||
- | | '' | ||
- | **Table | + | **Table |
- | === 39, 47, 55: Scope {0, 1, 2} continuous trigger === | + | == 0 - 7: ADC {0 - 7} offset correction summand |
- | If set to 1, the trigger is armed and will be rearmed automatically after every capture completion. | + | Correction summand for a possible offset deviation of the ADC. The offset correction precedes the gain correction. |
- | === 64, 80: FMC {0, 1} status LED select === | + | == 8 - 15: ADC {0 - 7} gain correction factor |
- | There is one tricolor LED on the FMC front panel labeled //status// that can be controlled | + | Correction factor for a possible gain deviation of the ADC. The default value 0x8000 corresponds to a multiplication |
- | ^ value^input | + | == 16 - 19: BPM {0 - 3} capacitance correction factor == |
- | | 0|ADC clock, blink frequency divided by {{:ds: | + | |
- | | 1|AD9510 monitoring clock, blink frequency divided by {{: | + | |
- | | 2, 3|static value from register ’status LED value’ | + | |
- | === 65, 81: FMC {0, 1} status LED value === | + | The capacitances of the two corresponding capacitor plates of a single BPM can differ. Data 0 is fed unchanged into the BPM algorithm, while data 1 is multiplied by a correction factor. The default |
- | The static lighting pattern defined by this register becomes active if the corresponding register ’status LED select’ is set to 2 or 3. | + | == 20: BPM linear regression length - 1 == |
- | ^ bit^color | + | Number of samples over which the linear regression is calculated if no external RF pulse signal is present. This value is valid for all four BPMs. If an external RF pulse signal is present, the result of the linear regression will be output and a new calculation will be started on every rising edge of the RF pulse signal. For this to work, this register has to be set to a value that is longer than the interval between the RF pulses. |
- | | 0|red | | + | |
- | | 1|green | + | |
- | | 2|blue | + | |
- | === 66, 82: FMC {0, 1} SPI cs === | + | Allowed values: 0x002 - 0xFFF |
- | Chip select signals (active high) of the SPI bus to the four ADCs and to the AD9510 PLL and clock distribution. | + | The lower limit is determined by the throughput |
- | ^ bit^device | + | == 21: Log2 of BPM averaging length == |
- | | 0|ADC 0 | | + | |
- | | 1|ADC 1 | | + | |
- | | 2|ADC 2 | | + | |
- | | 3|ADC 3 | | + | |
- | | 4|PLL and clock distribution | + | |
- | === 67, 83: FMC {0, 1} SPI read/write === | + | Dual logarithm of the number of linear regression results over which the averaging is calculated. This value is valid for all four BPMs. |
- | 0: write mode, 1: read mode | + | Allowed range: 0 .. 20. Higher values will be set to the maximum allowed value. This corresponds to an averaging length of 1, 2, 4, … , 1,048,576. |
- | === 68, 84: FMC {0, 1} SPI address === | + | == 22: Gate signal input select |
- | The address of the register that shall be accessed. | + | ^ value^input |
+ | | '' | ||
+ | | '' | ||
+ | | '' | ||
- | === 69, 85: FMC {0, 1} SPI write data === | + | The gate signal input can be switched between one of the eight MLVDS lines on the backplane and the two MMCX connectors labeled //TRIG// on the FMC front panels. |
- | The data that shall be written to a register. | + | == 23: RF signal input select == |
- | === 70, 86: FMC {0, 1} SPI trigger === | + | ^ value^input |
+ | | '' | ||
+ | | '' | ||
+ | | '' | ||
- | Write a 1 to this register to start a read or write access on the SPI bus. The register does not have to be reset to 0 before | + | The RF signal input can be switched between one of the eight MLVDS lines on the backplane and the two MMCX connectors labeled //TRIG// on the FMC front panels. |
- | === 71, 87: FMC {0, 1} ADC resetn === | + | == 24: Intensity normalization exponent |
- | Low active reset signal to the four ADCs in parallel. Tie to 0 and back to 1 to initiate a reset. | + | The intensity calculation |
- | === 72, 88: FMC {0, 1} I2c read/write === | + | With each increment of this exponent by one, the result will double. Keep in mind that the result can saturate when setting this value to larger than zero. |
- | 0: write mode, 1: read mode | + | == 25: Moving average filter length - 1 == |
- | === 73, 89: FMC {0, 1} I2C device address === | + | The averaging length minus 1 of the moving average filter on the ADC data. All possible values from 0 to 1023 are allowed, resulting in an averaging length between |
- | The address of the connected VCXO is 0x49. | + | == 25: IIR filter enable == |
- | === 74, 90: FMC {0, 1} I2C register address === | + | Bitmask which enables the IIR filter per BPM. The filter is intended to suppress a 70 kHz interference on certain BPMs. Bit 0 enables the filter on the data of ADC 0 and 1, bit 1 on the data of ADC 2 and 3 and so on. |
- | The address of the register that shall be accessed. | + | == 32, 40, 48: Scope {0, 1, 2} capture length - 1 == |
- | === 75, 91: FMC {0, 1} I2C write data === | + | The number of samples minus one that are stored after a scope has been triggered. Each sample consists of 16 bytes. |
- | The data that shall be written to a register. | + | Scope 0 can only handle even numbers of samples. Uneven numbers will be automatically handled as the next higher even number. For scopes 1 and 2, also uneven numbers are allowed. |
- | === 76, 92: FMC {0, 1} I2C trigger | + | == 33, 41, 49: Scope {0, 1, 2} trigger |
- | Write a 1 to this register to start a read or write access | + | ^ |
+ | | '' | ||
+ | | '' | ||
+ | | '' | ||
- | === 77, 93: FMC {0, 1} PLL resetn === | + | == 34, 42, 50: Scope {0, 1, 2} arm trigger |
- | Low active reset signal | + | Writing a 1 to this register will arm the trigger once. The register does not have to be reset to 0 before the next arm trigger, just write another 1 to it. If the corresponding register // |
- | === 78, 94: FMC {0, 1} Clock switch select === | + | When in ’waiting for trigger’ state (see status register ’capture status’ in chapter [[# |
- | There is a separate clock switch in front of the AD9510 PLL reference clock input. | + | == 43, 51: Scope {1, 2} capture mode == |
- | ^ value^connect to | + | ^ value^capture mode ^ |
- | | 0|MMCX connector labeled //REF// on the front panel of the FMC board | | + | | '' |
- | | 1|clock output from the FPGA via the FMC connector | + | | '' |
- | === 79, 95: FMC {0, 1} VCXO output enable === | + | A capture mode register is only available for scopes |
- | Enables the frequency output of the VCXO. | + | == 127: Reset == |
- | === 96 - 99 and 104 - 107: FMC {0, 1} ADC {0 - 3} clock delay === | + | Writing a 1 to this register triggers a reset on the gateware, which also resets all configuration registers to their default values.\\ |
+ | The reset will be automatically lifted so that the register does not have to be written to 0 after initiating a reset. | ||
- | There is a configurable input delay for setting the correct digital interface timing for both the clock and the data signals. Increasing this value increases the delay of the clock, so that the data is sampled later. | + | ==== 6.4 Capturing procedure ==== |
- | === 100 - 103 and 108 - 111: FMC {0, 1} ADC {0 - 3} data delay === | + | === 6.4.1 Known number of samples |
- | See above. Increasing this value increases | + | A typical procedure for capturing a predefinable number of samples starting from the rising edge of the gate signal |
- | === 112: AFC LED select === | + | * write the number of samples minus 1 to the configuration register //capture length - 1// |
+ | * write a 0 to the configuration register //trigger mode// | ||
+ | * write a 0 (= default) to the configuration register //capture mode// | ||
+ | * write a 1 to the configuration register //arm trigger// | ||
+ | * you can check the status register ’capture status’ for the progress: '' | ||
+ | * you can check the current write address by polling the status register //next write address// | ||
- | There is one tricolor LED at the center of the AFC front panel labeled //L3// that can be controlled by the gateware. | + | === 6.4.2 Unknown number of samples === |
- | ^ value^input | + | BPM results are only calculated while the gate signal is high. If you want to capture a complete high period of e.g. BPM average samples, the total number of samples is unknown. Proceed as follows: |
- | | 0|PCIe reference clock, blink frequency divided by {{:ds: | + | |
- | | 1|static value from register 113 ’AFC LED value’ | + | |
- | === 113: AFC LED value === | + | * write the maximum |
+ | * write a 0 to the configuration register //trigger mode// | ||
+ | * write a 1 to the configuration register //capture mode// | ||
+ | * write a 1 to the configuration register //arm trigger// | ||
+ | * you can check the status register //capture status// as above | ||
+ | * the value of the status register //next write address// will be static after completion and indicates how many samples have been captured | ||
- | Static lighting pattern if register 112 ’AFC LED select’ | + | ===== 7 Extended gateware software interface ===== |
- | ^ bit^color | + | Besides the interface documented in chapter [[# |
- | | 0|red | | + | |
- | | 1|green | + | |
- | | 2|blue | + | |
- | === 114: Gate override === | + | While the productive interface is intended to be kept as downward compatible as possible, the extended interface may be subject to major changes during the development process. |
- | For testing purposes without an external gate signal this register can be set to 1 to simulate a gate signal via the register | + | ==== 7.1 Extended |
- | === 115: Gate override value === | + | === 7.1.1 Additional status registers |
- | Can be used to simulate a gate signal when register 114 ’gate override’ is 1. | + | The following additional status registers can be read by software: |
- | === 116: MLVDS direction === | + | ^ index^ |
- | + | | '' | |
- | Determines the direction of the eight MLVDS lines on the AMC connector. A ’0’ corresponds to an input to the FPGA and a ’1’ to an output from the FPGA. | + | | '' |
- | + | | '' | |
- | === 117: MLVDS output value === | + | | '' |
- | + | | '' | |
- | Determines the logic levels of the eight MLVDS lines if they are configured as outputs (see previous register). | + | | '' |
- | + | | '' | |
- | === 118: Observer scope mode === | + | | '' |
- | + | | '' | |
- | When observer scope mode is 1, scope 0 operates in an easy to use mode for storing ADC data. | + | | '' |
- | + | | '' | |
- | Setting this register to 0 enables additional functionality like combining and choosing different signals to store and a more powerful and flexible two-stage trigger. | + | | '' |
- | + | | '' | |
- | Registers 119 to 127 are only relevant in non productive mode. | + | | '' |
- | + | | '' | |
- | === 119: Observer valid signal select === | + | | '' |
- | + | | '' | |
- | Determines the data valid input to the observer. Samples are only stored when the valid signal is high. | + | | '' |
- | + | | '' | |
- | ^ value^input | + | | '' |
- | | 0, 3|constant | + | | '' |
- | | 1|BPM result valid | | + | | '' |
- | | 2|BPM averaging result valid | | + | | '' |
- | + | | '' | |
- | === 120, 121: Observer multiplexer {0, 1} select === | + | | '' |
- | + | | '' | |
- | The observer stores samples that are 128 bits wide, which consist of two concatenated 64 bits wide multiplexer outputs. Each multiplexer can choose between eight different input vectors. Like this, each signal can be observed in parallel to any other signal. | + | | '' |
- | + | | '' | |
- | ^ value^input vector(64 bits) ^ | + | | '' |
- | | 0|corrected ADC data of ADCs 0 - 3 | | + | |
- | | 1|corrected ADC data of ADCs 4 - 7 | | + | |
- | | 2|BPM 0 and 1 result, additional information | + | |
- | | 3|BPM 2 and 3 result, additional information | + | |
- | | 4|BPM 0 and 1 averaging result, additional information | + | |
- | | 5|BPM 2 and 3 averaging result, additional information | + | |
- | | 6|SPI and I2C signals, MLVDS signals, | + | |
- | | 7|test counter | + | |
- | + | ||
- | For a detailed description of the input vectors see [[# | + | |
- | + | ||
- | === 122: Observer number of samples - 1 === | + | |
- | + | ||
- | The number of samples minus one that are stored after the observer has been triggered. Each sample consists of 16 bytes. | + | |
- | + | ||
- | === 123: Observer trigger select === | + | |
- | + | ||
- | Analog | + | |
- | + | ||
- | === 124: Observer trigger compare vector (t = -1) === | + | |
- | + | ||
- | 64 bit wide compare vector that is compared with the observer input vector determined by register 123 ’observer trigger select’. If the two pattern match, the next sample will be compared | + | |
- | + | ||
- | === 125: Observer trigger compare vector (t = -1) === | + | |
- | + | ||
- | See above. If the patterns do not match, the next sample will be compared | + | |
- | + | ||
- | === 126: Observer trigger compare bit mask === | + | |
- | + | ||
- | Determines which bits of the input vector shall be compared with that of the compare vectors. Valid for both trigger compare vectors (registers 124 and 125). For triggering, the patterns must match for all bits whose bit mask is 1. | + | |
- | + | ||
- | === 126: Observer arm trigger === | + | |
- | + | ||
- | Starts the comparing process. Data is captured if the patterns defined by the previous three registers match. | + | |
- | + | ||
- | ==== 7.1.2 Additional status registers ==== | + | |
- | + | ||
- | The following additional status registers can be read by software: | + | |
- | ^ **index**^ | ||
- | | '' | ||
- | | '' | ||
- | | '' | ||
- | | '' | ||
- | | '' | ||
- | | '' | ||
- | | '' | ||
- | | '' | ||
- | | '' | ||
- | | '' | ||
- | | '' | ||
- | | '' | ||
- | | '' | ||
- | | '' | ||
- | | '' | ||
- | | '' | ||
- | | '' | ||
- | | '' | ||
- | | '' | ||
- | | '' | ||
- | | '' | ||
- | | '' | ||
- | | '' | ||
- | | '' | ||
- | | '' | ||
- | | '' | ||
- | | '' | ||
- | | '' | ||
- | | '' | ||
- | | '' | ||
- | | '' | ||
- | | '' | ||
- | **Table 7.3:** List of additional status registers | + | **Table 7.1:** List of additional status registers |
- | === 64, 80: FMC {0, 1} SPI busy === | + | == 64, 80: FMC {0, 1} SPI busy == |
Indicates that a SPI read or write access is going on. The value of this register has to be checked to be 0 before triggering a SPI access. | Indicates that a SPI read or write access is going on. The value of this register has to be checked to be 0 before triggering a SPI access. | ||
- | === 65, 81: FMC {0, 1} SPI read data === | + | == 65, 81: FMC {0, 1} SPI read data == |
Contains the result of a read access to a SPI register. | Contains the result of a read access to a SPI register. | ||
- | === 66, 82: FMC {0, 1} I2C busy === | + | == 66, 82: FMC {0, 1} I2C busy == |
Indicates that an I2C read or write access is going on. The value of this register has to be checked to be 0 before triggering an I2C access. | Indicates that an I2C read or write access is going on. The value of this register has to be checked to be 0 before triggering an I2C access. | ||
- | === 67, 83: FMC {0, 1} I2C read data === | + | == 67, 83: FMC {0, 1} I2C read data == |
Contains the result of a read access to an I2C register. | Contains the result of a read access to an I2C register. | ||
- | === 68, 84: FMC {0, 1} PLL status | + | == 68, 84: FMC {0, 1} PLL status == |
Value of the configurable output pin //status// of the AD9510 PLL and clock distribution IC. By default this pin indicates lock status of the PLL. | Value of the configurable output pin //status// of the AD9510 PLL and clock distribution IC. By default this pin indicates lock status of the PLL. | ||
- | === 69, 85: FMC {0, 1} VCXO initial RFREQ === | + | == 69, 85: FMC {0, 1} VCXO initial RFREQ == |
- | //RFREQ// is a factory calibrated multiplicator to the XTAL frequency of the Si571 programmable VCXO. Before the programming of a new output frequency this value has to be read (see [[#31_si571_programmable_vcxo|chapter 3.1]]). | + | //RFREQ// is a factory calibrated multiplicator to the XTAL frequency of the Si571 programmable VCXO. Before the programming of a new output frequency this value has to be read (see chapter |
- | === 69, 86: FMC {0, 1} VCXO RFREQ === | + | == 69, 86: FMC {0, 1} VCXO RFREQ == |
- | The VCXO output frequency is programmed to 125 MHz by the gateware. This register holds the value of //RFREQ// that has been programmed (see [[#31_si571_programmable_vcxo|chapter 3.1]]). | + | The VCXO output frequency is programmed to 125 MHz by the gateware. This register holds the value of //RFREQ// that has been programmed (see chapter |
- | === 71, 87: FMC {0, 1} measured ADC clock frequency | + | == 71, 87: FMC {0, 1} measured ADC clock frequency == |
The ADC clock is measured against the main processing clock. This register holds the number of detected ADC clock cycles during 1 second of the main processing clock. | The ADC clock is measured against the main processing clock. This register holds the number of detected ADC clock cycles during 1 second of the main processing clock. | ||
- | === 72, 88: FMC {0, 1} ADC FIFO underflow counter | + | == 72, 88: FMC {0, 1} ADC FIFO underflow counter == |
If the ADC clock is slower than the main processing clock, samples will be repeated by the clock domain crossing FIFO output logic. For each repetition the underflow counter will be incremented by 1. | If the ADC clock is slower than the main processing clock, samples will be repeated by the clock domain crossing FIFO output logic. For each repetition the underflow counter will be incremented by 1. | ||
- | === 73, 89: FMC {0, 1} ADC FIFO underflow counter | + | == 73, 89: FMC {0, 1} ADC FIFO underflow counter == |
If the ADC clock is faster than the main processing clock, samples will discarded by the clock domain crossing FIFO input logic. For each discarded sample the overflow counter will be incremented by 1. | If the ADC clock is faster than the main processing clock, samples will discarded by the clock domain crossing FIFO input logic. For each discarded sample the overflow counter will be incremented by 1. | ||
- | === {96 - 103}: ADC {0 - 7} max peak to peak === | + | == {96 - 103}: ADC {0 - 7} max peak to peak == |
The maximum and the minimum value of the ADC data is determined over a free running period of 1 second. This register contains the difference of the maximum and the minimum value. | The maximum and the minimum value of the ADC data is determined over a free running period of 1 second. This register contains the difference of the maximum and the minimum value. | ||
- | === 111: SDRAM initial calibration complete | + | == 111: SDRAM initial calibration complete == |
The communication to the SDRAM is controlled by an IP core by Xilinx which performs a timing calibration at start up. The value of this register will be 1 after completion of the initial calibration. | The communication to the SDRAM is controlled by an IP core by Xilinx which performs a timing calibration at start up. The value of this register will be 1 after completion of the initial calibration. | ||
- | === 124: observer triggered | + | === 7.1.2 Additional configuration registers |
- | Indicates that the observer has been triggered. | + | The following additional registers can be written by software: |
- | === 125: observer capture busy === | + | ^ |
+ | | '' | ||
+ | | '' | ||
+ | | '' | ||
+ | | '' | ||
+ | | '' | ||
+ | | '' | ||
+ | | '' | ||
+ | | '' | ||
+ | | '' | ||
+ | | '' | ||
+ | | '' | ||
+ | | '' | ||
+ | | '' | ||
+ | | '' | ||
+ | | '' | ||
+ | | '' | ||
+ | | '' | ||
+ | | '' | ||
+ | | '' | ||
+ | | '' | ||
+ | | '' | ||
+ | | '' | ||
+ | | '' | ||
+ | | '' | ||
+ | | '' | ||
+ | | '' | ||
+ | | '' | ||
+ | | '' | ||
+ | | '' | ||
+ | | '' | ||
+ | | '' | ||
+ | | '' | ||
+ | | '' | ||
+ | | '' | ||
+ | | '' | ||
+ | | '' | ||
+ | | '' | ||
+ | | '' | ||
+ | | '' | ||
+ | | '' | ||
+ | | '' | ||
+ | | '' | ||
- | Indicates that a capturing process is ongoing. | ||
- | === 126: build timestamp === | + | **Table 7.2:** List of additional configuration registers - part 1 |
- | Time when the bitstream was created. This information can be used to identify the gateware version (together with the Git commit information documented in [[# | + | ^ index^ |
+ | | '' | ||
+ | | '' | ||
+ | | '' | ||
+ | | '' | ||
+ | | '' | ||
+ | | '' | ||
+ | | '' | ||
+ | | '' | ||
+ | | '' | ||
+ | | '' | ||
+ | | '' | ||
+ | | '' | ||
+ | | '' | ||
+ | | '' | ||
+ | | '' | ||
+ | | '' | ||
- | Format: | ||
- | ^ | + | **Table 7.3:** List of additional configuration registers |
- | | 0 - 5|seconds | + | |
- | | 6 - 11|minutes | + | |
- | | 12 - 16|hours | + | |
- | | 17 - 22|last two decimal digits | + | |
- | | 23 - 26|month | + | |
- | | 27 - 31|day | + | |
- | ===== 7.2 Architecture information storage ===== | + | == 17: ADC test data enable |
- | The first seven eights of the Block RAM (see memory mapping, table 6.1) are used to store information about the observer signals, the registers and the gateware version. | + | If set to 1, all eight ADC data inputs will be overridden by a counter value which is incremented by 1 every 125 MHz clock cycle. |
- | ==== 7.2.1 Observer signal information ==== | + | == 39, 47, 55: Scope {0, 1, 2} continuous trigger |
- | Information about the signals connected | + | If set to 1, the trigger |
- | * name of signal (30 bytes) | + | == 56: AFC LED select == |
- | * display type of signal (1 byte) | + | |
- | * bit index in signal (1 byte) | + | |
- | ^ | + | There is one tricolor LED at the center |
- | | '' | + | |
- | | '' | + | |
- | | | + | |
- | | '' | + | |
- | | '' | + | |
- | | | + | |
- | | | + | |
- | | '' | + | |
- | | '' | + | |
- | | | + | |
- | | '' | + | |
- | | '' | + | |
- | | | + | |
- | | | + | |
- | | '' | + | |
- | **Table 7.4:** Observer signal information storage format | + | ^ value^input |
+ | | '' | ||
+ | | '' | ||
- | Table 7.4 shows the storage format of the 512 entries, each of which has a width of 32 bytes. The coding of the display type byte is the following: | + | == 57: AFC LED value == |
- | ^ value^display type ^ | + | Static lighting pattern if register 112 ’AFC LED select’ = 1. |
- | | 0|hexadecimal | + | |
- | | | + | |
- | | 2|unsigned | + | |
- | | 3|binary | + | |
- | | 4|analog | + | |
- | The names are stored as ASCII strings. If a name is shorter than 30 bytes, the remaining bytes are filled with Null characters. | + | ^ bit^color |
+ | | 0|red | | ||
+ | | 1|green | ||
+ | | 2|blue | ||
- | The observer signal information is used by the FPGA Observer software to display the observer signals in the Data Acquisition tab (see [[# | + | == 58: Gate override == |
- | ==== 7.2.2 Register information ==== | + | For testing purposes without an external gate signal this register can be set to 1 to simulate a gate signal via the register 115 ’gate override value’. |
- | Information about the 128 configuration registers and the 128 status registers is stored in the third quarter of the Block RAM. Following information is stored for every register: | + | == 59: Gate override value == |
- | * name of register | + | Can be used to simulate a gate signal when register |
- | * number of bits (1 byte) | + | |
- | ^ | + | == 60: MLVDS direction == |
- | | '' | + | |
- | | '' | + | |
- | | | + | |
- | | '' | + | |
- | | '' | + | |
- | | '' | + | |
- | | | + | |
- | | '' | + | |
- | **Table 7.5:** Register information storage format | + | Determines the direction of the eight MLVDS lines on the AMC connector. A ’0’ corresponds to an input to the FPGA and a ’1’ to an output from the FPGA. |
- | Table 7.5 shows the storage format of the 256 entries, each of which has a width of 32 bytes. The names are stored as ASCII strings. If a name is shorter than 31 bytes, the remaining bytes are filled with Null characters. If not all registers are in use, a width of 0 bits indicates that a register is not present. | + | == 61: MLVDS output value == |
- | The register information is used by the FPGA Observer software to display | + | Determines |
- | ==== 7.2.3 Gateware information ==== | + | == 64, 80: FMC {0, 1} status LED select |
- | The address range from 0x80006000 to 0x80006FFF | + | There is one tricolor LED on the FMC front panel labeled //status// that can be controlled by the gateware. |
- | The gateware | + | ^ |
+ | | '' | ||
+ | | '' | ||
+ | | '' | ||
- | ====== 8 Test software ====== | + | == 65, 81: FMC {0, 1} status LED value == |
- | ===== 8.1 FPGA Observer ===== | + | The static lighting pattern defined by this register becomes active if the corresponding register ’status LED select’ is set to 2 or 3. |
- | There is a graphical test software intended to be run on the CPU unit. It is implemented in Python using the GTK 3 GUI toolkit. | + | ^ bit^color |
+ | | 0|red | | ||
+ | | 1|green | ||
+ | | 2|blue | ||
- | ==== 8.1.1 Installation and usage ==== | + | == 66, 82: FMC {0, 1} SPI cs == |
- | The sources | + | Chip select signals (active high) of the SPI bus to the four ADCs and to the AD9510 PLL and clock distribution. |
- | === Installation === | + | ^ bit^device |
+ | | 0|ADC 0 | | ||
+ | | 1|ADC 1 | | ||
+ | | 2|ADC 2 | | ||
+ | | 3|ADC 3 | | ||
+ | | 4|PLL and clock distribution | ||
- | Connect to the CPU unit e.g. via ssh. Clone the // | + | == 67, 83: FMC {0, 1} SPI read/write == |
- | '' | + | 0: write mode, 1: read mode |
- | Install the PCIe driver: | + | == 68, 84: FMC {0, 1} SPI address == |
- | '' | + | The address of the register that shall be accessed. |
- | '' | + | |
- | Install the FPGA Observer software: | + | == 69, 85: FMC {0, 1} SPI write data == |
- | '' | + | The data that shall be written to a register. |
- | '' | + | |
- | {{: | + | == 70, 86: FMC {0, 1} SPI trigger == |
- | **Figure 8.1:** FPGA Observer - Register Access tab | + | Write a 1 to this register to start a read or write access on the SPI bus. The register does not have to be reset to 0 before the next SPI trigger, just write another |
- | === Usage === | + | == 71, 87: FMC {0, 1} ADC resetn |
- | For the PCIe driver | + | Low active reset signal |
- | Connect to the CPU unit e.g. via '' | + | == 72, 88: FMC {0, 1} I2c read/write == |
- | '' | + | 0: write mode, 1: read mode |
- | A GUI should open and a choice of FPGA serial numbers should be displayed on the upper left corner. If the list is empty, either the loading of the FPGAs finished after powering the CPU unit or the PCIe driver did not install correctly. The FPGA serial numbers can be used to identify the AFC board you like to access. Choose a serial number and click // | + | == 73, 89: FMC {0, 1} I2C device address == |
- | ==== 8.1.2 Register Access tab ==== | + | The address of the connected VCXO is 0x49. |
- | The names and widths of the registers are read from an information memory region in the FPGA (see [[# | + | == 74, 90: FMC {0, 1} I2C register address == |
- | The //read// button reads all the status registers either once or continuously if the // | + | The address of the register that shall be accessed. |
- | ==== 8.1.3 Data Acquisition tab ==== | + | == 75, 91: FMC {0, 1} I2C write data == |
- | The names and widths of the signals connected | + | The data that shall be written |
- | The two combo boxes // | + | == 76, 92: FMC {0, 1} I2C trigger |
- | {{: | + | Write a 1 to this register to start a read or write access on the I2C bus. The register does not have to be reset to 0 before the next I2C trigger, just write another 1 to it. |
- | **Figure 8.2:** FPGA Observer - Data Acquisition tab | + | == 77, 93: FMC {0, 1} PLL resetn == |
- | The //Number Of Samples// entry determines how many samples will be stored after a trigger event when the //capture// button has been pressed. If the // | + | Low active reset signal to the PLL and clock distribution. Tie to 0 and back to 1 to initiate a reset. |
- | The individual //Trigger Active (&)// check buttons define on which signals the trigger will listen. All of the enabled conditions have to become true for a trigger event. | + | == 78, 94: FMC {0, 1} Clock switch select == |
- | The trigger conditions for t = -1 and t = 0 contain the compare vectors | + | There is a separate clock switch in front of the AD9510 PLL reference |
- | When the data acquisition completes, | + | ^ value^connect to ^ |
+ | | '' | ||
+ | | '' | ||
- | {{: | + | == 79, 95: FMC {0, 1} VCXO output enable == |
- | **Figure 8.3:** GTKWave | + | Enables the frequency output of the VCXO. |
- | ==== 8.1.4 BPM Calculation tab ==== | + | == 96 - 99 and 104 - 107: FMC {0, 1} ADC {0 - 3} clock delay == |
- | The result of the BPM algorithm | + | There is a configurable input delay for setting |
- | {{: | + | == 100 - 103 and 108 - 111: FMC {0, 1} ADC {0 - 3} data delay == |
- | **Figure 8.4:** FPGA Observer - BPM Calculation tab | + | See above. Increasing this value increases the delay of the data, so that the data is sampled at an earlier position. |
- | ==== 8.1.5 Scope tabs ==== | + | ===== 8 Hardware properties ===== |
- | {{: | + | ==== 8.1 LEDs driven by the FPGA gateware ==== |
- | + | ||
- | **Figure 8.5:** FPGA Observer - Scope 0 tab | + | |
- | + | ||
- | ==== 8.1.6 Peripherals Configuration tab ==== | + | |
- | + | ||
- | === Configuration === | + | |
- | + | ||
- | A configuration file can be loaded to program the three different peripheral devices documented in [[# | + | |
- | The configuration file is a comma separated values (.csv) file with the following syntax: | + | |
- | + | ||
- | //device number// (1 hexadecimal digit), //register number// (2 hexadecimal digits), //value// (2 hexadecimal digits) | + | |
- | + | ||
- | The device number is encoded as follows: | + | |
- | + | ||
- | ^number | + | |
- | |0 - 3 |ADCs 0 - 3 on FMC0 | | + | |
- | |4 |all ADCs on FMC0 in parallel | + | |
- | |5 |PLL on FMC0 | | + | |
- | |6 |VCXO on FMC0 | | + | |
- | |7 - A |ADCs 0 - 3 on FMC0 | | + | |
- | |B |all ADCs on FMC0 in parallel | + | |
- | |C |PLL on FMC0 | | + | |
- | |D |VCXO on FMC0 | | + | |
- | + | ||
- | A correct configuration of the VCXOs via software can not be guaranteed due to the I2C configuration timeout of 10 ms of the VCXOs (see [[# | + | |
- | + | ||
- | === Reading the device registers === | + | |
- | + | ||
- | The complete register bank of the first ADC, the PLL and the VCXO on FMC 0 can be read and stored to three individual log files in the '' | + | |
- | + | ||
- | {{: | + | |
- | + | ||
- | **Figure 8.6:** FPGA Observer - Peripherals Configuration tab | + | |
- | + | ||
- | ==== 8.1.7 Gateware Information tab ==== | + | |
- | + | ||
- | {{: | + | |
- | + | ||
- | **Figure 8.7:** FPGA Observer - Gateware Information tab | + | |
- | + | ||
- | Information about the gateware version is displayed in this tab. The URL of the remote server of the Git repository, the latest commit hash and the latest commit date are read from an information memory region in the FPGA (see [[# | + | |
- | + | ||
- | ===== 8.2 Test scripts ===== | + | |
- | + | ||
- | ==== 8.2.1 PCIe access test script ==== | + | |
- | + | ||
- | There is a PCIe access test script under '' | + | |
- | + | ||
- | ====== 9 Helper scripts ====== | + | |
- | + | ||
- | ===== 9.1 VHDL beautification ===== | + | |
- | + | ||
- | There is a script '' | + | |
- | + | ||
- | The script expects one parameter: '' | + | |
- | + | ||
- | The script applies several corrections and changes to the //Emacs// formatting result: | + | |
- | + | ||
- | * correction of the handling of the comparison operator '' | + | |
- | * correction of the handling of initializations like '' | + | |
- | * enforcing of spaces around the operators '' | + | |
- | * no indentation for closing brackets | + | |
- | * aligning of full comment lines to the indentation level of the following VHDL command | + | |
- | * indentation with tabs instead of spaces | + | |
- | + | ||
- | ===== 9.2 Remote power cycling of the CPU unit ===== | + | |
- | + | ||
- | Whenever the bitstream of an FPGA is reloaded, the CPU unit has to be rebooted via its Hot Swap Handle in order to establish a PCIe connection. A software reboot does not work. | + | |
- | + | ||
- | An alternative possibility of remote power cycling the CPU unit is via the MCH. | + | |
- | + | ||
- | The script '' | + | |
- | + | ||
- | The script takes about 60 seconds to complete. | + | |
- | + | ||
- | ===== 9.3 Plotting of measurement data ===== | + | |
- | + | ||
- | The script '' | + | |
- | + | ||
- | ===== 9.4 Generation of a VHDL file for monitoring and control ===== | + | |
- | + | ||
- | The monitoring and control configuration of the gateware is defined by the configuration files '' | + | |
- | + | ||
- | The script '' | + | |
- | + | ||
- | The script is also executed by the gateware build flow documented in [[# | + | |
- | + | ||
- | ===== 9.5 Generation of documentation ===== | + | |
- | + | ||
- | ==== 9.5.1 PDF ==== | + | |
- | + | ||
- | There is a script '' | + | |
- | + | ||
- | ==== 9.5.2 Markdown ==== | + | |
- | + | ||
- | There is a script '' | + | |
- | + | ||
- | The result of //Pandoc// is postprocessed for multiple reasons: | + | |
- | + | ||
- | * conversion of the math syntax to Gitlab’s .md format | + | |
- | * corrections of the bibliography, | + | |
- | * corrections of the references to figures, tables and equations | + | |
- | * enumeration of chapters, sections and subsections | + | |
- | * adding of captions for figures and equations | + | |
- | * enumeration of figures, tables and equations | + | |
- | * generation of a table of contents | + | |
- | * implementation of citations | + | |
- | + | ||
- | Additional documentation which is not included in the Latex sources is appended from the file '' | + | |
- | + | ||
- | ==== 9.5.3 DokuWiki ==== | + | |
- | + | ||
- | There is a script '' | + | |
- | + | ||
- | The script converts the Latex documentation sources to the DokuWiki format in four steps: | + | |
- | + | ||
- | - conversion of Latex sources to Markdown | + | |
- | - preprocessing of Markdown before the conversion to DokuWiki | + | |
- | - calling //Pandoc// to convert Markdown to DokuWiki | + | |
- | - postprocessing for correction, extension of functionality and a different style | + | |
- | + | ||
- | The preprocessing actions are: | + | |
- | + | ||
- | * removing table of contents since it is automatically generated by DokuWiki | + | |
- | + | ||
- | The postprocessing actions are: | + | |
- | + | ||
- | * conversion or equations to images since DokuWiki can not render equations | + | |
- | * replacement of HTML tags, Latex color tags, etc. since DokuWiki can not handle them | + | |
- | * conversion of the reference format | + | |
- | + | ||
- | ====== 10 Continuous integration environment ====== | + | |
- | + | ||
- | There is a continuous integration environment setup for the // | + | |
- | + | ||
- | At the moment the Gitlab Runner is running on the Linux server //sdlx035// located in a server room in the basement. | + | |
- | + | ||
- | The benefits of continuous integration are: | + | |
- | + | ||
- | * every change will be tested automatically | + | |
- | * it is ensured that no files are missing in the repository | + | |
- | * the master branch can be kept functional at any time | + | |
- | * build results like e.g. bitstreams are automatically generated and can be archived | + | |
- | + | ||
- | {{: | + | |
- | + | ||
- | **Figure 10.1:** Gitlab: continuous integration pipelines | + | |
- | + | ||
- | ===== 10.1 Installation ===== | + | |
- | + | ||
- | There is an installation script '' | + | |
- | + | ||
- | After the installation, | + | |
- | + | ||
- | On the newly installed Gitlab Runner server, open a terminal and type '' | + | |
- | + | ||
- | Enter the following information: | + | |
- | + | ||
- | * gitlab-ci coordinator URL: e.g. '' | + | |
- | * gitlab-ci token: enter the registration token copied before | + | |
- | * gitlab-ci description: | + | |
- | * gitlab-ci tags: leave empty | + | |
- | * executor: '' | + | |
- | + | ||
- | You can add multiple repositories with different tokens by running '' | + | |
- | + | ||
- | ===== 10.2 Pipeline Stages ===== | + | |
- | + | ||
- | Each '' | + | |
- | + | ||
- | The following pipeline stages are defined: | + | |
- | + | ||
- | * documentation | + | |
- | * simulation | + | |
- | * FPGA build | + | |
- | + | ||
- | {{: | + | |
- | + | ||
- | **Figure 10.2:** Gitlab: Pipeline stages | + | |
- | + | ||
- | ==== 10.2.1 Documentation ==== | + | |
- | + | ||
- | The script '' | + | |
- | + | ||
- | The log file of Pdflatex and - if successful - the PDF of the documentation are archived. | + | |
- | + | ||
- | ==== 10.2.2 Simulation ==== | + | |
- | + | ||
- | The script '' | + | |
- | + | ||
- | The log file of the simulation and - if successful - a file with the BPM results from the simulation are archived. | + | |
- | + | ||
- | ==== 10.2.3 FPGA build ==== | + | |
- | + | ||
- | The script '' | + | |
- | + | ||
- | Different log files from synthesis and implementation, | + | |
- | + | ||
- | {{: | + | |
- | + | ||
- | **Figure 10.3:** Gitlab: Pipeline progress console | + | |
- | + | ||
- | ===== 10.3 Build results ===== | + | |
- | + | ||
- | For each of the pipeline stages the archiving of build results can be configured for an adjustable time period, which is set to one week. If the period has passed and the build results have been deleted, they can be generated again by restarting the pipeline. | + | |
- | + | ||
- | The build results can be downloaded from the Gitlab web front end where they are called //job artifacts// (see figure 10.3). | + | |
- | + | ||
- | <color green> | + | |
- | having to set up a build environment.**</ | + | |
- | + | ||
- | ===== 10.4 Settings ===== | + | |
- | + | ||
- | You can define individual settings for the CI/CD section of each Git repository in the Gitlab web front end. The following settings should fit for most cases: | + | |
- | + | ||
- | * Use git clone to get the recent application code, otherwise the pipelines might fail during git fetch:\\ | + | |
- | //Settings CI/CD General pipelines Git strategy for pipelines: git clone// | + | |
- | * Increase the timeout to allow FPGA build to finish in any case:\\ | + | |
- | //Settings CI/CD General pipelines Timeout: 6h// | + | |
- | + | ||
- | ====== 11 Programming and hardware configuration ====== | + | |
- | + | ||
- | ===== 11.1 Programming the gateware ===== | + | |
- | + | ||
- | ==== 11.1.1 Using a JTAG programmer ==== | + | |
- | + | ||
- | Before being able to access the FPGA you need to program the JTAG switch on the AFC board using a script from the // | + | |
- | + | ||
- | //Tools Run Tcl Script//: '' | + | |
- | + | ||
- | You should now see a // | + | |
- | + | ||
- | Choose the correct bitstream (.bit file) and press //OK//. The programming takes about one minute. | + | |
- | + | ||
- | ==== 11.1.2 Using a JTAG Switch Module ==== | + | |
- | + | ||
- | If there is a JTAG Switch Module (JSM) in the MicroTCA crate, the bitstream can also be programmed remotely via a so called Xilinx Virtual Cable: | + | |
- | + | ||
- | * download '' | + | |
- | * download '' | + | |
- | * convert '' | + | |
- | * upload '' | + | |
- | * open Vivado Hardware Manager | + | |
- | * //Open Target New Target Next Local Server Add Xilinx Virtual Cable (XVC)// | + | |
- | * // | + | |
- | * Port: find correct port number in //NAT-MCH GUI JSM// | + | |
- | * // | + | |
- | * //Open target// | + | |
- | * you should see the FPGA now in Vivado Hardware Manager and can program it | + | |
- | + | ||
- | The first four steps are persistent and only have to be executed initially. | + | |
- | + | ||
- | ==== 11.1.3 Storing a bitstream persistently in the SPI Flash ==== | + | |
- | + | ||
- | There is a 256 MB SPI Flash memory on the AFCv3.1 board for persistent bitstream storage. | + | |
- | + | ||
- | === File format conversion === | + | |
- | + | ||
- | First you have to convert the bitstream (.bit) file to a .mcs file. There is a script in the // | + | |
- | + | ||
- | '' | + | |
- | + | ||
- | The .mcs file will be generated in the same folder as the .bit file. | + | |
- | + | ||
- | === Programming === | + | |
- | + | ||
- | Program the JTAG switch on the AFC board as described in [[# | + | |
- | Right click on it and choose //Add Configuration Memory Device// and choose // | + | |
- | + | ||
- | You should now see a // | + | |
- | + | ||
- | Right click on it and choose //Program Configuration Memory Device// | + | |
- | Choose the .mcs file you created before and press //OK//. The programming is really slow and can take up to half an hour. | + | |
- | + | ||
- | ===== 11.2 Configuration of the MCH ===== | + | |
- | + | ||
- | ==== 11.2.1 Via the MCH’s web interface ==== | + | |
- | + | ||
- | === Base configuration === | + | |
- | + | ||
- | //MCH global parameter SSH access: enabled// | + | |
- | This will trigger SSH key generation which takes some minutes to complete. | + | |
- | + | ||
- | //PCIe parameter Upstream slot power up delay: 5 sec//\\ | + | |
- | Delay before the CPU unit will power up on start up. For making sure that the bitstreams are loaded to the AFC’s FPGAs from Flash memory before the CPU unit boots you might have to increase this value. | + | |
- | + | ||
- | //PCIe parameter PCIe hot plug delay for AMCs: 0 sec//\\ | + | |
- | Delay before the AFC boards will power up on start up. | + | |
- | + | ||
- | === Switch PCIe x80 === | + | |
- | + | ||
- | Set the CPU-Unit as upstream AMC source in ’Virtual Switch 0’: | + | |
- | + | ||
- | //PCIe Virtual Switches Upstream AMC: AMC1/ | + | |
- | (for CPU unit in AMC slot 1) | + | |
- | + | ||
- | Make sure you enable //PCIe downstream ’4..7’// | + | |
- | + | ||
- | ==== 11.2.2 Via USB ==== | + | |
- | + | ||
- | The most comfortable way of configuring the MCH is via its web interface. If you have accidentally disabled the webserver, set an invalid IP or DHCP configuration or reset the MCH settings to default, you can access the MCH via an USB connection to the micro USB port on the left side of the front panel. | + | |
- | + | ||
- | On a Linux PC, connect a micro USB cable and check via '' | + | |
- | + | ||
- | Now typing '' | + | |
- | + | ||
- | ===== 11.3 Enabling network boot on the CPU unit ===== | + | |
- | + | ||
- | Shortly after powering the CPU unit, press F2 to enter the BIOS. | + | |
- | + | ||
- | In the //Main// tab, go to //Boot Features// and select the following (using F6 for enabling and F5 for disabling): | + | |
- | + | ||
- | * '' | + | |
- | * '' | + | |
- | * '' | + | |
- | + | ||
- | In the // | + | |
- | + | ||
- | In the //Boot// tab, go to //Legacy// and //Boot Type Order//. There should be an //Others// entry that has to be shifted to the top of the list using F6. In newer BIOS versions, there is a //Boot Option #1// entry in the //Boot// tab, which has to be chosen to //IBA GE Slot 1600 v1513//. | + | |
- | + | ||
- | Save the settings by pressing F4. | + | |
- | + | ||
- | The CPU unit should boot from network after the next reboot. For the loading of the correct network image, the MAC address of the desired Ethernet port of the CPU unit has to registered in the DHCP server responsible for distributing the locations from which to load the network images. | + | |
- | + | ||
- | ===== 11.4 MMC firmware ===== | + | |
- | + | ||
- | The NXP LPC1764FBD100 microcontroller on the AFC board is responsible for: | + | |
- | + | ||
- | * implementing the AMC protocol: | + | |
- | * power control | + | |
- | * provision of sensor data | + | |
- | * driving the AMC standard LEDs | + | |
- | * upstart procedure | + | |
- | * configuration of the onboard devices: | + | |
- | * PLL configuration | + | |
- | * clock switch configuration | + | |
- | * control of various pins on the FPGA and the FMC connectors: | + | |
- | * FPGA reset pin | + | |
- | * FMC power good pins | + | |
- | + | ||
- | ==== 11.4.1 Building the MMC firmware ==== | + | |
- | + | ||
- | === Getting the sources === | + | |
- | + | ||
- | The source code of LNLS’s openMMC code can be found on https:// | + | |
- | + | ||
- | === Installation of required software === | + | |
- | + | ||
- | An ARM cross compiler is needed to compile the code on a x86-64 architecture. | + | |
- | + | ||
- | A prebuild GCC for ARM can be downloaded from e.g. https:// | + | |
- | + | ||
- | Install the downloaded TAR archive by: | + | |
- | + | ||
- | '' | + | |
- | + | ||
- | === Building === | + | |
- | + | ||
- | * Create a '' | + | |
- | * '' | + | |
- | * '' | + | |
- | + | ||
- | The firmware files will be created in the '' | + | |
- | + | ||
- | * '' | + | |
- | * '' | + | |
- | + | ||
- | ==== 11.4.2 Programming the MMC firmware ==== | + | |
- | + | ||
- | For programming the MMC firmware into the LPC microcontroller you need to install a proprietary software from NXP called LPCxpresso. | + | |
- | + | ||
- | === Installation of LPCxpresso on Linux === | + | |
- | + | ||
- | Download LPCxpresso from the NXP website [[# | + | |
- | + | ||
- | === Programming === | + | |
- | + | ||
- | Disconnect the AFC board completely. The power for programming the microcontroller will come from the LPC-Link programmer. Connect and power the LPC-Link programmer via USB and connect the customized cable to the // | + | |
- | + | ||
- | Program the device via: | + | |
- | + | ||
- | '' | + | |
- | + | ||
- | '' | + | |
- | + | ||
- | You can find the openMMC firmware binary under '' | + | |
- | + | ||
- | ==== 11.4.3 Differences in the MMC firmwares of Creotech and LNLS ==== | + | |
- | + | ||
- | LNLS’s OpenMMC firmware routes a 100 MHz clock to the PCIe reference clock input, whereas Creotech’s MMC firmware routes a 125 MHz clock to this pin. The frequency of //sys_clk// is 125 MHz for both. | + | |
- | + | ||
- | The OpenMMC firmware should forward the signal of the reset button on the AFC front panel to the FPGA pin AG26 after some seconds. However, no reaction can be observed on this pin. It is unclear if or how Creotechs MMC firmware handles reset button actions, since there is no observable reaction. | + | |
- | + | ||
- | The current gateware is functional with the OpenMMC firmware. For running it together with Creotech’s MMC firmware, the PCIe reference clock frequency in the IP // | + | |
- | + | ||
- | ===== 11.5 Configuration of the timing receiver ===== | + | |
- | + | ||
- | The timing receiver is connected to the AFC boards via eight differential MLVDS lines via the backplane of the MicroTCA crate. By default, the gateware is configured to use the first MLVDS line as an input for the gate signal. | + | |
- | + | ||
- | The GPIOs of the timing receiver can be controlled via command line commands available in the network boot image of the CPU unit: | + | |
- | + | ||
- | * display all available GPIOs: '' | + | |
- | * enable the outputs to the backplane: '' | + | |
- | * display the properties of a single GPIO: e.g. '' | + | |
- | * enable the output of a single GPIO: e.g. '' | + | |
- | * drive the output of a single GPIO high: e.g. '' | + | |
- | + | ||
- | ====== 12 Hardware properties ====== | + | |
- | + | ||
- | ===== 12.1 LEDs on the AFC and FMC front panels ===== | + | |
- | + | ||
- | ==== 12.1.1 LEDs driven by the FPGA gateware ==== | + | |
There are three tricolor LEDs connected to FPGA pins: | There are three tricolor LEDs connected to FPGA pins: | ||
- | * //L3// in the center of the AFC front panel:\\ | + | * //L3// in the center of the AFC front panel: Currently displays the PCIe reference clock divided by {{: |
- | Currently displays the PCIe reference clock divided by {{: | + | * //LD1// (v1.0) or //STATUS// (v1.2 and v2.3) on the right of the two FMC board’s front panels: Currently display the ADC clock frequencies divided by {{: |
- | * //LD1// (v1.0) or //STATUS// (v1.2 and v2.3) on the right of the two FMC board’s front panels:\\ | + | |
- | Currently display the ADC clock frequencies divided by {{: | + | |
Each tricolor LED consists of three independent LEDs (red, green and blue). | Each tricolor LED consists of three independent LEDs (red, green and blue). | ||
- | ==== 12.1.2 LEDs driven by the MMC ==== | + | ==== 8.2 Differences between FMC ADC 250 M 16B 4CH versions |
- | + | ||
- | * In Service (//L1//), green | + | |
- | * Alarm (//L2//), red | + | |
- | * Hot Swap (//HS//), blue | + | |
- | + | ||
- | === Lighting patterns of the Hot Swap LED === | + | |
- | + | ||
- | Insertion of an AFC board: | + | |
- | + | ||
- | ^**event** | + | |
- | |AMC inserted into chassis with handle open |Open | + | |
- | |AMC handle closed | + | |
- | |Activation granted and AMC powers up |Closed | + | |
- | + | ||
- | Source: [[# | + | |
- | + | ||
- | Removal of an AFC board: | + | |
- | + | ||
- | ^**event** | + | |
- | |AMC handle pulled open | + | |
- | |Deactivation granted and AMC powers down (AMC can now be removed) | + | |
- | + | ||
- | Source: [[# | + | |
- | + | ||
- | ===== 12.2 MCH PCIe status LEDs ===== | + | |
- | + | ||
- | The lighting patterns of the PCIe status LEDs on the MCH show the link status and the link speed of the PCIe connections: | + | |
- | + | ||
- | ^**LED state** | + | |
- | |off |no PCIe link | | + | |
- | |1 blink/ | + | |
- | |2 blinks/ | + | |
- | |on |8 GBaud | | + | |
- | + | ||
- | Source: [[# | + | |
- | + | ||
- | ===== 12.3 Differences between hardware versions ===== | + | |
- | + | ||
- | ==== 12.3.1 Differences between AFC version | + | |
- | + | ||
- | Both boards carry 2 GiBytes of DDR3-SDRAM, divided in four modules of 512 MiBytes each. The SDRAM model can be determined via the FBGA code printed on the modules using the Micron part decoder webpage [[# | + | |
- | + | ||
- | === AFC version 2 === | + | |
- | + | ||
- | * FBGA code: D9PBC, translates to Micron MT41J512M8RA-125: | + | |
- | * operates at 1.5 V | + | |
- | + | ||
- | === AFC version 3.1 === | + | |
- | + | ||
- | * FBGA code: D9QBV, translates to Micron MT41K512M8RH-125 IT:E | + | |
- | * compatible to older MT41J family, operates at 1.5 V or 1.35 V | + | |
- | + | ||
- | === Differences between FMC ADC 250 M 16B 4CH versions === | + | |
The //LD1// (v1.0) or //STATUS// (v1.2 and v2.3) LED on the right of the FMC board front panel is connected differently between v1.0 and (v1.2 and v2.3). When using the location constraints for v1.2 and v2.3 together with a v1.0 board, the LED lights as follows: | The //LD1// (v1.0) or //STATUS// (v1.2 and v2.3) LED on the right of the FMC board front panel is connected differently between v1.0 and (v1.2 and v2.3). When using the location constraints for v1.2 and v2.3 together with a v1.0 board, the LED lights as follows: | ||
- | * wanted red off | + | * wanted red: off |
- | * wanted green lights green | + | * wanted green: lights green |
- | * wanted blue lights red | + | * wanted blue: lights red |
Also, the MMCX input //TRIG// seems to be connected differently on v1.0. Feeding HF-Pulses into v1.0 boards does not work with the current bitstream. | Also, the MMCX input //TRIG// seems to be connected differently on v1.0. Feeding HF-Pulses into v1.0 boards does not work with the current bitstream. | ||
- | ===== 12.4 Maximum achievable data rate to and from SDRAM ===== | + | ==== 8.3 Analog characteristics |
- | The gross data rate of the SDRAM interface is 800 MT/s with 32 bits/ | + | === 8.3.1 ADC input filter === |
- | + | ||
- | The storage of the samples of all eight ADCs in parallel at a sampling rate of 125 MHz results in a write data rate of: | + | |
- | + | ||
- | {{: | + | |
- | + | ||
- | The SDRAM capacity of 2 GiBytes would be sufficient to store the stream data of all eight ADCs for 1.07 seconds. | + | |
- | + | ||
- | ===== 12.5 Analog characteristics ===== | + | |
- | + | ||
- | ==== 12.5.1 ADC input filter | + | |
The FMC ADC boards were originally designed for very high input frequencies and are equipped with input filters that show a pronounced high pass characteristic. There are different versions of the boards which have a different ADC input filter circuitry. | The FMC ADC boards were originally designed for very high input frequencies and are equipped with input filters that show a pronounced high pass characteristic. There are different versions of the boards which have a different ADC input filter circuitry. | ||
- | {{: | + | {{: |
- | **Figure | + | **Figure |
- | {{: | + | {{: |
- | **Figure | + | **Figure |
- | The part labeled //TR1(B) BD0205F5050A00// | + | The part labeled //TR1(B) BD0205F5050A00// |
For being able to use the FMC ADC boards in the Cryring BPM system, the baluns have to be replaced by more suitable components. | For being able to use the FMC ADC boards in the Cryring BPM system, the baluns have to be replaced by more suitable components. | ||
Line 2124: | Line 1551: | ||
Two approaches have been implemented on versions 1.0 and 1.2 (probably by Piotr Miedzik): | Two approaches have been implemented on versions 1.0 and 1.2 (probably by Piotr Miedzik): | ||
- | | + | |
- | | + | |
- | {{: | + | {{: |
- | **Figure | + | **Figure |
- | {{: | + | {{: |
- | **Figure | + | **Figure |
- | {{: | + | {{: |
- | **Figure | + | **Figure |
The heatspreader under the bottom of the FMC ADC board has to be unscrewed to access the baluns. | The heatspreader under the bottom of the FMC ADC board has to be unscrewed to access the baluns. | ||
Line 2143: | Line 1570: | ||
There is a significant difference in the ADC input filter circuitry between versions 1.0 and 1.2 and version 2.3. In version 2.3 the transmission line transformers //L11 {A, B, C, D}// have been removed and the RC filter has been modified. | There is a significant difference in the ADC input filter circuitry between versions 1.0 and 1.2 and version 2.3. In version 2.3 the transmission line transformers //L11 {A, B, C, D}// have been removed and the RC filter has been modified. | ||
- | Figure | + | Figure |
- | {{: | + | {{: |
- | **Figure | + | **Figure |
- | ===== 12.6 Required changes for PLL lock ===== | + | ==== 8.4 Required changes for PLL lock ==== |
- | For being able to drive the reference clock of the PLLs on the FMC ADC boards from an FPGA output pin, a pullup resistor indicating the clock direction has to be desoldered. The resistor is labeled //R132// in the FMC ADC schematic [[#15_references|[17]]]. Figure | + | For being able to drive the reference clock of the PLLs on the FMC ADC boards from an FPGA output pin, a pullup resistor indicating the clock direction has to be desoldered. The resistor is labeled //R132// in the FMC ADC schematic [[#10_references|[13]]]. Figure |
Without this change slight clock frequency differences between the processing clock on the AFC board and the ADC clocks on the FMC boards occur. There are synchronization FIFOs which ensure correct clock domain crossings, but in this case a small fraction of ADC samples might have to be discarded or repeated once, depending on which frequency is higher. This will always affect all the samples of a FMC board in parallel, so that no differences between the two input data streams of a single BPM will occur and no measurable effect on the BPM results should be observed. | Without this change slight clock frequency differences between the processing clock on the AFC board and the ADC clocks on the FMC boards occur. There are synchronization FIFOs which ensure correct clock domain crossings, but in this case a small fraction of ADC samples might have to be discarded or repeated once, depending on which frequency is higher. This will always affect all the samples of a FMC board in parallel, so that no differences between the two input data streams of a single BPM will occur and no measurable effect on the BPM results should be observed. | ||
Line 2157: | Line 1584: | ||
With this changes applied and with the correct settings of the PLL, the ADC samples of both FMC boards will arrive exactly in parallel and no samples will be lost or repeated. | With this changes applied and with the correct settings of the PLL, the ADC samples of both FMC boards will arrive exactly in parallel and no samples will be lost or repeated. | ||
- | {{:ds: | + | Update: Even though removing the resistor |
- | **Figure 12.7:** Pullup resistor which needs to be removed for the PLL to lock | + | {{:ds: |
- | ===== 12.7 List of AFC v3.1 boards ===== | + | **Figure 8.7:** Pullup resistor which needs to be removed for the PLL to lock |
- | ^**AFC serial number** | + | ==== 8.5 List of ADC-FMC |
- | |111154 | + | |
- | |191087 | + | |
- | |240030 | + | |
- | |260046 | + | |
- | |261056 | + | |
- | |290148 | + | |
- | The FPGA serial | + | ^ |
+ | | ADC 1|4 | ||
+ | | ADC 2|- | ||
+ | | ADC 3|1 | ||
+ | | ADC 4|- | ||
+ | | ADC 5|4 | ||
+ | | | ||
+ | | ADC 7|- | ||
+ | | ADC 8|- | ||
+ | | ADC 9|- | ||
+ | | ADC 10|1 | ||
+ | | ADC 11|5 | ||
+ | | ADC 12|2 | ||
+ | | ADC 13|2 | ||
+ | | ADC 14|3 | ||
+ | | ADC 15|5 | ||
- | ====== 13 Test coverage ====== | + | ==== 8.6 Productive setup ==== |
- | ===== 13.1 BPM algorithm ===== | + | The following AFC version 3.1 boards are installed in the productive setup in the Cryring container: |
- | ==== 13.1.1 Simulation | + | ^AFC number |
+ | |1 | ||
+ | |2 | ||
+ | |3 | ||
+ | |4 | ||
+ | |5 | ||
+ | |||
+ | The FPGA serial number is not printed anywhere, but can only be read from the DNA_PORT primitive by the gateware. In this gateware the FPGA serial number is read out and stored in a register (see chapter [[# | ||
+ | |||
+ | The signal cables are connected as follows: | ||
+ | |||
+ | ^signal | ||
+ | |YR2DX1HL | ||
+ | |YR2DX1HR | ||
+ | |YR2DX2VO | ||
+ | |YR2DX2VU | ||
+ | |YR10DX1HL | ||
+ | |YR10DX1HR | ||
+ | |YR10DX2VO | ||
+ | |YR10DX2VU | ||
+ | |YR3DX3VO | ||
+ | |YR3DX3VU | ||
+ | |YR3DX4HL | ||
+ | |YR3DX4HR | ||
+ | |YR12DX1HL | ||
+ | |YR12DX1HR | ||
+ | |YR12DX2VO | ||
+ | |YR12DX2VU | ||
+ | |YR7DX1HL | ||
+ | |YR7DX1HR | ||
+ | |YR7DX2VO | ||
+ | |YR7DX2VU | ||
+ | |YR3DX1HL | ||
+ | |YR3DX1HR | ||
+ | |YR3DX2VO | ||
+ | |YR3DX2VU | ||
+ | |YR8DX1HL | ||
+ | |YR8DX1HR | ||
+ | |YR8DX2VO | ||
+ | |YR8DX2VU | ||
+ | |YR6DX1HL | ||
+ | |YR6DX1HR | ||
+ | |YR6DX2VO | ||
+ | |YR6DX2VU | ||
+ | |YR11DX1HL | ||
+ | |YR11DX1HR | ||
+ | |YR11DX2VO | ||
+ | |YR11DX2VU | ||
+ | |||
+ | ===== 9 Test coverage ===== | ||
+ | |||
+ | ==== 9.1 BPM algorithm ==== | ||
+ | |||
+ | === 9.1.1 Simulation === | ||
This test simulates the VHDL code of the gateware and is automatically run by the CI/CD pipelines of Gitlab. | This test simulates the VHDL code of the gateware and is automatically run by the CI/CD pipelines of Gitlab. | ||
Line 2183: | Line 1672: | ||
All ADC data inputs are driven by the same repeated pattern of positive and negative values, but with different amplitudes. | All ADC data inputs are driven by the same repeated pattern of positive and negative values, but with different amplitudes. | ||
- | For the same patterns on both inputs of a BPM with the amplitudes {{: | + | For the same patterns on both inputs of a BPM with the amplitudes {{: |
- | {{: | + | {{: |
and equation 4.1 simplifies to: | and equation 4.1 simplifies to: | ||
- | {{: | + | {{: |
- | **Equation | + | |
+ | **Equation | ||
so that the expected BPM result can be calculated as: | so that the expected BPM result can be calculated as: | ||
- | {{: | + | {{: |
- | ^ | + | ^ BPM^ ADC^ relative amplitudes {{: |
- | | 0| 0| | + | | 0| 0| |
- | | | + | | |
- | | 1| 2| | + | | 1| 2| |
- | | | + | | |
- | | 2| 4| | + | | 2| 4| |
- | | | + | | |
- | | 3| 6| | + | | 3| 6| |
- | | | + | | |
The simulation results are consistent with the expectations considering possible numeric calculation deviations in the numerous calculation steps, which might influence the least significant bits. | The simulation results are consistent with the expectations considering possible numeric calculation deviations in the numerous calculation steps, which might influence the least significant bits. | ||
- | ==== 13.1.2 Using a function generator as data source | + | === 9.1.2 Using a function generator as data source === |
- | === Digital gain setting | + | == Digital gain setting == |
- | The following measurement was made using a function generator which was configured to output two phase aligned sines with an amplitude of {{: | + | The following measurement was made using a function generator which was configured to output two phase aligned sines with an amplitude of {{: |
The linear regression length and the averaging length were both set to 1024. | The linear regression length and the averaging length were both set to 1024. | ||
Line 2221: | Line 1711: | ||
After that, the digital gain correction of the other ADC input was used to set different amplitudes in order to avoid possible nonlinearities of the function generator gain. | After that, the digital gain correction of the other ADC input was used to set different amplitudes in order to avoid possible nonlinearities of the function generator gain. | ||
- | The results were read from the FPGA Observer GUI which displays the BPM results divided by {{: | + | The results were read from the FPGA Observer GUI which displays the BPM results divided by {{: |
- | ^ | + | ^ relative amplitude^ |
- | | | + | | |
- | | | + | | |
- | | | + | | |
- | | | + | | |
- | | | + | | |
- | | | + | | |
- | | | + | | |
- | | | + | | |
The measurement results are consistent with the expectations considering noise and possible numeric calculation deviations in the numerous calculation steps, which might influence the least significant bits. | The measurement results are consistent with the expectations considering noise and possible numeric calculation deviations in the numerous calculation steps, which might influence the least significant bits. | ||
- | ===== 13.2 Reliability tests ===== | + | ==== 9.2 Reliability tests ==== |
A test of the BPM scope and the BPM averaging scope was run overnight. The two inputs of a BPM were fed by two function generator outputs with the following settings: | A test of the BPM scope and the BPM averaging scope was run overnight. The two inputs of a BPM were fed by two function generator outputs with the following settings: | ||
Line 2241: | Line 1731: | ||
* sine signal | * sine signal | ||
* frequency: 1 MHz | * frequency: 1 MHz | ||
- | * output to ADC0: 2.001 {{: | + | * output to ADC0: 2.001 {{: |
- | * output to ADC1: 0.667 {{: | + | * output to ADC1: 0.667 {{: |
According to equation 13.1, the expected BPM result for the chosen input amplitudes is 0.5. | According to equation 13.1, the expected BPM result for the chosen input amplitudes is 0.5. | ||
Line 2248: | Line 1738: | ||
The test was run for 12 hours, during which the scopes were read continuously and histograms of the occurring results were created. | The test was run for 12 hours, during which the scopes were read continuously and histograms of the occurring results were created. | ||
- | {{: | + | {{: |
- | **Figure | + | **Figure |
The gateware was configured as follows: | The gateware was configured as follows: | ||
Line 2259: | Line 1749: | ||
* number of samples per capture: 1024 | * number of samples per capture: 1024 | ||
- | {{: | + | {{: |
- | **Figure | + | **Figure |
The resulting histograms do not show Gaussian distributions, | The resulting histograms do not show Gaussian distributions, | ||
- | ====== 14 System limitations ====== | + | ===== 10 References ===== |
- | + | ||
- | ===== 14.1 CPU unit reboot ===== | + | |
- | + | ||
- | The FPGA bitstreams have to be loaded before the CPU unit boots, otherwise the PCIe driver will not detect the FPGAs. This can be ensured by setting suitable power up delays in the MCH (see [[# | + | |
- | + | ||
- | Whenever a new bitstream is loaded to a FPGA, the CPU unit has to rebooted either via its Hot Swap Handle or remotely via the MCH (see [[# | + | |
- | + | ||
- | ====== 15 References | + | |
[1] A. Reiter, R. Singh: Comparison of beam position calculation methods for application in digital acquisition systems. //Nuclear Instruments and Methods in Physics Research Section A: Accelerators, | [1] A. Reiter, R. Singh: Comparison of beam position calculation methods for application in digital acquisition systems. //Nuclear Instruments and Methods in Physics Research Section A: Accelerators, | ||
Line 2297: | Line 1779: | ||
[11] AFC v3.1 schematics, https:// | [11] AFC v3.1 schematics, https:// | ||
- | [12] NXP: LPCxpresso download web page, https:// | + | [12] FMC ADC 250M 16B 4ch v1.2 schematics, https:// |
- | + | ||
- | [13] NXP: AMC documentation, | + | |
- | + | ||
- | [14] NAT GmbH: MCH technical reference manual, https:// | + | |
- | + | ||
- | [15] Micron: FBGA and Component Marking Decoder, https:// | + | |
- | + | ||
- | [16] FMC ADC 250M 16B 4ch v1.2 schematics, https:// | + | |
- | + | ||
- | [17] FMC ADC 250M 16B 4ch v2.3 schematics, https:// | + | |
- | [18] Anaren: Balun BD0205F5050A00 datasheet, https://git.gsi.de/BEA_HDL/ | + | [13] FMC ADC 250M 16B 4ch v2.3 schematics, https://github.com/lnls-dig/fmc250-hw/ |
+ | [14] Anaren: Balun BD0205F5050A00 datasheet, https:// |