This shows you the differences between two versions of the page.
Both sides previous revision Previous revision | |||
ds:projects:unimon:gateware [2023/01/24 13:40] rgeissler |
ds:projects:unimon:gateware [2023/02/09 10:10] (current) rgeissler |
||
---|---|---|---|
Line 6: | Line 6: | ||
The documentation is also available as a PDF: {{: | The documentation is also available as a PDF: {{: | ||
- | Up to date DokuWiki and PDF versions | + | An up to date PDF version |
+ | |||
+ | Documentation that is common to multiple FPGA based projects can be found here: [[: | ||
+ | It is also available as a PDF: {{: | ||
===== Resources ===== | ===== Resources ===== | ||
Line 26: | Line 29: | ||
This document describes the gateware (= FPGA firmware) implementation of the UniMon Tank Signal Monitor. | This document describes the gateware (= FPGA firmware) implementation of the UniMon Tank Signal Monitor. | ||
- | The gateware | + | The system uses 4 IOxOS ADC_3117 FMC cards which are mounted on two OpenHardware AFC v4.0 FMC carriers. Each of the FMCs provides 20 ADCs with a resolution of 16 bits. The sampling rate of the ADCs is adjustable between 1 kHz and 5 MHz. Each ADC input is individually configurable as a differential or single ended and different gain settings are available. |
+ | |||
+ | {{: | ||
+ | |||
+ | **Figure 1.1:** UniMon crate in the HKR | ||
+ | |||
+ | The gateware of each AFC stores the data stream of respectively 40 ADCs to a ring buffer, from which it can be read via PCIe. | ||
+ | |||
+ | ===== 2 Common FPGA based projects documentation ===== | ||
+ | |||
+ | This project incorporates the code from the // | ||
+ | |||
+ | ==== 2.1 Common monitoring and control features ==== | ||
+ | |||
+ | Documentation about the register bank, the architecture information storage and the observer can be found here:\\ | ||
+ | https:// | ||
+ | |||
+ | ==== 2.2 FPGA Observer ==== | ||
+ | |||
+ | There is a expert GUI that can be used together with multiple projects: | ||
+ | https:// | ||
+ | |||
+ | ==== 2.3 Build flow and simulation ==== | ||
+ | |||
+ | You can find instructions on how to build and simulate | ||
+ | https://git.gsi.de/ | ||
+ | |||
+ | ==== 2.4 Helper scripts ==== | ||
+ | |||
+ | You can find usefull scripts here:\\ | ||
+ | https:// | ||
- | The system uses a total of 4 IOxOS ADC_3117 FMC cards. Each of them provides 20 ADCs with a maximum sampling rate of 5 MHz and a resolution of 16 bits. | + | ==== 2.5 Continuous integration environment ==== |
- | There are two AMC FMC carriers which carry respectively two FMC cards. The gateware of each AMC card stores | + | Information about the continuous integration setup can be found here:\\ |
+ | https:// | ||
- | The sampling rate of the ADCs is adjustable between 1 kHz and 5 MHz. Due to the parallel storage of the data streams in a ring buffer, the sampling rates of the all the ADCs mounted on a single AMC card must beequal. | + | ==== 2.6 Programming |
- | Each ADC input is individually configurable as differential (two wires) or as single ended relative | + | You can find instructions on how to program |
+ | https://git.gsi.de/ | ||
- | ===== 2 Peripheral devices ===== | + | ===== 3 Peripheral devices ===== |
The gateware communicates with the following devices on each of the IOxOS ADC_3117 FMC cards: | The gateware communicates with the following devices on each of the IOxOS ADC_3117 FMC cards: | ||
Line 43: | Line 78: | ||
* 10 * EXAR XRA1405 GPIO expander | * 10 * EXAR XRA1405 GPIO expander | ||
- | ==== 2.1 Analog Devices LTC2323-16 ADC ==== | + | ==== 3.1 Analog Devices LTC2323-16 ADC ==== |
Each LTC2323-16 consists of two ADCs that are sampled in parallel. | Each LTC2323-16 consists of two ADCs that are sampled in parallel. | ||
Line 51: | Line 86: | ||
Each ADC provides a differential input pair and is directly controlled by the FPGA without any external clock. | Each ADC provides a differential input pair and is directly controlled by the FPGA without any external clock. | ||
- | The FPGA has to read the samples at a clock speed of 105 MHz. The sampling rate of the ADCs is controlled by the distance between the so called conversion pulses that have to be generated by the FPGA [[#12_references|[1]]]. | + | The FPGA has to read the samples at a clock speed of 105 MHz. The sampling rate of the ADCs is controlled by the distance between the so called conversion pulses that have to be generated by the FPGA [[#6_references|[1]]]. |
- | Due to this clock generation logic of the ADCs, the sampling frequency can be set with a certain granularity described in chapter [[# | + | Due to this clock generation logic of the ADCs, the sampling frequency can be set with a certain granularity described in chapter [[# |
- | ==== 2.2 EXAR XRA1405 GPIO expander ==== | + | ==== 3.2 EXAR XRA1405 GPIO expander ==== |
The GPIO expanders are used for multiple purposes. Among these is the control of the inputs to each ADC. | The GPIO expanders are used for multiple purposes. Among these is the control of the inputs to each ADC. | ||
- | === 2.2.1 ADC input selection === | + | === 3.2.1 ADC input selection === |
There are dedicated switch devices in front of each of the differential ADC inputs, which can be controlled independently. | There are dedicated switch devices in front of each of the differential ADC inputs, which can be controlled independently. | ||
Line 70: | Line 105: | ||
* fixed calibration voltage of + 4.128 V | * fixed calibration voltage of + 4.128 V | ||
- | === 2.2.2 Gain selection === | + | === 3.2.2 Gain selection === |
The gain of each differential ADC input pair can be controlled by a dedicated fixed gain amplifier device which can be programmed to an amplifications of 1, 2, 5 or 10. | The gain of each differential ADC input pair can be controlled by a dedicated fixed gain amplifier device which can be programmed to an amplifications of 1, 2, 5 or 10. | ||
Line 76: | Line 111: | ||
In front of each ADC input there is an attenuator with a fixed gain of 0.4. | In front of each ADC input there is an attenuator with a fixed gain of 0.4. | ||
- | In total the following gains are available: | + | In total the following gains are available |
^ amplifier gain ^ input range ^ resolution per LSB ^ | ^ amplifier gain ^ input range ^ resolution per LSB ^ | ||
Line 84: | Line 119: | ||
| 10 | +/- 1.024 V | 31.25 {{: | | 10 | +/- 1.024 V | 31.25 {{: | ||
- | ===== 3 Gateware implementation ===== | + | ===== 4 Gateware implementation ===== |
- | ==== 3.1 Clocking ==== | + | ==== 4.1 Clocking ==== |
The gateware uses only one primary clock. | The gateware uses only one primary clock. | ||
- | === 3.1.1 PCIe reference clock === | + | === 4.1.1 PCIe reference clock === |
The PCIe reference clock has a nominal frequency of 100 MHz and feeds the reference clock input of the PCIe IP core by Xilinx, which contains a PLL producing a 125 MHz output clock for the AXI interface named // | The PCIe reference clock has a nominal frequency of 100 MHz and feeds the reference clock input of the PCIe IP core by Xilinx, which contains a PLL producing a 125 MHz output clock for the AXI interface named // | ||
Line 102: | Line 137: | ||
The SDRAM interface IP core contains a MMCM which generates a 100 MHz clock named // | The SDRAM interface IP core contains a MMCM which generates a 100 MHz clock named // | ||
- | ==== 3.2 Resets ==== | + | ==== 4.2 Resets ==== |
- | === 3.2.1 PLL not in lock === | + | === 4.2.1 PLL not in lock === |
As long as the PLL in the MMCM producing the main processing clock //clk_100// is not yet in lock, the design is held in reset. After this the lock should be stable until the next power cycle. | As long as the PLL in the MMCM producing the main processing clock //clk_100// is not yet in lock, the design is held in reset. After this the lock should be stable until the next power cycle. | ||
- | === 3.2.2 Reset button === | + | === 4.2.2 PCIe reset === |
+ | |||
+ | The design will be reset whenever the PCIe connection is re-initialized. This happens e.g. when the FEC is rebooted. | ||
+ | |||
+ | === 4.2.3 Reset button === | ||
There is a push button labeled //RST// at the center of the AFC front panel which is connected to the microcontroller for the MMC firmware. The OpenMMC firmware forwards a button press with a duration of at least two seconds to the FPGA pin AG26 as an active low signal to initiate a reset of the gateware. | There is a push button labeled //RST// at the center of the AFC front panel which is connected to the microcontroller for the MMC firmware. The OpenMMC firmware forwards a button press with a duration of at least two seconds to the FPGA pin AG26 as an active low signal to initiate a reset of the gateware. | ||
- | Resetting the FPGA leads to the loss of the PCIe connection. To re-enable the connection, the CPU unit has to be power cycled. | + | Resetting the FPGA leads to the loss of the PCIe connection. To re-enable the connection, the FEC has to be rebooted. |
- | ==== 3.3 Data flow diagram ==== | + | ==== 4.3 Data flow diagram ==== |
{{: | {{: | ||
- | **Figure | + | **Figure |
- | Figure [[# | + | Figure [[# |
* processing clocks and clock domain crossings | * processing clocks and clock domain crossings | ||
Line 128: | Line 167: | ||
* data width conversions of AXI connections | * data width conversions of AXI connections | ||
- | ==== 3.4 PCIe Interface ==== | + | ==== 4.4 PCIe Interface ==== |
This gateware uses the Xilinx IP core // | This gateware uses the Xilinx IP core // | ||
Line 138: | Line 177: | ||
The PCIe reference clock is routed to the FPGA via the MMC firmware and is configured to be driven by the 100 MHz //FCLKA// clock coming from the AMC connector. | The PCIe reference clock is routed to the FPGA via the MMC firmware and is configured to be driven by the 100 MHz //FCLKA// clock coming from the AMC connector. | ||
- | ==== 3.5 SDRAM interface ==== | + | ==== 4.5 SDRAM interface ==== |
For the communication with the SDRAM, an IP core by Xilinx is used. The clock frequency of the SDRAM interface’s AXI bus is 100 MHz, but has a different source so that an AXI clock converter is used to connect it to the rest of the AXI infrastructure which is clocked at the main processing clock of 100 MHz. | For the communication with the SDRAM, an IP core by Xilinx is used. The clock frequency of the SDRAM interface’s AXI bus is 100 MHz, but has a different source so that an AXI clock converter is used to connect it to the rest of the AXI infrastructure which is clocked at the main processing clock of 100 MHz. | ||
- | ===== 4 Build flow and simulation ===== | + | ==== 4.6 Observer |
- | The build flow is designed and tested to be run on a Linux operation system. The bitstream generation should also work on a Windows installation, | + | This project incorporates the observer interface from the //FPGA_Common// Git submodule (see https://git.gsi.de/ |
- | ==== 4.1 Prerequisites ==== | + | The following signals are connected to the observer inputs: |
- | An installation | + | ^ |
+ | | '' | ||
+ | | '' | ||
+ | | '' | ||
+ | | '' | ||
+ | | '' | ||
+ | | '' | ||
+ | | '' | ||
+ | | '' | ||
+ | | '' | ||
+ | | '' | ||
+ | | '' | ||
- | ==== 4.2 Build flow ==== | + | The rest of the multiplexer inputs are connected |
- | + | ||
- | === 4.2.1 Scripted build flow === | + | |
- | + | ||
- | For a completely automatic script based build flow without using the Vivado GUI proceed as follows: | + | |
- | + | ||
- | * navigate to the root folder of the repository in a terminal | + | |
- | * type '' | + | |
- | + | ||
- | A project will be generated in the folder '' | + | |
- | + | ||
- | === 4.2.2 Vivado GUI based build flow === | + | |
- | + | ||
- | == Via a Bash script == | + | |
- | + | ||
- | * navigate to the root folder of the repository in a terminal | + | |
- | * type '' | + | |
- | + | ||
- | This will open the Vivado GUI and set up a project, which can take some minutes. The project will be generated in the folder '' | + | |
- | + | ||
- | == Via the Vivado GUI == | + | |
- | + | ||
- | If you intend to use the Vivado GUI itself to set up the project proceed as follows: | + | |
- | + | ||
- | * open Vivado | + | |
- | * use the TCL console in the bottom of the GUI to navigate to '' | + | |
- | * type '' | + | |
- | + | ||
- | ==== 4.3 Simulation ==== | + | |
- | + | ||
- | === 4.3.1 Vivado GUI based simulation === | + | |
- | + | ||
- | Prerequisite: | + | |
- | + | ||
- | Click on //Run Simulation// | + | |
- | + | ||
- | === 4.3.2 Scripted simulation === | + | |
- | + | ||
- | The scripted simulation checks that the simulation results match a predefined reference pattern. | + | |
- | + | ||
- | * navigate to the root folder of the repository in a terminal | + | |
- | * type '' | + | |
- | * you will find the output files of the simulation in the folder '' | + | |
- | + | ||
- | === 4.3.3 Peripherals simulation models === | + | |
- | + | ||
- | The toplevel simulation includes a Verilog simulation model from Micron, the manufacturer of the AFC’s SDRAM, which allows the simulation of the behavior of the external SDRAM. | + | |
- | + | ||
- | The SDRAM interface IP needs an initial calibration process which finishes after about 120 us. If the communication to the SDRAM is of interest | + | |
===== 5 Gateware software interface ===== | ===== 5 Gateware software interface ===== | ||
Line 206: | Line 208: | ||
The following mapping is applied: | The following mapping is applied: | ||
- | ^ | + | ^ |
- | | 0x00000000| | + | | |
- | | 0x00004000| | + | | |
- | | 0x80000000| | + | | |
+ | | '' | ||
**Table 5.1:** Memory mapping | **Table 5.1:** Memory mapping | ||
+ | |||
+ | The // | ||
+ | https:// | ||
==== 5.1 PCIe Driver ==== | ==== 5.1 PCIe Driver ==== | ||
Line 221: | Line 227: | ||
* '' | * '' | ||
- | === 5.1.1 Reading a register === | + | === 5.1.1 Reading |
Example in C: | Example in C: | ||
Line 232: | Line 238: | ||
read(fd, &value, sizeof(uint64_t)); | read(fd, &value, sizeof(uint64_t)); | ||
</ | </ | ||
- | === 5.1.2 Writing a register === | + | === 5.1.2 Writing |
It is important to write the whole register width of 64 bits. If a register has less than 64 bits, the unused MSBs have to be written to any value. 32 bit write accesses will not have any effect. | It is important to write the whole register width of 64 bits. If a register has less than 64 bits, the unused MSBs have to be written to any value. 32 bit write accesses will not have any effect. | ||
Line 286: | Line 292: | ||
The following status registers can be read by software: | The following status registers can be read by software: | ||
- | ^ index^ | + | ^ index^ |
- | | '' | + | | '' |
- | | '' | + | | '' |
- | | …| | + | | …| |
- | | '' | + | | '' |
- | | '' | + | | '' |
- | | '' | + | | '' |
- | | '' | + | | '' |
- | | '' | + | | '' |
- | | '' | + | | '' |
+ | | '' | ||
Line 307: | Line 314: | ||
Address where the next data sample will be stored during the scope’s capturing process. The recently stored ADC data can be found below this address, or - in case of a ringbuffer wraparound - also below the end of the memory region. | Address where the next data sample will be stored during the scope’s capturing process. The recently stored ADC data can be found below this address, or - in case of a ringbuffer wraparound - also below the end of the memory region. | ||
+ | |||
+ | == 111: SDRAM initial calibration complete == | ||
+ | |||
+ | The communication to the SDRAM is controlled by an IP core by Xilinx which performs a timing calibration at start up. The value of this register will be '' | ||
== 124: build timestamp == | == 124: build timestamp == | ||
- | Time when the bitstream was created. This information can be used to identify the gateware version (together with the Git commit information documented in chapter [[# | + | Time when the bitstream was created. This information can be used to identify the gateware version (together with the Git commit information documented in https://git.gsi.de/ |
Format: | Format: | ||
Line 323: | Line 334: | ||
== 126: module ID == | == 126: module ID == | ||
- | The module ID can be used to identify the type of the current bitstream. The module ID of the UniMon gateware is '' | + | The module ID can be used to identify the type of the current bitstream.\\ |
+ | The module ID of the UniMon gateware is '' | ||
The fields are defined as follows: | The fields are defined as follows: | ||
Line 397: | Line 409: | ||
The following gains are available: | The following gains are available: | ||
- | ^ value^ | + | ^ value^ |
- | | 0| | + | | '' |
- | | 1| | + | | '' |
- | | 2| | + | | '' |
- | | 3| 10| +/- 1.024 V| 31.25 {{: | + | | '' |
This register holds the concatenation of the gain settings of the FMC’s 20 ADCs: | This register holds the concatenation of the gain settings of the FMC’s 20 ADCs: | ||
Line 417: | Line 429: | ||
The following input settings are available: | The following input settings are available: | ||
- | ^ value^input | + | ^ value^input |
- | | 0|differential p / n input pin | | + | | '' |
- | | 1|ground | + | | '' |
- | | 2|not implemented | + | | '' |
- | | 3|calibration voltage of +4.128 V | | + | | '' |
This register holds the concatenation of the input settings of the FMC’s 20 ADCs: | This register holds the concatenation of the input settings of the FMC’s 20 ADCs: | ||
Line 444: | Line 456: | ||
The reset will be automatically lifted so that the register does not have to be written to 0 after initiating a reset. | The reset will be automatically lifted so that the register does not have to be written to 0 after initiating a reset. | ||
- | ===== 6 Extended gateware software interface ===== | + | ===== 6 References ===== |
- | + | ||
- | Besides the interface documented in chapter [[# | + | |
- | + | ||
- | While the productive interface is intended to be kept as downward compatible as possible, the extended interface may be subject to major changes during the development process. | + | |
- | + | ||
- | ==== 6.1 Extended register map ==== | + | |
- | + | ||
- | === 6.1.1 Additional status registers === | + | |
- | + | ||
- | The following additional status registers can be read by software: | + | |
- | + | ||
- | ^ index^ | + | |
- | | '' | + | |
- | + | ||
- | + | ||
- | **Table 6.1:** List of additional status registers | + | |
- | + | ||
- | == 111: SDRAM initial calibration complete == | + | |
- | + | ||
- | The communication to the SDRAM is controlled by an IP core by Xilinx which performs a timing calibration at start up. The value of this register will be 1 after completion of the initial calibration. | + | |
- | + | ||
- | === 6.1.2 Additional configuration registers === | + | |
- | + | ||
- | The following additional registers can be written by software: | + | |
- | + | ||
- | ^ index^ | + | |
- | | '' | + | |
- | | '' | + | |
- | | '' | + | |
- | | '' | + | |
- | | '' | + | |
- | | '' | + | |
- | | '' | + | |
- | | '' | + | |
- | | '' | + | |
- | + | ||
- | + | ||
- | **Table 6.2:** List of additional configuration registers | + | |
- | + | ||
- | == 112: Observer ignore valid == | + | |
- | + | ||
- | If set to 1, the valid signal connected to the observer multiplexer input will be ignored and the data will be sampled at every clock cycle. | + | |
- | + | ||
- | == 113, 114: Observer multiplexer {0, 1} select == | + | |
- | + | ||
- | The observer stores samples that are 128 bits wide, which consist of two concatenated 64 bits wide multiplexer outputs. Each multiplexer can choose between 16 different input vectors. Like this, each signal can be observed in parallel to any other signal. | + | |
- | + | ||
- | ^ value^input vector(64 bits) | + | |
- | | 0|corrected ADC data of ADCs 0 - 3 |corrected ADC data valid (0) | | + | |
- | | 1|corrected ADC data of ADCs 4 - 7 |corrected ADC data valid (0) | | + | |
- | | 2|corrected ADC data of ADCs 8 - 11 | + | |
- | | 3|corrected ADC data of ADCs 12 - 15 |corrected ADC data valid (0) | | + | |
- | | 4|corrected ADC data of ADCs 16 - 19 |corrected ADC data valid (0) | | + | |
- | | 5|corrected ADC data of ADCs 20 - 23 |corrected ADC data valid (1) | | + | |
- | | 6|corrected ADC data of ADCs 24 - 27 |corrected ADC data valid (1) | | + | |
- | | 7|corrected ADC data of ADCs 28 - 31 |corrected ADC data valid (1) | | + | |
- | | 8|corrected ADC data of ADCs 32 - 35 |corrected ADC data valid (1) | | + | |
- | | 9|corrected ADC data of ADCs 36 - 39 |corrected ADC data valid (1) | | + | |
- | | | + | |
- | + | ||
- | The rest of the multiplexer inputs are connected to zero. | + | |
- | + | ||
- | == 115: Observer trigger select == | + | |
- | + | ||
- | Analog to register 113 and 114. Determines on which observer input vector the trigger listens. | + | |
- | + | ||
- | == 116: Observer trigger compare vector (t = 0) == | + | |
- | + | ||
- | 64 bit wide compare vector that is compared with the observer input vector determined by register 115 ’observer trigger select’. If the two patterns match, the next sample will be compared to the compare vector determined by register 117: ’trigger compare vector (t = 1)’. | + | |
- | + | ||
- | == 117: Observer trigger compare vector (t = 1) == | + | |
- | + | ||
- | See above. If the patterns do not match, the next sample will be compared to the compare vector determined by register 116: ’trigger compare vector (t = 0)’. If the patterns match the data acquisition is triggered. | + | |
- | + | ||
- | == 118: Observer trigger compare bit mask == | + | |
- | + | ||
- | Determines which bits of the input vector shall be compared with that of the compare vectors. Valid for both trigger compare vectors (registers 116 and 117). For triggering, the patterns must match for all bits whose bit mask is set to 1. | + | |
- | + | ||
- | == 119: Observer capture == | + | |
- | + | ||
- | Starts the comparing process. Data is captured if the patterns defined by the three previous registers match. | + | |
- | + | ||
- | == 120: Observer cancel == | + | |
- | + | ||
- | Stops the comparing process. | + | |
- | + | ||
- | ==== 6.2 Architecture information storage ==== | + | |
- | + | ||
- | The Block RAM (see memory mapping, table [[# | + | |
- | + | ||
- | === 6.2.1 Register information === | + | |
- | + | ||
- | Information about the 128 status registers and the 128 configuration registers is stored in the first half of the Block RAM. Following information is stored for every register: | + | |
- | + | ||
- | * name of register (31 bytes) | + | |
- | * number of bits (1 byte) | + | |
- | + | ||
- | ^ | + | |
- | | '' | + | |
- | | '' | + | |
- | | | + | |
- | | '' | + | |
- | | '' | + | |
- | | '' | + | |
- | | | + | |
- | | '' | + | |
- | + | ||
- | + | ||
- | **Table 6.3:** Register information storage formats | + | |
- | + | ||
- | Table [[# | + | |
- | + | ||
- | The register information is used by the FPGA Observer software to display the registers in the Register Access tab (see chapter [[# | + | |
- | + | ||
- | === 6.2.2 Gateware information === | + | |
- | + | ||
- | The address range from 0x00006000 to 0x00006FFF is used to store information about the gateware version. The information is stored as an ASCII string of variable length (maximum 4 kiB), which is assembled from information from the Git repository. It contains the URL of the remote server of the Git repository, the latest commit hash and the latest commit date. | + | |
- | + | ||
- | The gateware information is used by the FPGA Observer software to display the information in the Gateware Information tab (see chapter [[# | + | |
- | + | ||
- | === 6.2.3 Observer signal information === | + | |
- | + | ||
- | The address range from 0x00007000 to 0x00007FFF is used to store information about the signals connected to the 16 observer multiplexer inputs. Following 32 bytes wide information is stored for every signal connected to each of the 16 64 bits wide multiplexer inputs: | + | |
- | + | ||
- | * name of signal (bytes 0 to 28) | + | |
- | * number of bits (the 6 LSBs of byte 29) | + | |
- | * bit offset inside the 64 bits wide observer input vector (the 6 LSBs of byte 30) | + | |
- | * display type of signal (the 4 LSBs of Byte 31) | + | |
- | * observer input number (the 4 MSBs of Byte 31) | + | |
- | + | ||
- | The used storage size depends on the number of signals connected to the observer multiplexer inputs. If the field //number of bits// of one entry reads 0, it indicates that the corresponding 32 byte information is not used. | + | |
- | + | ||
- | The coding of the display type is the following: | + | |
- | + | ||
- | ^ value^display type ^ | + | |
- | | 0|unsigned analog | + | |
- | | 1|signed analog | + | |
- | | 2|unsigned int | | + | |
- | | 3|signed int | | + | |
- | | 4|hexadecimal | + | |
- | + | ||
- | The names are stored as ASCII strings. If a name is shorter than 29 bytes, the remaining bytes are filled with Null characters. | + | |
- | + | ||
- | The observer signal information is used by the FPGA Observer software to display the //Observer Trigger// tab and the // | + | |
- | + | ||
- | ===== 7 Test software ===== | + | |
- | + | ||
- | ==== 7.1 FPGA Observer ==== | + | |
- | + | ||
- | There is a graphical test software intended to be run on the CPU unit. It is implemented in Python using the GTK 3 GUI toolkit. | + | |
- | + | ||
- | === 7.1.1 Installation and usage === | + | |
- | + | ||
- | The sources and an installation script can be found under '' | + | |
- | + | ||
- | == Installation == | + | |
- | + | ||
- | Connect to the CPU unit e.g. via ssh. Clone the // | + | |
- | + | ||
- | '' | + | |
- | + | ||
- | Install the PCIe driver: | + | |
- | + | ||
- | '' | + | |
- | '' | + | |
- | + | ||
- | Install the FPGA Observer software: | + | |
- | + | ||
- | '' | + | |
- | '' | + | |
- | + | ||
- | == Usage == | + | |
- | + | ||
- | For the PCIe driver to work, the bitstreams of the FPGAs have to be loaded before powering the CPU unit. If that is not the case, power cycle the CPU unit by pulling out the Hot Swap Handle and pushing it in again. | + | |
- | + | ||
- | Connect to the CPU unit e.g. via '' | + | |
- | + | ||
- | '' | + | |
- | + | ||
- | A GUI should open and a choice of FPGA serial numbers should be displayed on the upper left corner. If the list is empty, either the loading of the FPGAs finished after powering the CPU unit or the PCIe driver did not install correctly. The FPGA serial numbers can be used to identify the AMC board you like to access. Choose a serial number and click // | + | |
- | + | ||
- | === 7.1.2 Register Access tab === | + | |
- | + | ||
- | {{: | + | |
- | + | ||
- | **Figure 7.1:** FPGA Observer - Register Access tab | + | |
- | + | ||
- | The names and widths of the registers are read from an information memory region in the FPGA (see chapter [[# | + | |
- | + | ||
- | The //read// button reads all the status registers either once or continuously if the // | + | |
- | + | ||
- | === 7.1.3 Scope tab === | + | |
- | + | ||
- | {{: | + | |
- | + | ||
- | **Figure 7.2:** FPGA Observer - Scope tab | + | |
- | + | ||
- | The scope tab displays the signals of all the ADCs that are mounted on a single AFC carrier. | + | |
- | + | ||
- | For this purpose the write pointer of the ring buffer is polled and the required number of samples are read from positions before this pointer. | + | |
- | + | ||
- | === 7.1.4 Gateware Information tab === | + | |
- | + | ||
- | {{: | + | |
- | + | ||
- | **Figure 7.3:** FPGA Observer - Gateware Information tab | + | |
- | + | ||
- | The URL of the remote server of the Git repository, the latest commit hash and the latest commit date are read from an information memory region in the FPGA (see chapter [[# | + | |
- | + | ||
- | The bitstream generation date is read from status register 126 ’build timestamp’. | + | |
- | + | ||
- | ==== 7.2 Test scripts ==== | + | |
- | + | ||
- | === 7.2.1 PCIe access test script === | + | |
- | + | ||
- | There is a PCIe access test script '' | + | |
- | + | ||
- | ===== 8 Helper scripts ===== | + | |
- | + | ||
- | ==== 8.1 VHDL beautification ==== | + | |
- | + | ||
- | There is a script '' | + | |
- | + | ||
- | The script expects one parameter: '' | + | |
- | + | ||
- | The script applies several corrections and changes to the //Emacs// formatting result: | + | |
- | + | ||
- | * correction of the handling of the comparison operator '' | + | |
- | * correction of the handling of initializations like '' | + | |
- | * enforcing of spaces around the operators '' | + | |
- | * no indentation for closing brackets | + | |
- | * aligning of full comment lines to the indentation level of the following VHDL command | + | |
- | * indentation with tabs instead of spaces | + | |
- | + | ||
- | ==== 8.2 Remote power cycling of the CPU unit ==== | + | |
- | + | ||
- | Whenever the bitstream of an FPGA is reloaded, the CPU unit has to be rebooted via its Hot Swap Handle in order to establish a PCIe connection. A software reboot does not work. | + | |
- | + | ||
- | An alternative possibility of remote power cycling the CPU unit is via the MCH. | + | |
- | + | ||
- | The script '' | + | |
- | + | ||
- | The script takes about 60 seconds to complete. | + | |
- | + | ||
- | ==== 8.3 Generation of a VHDL file for monitoring and control ==== | + | |
- | + | ||
- | The monitoring and control configuration of the gateware is defined by the configuration files '' | + | |
- | + | ||
- | The script '' | + | |
- | + | ||
- | The script is also executed by the gateware build flow documented in chapter [[# | + | |
- | + | ||
- | ==== 8.4 Generation of documentation ==== | + | |
- | + | ||
- | === 8.4.1 PDF === | + | |
- | + | ||
- | There is a script '' | + | |
- | + | ||
- | === 8.4.2 Markdown === | + | |
- | + | ||
- | There is a script '' | + | |
- | + | ||
- | The version 2.10 of //Pandoc// is used here (https:// | + | |
- | + | ||
- | The result of //Pandoc// is postprocessed for multiple reasons: | + | |
- | + | ||
- | * conversion of the math syntax to Gitlab’s .md format | + | |
- | * corrections of the bibliography, | + | |
- | * corrections of the references to figures, tables and equations | + | |
- | * enumeration of chapters, sections and subsections | + | |
- | * adding of captions for figures and equations | + | |
- | * enumeration of figures, tables and equations | + | |
- | * generation of a table of contents | + | |
- | * implementation of citations | + | |
- | + | ||
- | Additional documentation which is not included in the Latex sources is appended from the file '' | + | |
- | + | ||
- | === 8.4.3 DokuWiki === | + | |
- | + | ||
- | There is a script '' | + | |
- | + | ||
- | The script converts the Latex documentation sources to the DokuWiki format in four steps: | + | |
- | + | ||
- | * conversion of Latex sources to Markdown | + | |
- | * preprocessing of Markdown before the conversion to DokuWiki | + | |
- | * calling //Pandoc// to convert Markdown to DokuWiki | + | |
- | * postprocessing for correction, extension of functionality and a different style | + | |
- | + | ||
- | The preprocessing actions are: | + | |
- | + | ||
- | * removing table of contents since it is automatically generated by DokuWiki | + | |
- | + | ||
- | The postprocessing actions are: | + | |
- | + | ||
- | * conversion or equations to images since DokuWiki can not render equations | + | |
- | * replacement of HTML tags, Latex color tags, etc. since DokuWiki can not handle them | + | |
- | * conversion of the reference format | + | |
- | + | ||
- | ===== 9 Continuous integration environment ===== | + | |
- | + | ||
- | There is a continuous integration environment setup for the // | + | |
- | + | ||
- | At the moment the Gitlab Runner is running on the Linux server //sdlx035// located in a server room in the basement. | + | |
- | + | ||
- | The benefits of continuous integration are: | + | |
- | + | ||
- | * every change will be tested automatically | + | |
- | * it is ensured that no files are missing in the repository | + | |
- | * the master branch can be kept functional at any time | + | |
- | * build results like e.g. bitstreams are automatically generated and can be archived | + | |
- | + | ||
- | ==== 9.1 Installation ==== | + | |
- | + | ||
- | There is an installation script '' | + | |
- | + | ||
- | After the installation, | + | |
- | + | ||
- | On the newly installed Gitlab Runner server, open a terminal and type '' | + | |
- | + | ||
- | Enter the following information: | + | |
- | + | ||
- | * gitlab-ci coordinator URL: e.g. '' | + | |
- | * gitlab-ci token: enter the registration token copied before | + | |
- | * gitlab-ci description: | + | |
- | * gitlab-ci tags: leave empty | + | |
- | * executor: '' | + | |
- | + | ||
- | You can add multiple repositories with different tokens by running '' | + | |
- | + | ||
- | ==== 9.2 Pipeline Stages ==== | + | |
- | + | ||
- | {{: | + | |
- | + | ||
- | **Figure 9.1:** Gitlab: continuous integration pipelines | + | |
- | + | ||
- | Each '' | + | |
- | + | ||
- | The following pipeline stages are defined: | + | |
- | + | ||
- | * documentation | + | |
- | * simulation | + | |
- | * FPGA build | + | |
- | + | ||
- | {{: | + | |
- | + | ||
- | **Figure 9.2:** Gitlab: Pipeline stages | + | |
- | + | ||
- | === 9.2.1 Documentation === | + | |
- | + | ||
- | The script '' | + | |
- | + | ||
- | The log file of Pdflatex and - if successful - the PDF of the documentation are archived. | + | |
- | + | ||
- | === 9.2.2 Simulation === | + | |
- | + | ||
- | The script '' | + | |
- | + | ||
- | The log file of the simulation and - if successful - a file with the output from the simulation are archived. | + | |
- | + | ||
- | === 9.2.3 FPGA build === | + | |
- | + | ||
- | The script '' | + | |
- | + | ||
- | Different log files from synthesis and implementation, | + | |
- | + | ||
- | {{: | + | |
- | + | ||
- | **Figure 9.3:** Gitlab: Pipeline progress console | + | |
- | + | ||
- | === 9.2.4 Timing check === | + | |
- | + | ||
- | The script '' | + | |
- | + | ||
- | Since timing failures do not necessarily result in system malfunctions, | + | |
- | + | ||
- | ==== 9.3 Build results ==== | + | |
- | + | ||
- | For each of the pipeline stages the archiving of build results can be configured for an adjustable time period, which is set to one week. If the period has passed and the build results have been deleted, they can be generated again by restarting the pipeline. | + | |
- | + | ||
- | The build results can be downloaded from the Gitlab web front end where they are called //job artifacts// (see figure [[# | + | |
- | + | ||
- | <color green> | + | |
- | + | ||
- | ==== 9.4 Settings ==== | + | |
- | + | ||
- | You can define individual settings for the CI/CD section of each Git repository in the Gitlab web front end. The following settings should fit for most cases: | + | |
- | + | ||
- | * Use git clone to get the recent application code, otherwise the pipelines might fail during git fetch: //Settings CI/CD General pipelines Git strategy for pipelines: git clone// | + | |
- | * Increase the timeout to allow FPGA build to finish in any case: //Settings CI/CD General pipelines Timeout: 6h// | + | |
- | + | ||
- | ===== 10 Programming and hardware configuration ===== | + | |
- | + | ||
- | ==== 10.1 Programming the gateware ==== | + | |
- | + | ||
- | === 10.1.1 Using a JTAG Switch Module === | + | |
- | + | ||
- | ==== 10.2 Configuration of the MCH ==== | + | |
- | + | ||
- | === 10.2.1 Via the MCH’s web interface === | + | |
- | + | ||
- | == Base configuration == | + | |
- | + | ||
- | //MCH global parameter SSH access: enabled// | + | |
- | This will trigger SSH key generation which takes some minutes to complete. | + | |
- | + | ||
- | //PCIe parameter Upstream slot power up delay: 5 sec//\\ | + | |
- | Delay before the CPU unit will power up on start up. For making sure that the bitstreams are loaded to the AMC’s FPGAs from Flash memory before the CPU unit boots you might have to increase this value. | + | |
- | + | ||
- | //PCIe parameter PCIe hot plug delay for AMCs: 0 sec//\\ | + | |
- | Delay before the AMC boards will power up on start up. | + | |
- | + | ||
- | == Switch PCIe x80 == | + | |
- | + | ||
- | Set the CPU-Unit as upstream AMC source in ’Virtual Switch 0’: | + | |
- | + | ||
- | //PCIe Virtual Switches Upstream AMC: AMC1/ | + | |
- | (for CPU unit in AMC slot 1) | + | |
- | + | ||
- | === 10.2.2 Via USB === | + | |
- | + | ||
- | The most comfortable way of configuring the MCH is via its web interface. If you have accidentally disabled the webserver, set an invalid IP or DHCP configuration or reset the MCH settings to default, you can access the MCH via an USB connection to the micro USB port on the left side of the front panel. | + | |
- | + | ||
- | On a Linux PC, connect a micro USB cable and check via '' | + | |
- | + | ||
- | Now typing '' | + | |
- | + | ||
- | ==== 10.3 Enabling network boot on the CPU unit ==== | + | |
- | + | ||
- | Shortly after powering the CPU unit, press F2 to enter the BIOS. | + | |
- | + | ||
- | In the //Main// tab, go to //Boot Features// and select the following (using F6 for enabling and F5 for disabling): | + | |
- | + | ||
- | * '' | + | |
- | * '' | + | |
- | * '' | + | |
- | + | ||
- | In the // | + | |
- | + | ||
- | In the //Boot// tab, go to //Legacy// and //Boot Type Order//. There should be an //Others// entry that has to be shifted to the top of the list using F6. In newer BIOS versions, there is a //Boot Option #1// entry in the //Boot// tab, which has to be chosen to //IBA GE Slot 1600 v1513//. | + | |
- | + | ||
- | Save the settings by pressing F4. | + | |
- | + | ||
- | The CPU unit should boot from network after the next reboot. For the loading of the correct network image, the MAC address of the desired Ethernet port of the CPU unit has to registered in the DHCP server responsible for distributing the locations from which to load the network images. | + | |
- | + | ||
- | ==== 10.4 Configuration of the timing receiver ==== | + | |
- | + | ||
- | The GPIOs of the timing receiver can be controlled via command line commands available in the network boot image of the CPU unit: | + | |
- | + | ||
- | * display all available GPIOs: '' | + | |
- | * enable the outputs to the backplane: '' | + | |
- | * display the properties of a single GPIO: e.g. '' | + | |
- | * enable the output of a single GPIO: e.g. '' | + | |
- | * drive the output of a single GPIO high: e.g. '' | + | |
- | + | ||
- | ===== 12 References ===== | + | |
- | [1] IOxOS ADC_3117 | + | [1] Analog Devices LTC2323-16 |
- | [2] Analog Devices LTC2323-16 | + | [2] IOxOS ADC_3117 |