pCT - User contributions [en]

Client benchmark.cpp

2020-09-17T21:49:11Z

Wac002: Created page with " <syntaxhighlight lang="cpp"> #include <errno.h> #include <netdb.h> #include <stdio.h> #include <stdlib.h> #include <sys/socket.h> #include <unistd.h> #include <chrono> #incl..."

<syntaxhighlight lang="cpp">
#include <errno.h>
#include <netdb.h>
#include <stdio.h>
#include <stdlib.h>
#include <sys/socket.h>
#include <unistd.h>

#include <chrono>
#include <cstring>
#include <iostream>
#include <string>
#include <thread>

int main(int argc, char *argv[]) {
int s;
unsigned short port;
struct sockaddr_in server;
struct hostent *hp;
char buf[4096];

// if (argc != 3) {
// printf("Usage: %s <host address> <port> \n", argv[0]);
// exit(1);
//}
port = htons(atoi(argv[2]));

/* Create a datagram socket in the internet domain and use the
* default protocol (UDP).
*/

if ((s = socket(AF_INET, SOCK_DGRAM, 0)) < 0) {
perror("socket()");
exit(1);
}
struct timeval tv;
tv.tv_sec = 0;
tv.tv_usec = 2000;
if (setsockopt(s, SOL_SOCKET, SO_RCVTIMEO, &tv, sizeof(tv)) < 0) {
perror("Error setting timeout");
}
// printf(tvStart.tv_sec);
/* Set up the server name */
server.sin_family = AF_INET; /* Internet Domain */
server.sin_port = port; /* Server Port */
std::string ip = "192.168.2.20";
hp = gethostbyname(ip.c_str());
if (hp == 0) perror("Unknown host");

bcopy((char *)hp->h_addr, (char *)&server.sin_addr, hp->h_length);

char msg[] = {0x20, 0xFF, 0xFF, 0x8}; // Send command no ack, request
/* Send the message in buf to the server */
unsigned int slen = sizeof(struct sockaddr_in);

struct timeval tval;
int rslt;
int packNum = 65535;

std::chrono::high_resolution_clock nowTimePoint2;

auto statSize = 10;
std::cout << "start\n";
auto recvdBytes = 0;
auto timeouts = 0;
msg[0] = 0x60;
if (sendto(s, msg, (sizeof(msg)), 0, (struct sockaddr *)&server,
sizeof(server)) < 0) {
perror("sendto()");
exit(2);
}
std::this_thread::sleep_for(std::chrono::milliseconds(100));

for (size_t l = 2480; l <= 4080; l += 100) {
timeouts = 0;
msg[3] = (l / 16);
// printf("%hhX", msg[3]);
std::cout << (l / 16) * 16 << ',';

for (size_t i = 0; i < statSize; i++) {
msg[0] = 0x20;
auto start = std::chrono::time_point_cast<std::chrono::nanoseconds>(
nowTimePoint2.now());
if (sendto(s, msg, (sizeof(msg)), 0, (struct sockaddr *)&server,
sizeof(server)) < 0) {
perror("sendto()");
exit(2);
}

for (size_t j = 0; j < packNum; j++) {
recvdBytes =
recvfrom(s, buf, sizeof(buf), 0, (struct sockaddr *)&server, &slen);
if (recvdBytes <= 0) {
// std::cout << "ERROR IN RECV\n";
timeouts++;
}
}

auto stop = std::chrono::time_point_cast<std::chrono::nanoseconds>(
nowTimePoint2.now());
auto duration = stop - start;
std::cout << timeouts << ',';
std::cout << static_cast<unsigned int>(
(packNum * 1000.f * 1000.f * 1000.f) / duration.count() *
1.f)

<< ',';
msg[0] = 0x60;
if (sendto(s, msg, (sizeof(msg)), 0, (struct sockaddr *)&server,
sizeof(server)) < 0) {
perror("sendto()");
exit(2);
}
}

std::cout << '\n';
}

/* Deallocate the socket */
close(s);
}

</syntaxhighlight>

Control & Readout Software Documentation and Howto's

2020-09-17T21:48:15Z

Wac002:

[[Main Page]] -> [[Control & Readout Software Documentation and Howto's]]

== Getting Started ==
*[[Media:Requirements.pdf | Requirements]]
*[[Building Boost | Building Boost]]
*[[Building IPBus Software | Building IPBus Software]]
*[[Building ROOT | Building ROOT]]
*[[Building pct-online software | Building pct-online software]]
*[[Using IPbus | Using IPbus]]
*[[Running pDTPClient | Running pDTPClient]]
*[[Running pRUParser | Running pRUParser]]

== Documentation ==
The test software communicates with the readout unit and the ALPIDE via a series of registers. Once the test is initiated, a communication link with the readout unit will be set up. The register values specified in the test will set up the readout board and the ALPIDE. Data is then transferred from the ALPIDE to the readout unit via a firefly cable and offload from the readout unit is done via a 10 Gb high speed link.
The data is transferred to the the pRU parser, which is responsible for debugging and writing the data to a .root format. This data can then be analyzed to verify the results of the test in question.

=== ALPIDE Registers ===
Alpide Registers are registers that can be set on the ALPIDE chip.

This includes configuration registers to specify masking, pusling, strobe generation and configuration of the electrical part on the ALPIDE.

Specification on registers can be found in the ALPIDE manual

=== Readout Unit Registers ===
Readout unit registers are registers that can be set on the readout board.

This includes offload, triggering, global regs, alpide control and monitoring of different register via alpide data

Specifications on registers can be found in their respective modules

== Test examples ==

*[[Media:Digital Scan.pdf | Digital Scan]]
*[[Media:Analog Scan.pdf | Analog Scan]]
*[[Media:Threshold Scan.pdf | Threshold Scan]]
*[[Media:Single Event.pdf | Single Event]]

== Module Registers ==
*[[Media:Alpide control.pdf | Alpide control]]
*[[Media:Alpide data.pdf | Alpide data]]
*[[Media:Global regs.pdf | Global regs]]
*[[Media:Trigger manager.pdf | Trigger manager]]
*[[Media:Offload.pdf | Offload]]

== ALPIDE Manual ==
*[[Media:ALPIDEmanual_v0_3.pdf | Alpide manual]]

== .root format ==
* readout unit id
* stave id
* chip id
* frame id
* abs time (120 Mhz clock)
* bunch counter (40 Mhz clock)
* xy (hit coordinate)

== Network Stack Tuning Guide ==
Disclaimer: This guide applies to the Linux distribution CentOS and the Intel X710-DA2 network card using the User Datagram Protocol (UDP). (There might be some small variations with other distributions and other network cards).
=== Intel X710-DA2 Considerations ===
The X710-DA2 needs to be installed in a 8.0 GT/s, x8 lanes PCIe slot, additionally the card needs to be on its own PCI-E bus (many physical PCI-E slots share the same bus)[https://www.intel.com/content/www/us/en/support/articles/000005811/network-and-io/ethernet-products.html]. To determine what PCIe slots your motherboard has, please refer to the documentation provided by the manufacturer of your hardware.

* To identify which bus your card is installed in and that it is alone on that bus on run the command:
<code>$ sudo lshw -numeric</code>

==== Network card ring buffer ====
There is an HW buffer on the card itself, this buffer can be viewed with the command:

<code>$ ethtool -g <interface name></code>
and adjusted to the maximum size with the command:

<code>$ ethtool -G <interace name> rx 4096</code>

=== CentOS Considerations ===
* To list the relevant network settings in the kernel use the command:
<code>$ sudo sysctl -a | grep net.ipv4 and sudo sysctl -a | grep net.core</code>

=== Identifying possible bottlenecks ===
* To list network statistics from the network card(s) use the command:
<code>$ ethtool -S <interface name> </code>
* To list UDP drops realtime every second use the command:
<code>$ netstat -sune 1</code>
* To view package drops directly in the kernel use the command:
<code>$ sudo dropwatch -l kas</code> and write start

=== Benchmarking the system ===
To run a benchmark on you fast the system is you can either set the test registers with test data via the DCS(IPBus) or a custom firmware can be obtained from the [https://git.app.uib.no/pct/pDTP pDTP repo] under CI/CD -> Pipelines -> Artifacts NOTE: This requires using Vivado to upload the firmware to the VCU118 board.

Test code for a simple C++ client can be found here: [[client_benchmark.cpp]]. (Rename file type to CPP)
This client will send a stream request to hardware for 65535 packages, 15 times in the package size range from 2480 to 4080

To build:
<syntaxhighlight lang="bash">
$ g++ -O2 -Wno-narrowing client_benchmark.cpp -o client_executable
</syntaxhighlight>

To run:
<syntaxhighlight lang="bash">
./client_executable
</syntaxhighlight>

Control & Readout Software Documentation and Howto's

2020-09-17T21:19:05Z

Wac002:

[[Main Page]] -> [[Control & Readout Software Documentation and Howto's]]

== Getting Started ==
*[[Media:Requirements.pdf | Requirements]]
*[[Building Boost | Building Boost]]
*[[Building IPBus Software | Building IPBus Software]]
*[[Building ROOT | Building ROOT]]
*[[Building pct-online software | Building pct-online software]]
*[[Using IPbus | Using IPbus]]
*[[Running pDTPClient | Running pDTPClient]]
*[[Running pRUParser | Running pRUParser]]

== Documentation ==
The test software communicates with the readout unit and the ALPIDE via a series of registers. Once the test is initiated, a communication link with the readout unit will be set up. The register values specified in the test will set up the readout board and the ALPIDE. Data is then transferred from the ALPIDE to the readout unit via a firefly cable and offload from the readout unit is done via a 10 Gb high speed link.
The data is transferred to the the pRU parser, which is responsible for debugging and writing the data to a .root format. This data can then be analyzed to verify the results of the test in question.

=== ALPIDE Registers ===
Alpide Registers are registers that can be set on the ALPIDE chip.

This includes configuration registers to specify masking, pusling, strobe generation and configuration of the electrical part on the ALPIDE.

Specification on registers can be found in the ALPIDE manual

=== Readout Unit Registers ===
Readout unit registers are registers that can be set on the readout board.

This includes offload, triggering, global regs, alpide control and monitoring of different register via alpide data

Specifications on registers can be found in their respective modules

== Test examples ==

*[[Media:Digital Scan.pdf | Digital Scan]]
*[[Media:Analog Scan.pdf | Analog Scan]]
*[[Media:Threshold Scan.pdf | Threshold Scan]]
*[[Media:Single Event.pdf | Single Event]]

== Module Registers ==
*[[Media:Alpide control.pdf | Alpide control]]
*[[Media:Alpide data.pdf | Alpide data]]
*[[Media:Global regs.pdf | Global regs]]
*[[Media:Trigger manager.pdf | Trigger manager]]
*[[Media:Offload.pdf | Offload]]

== ALPIDE Manual ==
*[[Media:ALPIDEmanual_v0_3.pdf | Alpide manual]]

== .root format ==
* readout unit id
* stave id
* chip id
* frame id
* abs time (120 Mhz clock)
* bunch counter (40 Mhz clock)
* xy (hit coordinate)

== Network Stack Tuning Guide ==
Disclaimer: This guide applies to the Linux distribution CentOS and the Intel X710-DA2 network card using the User Datagram Protocol (UDP). (There might be some small variations with other distributions and other network cards).
=== Intel X710-DA2 Considerations ===
The X710-DA2 needs to be installed in a 8.0 GT/s, x8 lanes PCIe slot, additionally the card needs to be on its own PCI-E bus (many physical PCI-E slots share the same bus)[https://www.intel.com/content/www/us/en/support/articles/000005811/network-and-io/ethernet-products.html]. To determine what PCIe slots your motherboard has, please refer to the documentation provided by the manufacturer of your hardware.

* To identify which bus your card is installed in and that it is alone on that bus on run the command:
<code>$ sudo lshw -numeric</code>

==== Network card ring buffer ====
There is an HW buffer on the card itself, this buffer can be viewed with the command:

<code>$ ethtool -g <interface name></code>
and adjusted to the maximum size with the command:

<code>$ ethtool -G <interace name> rx 4096</code>

=== CentOS Considerations ===
* To list the relevant network settings in the kernel use the command:
<code>$ sudo sysctl -a | grep net.ipv4 and sudo sysctl -a | grep net.core</code>

=== Identifying possible bottlenecks ===
* To list network statistics from the network card(s) use the command:
<code>$ ethtool -S <interface name> </code>
* To list UDP drops realtime every second use the command:
<code>$ netstat -sune 1</code>
* To view package drops directly in the kernel use the command:
<code>$ sudo dropwatch -l kas</code> and write start

=== Benchmarking the system ===
To run a benchmark on you fast the system is you can either set the test registers with test data via the DCS(IPBus) or a custom firmware can be obtained from the [https://git.app.uib.no/pct/pDTP pDTP repo] under CI/CD -> Pipelines -> Artifacts NOTE: This requires using Vivado to upload the firmware to the VCU118 board.
Test code for a simple C++ client can be found here: [[File:client_benchmark.txt]]. (Rename file type to CPP)

To build:
<syntaxhighlight lang="bash">
$ g++ -O2 -Wno-narrowing client_benchmark.cpp -o client_executable
</syntaxhighlight>

To run:
<syntaxhighlight lang="bash">
./client_executable
</syntaxhighlight>

Running pDTPClient

2020-09-17T20:29:04Z

Wac002:

[[Main Page]] -> [[Control & Readout Software Documentation and Howto's]] -> [[Running pDTPClient]]

== Use case ==
The main task of this program is to get data from the proton Readout Unit (pRU) hardware to a host computer for storage or parsing.

The pDTPClient handles the proton Data Transfer Protocol (pDTP) using UDP.
pDTP has three main modes; Request read (RQR), a single packet from 1 byte to 4080 bytes. Request stream (RQS), a single packet up to 65535 packets. And request full stream (RQFS). For more in-depth details about pDTP see:
[[Media:PDTP.pdf | pDTP spesification]]

If the file writer is enabled the client will produce time and date stamped file that in a binary format.

== Simplified diagram of offload chain ==
[[File:offload-system.png|750px]]

== Optimal configuration ==
Due to the RUs ability to max out the 10Gbit link with small package sizes (high burst) and the Linux kernels inability to process large amounts of packages, an optimal configuration is advised.

There is also a consideration to be made with the size of the queue between the pDTPClient and the file writer/parser.

The config file in the next section is a good starting point on an optimal configuration.

== The config file ==
The config file exists to load the diffrent settings of the client in a easy manner.
<nowiki>#Config file for the settings to the pDTPClient
#IP adress to the PRU
IP = 192.168.100.200
#IP = 127.0.0.1
#PORT 30000 for data and 29070 for loopback
PORT = 30000
#Size of request stream 1 to 65535
SSIZE = 10
#Size of packet, 1 to 4080
PSIZE = 4080
#Start mode: RQT, RQR, RQS, RQFS, GS, ABRT
MODE = RQS
#Time to run the client in seconds: 0 for disable.
RTIME = 0
#UDP recv timeout in microsecounds, 0 to disable
UDPTIMEO = 4
#Request a certain number of words, then stop the client, 0 to disable
NUMREQ = 0
#Set the verbosity for log messsages to console
VERB = 2
#Set the size of the SPSC queue
SPSCSIZE = 10000
#Poll PRU, yes = 1, no = 0
POLL = 1
#FileWriter Enable = 1, Disable = 0
ENABLEFILE = 1
#How to handle the buffer fill level, 0 = no RQS,RQFS, 1 = RQS, no RQFS or
#2 = RQS and RQFS.
SMODE = 0
#Set no wait flag
NOWAIT = 0
#Set maximize flag
MAXIMIZE = 0</nowiki>

== Starting the client ==
The client can be run with the command:

<code>./pDTPClient --config <location/config.conf></code>

== Using a hardware emulator ==
The hardware emulator can be obtained from the [https://git.app.uib.no/pct/pDTP pDTP repo]

=== Starting the emulator ===
Load server with only constant data:
<syntaxhighlight lang="bash">
python3 server_emulator.py
</syntaxhighlight>
Load a file with data:
<syntaxhighlight lang="bash">
python3 server_emulator.py -f <FILENAME>
</syntaxhighlight>
Increase verbosity (prints all debug messages to console. Note that these can be viewed in debug.log as well):
<syntaxhighlight lang="bash">
python3 server_emulator.py -v
</syntaxhighlight>

=== Adapt the config file ===
You can adapt the config file to use <code>IP = localhost</code> to connect to the emulator when it is running.

Running pDTPClient

2020-09-17T20:26:56Z

Wac002:

[[Main Page]] -> [[Control & Readout Software Documentation and Howto's]] -> [[Running pDTPClient]]

== Use case ==
The main task of this program is to get data from the proton Readout Unit (pRU) hardware to a host computer for storage or parsing.

The pDTPClient handles the proton Data Transfer Protocol (pDTP) using UDP.
pDTP has three main modes; Request read (RQR), a single packet from 1 byte to 4080 bytes. Request stream (RQS), a single packet up to 65535 packets. And request full stream (RQFS). For more in-depth details about pDTP see:
[[Media:PDTP.pdf | pDTP spesification]]

If the file writer is enabled the client will produce time and date stamped file that in a binary format.

== Simplified diagram of offload chain ==
[[File:offload-system.png|750px]]

== Optimal configuration ==
Due to the RUs ability to max out the 10Gbit link with small package sizes (high burst) and the Linux kernels inability to process large amounts of packages, an optimal configuration is advised.

There is also a consideration to be made with the size of the queue between the pDTPClient and the file writer/parser.

The config file in the next section is a good starting point on an optimal configuration.

== The config file ==
The config file exists to load the diffrent settings of the client in a easy manner.
<nowiki>#Config file for the settings to the pDTPClient
#IP adress to the PRU
IP = 192.168.100.200
#IP = 127.0.0.1
#PORT 30000 for data and 29070 for loopback
PORT = 30000
#Size of request stream 1 to 65535
SSIZE = 10
#Size of packet, 1 to 4080
PSIZE = 4080
#Start mode: RQT, RQR, RQS, RQFS, GS, ABRT
MODE = RQS
#Time to run client in seconds: 0 for disable.
RTIME = 0
#UDP recv timeout in microsecounds, 0 to disable
UDPTIMEO = 4
#Request a certain number of words, then stop the client, 0 to disable
NUMREQ = 0
#Set the verbosity for log messsages to console
VERB = 2
#Set the size of the SPSC queue
SPSCSIZE = 10000
#Poll PRU, yes = 1, no = 0
POLL = 1
#FileWriter Enable = 1, Disable = 0
ENABLEFILE = 1
#How to handle the buffer fill level, 0 = no RQS,RQFS, 1 = RQS, no RQFS or
#2 = RQS and RQFS.
SMODE = 0
#Set no wait flag
NOWAIT = 0
#Set maximize flag
MAXIMIZE = 0</nowiki>

== Starting the client ==
The client can be run with the command:

<code>./pDTPClient --config <location/config.conf></code>

== Using a hardware emulator ==
The hardware emulator can be obtained from the [https://git.app.uib.no/pct/pDTP pDTP repo]

=== Starting the emulator ===
Load server with only constant data:
<syntaxhighlight lang="bash">
python3 server_emulator.py
</syntaxhighlight>
Load a file with data:
<syntaxhighlight lang="bash">
python3 server_emulator.py -f <FILENAME>
</syntaxhighlight>
Increase verbosity (prints all debug messages to console. Note that these can be viewed in debug.log as well):
<syntaxhighlight lang="bash">
python3 server_emulator.py -v
</syntaxhighlight>

=== Adapt the config file ===
You can adapt the config file to use localhost to connect to the emulator when it is running.

Using IPbus

2020-09-09T14:10:21Z

Wac002:

[[Main Page]] -> [[Control & Readout Software Documentation and Howto's]] -> [[Using IPbus]]

== Use case ==
IPbus is a control protocol that uses UDP/IP to send values into the registers on the readout unit (RU). The software on the host computer uses the Hardware Access Library (uHAL) to send the commands to the RU.
[[File:ipbus-conect6.png|750px]]

== Configuration files ==
There is a main connection file (pRU-connections.xml), this file contains the address and port of the RU, for information on the content of the connection file please see the [[:Media:PRU Ethernet Configurations.pdf | pRU Ethernet Configurations]] document.

Following there is a Module registers file (module-address.xml), this file contains a table with all the registers present in the RU.

Finally, there are files for each register
* global_regs.xml
* trigger_manager.xml
* offload.xml
* alpide_control.xml
* alpide_data.xml

As of the writing of this page, these files can be found on the production_test repository in the ipbus branch under the BSP folder.

WARNING: To prevent undefined behavior the register files MUST match the firmware version!

== Smart Dummy Hardware ==
To be able to read values from registers, a custom DummyHardwareUdp.exe can be built from this repository:
[https://github.com/norscope/ipbus-software/tree/smartdummy IPbus Custom Repo]
NOTE: Remember to switch to the smartdummy branch.

This adds a file reader in the DummyHardware.cpp that reads a file in the directory of the executable, the file is called register-values.txt, an example file can be found here: [[:Media:register-values.txt | Example file]]

=== How to Build on Ubuntu 20.04 LTS ===
==== Requierd packages ====
<syntaxhighlight lang="bash">
$ sudo apt update
$ sudo apt install git g++ make erlang python3 python-is-python3 libboost1.71-all-dev libpugixml-dev
</syntaxhighlight>

==== Building ====
In the directory where the repository have been downloaded:

<syntaxhighlight lang="bash">
$ make
$ sudo make install prefix=/home/user/ipbus-install
</syntaxhighlight>

==== Adding paths to envirorment ====
<syntaxhighlight lang="bash">
$ export LD_LIBRARY_PATH=/home/user/ipbus-install/lib:$LD_LIBRARY_PATH
$ export PYTHONPATH=/home/user/software-dev/ipbus-install/lib/python3.8/site-packages:$PYTHONPATH
</syntaxhighlight>

==== Running the hardware emulator ====
Change directory to ipbus-install/bin/uhal/tests
NOTE: Make sure the register-values.txt file is present in this folder. Filename is in all lowercase.

<syntaxhighlight lang="bash">
./DummyHardwareUdp.exe -p 50001 -v 2 -V
</syntaxhighlight>

==== Testing the emulator ====
From the production_test repo, in the folder tests_vcu118 run:

<syntaxhighlight lang="bash">
$ python test_ipbus_emulator.py
</syntaxhighlight>

This should output 0xfeedbeef

== Additonal information ==
For more information, please consult the IPbus wiki at: [https://ipbus.web.cern.ch/ipbus/doc/user/html/software/uhalQuickTutorial.html IPbus wiki]

File:Register-values.txt

2020-09-09T14:06:15Z

Wac002: Wac002 uploaded a new version of File:Register-values.txt

Using IPbus

2020-09-09T13:55:36Z

Wac002:

[[Main Page]] -> [[Control & Readout Software Documentation and Howto's]] -> [[Using IPbus]]

== Use case ==
IPbus is a control protocol that uses UDP/IP to send values into the registers on the readout unit (RU). The software on the host computer uses the Hardware Access Library (uHAL) to send the commands to the RU.
[[File:ipbus-conect6.png|750px]]

== Configuration files ==
There is a main connection file (pRU-connections.xml), this file contains the address and port of the RU, for information on the content of the connection file please see the [[:Media:PRU Ethernet Configurations.pdf | pRU Ethernet Configurations]] document.

Following there is a Module registers file (module-address.xml), this file contains a table with all the registers present in the RU.

Finally, there are files for each register
* global_regs.xml
* trigger_manager.xml
* offload.xml
* alpide_control.xml
* alpide_data.xml

As of the writing of this page, these files can be found on the production_test repository in the ipbus branch under the BSP folder.

WARNING: To prevent undefined behavior the register files MUST match the firmware version!

== Smart Dummy Hardware ==
To be able to read values from registers, a custom DummyHardwareUdp.exe can be built from this repository:
[https://github.com/norscope/ipbus-software/tree/smartdummy IPbus Custom Repo]
NOTE: Remember to switch to the smartdummy branch.

This adds a file reader in the DummyHardware.cpp that reads a file in the directory of the executable, the file is called register-values.txt, an example file can be found here: [[:Media:register-values.txt | Example file]]

=== How to Build on Ubuntu 20.04 LTS ===
==== REQUIRED PACKAGES ====
<syntaxhighlight lang="bash">
$ sudo apt update
$ sudo apt install git g++ make erlang python3 python-is-python3 libboost1.71-all-dev libpugixml-dev
</syntaxhighlight>

==== BUILDING ====
In the directory where the repository have been downloaded:

<syntaxhighlight lang="bash">
$ make
$ sudo make install prefix=/home/user/ipbus-install
</syntaxhighlight>

==== ADDING PATHS TO ENVIRORMENT ====
<syntaxhighlight lang="bash">
$ export LD_LIBRARY_PATH=/home/user/ipbus-install/lib:$LD_LIBRARY_PATH
$ export PYTHONPATH=/home/user/software-dev/ipbus-install/lib/python3.8/site-packages:$PYTHONPATH
</syntaxhighlight>

==== RUNNING THE EMULATOR ====
Change directory to ipbus-install/bin/uhal/tests
NOTE: Make sure the register-values.txt file is present in this folder.

<syntaxhighlight lang="bash">
./DummyHardwareUdp.exe -p 50001 -v 2 -V
</syntaxhighlight>

== Additonal information ==
For more information, please consult the IPbus wiki at: [https://ipbus.web.cern.ch/ipbus/doc/user/html/software/uhalQuickTutorial.html IPbus wiki]

Using IPbus

2020-09-08T10:17:07Z

Wac002:

Using IPbus

2020-09-08T09:44:39Z

Wac002:

File:Register-values.txt

2020-09-08T08:16:28Z

Wac002:

Using IPbus

2020-09-08T08:15:11Z

Wac002:

Using IPbus

2020-09-08T08:09:54Z

Wac002:

Using IPbus

2020-09-08T08:09:03Z

Wac002:

Using IPbus

2020-06-23T09:36:34Z

Wac002:

File:Ipbus-conect6.png

2020-06-23T09:35:02Z

Wac002: File uploaded with MsUpload

File uploaded with MsUpload

Using IPbus

2020-06-23T09:32:27Z

Wac002:

File:Ipbus-conect5.png

2020-06-23T09:31:54Z

Wac002: File uploaded with MsUpload

File uploaded with MsUpload

Using IPbus

2020-06-22T10:24:17Z

Wac002:

Using IPbus

2020-06-22T10:23:44Z

Wac002:

File:Ipbus-conect2.png

2020-06-22T10:22:41Z

Wac002: File uploaded with MsUpload

File uploaded with MsUpload

File:Ipbus-conect.png

2020-06-22T10:20:50Z

Wac002: Wac002 uploaded a new version of File:Ipbus-conect.png

File uploaded with MsUpload

Using IPbus

2020-06-22T07:59:08Z

Wac002:

File:Ipbus-conect.png

2020-06-22T07:57:44Z

Wac002: File uploaded with MsUpload

File uploaded with MsUpload

Using IPbus

2020-06-21T15:24:19Z

Wac002:

Using IPbus

2020-06-21T15:23:31Z

Wac002:

Using IPbus

2020-06-21T15:21:29Z

Wac002:

Using IPbus

2020-06-21T15:19:39Z

Wac002:

Using IPbus

2020-06-21T15:17:58Z

Wac002:

Using IPbus

2020-06-21T15:03:26Z

Wac002:

[[Main Page]] -> [[Control & Readout Software Documentation and Howto's]] -> [[Using IPbus]]

== Use case ==
IPbus is a control protocol that uses UDP/IP to send values into the registries on the readout unit (RU). Hardware Access Library (uHAL) is used on the host computer to send the commands.

== Configuration files ==
There is a main connection file pRU-connections.xml, this file contains the address and port of the RU, for information on the content of the connection file please see the pRU Ethernet Configuration document.

Following there is a Module registers file (module-address.xml), this file contains a table with all the registries present in the RU.

Finally, there are files for each register
* global_regs.xml
* trigger_manager.xml
* offload.xml
* alpide_control.xml
* alpide_data.xml

PTB-specific Module R

As of the writing of this page, these files can be found on the production_test repository in the ipbus fork under the BSP folder.

Using IPbus

2020-06-21T14:12:05Z

Wac002:

[[Main Page]] -> [[Control & Readout Software Documentation and Howto's]] -> [[Using IPbus]]

== Use case ==
IPbus is a control protocol is used to send values into the registries on the readout unit using the Hardware Access Library (uHAL)

Using IPbus

2020-06-21T14:07:07Z

Wac002: Created page with "Main Page -> Control & Readout Software Documentation and Howto's -> Using IPbus"

[[Main Page]] -> [[Control & Readout Software Documentation and Howto's]] -> [[Using IPbus]]

Control & Readout Software Documentation and Howto's

2020-06-21T14:06:05Z

Wac002:

Running pDTPClient

2020-02-04T14:27:57Z

Wac002:

[[Main Page]] -> [[Control & Readout Software Documentation and Howto's]] -> [[Running pDTPClient]]

== Use case ==
The main task of this program is to get data from the proton Readout Unit (pRU) hardware to a host computer for storage or parsing.

The pDTPClient handles the proton Data Transfer Protocol (pDTP) using UDP.
pDTP has three main modes; Request read (RQR), a single packet from 1 byte to 4080 bytes. Request stream (RQS), a single packet up to 65535 packets. And request full stream (RQFS). For more in-depth details about pDTP see:
[[Media:PDTP.pdf | pDTP spesification]]

If the file writer is enabled the client will produce time and date stamped file that in a binary format.

== Simplified diagram of offload chain ==
[[File:offload-system.png|750px]]

== Optimal configuration ==
Due to the RUs ability to max out the 10Gbit link with small package sizes (high burst) and the Linux kernels inability to process large amounts of packages, an optimal configuration is advised.

There is also a consideration to be made with the size of the queue between the pDTPClient and the file writer/parser.

The config file in the next section is a good starting point on an optimal configuration.

== The config file ==
The config file exists to load the diffrent settings of the client in a easy manner.
<nowiki>#Config file for the settings to the pDTPClient
#IP adress to the PRU
IP = 192.168.100.200
#IP = 127.0.0.1
#PORT 30000 for data and 29070 for loopback
PORT = 30000
#Size of request stream 1 to 65535
SSIZE = 10
#Size of packet, 1 to 4080
PSIZE = 4080
#Start mode: RQT, RQR, RQS, RQFS, GS, ABRT
MODE = RQS
#Time to run client in seconds: 0 for disable.
RTIME = 0
#UDP recv timeout in microsecounds, 0 to disable
UDPTIMEO = 4
#Request a certain number of words, then stop the client, 0 to disable
NUMREQ = 0
#Set the verbosity for log messsages to console
VERB = 2
#Set the size of the SPSC queue
SPSCSIZE = 10000
#Poll PRU, yes = 1, no = 0
POLL = 1
#FileWriter Enable = 1, Disable = 0
ENABLEFILE = 1
#How to handle the buffer fill level, 0 = no RQS,RQFS, 1 = RQS, no RQFS or
#2 = RQS and RQFS.
SMODE = 0
#Set no wait flag
NOWAIT = 0
#Set maximize flag
MAXIMIZE = 0</nowiki>

== Starting the client ==
The client can be run with the command:

<code>./pDTPClient --config <location/config.conf></code>

Running pDTPClient

2020-02-04T14:27:14Z

Wac002:

[[Main Page]] -> [[Control & Readout Software Documentation and Howto's]] -> [[Running pDTPClient]]

== Use case ==
The main task of this program is to get data from the proton Readout Unit (pRU) hardware to a host computer for storage or parsing.

The pDTPClient handles the proton Data Transfer Protocol (pDTP) using UDP.
pDTP has three main modes; Request read (RQR), a single packet from 1 byte to 4080 bytes. Request stream (RQS), a single packet up to 65535 packets. And request full stream (RQFS). For more in-depth details about pDTP see:
[[Media:PDTP.pdf | pDTP spesification]]

If the file writer is enabled the client will produce time and date stamped file that in a binary format.

== Simplified diagram of offload chain ==
[[File:offload-system.png|750px]]

== Optimal configuration ==
Due to the RUs ability to max out the 10Gbit link with small package sizes (high burst) and the Linux kernels inability to process large amounts of packages, an optimal configuration is advised.

There is also a consideration to be made with the size of the queue between the pDTPClient and the file writer/parser.

== The config file ==
The config file exists to load the diffrent settings of the client in a easy manner.
<nowiki>#Config file for the settings to the pDTPClient
#IP adress to the PRU
IP = 192.168.100.200
#IP = 127.0.0.1
#PORT 30000 for data and 29070 for loopback
PORT = 30000
#Size of request stream 1 to 65535
SSIZE = 10
#Size of packet, 1 to 4080
PSIZE = 4080
#Start mode: RQT, RQR, RQS, RQFS, GS, ABRT
MODE = RQS
#Time to run client in seconds: 0 for disable.
RTIME = 0
#UDP recv timeout in microsecounds, 0 to disable
UDPTIMEO = 4
#Request a certain number of words, then stop the client, 0 to disable
NUMREQ = 0
#Set the verbosity for log messsages to console
VERB = 2
#Set the size of the SPSC queue
SPSCSIZE = 10000
#Poll PRU, yes = 1, no = 0
POLL = 1
#FileWriter Enable = 1, Disable = 0
ENABLEFILE = 1
#How to handle the buffer fill level, 0 = no RQS,RQFS, 1 = RQS, no RQFS or
#2 = RQS and RQFS.
SMODE = 0
#Set no wait flag
NOWAIT = 0
#Set maximize flag
MAXIMIZE = 0</nowiki>

== Starting the client ==
The client can be run with the command:

<code>./pDTPClient --config <location/config.conf></code>

Running pDTPClient

2020-02-04T14:19:27Z

Wac002:

[[Main Page]] -> [[Control & Readout Software Documentation and Howto's]] -> [[Running pDTPClient]]

== Use case ==
The main task of this program is to get data from the proton Readout Unit (pRU) hardware to a host computer for storage or parsing.

The pDTPClient handles the proton Data Transfer Protocol (pDTP) using UDP.
pDTP has three main modes; Request read (RQR), a single packet from 1 byte to 4080 bytes. Request stream (RQS), a single packet up to 65535 packets. And request full stream (RQFS). For more in-depth details about pDTP see:
[[Media:PDTP.pdf | pDTP spesification]]

If the file writer is enabled the client will produce time and date stamped file that in a binary format.

== Simplified diagram of offload chain ==
[[File:offload-system.png|750px]]

== Optimal configuration ==
Due to the RUs ability to max out the 10Gbit link with small package sizes (high burst) and the Linux kernels inability to process large amounts of packages, and optimal configuration is advised.

There is also a consideration to be made with the size of the queue between the pDTPClient and the file writer/parser.

== The config file ==
The config file exists to load the diffrent settings of the client in a easy manner.
<nowiki>#Config file for the settings to the pDTPClient
#IP adress to the PRU
IP = 192.168.100.200
#IP = 127.0.0.1
#PORT 30000 for data and 29070 for loopback
PORT = 30000
#Size of request stream 1 to 65535
SSIZE = 10
#Size of packet, 1 to 4080
PSIZE = 4080
#Start mode: RQT, RQR, RQS, RQFS, GS, ABRT
MODE = RQS
#Time to run client in seconds: 0 for disable.
RTIME = 0
#UDP recv timeout in microsecounds, 0 to disable
UDPTIMEO = 4
#Request a certain number of words, then stop the client, 0 to disable
NUMREQ = 0
#Set the verbosity for log messsages to console
VERB = 2
#Set the size of the SPSC queue
SPSCSIZE = 10000
#Poll PRU, yes = 1, no = 0
POLL = 1
#FileWriter Enable = 1, Disable = 0
ENABLEFILE = 1
#How to handle the buffer fill level, 0 = no RQS,RQFS, 1 = RQS, no RQFS or
#2 = RQS and RQFS.
SMODE = 0
#Set no wait flag
NOWAIT = 0
#Set maximize flag
MAXIMIZE = 0</nowiki>

== Starting the client ==
The client can be run with the command:

<code>./pDTPClient --config <location/config.conf></code>

Running pDTPClient

2020-02-04T08:30:59Z

Wac002:

File:Offload-system.png

2020-02-04T08:06:49Z

Wac002: File uploaded with MsUpload

File uploaded with MsUpload

Control & Readout Software Documentation and Howto's

2020-02-04T07:50:04Z

Wac002:

[[Main Page]] -> [[Control & Readout Software Documentation and Howto's]]

== Getting Started ==
*[[Media:Requirements.pdf | Requirements]]
*[[Building Boost | Building Boost]]
*[[Building IPBus Software | Building IPBus Software]]
*[[Building ROOT | Building ROOT]]
*[[Building pct-online software | Building pct-online software]]
*[[Running pDTPClient | Running pDTPClient]]
*[[Running pRUParser | Running pRUParser]]

== Documentation ==
The test software communicates with the readout unit and the ALPIDE via a series of registers. Once the test is initiated, a communication link with the readout unit will be set up. The register values specified in the test will set up the readout board and the ALPIDE. Data is then transferred from the ALPIDE to the readout unit via a firefly cable and offload from the readout unit is done via a 10 Gb high speed link.
The data is transferred to the the pRU parser, which is responsible for debugging and writing the data to a .root format. This data can then be analyzed to verify the results of the test in question.

=== ALPIDE Registers ===
Alpide Registers are registers that can be set on the ALPIDE chip.

This includes configuration registers to specify masking, pusling, strobe generation and configuration of the electrical part on the ALPIDE.

Specification on registers can be found in the ALPIDE manual

=== Readout Unit Registers ===
Readout unit registers are registers that can be set on the readout board.

This includes offload, triggering, global regs, alpide control and monitoring of different register via alpide data

Specifications on registers can be found in their respective modules

== Test examples ==

*[[Media:Digital Scan.pdf | Digital Scan]]
*[[Media:Analog Scan.pdf | Analog Scan]]
*[[Media:Threshold Scan.pdf | Threshold Scan]]
*[[Media:Single Event.pdf | Single Event]]

== Module Registers ==
*[[Media:Alpide control.pdf | Alpide control]]
*[[Media:Alpide data.pdf | Alpide data]]
*[[Media:Global regs.pdf | Global regs]]
*[[Media:Trigger manager.pdf | Trigger manager]]
*[[Media:Offload.pdf | Offload]]

== ALPIDE Manual ==
*[[Media:ALPIDEmanual_v0_3.pdf | Alpide manual]]

== .root format ==
* readout unit id
* stave id
* chip id
* frame id
* abs time (120 Mhz clock)
* bunch counter (40 Mhz clock)
* xy (hit coordinate)

== Network Stack Tuning Guide ==
Disclaimer: This guide applies to the Linux distribution CentOS and the Intel X710-DA2 network card using the User Datagram Protocol (UDP). (There might be some small variations with other distributions and other network cards).
=== Intel X710-DA2 Considerations ===
The X710-DA2 needs to be installed in a 8.0 GT/s, x8 lanes PCIe slot, additionally the card needs to be on its own PCI-E bus (many physical PCI-E slots share the same bus)[https://www.intel.com/content/www/us/en/support/articles/000005811/network-and-io/ethernet-products.html]. To determine what PCIe slots your motherboard has, please refer to the documentation provided by the manufacturer of your hardware.

* To identify which bus your card is installed in and that it is alone on that bus on run the command:
<code>$ sudo lshw -numeric</code>

==== Network card ring buffer ====
There is an HW buffer on the card itself, this buffer can be viewed with the command:

<code>$ ethtool -g <interface name></code>
and adjusted to the maximum size with the command:

<code>$ ethtool -G <interace name> rx 4096</code>

=== CentOS Considerations ===
* To list the relevant network settings in the kernel use the command:
<code>$ sudo sysctl -a | grep net.ipv4 and sudo sysctl -a | grep net.core</code>

=== Identifying possible bottlenecks ===
* To list network statistics from the network card(s) use the command:
<code>$ ethtool -S <interface name> </code>
* To list UDP drops realtime every second use the command:
<code>$ netstat -sune 1</code>
* To view package drops directly in the kernel use the command:
<code>$ sudo dropwatch -l kas</code> and write start

Running pDTPClient

2020-02-03T15:08:14Z

Wac002:

Running pDTPClient

2020-02-03T14:25:33Z

Wac002:

Running pDTPClient

2020-02-03T13:53:53Z

Wac002:

Running pDTPClient

2020-02-03T13:27:11Z

Wac002:

[[Main Page]] -> [[Control & Readout Software Documentation and Howto's]] -> [[Running pDTPClient]]

== Getting Started ==
*[[Media:Requirements.pdf | Requirements]]
pDTPClient
The main task of this program is to get data from the proton Readout Unit (pRU) hardware to a host computer for storage or parsing.

The pDTPClient handles the proton Data Transfer Protocol (pDTP) using UDP. pDTP have three main modes; Request read (RQR), a single packet from 1 byte to 4080 byte. Request stream (RQS), a single packet upto 65535 packets. And request full stream (RQFS)

The config file:

Starting the client:

Running pDTPClient

2020-02-03T13:24:57Z

Wac002: User guide for the pDTPClient

pDTPClient
The main task of this program is to get data from the proton Readout Unit (pRU) hardware to a host computer for storage or parsing.

The pDTPClient handles the proton Data Transfer Protocol (pDTP) using UDP. pDTP have three main modes; Request read (RQR), a single packet from 1 byte to 4080 byte. Request stream (RQS), a single packet upto 65535 packets. And request full stream (RQFS)

The config file:

Starting the client:

Control & Readout Software Documentation and Howto's

2019-12-28T14:28:14Z

Wac002:

== Getting Started ==
*[[Media:Requirements.pdf | Requirements]]

== Documentation ==
The test software communicates with the readout unit and the ALPIDE via a series of registers. Once the test is initiated, a communication link with the readout unit will be set up. The register values specified in the test will set up the readout board and the ALPIDE. Data is then transferred from the ALPIDE to the readout unit via a firefly cable and offload from the readout unit is done via a 10 Gb high speed link.
The data is transferred to the the pRU parser, which is responsible for debugging and writing the data to a .root format. This data can then be analyzed to verify the results of the test in question.

=== ALPIDE Registers ===
Alpide Registers are registers that can be set on the ALPIDE chip.

This includes configuration registers to specify masking, pusling, strobe generation and configuration of the electrical part on the ALPIDE.

Specification on registers can be found in the ALPIDE manual

=== Readout Unit Registers ===
Readout unit registers are registers that can be set on the readout board.

This includes offload, triggering, global regs, alpide control and monitoring of different register via alpide data

Specifications on registers can be found in their respective modules

== Test examples ==

*[[Media:Digital Scan.pdf | Digital Scan]]
*[[Media:Analog Scan.pdf | Analog Scan]]
*[[Media:Threshold Scan.pdf | Threshold Scan]]
*[[Media:Single Event.pdf | Single Event]]

== Module Registers ==
*[[Media:Alpide control.pdf | Alpide control]]
*[[Media:Alpide data.pdf | Alpide data]]
*[[Media:Global regs.pdf | Global regs]]
*[[Media:Trigger manager.pdf | Trigger manager]]
*[[Media:Offload.pdf | Offload]]

== ALPIDE Manual ==
*[[Media:ALPIDEmanual_v0_3.pdf | Alpide manual]]

== .root format ==
* readout unit id
* stave id
* chip id
* frame id
* abs time (120 Mhz clock)
* bunch counter (40 Mhz clock)
* xy (hit coordinate)

== Network Stack Tuning Guide ==
Disclaimer: This guide applies to the Linux distribution CentOS and the Intel X710-DA2 network card using the User Datagram Protocol (UDP). (There might be some small variations with other distributions and other network cards).
=== Intel X710-DA2 Considerations ===
The X710-DA2 needs to be installed in a 8.0 GT/s, x8 lanes PCIe slot, additionally the card needs to be on its own PCI-E bus (many physical PCI-E slots share the same bus)[https://www.intel.com/content/www/us/en/support/articles/000005811/network-and-io/ethernet-products.html]. To determine what PCIe slots your motherboard has, please refer to the documentation provided by the manufacturer of your hardware.

* To identify which bus your card is installed in and that it is alone on that bus on run the command:
sudo lshw -numeric

==== Network card ring buffer ====
There is an HW buffer on the card itself, this buffer can be viewed with the command:

ethtool -g <interface name>
and adjusted to the maximum size with the command:
ethtool -G <interace name> rx 4096

=== CentOS Considerations ===
* To list the relevant network settings in the kernel use the command:
sudo sysctl -a | grep net.ipv4 and sudo sysctl -a | grep net.core

=== Identifying possible bottlenecks ===
* To list network statistics from the network card(s) use the command:
ethtool -S <interface name>
* To list UDP drops realtime every second use the command:
netstat -sune 1
* To view package drops directly in the kernel use the command:
sudo dropwatch -l kas and write start