Running pRUParser: Difference between revisions
No edit summary |
No edit summary |
||
(13 intermediate revisions by 3 users not shown) | |||
Line 1: | Line 1: | ||
[[Main Page]] -> [[Control & Readout Software Documentation and Howto's]] -> [[Running pRUParser]] | |||
== Software Function == | == Software Function == | ||
The pRUParser takes a raw, binary, offload stream of pRU data from a file created by the pDTP Client. | The pRUParser takes a raw, binary, offload stream of pRU data from a file created by the pDTP Client. | ||
It will sort, error check and prunes this in to a raw, binary, stream and save this do disk for further prosessing. | It will sort, error check and prunes this in to a raw, binary, stream and save this do disk for further prosessing. | ||
It will lastly print a summary of any errors it detected. | It will lastly print a summary of any errors it detected. | ||
== Protocol Dependencies == | |||
The pRUParser uses the pRU Data Format Specification v0.2 | |||
'''TODO''' add link to protocol document/page | |||
== Usage == | == Usage == | ||
Line 15: | Line 23: | ||
$./run-pru-parser -i relative/path/to/input/file -o relative/path/to/output/file -v verbosity level (0 or 1) | $./run-pru-parser -i relative/path/to/input/file -o relative/path/to/output/file -v verbosity level (0 or 1) | ||
</syntaxhighlight> | </syntaxhighlight> | ||
== Output == | |||
During the parse, if verbosity is greater than 0, a status summaries will be printed: | |||
<syntaxhighlight lang="bash"> | |||
Starting pipeline ... | |||
Input policy [File reader]: file readout/testDataX288.bin size kxcd | |||
Filter policy [Forward]: buffer of kxcd byte(s) | |||
This parse had a speed of: kx.cd Mb/s | |||
Ouput policy [File writer]: write to file readout/pData.bin size kxcd | |||
... done | |||
</syntaxhighlight> | |||
Otherwise, only a the parse benchmark will be printed | |||
<syntaxhighlight lang="bash"> | |||
This parse had a speed of: kx.cd Mb/s | |||
</syntaxhighlight> | |||
Then a summary is printed: | |||
<syntaxhighlight lang="bash"> | |||
SUMMARY ------------- | |||
Total number of pRU frames parsed: n | |||
Total number of pRU empty words found: n | |||
Total number of pRU delimiter words found: n | |||
Unfinished pRU words in buffer: n | |||
</syntaxhighlight> | |||
If any pRU empty words are received a shot summary will be printed: | |||
<syntaxhighlight lang="bash"> | |||
EMPTY FRAMES FOUND: n | |||
</syntaxhighlight> | |||
If no errors are found, this will be printed: | |||
<syntaxhighlight lang="bash"> | |||
No errors found. | |||
</syntaxhighlight> | |||
Otherwise an error summary will be printed: | |||
<syntaxhighlight lang="bash"> | |||
INPUT_SIZE_ERROR: n, MULTIPLE_PRU_HEADER_ERROR: n, DATA_WORD_MISSING_HEADER_ERROR: n, | |||
TRAILER_WORD_MISSING_HEADER_ERROR: n, PRU_TRAILER_FLAGS_ERROR: n, NO_DATA_WORD_ERROR: n, | |||
FRAME_SIZE_ERROR: n, MISSING_CHIP_HEADER_ERROR: n, MISSING_REGION_HEADER_ERROR: n, | |||
MISSING_CHIP_TRAILER_ERROR: n, MULTIPLE_CHIP_HEADER_ERROR: n, ALPIDE_TRAILER_FLAGS_ERROR: n, | |||
PADDING_ERROR: n, UNKNOWN_ALPIDE_WORD_ERROR: n | |||
</syntaxhighlight> | |||
If any of the non fatal alpide trailer flags has been detected, a summary will be printed: | |||
<syntaxhighlight lang="bash"> | |||
ALPIDE Busy Transition: n | |||
ALPIDE Strobe Extended: n | |||
</syntaxhighlight> | |||
== Parser specification == | |||
=== I/O formats === | |||
=== Decoder === | |||
[[File:PRU Parser ALPIDE Decoder flow chart.jpeg|x400px]] | |||
=== Handling of format errors === | |||
* recovery | |||
* fatal errors | |||
* propagation/monitoring | |||
=== Generation of test data === | |||
Clone into the WP3 repository | |||
<syntaxhighlight lang="bash"> | |||
cd wp3/software/simulation | |||
pip3 install -r requirements.txt | |||
python3 test_cases.py | |||
</syntaxhighlight> | |||
cd to the desired test case folder. The different test cases are described in test_cases.py | |||
<syntaxhighlight lang="bash"> | |||
xxd -r -p testcase_pruformat.txt outputfilename.bin | |||
</syntaxhighlight> | |||
=== Known Issues === | |||
* Unable to analyze datastream if idle (FF) is thrown into the datastream, needs masking to handle this. (Testcase 5) | |||
** idle is masked by the RU firmware in normal operation | |||
** need to be handled as optional debug feature | |||
** extended error checking if error occurred | |||
* Mistakenly throws missing region header error when we have a frame consisting of only alpide header+trailer with alpide trailer set to b8, which should be a busy violation. | |||
* Parser reports more pru frame errors than there are frames when busies are thrown in randomly. Reports missing trailer errors when all trailers are present. | |||
* All frames with errors are discarded, should be saved somewhere. | |||
** currently all frames with errors are stored in a temporary internal buffer, needs to be saved to file | |||
** consider an error descriptor struct to describe frames in the raw buffer | |||
* Need more detail on ALPIDE trailer flags, the different errors should be stored along with the frame id somewhere. (Additional fields in .root format) | |||
* Random empty frames with/without compression leads to missing region headers. | |||
* No specific check for fatal or data overrun in ALPIDE trailer, which should be a critical warning, and the frame should be saved. | |||
* Conversion of data to .root format is only started after all the data is read out. This should be parallelized so that the conversion may start once the first pru-trailer is received. | |||
** implement after stable error handling is implemented |
Latest revision as of 09:23, 1 September 2020
Main Page -> Control & Readout Software Documentation and Howto's -> Running pRUParser
Software Function
The pRUParser takes a raw, binary, offload stream of pRU data from a file created by the pDTP Client. It will sort, error check and prunes this in to a raw, binary, stream and save this do disk for further prosessing. It will lastly print a summary of any errors it detected.
Protocol Dependencies
The pRUParser uses the pRU Data Format Specification v0.2
TODO add link to protocol document/page
Usage
In the readout/ folder in your build directory: For help run:
$./run-pru-parser -h
To parse a file, run:
$./run-pru-parser -i relative/path/to/input/file -o relative/path/to/output/file -v verbosity level (0 or 1)
Output
During the parse, if verbosity is greater than 0, a status summaries will be printed:
Starting pipeline ...
Input policy [File reader]: file readout/testDataX288.bin size kxcd
Filter policy [Forward]: buffer of kxcd byte(s)
This parse had a speed of: kx.cd Mb/s
Ouput policy [File writer]: write to file readout/pData.bin size kxcd
... done
Otherwise, only a the parse benchmark will be printed
This parse had a speed of: kx.cd Mb/s
Then a summary is printed:
SUMMARY -------------
Total number of pRU frames parsed: n
Total number of pRU empty words found: n
Total number of pRU delimiter words found: n
Unfinished pRU words in buffer: n
If any pRU empty words are received a shot summary will be printed:
EMPTY FRAMES FOUND: n
If no errors are found, this will be printed:
No errors found.
Otherwise an error summary will be printed:
INPUT_SIZE_ERROR: n, MULTIPLE_PRU_HEADER_ERROR: n, DATA_WORD_MISSING_HEADER_ERROR: n,
TRAILER_WORD_MISSING_HEADER_ERROR: n, PRU_TRAILER_FLAGS_ERROR: n, NO_DATA_WORD_ERROR: n,
FRAME_SIZE_ERROR: n, MISSING_CHIP_HEADER_ERROR: n, MISSING_REGION_HEADER_ERROR: n,
MISSING_CHIP_TRAILER_ERROR: n, MULTIPLE_CHIP_HEADER_ERROR: n, ALPIDE_TRAILER_FLAGS_ERROR: n,
PADDING_ERROR: n, UNKNOWN_ALPIDE_WORD_ERROR: n
If any of the non fatal alpide trailer flags has been detected, a summary will be printed:
ALPIDE Busy Transition: n
ALPIDE Strobe Extended: n
Parser specification
I/O formats
Decoder
Handling of format errors
- recovery
- fatal errors
- propagation/monitoring
Generation of test data
Clone into the WP3 repository
cd wp3/software/simulation
pip3 install -r requirements.txt
python3 test_cases.py
cd to the desired test case folder. The different test cases are described in test_cases.py
xxd -r -p testcase_pruformat.txt outputfilename.bin
Known Issues
- Unable to analyze datastream if idle (FF) is thrown into the datastream, needs masking to handle this. (Testcase 5)
- idle is masked by the RU firmware in normal operation
- need to be handled as optional debug feature
- extended error checking if error occurred
- Mistakenly throws missing region header error when we have a frame consisting of only alpide header+trailer with alpide trailer set to b8, which should be a busy violation.
- Parser reports more pru frame errors than there are frames when busies are thrown in randomly. Reports missing trailer errors when all trailers are present.
- All frames with errors are discarded, should be saved somewhere.
- currently all frames with errors are stored in a temporary internal buffer, needs to be saved to file
- consider an error descriptor struct to describe frames in the raw buffer
- Need more detail on ALPIDE trailer flags, the different errors should be stored along with the frame id somewhere. (Additional fields in .root format)
- Random empty frames with/without compression leads to missing region headers.
- No specific check for fatal or data overrun in ALPIDE trailer, which should be a critical warning, and the frame should be saved.
- Conversion of data to .root format is only started after all the data is read out. This should be parallelized so that the conversion may start once the first pru-trailer is received.
- implement after stable error handling is implemented