PRETUS: A plug-in based platform for real-time ultrasound imaging research

We present PRETUS — a Plugin-based Real Time UltraSound software platform for live ultrasound image analysis and operator support. The software is lightweight; functionality is brought in via independent plug-ins that can be arranged in sequence. The software allows to capture the real-time stream of ultrasound images from virtually any ultrasound machine, applies computational methods and visualizes the results on-the-fly. Plug-ins can run concurrently without blocking each other. They can be implemented in C++ and Python. A graphical user interface can be implemented for each plug-in, and presented to the user in a compact way. The software is free and open source, and allows for rapid prototyping and testing of real-time ultrasound imaging methods in a manufacturer-agnostic fashion. The software is provided with input, output and processing plug-ins, as well as with tutorials to illustrate how to develop new plug-ins for PRETUS.

0 and 200 can be set using the command line argument -filemanager_framerate. Second, if the mhd headers have the field AcquisitionFrameRate, then this value will be used, and can be different for each image. Additional options allow the last image to loop around when it is read or to ignore the header information.
This plug-in provides a widget that allows to scroll through the images rapidly and to play/pause the streaming. In pause mode, the same image keeps being transmitted at the default framerate, allowing the rest of the plug-ins to continue operating on the paused frame.

A.2. Video manager plug-in
This plug-in allows a video file to be read from the file system and transmits it through the pipeline. Opencv is used to read the video files so supported format depends on local configuration of opencv.
The video by default loops around when finished, but this can be disabled by the user using the command line argument -videomanager_loop 0. The video starts from the beginning by default, but an arbitrary start time can be set with -videomanager_start_time <mm:ss>. The video can also be played faster by setting a fast-forward factor with -videomanager_ff <factor>. This plug-in's widget enables interactively moving around in the video with a slider and to play/pause the streaming. In pause mode, the same video frame keeps being transmitted at the default frame-rate, allowing the rest of the plug-ins to continue operating on the paused frame.

A.3. Frame grabber plug-in
This plug-in allows a stream of images to be received in realtime from a video source, such as the video output of an ultrasound system, by using the Epiphone DVI2USB3.0 frame grabber (https://www.epiphan.com/products/dvi2usb-3-0/). The plug-in is currently implemented to convert the images to grayscale and pass it on to the rest of the pipeline as a single channel, 8 bit images.

A.4. Cpp algorithm plug-in
This plug-in performs a simple binary thresholding on the input image. The plug-in is conceived as a tutorial to illustrate how to develop C++plug-ins for PRETUS.
The Cpp Algorithm plug-in performs the thresholding operation using the ITK library. The threshold value can be set via command-line argument (cppalgorithm_th <th>) and edited in real-time using the slider in the plug-in's widget. An overlay of the input image and the thresholded image are shown on the plug-in's image widget.

A.5. Python algorithm plug-in
This plug-in performs a Gaussian blur on the input image. The plug-in is conceived as a tutorial to illustrate how to develop Python plug-ins for PRETUS.
The Python Algorithm plug-in performs the Gaussian blur operation using the SimpleITK Python library. The sigma value for the Gaussian kernel can be set via command-line argument (pythonalgorithm_sigma <sigma>) and edited in real time using the slider in the plug-in's widget. The blurred version of the input image is shown on the plug-in's image widget. The plugin's worker waits a user-defined time (within the Python code) to simulate a longer task execution.

A.6. Standard plane detection (SonoNet)
This plug-in implements the fetal scan plane detection method described in [13]. The plug-in runs the method in every frame received from the input stream (which can be selected by the user).
The algorithm yields a 13-element vector with a score indicating the probability of the image belonging to each class above. The plug-in packs this information into four fields in the output image header: • ''Standardplanedetection_labels'', a string array with the original class labels in order.

A.7. Image file writer plug-in
This plug-in allows images to be written to file, in realtime. The plug-in can write images from any stream, or multiple Streams, or all. Each received frame is written as a single image in mhd/raw format, which is well supported by imaging libraries This plug-in handles the header field ''DO_NOT_WRITE'' by not writing to file any image that has that key in the header, even if the image belongs to a stream that is being written. This allows other plug-ins to transmit images for visualization or for other plug-ins but not write them to file. For example, this is useful in the standard plane detection plug-in, where the user may not want to write the 'background' images to file, but still wants to visualize them in real time.
This plug-in implements a widget that shows the number of images that have been saved and allows to stop/resume the image saving via a checkbox.

A.8. GUI plug-in
The graphical user interface (GUI) plug-in is designed to display a stream of images and widgets around the images with information of the other plug-ins in the pipeline. The organization of the visualization window in shown in Fig. A.4.
All plug-ins can implement two types of widgets, declared in the Plugin parent class: plug-in widgets, that can be placed in panels B or C in the figure, and image widgets, that can be placed in panel A. By default, the GUI plug-in creates a colored frame around each widget that matches a colored frame around the image widget of the same plug-in (if available), as shown in Fig. 2. This can be disabled with the command line argument --gui_usecolors 0.
The GUI plug-in itself implements a widget (by default located in panel B) that allows to control the size of all image widgets.