Application of Artificial Intelligence Algorithms Within the Medical Context for Non-Specialized Users: the CARTIER-IA Platform

The use of advanced algorithms and models such as Machine Learning, Deep Learning and other related approaches of Artificial Intelligence have grown in their use given their benefits in different contexts. One of these contexts is the medical domain, as these algorithms can support disease detection, image segmentation and other multiple tasks. However, it is necessary to organize and arrange the different data resources involved in these scenarios and tackle the heterogeneity of data sources. This work presents the CARTIER-IA platform: a platform for the management of medical data and imaging. The goal of this project focuses on providing a friendly and usable interface to organize structured data, to visualize and edit medical images, and to apply Artificial Intelligence algorithms on the stored resources. One of the challenges of the platform design is to ease these complex tasks in a way that non-AI-specialized users could benefit from the application of AI algorithms without further training. Two use cases of AI application within the platform are provided, as well as a heuristic evaluation to assess the usability of the first version of CARTIER-IA.


I. Introduction
A RTIFICIAL Intelligence (AI) algorithms have grown in popularity and increased its range of uses over the years. The possibility of applying them to different problems and contexts provide a wide support in complex scenarios in which data is continuously being generated.
One of these complex scenarios is the medical context. These algorithms and approaches are becoming very relevant when analyzing medical data [1]. However not only structured or tabular data can be involved in this context; medical imaging are also crucial resources within the medical domain.
The analysis of medical imaging involves complex tasks such as disease detection, segmentation, assessment of organ functions, etc. [2]- [4]. In this sense, artificial intelligence algorithms can provide support to these tasks with similar performance compared to human skills [5].
However, as introduced, data is being continuously generated in medical scenarios, which makes its management a convoluted responsibility. In fact, not only several data sources can be involved, but also different data structures. This data heterogeneity is a challenge both for its management and the application of AI algorithms.
Because of this, one of the main challenges of applying AI algorithms in real medical scenarios relies on the unification and accessibility of the generated data. For this reason, information systems are required to gather, clean, organize and structure data in order to apply AI algorithms in a friendly, secure and anonymized manner.
This work presents a platform for the management of structured data and imaging resources in the medical context with advanced features such as their visualization, edition and application of AI on the stored resources.
Powerful tools such as information dashboards can be easily integrated in the platform [6], [7] to explore structured data. This kind of tools provide support to knowledge generation, which is very relevant in this context [8].
On the other hand, DICOM editors and AI integration are also crucial components, which allow the modification and advanced exploration of imaging data.
The starting point of this project stems from the need of using a collaborative platform to gather these heterogeneous types of data. Unifying medical data sources through a collaborative platform eases their exploration and analysis, as well as enabling the possibility of sharing knowledge across different projects.
In this case, the platform was built for research purposes in the field of cardiology, but its flexibility enables its use for other fields in which structured data and medical imaging need to be unified.
In fact, the features of this platform can also provide support to educational purposes, in which the application of AI scripts is guided and explained to novice or non-specialized users [9], [10].
Relying on a web-collaborative platform also allows the integration of artificial intelligence algorithms. In fact, cardiac imaging is particularly interesting for the application of AI algorithms, since many tasks are related to the assessment of volumes, distances and motion of different structures in the heart and, in this regard, deep learning techniques have been proven to achieve good results [11].
Cardiac imaging is usually composed of DICOM (Digital Imaging and Communication On Medicine) files [12] from echocardiographic, magnetic resonance, or computed tomography, among the most important. Other online medical imaging platforms implement the DICOM protocol and are available for these purposes, some of them even in an open-source format [13].
On the other hand, there exist solutions based on application programming interfaces with pretrained models' repositories for use in the medical imaging field. Nevertheless, these repositories are mostly oriented towards advanced users with expertise in programming and data science knowledge [14].
As can be seen, different solutions arise to manage medical imaging and execute AI algorithms over them. However, in this scenario, it is necessary not only to unify data sources, but also these kinds of services.
For these reasons, the development of a technological ecosystem [15], [16] is an appropriate solution to merge both functionalities into a user-friendly web-based interface.
The CARTIER-IA platform can be seen as a technological ecosystem that support all data-management related tasks (including structured data and medical imaging collection) and also enable both healthcare professionals and data scientists to apply AI models to the stored images.
Deep learning and machine learning models can be stored by AI developers through Python scripts, as it will be detailed in section 2.C. Using this approach, scripts can be executed through the web interface by any user interested in analyzing the image, with the goal of providing the benefits of these scripts without requiring pythonprogramming skills nor advanced knowledge regarding Artificial Intelligence algorithms.
In this respect, given the fact that not every user is skilled in programming AI algorithms, the platform needs a user-centered approach to provide friendly interfaces and bring AI-driven tasks closer to non-specialized users.
In this work we present the integration of AI algorithms into the CARTIER-IA platform's DICOM viewer and editor. A heuristic evaluation of the tool is provided to test its usability and improve the image processing and AI application workflow with the goal of offering better user experience.
The structure of this paper is as follows: section II outlines the technical details of the platform as well as the heuristic evaluation methodology, section III explains the main functionality blocks of the CARTIER-IA platform, section IV describes two use cases of the AI integration within the platform and section V provides the heuristic evaluation results regarding the image editor and script application tool. Finally, section VI and section VII discuss the results and present the conclusions, respectively.

A. Technical Details of the Platform
The platform relies on different technologies and frameworks which are integrated using a client-server architecture.
The front-end employs HTML, CSS, and JavaScript to send data to the server. On the other hand, the DICOM viewer and editor is also located at the front end, and it is implemented through the Cornerstone.js library.
On the other hand, the back end performs more complex tasks to fulfil the requirements of the platform, such as the data storage, data processing and an Artificial Intelligence environment.
The technology employed to implement this client-server approach as a web application is Django, a Python-based web framework [17]. The web application is also connected through web requests to other services such as a REDCap instance to manage additional projects and information.
Due to the necessity of pre-validate DICOM images and structured data, upload processes can be time-consuming tasks. For this reason, job queries have been implemented to carry out these data uploads asynchronously as background jobs. This allow users to navigate the platform while their data is uploading.
Finally, to implement the integrated AI environment, the back end is supported by libraries such as OpenCV and TensorFlow, in order to enable the execution of deep learning models and other AI-related scripts.

B. Usability Study: Heuristic Evaluation
Integrating complex tasks such as AI algorithms in a web interface in which other diverse functionalities are involved is a challenge, especially regarding providing a good user experience.
For this reason, the platform needs to be thoroughly tested in terms of its usability. One of the preliminary studies that has been carried out to identify interface design weaknesses in the platform's first version is a heuristic evaluation.
Although there are several heuristics sets to perform heuristic evaluations, the most popular are the ten heuristics by Nielsen [18]. There are also specific heuristics related to the medical domain, but there aren't focused on this kind of platforms (they are mostly related to the evaluation of Electronic Health Records [19], [20]).
However, due to the fact that CARTIER-IA platform is mainly focused on research tasks, image edition and AI algorithms application, the previous heuristics are not the best fit for this usability study.
For these reasons, the Nielsen's heuristics were the selected instrument to perform the heuristic evaluation on the CARTIER-IA platform. This set is composed of ten heuristics, which are listed below [18]. HR1: Visibility of system status.
HR2: Match between system and the real world. A total of six experts were involved in the heuristic evaluation. Four of these experts were HCI experts (web developers and researchers), and two of them both HCI experts and domain experts (a Ph.D. student and clinical data scientist) [21]. In fact, these double experts had used the CARTIER-IA platform as users before performing the heuristic evaluation.
The heuristic evaluation was carried out using a template with guidelines to support the evaluation and issues' reporting. Each evaluator had only access to his/her own report, in order to avoid biases. The evaluation template had three fields to collect the evaluator's name, the name of the tool evaluated, and the browser that they employed to access the platform.
Finally, the template provided a table with three columns (heuristic name, score from 1 to 10 and problems detected) and one row per problem detected within each heuristic.

A. Data Collection
As introduced in section I, one of the motivations of developing the platform is to unify data from different sources and arrange them into a more friendly structure. Due to this requirement, the CARTIER-IA platform provides two types of data upload processes.
First, a structured data uploader. The platform allows users to upload spreadsheets of data at different levels, containing information associated to patients, image studies or files. The platform also supports longitudinal structures (repeated measurements or data for the same patient over time).
In addition, data schemas are flexible to vary among different projects, so a project might contain a structured data schema completely different from another.
This flexibility is accomplished through the Django ORM (a database-abstraction API). The Django ORM API provides access to a relational database with the structure shown in Fig. 1. The platform is mainly organized through projects, which will hold data from different patients. Structured data is stored as JSON object at different levels (patient, study or file), which provides the support to modify the data schemas across projects. On the other hand, the platform provides a second data uploader; an image data uploader. This uploader allows users to upload a set of image studies through compressed files. The service handles the DICOM format, read some of the metadata tags, check the integrity of data anonymization of each file and validates the metadata against already stored structured data, linking them if applicable.
In this regard, imaging data relies on three entities: studies, series, and files. These entities provide the structure to manage data about the DICOM images that will be uploaded to the platform.
The image files are referenced through these entities and stored through file systems. The type of file system can be modified, allowing flexibility in the selection of the storage technology.
Finally, the platform integrates an external tool to make the data management more powerful. A REDCap instance is connected to the platform to enable the importation of its data, thus providing another layer of data unification. REDCap (Research Electronic Data Capture) is an electronic data capture (EDC) software and workflow methodology for creating and designing clinical research databases [22].

B. Image Edition
Another main functionality of the platform is its image editor. When DICOM images are uploaded, users can explore them more closely through this tool. One of the benefits of this image editor is that is fully integrated within the platform, so it is not necessary to use external tools to carry out image modifications. This is possible because, as explained in section II, image edition takes place in the browser through the Cornerstone.js (https://github. com/cornerstonejs/cornerstone) framework, an open-source library to parse and render DICOM files.
Thanks to this approach, users can edit the images they are currently exploring, and decide later if they can make these annotations and modifications persistent.

C. Artificial Intelligence Support
The feature in which this paper is focused is the Artificial Intelligence integration within the platform. This feature has two main motivations: • To offer the benefits from Artificial Intelligence algorithms in situ, without the necessity of leaving the platform to applying these algorithms

• To provide a friendly interface to apply AI scripts and open their use to non-specialized users
This feature allows researchers to upload their AI scripts into the platform and make them available to other users. Only researchers with privileges can add new scripts, which need to be thoroughly tested by the corresponding researcher before integrating them into the platform to ensure a reliable functionality.
The platform also provides an uploader to define the algorithm's meta-data. In this case, algorithms' meta-data is highly important to properly integrate the scripts within the platform. These meta-data provide information about the algorithm's output (a modified image, a set of measures, a segmentation mask, etc.), its applicability (as their application might be limited to specific DICOM modalities) or other parameters depending on the output.
It is important to clarify that the algorithms need to be pre-trained before their integration into the platform. For this reason, the uploader also provides a field to upload the exported model or the models' weights depending on the type of AI algorithm employed.
To sum up, to integrate an algorithm into the platform it is necessary to provide the pre-trained model, and the script that makes use of the pre-trained model with the goal of enabling their invocation by the platform's AI module.
Once an algorithm has been integrated, it will be available at image editor. To apply an algorithm, the user just needs to click the AI button and select one of the available scripts for the current image being displayed. When the user confirms the application, the platform will yield the result which, depending on the algorithm's output type, could result in displaying a new image, an inferred diagnosis or the addition of AI-driven measurements as new structured data.

IV. Use Cases
This section provides two application uses of the AI integration within the CARTIER-IA platform as an example of how the platform behaves when dealing with different types of AI algorithms.

A. Manual vs. Artificial Intelligence Segmentation
Segmentation of medical images is a relevant procedure within the field of medical image processing. Its ultimate goal is to identify different elements and features in medical images to detect abnormalities or other characteristics of interest.
For this reason, one of the most relevant features of the image viewer is the possibility of performing the segmentation of the stored DICOM images in place.
As explained in the previous section, the image viewer relies on different tools to provide a complete set of image processing functionalities. Among them, the platform offers different brushes to perform image segmentations manually and store them as JSON objects that can be retrieved and further processed.
But along with the manual segmentation, researchers can integrate deep learning models whose outputs are automatically generated segmentations. To do that, users can select among the available AI algorithms in the platform and simply confirm their choice (red rectangle in Fig. 2). The algorithm choice is processed in the back end, which consults the algorithm's meta-data and, depending on the output, performs different actions. In this case, the output is a segmented image, so this result is sent back to the client and displayed in the viewer next to the original image (Fig. 2).
This interface organization allows users to compare their manual segmentation with the algorithm's result, which could provide new information or assist the user with their own image segmentation. And it can also be used in the reverse scenario. If it is a trained physician or technician who performs the manual segmentation, this interface could be used to improve the artificial intelligence algorithm by active learning.

B. Measurements
The application of AI scripts is not only limited to image segmentation. As explained throughout this work, the platform manages both imaging data and structured data. Structured data also provides crucial information regarding patients and their monitoring, diagnoses, treatments, etc.
In this context, the CARTIER-IA platform also supports the execution of AI algorithms that, based on the input image or even input structured data, return a dataset containing new inferred information. The algorithm's results are persistently stored along with the rest of the patient's, study's or file's structured data, making them available for other users when exploring the project.
The process to apply these kinds of algorithms is exactly the same. However, in this case, instead of returning a new image, the back end executes the algorithm, stores the newly generated variables and sends a confirmation to the client (Fig. 3).
After the confirmation, users can see the measures yielded by the algorithm at the specified level. Fig. 4 shows a new variable generated by the algorithm "ai_DummyECO-script", which is stored under a new category with the format "<algorithm_name>_vars" to differentiate them from the original study variables.

V. Heuristic Evaluation Results
Each expert was identified by a number (E1, E2, E3, E4, E5, E6) in order to present the outputs of the heuristic evaluation. The heuristic evaluation was performed on the whole platform, but only the DICOM editor and AI tool-related issues are being discussed given the focus of this work. Fig. 5 shows the total average value assigned to the problems identified under each heuristic. Values close to 1 indicate that experts detected non-relevant issues, and values close to 10 implies that the issues are relevant and severe. A zero value represents that experts did not identify any problems in that heuristic. Not only is the severity of the problems important, but also the absolute number of issues to solve in each heuristic (Fig. 6). The heuristics with the largest number of usability issues identified for the DICOM viewer and AI tool were HR1 (Visibility of system status) and HR4 (Consistency and standards), both with 5 problems  detected. The average severity scores for these heuristics are 9.4 and 8.6, respectively.
The HR1 problems are mainly related to the absence of progress bars and the actions that can take place within the image editor, which are not clearly explained.
Regarding HR4-related issues, they are focused on the correctness of the metaphors used for the icons that represent each functionality button. In addition, some misfunctions on these image processing functionalities were also identified.
On the other hand, there are three heuristics that also obtained high severity ratings: HR3 and HR9 with a score of 9 and HR8 with a score of 10.
Regarding HR8 (Aesthetic and minimalist design), this high score is due to the fact that only one issue was encountered within this category (also related with the DICOM editor's icons), but E6 assigned a score of 10 because of its relevance. Specifically, this issue pointed out the great quantity of icons employed for the editor toolbar and their difficulty to clearly convey their meaning.
Only one issue was identified under HR3 (User control and freedom) but also with a high severity rating (9). In this case, E6 identified the impossibility of undo or redo actions taken place within the tool.
Four issues were identified in the HR9 (Help users recognize, diagnose, and recover from errors), obtaining a score of 9, too. In this case, experts identified the lack of information when an AI script or a DICOM image fails and the impossibility of recovering from this kind of errors. This heuristic also includes the issue that the DICOM editor does not support the reset of the modifications made on the images.
Finally, a lower number of issues were encountered in the rest of heuristics: • Better explanations regarding the editor's functionalities (HR2 and HR6) • Better explanations regarding the results yielded by the functionalities supported on the editor (HR6) • Keyboard shortcuts and AI integration for advanced users (HR7) • Lack of documentation, specifically regarding the AI tool, which could be complex to understand (HR10)

VI. Discussion
The heuristic evaluation identified different design issues regarding the image editor and AI algorithms' application. These usability evaluations are crucial to iteratively provide more robust and friendly interfaces to perform complex tasks such as the ones supported by the CARTIER-IA platform.
The results derived from the heuristic evaluation shown very high scores. This is due the great relevance that experts gave to usability in the image editor tool. The image editor is a powerful component of the CARTIER-IA platform, because not only provides edition functionalities, but also is the integration point for applying AI algorithms. For these reasons, offering good user experience in the image editor interface is crucial, and thus every usability issue encountered has high relevance.
The majority of issues were related to the toolbar, which relies on several icons to depict the supported functionalities. However, these icons were not very clear to the experts. In addition to this topic, some experts also pointed out the necessity of explaining the functionalities more thoroughly, especially the AI algorithms.
In this version of the platform, algorithms are listed in the interface and the user can apply them directly. However, only the names of the algorithms are displayed, which can be confusing, as the algorithm's name could provide little or no information at all regarding its outputs and results.
One of the design parameters for the AI integration was to make the application process straightforward both for skilled and nonskilled users. However, simplifying too much this process can also have drawbacks. Non-skilled users could question the algorithms' outputs if there are no further explanations regarding the process nor the interpretation of the results, because the AI tool works as a black box in its current version.
It is crucial to find balance between implementing a simple interface but also displaying enough information to understand the actions carried out within the platform.
Providing user-friendly interfaces in the health domain could make convoluted tasks more straightforward and thus, save time for physicians. As it has been shown, this kind of interfaces could also bring closer tasks for which users are not specialized nor trained (such as AI algorithms programming).
Another important benefit from integrating AI algorithms in a medical data management platform is that the trained models can be improved. Although in its current version the platform only provides an interface for executing AI algorithms (because models need to be pre-trained before their integration into the platform), this approach sets the foundations for future improvements, including the possibility of training AI models directly from the platform.
For example, manual segmentations can be carried-out by the users within the platform, which results in new data to train the existing models. On the other hand, users can also label the algorithms' outputs depending on their performance, thus laying the foundations for improving the models through active learning.
On the other hand, there is room for improvement regarding the algorithms' validation. Currently, researchers are responsible of the validation of their scripts, but another validation layer can be implemented to analyze and test these scripts automatically before carrying out the integration. The metrics obtaining from the testing of the scripts could complement the information of each algorithm to generate more confidence regarding the platform's AI support.
Finally, we want to mention that a heuristic evaluation does not ensure identifying all the problems that could affect a real user in a real context while using the platform. There are studies that point out that the problems detected by experts are not necessarily the actual problems that will affect the end users of the platform [23]. To alleviate this limitation, we included one expert with extensive knowledge of the platform's domain, although subsequent research will explore usability from the researchers and physicians' point of view.

VII. Conclusions
This paper presents a collaborative platform for the management of medical data and imaging. The platform has several features to provide support for a variety of functionalities, such as a DICOM viewer and editor. Among these tools there is the possibility of integrating artificial intelligence scripts to make the application process straightforward to non-specialized users.
Given the implication of all these features within the platform, a heuristic evaluation has been carried out to identify usability issues of the DICOM viewer and AI algorithms' integration in the current version of the platform. This evaluation gives hints on the aspects that need to be improved to provide better user experience to researchers and physicians.
Future research lines will involve the resolution of every usability issue identified, as well further usability tests including other techniques such as the PSSUQ questionnaire or usability labs.