Extensions

This section explains the types of Kaapana extensions and how they work. For descriptions of available workflow and application extensions, refer to the Workflows and Applications. To learn how to integrate custom components into the platform as extensions, refer to the Developing Applications and Developing Workflow Extensions.

The Extension functional unit in Kaapana serves as an app store. It allows users to install/uninstall applications, workflows, and even platforms (experimental feature). Technically, an extension is a Helm chart.

Each extension in the Kaapana repository consists of two folders: docker and <extension-name>-chart. For more information about the file structure, refer to the Helm Charts section Advanced: How Kaapana uses Helm.

There are two types of extensions:

Workflow Extensions: Consist of single or multiple executable DAGs in Apache Airflow. After installing a workflow extension, you can see the DAGs available under Workflow Execution menu.
Applications: These provide additional functionalities such as opening a VS Code server, a JupyterLab notebook, or an MITK Workbench instance.

In addition to the distinction in type, there is also an distinction in maturiy, namely stable or experimental. Stable extensions have been tested and maintained, while experimental extensions are not. The filters on the Extensions page allow users to filter extensions based on type, maturiy, and hardware requirements, i.e. GPU or CPU. The extension list is updated in real time based on the selected filters. The Extensions page also displays the current Helm and Kubernetes status of each extension, such as Running, Completed, Failed, or Error.

Note

Kaapana supports multi-installable extensions, which will have a “Launch” button instead of “Install”. Each time a multi-installable extension is launched, it is deployed as a separate Helm release.

Hint

To install a specific version of an extension, use the dropdown in the version column.

The section Developing Workflow Extensions also explains how to write and add your own extensions.

Uploading Extensions to the Platform

Kaapana also provides an experimental upload component for extensions. This allows users to upload both Docker images and Helm charts to the platform. Currently, this component only accepts two file types: “.tar” for exported images and “.tgz” for Helm charts.

This feature is intended to be used by developers who have knowledge about configuring Helm charts and Kubernetes resources. It is strongly recommended to read the following sections before uploading anything to platform: Advanced: How Kaapana uses Helm and Writing Dockerfile

Chart Upload:

Uploaded chart files are checked for basic safety measures, such as whether they are running any resource under the admin namespace.
To create a zipped chart file that can be uploaded to Kaapana, run the following Helm command inside the chart folder:

helm dep up
helm package .

Hint

If the build step is already completed, all the chart tgz files -and their respective folders- should be available under kaapana/build/kaapana-admin-chart/kaapana-extension-collection/charts. The structure should be the same with the DAGs and services already available there.
For any Kubernetes resource yaml inside the templates folder (i.e. deployment, job), the image tag should be referenced correctly (example field that needs to be changed).

Image Upload:

To save an image as a .tar file, use docker or podman.
Uploaded images are automatically imported into the microk8s ctr environment (for details see the images import command here) .
A useful command to check if the image is imported with the correct tag into the container runtime is microk8s ctr images ls | grep <image-tag>

Hint

Since the images uploaded via this component are not already available in a registry, the imagePullPolicy field in the corresponding Kubernetes resource yaml files (example value to be changed) should be changed to IfNotPresent.

If you have any issues regarding the upload mechanism, check Container Upload Failed.

Extension Parameters

Introduced in version 0.2.0, Extensions support specifying parameters as environment variables. This functionality can be customized according to the requirements of the extension. Some examples of available parameters are task_ID`s for **nnUNet** and the :code:`service_type` field for MITK Workbench. Parameters can be of type string, boolean, single_selectable, or multi_selectable. Parameters should be defined in the values.yaml file of the chart. Each of them should follow this structure:

extension_params:
  <parameter_name>:
    default: <default_value>
    definition: "definition of the parameter"
    type: oneof (string, bool, list_single, list_multi)
    value: <value_entered_by_the_user>

Workflows

This is a list of built-in workflow-extensions, that can be installed.

Note

The list of executable workflows in the Workflow Execution view is only refreshed once every minute. This interval is configurable as the parameter dag_dir_list_interval in the file airflow.cfg.

classification-inference

Workflow Overview

A classification inference pipeline based on a trained model is executed.

1) DICOM data is fetched from the PACS
2) DICOM data is converted to .nifti
3) Images are preprocessed: resampled and normalized
4) The model runs inference on the input data
5) DICOM SEG objects are sent to the internal platform PACS

Input data:

A trained classification model checkpoint

classification-training

Workflow Overview

A classification training pipeline based on ResNet18 is executed. The labels should be custom tags that can be added in the Datasets view.

1) DICOM data is fetched from the PACS
2) DICOM data is converted to .nifti
3) Images are preprocessed: resampled and normalized
4) The model is trained on the input data
5) DICOM SEG objects are sent to the internal platform PACS

Input data:

A dataset with DICOM images and tags that represent the labels for classification. The tags should be added to the dataset in the Datasets view.

body-and-organ-analysis

Method: “BOA: Body and Organ Analysis”
Authors:  Haubold, J., Baldini, G., Parmar, V., Schaarschmidt, B. M., Koitka, S., Kroll, L., van Landeghem, N., Umutlu, L., Forsting, M., Nensa, F., & Hosch, R.
Cite as: Haubold, J., Baldini, G., Parmar, V., Schaarschmidt, B. M., Koitka, S., Kroll, L., van Landeghem, N., Umutlu, L., Forsting, M., Nensa, F., & Hosch, R. (2023). BOA: A CT-Based Body and Organ Analysis for Radiologists at the Point of Care. Investigative radiology, 10.1097/RLI.0000000000001040. Advance online publication. https://doi.org/10.1097/RLI.0000000000001040
Code: https://github.com/UMEssen/Body-and-Organ-Analysis/tree/main

Workflow Overview:

Runs inference on the CT images using all selected models. Submodels associated with the “total” model are only executed if they are explicitly selected and the “total” model is selected. For each input series, a dedicated workflow is started. For more information, check out their repository.

Converts the DICOM input to a NIfTI file.
Executes the BOA command line tool on the input image.
Verifies the outputs:
- Checks for invalid results, such as images with only background.
- If strict mode is enabled, all expected outputs must be present to continue.
Converts all NIfTI segmentation results into a single DICOM SEG file.
Uploads the results:
- The DICOM SEG file is uploaded to the PACS.
- All other output files are uploaded to MinIO at <project-bucket>/body-and-organ-analysis/<dicom-series-uid>.

nnunet-predict

Method: “Automated Design of Deep Learning Methods for Biomedical Image Segmentation”
Authors:  Fabian Isensee, Paul F. Jäger, Simon A. A. Kohl, Jens Petersen, Klaus H. Maier-Hein
Cite as: arXiv:1904.08128 [cs.CV]

Workflow Overview

A nnU-Net inference is executed.

1) Model is downloaded
2) DICOM is converted to .nifti files
3) The model runs inference on the input data
4) Segmentations are converted to DICOM Segmentation (DICOM SEG) objects
5) DICOM SEG objects are sent to the internal platform PACS

Input data:

A trained nnU-Net model that is already installed via nnunet-install-model workflow.

nnunet-training

Method: “Automated Design of Deep Learning Methods for Biomedical Image Segmentation”
Authors:  Fabian Isensee, Paul F. Jäger, Simon A. A. Kohl, Jens Petersen, Klaus H. Maier-Hein
Cite as: arXiv:1904.08128 [cs.CV]

Workflow Overview

A nnU-Net training is executed.

1) Segmentation objects are downloaded
2) Referenced DICOM images are downloaded
3) DICOM images are converted to .nifti files
4) Segmentation masks are converted to .nifti files
5) If specified in input form, segmentation masks are filtered based on keywords “Keep: <label>” and “Ignore: <label>”
6) If specified in input form, multiple labels are fused into a new label
7) If specified in input form, label are renamed
8) The segmentation masks are evaluated for overlapping segmentations and if the overlap is above a certain threshold, the segmentation object is removed from training data
9) nnU-Net training is planned and training data is preprocessed
10) The actual training is executed
11) The trained model is zipped
12) The zipped model is converted to a DICOM object
13) The DICOM object is sent to the internal platform PACS
14) A training report is generated
15) The model and training logs are uploaded to Minio
16) The training report is uploaded to a location, where it can be rendered by a static website
17) The training report is converted to a DICOM object
18) The DICOM object is sent to the internal platform PACS

Input data:

Segmentation objects. Please avoid overlapping segmentations and specify the segmentation labels that you want to use for the training in the SEG field.

nnunet-install-model

Method: “Automated Design of Deep Learning Methods for Biomedical Image Segmentation”
Authors:  Fabian Isensee, Paul F. Jäger, Simon A. A. Kohl, Jens Petersen, Klaus H. Maier-Hein
Cite as: arXiv:1904.08128 [cs.CV]

Workflow Overview
Models that are stored as DICOM files the internal PACS are installed into the models directory of Kaapana.
Installed models can be used in nnunet-predict workflow.

Input data:

Dataset that stores nnunet models as DICOM files. If the dataset contains any modality other than OT, the workflow will fail. Use the Datasets view to filter for the right model.

nnunet-uninstall-models

Method: “Automated Design of Deep Learning Methods for Biomedical Image Segmentation”
Authors:  Fabian Isensee, Paul F. Jäger, Simon A. A. Kohl, Jens Petersen, Klaus H. Maier-Hein
Cite as: arXiv:1904.08128 [cs.CV]

Workflow Overview

Installed models inside the models directory of Kaapana are uninstalled. Models still persist in the internal PACS as DICOM files.

Input data:

Installed model name.

TotalSegmentator

Method: “TotalSegmentator: robust segmentation of 104 anatomical structures in CT images”
Authors:  Wasserthal J., Meyer M., Breit H., Cyriac J., Yang S., Segeroth M.
DOI: 10.48550/arXiv.2208.05868
Code: https://github.com/wasserth/TotalSegmentator

Workflow Overview
1) Model is downloaded
2) DICOM is converted to .nrrd files
3) TotalSegmentator with all its subtasks is applied to the input data
4) .nrrd segmentations will be converted to DICOM Segmentation (DICOM SEG) object
5) DICOM SEGs will be sent to the internal platform PACS

Input data:

Any CT scans.

TotalSegmentator v2

Method: “TotalSegmentator: robust segmentation of  117 (CT) and 50 (MR) body structures”
Authors:  Wasserthal J., Meyer M., Breit H., Cyriac J., Yang S., Segeroth M.
DOI: 10.48550/arXiv.2208.05868
Code: https://github.com/wasserth/TotalSegmentator

Workflow Overview
1) Model is downloaded.
2) DICOM is converted to NIfTI files
3) TotalSegmentator with selected its subtasks is applied to the input data
4) NIfTI segmentations will be converted to DICOM Segmentation (DICOM SEG) object
5) DICOM SEGs will be sent to the internal platform PACS

Input data:

Any CT or MR scans.

Automatic organ segmentation (shapemodel-organ-seg)

Method: “3D Statistical Shape Models Incorporating Landmark-Wise Random Regression Forests for Omni-Directional Landmark Detection”
Authors:  Tobias Norajitra and Klaus H. Maier-Hein
DOI: 10.1109/TMI.2016.2600502

Workflow Overview
1) DICOMs are converted to .nrrd files
2) Input images are normalized
3) Parallel segmentation of liver,spleen and kidneys (left and right)
4) .nrrd segmentations will be converted to DICOM Segmentation (DICOM SEG) object
5) DICOM SEGs are sent to the internal platform PACS

Input data:

Filter for abdominal CT scans using the Datasets view.

Radiomics (radiomics-dcmseg)

Workflow Overview
1) Selected DICOM SEGs are converted to .nrrd files
2) Corresponding CT file is downloaded form the PACS
3) Downloaded CT files are converted to *.nrrd
4) Radiomics is applied on selected DICOMs
5) Extracted radiomics data are pushed to the bucket radiomics in Minio and can be downloaded there

Input data:

DICOM Segmentations

MITK Flow

Workflow Overview
1) A MITK instance is launched within a noVNC application
2) Access the noVNC application with MITK running through the Pending applications page
3) In MITK, load the first task from the Kaapana Task List Load task 1/x
4) Modify or create segmentations
5) Accept the segmentations by clicking Accept segmentation. Only accepted segmentations are stored
6) Load the next task
7) After completing manual interactions, click Finish Manual Interaction on the Pending applications page. Newly created segmentations are uploaded to the PACS

Notes:
The mitk-flow workflow aims to generate segmentations using MITK tools.
Inside the initialized MITK application, a task list is created, containing all series selected in the workflow. Depending on the input data, there are two possibilities to create new segmentations:

1) If the input data is an image series, a new segmentation can be directly created.

2) If a segmentation is selected as input data, the corresponding image and segmentation are preloaded. The modified segmentation is then stored as a new segmentation in the PACS.

Once you have completed your work with all series (all tasks are done), all accepted segmentations will be sent to the PACS upon finishing the manual interaction.
In the datasets view, the segmentations are tagged as “MITK-flow”.
If no segmentations were created or no project was saved, the workflow will fail because the DcmSendOperator fails when no data is sent.

Input data:

DICOMs

Task API workflows

register-dicoms
1) Download dataset with the target image
2) Download dataset with the moving images that will be registerd to the target image
3) Convert dicom images to nrrd
4) Register moving images to the target image
5) Convert registered images from nrrd to dicom
6) Send registered images to the internal CTP server

test-task-operator:

Minimal workflow to test the Task API and the KaapanaTaskOperator

Applications

Code server for Airflow

Purpose

The Code Server is used for developing and debugging Airflow DAGs and operators. It runs a VS Code–compatible server inside a container and mounts the workflows directory of Kaapana.

Mount point:

<fast_data_dir>/workflows

Capabilities:
- Edit DAG files directly inside the Code Server.
- Access the full workflow directory, including retained workflow data and downloaded models/scripts.
- Inspect DAG definitions and troubleshoot failed workflows without starting an operator-specific Code Server.
- Add and execute small scripts on the mounted data for testing and debugging.
- Not intended for running production workloads.

Logs:

Logs are not directly accessible from within the Code Server. Use the Kaapana platform UI or the Airflow UI to view log output.

VS Code settings:

If you want to use your custom VS Code settings inside the Code Server, save them under /kaapana/app/.vscode/settings.json.

Code server for development

This extension provides a VS Code Server environment within the Kaapana platform for Python and R development. Users can explore and analyze data directly within the platform.

Overview

This extension is designed for users who need an interactive coding environment similar to JupyterLab, but with the additional flexibility of VS Code. It utilizes code-server to bring VSCode in the browser. Note that this extension is intended to be used with online deployments. If the container can not access the internet, an OFFLINE_WARNING.txt will appear next to this README.md file. The environment includes:

Python 3 and R support
Preinstalled Python, R, and Jupyter extensions in VSCode
Access to the project’s MinIO storage
Ability to install additional Python or R packages

Preinstalled Components

Pyhon Environment
- Python version: 3.x (installed via Ubuntu python3-full)
- Virtual environment path: /opt/venv
Installed python packages
- numpy
- pandas
- scipy
- matplotlib
- jupyterlab
- torch, torchvision, torchaudio
R Environment
- Installed from the official CRAN repository
- Includes the languageserver package for IDE integration
VS Code Extensions
- ms-python.python
- ms-toolsai.jupyter
- REditorSupport.r
- ms-toolsai.jupyter-keymap
- ms-toolsai.jupyter-renderers

Project Storage and Data Access

You can launch multiple instances of Code-Server simultaneously for different usecases. Upon launching of a new application you can specify a directory within the project-bucket in MinIO that is periodically synced to the file-system of the Code Server at /kaapana/minio/. You can also specify a comma-separated list of patterns that should be excluded during the syncing process. The sync interval is 20 seconds. If a file is simultaneously changed in MinIO and Code-Server, rclone will resolve this conflict during the next syncronisation.

It does not overwrite either side.
It renames one version with a .conflict suffix.
The file changed in MinIO keeps the original filename.
The file changed in Jupyterlab gets the conflict rename.
For more information check the rclone manual.

Known Limitations of The Environment

JupyterLab VSCode Extension works as expected in Firefox. In Chrome, Jupyter notebook cells may fail to render with a Service Worker SSL error:
```
Could not initialize webview: Error: Could not register service worker:
SecurityError: Failed to register a ServiceWorker for scope (...)
```
This is a known issue with code-server: https://github.com/coder/code-server/issues/3410
The Jupyter extension does not automatically select the correct Python environment, therefore the users must manually choose /opt/venv/bin/python as the interpreter each time they open a notebook.
The AI chat/agent sidebar introduced in newer code-server versions cannot currently be turned off via settings. Even with configuration options such as:
```
"chat.disableAIFeatures": true,
"chat.agent.enabled": false
```
the panel still appears in the beginning and must be manually closed. This is a known issue of code-server: https://github.com/coder/code-server/issues/754

Collabora

Collabora is a LibreOffice based office suite that allows users to create and edit documents, spreadsheets, and presentations directly in their browser.

Once installed, the documents can be accessed via Store > Documents

Extension Development Kit (EDK)

Starting from v0.4.0, Kaapana provides an extension development environment where users can build extensions directly inside the platform. EDK deploys a VS Code server environment for the development of container images and Helm charts, and a local registry where the built container images can be pushed to or pulled from. Note that currently, EDK only includes ease-of-use scripts for workflow extensions, not for application extensions. Once you install EDK, proceed to the VS Code server by using the first link next to the application name. There are a couple of scripts for building images, charts, or directly an entire extension. Please refer to the README.md inside for more details. To initialize the development environment, navigate to cd /kaapana/app and run ./init.sh inside the terminal. This script builds all the Kaapana base images and pushes them to the local registry. Once it completes, you can check the images in the local registry by following the second link. For convenience, the init script copies an example pyradiomics extractor DAG under the dag folder. You can build this extension directly by running ./build_extension.sh --dir /kaapana/app/dag/pyradiomics-feature-extractor. After this command is successfully executed, the extension should be available on the Extensions page (make sure that you change the Version filter to “All” on the Extensions page). This example DAG can be used as a template for building your own extensions. The easiest way to start modifying it is to change the script processing-containers/pyradiomics-feature-extractor/files/extract_features.py and rebuild it. This only changes the container that the operator PyradiomicsExtractorOperator pulls, and the rest of the DAG stays the same. Once you have a better understanding of this DAG, you can start adapting the DAG definition file or even add more operators and containers that they pull. In the end, you should be able to write and deploy your own custom workflow extensions directly inside EDK and test them easily on your platform.

JupyterLab

JupyterLab is an excellent tool to swiftly analyse data stored to the MinIO object store. It comes preinstalled with a wide array of commonly used Python packages for data analysis. While JupyterLab is great for exploratory data analysis, for more complex calculations, consider developing a dedicated Workflow Extension.

You can launch multiple instances of JupyterLab simultaneously for different usecases. Each instance has access to a directory inside the project bucket in MinIO. You can specify this directory, when you launch a new instance in the extensions view. Files in this directory are periodically synced bidirectionally between MinIO and the JupyterLab instance. You can also specify a comma-separated list of file patterns that should be excluded during the syncing process. Note, that the sync interval is 20 seconds. If a file is simultaneously changed in MinIO and JupyterLab, rclone will resolve this conflict during the next syncronisation.

It does not overwrite either side.
It renames one version with a .conflict suffix.
The file changed in MinIO keeps the original filename.
The file changed in Jupyterlab gets the conflict rename.
For more information check the rclone manual.

MinIO Sync

The MinIO Sync application is used to constantly sync a host directory located directly on the server with a folder inside a MINIO bucket following a sync strategy.

The following strategies are available:

Bidirectional: Syncs files from the host to minio and back
Host2Minio: Syncs files undirectional form host into minio
Minio2Host: Syncs files undirectional from MINIO to the host

MITK Workbench

The MITK Workbench is an instance of MITK running in a container and available to users via Virtual Network Computing (VNC). Multiple instances of MITK can be deployed simultaneously. For each deployment a dedicated MinIO bucket is created, named after the respective MITK instance. To import data into the running MITK container, upload your data to the /input directory within this MinIO bucket. All data stored at this path of the MinIO bucket will be transferred to the /input directory of the MITK container. If you wish to retrieve your results from the MITK application, ensure to save them to the /output directory within the MITK container. Any data placed in this directory will be automatically transferred to the /output directory within the dedicated MinIO bucket.

Mount point:
<slow_data_dir>/applications/mitk/<mitk-instance-name>/input
<slow_data_dir>/applications/mitk/<mitk-instance-name>/output

Slicer Workbench

The Slicer workbench is an instance of 3D Slicer running in a container and available to users via Virtual Network Computing (VNC). Multiple instances of Slicer can be deployed simultaneously. For each deployment a dedicated MinIO bucket is created, named after the respective Slicer instance. To import data into the running Slicer container, upload your data to the /input directory within this MinIO bucket. All data stored at this path of the MinIO bucket will be transferred to the /input directory of the Slicer container. If you wish to retrieve your results from the Slicer application, ensure to save them to the /output directory within the Slicer container. Any data placed in this directory will be automatically transferred to the /output directory within the dedicated MinIO bucket.

Mount point:
<slow_data_dir>/applications/slicer/<slicer-instance-name>/input
<slow_data_dir>/applications/slicer/<slicer-instance-name>/output

Tensorboard

Tensorboard can be launched to analyse results generated during a training. Multiple instances of Tensorboard can be deployed simultaneously. For each deployment a dedicated MinIO bucket is created, named after the respective Tensorboard instance. Data stored within this bucket are available to the Tensorboard application.

Mount point:

<slow_data_dir>/applications/tensorboard/<tensorboard-instance-name>

New Registry

This extension can be used to access container images from custom registries. The Development Guide explains detailed steps how to install a workflow extension from a custom registry.

Each instance of New Registry creates a registry-secret either in the Service-Secret scope for pulling images on a system level or in the Project-Secret scope in a single project for pulling processing-container images. Therefore, you might have to install the same registry in multiple scopes. You can select the scope when you launch the extension.

Configuration

display_name: Choose a unique and identifiable name for each instance of New Registry, e.g. REGISTRY-<project-name>-<registry-name> or REGISTRY-service-<registry-name>. Only use alphanumerical charackters and dots, dashes, and underscores.
secret_scope: Defines the scope of the secret
- Service-Secret: Install the secret for system-wide functionalities like installing a new workflow extension
- Project-Secret: Install the secret in the selected project for pulling processing-containers
custom_registry_url: This should be the absolute URL to the registry containing protocol and port: <protocol>://<registry>:<port>/<repository>
credentials_curstom_registry_username: The username to access the registry
credentials_curstom_registry_password: The password to access the registry
test_pull_image: This is an optional configuration if you want to test, if the connection to the registry works. You can specify an image that will be pulled after launching the extensions, e.g. <registry>:<port>/<repository>/<image>:<tag> Don’t specify the protocol here.

Installation errors

Common reasons for installation errors are:

You can only launch a single New Registry instance per registry in the Service-Secret scope.
You can only launch a single New Registry instance per registry and project for the Project-Secret scope.
The display_name must only contain supported charackters.