Uploading a Workflow Extension via the Extensions View

Workflow extensions can be uploaded to Kaapana using the drag-and-drop functionality in the Extensions View.

To make a new workflow extension fully available on the platform, the following artifacts must be uploaded:

All processing-container images used by the workflow.
A Docker image containing the Airflow code (DAG files and custom operator implementations).
The packaged Helm chart for the workflow extension.

This guide demonstrates how to prepare a workflow extension and how to upload it to the platform, using the otsus-method-workflow extension as an example. The following steps are explicitly covering the usecase, when processing-container images, Airflow code, and the extension Helm chart are uploaded via drag-and-drop.

Prerequisites

The following prerequisites are assumed:

Access to the kaapana code directory.
Base images available on the system. See the documentation for more information.

Build Processing Containers

From the root level of the kaapana code directory navigate to the directory of the only custom processing-container image used in the otsus-method workflow:
```
cd templates_and_examples/examples/processing-pipelines/otsus-method/processing-containers/otsus-method
```

Build the processing-container with the expected image tag:

docker build -t <registry>/<repository>/otsus-method:<platform-version> .

Prepare the Airflow Code and Build the DAG Installer Image

Adapt image_pull_policy of Airflow operators

Add the line image_pull_policy="IfNotPresent", to both operators in templates_and_examples/examples/processing-pipelines/otsus-method/extension/docker/files/otsus-method/.

class OtsusMethodOperator(KaapanaBaseOperator):
    def __init__(
        self,
        dag,
        name="otsus-method",
        execution_timeout=timedelta(seconds=120),
        *args,
        **kwargs,
    ):
        super().__init__(
            dag=dag,
            name=name,
            image=f"{DEFAULT_REGISTRY}/otsus-method:{KAAPANA_BUILD_VERSION}",
            image_pull_secrets=["registry-secret"],
            image_pull_policy="IfNotPresent",
            execution_timeout=execution_timeout,
            *args,
            **kwargs,
        )

and

class OtsusNotebookOperator(KaapanaBaseOperator):
    def __init__(
        self,
        dag,
        name="otsus-notebook-operator",
        execution_timeout=timedelta(minutes=20),
        *args,
        **kwargs,
    ):
        super().__init__(
            dag=dag,
            name=name,
            image=f"{DEFAULT_REGISTRY}/otsus-method:{KAAPANA_BUILD_VERSION}",
            image_pull_secrets=["registry-secret"],
            image_pull_policy="IfNotPresent",
            execution_timeout=execution_timeout,
            ram_mem_mb=1000,
            ram_mem_mb_lmt=3000,
            *args,
            **kwargs,
        )

Build the DAG installer image containing the Airflow code with the correct image tag:

cd ../../extension/docker/
docker build -t <registry>/<repository>/dag-otsus-method:0.0.0 .

Important

The image version 0.0.0 is expected here as it matches with the default version in the dag-installer chart.

Save and Upload Docker Images

Save all required Docker images into a single archive:

docker save <registry>/<repository>/otsus-method:<platform-version> <registry>/<repository>/dag-otsus-method:0.0.0 -o images.tar

Upload the images.tar file via the Extensions View in the Kaapana Web Interface.

Note

In case of issues the corresponding FAQ can be helpful.

Prepare, Package, and Upload the Helm Chart

Navigate to the helm chart directory:
```
cd ../otsus-method-workflow/
```

Update values.yaml with the following settings:

---
global:
    image: "dag-otsus-method"
    action: "copy"
    pull_policy_images: IfNotPresent

Update chart dependencies and package the Helm chart:
```
helm dep up
helm package .
```
Upload the packaged Helm chart otsus-method-workflow-0.0.0.tgz via the Extensions View.

Note

The version of the workflow corresponds to the version in the Chart.yaml file.

In case of issues the corresponding FAQ can be helpful.