tracking: Test demo upgrades on nightly versions for 26.3.0

[sbernauer]

Part of stackabletech/issues#812

This is testing that upgrades from the stable release to the nightly release of the operators
and products do not negatively impact the products.

Note

Record the results (issues, anomalies, or success) during the process in a comment on this issue.
Eg:

:green_circle: **airflow-scheduled-job**

The CRD had been updated and I needed to change the following in the manifest:
...

The following emojis can be used to indicate the status:

• hourglass: In progress / waiting
• red_circle: In progress with errors
• orange_circle: Completed with minor issues or anomalies
• green_circle: Completed without issues or errors

List of demos

Replace the items in the lists below with the applicable Pull Requests (if any). Also put your
GitHub handle next to the demo to indicate a particular demo was picked up for testing.

• airflow-scheduled-job @NickLarsenNZ
• argo-cd-git-ops @NickLarsenNZ decided to skip this (see comment)
• data-lakehouse-iceberg-trino-spark
• end-to-end-security @Techassi
• hbase-hdfs-load-cycling-data @NickLarsenNZ
• jupyterhub-keycloak @adwk67
• jupyterhub-pyspark-hdfs-anomaly-detection-taxi-data @adwk67
• logging @Techassi
• nifi-kafka-druid-earthquake-data @NickLarsenNZ
• nifi-kafka-druid-water-level-data @NickLarsenNZ
• signal-processing @Techassi
• spark-k8s-anomaly-detection-taxi-data @NickLarsenNZ
• trino-iceberg @sbernauer
• trino-taxi-data @sbernauer

List of stacks

Similar to the demos above we need to test all stacks that don't have a demo.
Replace the items in the lists below with the applicable Pull Requests (if any). Also put your
GitHub handle next to the stack to indicate a particular stack was picked up for testing.

Tip

Some of the stacks have a tutorial to follow.

• airflow-trino-dbt (this was merged into the airflow demo and was only removed from the stacks recently: it has also been removed from the template: @adwk67 )
• monitoring The only thing that get's upgraded is Prometheus, which is not a supported tool, hence not super important to test. tracking: Test demos on nightly versions for 26.3.0 #372 makes sure that a fresh install works
• observability @NickLarsenNZ Will do it ias part of tracking: Test demos on nightly versions for 26.3.0 #372
• openldap (comes up)
• tutorial-openldap @Techassi

Stable to Nightly Upgrade Testing Instructions

These instructions are for deploying and completing the stable demo, and then
upgrading operators, CRDs, and products to the nightly versions well as upgrading
the operators and CRDS.

Tip

Be sure to select the stable docs version on https://docs.stackable.tech/home/stable/demos/.

# For demo testing, , install demo (stable operators) for the stable release (25.11).
stackablectl demo install <DEMO_NAME>

# For stack testing, install stack via
stackablectl stack install <STACK_NAME>

# --- IMPORTANT ---
# Run through the (still) nightly demo/stack instructions (refer to the list above).

# Install nightly version of operators
stackablectl release upgrade dev

# Bump every stacklet to the version that used by the nightly demo.
# Typically, this is the latest supported version.
kubectl patch hbaseclusters/hbase --type='json' -p='[{"op": "replace", "path": "/spec/image/productVersion", "value":"x.x.x"}]' # changed

# Check that the operator logs look good (no unexpected errors etc.)

# Run through the (still) nightly demo/stack instructions again.

[Techassi]

🟠 logging

• 🟢 Stable from scratch
  • PR to adjust image: docs(demos/logging): Adjust OpenSearch dashboard image #379
• 🟠 Upgrade to nightly
  • Initial upgrade worked
  • Bumping OpenSearch 3.4.0 results in a weird mount error (@siegfriedweber wrote a guide on migration steps needed: doc: Add SDP upgrade instructions opensearch-operator#124)

[NickLarsenNZ]

🟢 airflow-scheduled-job

• 🟢 Stable from scratch
  • Docs PR: docs(demo/airflow-scheduled-job): Update instructions #378
• 🟢 Upgrade to nightly
  • When airflow is upgraded, you need to replace image refs (and rollout STSs) for additional containers to:

    image: oci.stackable.tech/sdp/airflow:3.1.6-stackable0.0.0-dev
    

  • Also need to edit the ConfigMap to update Spark version used by the DAG.

product upgrade steps

# Airflow 3.0.6 -> 3.1.6
kubectl -n airflow-demo patch airflowclusters/airflow --type='json' -p='[{"op": "replace", "path": "/spec/image/productVersion", "value":"3.1.6"}]'
# Kafka 4.1.0 -> 4.1.1
kubectl -n airflow-demo patch kafkaclusters/kafka --type='json' -p='[{"op": "replace", "path": "/spec/image/productVersion", "value":"4.1.1"}]'
# Opa 1.8.0 -> 1.12.3
kubectl -n airflow-demo patch opaclusters/opa-airflow --type='json' -p='[{"op": "replace", "path": "/spec/image/productVersion", "value":"1.12.3"}]'
# Spark 4.0.1 -> 4.1.1
# Need to edit the ConfigMap