VA Kafka upgrade preparation: 25.10 / 26.1 to 26.7

BUILD
  make bundle

Copy kafka-migrate.tar to a VA node. Extracting creates kafka-migrate/.

COMMON CUSTOMER PROCEDURE (25.10 AND 26.1)
Run this bundle on the SOURCE appliance before stopping services or backing up.
  appdctl --version
  tar -xf kafka-migrate.tar
  cd kafka-migrate
  sudo bash run.sh run

Wait for run to succeed, then:
  sudo bash run.sh verify
  sudo cat /var/appd/config/kafka-identity.json
  appdcli stop appd
  appdcli platform backup

Use the normal platform backup arguments/destination required by your build.
Keep /var/appd/config/kafka-identity.json in the backup. For 25.10 also retain
/var/appd/config/zk-kraft-migration, including its completed migration handoff.
Retain all broker AND controller PV/PVC manifests, disks and Kafka Secrets.
Restore on 26.7 using the updated restore helper, Helmfile and Kafka chart that
stage this identity and restore Kafka/node pool IDs before reconciliation.
When reattaching disks to replacement VMs, map local PV node affinity to the
new nodes hosting those disks. The identity file is not a data or Secrets backup.

Optional: sudo bash run.sh check | verify | status (choose one action).
All checks are included in run. Rerun run to resume after resolving a failure.

BEFORE RUNNING
Quiesce application writes and suspend appdcli start/sync/chart reconciliation.
Keep source Kafka and Strimzi running; on 25.10 keep ZooKeeper running too.
Do not run preparation after appdcli stop: live Kafka identity must still exist.
VA must provide Linux AMD64, Python 3, MicroK8s, appdctl and sha256sum.
The appduser account must exist. Migration still runs with sudo, but backup
artifacts are owned by appduser and its primary group. Directories stay 0700;
files stay 0600 so appduser can include them in the platform backup.

VERSION DETECTION
appdctl --version is read automatically. The major.minor release determines
behavior; for example 26.7.1-5362 is release 26.7, not the first two characters.
Unsupported or unreadable versions stop before migration.
  25.10: existing ZooKeeper-to-KRaft migration, then identity capture.
  26.1: validate existing finalized KRaft and capture identity only.
  26.7: identity capture only, useful before subsequent stop/start or backup.
26.1/26.7 never execute migration, change Kafka specs, create controllers or
archive/delete ZooKeeper storage. They require both existing node pools and
Ready Kafka at the current generation. Transitional/non-KRaft sources fail.

FIXED VA SETTINGS
Kafka namespace kafka; cluster appd; context microk8s.
25.10 migration requires Strimzi 0.45.1 and Kafka 3.9.1.
26.1/26.7 capture uses the source Kafka/operator versions already installed.
Operator kafka-operator/strimzi-cluster-operator; context microk8s.
The following storage/migration settings apply only to the 25.10 path:
Existing broker and ZooKeeper replica counts are preserved (one or three each).
One broker creates one controller; three brokers create three controllers.
No three-node requirement. Controller spreading across hosts is preferred.
Controller pool kafka-controller: local-data PVCs, 6Gi per controller.
Kafka.spec.kafka.image supplies localhost:32000/appdynamics/kafka:<tag>.
The same image runs brokers and controllers through the existing VA registry.
No Kafka admin credentials, image archive or downloads are required.

WHAT RUN DOES ON 25.10
Checks Strimzi Ready and records migration checkpoints and PVC identities.
Adopts broker storage, creates controllers and starts metadata migration.
Waits for Strimzi Ready=True at the current generation and KRaftPostMigration,
then KRaft. Checks cluster identity and retained PVC/PV identities.
Finalizes KRaft, lets Strimzi stop ZooKeeper, archives and deletes ZooKeeper PVCs.
Broker AND controller PVCs/PVs are retained for VA backup/restore.
After migration completes, saves /var/appd/config/kafka-identity.json.

IDENTITY CAPTURE ON ALL SUPPORTED RELEASES
Records schema version, original cluster ID, broker/controller node IDs,
Kafka version, appliance build, every data claim/PV identity and PV specs.
Checks consistent cluster IDs, unique node IDs, complete Bound claims and PV
claim references. Existing conflicting saved identity stops the run.
The file is appduser-owned, mode 0600. Existing config directory permissions
are preserved. Migration artifact ownership remains as described below.
check validates live source prerequisites; verify on 26.1/26.7 and completed
25.10 migrations compares live identity and backing storage with the saved file.
status shows saved identity and resources on 26.1/26.7; it shows migration
checkpoints on 25.10. Capture does not invent migration checkpoints or handoff
files for a native KRaft appliance.

KRaft finalization ends rollback to ZooKeeper. Do not run the 25.10 chart after it.
Checkpoints, status reports, handoff details and ZooKeeper archives: /var/appd/config/zk-kraft-migration.
The updated bundle also repairs ownership of artifacts from previous runs.
To repair an already completed migration without rerunning migration, execute:
  sudo bash run.sh status
Ownership is repaired before the status command reads Kubernetes state.
Retain this directory and broker/controller data, cluster/node IDs and Secrets
in the subsequent backup/restore flow. Allow disk space for ZooKeeper archives.
Migration success relies on Strimzi status; records, offsets, ACLs and Kafka
quorum are not independently inspected. No test topic is created.
The generic bundle has not been run on live 25.10/26.1 appliances in this change.
Live preparation and subsequent backup/restore still require validation.

Failures stop the script and retain checkpoints; there is no automatic rollback.
Concurrent runs are blocked. After a killed process, first establish that no
migration is running before removing kafka/appd-zk-kraft-migration-v1-lock.
