How We Migrate to Fission – serverless-v4 Engineering Deep Dive

Introduction

In the August update on evrtng.cloud, we explained why we are leaving Apache OpenWhisk and migrating to Fission [1]. Market, strategy, and the OpenWhisk wind-down narrative live there. This article is the engineering companion: what changes technically under the hood, and what the migration means for developers operating Functions on evrtng. We skip the market analysis and look directly into the stack.

Why Fission – Three Technical Reasons

The August article made the strategic why in detail [1]. Here only the engineering perspective, compressed to three points. For those who know the Pekko fork we run as a bridge: the fork was never an end state, but the prerequisite to stay operational while the next step was planned. The series continuity: Article 1 explained the OpenWhisk decision [4], Article 2 the invocation flow [5], this article the migration.

First: Fission is Kubernetes-native. State lives in Custom Resource Definitions and therefore in etcd, not in a separate CouchDB instance for control flow. What was a distributed in-house development in OpenWhisk (controller, own message bus, own database for subject state) is a thin layer of controllers over the Kubernetes API in Fission. Backups, RBAC, and observability use the existing Kubernetes toolchain directly, without separate bridges.

Second: poolmgr instead of spawn-per-invocation. OpenWhisk had warm containers retrofitted; we described that in Article 2 [5]. Fission consistently does what was an optional optimization path in OW: warm pods stay in the pool, the executor specializes them on demand. The cold-start path is shorter because no pod is spawned per invocation and destroyed again.

Third: dynamic tenancy in v1.27.0 [2]. A tenant is a CR, not a database entry. Fission’s tenant controller generates HMAC keys, RBAC, and ServiceAccounts automatically. That replaces our entire OW subject provisioning path, which so far consisted of CouchDB documents, manual limits, and separate auth keys.

Architecture Delta

The table shows the rows that change between v3 and v4. Anything not listed stays identical.

Nine rows change, the rest of the cluster does not. That’s the core point of the migration: swapped the heart, not the body.

What Stays

What stays: Traefik as edge router with Let’s Encrypt, APISIX as gateway with rate limiting and billing logs, CouchDB for billing and subject data, Redis, Strimzi Kafka (now only for MQ triggers via KEDA), Kata+Firecracker as RuntimeClass, Prometheus and Grafana with eight custom dashboards, K8up for backups on S3. The openwhisk namespace remains for these stateful services and our custom workloads (billing, provisioning, quotas). Fission gets its own namespace. That minimizes migration risk – CouchDB and Kafka PVCs can be restored from backup into the same namespace names. The isolation level is the same across both stack generations – Kata-Firecracker remains the constant isolation level, as the August article notes [1].

Invocation Flow v4

The flow from Article 2 [5] shortens. Four steps instead of nine.

1. Traefik TLS -> APISIX: At the edge, Traefik terminates TLS and forwards to APISIX. APISIX applies rate limits and pushes invocation logs via http-logger to the billing-consumer.
2. APISIX -> fission-router: Behind APISIX sits the fission-router as a ClusterIP service, two replicas. HA runs over endpointSliceCache and incrementalRoutes, not over Leader-Election – the router is stateless. Leader-Election is used by executor, buildermgr, and tenantController. It is what controller plus nginx was in OW.
3. Router -> warm pool pod: The router forwards to a pooled function pod, not a freshly spawned one. The pod runs with runtimeClassName: kata-fc on the evr-invoke node pool, poolsize 3 per environment.
4. No Kafka hop anymore: For synchronous calls, the Kafka detour over the invoker is gone. Kafka only runs for MQ triggers, driven by KEDA.

API and CLI – What Changes for Developers

This is the part the August article doesn’t cover. Developers want to know: what changes in my code, in my CLI, in my URLs.

CLI mapping:

Function signature in Node.js – this is a breaking change for users:

// v3 - OpenWhisk
async function main(params) {
  return { statusCode: 200, body: { ok: true } };
}

// v4 - Fission
module.exports = async function(ctx) {
  return { status: 200, body: { ok: true } };
};

Three deltas: main(params) becomes module.exports = async function(ctx), the argument is a context object instead of just params, and statusCode becomes status. Anyone migrating OW functions has to adapt the entry point. Currently this happens manually, we don’t offer a shim. Adapting a function takes minutes, a shim would have become an ongoing burden – it would have to keep both signature worlds alive, including the semantic differences in the context object.

API path: /api/v1/* becomes /fission-function/* as the public path. /v1/* stays routed for compatibility, so old clients don’t break immediately. Activation records disappear as a concept: OpenWhisk stored every execution in CouchDB test_activations, retrievable at /api/v1/namespaces/_/activations/<id>. Fission has no activation database. Metrics live in Prometheus, there’s no direct equivalent API. Anyone who needed activation IDs has to move to logs and metrics. That’s the most visible break for users who evaluated activation history in their own tools.

Code Examples from the Repo

Four snippets that show what changes internally.

FissionTenant CR – a tenant is now a Kubernetes object, not a database entry:

apiVersion: fission.io/v1
kind: FissionTenant
metadata: { name: acme-corp }
spec: { namespace: acme-corp }
# Fission generates HMAC key, RBAC, SA automatically

runtimePodSpec – the isolation level is anchored globally at the environment, not per function:

runtimePodSpec:
  enabled: true
  podSpec:
    runtimeClassName: kata-fc
    nodeSelector: { worker.gardener.cloud/pool: evr-invoke }
    tolerations: [{ key: sandbox, operator: Exists }]

Function signature delta: see section above.

Billing HTTP input – a Kafka consumer became a Flask endpoint:

@app.route("/ingest", methods=["POST"])
def ingest():
    logs = request.json  # APISIX http-logger POSTs
    # same billing logic as v3 (pricing, CouchDB write)

The pricing logic, GBS tiers, and CouchDB writes are identical to v3. Only the input channel changed: HTTP instead of Kafka.

Isolation Deep Dive – Kata and Firecracker

In Article 2 [5] we only briefly mentioned process isolation. Here is the follow-up. Every function pod in v4 runs in a Firecracker microVM [3]. The runtimeClassName: kata-fc in the runtimePodSpec makes Kubernetes start the pod via the kata-fc shim instead of a normal container runtime. The five nodes in the evr-invoke pool carry the taint sandbox=true:NoSchedule; only pods with the corresponding toleration land there. Function workloads of different customers share hardware, but not a kernel.

AWS Lambda pioneered Firecracker at hyperscale [1]. EVRTNG delivers it via standard Kubernetes RuntimeClass, without custom integration in the FaaS engine. That’s the reason the isolation level stays unchanged between v3 and v4 – it doesn’t depend on the FaaS, but on Kubernetes.

Lessons Learned

Three concrete findings, not glorified.

First: the Kata devmapper saga. Fission’s runtimePodSpec allows runtimeClassName, but Kata with Firecracker needs a snapshotter for containerd. We started with the devmapper snapshotter, LVM on loopback. That was fragile – the repo contains 15 kata-* files (kata-devmapper-diag, kata-isolation-test, kata-overlay-install and others), produced over days of debugging, until the first function ran in the microVM. Falling back to overlayfs fixed the problem. Anyone running Kata+Firecracker on Kubernetes should check overlayfs before trying devmapper. Devmapper is the documented default for Firecracker, but the documented default assumes real block storage backing, not a loopback file.

Second: Fission v1.27.0 [2] requires Kubernetes >=1.32. Our Gardener shoot ran on 1.31. Two options: shoot upgrade to 1.32 or pin to Fission v1.24.x (the last line that supports 1.28). Decision: shoot upgrade. A FaaS engine pin to an older version would have become an ongoing burden, a Kubernetes version upgrade is a day’s work. We take the upgrade and accept that other components have to be re-validated on a 1.32 jump.

Third: rethink the billing pipeline. OpenWhisk emitted Kafka events – that was convenient, the consumer only had to consume. Fission emits nothing. APISIX http-logger as replacement is shorter and more direct, but the consumer had to be rewritten from a Kafka consumer to a Flask HTTP endpoint. Activation records disappear as a billing source; instead APISIX logs deliver all fields we need: URI, status, latency, namespace. The pricing logic stayed identical, the input channel was rewritten. Anyone running a similar setup should plan for Fission not being an event source – you have to put your own in front, in our case APISIX.

Outlook

The migration is underway. This article covers the engineering delta, the August article the strategy [1]. The next post in the series is about multi-tenancy with Fission dynamic tenancy in production: FissionTenant CR, HMAC flow, namespace provisioning in depth. Anyone who wants to know how a tenant is provisioned technically and how the HMAC keys flow through the stack finds that there.

1. August strategy article: https://evrtng.cloud/2026/08/15/serverless-computing-2026-update/
2. Fission v1.27.0 Release: https://github.com/fission/fission/releases/tag/v1.27.0
3. Kata Containers: https://github.com/kata-containers/kata-containers
4. Article 1 (OpenWhisk Architecture): https://functions.evrtng.cloud/2026/03/30/warum-wir-apache-openwhisk-gewaehlt-haben-2/
5. Article 2 (Invocation Flow): https://functions.evrtng.cloud/2026/07/17/so-funktioniert-ein-function-aufruf-bei-evrtng/