Serverless Jupyter Notebooks: Comparing Deployment Models¶
This project investigates how to run Jupyter notebook servers on Kubernetes more efficiently by comparing two deployment models: traditional JupyterHub with KubeSpawner and HPA versus serverless deployments using Knative Serving. The goal is to determine which method is more resource-efficient, faster to start, and easier to scale for educational workloads where demand is bursty and idle periods dominate.
Motivation¶
In educational settings, Jupyter notebook servers exhibit highly bursty usage patterns — active during class sessions and lab periods, idle the rest of the time. Traditional JupyterHub deployments with Horizontal Pod Autoscaler (HPA) provision dedicated pods per user that remain running even during idle periods, wasting CPU and memory resources. The serverless model promises scale-to-zero: pods are automatically created on demand and destroyed when not in use, matching resource consumption to actual workload.
Knative vs. HPA: Empirical Comparison¶
The experiment was conducted on a MicroK8s cluster (3 nodes, 2 CPUs and 128GB RAM each) with simulated HTTP requests to JupyterLab instances. Key findings:
| Feature | Knative Serverless | Kubernetes HPA |
|---|---|---|
| Cold Start Latency | ~3.2s (3× faster) | Higher |
| Warm Start Latency | ~150ms | ~200ms |
| Scale to Zero | Yes (native) | No (manual only) |
| Resource Efficiency | High — no idle pod costs | Lower — idle pods consume resources |
| Setup Complexity | Moderate | Easier |
Knative starts pods approximately 3× faster than HPA during cold starts. This is largely due to Knative's optimized request-based scaling model, which bypasses HPA's monitoring-interval delay. Warm starts (where the pod is already running) are comparable between both approaches, with Knative maintaining a slight advantage.
Scale-to-Zero and Resource Efficiency¶
The most significant operational advantage of the Knative model is automatic scale-to-zero. When no users are connected, Knative scales the pod count to zero, eliminating idle resource consumption. HPA-based deployments, by contrast, maintain a minimum replica count that wastes resources during off-hours — a critical concern in educational environments where servers may be idle for 18+ hours per day.
This has direct cost implications for institutions providing Jupyter infrastructure: scale-to-zero can reduce compute costs by an order of magnitude compared to always-on deployments, without sacrificing startup latency during active use.
Implications for GPU-Enabled Kernel Architecture¶
The serverless notebook model has important implications for GPU-backed Jupyter kernels. GPU instances are expensive and typically allocated per-user in traditional deployments, leading to idle GPU time during student thinking periods. A serverless kernel architecture — where GPU-backed kernels are spun up on demand for execution cells and scaled to zero during idle analysis periods — could dramatically reduce GPU costs while maintaining the interactive experience.
Future plans include building a custom kernel manager for HTTP-based execution that decouples the notebook UI from the execution backend, storing notebook state in Redis or a database for multi-pod resilience. This would enable GPU kernels to be treated as ephemeral, serverless resources rather than long-lived per-user allocations.
Architecture¶
The project provides:
- A custom Knative spawner for JupyterHub (
jhub-knative-spawner) that replaces KubeSpawner's pod creation with Knative Service creation, inheriting automatic scaling and scale-to-zero. - A JupyterLab container image (
jupyterlab-knative) configured for Knative deployment with an appropriate startup script. - A Knative Service definition demonstrating the deployment configuration.
Related Projects¶
No direct project page — this is infrastructure research. See our full project list for related tools like the CSE120 grading pipeline deployment.
- Multi-Modal Attendance — Tendy shares the infrastructure deployment focus.
- Skill Networks — SkillNet's crawl infrastructure also uses Kubernetes-native patterns.
Status Note¶
This project was never fully completed but represents a promising direction for educational compute infrastructure. The 3.19s cold-start results and scale-to-zero capability are directly applicable to the lab's GPU compute architecture on csed-nebula. The idea of ephemeral, on-demand notebook kernels — where GPU resources are spun up per execution cell and scaled to zero during idle analysis periods — would dramatically reduce GPU costs in educational settings. The custom kernel manager for HTTP-based execution (decoupling the notebook UI from the execution backend) and the jhub-knative-spawner are worth completing as a publication.