# [[Session Affinity in Kubernetes]] ![[Session Affinity in Kubernetes.svg]] **Session affinity** (also called "sticky sessions") is a load balancing feature that ensures requests from the same client always go to the same backend pod/server. ## How It Works ### Without Session Affinity (Round Robin) User → Load Balancer → Pod 1 (Login succeeds, session saved in Pod 1) User → Load Balancer → Pod 2 (Session not found, logged out!) User → Load Balancer → Pod 3 (Session not found, logged out!) User → Load Balancer → Pod 1 (Session found, works again) ### With Session Affinity (ClientIP) User (IP: 1.2.3.4) → Load Balancer → Pod 1 (Login succeeds) User (IP: 1.2.3.4) → Load Balancer → Pod 1 (Same pod, session found ✅) User (IP: 1.2.3.4) → Load Balancer → Pod 1 (Same pod, session found ✅) ## Configuration Example In a Kubernetes Service: apiVersion: v1 kind: Service metadata: name: grafana namespace: observability spec: sessionAffinity: ClientIP # Route based on client IP sessionAffinityConfig: clientIP: timeoutSeconds: 10800 # Keep affinity for 3 hours (default: 10800) ports: - port: 3000 targetPort: 3000 selector: app: grafana ## Types of Session Affinity ### 1. ClientIP (Kubernetes Native) - Routes based on source IP address - Works at the Kubernetes Service level - Simple and effective for most cases - Configured via `sessionAffinity: ClientIP` ### 2. Cookie-based (Application Load Balancers) - Uses a cookie to track which backend to use - More reliable if users change IPs (mobile networks, VPNs) - Requires application-level or ingress controller configuration - Not available in standard Kubernetes Services ## Common Use Cases ### When You Need Session Affinity **Local Session Storage** - Applications that store sessions in memory or local disk - Example: Grafana with SQLite session storage **WebSocket Connections** - Real-time applications requiring persistent connections - Example: Chat applications, collaborative tools **File Upload/Download** - Long-running transfers that must complete on the same pod - Example: Multi-part uploads ### When You Don't Need It **Stateless Applications** - Applications using JWT tokens or external session storage - Example: APIs with token-based authentication **Shared Session Storage** - Applications using Redis, Memcached, or database sessions - Any pod can handle any request ## Trade-offs ### Pros ✅ Works with applications that use local session storage ✅ Simple to configure (one line in Service spec) ✅ No code changes needed ✅ Native Kubernetes feature ### Cons ❌ If a pod dies, sessions on that pod are lost ❌ Can cause uneven load distribution (one user = one pod) ❌ Doesn't work well if client IP changes (VPN switches, mobile network) ❌ May prevent effective autoscaling ## Better Alternatives for Production Instead of session affinity, consider **shared session storage**: ### Redis/Memcached Application connects to Redis for session storage: - name: SESSION_STORE value: "redis://redis.default.svc.cluster.local:6379" **Benefits:** - Any pod can handle any request - Sessions survive pod restarts - True horizontal scaling - Better load distribution ### JWT Tokens (Stateless) Application uses JWT for authentication: - name: JWT_SECRET valueFrom: secretKeyRef: name: app-secrets key: jwt-secret **Benefits:** - No server-side session storage needed - Infinitely scalable - No session affinity required - Simplest architecture ### Shared Database Sessions Application stores sessions in PostgreSQL: - name: SESSION_DATABASE value: "postgresql://sessions-db.default.svc.cluster.local:5432/sessions" **Benefits:** - Sessions persist across pod restarts - Centralized session management - Easy to query/audit sessions ## Troubleshooting ### Problem: Users Getting Logged Out Randomly **Diagnosis:** # Check if session affinity is configured kubectl get service grafana -n observability -o yaml | grep sessionAffinity # Check number of pods kubectl get pods -l app=grafana **Solution:** Add session affinity to the Service: spec: sessionAffinity: ClientIP sessionAffinityConfig: clientIP: timeoutSeconds: 10800 ### Problem: Uneven Load Distribution **Diagnosis:** # Check requests per pod kubectl top pods -l app=grafana **Cause:** Session affinity can cause some pods to handle more traffic than others. **Solutions:** 1. Use shared session storage (Redis) and remove session affinity 2. Increase timeout to reduce connection churn 3. Monitor and adjust replica counts based on actual load ### Problem: Sessions Lost After Pod Restart **Diagnosis:** Pods use `emptyDir` or local storage for sessions. **Solution:** Migrate to shared session storage: # Option 1: Use PersistentVolumeClaim volumes: - name: session-data persistentVolumeClaim: claimName: grafana-sessions # Option 2: Use Redis/Memcached env: - name: GF_SESSION_PROVIDER value: "redis" - name: GF_SESSION_REDIS_ADDR value: "redis:6379" ## Real-World Example: Grafana with Session Affinity ### The Problem We had Grafana with: - 4 pods (KEDA autoscaling) - Local SQLite database for sessions - Users logging in but immediately getting logged out ### The Diagnosis $ kubectl get pods -n observability -l app=grafana NAME READY STATUS RESTARTS AGE grafana-84ff54759d-2gzts 1/1 Running 0 5m grafana-84ff54759d-ml5nc 1/1 Running 0 5m grafana-84ff54759d-nrcln 1/1 Running 0 5m grafana-84ff54759d-v6gpz 1/1 Running 0 5m # User logs in → hits Pod 1 (session stored in Pod 1's SQLite) # Next request → hits Pod 2 (no session found) → logged out ### The Solution apiVersion: v1 kind: Service metadata: name: grafana namespace: observability spec: sessionAffinity: ClientIP sessionAffinityConfig: clientIP: timeoutSeconds: 10800 # 3 hours type: LoadBalancer ports: - port: 3000 targetPort: 3000 selector: app: grafana ### The Result ✅ All requests from the same user go to the same pod ✅ Sessions persist throughout the user's session ✅ Login works correctly with multiple Grafana pods ## References - Kubernetes Service Documentation: https://kubernetes.io/docs/concepts/services-networking/service/#session-affinity - Service Session Affinity: https://kubernetes.io/docs/reference/networking/virtual-ips/#session-affinity %% # Excalidraw Data ## Text Elements ## Drawing ```json { "type": "excalidraw", "version": 2, "source": "https://github.com/zsviczian/obsidian-excalidraw-plugin/releases/tag/2.1.4", "elements": [ { "id": "4y8R7iOA", "type": "text", "x": 118.49495565891266, "y": -333.44393157958984, "width": 3.8599853515625, "height": 24, "angle": 0, "strokeColor": "#1e1e1e", "backgroundColor": "transparent", "fillStyle": "solid", "strokeWidth": 2, "strokeStyle": "solid", "roughness": 1, "opacity": 100, "groupIds": [], "frameId": null, "roundness": null, "seed": 967149026, "version": 2, "versionNonce": 939059582, "isDeleted": true, "boundElements": null, "updated": 1713723615080, "link": null, "locked": false, "text": "", "rawText": "", "fontSize": 20, "fontFamily": 4, "textAlign": "left", "verticalAlign": "top", "containerId": null, "originalText": "", "lineHeight": 1.2 } ], "appState": { "theme": "dark", "viewBackgroundColor": "#ffffff", "currentItemStrokeColor": "#1e1e1e", "currentItemBackgroundColor": "transparent", "currentItemFillStyle": "solid", "currentItemStrokeWidth": 2, "currentItemStrokeStyle": "solid", "currentItemRoughness": 1, "currentItemOpacity": 100, "currentItemFontFamily": 4, "currentItemFontSize": 20, "currentItemTextAlign": "left", "currentItemStartArrowhead": null, "currentItemEndArrowhead": "arrow", "scrollX": 583.2388916015625, "scrollY": 573.6323852539062, "zoom": { "value": 1 }, "currentItemRoundness": "round", "gridSize": null, "gridColor": { "Bold": "#C9C9C9FF", "Regular": "#EDEDEDFF" }, "currentStrokeOptions": null, "previousGridSize": null, "frameRendering": { "enabled": true, "clip": true, "name": true, "outline": true } }, "files": {} } ``` %%