Octopus shared cache optimization with thundering herd prevention

[mgazza]

Summary

Implements critical performance optimization for Octopus Energy integration by splitting cache architecture and preventing thundering herd on cache expiry, reducing API load by ~99.9%.

Problem

Current inefficiency: Every PredBat instance (1000+ users in SaaS deployment) independently:

1. Fetches the SAME Octopus Agile rates every 30 minutes
2. Stores duplicate copies in per-user cache files
3. Makes redundant API calls (500 users on AGILE-24-10-01 = 500 identical API calls)
4. Thundering herd: When cache expires, all 1000 pods fetch simultaneously

Example:

• 500 users on AGILE-24-10-01 tariff
• Each pod fetches independently every 30 min
• 500 identical API calls + 500 duplicate storage entries
• Cache expiry at 4:30:00 → 1000 simultaneous API calls 💥

Solution

1. Split Cache Architecture

User-specific cache (/tmp/cache/{user_id}/octopus_user.yaml):

• Account agreements (which tariffs apply when)
• Intelligent device settings
• Saving session enrollments

Shared tariff cache (/tmp/cache/shared/tariffs/{product}_{tariff}.yaml):

• One file per unique tariff
• Shared across ALL users on that tariff
• No duplicate storage

Shared URL cache (/tmp/cache/shared/urls/{hash}.yaml):

• SHA256-hashed URL responses
• 30-minute TTL with automatic expiry

2. Stale-While-Revalidate (Thundering Herd Prevention)

Three-tier cache strategy:

1. Fresh (< 30 min): Return cached data immediately
2. Stale (30-35 min): Serve stale data while ONE pod refreshes
3. Too stale (> 35 min): Must fetch fresh data

How it works when cache expires:

• First pod: Acquires atomic file lock, refreshes cache
• Other 999 pods: See lock exists, serve 5-min-stale data
• No blocking, eventually consistent within seconds

Lock implementation:

• Uses O_CREAT | O_EXCL for atomic, non-blocking lock
• Automatic cleanup after refresh
• No deadlock risk

Changes

Commit 1: Shared cache architecture

• octopus.py:342-352 - Initialize shared cache directory structure
• octopus.py:449-514 - Add tariff/URL cache helper methods
• octopus.py:516-563 - Split save/load cache methods
• Cache key generation uses product_code + tariff_code (no date component)

Commit 2: Stale-while-revalidate

• octopus.py:1016-1069 - Implement three-tier cache with atomic locking
• Prevents thundering herd on cache expiry
• Only ONE pod fetches, others serve stale data

Expected Impact

API Load Reduction: 99.9%

Before:

• Normal operation: 500 users × 1 call = 500 API calls per 30 min
• Cache expiry: 1000 pods × 1 call = 1000 simultaneous API calls

After:

• Normal operation: 1 API call per tariff per 30 min
• Cache expiry: 1 pod fetches, 999 serve stale = 1 API call

Storage Efficiency: 99.9%

Before: 1000 users × 10 tariffs = 10,000 duplicate entries
After: 10 unique tariff files shared by all users

Instant Propagation

• Pod A fetches new Agile rates at 4:05pm
• Pod B checks at 4:06pm → cache hit (no API call)
• All pods benefit from single fetch

Reliability

• No thundering herd overwhelming Octopus API
• No rate limiting failures
• 5-minute staleness is acceptable for battery optimization

Compatibility

• ✅ Backward compatible: Single-user installations continue working (no pods to coordinate)
• ✅ No migration needed: Old cache files ignored, new structure created
• ✅ Octopus Agile timing: 30-min polling handles variable publication time (4pm-6pm)
• ✅ Multi-user safe: File locking prevents race conditions in shared deployments

Use Cases

Single-User (Home Assistant Add-on)

• Cache split still beneficial (cleaner organization)
• No thundering herd (only one instance)
• Works exactly as before

Multi-User (SaaS Deployment)

• Massive API reduction (99.9%)
• Shared cache across all customer pods
• Thundering herd prevention critical

Testing

Tested in PredBat SaaS infrastructure with 1000+ customer instances. Will benefit open-source users running multiple instances or preparing for multi-user deployments.

🤖 Generated with Claude Code

Co-Authored-By: Claude noreply@anthropic.com

[mgazza]

Closing fork PR - created upstream PR instead: springfall2008#3045