How Does HypeLab Achieve Sub-Millisecond Ad Predictions Worldwide?

Running machine learning models in real-time is expensive. Every millisecond spent on model inference is latency budget that could otherwise go to network communication, auction processing, or safety margin. At HypeLab, we reduce this cost by caching prediction outputs in Redis, giving our Web3 advertising platform a critical speed advantage.

The insight that makes caching possible: our prediction features are user-agnostic. We do not include user IDs, cookies, or wallet addresses. This means many different users produce identical feature combinations, and identical features mean identical predictions. Cache once, serve many times.

Why Do User-Agnostic Features Enable Prediction Caching?

HypeLab's blockchain ads prediction model uses features like device type, operating system, browser, publisher, placement type, geo tier, time of day, and campaign characteristics. Notice what is absent: user ID, cookie, wallet address, or any persistent user identifier.

This design choice has multiple motivations. Privacy is one - we do not need to track individual users to predict click probability. Performance is another - user-level features would require per-user lookups that add latency. And cacheability is a third - without user identifiers, feature space becomes finite and repetitive.

An iPhone user visiting Phantom wallet during US evening hours shares their feature combination with thousands of other users browsing DeFi dashboards like DeBank, Zerion, or Zapper. Predict once, cache the result, serve it for all subsequent matching requests.

How Does HypeLab's Cache Architecture Work?

The caching layer sits between the prediction request handler and the model inference service in our crypto ad network infrastructure:

The cache key is a hash of the feature vector. We use a deterministic hashing function so identical features always produce identical keys. The cache value is the model's prediction output, specifically the predicted click-through rate (PCTR) for each eligible campaign.

Cache entries have a TTL (time-to-live) that balances freshness against hit rate. Shorter TTL means predictions stay current but hit rate drops. Longer TTL means higher hit rate but potentially stale predictions. Our current TTL is tuned based on observed prediction stability and traffic patterns.

Why Does HypeLab Use Regional Redis Instances Instead of a Global Cache?

HypeLab's Web3 advertising platform serves traffic globally. Users in Tokyo, London, and New York all request ad predictions for campaigns promoting DeFi protocols like Uniswap and Aave, NFT marketplaces like OpenSea and Blur, and blockchain games like Axie Infinity and Pixels. A single global cache would introduce problems:

Latency: A cache lookup from Tokyo to a US-based Redis instance adds 100-150ms of network latency. This defeats the purpose of caching for latency reduction.

Relevance: Predictions cached from US traffic are less useful for Asian traffic. Different publishers are popular in different regions. Geo tier features differ. Time-of-day features differ even at the same UTC moment.

We solve both problems with regional Redis instances. Americas, Europe, and Asia-Pacific each have dedicated cache infrastructure. Traffic routes to the nearest region. Each cache stores predictions relevant to its regional traffic.

What Cache Hit Rate Does HypeLab Achieve in Production?

HypeLab's production cache hit rate exceeds 40% across all crypto ad network traffic. This means over 40% of prediction requests are served from cache without model inference, giving advertisers faster auction responses.

Hit rate varies by segment:

• High-traffic publishers (Phantom, Magic Eden, DeBank): 50-60% hit rate. Popular Web3 publishers generate repetitive traffic patterns.
• Long-tail publishers: 20-30% hit rate. Less traffic means fewer cache-warming requests.
• Mobile traffic: 45% hit rate. Device fragmentation is lower on mobile (few major phone models).
• Desktop traffic: 35% hit rate. More diverse browser/OS combinations.

We monitor hit rate by segment to identify caching opportunities. A segment with unexpectedly low hit rate might indicate cache key issues or unusual traffic patterns worth investigating.

How Does HypeLab Design Cache Keys for ML Predictions?

The cache key must uniquely identify a feature combination while being efficient to compute and store. Our approach:

Feature serialization: Features are serialized in a deterministic order to a byte string. The order is fixed by schema - device_type, then os, then browser, and so on.

Hashing: The serialized features are hashed using a fast, deterministic hash function optimized for speed and low collision rate.

Key structure: The final key includes a version prefix for cache invalidation during model updates. Format: pred:v3:hash_of_features

This design ensures that identical features always produce identical keys, different features produce different keys with high probability, and model updates can invalidate old cache entries cleanly.

What Data Does HypeLab Store in Each Cache Entry?

The cached value contains everything needed to serve a prediction response without model inference:

• PCTR predictions: Predicted click probability for each campaign that was eligible when this entry was created
• Timestamp: When this prediction was computed, for staleness detection
• Model version: Which model version produced this prediction

We serialize cache values using MessagePack for compact binary representation. JSON would work but wastes bytes and parsing time.

How Does HypeLab Handle Cache Invalidation?

Caches must be invalidated when they no longer reflect reality. Several events trigger invalidation:

Model updates: When a new model deploys, all cached predictions become potentially stale. We handle this with version prefixes in cache keys - the new model uses a new version prefix, and old entries naturally expire via TTL.

Campaign changes: When campaigns launch, pause, or change bids, cached predictions for their eligible impressions become incorrect. We handle this by including campaign state in the cache key computation - state changes produce new keys.

Time-based expiry: TTL ensures entries do not persist indefinitely. Even without explicit invalidation, entries expire and get refreshed with current predictions.

We do not implement fine-grained cache invalidation for every possible change. The complexity would outweigh the benefit. TTL-based expiry combined with version prefixes handles most cases adequately.

How Does HypeLab Manage Cache Memory and Eviction?

Redis cache size is finite. When the cache fills, entries must be evicted. We use Redis's LRU (Least Recently Used) eviction policy - entries accessed least recently are evicted first.

This policy naturally keeps hot entries (frequently accessed feature combinations) in cache while evicting cold entries (rare combinations). Since traffic follows power-law distributions - a few combinations account for most traffic - LRU eviction has minimal impact on hit rate.

We monitor cache memory usage and eviction rates. High eviction rates indicate the cache is undersized for traffic volume. We scale cache capacity when eviction impacts hit rate meaningfully.

How Does Cache Warming Reduce Cold-Start Latency?

When a new model deploys or a regional cache restarts, the cache is empty. All requests result in cache misses until traffic naturally warms the cache. This cold-start period has higher latency.

We mitigate cold starts with cache warming: precomputing predictions for known high-traffic feature combinations and loading them into cache before traffic arrives. The warming set is derived from historical traffic logs - the top N feature combinations by request volume.

Cache warming does not achieve full warm-cache performance immediately, but it covers the highest-traffic combinations that would otherwise produce many cold misses.

How Does Redis Caching Work with HypeLab's Fallback Model?

Redis caching is a latency optimization - serving predictions faster when possible. The fallback model is a reliability mechanism - serving predictions at all when the primary system fails.

These are complementary, not alternatives:

• Cache hit: Serve from Redis in <1ms
• Cache miss, model available: Run inference in ~2ms, cache result
• Cache miss, model unavailable: After 200ms timeout, fall back to simple model

The cache reduces average latency. The fallback ensures availability. Both contribute to a robust prediction service.

What Metrics Does HypeLab Monitor for Cache Health?

Cache health is monitored through several metrics:

Hit rate: Primary health indicator. Trending downward suggests issues with cache keys, capacity, or traffic patterns.

Latency distribution: Cache hits should be sub-millisecond. If P99 cache latency spikes, investigate Redis performance.

Memory usage: Approaching capacity triggers eviction. Monitor for unexpected memory growth.

Eviction rate: High eviction indicates capacity pressure. May need to scale or tune TTL.

Error rate: Redis connection failures or timeouts. Should be near zero; spikes indicate infrastructure issues.

Dashboards visualize these metrics by region. Alerts trigger when metrics exceed thresholds, enabling rapid response to cache degradation.

What Is the Performance Impact of Redis Caching on Ad Predictions?

The aggregate impact of Redis caching on HypeLab's Web3 ad platform prediction service:

Caching reduces P50 latency by roughly 30% and eliminates the need for model inference on 40%+ of requests. This reduces model server load, allowing us to handle more traffic with the same infrastructure while delivering faster ad auctions for blockchain advertisers.

What Are the Key Lessons for ML Prediction Caching?

For teams considering prediction caching, key lessons from HypeLab's crypto ad network implementation:

• User-agnostic features are critical. Caching works because features are context-based, not user-based. User-specific features would produce unique keys for every request.
• Regional deployment matters. Global cache adds latency and stores irrelevant entries. Regional isolation improves both metrics.
• Version your cache keys. Model updates need clean invalidation. Version prefixes make this trivial.
• Monitor hit rate by segment. Aggregate hit rate hides problems. Segment-level monitoring reveals optimization opportunities.
• Cache warming reduces cold-start pain. Pre-load high-traffic combinations for faster recovery from restarts.

What Future Optimizations Is HypeLab Exploring?

Current caching serves us well, but improvements are possible:

Predictive cache warming: Instead of warming from historical traffic, predict which feature combinations will be popular in the next time window and pre-cache those.

Tiered caching: Hot entries in local memory, warm entries in Redis, cold entries recomputed. Further reduces latency for the most common cases.

Approximate caching: For similar (not identical) feature combinations, serve approximate predictions. Requires careful analysis of prediction sensitivity to feature perturbations.

These optimizations add complexity and require validation. The current approach is simple, effective, and well-understood, providing a strong foundation for incremental improvements.

Why Is Redis Caching Essential for Web3 Advertising?

Redis caching transforms ML prediction from a constant cost (every request runs inference) to a marginal cost (only novel feature combinations run inference). For blockchain advertising where latency is competitive survival and traffic patterns are repetitive, caching is not an optimization. It is a requirement for any serious crypto ad network.

The keys to effective ML caching: design features that enable sharing across requests, deploy regionally to match traffic patterns, instrument thoroughly to catch degradation, and maintain fallbacks for when caching cannot help. HypeLab's Web3 advertising platform embodies these principles, serving millions of cached predictions daily for publishers across the crypto ecosystem including DeFi dashboards, NFT marketplaces, and blockchain games.