Skip to content

Session 8 - Module C: Performance Optimization - Test Solutions

Answer Key and Detailed Explanations

Question 1: Multi-Layer Caching Architecture

Correct Answer: B) It provides both ultra-fast access and distributed caching capabilities

Explanation: The multi-layer caching system combines L1 memory cache for ultra-fast millisecond access with L2 Redis for distributed caching across multiple instances. L1 provides the fastest possible access for frequently used data, while L2 Redis enables sharing cached data across multiple agent instances and survives application restarts. This architecture maximizes both performance and scalability.

# L1 provides millisecond access, L2 provides distributed capabilities
self._backends = {
    CacheLevel.L1_MEMORY: MemoryCacheBackend(config, self._stats),
    CacheLevel.L2_REDIS: RedisCacheBackend(config, self._stats)
}

Question 2: Cache Eviction Algorithm

Correct Answer: C) Both recency score and frequency score combined

Explanation: The intelligent eviction algorithm uses a combined scoring approach that considers both temporal (recency) and usage (frequency) factors. The recency score measures time since last access, while the frequency score is inversely related to access count. The combined score (recency_score * frequency_score) prioritizes removing entries that are both stale and infrequently used.

# Combined scoring for intelligent eviction
recency_score = (now - entry.last_accessed).total_seconds()
frequency_score = 1.0 / (entry.access_count + 1)
combined_score = recency_score * frequency_score

Question 3: Budget-Based Cost Optimization

Correct Answer: C) Trigger automatic scaling down of non-critical services

Explanation: At 95% budget utilization, the system triggers auto_scale_down_threshold which implements automatic scaling down of non-critical services. This is a protective measure that occurs before the emergency shutdown at 100%. It allows the system to continue operating while aggressively reducing costs to stay within budget.

# Automatic scale-down at 95% budget utilization
auto_scale_down_threshold: float = 0.95  # Trigger automatic scaling at 95%
emergency_shutdown_threshold: float = 1.0  # Emergency stop at 100%

Question 4: Model Selection Algorithm Weights

Correct Answer: C) It weights cost and quality at 40% each, speed at 20%

Explanation: The intelligent model selection algorithm uses a balanced approach with cost efficiency and quality being equally important (40% each), while speed has lower priority (20%). This weighting reflects the business priority of optimizing costs while maintaining quality, with performance being important but secondary.

# Balanced scoring with cost and quality priority
combined_score = (cost_score * 0.4 + quality_score * 0.4 + speed_score * 0.2)

Question 5: Connection Pooling Purpose

Correct Answer: B) To reduce latency by reusing HTTP connections across requests

Explanation: Connection pooling dramatically reduces latency by maintaining persistent HTTP connections that can be reused across multiple requests. This eliminates the overhead of establishing new TCP connections, SSL handshakes, and DNS resolution for each request. The pooling configuration includes keepalive timeouts and connection limits to optimize performance while managing resources.

# Connection pooling reduces latency through reuse
connector = aiohttp.TCPConnector(
    limit=20,  # Max connections
    limit_per_host=10,
    keepalive_timeout=30,  # Maintain warm connections
    enable_cleanup_closed=True
)

Key Performance Optimization Concepts

Caching Strategies

  • L1 Memory Cache: Provides millisecond access for hot data
  • L2 Redis Cache: Enables distributed caching with persistence
  • Intelligent Eviction: Uses recency and frequency for optimal cache management
  • Cache Promotion: Automatically moves popular data to faster cache levels

Cost Management

  • Multi-tier Budgeting: Daily, weekly, and monthly budget tracking
  • Automated Protection: Automatic scaling controls at budget thresholds
  • Model Right-sizing: Intelligent selection based on requirements and costs
  • Emergency Controls: Graceful degradation when budget limits are reached

Performance Optimization

  • Connection Pooling: Reuses HTTP connections for reduced latency
  • Request Batching: Groups requests for improved throughput
  • Context Management: Intelligent pruning of conversation history
  • Memory Optimization: Efficient resource usage with garbage collection control

Business Value

These optimization techniques deliver: - Cost Savings: Up to 90% cost reduction through intelligent model selection - Performance Gains: Millisecond cache access and reduced connection overhead - Scalability: Distributed caching and connection pooling support high concurrency - Reliability: Budget protection prevents service disruption from cost overruns

← Back to Module C