# Accuracy vs Latency

Quick Reference: Throughput vs Latency | Step 9: Observability

# Quick Reference

Accuracy: Precise results, may take longer

Latency: Fast results, may be approximate

Trade-off: Exact algorithms vs approximate algorithms

# Clear Definition

Accuracy vs Latency trade-off: Exact algorithms provide accurate, precise results but may be computationally expensive and slow. Approximate algorithms provide fast results with acceptable accuracy, trading precision for speed.

💡 Key Insight: Use approximate algorithms when "good enough" accuracy is acceptable and speed is critical. Use exact algorithms when accuracy is non-negotiable (financial, medical, legal). Many systems use a hybrid approach - approximate for real-time, exact for batch processing.

# Core Concepts

# Exact Algorithms

Characteristics:

• Process all data
• Guaranteed accuracy
• Deterministic results
• Higher computational cost
• Higher latency

Examples:

• Full database scans
• Exact count queries
• Precise calculations
• Complete data analysis
• Deterministic sorting

When to Use:

• Financial transactions (must be exact)
• Medical records (accuracy critical)
• Legal documents (precision required)
• Audit trails (exact counts needed)
• Regulatory compliance (no approximations)

Trade-offs:

• ✅ Guaranteed accuracy
• ✅ Deterministic results
• ✅ No false positives/negatives
• ❌ Higher latency
• ❌ Higher computational cost
• ❌ May not scale to large datasets

# Approximate Algorithms

Characteristics:

• Process subset or use heuristics
• Acceptable accuracy (usually 95-99%)
• Probabilistic results
• Lower computational cost
• Lower latency

Examples:

• Bloom Filters: Fast membership testing (may have false positives)
• HyperLogLog: Cardinality estimation (small error margin)
• Sampling: Process subset of data
• Sketching: Compact data structures
• Monte Carlo: Statistical approximation

When to Use:

• Real-time analytics (speed critical)
• Large-scale counting (exact count expensive)
• Recommendation systems (approximate OK)
• Monitoring dashboards (trends more important than exact)
• Search ranking (approximate relevance OK)

Trade-offs:

• ✅ Lower latency
• ✅ Lower computational cost
• ✅ Scales to large datasets
• ✅ Real-time processing possible
• ❌ May have errors
• ❌ Probabilistic results
• ❌ May need exact verification

# Common Approximate Algorithms

# 1. Bloom Filters

What it Does: Fast membership testing (is item in set?)

Accuracy: May have false positives, no false negatives

Latency: O(k) where k is number of hash functions (very fast)

Memory: Compact (bits per item)

Use Cases:

• Cache lookups (check if in cache)
• Duplicate detection
• Database query optimization
• Distributed systems (check if data exists)

Example:

from pybloom_live import BloomFilter

# Create bloom filter
bf = BloomFilter(capacity=1000000, error_rate=0.001)

# Add items
bf.add("user123")
bf.add("user456")

# Check membership (very fast)
if "user123" in bf:  # Fast check
    # May need to verify in actual database
    pass

# 2. HyperLogLog

What it Does: Estimate cardinality (count unique items)

Accuracy: ~0.81% standard error

Latency: O(1) per item (very fast)

Memory: ~12KB for millions of items

Use Cases:

• Count unique visitors
• Count distinct values
• Analytics (approximate counts)
• Large-scale counting

Example:

from hyperloglog import HyperLogLog

# Create HyperLogLog
hll = HyperLogLog(0.01)  # 1% error

# Add items
for user_id in user_ids:
    hll.add(user_id)

# Get approximate count (very fast)
approx_count = len(hll)  # Fast, approximate

# 3. Sampling

What it Does: Process subset of data

Accuracy: Depends on sample size and distribution

Latency: Proportional to sample size (much faster)

Memory: Much less (only sample)

Use Cases:

• Large dataset analysis
• A/B testing
• Quality assurance
• Statistical analysis

Example:

import random

# Sample 1% of data
sample_size = len(data) // 100
sample = random.sample(data, sample_size)

# Process sample (much faster)
result = process(sample)

# Extrapolate to full dataset
full_result = result * 100

# 4. Count-Min Sketch

What it Does: Estimate frequency of items

Accuracy: Overestimates (never underestimates)

Latency: O(d) where d is number of hash functions

Memory: Compact (counters)

Use Cases:

• Top-K items
• Frequency estimation
• Heavy hitters detection
• Stream processing

# 5. Locality-Sensitive Hashing (LSH)

What it Does: Approximate nearest neighbor search

Accuracy: May miss some neighbors

Latency: Much faster than exact search

Memory: Compact hash tables

Use Cases:

• Similarity search
• Recommendation systems
• Image search
• Duplicate detection

# Decision Framework

# Step 1: Assess Accuracy Requirements

Questions to Ask:

• What's the acceptable error margin?
• What happens if result is approximate?
• Is exact accuracy required by law/regulation?
• Can we verify approximate results later?

Accuracy Critical (Use Exact):

• Financial calculations
• Medical records
• Legal documents
• Audit trails
• Regulatory compliance

Approximate Acceptable (Use Approximate):

• Analytics dashboards
• Recommendations
• Monitoring metrics
• Search results
• Real-time insights

# Step 2: Evaluate Latency Requirements

Questions to Ask:

• What's the acceptable latency?
• Is real-time processing required?
• Can we process in batch?
• What's the cost of delay?

Low Latency Critical (Consider Approximate):

• Real-time dashboards
• User-facing features
• Live monitoring
• Interactive systems

Latency Flexible (Can Use Exact):

• Batch processing
• Background jobs
• Reports
• Analytics

# Step 3: Consider Data Scale

Large Scale (Consider Approximate):

• Millions/billions of items
• Real-time streams
• High-volume data
• Limited memory

Small Scale (Can Use Exact):

• Thousands of items
• Batch processing
• Sufficient resources
• Accuracy critical

# Step 4: Choose Strategy

Pure Exact:

• Use when: Accuracy non-negotiable
• Example: Financial transactions

Pure Approximate:

• Use when: Speed critical, accuracy acceptable
• Example: Real-time analytics

Hybrid (Recommended):

• Approximate for real-time
• Exact for batch verification
• Example: Real-time dashboard + nightly exact reports

# Real-World Examples

# Example 1: Unique Visitor Counting

Challenge: Count unique visitors to website (millions of visitors)

Exact Approach:

• Store all visitor IDs
• Deduplicate
• Count
• Latency: High (millions of operations)
• Memory: High (store all IDs)

Approximate Approach:

• Use HyperLogLog
• Add visitor ID to sketch
• Get approximate count
• Latency: Low (O(1) per visitor)
• Memory: Low (~12KB)
• Accuracy: ~99% (acceptable for analytics)

Trade-off: 99% accuracy with 1000x faster processing

# Example 2: Cache Lookup

Challenge: Check if data exists in cache (millions of keys)

Exact Approach:

• Query cache for each key
• Network round-trip per check
• Latency: High (network overhead)

Approximate Approach:

• Use Bloom Filter
• Check filter first (in-memory)
• Only query cache if filter says "maybe"
• Latency: Low (in-memory check)
• Accuracy: May have false positives (acceptable)

Trade-off: Occasional false positive vs significant latency improvement

# Example 3: Financial Transaction

Challenge: Calculate account balance

Exact Approach:

• Sum all transactions exactly
• Verify against ledger
• Latency: Acceptable (accuracy critical)
• Accuracy: 100% (required)

Approximate Approach:

• NOT RECOMMENDED
• Financial data must be exact
• Regulatory requirement

Trade-off: Accuracy is non-negotiable, latency acceptable

# Hybrid Approaches

# Pattern 1: Approximate + Exact Verification

┌─────────────────────────────────────┐
│  1. Approximate (Fast)              │
│     - Quick estimate                │
│     - Real-time response            │
├─────────────────────────────────────┤
│  2. Exact (Background)              │
│     - Verify approximate result     │
│     - Update if needed              │
│     - Batch processing              │
└─────────────────────────────────────┘

Example: Real-time dashboard shows approximate count, nightly job calculates exact count.

# Pattern 2: Tiered Accuracy

┌─────────────────────────────────────┐
│  Tier 1: Real-time (Approximate)    │
│  - Fast, acceptable accuracy        │
├─────────────────────────────────────┤
│  Tier 2: Near Real-time (Better)   │
│  - Slower, better accuracy          │
├─────────────────────────────────────┤
│  Tier 3: Batch (Exact)              │
│  - Slowest, 100% accuracy           │
└─────────────────────────────────────┘

Example: Real-time metrics (approximate), hourly reports (better), daily reports (exact).

# Pattern 3: Sampling with Confidence Intervals

• Use sampling for speed
• Calculate confidence intervals
• Provide accuracy bounds
• User knows accuracy level

# Best Practices

# 1. Understand Accuracy Requirements

• Define acceptable error margin
• Know when exact is required
• Understand impact of approximation

# 2. Use Appropriate Algorithms

• Bloom Filters for membership
• HyperLogLog for counting
• Sampling for large datasets
• Choose based on use case

# 3. Monitor Accuracy

• Track error rates
• Compare approximate vs exact
• Alert on accuracy degradation
• Validate periodically

# 4. Implement Hybrid Approaches

• Approximate for real-time
• Exact for batch verification
• Best of both worlds

# 5. Communicate Accuracy to Users

• Show confidence intervals
• Indicate approximate values
• Provide exact values when available
• Set expectations

# 6. Verify Periodically

• Run exact calculations in background
• Compare with approximate
• Adjust algorithms if needed
• Maintain accuracy guarantees

# Quick Reference Summary

Accuracy: Precise, exact results but slower processing. Required for financial, medical, legal data.

Latency: Fast results with acceptable accuracy. Suitable for analytics, recommendations, real-time systems.

Key Algorithms:

• Bloom Filters: Membership testing (fast, may have false positives)
• HyperLogLog: Cardinality estimation (fast, ~1% error)
• Sampling: Process subset (fast, depends on sample)
• Count-Min Sketch: Frequency estimation (fast, overestimates)

Best Practice: Use approximate algorithms when "good enough" accuracy is acceptable and speed is critical. Use exact algorithms when accuracy is non-negotiable. Many systems use hybrid approaches.

Remember: Not all problems need exact solutions. Often, approximate algorithms provide the right balance of speed and accuracy for real-world systems.

Previous Topic: Throughput vs Latency ←

Back to: Step 11 Overview | Main Index