QAVR - Q-Value Augmented Vector Retrieval

Memory system that learns which information is most useful over time.

Concept

Standard vector retrieval returns results by semantic similarity alone. QAVR adds learned utility scoring based on actual usage outcomes:

code

Final Score = (1 - α) × Semantic Similarity + α × Q-Value

Where:

•Semantic Similarity: How relevant the memory is to the query
•Q-Value: Learned utility score (0.0 - 1.0) based on past usefulness
•α: Blending factor (increases as more data collected)

How It Works

Cold Context (< 100 interactions)

•Pure semantic similarity (α = 0)
•Q-values being collected but not used
•Learning phase

Warm Context (≥ 100 interactions)

•Q-value re-ranking active (α = 0.3)
•Memories that led to successful outcomes ranked higher
•Continuous learning from feedback

Q-Value Updates

After each interaction:

python

# Positive outcome (task succeeded, user satisfied)
q_new = q_old + learning_rate * (reward - q_old)
reward = 1.0 for success, 0.0 for failure

# Temporal decay (unused memories fade)
q_decayed = q_old * decay_factor  # e.g., 0.99 per day

Implementation

Storage Format

json

{
  "memories": {
    "memory_id_1": {
      "q_value": 0.75,
      "access_count": 12,
      "last_accessed": "2026-01-26",
      "success_count": 9,
      "failure_count": 3
    }
  },
  "contexts": {
    "debugging": {"interactions": 82, "mode": "cold"},
    "coding": {"interactions": 156, "mode": "warm"}
  },
  "config": {
    "learning_rate": 0.1,
    "decay_factor": 0.99,
    "warm_threshold": 100
  }
}

Integration with Vector DB

python

def qavr_query(query_text, collection, n_results=5):
    # Get semantic results
    results = collection.query(
        query_texts=[query_text],
        n_results=n_results * 2  # Over-fetch for re-ranking
    )

    # Apply Q-value re-ranking if warm
    if context_is_warm():
        results = rerank_by_qvalue(results, alpha=0.3)

    return results[:n_results]

Feedback Signals

QAVR learns from implicit signals:

Signal	Interpretation	Reward
Memory used in successful task	Highly useful	+1.0
Memory retrieved but not used	Somewhat relevant	+0.1
Memory retrieved, task failed	Possibly misleading	-0.2
Memory not retrieved for days	Decaying relevance	decay

Benefits

•Personalization: Learns YOUR usage patterns
•Noise Reduction: Unhelpful memories sink to bottom
•Efficiency: Most useful info surfaces first
•Adaptation: Adjusts as your needs change

Configuration

json

{
  "qavr": {
    "enabled": true,
    "learning_rate": 0.1,
    "decay_factor": 0.99,
    "warm_threshold": 100,
    "alpha_warm": 0.3,
    "contexts": ["debugging", "coding", "research"]
  }
}

Monitoring

Check QAVR status:

•Total memories tracked
•Context modes (cold/warm)
•Top Q-value memories
•Learning progress