Custom Ranking

What is Custom Ranking?

Custom ranking is the practice of defining a ranking formula beyond pure text relevance (BM25). It blends additional, structured signals—popularity, price, margin, rating, stock, recency, personalization—to order results that still respect query match.

How It Works (quick)

• Base relevance: Retrieve candidates via lexical/semantic recall.
• Signals: Compute attributes (e.g., in_stock, rating, margin, click_through_rate).
• Scoring: Combine with weights/transformations (log, min-max, z-score) into a single score.
• Policies & caps: Apply boost caps, time decay, and hard filters (e.g., must be in stock).
• Feedback loop: A/B test weights; update from click/conversion data or learning-to-rank models.

Why It Matters in E-commerce

• Aligns with goals: Balance relevance with profitability, availability, and quality.
• Consistency: Predictable ordering across categories and seasons.
• Better UX: Reduce “dead” results (OOS, discontinued) and surface trusted items.

Best Practices

• Guardrails first: Hard-filter out-of-stock, disabled, or restricted items before scoring.
• Normalize signals: Scale per category/locale to avoid dominance (e.g., price).
• Small weights: Start with gentle nudges (+10–30%); cap boosts to protect relevance.
• Diversity rules: Prevent one brand or variant from flooding top slots.
• Measure the right KPIs: NDCG@k, CTR, conversion, margin per session, return rate.
• Evolve to LTR: Use learning-to-rank for data-driven weights; keep a simple fallback.

Challenges & Trade-offs

• Overfitting to margin/popularity: Hurts discovery and long-tail.
• Noisy signals: Ratings/reviews can be sparse or biased; smooth and threshold.
• Locale/category bias: A single global formula may misrank niches—segment it.
• Explainability: Complex formulas can be hard to debug; log features per result.

Examples

• Blend BM25 score + in_stock (hard filter) + rating (capped) + margin (small boost) + recency (decay).
• Category “sneakers”: prioritize size-availability for the shopper’s size; cap per-brand results.
• Seasonal campaign: temporary freshness boost with automatic expiry.

Summary

Custom ranking balances text relevance with business and quality signals. With guardrails, normalization, caps, and ongoing testing—or LTR—you’ll rank items that are both relevant and right for the business.