• Airbnb

• Instacart

• Uber

• ElasticSearch

Search System

Characteristic

• Relevance between the search query and the search result is critical
• Query understanding
• Strict latency
• Heterogeneous query conditions, e.g., text-based search on e-commerce platform, location-based search on Airbnb

Design Overview

• Problem Definition/Formulation

• Metric Measurement

  • Offline
    • Recall
    • AUC
    • Normalized Cross Entropy (NCE)
    • CTR calibration
    • nDCG
  • Online
    • Revenue
    • nDCG

• Data Collection

  • Positive Samples
  • Negative Samples

• Feature Engineering

  • Feature examples:
    • item-wise ones
      • Airbnb: various properties of the listing, e.g., price, amenities, location, number of bedrooms, guest rules, historical booking count, etc.
  • Engineering Tips:
    • Feature normalization is important
    • Normalization transformation and log transformation

• Model Training

• Model Evaluation/Serving

• Model Performance Monitoring

• Challenges

Reference

• Airbnb 2018 Listing Embeddings in Search Ranking

• Instacart How Instacart Uses Embeddings to Improve Search Relevance

• System Design for Recommendations and Search

• Paper 2016 Amazon Amazon Search: The joy of ranking products

• Paper 2019 Airbnb Applying Deep Learning To Airbnb Search

• Paper 2020 Airbnb Airbnb Improving Deep Learning For Airbnb Search