Data flow · SERP Rank Lab
SERP Rank Lab — relevance ranking vs commission ranking
SERP ranking data flow: metric vs money
Data flow of a flight metasearch SERP lab: ingested measured feeds are fenced off from a modeled SERP layer that turns real route and price pairs into offers; two oracle ranking policies order those offers by relevance or by partner commission, position-biased simulated clicks are corrected with inverse-propensity weighting, and the readout shows the offline NDCG@10 and the money metric revenue per search pulling in opposite directions, 38 percent apart.
01
INGEST
02
MODEL
03
RANK A
04
RANK B
05
READOUT
MEASURED
FEEDS
MODELED
SERP
RANKING
MONEY
CLICKS
ESTIMATE
MODELED ≠ MEASURED
+38% REVENUE
RAW
Ingested feeds
orders · bookings_legs
events · islands · no FK
SES
Sessions
events · 30 min idle
join key: user_id
SER
SERP offers
query · route × price
serp_offers · modeled
NDC
NDCG@10 · MRR
offline metric
1.000 → 0.932
REL
Relevance policy
rank by relevance
oracle · sees relevance
REV
Revenue policy
rank by commission
oracle · sees commission
MON
revenue / search
partner commission
868.9 → 1,196.6
PBM
Click model · PBM
click = examine × relevant
examine_prob (1/k)^η
IPW
IPW correction
1 / propensity(k)
naive 0.163 → 0.629
LEGEND
RAW
measured feeds
SER
modeled SERP
REL
ranking policy
PBM
click & estimate
MON
money metric · focal
data flow
headline result