Nikita Boyarkin
← Volta Neobank

RAT v2 · project 20 of 23

A/B: a segment offer lifts gap segments (+3.3pp 45+, +4.2pp families, +3.8pp travelers) — Holm-significant, but the anchor barely moves.

Situation
The generic upsell doesn't transfer — the previous project recommended segment offers, but they went untested
Task
Own the head-to-head test of the generic upsell against the segment offer the earlier project left unverified
Action
Randomized A/B, sample-size and SRM checks, z-test by segment with Holm correction, segment × arm interaction (DiD)
Result
45+ 1.6%→5.0%, families 4.4%→8.6%, travelers 8.8%→12.6%; anchor +0.86pp — the gap is not closed
Stack
Pythonpandas / NumPySciPy / Statsmodelsscikit-learnMatplotlib / Seabornuv + ruff
Source View on GitHub →
On this page
  1. Situation
  2. Task
  3. Actions
  4. Result
  5. Recommendations
  6. Documentation

Volta — Segment Premium Offers

Situation

Project 15 showed the generic upsell doesn’t transfer and recommended segment offers — but that was untested. This project tests generic (control) vs segment (treatment).

Task

The preceding project recommended segment offers but left them untested, so I owned the head-to-head test of the generic upsell against the segment offer.

Actions

  • Randomized A/B, sample-size/SRM checks.
  • z-test by segment with 95% CIs, Holm correction.
  • Segment × arm interaction (DiD), residual-gap analysis.

Result

  • Treatment lifts gap segments: 45+ 1.6%→5.0%, families 4.4%→8.6%, travelers 8.8%→12.6% — all Holm-significant.
  • The anchor barely moves (+0.86pp): it’s already served by the generic offer.
  • Residual gap: 45+ treatment (5.0%) is still ~3.8× below the anchor’s generic offer (18.6%).

Recommendations

  • Ship segment offers — a positive, significant ROI.
  • Plan for a persistent gap; don’t expect parity with the anchor.
  • For 45+, combine with the trust track, not the offer alone.

Documentation

Charts

Source: github.com/NikitaBoyarkin/volta-banking — 22 projects; figures recomputed from the repo's own datasets (data/*.csv) via its analysis scripts. Funnel counts from data/volta_funnel_data.csv (10,000 users); A/B, retention, segmentation, churn, RFM, CLV, attribution, anomalies, spend, support, NPS, JTBD, unit economics, premium, KYC deep-dive, referral, assisted CAC, FX sourcing, premium offers, anchor CAC and dormant win-back follow the published project narrative (README + part pages).

Segment premium offers: conversion, control vs treatment

Randomized A/B: generic upsell (control) vs segment offer (treatment) across four segments. Treatment lifts gap segments but does not reach the anchor.

0 5 10 15 20 Young Professionals — Control (generic): 18.62 18.62 Digital Newcomers 45+ — Control (generic): 1.61 1.61 Family Budgeters — Control (generic): 4.42 4.42 Travelers — Control (generic): 8.78 8.78 Young Professionals — Treatment (segment): 19.49 19.49 Digital Newcomers 45+ — Treatment (segment): 4.95 4.95 Family Budgeters — Treatment (segment): 8.58 8.58 Travelers — Treatment (segment): 12.57 12.57 Young Professionals Digital Newcomers 45+ Family Budgeters Travelers Segment Premium conversion, % Control (generic) Treatment (segment)
Key takeaways
  • Treatment lifts gap segments: 45+ 1.6%→5.0%, families 4.4%→8.6%, travelers 8.8%→12.6% — all Holm-significant.
  • The anchor barely moves (+0.86 pp): 45+ treatment (5.0%) is still ~3.8× below the anchor's generic offer (18.6%).

Segment premium offers: treatment lift

Treatment lift over control by segment (pp). Gap segments yield 3–5× the anchor's lift.

0 2 4 6 Young Professionals — Lift: 0.86 0.86 Digital Newcomers 45+ — Lift: 3.33 3.33 Family Budgeters — Lift: 4.16 4.16 Travelers — Lift: 3.78 3.78 Young Professionals Digital Newcomers 45+ Family Budgeters Travelers Segment Lift, pp
Key takeaways
  • Gap segments yield 3–5× the anchor's lift: +3.33 pp 45+, +4.16 pp families, +3.79 pp travelers vs +0.86 pp at the anchor.
  • The offer adds +3.3…+4.2 pp where generic failed, but narrows the gap rather than closing it — there is no silver bullet.

Volta project map

Volta overview →

Core

Extended

Market & Jobs

RAT v2

Causal