When should a business personalize?
A randomized customer experiment connects causal estimates to a contact decision—and tests whether added complexity earns its place.
64,000 customers Three randomized arms 12,800 held-out customersLearned policy minus simple campaign, per customer.
Includes zero; superiority is not established.
For this population and the predefined cost scenario.
What the experiment supports
The learned candidate did not establish an improvement over the uniform men's email campaign. Selection was frozen before evaluating the final sample. The study preserves that result instead of optimizing against the test set.
“Men's” and “women's” describe campaign content. Customer gender is not observed. Financial contribution uses assumed margin and contact cost; it is not a measured profit figure.
Inspect the decision assumptions
Choose a predefined scenario. These estimates reuse the evaluation sample; browsing them does not establish a new confirmatory winner. Capacity limits are evaluated for this fixed customer batch.
| Policy | Value/customer | Pointwise 95% interval | Contacted |
|---|
How to challenge the result
Inspect the experiment and evaluation
Training, development and evaluation sets are separate. Assignment probabilities are known. A cross-fitted doubly robust learner competes with uniform campaigns and a transparent purchase-segment rule. The primary policy contrast uses paired customer scores and a 2,000-resample bootstrap.
Understand the score
Gamma_a = mu_a(X) + I(A=a) × (Y − mu_a(X)) / p_a
For the chosen action, scenario value is margin times the estimated spending score, less contact cost. Correct randomization supports value estimation even if the spending regression is imperfect.
Check uncertainty and transport limits
99.14% of the evaluation customers spent nothing. Sparse purchases and heavy tails limit precision. Intervals condition on the trained policies; they do not include variation from retraining on another experiment. Historical results do not establish current Indian campaign performance. The scenario grid is exploratory, with pointwise intervals rather than simultaneous confidence bands.
Technical report · Decision memo · Study protocol · Data provenance
Download this page to use the scenario explorer offline. All displayed values are saved aggregate results; no customer-level data or external service is used.