Experimentation & Growth Testing

Run tests that hold up through seasonal swings.

Demand in the region swings around Ramadan, Eid, White Friday and the summer. A test that runs across one of those shifts can read a seasonal swing as a win. Traffic split between languages and countries makes each test smaller still.

Sinfa's Experimentation advisory for MENA teams designs A/B tests that reach decisions despite sharp seasonal swings and traffic split across languages and countries. We plan test windows around the Hijri calendar, size samples per audience, judge results on delivered revenue, and fix stopping rules before launch so results survive the next peak.

Last updated

Seasonal swings and split traffic make tests hard to read.

Shopping behavior changes during Ramadan: browsing moves late into the night, basket sizes shift, and categories rise and fall. White Friday and Eid bring their own peaks. A test that runs across one of these changes measures the calendar as much as the variant.

Traffic is also split. Arabic, French and English visitors, and visitors from each country, can respond differently, so a test sized on total traffic may lack the power to read each audience.

Order-based metrics add a third risk. A variant that raises cash-on-delivery orders can also raise refusals. Without a guardrail on delivered revenue, the test can pick a loser.

Designed for evidence over velocity.

  1. Program audit We review your existing testing program: test selection, statistical methodology, win rate analysis, and whether results are informing decisions.
  2. Hypothesis development Each test starts with a specific, testable hypothesis linked to a mechanism. A weak hypothesis asks 'will this layout convert better?' A strong one states 'reducing cognitive load at this step will improve completion rate because...'
  3. Statistical design We set sample sizes using power analysis, choose primary and guardrail metrics before the test runs, and pick the statistical method that fits the question you need answered.
  4. Decision criteria We pre-specify what constitutes a win, a loss, and an inconclusive result, and what the team does in each case. Without a decision rule, a result gives you data to argue over and nothing to act on.
  5. Results interpretation We review results in context: statistical significance alone is not sufficient. We assess effect size, confidence interval, and whether the result is robust enough to act on.

Who this is for

  • Growth leads at $3M–$150M regional brands Who run tests but see results reverse after the next seasonal peak.
  • Heads of e-commerce with multilingual sites Who need to know whether a winning test holds for each language their visitors use.
  • Product teams building their first testing program Who want the design rules right from the start, before a year of unreadable tests piles up.

What do clients ask before starting Experimentation & Growth Testing?

Should we stop testing during Ramadan?

You can keep testing, but run tests that start and end within the period, and don't roll results out to the rest of the year without a check. Behavior during Ramadan differs from behavior after it.

Do we need separate tests for each language?

Not in every case. Where it matters, we size the test so you can read each language on its own, and pool languages where the change is the same for all of them.

What should our primary metric be if many orders are cash on delivery?

Orders can stay the primary metric for speed, with delivered, paid revenue as a guardrail checked once deliveries settle. For changes to payment or checkout, delivered revenue should be primary.

Book a consultation with Sinfa · [email protected]

Editions: North America · Europe · MENA