Crypto moves quickly, but speed without discipline burns capital. A structured experimentation program transforms your growth hypotheses into validated strategies – so you invest more in what works and stop what does not before it depletes your treasury.
The Challenge
Growth decisions rely on conviction rather than evidence
The founder thinks Twitter threads drive growth. The marketing lead thinks paid acquisition is the answer. The BD team believes partnerships will break through. Everyone has an opinion, nobody has data, and the loudest voice wins the budget. In crypto, where market conditions shift weekly, conviction without evidence is just expensive guessing. The companies that scale are the ones that test their assumptions before committing resources.
Traditional experimentation tools do not work for web3 products
Optimizely and LaunchDarkly were built for SaaS products with logged-in users and persistent sessions. Crypto products have pseudonymous wallet addresses, fragmented sessions across devices, and conversion events that happen on-chain rather than on your website. Running clean experiments requires custom instrumentation that can attribute wallet-level behavior across the full journey from ad impression to on-chain action. Off-the-shelf tools cannot do this.
Your team ships updates without measuring their impact
A new onboarding flow went live last month. Did it improve activation? Nobody knows because there was no control group, no baseline measurement, and no success metric defined before the change shipped. The team assumes it helped because the metrics went up that week, but three other things changed simultaneously. Without experimental discipline, you cannot separate signal from noise, and you end up scaling things that are not actually working.
Experimentation velocity is close to zero
Your growth team runs maybe one or two tests per quarter, and each one takes weeks to set up because it depends on engineering resources that are allocated to protocol work. At that velocity, it takes years to find the growth levers that matter. High-growth companies run 10-20 experiments per month. The gap between your testing velocity and theirs is the gap between their growth rate and yours.
How We Can Help
We create the experimentation program from scratch – the infrastructure, processes, and culture. We begin by determining what you truly know and what you merely assume. We audit each growth channel and funnel stage, uncover the assumptions within your existing strategy, and turn them into testable hypotheses prioritized by potential impact and effort.
Next comes the experimentation infrastructure. For crypto products, that requires a testing framework capable of handling wallet-based identity, cross-device attribution, and on-chain conversion tracking. We implement the event collection, identity resolution, and statistical analysis tools required for rigorous experimentation in a web3 environment. This infrastructure operates separately from your protocol engineering pipeline, ensuring growth experiments do not compete with product development.
After the infrastructure goes live, we set the experimentation cadence. The weekly cycle includes reviewing previous results, designing the next round of tests, launching experiments, and monitoring statistical significance. The objective is to keep three to five experiments running concurrently at all times, supported by a structured framework for scaling winners, refining inconclusive results, or ending losing tests.
We also teach your team experimentation methodology. That means more than learning the tools – it includes forming strong hypotheses, establishing appropriate sample sizes, avoiding premature reviews of results, and interpreting data honestly when outcomes differ from expectations. Shifting the culture from opinion-led to evidence-led growth is the engagement's most valuable result.
The program is self-sustaining once your team can independently develop hypotheses, design experiments, conduct tests, and allocate resources based on evidence. We work toward that outcome from the first day.
What we deliver
- Growth assumption audit converting strategy assumptions into testable hypotheses
- Custom experimentation infrastructure for wallet-based web3 products
- Weekly experimentation cadence with structured review and decision framework
- Experiment playbook covering hypothesis formation, design, analysis, and scaling
- Team training on experimentation methodology and statistical rigor
What separates a growing crypto company from one that stalls is not budget or brand awareness—it is experimentation velocity. A team testing 50 hypotheses each quarter will always outperform one debating 5 hypotheses in a meeting room.
Our Methodology
During the first two weeks, we audit your existing growth strategy and uncover every assumption that has not been tested. We map the funnel, evaluate channel performance data, and create a prioritized hypothesis backlog. Every hypothesis includes a defined expected impact, proposed test design, and minimum sample size required to achieve statistical significance.
From weeks three through six, we develop the experimentation infrastructure and launch the initial wave of tests. We implement tracking, identity resolution, and statistical analysis tools while concurrently shipping the highest-priority experiments. Early test outcomes guide how later waves are prioritized – the backlog remains a living document that changes with what we learn.
The last four weeks focus on establishing the operating rhythm and training your team. At this stage, the infrastructure is stable, results from the first wave are available, and your team has joined enough experimentation cycles to manage the process independently. We document the methodology in an internal playbook, implement automated reporting, and establish the weekly review cadence needed to maintain experiment velocity after the engagement concludes.
Our Working Process
The first 30 days deliver the hypothesis backlog and experimentation infrastructure. Your engineering and growth teams contribute to the audit and infrastructure choices. Deliverable: a working experimentation stack plus a prioritized backlog of 20-30 testable hypotheses.
Days 30-60 focus on peak experiment velocity. We operate three to five tests concurrently, evaluate results each week, and launch the next wave according to what we learn. Your team helps design experiments and participates in analysis, developing the muscle memory needed for rigorous testing.
Days 60-90 focus on handoff and scaling. We document the methodology, train the entire growth team, and put the weekly cadence in place. A detailed experiment report includes every test conducted, its results, and the strategic implications. Engagements typically last 3-4 months, while some clients continue with advisory support for quarterly experiment portfolio reviews.
Team structure: a senior growth strategist directs hypothesis development, a growth engineer develops the testing infrastructure, and a data analyst manages statistical analysis. You grant access to your codebase and analytics, along with at least one growth team member who can act as the internal experiment lead.
- Sustained experiment velocity of 10-20 tests per quarter
- Evidence-based channel and budget allocation
- Measurable conversion rate improvements at key funnel stages
- Internal team capability for independent experimentation
If your crypto / defi company needs growth experimentation leadership, we should talk.
Expand your marketing team output with our experts
Let us take a custom approach to your growth goals by assembling and leading the best-in-class marketing team to support your next stage.
Frequently asked questions
What does a growth experimentation program cost for a crypto company?
A 90-day experimentation program build-out generally costs $40K-$80K. That includes the hypothesis audit, infrastructure development, initial experiment wave, and team training. The investment recoups itself when just one experiment uncovers a growth lever worth scaling – which typically occurs within the first 30 days. Companies that already have analytics infrastructure can begin with a lighter 30-day engagement centered on hypothesis development and experiment design for $15K-$25K.
How soon does experimentation generate actionable results?
Initial experiment results arrive within two to three weeks after the infrastructure is deployed. The timeline for statistical significance depends on traffic volume – high-traffic DeFi protocols may achieve significance within days, whereas lower-traffic products can require two to three weeks for each test. The most impactful findings usually emerge from the first experiment wave, when the clearest bottlenecks are tested and addressed.
How will your experimentation team work with our current teams?
We collaborate directly with your growth, engineering, and data teams. Experiments are jointly designed, operated within your current infrastructure, and evaluated during shared weekly meetings. Knowledge transfer is the stated objective – your team takes part at every stage, enabling them to manage the program once we leave. We contribute production code to your repo and work with your data tools rather than using a separate black-box system.
How is Winston Francois different from engaging a growth experimentation consultant?
Consultants provide guidance about what to test. In a single engagement, we develop the infrastructure, conduct the tests, evaluate the findings, and train your team. We also know the unique difficulties involved in experimenting with web3 products: wallet-based identity, on-chain conversion events, pseudonymous users, and fragmented sessions. A generalist experimentation consultant would need weeks simply to understand these limitations.
Can growth experiments run without interrupting our core product development?
Yes, and this is a fundamental design principle behind our experimentation stack. Growth experiments use an independent deployment pipeline with feature flags, ensuring they never obstruct or disrupt protocol releases. The growth engineering layer operates above your product rather than within it. Your core team continues shipping the product as the experimentation program operates in parallel.
Which hypotheses should a crypto company test first?
Begin with conversion funnel bottlenecks – onboarding completion, wallet connect success rate, and first-transaction completion – because they offer the greatest immediate impact. Next, test acquisition channel effectiveness, messaging variations, and incentive structures. Prioritize the hypothesis backlog by expected impact multiplied by confidence that each test will be conclusive, rather than by what appears most interesting.
