The goal
The platform wanted to test a mobile-only fantasy application built around shorter, four-week leagues rather than a traditional full-season commitment, aimed at both retaining existing fantasy players and appealing to newer or more casual players unfamiliar with the traditional format. The draft experience in particular needed evaluation, along with the overall league experience across interest, credibility, and persuasion to keep using the app.
What we tested
Player behavior was recorded in context while participants completed a live draft inside the actual mobile application over a multi-week period, supplemented by a digital diary where players recorded their own reflections and answered structured questions about specific aspects of the experience. The study ran across six private leagues of eight players each, split deliberately between groups skewed toward inexperienced players, groups of experienced league veterans only, and a geographic mix, with both timed and untimed draft formats tested against each other.
What we found
The structured comparison across experience levels and draft formats surfaced specific friction points in the draft flow and onboarding sequence that were invisible when only testing with experienced fantasy players, since veterans compensated for confusing steps without consciously noticing them the way newer players did.
The takeaway
The platform came away with a genuinely better understanding of fantasy sports players' user experience needs across both experience levels, along with a clearer, more evidence-based picture of how users' thoughts, feelings, and interactions actually evolved across a full league cycle rather than just at a single point in time.
Testing timed and untimed drafts side by side also settled an internal debate that had been running without data for some time. The format that felt more natural to the design team was not necessarily the one that performed best for newer players, and only a direct comparison across both formats made that gap visible.
The six-league structure, split deliberately across experience level and geography, gave the product team confidence that the findings would hold up across a broader launch rather than reflecting the quirks of a single friend group's play style.