Study Completions Are Flat. Is the Problem in the Survey or Behind It?

By Rigoris Digital

Published
HealthcareFor Product LeadersUser Experience

The short answer

Flat study completions can reflect the participant interface, the platform work behind the study or the research and recruitment itself. Look at where participants stop and how coordinators spend their time. Software can address a workflow or interface constraint, but a build should follow that finding rather than stand in for research design.

A clay client and team member compare participant drop-off, coordinator workload and recruitment constraints in a health study.

TLDR

  • Your study is under target, but that alone doesn't tell you which piece of work to fund.
  • Separate participants not starting from participants starting and not finishing.
  • Administrative effort needs its own review; it doesn't appear in a study completion rate.

Three problems that look the same from the inside

The first is the interface. People click in and don't finish, and the loss isn't spread evenly across the questionnaire. It bunches on a few screens, usually the same few from one study to the next. A clinician who gave you the gap between two appointments and met one of those screens has given you nothing at all, and that's the cheapest of the three problems to fix.

A study that lands under target can prompt another recruitment push before anyone reviews the screens where participants stopped. Check those screens before treating a larger panel as the answer. Recruitment and completion are different questions, and spending on one doesn't establish that it was the constraint.

The second problem sits behind the study rather than inside it, and it turns up once the panel outlives the questionnaire. The same clinicians come back for study after study. They're owed money they'd like to see. The permission they gave you has to outlast the study it was given for, someone has to know who's allowed to see whom, and your own coordinators spend their working day in an administrative screen. A survey tool runs a questionnaire and hands you the answers. Almost nothing on that list is a questionnaire.

The third is that neither of those is your constraint. The limit is in the research rather than the software: who is reachable at all, what you're asking people to have a view on, and why anyone would answer. No build touches that, and we'd rather say so than sell around it.

How to tell which one you've got

Start with where people stop rather than with how many you invited. Pull drop-off by question across your last three studies rather than your worst one, because one bad study points at that study and three point at your instrument. If plenty of people start, few finish, and the loss keeps landing in the same places, it's the interface, and your existing platform can usually produce that answer in a morning.

If completion is fine and the pain is all on your side of the glass, it's the platform. That signal isn't in the study data at all, it's in how your team spends its week: a question about one participant that takes an export or a developer to answer, and work that has quietly moved out of the system into files sitting beside it. None of that shows up in a completion rate, which is why it runs for years before anyone puts a cost against it.

If few people start at all, while the ones who do begin are finishing, a software project is unlikely to be your answer. That's a sampling and research design conversation, and it belongs before a build conversation rather than after one.

Most teams have two of these at once, and the order isn't a matter of taste. The interface is the fastest and the cheapest, and it pays back inside the next study, so it's worth doing first even when a larger build is already coming. The platform problem is the one that compounds: it doesn't cost you a study, it costs you a share of everybody's week, permanently. The third costs nothing to rule out and is the only one that can make the other two irrelevant, which is why it goes first in any honest conversation.

Choose the next piece of work

Write down the point where the work stops, the person affected and what happens instead. That gives you a question to investigate before you choose a project.

The health research overview links to the different pieces of work.

If the question is still what to build, read how product strategy defines a first scope before asking for a development quote.

Common questions

Few starts are a reason to investigate recruitment, reach and the research request. Plenty of starts followed by repeated drop-off at particular screens are a reason to investigate the participant experience. These observations guide the next investigation; the completion total alone doesn't establish which change is needed.

Review the work behind the study separately. Participant records, permissions, payments and repeated exports can create administrative effort without changing the completion rate. Follow a coordinator's routine task through the existing tools and identify where work leaves the system before scoping a platform change.

A build should follow evidence of a software or workflow constraint. It doesn't establish that the right participants are reachable or that the research request gives them a reason to respond. Resolve those research questions alongside the software review, rather than assuming a new platform will answer them.