When you’re checking off items from your clinical trial site selection checklist, you or your trial team probably have a number of steps to identify who to start first, later, and…possibly never.

But are the measures you’re using today giving you the best chance for a successful trial?

If you’re still looking at performance from a “pass/fail” perspective — or even less insightful, an “attended/didn’t attend” perspective, you’re missing out on some powerful metrics that can help you  identify or develop that high-performing clinical trial team that succeeds with the least risk and fewest delays. 

Add these metrics into your clinical trial site selection checklist, and you might be surprised at how it changes your site ranking.

Metric #1: Mastery – Does the Clinical Site Staff Know the Content?

When you think of training your investigators or your clinical trial sites, you probably envision a live presentation, or perhaps a web meeting.

How do you measure success? Is it simply by attendance?

Or do you include an assessment? If so, that assessment is a critical part of completing the training checklist item, and identifying who passed, and who needs more work. 

But think about the purpose of that training, and if you use one, that assessment. Are you pushing your CRAs and clinical sites through a training process just to complete your checklist? 

If you expect your clinical teams and staff to complete any sort of eLearning or trial-related content review, I bet that you’re doing so in order to prepare them to use that knowledge in the clinical trial.

If you’re putting your trial staff and team members through a training, you’re probably doing so in order to prepare them to use that knowledge in the field.

The term mastery generally describes someone’s level of knowledge. You can think of mastery as asking this question: “does the clinical site staff member know the content?”

If you’ve been using a straightforward pass/fail type of assessment, you understand this concept already. But – knowledge is not so straightforward – it’s more of a spectrum.

Cognitive scientists often break that spectrum into a matrix with four blocks:

• Masters – confidently correct
• Misinformed – confidently incorrect
• Doubters – unsure but correct
• Uninformed – only sure about one thing, which is that they don’t know the content

There is a lot on the line with any clinical trial. Looking at the groups above, your pass/fail model of mastery gives you both the masters and the doubters. But in your trial, where you may be dealing with great risk and potential adverse events, do you want the doubters? 

What if you could move the doubters into the masters group? You can, with enough mediation – but in order to do so, you have to know who the doubters are. That’s why you need to combine mastery with metric #2, confidence.

Metric #2: Confidence – Is the Clinical Site Staff Comfortable Making the Right Decision?

Imagine this scenario:

Your entire clinical trial team completes an online course covering your upcoming trial protocol, and related essential documents. At the end of the course, you present them with a multiple-choice quiz.

Some of the team might answer correctly and be completely sure about their answers because they have gained the knowledge and are ready to apply what they have learned in trial. Others, however, may get through the test by completely guessing. Perhaps they all pass. But – remember the doubters I introduced in Metric #1? And is it possible that the uninformed could have guessed correctly often enough to pass?

A simple “pass” doesn’t mean all of your site staff are ready to make the right decisions in order to adhere to the protocol in the field.

Using confidence as a metric is an excellent way to separate those who truly possess the knowledge from those who got lucky. And since most (78% to be exact) life sciences companies indicate they need to improve their ability to identify which sites are truly ready to start a trial, this is a metric that you can start using to leapfrog the status quo.

78% of life sciences companies say they need to improve how they identify which sites are ready to start a trial.

How to use confidence as a clinical trial readiness metric:

Using confidence is fairly simple, but it helps to understand what using it will reveal to you. We already discussed these four areas of the knowledge continuum, but let’s describe them just a tiny bit more:

• Masters – those that have the knowledge and the confidence to use that knowledge in the clinical trial.
• Misinformed – those that form a great risk because they don’t have the knowledge, and yet are highly confident that they do. These people are confidently wrong. Could these be the source of most missteps or deviations?
• Doubters – those that have the knowledge you need, but aren’t confident enough to apply it. They might perform more slowly in a real life situation.
• Uninformed – those that neither know your content, nor believe they do. These team members are definitely not ready. 

How to implement confidence:

Implement confidence scoring within your assessments and tests. You must devise a scoring scheme that incorporates points or negative points for both the correctness of the answer, and the confidence about the answer. 

For example, we designed our product Ready to incorporate confidence for all assessment questions, and then combine those data to assign an overall readiness score to individuals. 

The great thing about fully identifying who falls into the masters, misinformed, doubters, and uninformed groups is that you now have a clear understanding of how to proceed. For example, a simple plan could be:

• Start the sites that have masters as soon as possible.
• Retrain or coach the sites that have doubters in order to increase their confidence, so they can start next.
• De-prioritize the sites with the misinformed and the uninformed, because these sites will require more remediation and are not ideal for your first sites to start.

This may seem too simple, especially for the last step. How do we know how to approach these misinformed and uninformed sites? It’s a tough question, but there’s yet another interesting metric you can use to prioritize those sites and staff even further: mindset.

Metric #3: Mindset – Are Site Staff Capable of Improvement?

Mindset is an interesting metric, and one that is probably new to you. And there has been so much written about mindset, primarily focused on the research and writings of Carol Dweck. 

But let’s simplify it for our purposes: you have site staff who fall into the misinformed (confidently wrong) and uninformed (not confident and wrong) groups. If you need to prioritize who will be most likely to rise to become masters…how do you do it?

With all other things being equal (location, access, capability), the metric of mindset will tell you who is capable of improving their master and confidence. 

That’s because, in the simplest terms, mindset tells us if a member of your team believes that intelligence, talent, and capability is something can be developed:

• Fixed mindset – those with a fixed mindset think intelligence and talent are inborn and can’t be changed. They may be unwilling to go beyond their comfort zone to grow.
• Growth mindset – those with a growth mindset think abilities can be developed and improved with effort. 

If you can identify the site staff with a growth mindset, you isolate those who believe that with enough work and commitment, they can improve their capability. These staff members are worth further investment, because they know they can and will improve.

So, how do you measure mindset?

Unlike mastery and confidence, mindset is more difficult to measure with simple tools. It requires behavioral analysis that something like SurveyMonkey doesn’t capture. But there are powerful tools like our product, Ready, that do measure mindset.

The way that Ready looks at mindset is a combination of:

• How much time site staff invest in content you provide them
• Whether site staff revisit content to review it
• How much effort site staff put into engaging with your content (like reviewing additional resources or notes)

We calculate mindset to analyze and identify site staff that may not be fully knowledgeable, but who invest the time in improving. These people are excellent investment opportunities because they’re committed to growing, and will likely respond to additional opportunities to learn.

Bonus Metric: Sitewide Readiness – How Did the Entire Site Perform?

All of the metrics we’ve discussed in this article have been focused on individual site staff. But that’s not how we start or conduct trials – we conduct them by site. 

So although all of these metrics are valuable for an individual, we need to look at the entirety of the trial site’s performance in order to identify which sites are the best candidates for conducting a clinical trial successfully.

But once you have the data for each individual at any given site, you have enough values that you can use to calculate the capability and readiness of the site. Alternatively, you can use a tool like Ready to calculate and analyze the sites’ readiness for you.

The greatest benefit of combining the three metrics of mastery, confidence, and mindset is that you now have a greater number of measures – much better than a simple checklist – to use in order to stack rank sites.

For example, sites with a medium number of masters, but who have a lot of people with a growth mindset are great candidates for an early start. These sites have a team that will uplevel with less effort. 

Want more? Get the eBook.

There is so much more to clinical trial site readiness than the metrics we’ve discussed in this article.

If you’re still a little unsure about what “Clinical Trial Readiness” really means, and how it will improve your clinical trial performance, we can help!

Get our free eBook to learn everything you need to know about Clinical Trial Readiness, why it matters, and how you can optimize your clinical trial process to experience fewer delays, better performance, and accelerated trial timelines.