Human Factors Validation During the COVID-19 Pandemic

Introduction

UserWise proposes the following three (3) scenarios for proceeding with Human Factors Validation testing during the COVID-19 pandemic:

Conduct In-person usability testing per recommendations
Conduct remote usability testing
Conduct an Expert Review of the medical product by Human Factors Engineers

Conducting in-person usability testing ensures adequate realism and that we test a given medical product sufficiently for safety and effectiveness prior to market launch. Remote usability testing introduces variability in test sessions that could introduce study artifact, and typically remote usability testing reduces the realism of the study.

Overview of Remote Usability Testing

General advantages of remote usability testing include:

Minimizes transmission of COVID-19
Maximizes participant comfort amidst pandemic and makes recruitment possible
Allows medical product developers to proceed with development of life-saving products

General drawbacks to remote usability testing include:

Reduced study realism

Observed use issues may be more exaggerated than in real life due to less opportunity to recover
Some use issues may be less likely to detect compared to simulated use

Reduced ability to simulate and capture unforeseen use issues
Reduced ability to interrupt a participant to ensure their safety (e.g. if a lay user is about to eject medication into their eye)
Reduced ability to control unapproved medical products (and confidentiality) in the case that medical products are sent to the participant
Unrealistic/insufficient training if training would typically be offered and if that training contains a hands-on component

Remote Usability Testing Methods

Here are some methods for remote usability testing, and Pro’s/Con’s of each method:

Remote Usability Testing for Two Example Products

UserWise examined tasks from two (2) Human Factors Validation Study Protocols and assessed whether remote usability testing could be employed to examine each task.

We present two examples, here. Example 1 includes a diagnostic analyzer for human specimens used by lab and medical technicians. Example 2 includes a small insulin infusion pump for use at home by diabetes patients and caregivers.

Example 1: Diagnostic Analyzer for Human Specimens

Note: User manual comprehension testing can very easily be performed remotely, if users are reasonably available.

Example 2: Small Insulin Infusion Pump

Conduct an Expert Review by Human Factors Engineers

Amidst a pandemic, medical professionals may better serve public safety by doing their normal job rather than participating in a usability study.

As such, an expert review by human factors engineers may be appropriate in lieu of Human Factors Validation Testing.

This expert review could include:

A use error analysis to uncover hazard-related use scenarios
Review of similar on-market user interfaces in relation to the product’s use-related risk analysis to ensure all known risks are captured
Threshold analysis - Comparison of product’s user interface to similar on-market user interfaces

Tasks associated with equivalent on-market user interfaces may not require further usability testing

(optional) A heuristic analysis of the user interface to uncover defects
Remote Interviews with intended users asking questions such as:

Do you currently use products that are similar? What are some ways that someone could use those products incorrectly?
Can you imagine any mistakes that a colleague/ person might make when using this product?

An expert review should be paired with a human-factors-heavy post-market surveillance plan. If possible, the first 15 users to use the product, of each user profile, should be observed by human factors engineers, and each user should be questioned, after use, as to whether they experienced any moments of hesitation. In the case that a company representative needed to correct a user interaction during the user’s first real-life use of the product, this scenario would be marked as a “task failure” and assessed in relation to the use-related risk analysis.

Conclusion

As of March 28, 2020, medical product developers are struggling to identify a path forward for usability testing as the pandemic continues. As the clock ticks, novel, life-saving medical products are being withheld from the public. There is no one-size fits all solution for conducting usability testing while the pandemic continues. Critical tasks must be examined for each medical product on a case-by-case basis to conclude the best path forward.

Next Steps

If you need Human Factors Validation data for your medical product during this pandemic, UserWise is here to help! We can assist you with identifying and examining each critical task, strategizing the best method for validation, and then presenting your case to the FDA or other regulatory body. Whether you need help conducting in-person usability testing per the recommendations, executing remote usability testing, or performing an expert review of your product, we can quickly assist you to move your medical innovation forward.

Contact us today to set up a free 1 hour consult for your project or learn more about our usability engineering services and expertise. Also, check out the blog posts below with helpful resources for product development during the COVID-19 pandemic.

︎ UserWise | March 28, 2020

︎Previous