The Performance of 11 Fingertip Pulse Oximeters During Hypoxemia in Healthy Human Participants with Varied, Quantified Skin Pigment

Nonin Onyx Vantage 9590^® is the Only Pulse Oximeter Within FDA A_RMS Guidelines Across All Skin Pigmentations and Oxygen Saturation Ranges

Summary

In a controlled study conducted by researchers at the University of California, San Francisco and other institutions, the performance of 11 fingertip pulse oximeters was evaluated in 34 healthy human participants with varied skin pigmentation.* This study was the first to incorporate both objective and subjective measurements of skin pigmentation, addressing a historical lack of reliable standardization in how skin pigmentation is identified within pulse oximetry studies. The study sought to deliver a more accurate picture of pulse oximeter performance, especially in people with dark skin pigmentation.

“Device performance is largely unregulated and poorly characterized, especially in people with dark skin pigmentation,” the authors noted.

The study measured oxygen saturation (SpO₂) levels in participants with different skin pigmentation, ranging from 70–100% SpO₂. It compared the bias and absolute bias of the 11 pulse oximeters against a reference device (Radiometer ABL90 Flex Plus Hemoximeter) across the different skin pigmentation and SpO₂ ranges. Only four of the devices have FDA 510(k) clearance (Nonin Onyx 9590, Masimo MightSat, Contec, Biolight).

The data found that five of the 11 devices resulted in greater than 3% root mean square error (A_RMS) and did not meet FDA criteria for accuracy (less than 3% A_RMS). Seven of the fingertip pulse oximeters are part of the ubiquitous low-cost devices flooding the market. None of the low-cost devices met FDA A_RMS guidelines in dark skin pigmentations at both of the low blood oxygen saturations (70–80% and 80–90%). In the 80–90% saturation range, when many clinicians are initially alerted to issues, Nonin was the only device to perform under 2% A_RMS in the darkest skin pigmentations. The risk of low cost and non-regulated devices reading inaccurately and potentially misdiagnosing hypoxemia in patients of color continues to be a problem.

Equitable and accurate pulse oximetry has been a significant issue, which the COVID-19 pandemic highlighted. Nonin’s fingertip pulse oximeters outperform low-cost oximeters, especially in patients with darker skin pigmentation.

Methods

The study tested the performance of fingertip pulse oximeters in participants with skin pigmentation ranging from light to dark. Participants underwent controlled desaturation protocols to achieve stable oxygen saturation (SaO₂) levels between 70–100%, while pulse oximeter readings (SpO₂) were recorded. Skin pigmentation was assessed both subjectively using the Fitzpatrick scale (pFP) and objectively using spectrophotometry to measure individual typology angle (ITA). Participants were grouped into light, medium, and dark pigmentation categories based on both the pFP and ITA measurements.

The bias and absolute bias of each pulse oximeter was calculated by comparing the SpO₂ readings to the reference arterial SaO₂ measurements, across the different skin pigmentation groups. Statistical analyses included Bland-Altman plots, linear regression, and calculation of A_RMS to evaluate the performance of the pulse oximeters.

Key Takeaways

Only the Nonin Onyx 9590 met the FDA’s accuracy criteria of less than 3% A_RMS across all skin pigmentation levels (light, medium, dark) and all oxygen saturation ranges (70–80%, 80– 90%, 90–100%). The Nonin Onyx Vantage 9590 outperformed the Masimo MightySat® and all low-cost devices, particularly in participants with darker skin and lower SpO₂ levels. For example, in participants with the darkest skin pigmentation and SpO₂ levels between 80–90%, the Nonin Onyx Vantage 9590 had a bias of 0.8% (95% CI 0.5, 1.2), which was lower than nine other devices tested.

Claims fall short of actual performance
The study found that the performance of the 11 fingertip pulse oximeters varied widely, with some devices meeting regulatory accuracy standards while others did not, especially for participants with darker skin pigmentation.

None of the low-cost devices (under $60 USD) met the FDA accuracy guidelines for the darkly pigmented participants at low blood-oxygen saturation levels. In fact, some of these low-cost devices had A_RMS readings 2–3 times higher than the regulatory guidelines.

“Fingertip POXs (pulse oximeters) have variable performance, frequently not meeting regulatory requirements for clinical use, and at times contradicting claims made by manufacturers,” the authors said.

More objectivity is needed in device evaluation
The study compared the subjective Fitzpatrick skin type (pFP) assessments to the more objective individual typology angle (ITA) measurements using spectrophotometry.

The results showed that relying solely on the subjective pFP scale may not accurately represent the optical properties at the fingertip measurement site. The objective ITA measurements provided a more comprehensive assessment of the participants’ actual skin pigmentation across the full spectrum, providing a more confident demonstration of device performance.

Accuracy is essential
Unregulated, low-cost pulse oximeters often perform below FDA standards and inconsistently across patients. Clinicians and patients cannot afford to use devices that do not perform reliably. In contrast, Nonin’s technology exceeds FDA requirements and outperforms both low-cost and other FDA-cleared oximeters, providing accuracy and dependability across all skin pigmentations.

Learn More About Nonin’s Commitment to Health Equity

Key Terminology

• Bias is the mean difference between the pulse oximeter oxygen saturation (SpO₂) and the arterial blood oxygen saturation (SaO₂) measured by a reference hemoximeter.
• Absolute bias is the mean of the absolute differences between the pulse oximeter oxygen saturation (SpO₂) and the arterial blood oxygen saturation (SaO₂) measured by a reference hemoximeter. It represents the average magnitude of the difference between the two measurements, without regard to the direction of the difference.
• Average root mean square error (A_RMS) represents the square root of the mean of the squared differences between the pulse oximeter oxygen saturation (SpO₂) and the arterial blood oxygen saturation (SaO₂) measured by a reference hemoximeter. It is a measure of the overall accuracy of the pulse oximeter compared to the reference standard.