Dr. Douglas Clarkson.
Development & Quality Manager, University Hospitals Coventry & Warwickshire NHS Trusts
Currently I work in Coventry, within the bioengineering department there. Just over a year ago, I
became aware of the aspects of pulse oximetry accuracy through an NPAG meeting in London where Geoff Mathews from the Electrode Company gave a presentation on issues to do with the accuracy of the pulse oximeter probes.
This talk is based on a catch up with the issues that relate to the basics of the use of these devices and questions about the accuracy of the readings that they provide.
There is no doubt that pulse oximetry technology revolutionised many aspects of patient care and has saved tens of thousands of lives, certainly, since their introduction in the mid to late 1980s. In part, the technology was enabled by the availability of LEDs which were compact, convenient to use and were appropriate for the technology.
We all, at some point, see these curves where it is explained how the basics of these probes operate. For example, you have the two forms of haemoglobin; oxy and deoxyhemoglobina. For example, it the stats fall and the absorption of the deoxyhaemoglobin increases and the signal from your red LED at the detector will fall. There is an opposite effect at the infrared diode, but the dominant effect is the selectivity of sensing of the red LED.
There are other forms of haemoglobin; carboxyhaemoglobin for example, might be present if there is carbon monoxide being
inhaled. For smokers, they tend to have a slightly higher level in their system. Methaemoglobin might also change, within certain clinical conditions. Most of the calculations relate to the oxhaemoglobin and deoxyhaemoglobin values.
The actual source of the pulsatile signal in the finger is essentially related to the arterial component of pulsatile flow. The actual signal that is obtained relates to this component of flow within the finger. There are other elements of the
venous flow and the non pulsatile arterial flow.
There are also aspects of how you place the probe on the finger to get a good signal. People assume that you pass the signal across the nail bed with the LEDs on the nail bed side.
If you are able to pick up this oscillating signal from the red and the infrared LEDs, it would look something like this; where you had some kind of ratio of the signal that you detected.
The basic starting point for the technology is to detect these waveforms that relate to this separate signal from the red and the infrared LED.
A characteristic probe; if you take a probe, in this example; 664.5 nanometres red and 907 nanometres infrared, and you work out a signal based on the absorbance of the signals. As you decrease the sats, the R value increases because the absorption of the red is increasing. Each manufacturer has, in his mind, this curve to actually work from the values of the ratios of absorption back to a saturation value.
If, however, the wavelengths are actually not what he thinks they are, then there will be an error introduced into the measurements. This is really one of the central points of understanding where there may be issues with the probes that we are given to use.
Just an example; if you take apart a Masimo probe and play with it, we measure the spectral output of the device which just shows the characteristic spectral wavelengths within the LED. It is maybe a 50 nanometre bandwidth of wavelengths at half power.
If you look at the construction of the assembly of the diodes, they are in opposite directions, as it were. The signal that is provided to drive them is a bipolar signal. In one phase, one diode is activated, and then in the negative phase, the other diode is activated. It is quite simple to measure the voltage within the system that comes out. On the oscilloscope the signal, for example that the diode is detecting, is the one you would see on the oscilloscope.
The actual diodes are pulsed alternately with a gap between each phase. The system knows what diode has been activated and it measure the signal across that for that LED at that point. What it has got to do, in effect, it doesn’t have a continuous signal with each channel. Each channel is broken up because you are switching the signal off and on.
What does the standard say about how you check the accuracy of these devices? A lot of the content in the standard is really historical about techniques that people have investigated but you don’t actually find that people are using these techniques to actually check the accuracy of devices.
A lot of energy has been expended to try and build a rig that would simulate a finger to actually change the saturation of blood within a system, to see if you can replicate this variation of signal. You can circulate a known saturation of blood and then check what the probe is actually measuring. Although you get some interesting observations, none of these systems, in fact, have any practical value for testing devices in the field.
This is an example of the kind of circulary system that we have in place.
It is quite interesting that the Department of Health, maybe ten years ago, was trying to develop this technology to improve the accuracy of measurements of these devices, but nothing has really come of this. Although the standard refers to them, they are not really practical.
One of the emerging technologies which, is not really referenced in the standard, is LED spectral characterisation. Although the standard has been revised in 2011, it hasn’t picked up on the availability of this technology to actually provide some answers on the accuracy of probes. It might actually be, at some point, that what is actually required is a separate standard based around LED spectral characterisation, which sits by itself as a technique that would be a framework for testing these devices.
When the manufacturer tests his product, he gets a representative group of perhaps 40 individuals. They will do what is called a break-down technique where they take people down from normal saturations down to about 70% saturation. During this cycle they will measure their oxygen saturation with co-oximeter blood samples. They will get a correlation between their monitor and the absolute values of saturation.
Also, this is a technique which can be used with the Lightman technology where you relate the values that you measure with that on a probe, to the actual values against an individual. The way it actually operates is that the device measure the spectral output of the red LED, the infrared LED. It has a library of the absorption characteristics of haemoglobin at different saturation levels.
It estimates what the signal ratio will be based on given levels of saturation. It has got physiological data within itself which simulates a human response. It says; what would you expect the response to be for an individual with normal physiology as you reduce the saturation levels from high to low?
Then it gets the manufacturers data that has been used for the monitor that is being tested. It does a comparison between the two and works out the deviation between them.
In this example on the left, it has got the red characterisation, the red LED, the infrared LED and it is done at an estimation of the errors at different levels of saturation. In this example, it has said that it is really quite good. There is only a 1% error down at 70%.
This is a way of actually saying that, if this was on the patient and it showed 80%, it would be 80%. It is a way of testing the finger probe on the system.
You might notice on the right hand curve there is some kind of double peak that is present there. This is to do with edging of the infrared photodiode. You can also get characteristics of infrared diodes where you get outline peaks that will confound the measurements. If this system detects one of these outlying peaks it will say that this probe is not accurate because it has got these external channels that are actually degrading the accuracy of the signal.
The technique is to do a nanometre by nanometre measurement of the output using a solid state spectroradiometer. I think the whole test only takes about 30 seconds to actually scan the system.
Within the software you have got the detailed information of the RR ratio curve that the manufacturers are using. There is a high content of intellectual property in actually running the system. Just to measure the spectral values itself is maybe only 10% of the whole process.
Recently, there have been some evaluations of the Lightman technique to measure the accuracy of probes. These are currently in the literature.
In terms of other methods of testing pulse oximeter probes, the so-called functional testers are quite widely used. In this diagram the LED systems are as element three.
This intervening unit one, detects output from these LEDs. It re-drives a signal down to the detector of the pulse oximeter probe in six. It is detecting on and off phases from the LEDs in the finger probe, but it is driving a signal and it is making a ratio signal into the detector.
It is injecting an RR ratio value into the monitor which might be, for example, equivalent to a 98% or a 90% value. It is not actually testing the wavelength values of the LEDs themselves.
The standard mentions check devices, like this. It says, “They have a role, but they can’t be used to check the accuracy of the readings provided of the finger probe themselves.” They can actually test the monitor is responding to an RR ratio, but they can’t actually check; if that probe was on a patient and it says 90%, that it actually is 90%.
This is just an example of a typical device that you might find as a functional tester. As an experiment, what the Electrode Company did is that they took a basic Nellcor sensor and they exchanged the red LED for an orange LED so it looked quite different from a normal probe. The actual wavelength deviation could be as high as 80 nanometres, which is a massive difference in terms of the characteristics of the probe that you might expect.
If you put that over the Pronk and you ask it to inject a signal for 98% it will do that because all it is doing is switching the LEDs on and off and using that to send a phase signal from another LED into the detector. If you then put that probe on your finger it reads about 40%. The functional test is saying 98%, it looks okay; when you put it on your finger, it is obviously not okay. The difference being that on the Pronk device, you are checking an injected signal. You are not checking a signal that is derived from the output of the LEDs to calculate the saturation values. It is essentially a test on the monitor. You are using the probe as a means of injecting a signal into the monitor.
If you do that evaluation on the Lightman, it says that there is a massive percentage error on the device. It picks up the fact that the probe is drastically wrong. It is a test on the implied accuracy of the actual probe itself.
The actual manufacturer of the Pronk says very clearly, “You can’t use this device for checking the accuracy of the probe as it would be in a clinical situation.” It is a means of checking the functional value of the monitor with an injected signal.
If you do some comparisons of the Lightman with a typical functional tester, yes, it detects the LEDs. It doesn’t do the spectral measurement on the functional one. There is a spectral check on the infrared with the Lightman, not in the Pronk.
The Lightman detects a noise signal where you may have, because of a malfunction in the leads, your signals might be highly
noisy and it will pick this up.
The Lightman doesn’t check a heart rate signal on the monitor because it doesn’t actually operate in that way. It doesn’t actually work on the basis of injecting a signal into the monitor which the Pronk would do.
In a way, if you ask 100 biomed engineers, “With the functional tester, what are you checking?” They all say, “I am checking the probe accuracy.” They may actually think they are checking, if that probe was on a patient, it would give an accurate value of SpO2, whereas in fact, it doesn’t.
I think that there are two main aspects to this approach where you are looking at it on a technology side to say that; what is each tester measuring? Then you look at the clinical side and there are huge issues about the use of probes that are inaccurate. Over time, the people have shown that the percentage of devices, where audits have been done with the Lightman, the percentage of audits of probes that fail that they have, they are outside of the range of plus or minus three percent at specific saturation values is increasing.
The quality of probes is not improving. If you have a probe that has got a high reading, maybe a plus five, you think, 95%, “That patient is alright, it’s actually a 90,” and you need to intervene. Or you see a reading, 90, it is a low reader, it is actually 95, you are either going to over-treat the patient, or mistreat the patient. The clinical pathway of that patient might be quite severely changed. It can have quite significant clinical implications.
There are also specific situations, for example, in heart surgery, where you are doing paediatric heart surgery where the patients come in with a saturation of maybe 70 and you are expecting them to go out of theatre with a saturation of 85, for that as your endpoint of the operation. If that value is wrong, then the procedure is not quite successful.
Also in sleep studies where you are making a clinical judgement on management of a patient, accuracy is important. There are issues to do with clinical governance here which need to be addressed.
Just in summary, the current standard doesn’t really help people improve the accuracy of the probes that are used. Functional testers are testing certain parameters, but not perhaps, the more vital one of the accuracy on the patient. The standard doesn’t reference the technology of spectral measurement of the LEDs. I think there are issues within the biomed community to find ways to be sure that when the engineers come and test the device, he has confidence that the readings on the patient are what the monitor says they are.
This presentation may be downloaded here: http://www.ebme.co.uk/downloads/viewdownload/14-2014-seminar/93-5-determining-the-accuracy-of-spo2-values-dr-d-mcg-clarkson