Analytical Comparison of Three Quantitative Immunochemical Fecal Occult Blood Tests for Colorectal Cancer Screening

Colorectal cancer is a major public health issue in all industrialized countries. Screening using guaiac fecal occult blood tests (G-FOBT) reduces specific mortality related to colorectal cancer (1). Several studies have concluded that both the Magstream (Fujirebio) and OC-Sensor (Eiken Chemical Co.) automated immunochemical FOBTs (I-FOBT) offer a gain in sensitivity in the detection of advanced neoplasias, compared with G-FOBT, at a cost of lower specificity (2–4). For both tests, ideal balance between sensitivity and specificity can be reached by variation in hemoglobin (Hb) concentration cutoff and number of samples (5–8). For both tests, a gain in both sensitivity and specificity for the detection of advanced neoplasias was possible (4, 7, 8).

Since it has been established that I-FOBTs do better than G-FOBTs, these tests are expected to be used in all national screening programs using FOBT. Accordingly, the use of a fecal immunochemical test has been included in U.S. guidelines for colorectal cancer screening (9). However, several I-FOBTs are available and their performance is difficult to compare because the cutoff provided in studies is expressed in concentration of Hb in the collecting tube depending on the concentration of Hb in the feces, but also on the volume of buffer in the tube, and on the amount of feces introduced in the tube. Therefore, optimal test, optimal number of samples, or optimal Hb concentration cutoff are for the moment indeterminate (10). Moreover, seasonal variations in positivity rates of screening programs using OC-Sensor or Magstream I-FOBT have raised the question of the sensibility to temperature (11–13), and laboratory analyses have established a decrease in Hb concentration in OC-Sensor I-FOBT with increasing delay in the sample (11, 14).

As a summary, the performance of I-FOBT depends mainly on the test's sensitivity to Hb, its reproducibility of sampling and measurement, and the stability of Hb in the collecting tube, in particular with regard to temperature variations and delay from fecal sampling to test analysis.

Our study aimed to compare measurement precision and reproducibility, together with Hb measurement stability at varying storage temperatures and varying delays between sampling and analysis of 3 I-FOBTs previously used in colorectal cancer screening programs: Magstream (New Hemtube) analyzed using a Magstream HT automated instrument (Fujirebio), OC-Sensor (OC-Auto sampling bottle3) analyzed using an OC-Sensor DIANA instrument (Eiken Chemical Co.; distributed by Mast Diagnosis), and FOB Gold (distributed by SKD, France) analyzed using a SENTiFOB instrument (Sentinel diagnostics).

All 3 tests use polyclonal rabbit antibodies directed against human HbA. All tests are fully automated (Table 1). Analysis of OC-Sensor and FOB Gold is based on immunoturbidimetry, which involves a measurement of the absorbance of light through the tube, which increases with the importance of Hb antibodies complexes. Analysis of Magstream involves the use of an automated visual measurement of migration of agglutinated magnetic particles. In routine use, the crude pixel value generated by Magstream is converted into MSR units, an arbitrary unit proposed by the manufacturer. For the present study, we asked Fujirebio to provide software (not routinely integrated in the machine) to allow for the collection of the measurements (pixel values). Although Magstream is commercialized by Fujirebio as a qualitative test, a quantitative measure is provided by the instrument, so we considered the test as quantitative in the analysis.

First, freshly collected stools, obtained from 10 healthy subjects aged less than 50 years, were tested using all 3 tests to confirm the initial absence of Hb. These stools were mixed and homogenized, then divided into containers. In each of these containers, a volume of human whole blood lysate, the Hb content of which had previously been measured (Advia 2120, Siemens), was added to obtain all prespecified Hb concentrations in the stool. Each container was vigorously shaken after adjunction of blood. For each of the concentrations, sampling of all 3 tests was done using the same containers, ensuring that concentration was identical for all tests. Finally, collecting tubes were shaken after sampling and before analysis. This procedure was repeated 2 times leading to 2 distinct stool mixtures, one being analyzed to evaluate reproducibility and the other to evaluate stability to storage.

Two distinct experiments were conducted: the first one to evaluate and compare the reproducibility of the tests (experiment 1) and the other one to evaluate and compare their sensitivity to temperature and delay of storage (experiment 2). For each of these 2 experiments, a distinct stool mixture was performed, as described as follows.

To compare tests for a given Hb concentration, we initially explored, for each test, the relationship between concentration in the feces and the value provided by the instrument for a total of 10 values of Hb concentration in feces, varying from 0 to 350 μg Hb/g of feces. This range of concentrations was selected to: (i) cover the range of usual or proposed Hb positivity cutoffs of all tests and (ii) cover the range of physiological bleeding of colorectal lesions (15). To assess measurement reproducibility, 10 tubes were collected for each instrument and each concentration, and all tubes were repeatedly analyzed 5 times leading to a total of 50 readings per concentration and test. In this experiment, all prepared I-FOBTs were stored for 3 days at 10°C before being analyzed, approximating to the conditions of a screening program involving mailed samples.

A second experimental plan was developed to assess the influence of storage temperature and delay between sampling and analysis for different Hb concentrations in feces. For each I-FOBT, 4 storage temperatures: 4°C, 10°C, 20°C, and 30°C, and 5 delays between sampling and analysis: 1, 3, 5, 7, and 10 days were tested for 6 Hb concentrations in feces (0, 20, 75, 100, 150, and 250) μg Hb/g of feces. Due to the large number of combinations of the 3 aforementioned parameters in an exhaustive plan (360 combinations) and to the relatively slow operating rate of 1 of the machines (SENTiFOB, Sentinel Diagnostics), we used an optimal experience plan, determined using the ADX (Analysis and Design of eXperiments) tool developed by SAS software. Four factors were introduced in this experimental plan (I-FOBT, storage temperature, delay, and concentration), as well as their 2-level interactions. Optimization of the design was achieved by D-optimal optimization (maximization of the determinant of the information matrix), to do a linear regression evaluating the mean daily decrease in standardized concentrations (to avoid the impact of the slope of the relationship between fecal concentration and buffer concentration of blood) in the collecting tubes. However, due to the nonlinear relationship between concentration of blood in the feces and the collecting tube (roughly logarithmic) and to the semiquantitative nature of the Magstream test (see Results), each test was assessed individually and no overall analysis was conducted (see Statistical Methods next).

In our experimental plan, 32 combinations of storage temperature, delay between sampling and analysis, and fecal Hb concentration were used for each test. The experimental plan can be deducted from Table 2 which presents mean measurement obtained for each test, according to temperature, concentration, or delay, and therefore in which nonempty cells correspond to evaluated combinations of temperature*concentration*time*test. Ten tubes were collected for each of the included situations, each tube being analyzed 5 times.

Temperature of storage was monitored using a system of electronic measurement of temperature every 5 minutes, and radio frequency recording (AOIP instrument). All tubes, whatever test or concentration, were stored in the same conditions for each temperature (dedicated fridge for 4°C and 10°C temperatures with a range of variations of temperature of ±2°C, air-conditioned room for 20°C temperature with a range of variations from 20°C to 22°C, and a dedicated incubator for 30°C temperature with a range of variations of ±1°C).

For each test and each of the experimental concentrations, the intertube and intratube variances were determined by using a random effect model (SAS PROC MIXED). This procedure enabled us to compute the variation coefficients due to sampling (intertube) and reading (intratube).

In an exploratory analysis, we computed the mean measurement obtained for each test, according to temperature, concentration, or delay (Table 2). Then, for each temperature and processing delay, the mean measurement was standardized on the initial fecal concentration to allow comparison of variations in concentration due to storage duration and temperature between tests. For FOB Gold and OC-Sensor, the buffer concentration was directly proportional to fecal concentration (Fig. 1). The theoretical relationship between fecal Hb concentration (C_f), expressed in μg/g, and concentration of Hb in the buffer (C_b), expressed in ng/mL, is given by the following formula: C_b = (C_f * q_s)/ν_b, where q_s is the quantity of stools introduced in the tube (in mg) and v_b is the volume of buffer (in mL). However, as we did not check independently the values of q_s and v_b provided by the manufacturers, the buffer concentration provided by the automated analyzer was simply divided by the initial theoretical fecal concentration, to standardize data. For Magstream, the pixel value provided by the automated analyzer had an inverse logarithm relationship to fecal Hb concentration, and the mean pixel value was 330 in the absence of Hb in the sample (Fig. 1). The following transformation was therefore applied to data: (330 pixel)/log(initial fecal concentration). However, this transformation was less adapted to smaller concentrations. For each test, we evaluated the mean daily decrease in standardized Hb concentration according to temperature using mixed effect linear models.

Statistical analysis was done using SAS software, version 9.1 (SAS Institute).

First, the relationship of the values provided by each automated analyzer (3 days' storage at 10°C) was assessed according to the initial fecal Hb concentration. A linear relationship between measurement and fecal Hb content was observed for OC-Sensor and FOB Gold, but not for Magstream (Fig. 1).

For Magstream, the pixel values were bounded in the range 130 to 330 pixels. Above a concentration in the tested stools of 250 μg Hb/g, the pixel measurements in the tube did not change with concentration (the mean pixel measurement was 130 pixels). Therefore, the test could only be considered as quantitative within a fecal Hb concentration range of 20 to 200 μg Hb/g of feces.

Instrument measurements have been plotted in Figure 1 and the expected value is represented as a dotted line according to linear modeling for FOB Gold and OC-Sensor, and loess (local polynomial fitting) modeling for Magstream. We defined overlap between concentrations whether no cutoff could perfectly separate instrument measurements between 2 successive fecal concentrations. For Hb concentrations up to 150 μg Hb/g of feces, there was no overlap in the range of measurements obtained with the different Hb concentrations in feces selected in our protocol for OC-Sensor. For both Magstream and FOB Gold, overlap was observed between concentrations in the entire range of these tested concentrations.

Reproducibility could be explored in the entire concentration range for OC-Sensor and FOB Gold (Fig. 2). However, it was only explored for concentrations below 250 μg Hb/g of feces for Magtream, because for higher concentrations, the absence of measurement variation between concentrations using this test would lead to artificially good reproducibility.

The best reproducibility (smaller total variation coefficient) was observed with OC-Sensor, with the exception of concentrations below 75 μg Hb/g of feces, for which Magstream offered better reproducibility. However, variability for small concentrations using Magstream was reduced due to lack of quantitative measurements above 330 pixels. The worst reproducibility was observed with the FOB Gold test. Indeed, the mean total variation coefficient observed between 75 and 250 μg Hb/g of feces was 0.07 for OC-Sensor, 0.10 for Magstream, and 0.18 for FOB Gold.

For all tests, measurement variability involved intertube variability rather than intratube variability (the variation coefficients associated with sampling were far greater than those associated with reading). Intratube variability was low for all tests (the mean coefficient of variation due to reading between 75 and 250 μg Hb/g of feces was 0.025, 0.016, and 0.060 for FOB Gold, OC-Sensor, and Magstream, respectively).

The difference between these sources of variation was the highest for FOB Gold, for which the intertube variability was particularly important compared with the other tests. The mean coefficient of variation due to sampling between 75 and 250 μg Hb/g of feces was 0.15 for FOB Gold, whereas it was only 0.10 for OC-Sensor and 0.07 for Magstream.

Intertube variability associated with FOB gold tended to decrease as the Hb concentration in feces increased. Intertube variability associated with OC-Sensor was stable between 75 and 250 μg Hb/g of feces, but was high for concentrations below 75, or above 250 μg Hb/g of feces. The latter concentrations were higher than the upper limits of good performance recommended by the manufacturer. Intertube variability using Magstream increased as did fecal Hb concentration up to 150 μg Hb/g of feces, then decreased due to a nonlinear relationship between pixels and concentration.

Mean measurements according to test, temperature, concentration, and time are provided in Table 2. For all 3 tests, measurement was stable over time, independently of storage temperature when no Hb was added to feces. Relationship between instrument measurements (pixel or buffer concentration) and fecal Hb concentration was verified (data not shown). Figures 3 and 4 plot mean measurement per machine according to storage duration or temperature. A decrease in measurement was observed for all tests at increased storage durations, and at temperatures of 20°C and 30°C, but not 4°C or 10°C.

Figure 5 provides the mean of ratio of measurements standardized on the experimental fecal concentration for each combination of storage duration and temperature and for each concentration (see Materials and Methods). The stability of the Hb measurement provided by all 3 tests was good at 4°C and 10°C for OC-Sensor and Magstream. However, an increase in concentration provided by FOB Gold was observed within the first days, particularly at a storage temperature of 4°C. A similar (yet smaller) increase in concentration detected by OC-Sensor and Magstream over the first days cannot be excluded on the basis of our data.

At a storage temperature of 20°C, a substantial decrease in Hb concentration was observed over time using all 3 tests. The best stability was observed using OC-Sensor with a significant daily decrease in measurement of 1.7% (P < 0.01) at 20°C, compared with a daily decrease of 7.4% and 7.8% with Magstream and FOB Gold, respectively (P < 10⁻³). However, the decrease observed with Magstream depended on the initial Hb concentration. It was greater with the smaller concentration of 20 μg Hb/g of feces.

At 30°C, the performance of OC-Sensor was far better than that of FOB Gold and Magstream. Indeed, the mean daily decrease in measurement observed with OC-Sensor was 8.6% (P < 10⁻³), whereas the mean decrease in measurement observed on the first day was 30% for Magstream and 70% for FOB Gold. The decrease observed with OC-Sensor at 30°C depended on the initial concentration, being more important for small concentrations. On day 7, 38% of an initial concentration of 150 μg Hb/g of stool was still detected with OC-Sensor. On day 10, 25% of an initial concentration of 250 μg Hb/g of stool was still detected by the same test. On the contrary, from day 5, no Hb content was detected by FOB Gold or Magstream in the samples.

Our results show that the precision and the reproducibility of Hb measurements in feces are better with OC-Sensor than with Magstream, and better with Magstream (in the range of concentrations where the test can be considered as quantitative) than with FOB Gold. About stability at varying temperatures, our results show that independently of the test used, there is a substantial loss in Hb measurement as from 20°C. This loss is more important for FOB Gold than for the other tests. At 20°C and 30°C, denaturation of Hb is less important and occurs less quickly in the buffers used with OC-Sensor than with Magstream devices.

Independently of the test used, intratube variability was lower than intertube variability. Similar findings were found in an experimental study conducted in U.K., although intratube variability was assessed in solution of Hb, and intertube in artificially positive stools (16). The intertube variability estimated in our experiment was a consequence of both the tube characteristics and the biologist's reproducibility in using the test (sampling of feces using the immunochemical tests probes). In real screening settings, such a tube effect would probably be greater because a patient is certainly less reproducible in his sampling technique than a biologist, and patients do not mix their stool to homogenize their blood content before performing the test. Moreover, the tube effect measured in screening programs including several tubes per patient also includes the effect of the intermittency of the bleeding associated with colonic lesions. To the contrary, the intratube variability measured in our experiment should be similar to that which occurs in real screening settings, as it is solely an effect of the automated analyzer itself.

The lower intertube variability when using OC-Sensor could be explained by a more accurately calibrated quantity of stool incorporated in the sample (not measurable in our study). In addition, more overlap occurred between the evaluated concentrations with FOB Gold and Magstream than with OC-Sensor. Accordingly, the loss in precision of these tests (including intertube and intratube variability) could have consequences on positivity rates in real screening settings.

Temperature-related Hb degradation is expected; however, it can be delayed by the use of a suitable stabilizing agent in the buffer. Such stabilizing agents are included in all 3 tests. Nevertheless, the stability of Hb measurement at varying temperatures and over time was better with OC-Sensor than with Magstream and far better than FOB Gold. The superiority of OC-Sensor was also observed in the NHS evaluation report (16). At a temperature of 20°C, the decrease observed with FOB Gold and Magstream was at least twice that observed with OC-Sensor. At 30°C, the performance of Magstream and FOB Gold was very poor, whereas the OC-Sensor was much better and remained reliable, even at this high temperature. I-FOBT sensitivity variations related to storage duration have previously been described in both laboratory experiments and genuine screening settings (11–14). The interaction between storage duration and temperature has been quantified for the OC-Sensor test in a laboratory experiment: The daily decrease in fecal Hb measurement was 0.3% at 4°C, 2.2% at 20°C, and 3.7% at 28°C (14). Our findings (2% daily decrease at 20°C and 9% daily decrease at 30°C) are consistent with this observation, although the decrease observed at 30°C seemed more important. This could be explained by a difference in initial Hb concentration in feces, the relative decrease being higher for small initial concentrations. Such differences in sensitivity related to storage temperature and duration, and affecting the reliability of colorectal screening test programs, could have an important impact on screening organization and test choice, particularly in countries where ambient temperatures are high. It would seem that, whenever possible, the delay between sampling and test processing should be reduced to 3 days, or CRC screening programs should be stopped during the summer in countries with long period of very high temperatures (>30°C). In addition, patients should be advised to store fecal samples in the refrigerator at home before forwarding them by post. Acceptability of such a recommendation needs further investigations.

Our study has several limitations associated with experimental conditions and data analysis constraints. Since it was based on laboratory preparation of positive fecal samples, it is possible that these artificial positive samples behave differently than native positive samples, despite the fact that we used human feces. However, decrease in Hb from real positive samples has also been shown with OC-Sensor (11). On the one hand, mixed or nonmixed stool samples could behave differently in the application of I-FOBT. On the other hand, the mixture of stool samples from several subjects done in our protocol allowed us to avoid differential bias between concentrations and tests in case of undetected Hb previously present in 1 or several fecal samples. In addition, sampling error was probably underestimated since the tests were conducted by trained biologists, rather than by subjects from the general population. For example, no miss manipulation of tubes occurred, such as opening the wrong side of the FOB Gold test or spilling the OC-Sensor or FOB Gold buffer. Targeted concentrations were not checked in the samples by independent Hb measurement methods. Nevertheless, within each of the experiments (for example, storage evaluation), calibrated stool samples were obtained from the same negative FOB-tested stools. Finally, the nonlinear relationship between pixel value and concentration of Hb in the buffer for the Magstream test implied transformation of crude data. However, we selected the transformation which minimized underlying assumptions and independently of the results.

However, the fact that this study was entirely laboratory-performed enabled every parameter to be controlled, as from the introduction of blood into the stool. This is not the case for studies exploring the stability of fecal samples sent to the laboratory after having been performed by the patient at home, measurement of initial Hb concentration in the stool, and storage duration or temperature from home to laboratory being unknown (11, 14). This also enabled the 3 tests to be compared on the same stools. Moreover, the use of blood lysate guaranteed the best possible conditions for the antibody–antigen reaction. Finally, our laboratory study was designed to fit as most as possible real screening settings with the choice of a range of concentrations adapted to physiological bleeding of lesions and discussed cutoffs, temperatures close to usual weather conditions, and delays compatibles with mailing of samples by post to a central analysis center, as it is the case in FOBT-based screening programs.

Our results show better analytical and stability performances of OC-Sensor or Magstream than FOB Gold. Comparison between OC-Sensor and Magstream was made more complex since: (i) the area of optimal performance was different for both tests, OC-Sensor detecting smaller Hb concentrations, and (ii) the relationship between measurement of OC-Sensor and Magstream was not linear. The improved stability offered by OC-Sensor compared with Magstream and the semiquantitative nature of the Magstream offer strong arguments in favor of OC-Sensor. Nevertheless, several studies have shown the good performance of Magstream in population surveys, even using only 1 sample at the manufacturer's threshold (17, 18). Costs of the tests should also be considered, together with adaptation of screening programs to limit the effect of sensitivity of tests to temperature on their performances (12). Further studies and cost-effectiveness analysis are needed to compare Magstream and OC-Sensor in real screening settings, appropriately taking into account temperature and duration of storage.

No potential conflicts of interest were disclosed.

L. Guittet, E. Guillaume, R. Levillain, P. Beley, and G. Launoy elaborated the experimental plan. P. Beley and R. Levillain performed the laboratory analyses. L. Guittet and E. Guillaume performed the statistical analysis. Results were analyzed and discussed by all authors. Finally, L. Guittet, E. Guillaume, and G. Launoy wrote the article, which was carefully corrected by all authors.

The study was funded by the French National Cancer Institute (InCA). Instruments were provided on loan by manufacturing companies. Supporting companies had no access to crude data. Preliminary analysis was provided to them for the purposes of information; however, in no manner whatsoever did it influence our analysis or the drafting of this article.

The costs of publication of this article were defrayed in part by the payment of page charges. This article must therefore be hereby marked advertisement in accordance with 18 U.S.C. Section 1734 solely to indicate this fact.