Variance in multiplex suspension array assays: intraplex method improves reliability
© Hanley; licensee BioMed Central Ltd. 2007
Received: 06 August 2006
Accepted: 29 August 2007
Published: 29 August 2007
Flow cytometry based suspended microarray assays are susceptible to many sources of variance; multi-well replication and inter-instrument reproducibility is uncertain.
Method and results
An "intraplex" method was developed in order to minimize differences in sample readings between instruments. A full intraplex assay consists of a set m of microparticle set classifications assaying for the same analyte, with each of the m classifier sets having different sensitivity to analyte, and n classifier sets replicating each of the m levels of sensitivity, where m > 1 (generally m > 4 would be used).
The intraplex method can compensate adequately for the sources of variance that have been identified in suspended microarray assays. It requires no changes to current equipment in use, and is a superior method of constructing precision assays. Additionally, Luminex® users may want to consider the evidence that shows that despite calibration to the same standard, two instruments may not give similar results for all concentrations of analytes.
A suspended microarray assay system uses small particles, such as microspheres or microrods that contain some method for identifying a set of particles composing one assay. An chemical compound used to bind to a biological (or chemical) target molecule (analyte) is bound to the surface of a set of identical particles, which are generally in the size range of 3–15 microns. Differently labeled particles have different target molecules that they assay for. These particles are added to a liquid (such as serum or cell lysate) containing the potential analytes. (In systems such as "smart dust", the assay may be distributed in the field to detect analytes. A system such as "smart dust" may also use an alternative method of analyte signaling and readout.) The final step in the assay activates a reporter fluorophore that provides a signal. (Essentially, this is an ELISA assay on the surface of a small particle.) The particles are run through a flow cytometer, which may be optimized for the specific assay system. For each particle in the mixture, the cytometer identifies the classifier for the set the particle belongs to together with the fluorescence reading of the reporter fluorophore. Because the particle classifiers are designed to be unique for each analyte, it is possible to multiplex the assays together in a test tube in order to test for multiple analytes in one sample. Multi-well assay plates can be used to test many samples, and such assays then become a high throughput system.
The Luminex Corporation (Austin, Texas) is one vendor of specialized flow cytometry equipment, which they also license to BioRad (Bio-Rad Laboratories, Hercules, California). The Luminex assay examined in this study utilizes microspheres on the order of 5.6 microns in diameter, upon which antigens or antibodies have been covalently bonded (xMap™ assays). The Luminex xMap™ assay microspheres used in this study contain two classifier fluorophores. Each fluorophore has n levels of brightness that can be differentiated, and the two are proportionally varied to separate them into n2 different microsphere populations for identification (currently n2 = 100 for two fluorophores.) This study used classical sandwich assays to attach reporter molecules of streptavidin-linked phycoeryrthrin to the microspheres. Luminex also provides assays which utilize nucleotide hybridizations to attach reporter fluorophores, and other assays are possible. The reporter fluorophore intensity is then measured in a specialized flow cytometer together with the microsphere classifiers; the reporter fluorescence measurement is collected separately for each microsphere population in the mixture. For each microsphere classifier population a sample of microspheres is collected, and one or more of the following are then used as the reported value: median, mean, trimmed mean, or peak. Median is the most commonly used value. The system is usually deployed with one well containing the same analyte fluid, sometimes two, however, some laboratories use three replicate wells as a standard, and throw out outlier values when they occur.
The experimental sample fluid with n sets of microspheres flows up through a probe, which has a tip with 5 fine holes leading to a single channel at the top. The fluid travels through a system of tubing and valves into the flow cell, where (in the current equipment) two lasers are present. One laser stimulates the two marker fluorophores, and the other stimulates the reporter fluorophore. A system of avalanche photodiodes and a photomultiplier tube captures the fluorescence from marker and reporter emissions.
Users of the Luminex instrument with xMap™ microsphere arrays have had mixed success in correlating the results of the assays with ELISA assays and generating reproducible results for a given assay [1–7]. A solution offered at the Planet xMAP 2006 Symposium, where the results of a primarily Luminex authored paper  were presented, was to use more microspheres for each analyte. However, there are at least two significant matters not addressed by that recommendation: carryover between wells, and stochastic variance.
In response to the above, and a set of concerns from prior experimental work, the intraplex method was developed. This method compensates for various sources of variance that occur under typical real world laboratory conditions. Potential sources of variance that can be compensated for in whole or in part include: variation in size of microspheres affecting brightness ; carryover of microspheres between wells; stochastic variance issues (unpublished); and inter-instrument calibration differences (response curve for varying concentrations of analyte by the complete opto-electronic system).
In order to try to minimize inter-instrument and inter-well variances, the intraplex assay method was developed. Due to significant opportunities for confusion in this discussion, three terms are introduced for clarity: Suspended Microarray Particle (SMP), Suspended Microarray Particle Classifier Set (SMPCS) and Suspended Microarray Particle Classifier Set – IDentical Group (SMPCS-IDG). An SMP corresponds to a single microsphere, and an SMPCS corresponds to a set of microspheres that share a classifier. An SMPCS corresponds to what Luminex commonly calls "a microbead region", a "microbead set" or more colloquially, "a microbead" or simply "beads" and is usually interchangeable with bead number, since Luminex identifies their microbeads to users by numbers from 001 to 100 in the older systems in use.
What is new to intraplexing is the SMPCS-IDG, a superset of SMPCS's composing an identically responsive group. An SMPCS-IDG is a set of n SMPCS's that assay for the same analyte with the same level of sensitivity. This is explained in more detail below.
When processing this 5 × 5 intraplex, the first step of processing removes outlier values from each of the 5 SMPCSs making up each SMPCS-IDG if outliers exist. Step two averages the remaining n readings for each of the 5 sets, to obtain 5 averages, or "means of n." Then these means of n are themselves averaged to produce a single mean of m. The third step uses the mean of m as the denominator for each of the 5 means of n. (i.e. essentially the same as for the simple intraplex above, with greater statistical confidence generated for each of the m SMPCS-IDG's.) Like the simple intraplex, the end result is 5 ratios, called internal self-mean ratios. This complete technique should give a high level of precision where precision is needed.
Preparation of xMap™ microspheres
Microsphere preparation was done according to standard Luminex xMap™ microsphere coating protocols. The assays used had already been tested against rhesus serum samples and levels of signal were recorded. This signal level was accepted as sufficient indication that they were representative of a real world assay.
The virus antigens used in these experiments were:
SFV- Simian Foamy Virus,
SRV- Simian Type D Retrovirus,
SIV- Simian Immunodeficiency Virus.
Assays and microsphere classifiers available for use
Preparation of microtiter plates
MultiScreen HTS, BV (Millipore; Bedford, MA) 96 well filter plates were utilized for all assays. Preliminary studies of pipetting error indicated that volumes above 5 μl would have minimal error. All assays were conducted such that no fluid volume below 5 μl would be pipetted, and pipetting was done using a multi-channel pipetter. On the basis of preliminary evaporation studies, a total volume of at least 90 μl per well was used during incubations to minimize evaporation as a source of variance. In addition, all wells were filled within 2 minutes or less after each washing so that any difference between well concentrations due to evaporation was further minimized.
Using a setting to collect a minimum of 100 microspheres per sample, 3, 4, and 5 microsphere set intraplexes were used to assay for the same analyte. Serum titrations of 1:50, 1:100 and 1:200 were used with 32 replicate wells per titration. The aim was to find a method for improving the accuracy of xMap™ assays through better intra-well controls. In total, 25 SMPCS's (i.e. xMap™ microsphere regions) were multiplexed, including all elements of the intraplexes. One SMPCS was coated with BSA as a control to measure nonspecific binding. An additional set of 6 uncoated SMPCS's were used as an alternate experimental intraplex control.
The assays used in this study were developed previously for a simian virus detection project. They were manufactured using carboxylate xMap™ microspheres from Luminex (Luminex; Austin, TX) conjugated to multiple viral antigens; the viral antigens used were 4 microsphere sets for CMV, 5 sets for SFV, 5 for SRV and 3 for SIV. (Table 1) These assays, intended to bind Rhesus macaque antibody, were antigen attached to microspheres. The single Rhesus macaque serum used is known positive for SRV, CMV and SFV. This serum is known to be negative for SIV.
Three controls were used: uncoated microspheres, the SIV microsphere assays, and a BSA standard control for background. Serum from a single Rhesus macaque with known positive and negative characteristics for the assays used was the sole experimental sample (and thus a type of control). Samples were incubated for two hours on a shaker table, washed with PBS-Tween, then incubated for 40 minutes with R-Phycoerythrin-conjugated Affinipure F(ab) Fragment Goat anti-Human IgG Fcγ (Jackson ImmunoResearch Laboratories, Inc.; West Grove, PA), which was used as a conjugate reporter to detect the Rhesus macaque antigen specific IgG antibodies bound to antigen on microspheres. The plate contents were then washed with PBS-Tween, shaken to suspend the microspheres, washed again, resuspended, then read on a Luminex instrument. Plates were stored overnight at 4°C in a refrigerator and read on a Bioplex instrument the following morning.
Two instruments were used for these experiments: a Luminex Model 100 that is approximately 5 years old, and a Biorad Bioplex instrument installed in late December 2005 and commissioned for use in January 2006. Both instruments were under standard service contract. Prior to commencing the study, both instruments had been serviced by field technicians within the previous 2 months. Also prior to commencing the study, the Luminex instrument was upgraded to the latest software and firmware levels.
Each plate was run on both machines, first on the Luminex, and second on the Biorad Bioplex. Seven different statistics available from Luminex and Bioplex instrument software were examined for each instrument: mean, standard deviation, trimmed mean, median, trimmed standard deviation, peak and trimmed peak.
The mean is the simple arithmetic average of all fluorescent intensities for the microsphere set that pass gating criteria. The standard deviation is the standard deviation of the simple mean calculation. The trimmed mean is an average of the fluorescent intensities collected in a sample, using an algorithm that appears to remove data points on both sides of the median. The trimmed standard deviation is the standard deviation of the data points used in calculating the trimmed mean. The median is the most commonly used value for most instrument users.
Peak and trimmed peak values were not used because the Bioplex XML file does not present the "peak" values that are present in the Luminex CSV file. The peak value should correspond to a mode. Examination of distributions of individual microsphere events was done using data from the Bioplex XML file. However, these showed enough complexity, and since the precise algorithm used by the Luminex was unknown, attempting to calculate a facsimile peak value from Bioplex XML data was abandoned. Thus, it was not possible to include these data as a further test of normality of distribution for both datasets.
Distributions were examined for normality, focusing on what is usually available to users of the instrument. A simple preliminary test for normality of the distribution is to divide the mean by the median and the peak (mode) for the datasets. If the sample distribution is normal then these values are equal and the ratio is 1:1. If it is skewed, then the mean will be some multiple of the median if the skew is toward the high end, or some fraction of the median if the skew is towards the low end. While this test would not be correct under all conditions in the absence of the peak values, visual examination of some histograms of microsphere distributions taken from the Bioplex shows that it appears adequate for this instrument.
This examination showed that trimmed mean and trimmed standard deviation was the optimum data source for the instrument for this study, since analysis used standard deviations of individual readings (not shown), although the median is more commonly used by biologists employing this instrument.
Results and ratio analyses
Results from microsphere intraplex assays where m = 4 and m = 5 are presented. Several ratios were studied.
For the first ratio, the mean of a set of 6 uncoated microspheres was used as denominator. This mean value was then used to determine a ratio with all the other SMPCS's in each intraplex assay. This is termed an 'external ratio' because it was external to the intraplex set for a single assay.
The second type of ratio was as follows. Since several different intraplex assays were used together (i.e. a multiplexed intraplex), the mean of a different intraplex assay could be explored as a ratio denominator: for example, the ratio of each SRV SMPCS's fluorescent reporter intensity against the mean of the SFV SMPCS's fluorescent reporter intensities, and vice versa.
The third ratio is the mean of all values for each intraplex set to their own mean as denominator. Each SMPCS's reading is used as the numerator over the mean of all the values in that set. The ratio of all SMPCS reporters in the intraplex was taken against that mean. This is termed an internal ratio against the self mean.
Discussion of intraplex ratios
The amount of analysis that could be presented here is considerable. These figures and tables show the essence of what is important for understanding the improvement derived from this new assay technique. The primary work compared results for assay plates with 32 replicate wells where each plate was read on two different instruments.
For each well, a ratio between the fluorescent intensity (FI) and several denominators was taken. The denominators were: mean of uncoated control microsphere FI; FI mean of external real assays; FI self mean of the intraplex set; and FI of one arbitrarily selected SMPCS from the intraplex.
For each SMPCS, the mean, median, maximum, minimum, and standard deviation were calculated for each 32-well replicate serum titration.
Between instruments, the ratios of the mean, median, maximum, minimum and standard deviations were calculated for each serum titration. This was done for each permutation of denominators taken in step 1.
Difference of means test
Above, and are the mean of the respective reading sets for the two instruments, n1 and n2 is the number of readings, s1 and s2 are the standard deviations of the samples. For these tests the same set of 32 replicated sample wells was read, once on instrument A followed by repeating the same plate on instrument B, the anticipated results are identical.
Comparison of stability between instruments of three methods: A. internal self mean ratio; B. ratios based on an external assay; and C raw instrument data. Internal self mean (A) is the most reliable. Using external ratios, (2C) is a close second, and raw readings, (2C) show the greatest deviation between instruments.
Ratios N = 32 for all.
Mean z score
Median z score
Ratios on internal self mean
Rh. Sera 1:50
Rh. Sera 1:100
Rh. Sera 1:200
Ratios on real external mean (SRV/SFV mean and SFV/SRV mean)
Rh. Sera 1:50
Rh. Sera 1:100
Rh. Sera 1:200
Raw inter instrument comparisons
Rh. Sera 1:50
Rh. Sera 1:100
Rh. Sera 1:200
This study indicates that intraplex methodology provides significant benefits to suspended microarray assay precision, and that for an intraplex analysis the ratio to the internal self-mean would be optimal to use, although a developer may choose an external method for some circumstance, or use both internal and external methods together as cross validations. An intraplex should produce reliable results regardless of which specific instrument (appropriate for the assay manufacturer) is used. Intraplex ratios compensated for known assay error modes.
A graph of the internal self-mean clustering will show n ratios moving closer together, with a high or low outlier in most instances, since signal response levels will usually vary semi-logarithmically as the analyte concentration is lowered, frequently causing mean of m to have an apparent outlier. This clustering provides a measure correlated to concentration of analyte.
To achieve intra-plate standard concentration determination independence, intraplex assays can be run by an assay developer at differing levels of known analyte. Ratios for each analyte assay can then be generated for each intraplex assay batch. These ratios can then be used to provide an independent intra-assay correlation with analyte concentration. To make the assay even more precise, intraplex assays could be used together with the current system of creating a standard curve for each assay plate. Combining such results will allow diagnosis of problems with standard solutions, and provide potentially greater precision.
Intraplexing assays are useful for several purposes. Intraplexing should provide a means of making the serious issue of unpredictable large carryover events visible should they occur, and can compensate for them. An intraplex assay that is carefully calibrated by replication should show a characteristic set of relationships between the components of the assay. Proper analysis of results should enable outlier readings for an SMPCS to be discarded. Thus, an intraplex of 5 to 10 SMPCS's should provide a good degree of accuracy.
Having a value of n ≥ 5 for the remainder of an m × n intraplex after culling possible outliers provides useful statistical significance, although some may accept lower values of n and some may require higher. The processed data from an individual well, using intraplexing, can have a validity that is currently unavailable, thus avoiding requirements for sample replication in many uses. Validity will be generally based on t tests, but with a reasonable confidence. This can allow software vendors to make better judgments for users regarding the statistical significance of a result.
Users of suspended microarray assay systems should take note of this method and apply its results as appropriate to their systems. Much of these results apply to "smart dust", smart microspheres, bar coded microspheres, microrods and others. To confer optimum precision for research, clinical use and other applications on this sector of assay technology, the matters raised here also should be considered for these alternative assay methods. Additionally, users may want to take note of the potential for significant differences between instruments when instruments are calibrated to the same standard.
Elizabeth Reay is thanked for manuscript editing; Paul Luciw is thanked for use of laboratory facilities, Resmi Ravindran for collaboration, Joann Yee and the California Primate Research Center for generosity in supplying both the sera for these experiments, and use of facilities to run assays on their Bioplex. Imran Khan, Melanie Ziman, and Sara Mendoza contributed to creation of the monkey serum diagnostic microsphere sets used in this work. The laboratory of Thomas North is thanked for use of facilities, as is Jesse Deere, also of the North laboratory. This work was supported by BW Education and Forensics of Cheyenne, Wyoming, and KonnectWorld, Inc. of Davis, California.
- Earley MC, Jr RFV, Shapiro HM, Mandy FF, Kellar KL, Bellisario R, Pass KA, Marti GE, Stewart CC, Hannon WH: Report from a workshop on multianalyte microsphere assays. Cytometry. 2002, 50 (5): 239-242. 10.1002/cyto.10140.View ArticlePubMedGoogle Scholar
- Gosselin RC, Janatpour KA, Owings JT: Evaluation of multiplex testing for measuring phospholipids antibodies in thromboembolic patients.: Suzhou, China.2006, ,Google Scholar
- Kellar KL, Douglass JP: Multiplexed microsphere-based flow cytometric immunoassays for human cytokines. J Immunol Methods. 2003, 279 (1-2): 277-85. 10.1016/S0022-1759(03)00248-5.View ArticlePubMedGoogle Scholar
- Khan SS, Smith MS, Reda D, Suffredini AF, J. Philip McCoy J: Multiplex Bead Array Assays for Detection of Soluble Cytokines: Comparison of Sensitivity and Quantitative Values Among Kits from Multiple Manufacturers. Cytometry B Clin Cytom. 2004, 61 (1): 35-39. 10.1002/cyto.b.20021.View ArticlePubMedGoogle Scholar
- Kingsmore SF: Multiplexed protein measurement: technologies and applications of protein and antibody arrays. Nature Reviews Drug Discovery. 2006, 5: 310-321. 10.1038/nrd2006.PubMed CentralView ArticlePubMedGoogle Scholar
- Lash GE, Scaife PJ, Innes BA, Otun HA, Robson SC, Searle RF, Bulmer JN: Comparison of three multiplex cytokine analysis systems: Luminex, SearchLight(tm) and FAST Quant(c). Journal of Immunological Methods. 2006, 309: 205-208. 10.1016/j.jim.2005.12.007.View ArticlePubMedGoogle Scholar
- Liu MY, Xydakis AM, Hoogeveen RC, Jones PH, Smith EOB, Nelson KW, Ballantyne CM: Multiplexed Analysis of Biomarkers Related to Obesity and Metabolic Syndrome in Human Plasma, Using the Luminex-100 System. Clinical Chemistry. 2005, 51 (7): 1102-1109. 10.1373/clinchem.2004.047084.View ArticlePubMedGoogle Scholar
- Jacobson JW, Oliver KG, Weiss C, Kettman J: Analysis of Individual Data from Bead-Based Assays ("Bead Arrays"). Cytometry A. 2006, 69 (5): 384-390. 10.1002/cyto.a.20293.View ArticlePubMedGoogle Scholar
- Hanley B, Xing L, Cheng RH: Variance in multiplex suspension array assays: microsphere size variation impact. Theor Biol Med Model. 2007, 4 (1): 31-10.1186/1742-4682-4-31.PubMed CentralView ArticlePubMedGoogle Scholar
- Hanley B: Variance in multiplex suspension array assays: carryover of microspheres between sample wells. Journal of Negative Results in BioMedicine. 2007, 6 (6):
This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.