Optimization and enhancement of H&E stained microscopical images by applying bilinear interpolation method on lab color mode
© Kuru; licensee BioMed Central Ltd. 2014
Received: 10 November 2013
Accepted: 4 February 2014
Published: 6 February 2014
Hematoxylin & Eosin (H&E) is a widely employed technique in pathology and histology to distinguish nuclei and cytoplasm in tissues by staining them in different colors. This procedure helps to ease the diagnosis by enhancing contrast through digital microscopes. However, microscopic digital images obtained from this technique usually suffer from uneven lighting, i.e. poor Koehler illumination. Several off-the-shelf methods particularly established to correct this problem along with some popular general commercial tools have been examined to find out a robust solution.
First, the characteristics of uneven lighting in pathological images obtained from the H&E technique are revealed, and then how the quality of these images can be improved by employing bilinear interpolation based approach applied on the channels of Lab color mode is explored without losing any essential detail, especially for the color information of nuclei (hematoxylin stained sections). Second, an approach to enhance the nuclei details that are a fundamental part of diagnosis and crucially needed by the pathologists who work with digital images is demonstrated.
Merits of the proposed methodology are substantiated on sample microscopic images. The results show that the proposed methodology not only remedies the deficiencies of H&E microscopical images, but also enhances delicate details.
Non-uniform illumination problems in H&E microscopical images can be corrected without compromising crucial details that are essential for revealing the features of tissue samples.
Pathology is the study of cells and tissues in which structural and functional changes take place. Pathologists examine tissue slices,a cytology, body fluids for abnormal levels of chemicals and the presence of crystals; they carry out molecular studies to diagnose diseases under the microscope. Identifying abnormal cells and the morphology of tissues under the light microscope are essentially conducted by histological staining techniques. Among these, Hematoxylin & Eosin (H&E) technique is one of the most common employed techniques . This technique has become the oversight method of first choice for most practitioners of normal histology and histopathology since first introduced in 1876 . The technique allows both enhancing contrast and discerning between nuclei and cytoplasm in tissues by staining them in different colors. These colors are namely blue by applying hemalum, which is a complex formed from aluminium ions and oxidized hematoxylin, and reddish (namely red, pink and orange)by applying an aqueous or alcoholic solution of eosin. This bi-coloring process helps to better depict the microscopic morphology of tissues and cells, and eases the diagnosis through the microscope even though dyes are highly inconsistent for various reasonsb.
With the recent advances in digital imaging, the latest digital cameras coupled with powerful computer methods offer not only better image quality that is comparable with traditional silver halide film photography, but also greater flexibility for image manipulation and storage; hence, they are increasingly being used for image capture for microscopy – an area that demands high resolution, color fidelity and careful management . Acquiring better images as well as depicting slices better through microscopes depends essentially on three interrelated components: one of which is the proper alignment of the filament and condenser; the other one is the quality of microscopes; and the last one is the regular maintenance of microscopes including calibration. These issues are explained briefly in the following paragraph.
One of the most misunderstood and generally neglected concepts in optical microscopy is the proper configuration of the microscope with regards to illumination, which is a critical parameter that must be fulfilled in order to achieve optimum performance . The intensity and wavelength spectrum of light emitted by the illumination source is of significant importance, but even more essential is that light emitted from various locations on the lamp filament be collected and focused at the plane of the condenser aperture diaphragm . In this respect, the proper alignment of the filament and condenser is an essential requirement to establish good Koehler illumination. Moreover, when working with living cells it is imperative to avoid relatively high light intensities and long exposure times that are typically employed in recording images of fixed cells and tissues (where photo bleaching is the major consideration) . The misalignment of the filament and condenser along with low light situations lead to serious illumination problems; this reduces the quality of the digital images, in particular those obtained from the H&E technique.c Moreover, it should be noted that there are several types of microscopes ranging from a good quality that is unbelievably expensive to a less quality that is much less expensive. Quality of calibration is very much dependent on the quality of microscopes. Some less quality microscopes may not produce satisfactory results even though they are calibrated well which leads to an illumination problem at any circumstances. Since bulbs used in microscopy as an illuminator have a limited life, they should be changed regularly in terms of the hours they are used. Therefore, regular maintenance of microscopes is a vital issue both for depicting slices better and for acquiring better images. In this empirical study, an effective approach is presented by addressing this specific problem that is commonly encountered by pathologists and histologists.
The methodology presented in this study not only addresses the deficiencies of the general off-the-shelf commercial methods in terms of enhancing H&E images, but also outperforms better than the specific methods that have been established for remedying the particular illumination problem in microscopic images.
Methodology and results
To summarize, several off-the-shelf methods particularly established to correct illumination problem in H&E images along with some popular general commercial methods have been examined to find out a robust solution. Our evaluation of these previous studies has formed the basis for the software requirement analysis that consists of architectural, structural, behavioral and functional requirements. The methodology proposed in this study will be described in detail and its effectiveness will be demonstrated on sample microscopic images in the following two sections.
Correction of illumination
Enhancing the delicate details in nuclei
We may explore some other possibilities for images of H&E stained microscopical materials by using Lab Color mode by adopting some basic image processing algorithms to other channels of Lab color mode, namely “a” and “b”. Unsharp mask is a classic sharpening technique, which was widely used by photographers before computer invented. The main idea is to blend image with blurred version. It emphasis edges and makes the image sharper. There are three parameters which are amount of sharpen, radius of blur, and threshold. The first parameter specifies how strong to sharpen image. The algorithm creates blurred version of the image and for each pixel calculates the difference between the original and the blurred image . After that, it uses this difference to sharpen the image. The algorithm adds (difference × amount/100) to each pixel of the original image with amount specifying a percent of the difference to be added to the original image. The second parameter specifies radius of blur effect which creates defocused image. The higher value you enter, the wider will be edges. That’s why be careful when you adjust this parameter - a big radius will lead to unnatural effects (false halo around objects of the image). The third parameter specifies a threshold of the effect. If the difference between the pixels values of the original and the blurred images is less than the threshold, it is discarded. It allows to keep minor details unchanged and to apply sharpening only on noticeable details. For example, if you sharpen a photo which contains face, you would like to sharpen facial features (nose, lips, eyes, etc), but do not emphasize pimples, birthmark and other minor details. Zero is opted for the threshold to include all the minor details in this study. Here, an image processing algorithm is applied to both channels of L and b (nuclei) that is the prime important for pathologist to diagnose. In our study, some of the L* values are processed by referring only to b* values.
Interface of the methodology
The proposed methodology has been implemented both in Java and in .NET.m The interface of the application is depicted in Figure 19. The program allows the user to load an image acquired using the H&E staining technique, choose the source points and specify the reference L* values. Possible white areas can be seen easily in four quadrants of the images after they are highlighted by a magnification utility provided by the application since sometimes they are not evident by naked eye. Choosing points other than white areas may cause unexpected results. In order to facilitate the process of choosing suitable points, the program can highlight the probable white areas of the image (i.e. with L* values in CIE LAB model close to 100 and not smaller than 90n) and automatically determine the source points by dividing the image in four quadrants and picking in each quadrant the point among possible white points which is closest to the centroid of those points; if such a point does not exist in a quadrant, then the centroid of the points that are symmetric to the found points in other quadrants with respect to the XY axis is chosen as the source point and its mask value is set to the average of the mask values of the source points in other quadrants. “Correction of illumination” is implemented by the utility “optimize illumination” while “enhancing the details of nuclei” is implemented by the utility “sharpen” in the application. The course of illumination and enhancement can be adjusted using the factor selection screens in the application as displayed in Figure 19.
Microscopic digital imaging and telepathology have improved the practice of pathology in enabling examinations to be managed from remote locations. Telepathology allows pathologists to access second opinions in difficult cases as well as to participate in case discussions without traveling over long distances. Furthermore it allows hospitals without a pathologist to access expert opinions. However, microscopic digital images obtained from H&E technique suffer from uneven lighting. There are several common commercial methods that work in RGB mode to solve a widespread illumination problem generally occurred in photographs such as white balancing. These methods lead to a color shift in images which might not be very important for photographs. However, a color shift makes up an H&E image almost a different image on which it seems almost impossible to decide if the tissue has cancer cells. In addition to these commercial methods, several specific methods for remedying uneven illumination in microscopic images have been established. Some of the main drawbacks of these methods regarding H&E images are that it lacks in correcting the non-uniform illumination and it causes serious artifacts as mentioned in background section. These methods specific to microscopic images along with the commonly used commercial off-the-shelf methods are not successful to handle this kind of specific problem in H&E images. That’s why; these H&E images are left as they are as seen in medicine textbooks as well as in many publications. The problem needs to be solved specific to the features of the illumination problem and the features of the H&E images. This study by covering this specific problem area in pathology and histology presents a novel approach comprising both bilinear interpolation and Lab color mode to remedy the deficiencies of current methods. The effectiveness of the correction lies in quadratic estimation using XY plane as mentioned in the manuscript regarding Figure 13.
In the study, at first, the features of H&E images have been examined to reveal the patterns of illumination problems. In this respect, the distribution of illumination problem in a microscopic image is described. The perfect match between the H&E staining technique and Lab color mode is delineated and two methods employed in the channels of Lab separately as well as interrelated of channels are presented. The experimental results show that the illumination problem could be managed effectively and efficiently and delicate details can be enhanced not compromising precious data (H&E sections).
In this study, an effective and efficient methodology for addressing the problem of uneven lighting in pathological images obtained using Hematoxylin & Eosin staining technique has been proposed. The method exploits the features of such images by first converting them into the CIE LAB color model, in which the lightness and the chromaticity of pixels can be modified independent of each other, and then employing a new approach along with bilinear interpolation to mask out the uneven lighting that spreads through an image. Furthermore, enhancement of H&E images is implemented by a novel approach together with unsharp mask. The empirical studies on a diverse set of sample images illustrated in the manuscript demonstrate that the proposed method is effective in improving the quality of H&E images; unlike other ad-hoc methods implemented in RGB color space, a color shift is not observed in the resulting images and crucial details are preserved. The established application provides pathologists with little or no image processing experience, an effective means of optimizing and enhancing H&E images for printing, image analysis, or telepathology.
Some other basic image processing algorithms may be employed to other channels, a* and b*, of Lab color mode as well as L* channel as a future work. For instance, in terms of the chromaticity layers, a* and b*, the features of color problems of images could be analyzed. In this respect images should be captured from both non calibrated and calibrated microscopes and should be analyzed in order to recognize the patterns of color problems. New enhancement algorithms might be established to correct the problems in the chromaticity layers using acquired patterns of color problems.
aTissue samples may be obtained during surgery, after death or at a biopsy.
bDyes are not pure substances; slight variations in manufacturing will have a profound effect on the final product; each manufacturer has its unique way to make a particular dye; constantly evolving health and safety regulations cause alterations in manufacturing protocols; recent geographic shifts in where dyes are made has had a major impact on the nature of dyes. Thus, sometimes it is not possible to see distinct blue or red color on images. Interested readers can reach more information about the H&E technique from the book whose editors are Kumar and Kiernan .
cThe interested readers are referred to the articles written by Drent  and Fellers  for more information about the causes of both noise and poor Koehler illumination in digital microscopic images.
dNote that another name of auto-levels is white balancing.
eThe plug-in can be downloaded from the web site, http://rsbweb.nih.gov/ij/plugins/inserm514/Documentation/A_posteriori_shading_correction_514_v3/A_posteriori_shading_correction_514_v3.html. Some sections of the image cannot be restored as explained at the website. Interested readers can reach some more examples from this website.
fThe plug-in can be downloaded from the web site, http://www.optinav.com/Polynomial_Fit.htm.
gThe plug-in can be downloaded from the web site, http://imagejdocu.tudor.lu/doku.php?id=plugin:filter:fit_polynomial:start.
hMore examples can be reached from the web site, http://www.mecourse.com/landinig/software/cdeconv/cdeconv.html.
iMany images were obtained from a wide range of microscopes regarding the images we pulled out of publications and the images we obtained in our laboratory by employing several different set up options. All the microscopes from which our pathologists detected images are regularly well calibrated microscopes and very expensive ones in good quality.
jCorrection is applied for all channels namely red, green and blue separately by color separation and then these channels are combined together to form an image.
krgbImage1 = imread('specimen1.png');rgbImage2 = imread('corrected1.png');colorTransform = makecform('srgb2lab'); labImage1 = applycform(rgbImage1, colorTransform); labImage2 = applycform(rgbImage2,colorTransform);BChannel1 = labImage1(:, :, 3); BChannel2 = labImage2(:, :, 3);[h,p,CI,stats] = vartest2(double(BChannel1(:)),double(BChannel2(:)),0.05).
lNote that two of these images are the images depicted in Figure 1 on which several prominent methods are employed. A comparison of corrected illumination problem in these two images should be performed in terms of the proposed approach and the current prominent methods employed.
mThe program can be downloaded from http://goo.gl/UGQR3.
nThese values are not observed for the sections stained by H&E technique.
Conversion between RGB and CIE LAB color models
In the study of color perception, one of the first mathematically defined color spaces was the CIE XYZ color space, created by the International Commission on Illumination (CIE) in 1931. CIE XYZ may be thought of as derived parameters from CIE RGB color space, the red, green, blue colors. CIE LAB color space is based directly on the CIE XYZ color space as an attempt to linearize the perceptibility of color differences. The non-linear relations for L*, a*, and b* are intended to mimic the logarithmic response of the eye. In order to convert an image from RGB color space to CIE LAB color space (or vice versa), the CIE XYZ color space is used as an intermediate color space at transformation phases from one color space into other .
The author would like to thank Dr. Bulent Celasun, MD, a pathologist by whom the specific problem and needed urgent solution has been pointed out. The approach mentioned throughout the manuscript has been improved mainly based on the advices of the pathologists working in GATA and Baskent University in the leadership of Dr. Celasun. Previous versions to be tested were presented at the web page, http://www.patoloji.gen.tr/HE-ImageOptimizer.htm and the ultimate version has been established based on the feedbacks and the appreciations of many pathologists. The author is very grateful to Dr. Sertan Girgin, PhD, a computer engineer, for his advices to build up the methodology. The manuscript has been enhanced thanks to attendants of 6th IAPR International Conference on Pattern Recognition in Bioinformatics as well .
- Ruifrok AC, Johnston DA: Quantification of histochemical staining by color deconvolution. Anal Quant Cytol Histol. 2001, 23: 291-299.PubMedGoogle Scholar
- Kumar GL, Kiernan JA: Special stains and H&E. 2010, California, Dako: Carpinteria, 2Google Scholar
- Drent P: Digital imaging new opportunities for microscopy. 2010, Nikon,http://www.microscopyu.com/articles/digitalimaging/drentdigital.html,Google Scholar
- Parry-Hill M, Sutter RT, Davidson MW: Microscope alignment for kohler illumination. 2004, Nikon,http://www.microscopyu.com/tutorials/java/kohler/,Google Scholar
- Hazelwood KL, Olenych SG, Griffin JD, Murphy CS, Cathcart JA, Davidson MW: Introduction to digital imaging in microscopy. 2009, Nikon,http://hamamatsu.magnet.fsu.edu/articles/microscopyimaging.html,Google Scholar
- Sternberg S: Biomedical image processing. IEEE Comput. 1983, 16: 22-34.View ArticleGoogle Scholar
- Likar B, Maintz JB, Viergever MA, Pernus F: Retrospective shading correction based on entropy minimization. J Microsc. 2000, 197: 285-295. 10.1046/j.1365-2818.2000.00669.x.View ArticlePubMedGoogle Scholar
- Leong FJMN, Brady M, McGee JOD: Correction of uneven illumination (vignetting) in digital microscopy images. J Clin Pathol. 2003, 56: 619-621. 10.1136/jcp.56.8.619.PubMed CentralView ArticlePubMedGoogle Scholar
- Walker RA: Quantification of immunohistochemistry - issues concerning methods, utility and semiquantitative assessment I. Histopathology. 2006, 49: 406-410. 10.1111/j.1365-2559.2006.02514.x.View ArticlePubMedGoogle Scholar
- Taylor C, Levenson R: Quantification of immunohistochemistry – issues concerning methods, utility and semiquantitative assessment II. Histopathology. 2006, 49: 411-424. 10.1111/j.1365-2559.2006.02513.x.View ArticlePubMedGoogle Scholar
- Reyes-Aldasoro CC: Retrospective shading correction algorithm based on signal envelope estimation. Electron Lett. 2009, 45: 454-456. 10.1049/el.2009.0320.View ArticleGoogle Scholar
- Tuominen VJ, Ruotoistenmäki S, Viitanen A, Jumppanen M, Isola J: ImmunoRatio: a publicly available web application for quantitative image analysis of estrogen receptor (ER), progesterone receptor (PR), and Ki-67. Breast Canc Res. 2010, 12: R56-10.1186/bcr2615.View ArticleGoogle Scholar
- Rabinovich A, Agarwal S, Laris CA, Price JH, Belongie S: Unsupervised color decomposition of histologically stained tissue samples. Advances in Neural Information Processing Systems 16. Edited by: Thrun S. 2004, USA: MIT Press, 575-583.Google Scholar
- Fairchild M: Wiley-IS&T Series in Imaging Science and T. Color Appearance Models. 2005, UK: J Wiley PressGoogle Scholar
- Wyble DR, Fairchild MD: Prediction of munsell appearance scales using various color appearance models. Color Res Appl. 2000, 25: 132-144. 10.1002/(SICI)1520-6378(200004)25:2<132::AID-COL8>3.0.CO;2-D.View ArticleGoogle Scholar
- Dalheimer K, Welsh M: Running Linux. 2006, USA: O’Reilly Press, 5Google Scholar
- Gonzalez RC, Woods RE: Digital Image Processing. 2008, USA: Prentice Hall Press, 3Google Scholar
- Fellers TJ, Vogt KM, Davidson MW: Ccd signal-to-noise ratio. Nikon. 2009, Nikon,http://www.microscopyu.com/tutorials/java/digitalimaging/signaltonoise/,Google Scholar
- Kuru K, Girgin S: A bilinear interpolation based approach for optimizing hematoxylin and eosin stained microscopical images. The Procedings of 6th IAPR International Conference on Pattern Recognition in Bioinformatics. Edited by: Loog M, Wessels L, Reinders MJT, Ridder D. 2011, Berlin-Heidelberg: Springer, 168-178.View ArticleGoogle Scholar
- Ford A, Roberts A: Colour space conversions. Westminster University (August 1998) Technical Report.http://biology.duke.edu/johnsenlab/pdfs/tech/colorconversion.pdf,
This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly credited. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.