Research ArticleImaging

Wide-field computational imaging of pathology slides using lens-free on-chip microscopy

See allHide authors and affiliations

Science Translational Medicine  17 Dec 2014:
Vol. 6, Issue 267, pp. 267ra175
DOI: 10.1126/scitranslmed.3009850

Abstract

Optical examination of microscale features in pathology slides is one of the gold standards to diagnose disease. However, the use of conventional light microscopes is partially limited owing to their relatively high cost, bulkiness of lens-based optics, small field of view (FOV), and requirements for lateral scanning and three-dimensional (3D) focus adjustment. We illustrate the performance of a computational lens-free, holographic on-chip microscope that uses the transport-of-intensity equation, multi-height iterative phase retrieval, and rotational field transformations to perform wide-FOV imaging of pathology samples with comparable image quality to a traditional transmission lens-based microscope. The holographically reconstructed image can be digitally focused at any depth within the object FOV (after image capture) without the need for mechanical focus adjustment and is also digitally corrected for artifacts arising from uncontrolled tilting and height variations between the sample and sensor planes. Using this lens-free on-chip microscope, we successfully imaged invasive carcinoma cells within human breast sections, Papanicolaou smears revealing a high-grade squamous intraepithelial lesion, and sickle cell anemia blood smears over a FOV of 20.5 mm2. The resulting wide-field lens-free images had sufficient image resolution and contrast for clinical evaluation, as demonstrated by a pathologist’s blinded diagnosis of breast cancer tissue samples, achieving an overall accuracy of ~99%. By providing high-resolution images of large-area pathology samples with 3D digital focus adjustment, lens-free on-chip microscopy can be useful in resource-limited and point-of-care settings.

INTRODUCTION

Patients in developing countries often experience poor health care, as a result of either limited access to medical infrastructure or the lack of trained medical personnel. Recent advances in communication technologies and telemedicine offer opportunities to bridge some of these gaps by enabling rapid and secure sharing of medical data and patient health records with remote physicians. However, pathology is still largely constrained to advanced clinical settings, because it is partially bottlenecked by the requirements of high-throughput imaging devices that can automatically scan and digitize large sample areas (typically >1 cm2) (1); for example, to digitize a 1-cm2 sample with a traditional light microscope would typically require capture of >500 to 1000 individual images.

The current gold standard for pathologists to observe microscale objects within a tissue section or a smear is the bright-field optical transmission microscopy. Although widely used in clinical settings, traditional optical microscopy has several limitations. First, a high-quality pathology microscope and its carefully crafted objective lenses are costly. Additionally, the microscope’s field of view (FOV) is inversely proportional to the square of its total magnification factor, thus limiting its ability to observe large samples, such as whole histology slides, without mechanical scanning. Lateral scanning can expand the FOV of a microscope image; however, its usefulness is partially compromised owing to the potential sample tilt and out-of-plane regions that naturally occur in large pathology samples, which necessitate dynamic focus adjustment through additional depth scanning and digital signal processing (2).

Recent developments in optics and electronics technologies—mainly image sensor chips with larger numbers of pixels and smaller pixel sizes—together with the constantly reducing cost of high-performance computation, have led to the emerging field of computational microscopy, which endeavors to overcome the limitations of traditional light microscopes using novel imaging designs and reconstruction algorithms (314). For example, recent advances in incoherent holography methods and spatial light modulators have fostered new computational microscopy techniques that permit three-dimensional (3D) imaging of fluorescent samples (14, 15). Also, compressive sampling techniques (16) have enabled 3D reconstruction of objects from a relatively small number of image frames or pixels (3). All of these emerging computational imaging approaches and various others (5, 7, 1721) use reconstruction algorithms for improving the performances of lens-based imaging designs.

Lens-free on-chip imaging takes an alternative approach and uses an optoelectronic sensor array to directly sample an object’s diffraction pattern to reconstruct its image (9, 10, 22). On the basis of digital in-line holography, this recently emerging computational imaging approach does not require lenses to form an image, enabling its setup to be simple, compact, and cost-effective (9, 23). Because the light source–to–object distance (Z1) is much larger than the sample-to-sensor distance (Z2), the lens-free holograms of the objects are captured with unit magnification (Fig. 1A). Therefore, the FOV of a lens-free microscope is equal to the image sensor’s active area (for example, 20 to 30 mm2) and can be hundreds of times larger than that of a lens-based microscope, which is typically ~0.1 to 0.2 mm2 for high-resolution digital imaging (9).

Fig. 1. Lens-free imaging experimental setup and image reconstruction.

(A) Setup contained a partially coherent illumination source, a sample (a pathology slide), and an image sensor chip. The illumination source was placed 7 to 15 cm away from the pathology slide (Z1), whereas the image sensor was positioned closely 100 to 600 μm underneath the sample (Z2). Therefore, instead of recording a direct image of the specimen, an interference pattern was recorded by the image sensor, which was then reconstructed to reveal 3D images of the specimens. (B) Image reconstruction steps. The imaging process starts by capturing a sequence of low-resolution holograms, each subpixel shifted with respect to others at each height (Z2 distance). For every height, the pixel super-resolution algorithm synthesized one high-resolution hologram out of these multiple low-resolution holograms. These resulting “pixel super-resolved” holograms at different heights were then fed to a multi-height phase recovery algorithm, which included rotational field transformations, to retrieve the lost phase of the optical wave, thus enabling the reconstruction of the specimen with 3D digital focus adjustments. To accelerate the convergence of the iterative phase recovery algorithm, the phase solution of TIE was used as an initial phase guess. Lastly, digital colorization algorithms provided color representation of the lens-free reconstructed image.

Here, we demonstrate that lens-free holographic on-chip microscopy can image pathology slides over an ultrawide FOV (>20 mm2) with a spatial resolution and contrast sufficient for clinical diagnosis, which was confirmed through blinded analyses by an expert pathologist. In our study, we imaged invasive carcinoma cells within human breast sections, Papanicolaou (Pap) smears consistent with a high-grade squamous intraepithelial lesion, and sickle cell anemia blood smears. Before this work, imaging of tissue samples could not be achieved using lens-free holographic on-chip imaging owing to phase recovery problems. We solved these issues through several improvements. First, we digitally solved the transport-of-intensity equation (TIE) to create an initial guess for the multi-height iterative phase retrieval, avoiding phase stagnation in high-resolution imaging of tissue samples using pixel super-resolved in-line holography. Second, rotational field transformations were implemented to digitally compensate for unavoidable sample tilt and misalignments with respect to the sensor plane. Furthermore, we used a digital colorization technique based on statistical transformation of a monocolor reconstructed lens-free image into a color image, which provides improved speed compared to previous colorization techniques that were used in lens-free imaging. Using this approach, we demonstrated that despite the monocolor nature of holography, color images can be rapidly generated for tissue samples. Lens-free computational on-chip microscopy can transform pathology, especially at the point of care, by providing a compact 3D imaging technology that, importantly, can visualize large FOVs for telemedicine applications.

RESULTS

Lens-free imaging setup

The setup of a lens-free on-chip microscope based on multi-height phase retrieval is shown in Fig. 1A. The microscope was composed of a partially coherent illumination source (2-nm bandwidth), a scanning stage, a complementary metal-oxide semiconductor (CMOS) image sensor chip (1.12-μm pixel size), and a computer that controlled the setup. The pathology slide of interest was mounted on the stage through a 3D-printed sample holder and was illuminated to project an in-line hologram onto the sensor chip. Hologram intensity was recorded and saved to the computer for digital reconstruction of the object image. Because Z1Z2, the magnification of the hologram plane with respect to the object plane is unit (24), enabling a sample FOV as large as the active area of the image sensor chip (Fig. 1). The FOV was 20.5 mm2 for the CMOS sensor used in this work, but could be as large as ~18 cm2 for a charge-coupled device image sensor (9). To mitigate unit magnification of on-chip imaging and the finite pixel size of the image sensor, a pixel super-resolution technique was implemented to prevent spatial sampling errors (25, 26). By capturing a number of low-resolution holograms at subpixel shifts with respect to each other, the number of sample image points was increased and the pixel size was equivalently reduced.

Instead of using a highly coherent source that is typically used in holographic imaging, we adopted a partially coherent illumination source in our case to reduce noise due to speckle and multiple-reflection interference. Also, because of the small distance between the sample and sensor planes, this limited (partial) temporal coherence of our illumination did not pose a resolution limit for the reconstructed images. The stage was programmed to first apply subpixel lateral shifts to the specimen on a 6 × 6 or 8 × 8 orthogonal grid and then move incrementally in the z direction to reach the next height. This process was repeated for multiple heights (typically eight), generating a 3D stack of lens-free images, which were fed to the image-processing algorithm for pixel super-resolution synthesis and multi-height phase reconstruction (Fig. 1B).

Using the computational methods described in Fig. 1B, we achieved pathology slide imaging, including tissue samples, with 3D focus adjustment over a large FOV (~20 mm2). Unlike bright-field microscopy, lens-free holographic on-chip microscopy retrieves both the amplitude and phase images of the specimen. The amplitude image is related to the scattering and absorption properties of the object, similar to a traditional bright-field microscope image, whereas the phase image is related to the thickness and refractive index variations of the object, and therefore, its information content is similar to a phase-contrast microscope image.

Imaging human invasive ductal carcinoma cells

To demonstrate that our lens-free, holographic, on-chip technology can properly image histology slides, we chose to image a 7-μm-thick formalin-fixed paraffin-embedded (FFPE) tissue slice from a patient with breast adenocarcinoma. Figure 2A shows a full FOV reconstruction (FOV, 20.5 mm2) of the slide; the lens-free holograms that lead to this reconstruction are in Fig. 2D. To emphasize the wide FOV of lens-free imaging compared to that of traditional lens-based microscopy, the digital FOVs of 40× and 20× microscope objectives are also shown in Fig. 2A (note that these rectangular FOVs for lens-based digital microscopes are estimated as the upper bounds). Using the lens-free reconstructed images (Fig. 2B), a pathologist confirmed atypical disordered epithelium. The cells exhibited nuclear enlargement, irregular nuclear contours, open chromatin, and moderate delicate cytoplasm. All of these observations made by the pathologist were in agreement with the 40× microscope objective images [numerical aperture (NA) = 0.75] (Fig. 2C).

Fig. 2. Lens-free imaging of invasive ductal carcinoma of the human breast.

(A) A full FOV (20.5 mm2) lens-free amplitude image of the specimen (~7 μm thick). For comparison, the FOVs of conventional 40× and 20× microscope objectives are shown using white solid rectangles. (B) Zoomed-in regions outlined by dotted yellow squares in the lens-free image in (A), which show atypical disordered epithelium. The cells exhibited nuclear enlargement, irregular nuclear contours, open chromatin, and moderate delicate cytoplasm. (C) Microscope comparison images of (B) taken with a 40× objective lens (0.75 NA). (D) Super-resolved lens-free holograms before digitally reconstructed to yield the images shown in (B).

Pseudocolored lens-free images of invasive human carcinoma cells

The lens-free images in Fig. 2 were acquired using a single illumination wavelength (λ = 532 nm), making them monocolor. However, a pseudocolor mapping can digitally generate lens-free color images. To demonstrate color images, we imaged a 4-μm-thick FFPE tissue section from a patient with breast carcinoma. Although the raw lens-free image was captured using only one illumination wavelength, we digitally colorized the reconstructed image using a transformation that mapped intensity to color through prior learning statistics.

This color transformation was built by finding the average color for each intensity value in a set of microscope images, then matching the histograms of the lens-free images to the learning set and having each grayscale value mapped to a unique color. In this method, because all holograms were captured at a single wavelength, the image acquisition time was equal to the monocolor case, and there is no need for a multiwavelength experiment. The initial learning step needed to be carried out only once for a specific stain-tissue combination, and therefore, new samples of the same type can be rapidly colored using the same mapping function. Figure 3A illustrates a pseudocolored lens-free image of carcinoma cells invading a connective tissue region. A pathologist examining the lens-free reconstructed color image noted that the cells were arranged in irregular nests within the connective tissue (dashed squares in Fig. 3, B and C). Each nest contained cells that exhibited increased nuclear-to-cytoplasmic ratios, hyperchromasia, irregular nuclear contours, and scant cytoplasm compared with normal cells. After examining the lens-free reconstructed color images (Fig. 3, A and B), the same pathologist later verified her qualitative assessment by examining the same FOVs using 40× microscope images (Fig. 3C).

Fig. 3. Lens-free pseudocolor image of human breast carcinoma.

(A) Lens-free amplitude image that shows invasive carcinoma cells arranged in irregular nests within the connective tissue. The image was captured using a single illumination wavelength (λ = 550 nm) and automatically colored using a transformation that mapped intensity to color. (B) Zoomed-in images of zones 1 to 3 outlined in yellow squares (A). The cells within the dashed green boxes are invasive carcinoma cells with increased nuclear-to-cytoplasmic ratios, hyperchromasia, irregular nuclear contours, and scant cytoplasm compared with normal cells. (C) Microscope comparison images of (B) taken with a 40× objective lens (0.75 NA). Scale bar, 20 μm (B and C).

To validate the diagnostic quality of our lens-free colored images, a pathologist performed a blinded analysis of images obtained from breast cancer tissue. The tissue slides were separated into three groups: (i) benign (n = 1), (ii) atypical with ductal carcinoma in situ (DCIS) (n = 2), and (iii) invasive carcinoma (n = 2). From these slides, 25 different FOVs for each clinical condition were imaged and digitized by a conventional bright-field microscope as well as our lens-free microscope, creating a total of 150 images. To correlate breast tissue diagnosis between conventional and lens-free microscopy, two blind tests (one for each imaging platform) were administered by the board-certified pathologist (S.R.K.). The results of this comparison (Table 1) revealed only one false-positive diagnosis using our lens-free images with the other 74 readouts, matching the pathology reports provided by the vendor of the slides, yielding an overall accuracy of ~ 99%. By comparison, using a conventional microscope yielded an accuracy of 100%.

Table 1. Comparison of diagnosis accuracy using conventional and lens-free on-chip microscopy images of breast tissue.

A blinded board-certified pathologist (S.R.K.) reviewed and diagnosed 150 FOVs of breast tissue images taken by conventional and lens-free microscopes. Three clinical conditions were benign, atypical/DCIS, and invasive carcinoma. The pathologist’s diagnosis was compared with the report from the tissue slide vendor.

View this table:

Lens-free color images of a Pap smear

Our lens-free microscopy platform is not only amenable to breast cancer diagnoses, but we believe that it also could be used for analyzing any tissue or specimen, which can be imaged using transmission bright-field microscopy. As another example, cervical cancer screening would benefit from high-throughput and cost-effective imaging solutions, especially in resource-limited clinics. The pathologist or cytotechnologist is required to mechanically scan, using a light microscope, large-area (at least ~1 cm2, depending on the sample preparation technique) Pap smear samples in search for precancerous cells. The large 20.5-mm2 FOV of our lens-free imaging technology could assist the pathologist to minimize the mechanical scanning (Fig. 4A). Moreover, when a suspicious cell is detected, the pathologist typically refocuses the objective lens to different depth slices within the suspicious cell, to better assess the cell’s morphology before classifying the cell. Lens-free holographic imaging can address this need because it has the ability to digitally focus the sample image to different depths after the image capture using the reconstructed complex wave of the specimen, which includes both phase and amplitude information of the object (movie S1).

Fig. 4. Lens-free color imaging of normal and abnormal Pap smears.

(A) A photograph of a normal Pap smear slide (SurePath preparation). For comparison, the FOVs of our lens-free microscope and of a bright-field microscope equipped with a 40× objective lens were marked by a dashed and a solid rectangle, respectively. (B) Lens-free images of a normal Pap smear in (A) and their microscope comparison images taken with a 40× (0.75-NA) objective lens. (C and D) Abnormal Pap smears (ThinPrep preparation). (C) A lens-free color image of atypical squamous cells. The white arrows mark a cluster of squamous epithelial cells with nuclear crowding, increased nuclear-to-cytoplasmic ratios, and slightly irregular nuclear contours. (D) Lens-free color images that show cells with high nuclear-to-cytoplasmic ratios, markedly irregular nuclear contours, hyperchromasia, and scant cytoplasm (white arrows). Microscope comparison images of (C) and (D) were taken with a 40× objective lens (0.75 NA) and are shown directly below the holographic images.

Lens-free images of a normal Pap smear and their conventional microscope comparison images are shown side by side in Fig. 4B (SurePath preparation). Two different types of abnormal Pap smears from two different pathology slides (ThinPrep preparation) were also confirmed by a pathologist using lens-free holographic imaging (Fig. 4, C and D). In Fig. 4C, the images of atypical squamous cells displayed a cluster of squamous epithelial cells with nuclear crowding, increased nuclear-to-cytoplasmic ratios, and slightly irregular nuclear contours compared with normal cells. In Fig. 4D, cells demonstrated high nuclear-to-cytoplasmic ratios, irregular nuclear contours, hyperchromasia, and scant cytoplasm, characteristic of high-grade squamous epithelial lesions. The colorization technique that we used in Fig. 4 was based on YUV color space averaging (27).

Lens-free imaging of whole-blood smears

Blood smear is still considered as one of the standard methods to identify immature or abnormal cells that are indicative of various diseases, such as anemia, hemoglobin variants, and bone marrow disorders. Using our lens-free on-chip microscopy platform, we imaged normal and abnormal blood smears (Fig. 5). The full FOV image of a normal blood smear (Wright stain) was reconstructed using eight different heights (393, 411, 431, 448, 467, 484, 500, and 516 μm) and 10 multi-height phase recovery iterations (Fig. 5A). The phase solution of TIE was used as an initial guess for the multi-height phase recovery algorithm.

Fig. 5. Lens-free imaging of normal and sickle cell anemia blood smears.

(A) Full FOV amplitude images and zoomed-in regions of normal RBCs and white blood cells and their 20× objective lens (0.5 NA) microscope comparisons. (B) Super-resolved lens-free holograms before and after reconstruction. Normal white blood cells (orange arrows) can be seen amid normal RBCs. In sickle cell anemia, abnormal RBCs (yellow arrows) mix with target cells (exhibiting a bull’s-eye appearance, red arrows), which are typical for hemoglobinopathy disease. For comparison, 20× objective lens (0.5 NA) microscope images are shown directly below their holographic lens-free counterparts. Scale bars, 20 μm.

In the normal smears, mature red blood cells (RBCs) had uniform diameters of ~7 μm, were shaped as donuts (round with an indentation in the middle), and did not have a nucleus. In abnormal smears taken from a patient with sickle cell anemia, RBCs that were spindle-shaped (“sickled”) could be visualized as well as target cells (that is, RBCs with abnormal morphology) that showed a dark spot in the middle of the cell surrounded by a bright ring (Fig. 5B). The lens-free microscope resolution also enabled the identification of white blood cells present in the blood smear. However random the lens-free holograms appeared before reconstruction, our reconstruction technique was able to transform these holograms to images that matched visibly the microscope comparison images.

Multi-height phase retrieval and TIE

To image spatially dense and connected objects such as pathology slides, our iterative multi-height phase retrieval algorithm (fig. S1) worked by assuming an initial phase guess for the complex optical field of the sample and propagating it back and forth among different heights. At each plane, the amplitude of the current guess was averaged with the amplitude of the super-resolved hologram (that is, the measurement) while keeping the current status of the phase. In each iteration, the algorithm started from the lowest plane to the highest one, processing all the heights in between, and then went backward. As the iterations proceeded, the twin image artifact that was inconsistent from one height to another was gradually removed, and the estimate of the true complex field persisted (that is, convergence was achieved). Typically, eight heights, with vertical separations of ~15 μm between adjacent heights, and 10 to 20 iterations were used to achieve convergence; as few as three to four heights also generated satisfactory results (fig. S2A).

In this iterative algorithm, a simple initial guess could be taken using the amplitude of the super-resolved hologram at the lowest height with zero phase (28, 29); however, a better guess for improved convergence was generated using TIE, which analytically computed the initial phase guess on the basis of the measurements from two different heights (30, 31). Note, however, that we only used the TIE solution for the initial phase guess to our iterative multi-height phase retrieval algorithm, and therefore, our images were not affected by the low-resolution Fresnel approximation that is inherent to TIE. Although for relatively sparse and less connected objects, TIE solution might not always be needed, for dense and connected objects, such as histopathology slides, it provides substantial convergence advantages. The use of TIE markedly reduced the number of iterations necessary for the algorithm to converge (see fig. S2A and movie S2).

The TIE can be solved either by using elliptic equation solver or by fast Fourier transform (FFT)–based method; both provide comparable results in terms of image quality (fig. S3). To validate that our spatial resolution was not compromised using TIE initial phase, we imaged a high-resolution test chart, which contained multiple gratings with varying periods. The last resolvable grating determines the resolution of the imaging system. For our lens-free microscope, the last element of group 9, which corresponded to a grating linewidth of 548 nm, was successfully resolved using λ = 532 nm (fig. S2B).

The use of a low-cost axial translation stage

The main challenge of using a low-end axial translation stage (with a cost of ~$10 to $20) is that each recorded hologram at a given height exhibits a different tilt between the axially translated sensor chip and the sample plane (Fig. 6A and fig. S4). These uncontrolled tilts result in distortions that are apparent in the reconstructed images. To digitally mitigate these distortions (after image capture), we used a rotational field transformation technique (see Materials and Methods), which is a computational method that can reconstruct a complex image on any arbitrarily tilted plane (3234). The inclusion of this rotational transformation in our multi-height phase recovery algorithm (fig. S1) is depicted in Fig. 6B. The current guess of each iteration (blue dashed rectangle) was projected onto the image sensor plane (green solid rectangle) to enforce the measured intensity while retaining the phase. This tilt correction step in the forward model of our algorithm was of paramount importance to reconstruct the phase of the optical wave.

Fig. 6. Computational tilt correction in multi-height phase recovery algorithm.

(A) When using a low-end stage to mechanically modulate the sample-to-sensor distance, mechanical tilts between the sample and the image sensor planes might occur. (B) A modification of the multi-height phase recovery algorithm, which corrects for mechanical tilts between the sample and the image sensor planes in (A). (C and D) Sample-to-sensor distances at five different heights, when the tilt angle between the planes ranged from 3.5° to 0.2° (C). This experiment represented a poorly aligned lens-free imaging setup, and these tilted measurements are used to reconstruct images shown in (D). Zoomed-in images of zones 1 to 3 are shown on the right.

To test the robustness of this algorithm, we recorded five interference patterns using five highly tilted planes (Fig. 6C), mimicking a poorly designed experimental setup that is less accurate and more prone to tilt errors than a typical inexpensive 1D translation stage. Without the tilt correction, the reconstructed images were severely distorted (Fig. 6D, upper row), in contrast with the lens-free images shown in Fig. 6D (lower row), which used the tilt correction process detailed in Fig. 6B. This computational tilt correction and sample height estimation approach will enable cost-effective implementations of lens-free on-chip microscopy, digitally eliminating the need for expensive scanning hardware. It should be noted that the tilt angles between the sample and the sensor planes are calculated using an autofocus algorithm (fig. S5). Therefore, the entire correction process occurs after image acquisition without any prior knowledge on the tilt angles between the sample and sensor planes.

DISCUSSION

Our results demonstrate that the lens-free system based on holographic imaging provides high-resolution images of patient samples that can be used by pathologists for diagnoses. The reconstructed lens-free images exhibited sufficient image quality for clinical evaluation of pathology samples, and our blind diagnostic test by a board-certified pathologist achieved an overall accuracy of ~99% for examination of human breast cancer tissue samples. The wide FOV (20.5 mm2) of lens-free imaging is more than two orders of magnitude larger than the FOV of a lens-based microscope with similar resolution (for example, a 40× objective has a digital FOV of ~0.15 mm2).

Furthermore, lens-free imaging enabled digital focusing of the image plane to different depths—a highly desired attribute that would give pathologists more degrees of freedom in their examination of the samples, because often different parts of the specimen appear in focus at different depths for large-area pathology samples. Some examples of our 3D focusing capability are illustrated in movie S1, where lens-free images of atypical squamous cells on a Pap smear were reconstructed at different depths, revealing their irregular nuclear contours and chromatin distribution. Three-dimensional imaging performance cannot be achieved using other on-chip microscopes that are based on contact imaging (12, 35) because complex optical fields cannot be retrieved using a contact imaging geometry, which demands the objects to be flat and parallel (with submicron gap) with respect to the plane of the sensor chip. Pathology samples and other biological specimens naturally have 3D features, with uncontrolled modulation of the gap between the sample and sensor planes, both of which create spatial artifacts in contact or shadow imaging. Conversely, because holographic on-chip microscopy retrieves complex optical fields of the objects, the 3D nature of specimen and uncontrolled variations in tilt and height of the specimen are digitally corrected.

In addition to wide FOV and 3D imaging capability, another advantage of lens-free on-chip holographic imaging, especially for resource-limited settings, is its cost-effectiveness and design simplicity compared to a lens-based pathology microscope. In the current setup, we used a mechanical positioning stage mainly for two reasons. First, the positioner laterally shifted the sample for pixel super-resolution; this function of the stage can be replaced by source shifting using, for instance, an array of laser diodes or light-emitting diodes (LEDs), which provide cost-effective solutions for achieving pixel super-resolution, as we reported previously (23, 36). Furthermore, in our reconstructions, we used an algorithm to automatically determine the relative subpixel shifts of each lens-free hologram, without the need for a measurement or reading from the scanning system; therefore, even a simple and inexpensive mechanical stage would work well for implementing pixel super-resolution.

Second, the mechanical stage was used to modulate and control the sample-to-sensor distance (Fig. 1A), so that we could capture several defocused interference patterns for our multi-height phase recovery algorithm. However, for this purpose, a simple and inaccurate one-axis translation stage is sufficient because we digitally estimate the sample-to-sensor distance as well as uncontrolled tilts of the sample using an autofocus algorithm with ~1-μm precision, without the need for stage readings (fig. S5). This method offered a simple way for finding the focus in an automated fashion without the need for any feedback or quantified measurement from the scanning stage or the experimental setup, which released the alignment and complexity requirements of the system. We modified our multi-height phase recovery algorithm to digitally compensate for these uncontrolled tilts and variations in sample-to-sensor distances along our FOV, which permitted the use of a low-cost 1D translation stage (see Fig. 6 and fig. S4).

All the pathology slides reported in this work were reconstructed using 288 raw lens-free images (36 holograms per height to perform pixel super-resolution and eight heights to perform multi-height phase recovery), which can be translated into image acquisition times that are on the order of several seconds using the maximum frame rate of our optoelectronic image sensor chip (15 frames/s). Image acquisition time could be improved using faster CMOS imager chips and/or pulsing of illumination source(s). In terms of the image reconstruction time, using MATLAB and eight measurement heights, the entire processing time of a 1 mm × 1 mm sub-FOV took about 4.5 min with a single Dell desktop computer. Because all reconstruction steps could be processed in parallel for different sub-FOVs, using a cluster of 20 nodes (quad-core machines), the entire FOV reconstruction could be performed within ~4.5 min. A cluster of graphics processing units (GPUs) would speed up the total reconstruction time by 10- to 20-fold because our algorithms heavily rely on FFTs (22). Optimized algorithms running on more efficient software languages, such as C/C++, would also improve reconstruction times. Therefore, even with a single desktop computer using GPUs, the processing time for full FOV reconstructions can be reduced to less than a few minutes.

Another important consideration is the coherence of illumination, both spatially and temporally. For contact on-chip microscopy, partial coherence of the source introduces spatial artifacts owing to optical diffraction that occurs between the sample and sensor planes. This unavoidable artifact is especially more pronounced for nonplanar objects and submicron features of the specimen, imaged using contact on-chip microscopy (35). Conversely, for lens-free holographic on-chip microscopy, partial coherence of the source was engineered and used in our favor to retrieve high-resolution complex fields of the specimen to digitally reverse optical diffraction. This feature allowed the vertical gap between the sample and sensor planes to be substantially larger compared to contact imaging and also vary within the sample volume, without introducing spatial artifacts. For this performance, we engineered our spatial coherence diameter at the sensor plane to be >4 mm and temporal coherence length to be ~0.1 mm, permitting high-resolution imaging of connected tissue slides over >20-mm2 FOV with significantly reduced speckle and multiple-reflection interference noise terms.

The computational lens-free microscopy platform and the reconstructed images that we have presented here have been limited to bright-field transmission imaging, although fluorescence and reflection imaging modalities are also often used in optical microscopy. Fluorescence (37) and reflection imaging (38) techniques have been integrated onto a similar lens-free bright-field imaging platform to conduct multimodal imaging on a chip. However, spatial resolution, NAs, and space-bandwidth products of these additional on-chip imaging modalities, especially fluorescence, lag behind the performance metrics of the holographic transmission imaging approach (9). This is mostly due to reduced signal-to-noise ratio of reflection or fluorescence on-chip microscopy modalities compared to transmission on-chip imaging.

For this platform technology to scale up and enter the clinical and pathological microscopy market, modularization and standardization of reconstruction algorithms and hardware would be necessary. Translation of all the modularized pieces of software blocks into GPUs could create a scalable platform where the users can define the necessary batch of computational steps for their raw lens-free images based on, for example, resolution, FOV, and speed requirements of their applications. This approach would also need improvements in the graphical user interface (GUI) of our platform such that almost all the technical details are hidden from the users with a robust and intuitive GUI. Stated differently, a similar level of transformation that the early stages of personal computers went through to reach today’s easy-to-use operating systems would also be needed for wide-scale adoption and clinical use of this computational lens-free microscopy platform.

MATERIALS AND METHODS

Study design

This study evaluated the capabilities of lens-free holographic microscopy that uses the TIE equation, multi-height iterative phase retrieval, and rotational field transformations to image pathology slides as accurately as conventional bright-field microscopy with the advantages of wider FOV together with a simpler, more cost-effective, and compact setup. We imaged blood smears, Pap smears, and breast carcinoma as examples of frequently encountered pathology specimen with high clinical significance. We quantified the ability of lens-free microscopy to enable accurate diagnosis by conducting two blind tests, where a board-certified pathologist (S.R.K.) was asked to diagnose cancerous breast tissue samples on the basis of randomly selected lens-free images and traditional microscope images of the same specimen. All the pathology slides [hematoxylin and eosin (H&E)–stained] were purchased from a vendor with anonymous pathology surgical reports, which served as the gold standard. The breast tissue samples in this blind test were separated into three groups: benign, atypical with DCIS, and invasive carcinoma. Seventy-five different FOVs (25 for each group) were used for each imaging platform, totaling 150 images. In the first blind test, randomly shuffled FOVs created by a lens-free microscope were presented to the pathologist for classification. The second blind test was administered shortly after this and consisted of FOVs taken by a conventional microscope, where the images were again randomly shuffled.

Analytical phase retrieval using the TIE

In our work, the intensity derivative along the axial direction is approximated by the differentiation of intensity measurements at two different heights divided by the distance between them. The first height was picked as the lowest one, and the second height was picked among the other heights so that they were separated by about 100 μm with respect to each other. TIE was solved using a finite element method–based elliptic equation solver (39, 40). Owing to the fact that the phase at the boundary was difficult to measure in practice, we tapered the intensity derivative gradually to zero at the edges using a Tukey window and assumed a zero Dirichlet boundary condition at the edges of the aperture. The output of the equation solver was fed to the multi-height phase retrieval algorithm as the initial guess for the optical phase. To increase the speed of the TIE solver, a faster solution to the TIE could also be generated using an FFT-based approach, but it was in theory less accurate than the elliptic equation solver owing to its periodic assumption of the boundary conditions (41). Note, however, that this FFT-based method did not introduce any visible degradation of the reconstructed image quality compared to elliptic equation solver (fig. S3).

Field transformations among tilted planes

Rotational transformation of a complex optical field is a computational method that enables the reconstruction of an image on arbitrary tilted planes using the phase information of an optical wave (3234). For example, when trying to image a tilted surface using a bright-field microscope, the microscope user has to constantly refocus the microscope at different locations within the FOV. However, if one has access to the complex field information, the entire sample can be digitally focused all at once using rotational transformations. This method is computationally inexpensive because it involves two FFT operations and a single interpolation step in the Fourier domain. To implement it and to digitally focus the entire FOV of our lens-free on-chip microscope, the local tilt angles between the image sensor and the sample need to be determined. We automatically estimated these local tilt angles by using an autofocus algorithm (fig. S5) at different spatial locations on the sample FOV and finding their absolute heights. We then fitted a plane to match these heights (Fig. 6C) and calculated the tilt angles and a 3D rotation transformation matrix to implement the needed local field transformation, which was implemented in C language to minimize the processing time.

Multi-height phase recovery with tilt correction

To take into account the tilts between the image sensor and the sample planes, we modified the multi-height phase recovery algorithm. First, the tilt angles between different planes were evaluated using our autofocus algorithm (Supplementary Materials and Methods). Second, the multi-height phase recovery process was evoked without tilt correction for 10 iterations. The result of this previous step served as an initial guess for the modified multi-height algorithm. In this modified algorithm (Fig. 6B), the current guess was still propagated among different measured planes; however, after the propagation step, the current guess (blue dashed rectangle) was projected to the tilted image sensor plane (green solid rectangle) using rotational transformation. The tilted current guess was then registered to the measured hologram. After the registration step, the tilted current guess amplitude was averaged with the measured amplitude, whereas the phase of the current guess was maintained for the next cycle of iterations. After this step, the current guess was rotated back to a parallel plane (blue dashed rectangle), and it was digitally propagated to the next measured plane/height until convergence was achieved, which usually took 10 to 20 iterations.

Sample preparation and acquisition

All the pathology slides that were imaged in this paper were anonymous and provided by a vendor or a third party without any patient-related information. The human adenocarcinoma of breast sample (H&E-stained, Fig. 2), the normal human blood smear (Wright stain, Fig. 5), and the human sickle cell anemia smear (Fig. 5) were acquired from Carolina (item nos. 318766, 313158, and 317374, respectively). The human carcinoma of the breast slides (n = 5 slides, H&E-stained, Fig. 3 and Table 1) were acquired from the Translational Pathology Core Laboratory at University of California, Los Angeles (UCLA), with de-identified surgical pathology reports. The Pap smear slides (SurePath and ThinPrep preparations, Fig. 4 and fig. S2A) were de-identified and provided by UCLA Department of Pathology (Institutional Review Board no. 11-003335). The high-resolution test target (fig. S2) was purchased from Ready Optics (item no. 2012B).

SUPPLEMENTARY MATERIALS

www.sciencetranslationalmedicine.org/cgi/content/full/6/267/267ra175/DC1

Methods

Fig. S1. Block diagrams of multi-height phase retrieval and wave propagation algorithms.

Fig. S2. The effect of TIE solution on multi-height phase recovery algorithm convergence and spatial resolution.

Fig. S3. Comparison of lens-free reconstruction results using TIE that is solved by an FFT-based solver and an elliptic equation solver.

Fig. S4. Computational tilt correction in multi-height phase recovery algorithm using a one-axis translation stage.

Fig. S5. Schematic diagram of autofocus algorithm.

Movie S1. Demonstration of lens-free 3D focusing capability.

Movie S2. Evolution of the reconstructed lens-free amplitude images as a function of the number of iterations.

References (4248)

REFERENCES AND NOTES

  1. Acknowledgments: We acknowledge P. Sullivan of UCLA Department of Pathology and Laboratory Medicine for her support with clinical examination of our images as well as Y. Qian and Y. L. Chan of UCLA for their technical assistance. Funding: The Presidential Early Career Award for Scientists and Engineers, Army Research Office (W911NF-13-1-0419), and Office of Naval Research. A.G. is a Howard Hughes Medical Institute International Student Research Fellow at UCLA. Author contributions: Designed and conceived the project: A.G. and A.O. Mechanical setup design: P.-L.C. Setup automation: A.F. and W.L. Performed the experiments: A.G. and Y.Z. Data processing and code development: A.G. and Y.Z. Blinded pathology evaluation: S.R.K. Wrote the manuscript: A.G., Y.Z., and A.O. Supervised the project: A.O. Competing interests: A.G., Y.Z., A.F., and A.O. are co-inventors of a UCLA patent application on multi-height lens-free on-chip imaging. A.O. is a cofounder of a company that aims to commercialize computational imaging techniques. Data and materials availability: Lens-free holograms and image data can be made available under a material transfer agreement.
View Abstract

Navigate This Article