An improved data-mining technique combines information from multiple imaging modalities and other clinical parameters to predict the risk of cancer metastasis. In the new approach, imaging data from cancer patients are fed into a deep neural network whose output is processed by three different classification algorithms. A novel way of fusing the results from the three classifiers then yields a prediction of whether, by the time of a follow-up consultation months or years later, the cancers will have spread. The US-based researchers who developed the method demonstrated its effectiveness using diagnostic and treatment-planning images acquired for 188 patients with head-and-neck cancer.

As with cancers elsewhere in the body, early-stage cancers of the head and neck are treated using radiotherapy, with increasing success. When treatment fails, it is often down to the growth of new tumours far from the site of the initial disease. Predicting which patients are most likely to develop distant metastasis (DM) is vital, so that low-risk patients can be spared the severe side effects that accompany the systemic treatments used to control cancer proliferation.

One method of categorizing patients in this way is by extracting DM risk indicators from large imaging datasets — an approach known as “radiomics.” But while this technique has been shown to make good use of quantitative features found in single-modality datasets (see CT-based radiomics reveals prostate cancer risk), multi-modality datasets have not yet been used to their best advantage, due to the relatively simple way that disparate features from each image type are combined.

Now, writing in Physics in Medicine and Biology, Zhiguo Zhou at UT Southwestern Medical Center  and University of Central Missouri and colleagues describe a technique – which they call multifaceted radiomics (M-radiomics) – that makes more effective use of a dataset comprising positron-emission tomography (PET) and X-ray CT images.

The team compiled PET and CT images of 188 head-and-neck cancer patients who had follow-up consultations between six and 112 months after scanning. The images, which were acquired at various institutions, had already been studied by physicians, and 257 features – such as textural and geometrical characteristics – had been extracted for each patient. Also included in the dataset were other clinical parameters such as patient age and gender, and the extent to which the disease had already progressed at the time of imaging.

Zhou and colleagues inputted a subset of these data points into a deep neural network, which fused them into a single feature set. They then used this combined feature set to train a predictive model that could be tuned to optimize both sensitivity (the likelihood of making a true positive prediction of DM) and specificity (the likelihood of making a true false prediction of DM) simultaneously depending on clinical need. Usually, the outputs of this model would be sorted by a single classification algorithm to predict outcome: DM or no DM. The researchers note that combining three different classification algorithms improves the reliability of their approach.

Tested against a separate subset of patient data, the researchers found that their M-radiomics approach outperformed versions of the technique that lacked either the deep neural network, sensitivity–specificity optimization, or classifier-combination steps. They think that the method could be improved further by including image features outside of the physician-delineated tumour boundaries, and by developing a standardized, automatic feature-extraction procedure to minimize variability between images captured at different institutions.

Zhou and colleagues plan to validate their method with a multi-institutional prospective study – in contrast to the retrospective dataset that they used for their demonstration – which should show its value for aiding clinical decision making in similar high-risk head-and-neck cancer patients.

“Once validated, we hope clinical adoption of the model can be realized in two to three years,” says Zhou’s co-author, Jing Wang, at UT Southwestern Medical Center. As M-radiomics is a generalized framework, the approach could also be extended to predict treatment outcomes for primary cancers in other anatomical sites.

A collaborative team from Colorado State University and University of Illinois at Urbana-Champaign has developed a new type of nonlinear microscopy, called harmonic optical tomography (HOT), that can produce 3D images of biological tissues in minutes.

Nonlinear microscopy techniques have several benefits over traditional microscopy, as they can image deeper into a sample and are able to visualize many biological features without dyes or markers. One variety of nonlinear microscopy – second-harmonic generation microscopy – is particularly good at imaging molecules that are organized into filaments, such as collagen or muscle fibres.

The team, which specializes in producing holographic images of biological samples, combined this expertise with second-harmonic generation microscopy to produce the HOT microscope.

Making a microscope

The researchers began by developing theoretical models describing how to image the sample. While creating these models, they found that using out-of-focus laser light would actually provide a unique capability for 3D imaging.

They then constructed a microscope that could use these models to image samples. Crucial for development of this microscope was a custom-built high-power laser, which was then integrated into a custom holographic microscope. To achieve the specific illumination conditions to form the required second-harmonic generation signal, the researchers used unfocused light. This also allowed a larger field of the sample to be illuminated at any one time.

The team tested the microscope on myosin fibres in skeletal muscle samples. The wide-field second-harmonic generation microscopy alone was insufficient to produce high-quality data, as the images were corrupted by out-of-focus light and appeared blurry. The HOT microscope, however, uses computational algorithms to reconstruct the image to remove the stray light and produced clear artefact-free images. The HOT reconstruction revealed the distinctive fibre structure of the myosin.

Quicker and better images

Typically, producing these types of 3D images requires a laser scanning technique that sweeps through a sample pixel-by-pixel to produce a 2D image. These images are then stacked up to produce a 3D image. The HOT microscope still collects 2D images but is not restricted to pixel-by-pixel scanning. This means that HOT is considerably faster than traditional methods. As well as saving time, this makes the technique less vulnerable to unwanted vibrations and natural drift of microscope focus.

This type of microscopy can be used to image a variety of biological features and is sensitive to orientation of fibres within the sample. This could be particularly useful to image collagen in tumours, for example, where the orientation of the fibres can indicate disease prognosis. “Most investigators look at it in 2D and not 3D,” says senior author Gabriel Popescu. “Using this technique, we can use the orientation of the collagen fibres as a reporter of how aggressive the cancer is.”

This work is describe in Nature Photonics.