Retinal Imaging Technologies in Cerebral Malaria
Retinal Photography
Retinal photography allows non-ophthalmologists to acquire and grade fundus images, with well-documented practical applications in the assessment of retinopathy10,11. Furthermore, where a problem is identified or there is uncertainty, the same images can be reviewed by an ophthalmologist or another suitably trained clinician without further inconveniencing the patient. This is particularly pertinent in low-resource environments, where there are relatively few ophthalmologists per capita, and patients often travel considerable distances to access health care. Given the established utility of MR in differentiating CM from coma of other cause, and that the effect of CM is felt primarily in the low- and middle-income countries (LMIC) of the African region, the potential advantages of retinal photography become even more clear3,12.
Clinicians have been aware of fundal changes in the retina of patients with severe malaria since the late 19th century13. Since then, many papers have described the characteristic changes, with several papers in the 80s and 90s systematically examining the retinae of patients with CM and correlating findings with outcome, some with photographic documentation14–17. Example images are presented in Figure 2.
Despite this, we identified only five studies which reported outcomes related to retinal photography. Four reported photographic findings, one of which reported only minimal data on an unselected subgroup from a larger study, and one study sought to compare the effectiveness of different retinal cameras to binocular indirect ophthalmoscopy (BIO). Several studies evaluated artificial intelligence (AI) approaches to identifying MR using retinal photographic images. These will be addressed separately.
The four studies which reported their photographic findings are summarized in Table 1. The incidence of MR is roughly similar to that reported in clinical studies4. Interestingly, no adults had evidence of vessel discolouration, contrasting with the reported incidence of approximately 25% in African children18. It should be noted that vessel changes in MR are often very peripheral, and thus may be difficult to image using fundus photography.
Grading systems for MR have been developed at the University of Liverpool, with high levels of interobserver concordance in the hands of experienced ophthalmologists using BIO5,18,19. While BIO represents the gold standard for the detection of MR, it can be a difficult skill to learn, and grading requires both accurate examination and retention of much visual information while performing the examination. Grading of images from a retinal camera by ophthalmologists had an interobserver concordance of 100% in a study of Bangladeshi adults with severe malaria20. This is likely due to the static nature of camera images and the ability to digitally manipulate them with magnification and colour adjustment tools.
In a further study of Bangladeshi adults with MR, a subgroup was randomly selected to undergo retinal photography and this was graded and used as a control. Concordance between non-ophthalmologist observers was not reported, and the absence of an ophthalmologist’s assessment as the ‘gold standard’ test somewhat limits the utility of the data. However, the study did suggest that retinal photography was a more sensitive tool for identifying retinal whitening than ophthalmoscopy (direct and indirect) by a non-ophthalmologist21.
A comprehensive study of vessel changes and retinal whitening including clinical, photographic, angiographic and pathological data found that orange vessel changes are strongly associated with death22. This study is evaluated in detail later.
Only Soliz et al. directly compared the findings from retinal cameras to those of an ophthalmologist using BIO23. Of the three cameras that they evaluated, the handheld Pictor Plus camera appeared to be superior, with sensitivity and specificity for identifying MR in patients with CM of 100% and 87%, respectively. Of note, all cameras tested failed to adequately image the peripheral retina; of importance given the previously mentioned predilection of vessel changes to occur in the periphery. Though the results are impressive and suggest a role for fundus photography in screening for MR, the study is limited by small numbers. The authors also comment that design shortcomings may limit utility in the field, specifically the absence of real-time feedback on image quality. In resource-limited environments users may be technicians without substantial ophthalmic training and therefore may not recognize that an image is of poor quality, potentially causing a delay to treatment if re-imaging is required.
Fundus Fluorescein Angiography (FA)
Fluorescein is an organic compound that, when stimulated with blue light (λ = 465-490nm) emits a green light of longer wavelength (λ = 520-530nm)24. FA was developed in the 1960s and permits detailed imaging of the retinal vasculature and real-time, in-vivo visualisation of vascular pathology. Fluorescein dye is injected intravenously before serial photographs are taken using a camera with a blue excitation filter and a green emission filter. Injected fluorescein remains entirely intravascular in uncompromised blood vessels but will leak if the BRB is at all compromised. Sequential imaging allows visualisation of the retinal microvasculature in five phases: choroidal flush (pre-arterial), arterial, arteriovenous (laminar venous), venous and recirculation.
The similarities between the retinal and cerebral microcirculation mean that FA is a useful tool for understanding injury to the cerebral microvasculature in CM. Though it is possible to perform FA in low-resource environments with adapted equipment, it is not as straightforward as retinal photography and carries additional risks, such as anaphylaxis. Accordingly, its use as a screening tool is limited but it remains a useful tool to investigate pathophysiology.
We identified six studies which specifically addressed FA in CM. These are summarized in Table 2. Studies which used AI approaches to interpreting FA images in CM are reported separately.
Early studies using film cameras by Davis et al. and Hero et al. used FA in a small number of patients. Both studies identified angiographic abnormalities including capillary non-perfusion (CNP), vessel leak and disc leak8,25. Neither was sufficiently powered to accurately describe the range of changes seen or the prevalence of those changes. In addition, film cameras do not provide real-time feedback, so image quality may be compromised. Subsequent large studies using digital imaging have identified a range of angiographic findings which broadly can be grouped into CNP, leakage and intravascular filling defects (IVFD)26,27. Examples are shown in Figure 3.
CNP is caused when a capillary network fails to fill by the late arteriovenous phase. Images from earlier phases must not be used for grading, as the capillary network may not have had sufficient time to fill, thereby risking false positives. CNP is extremely common in CM, with some degree of macular CNP and peripheral CNP in 100% and 95% of gradable MR-positive cases, respectively, in one study27. It co-localises to areas of retinal whitening, suggesting that whitening arises because of an ischaemic insult.
Fluorescein leak seen in CM can be grouped into one of three types: large focal leak (LFL), punctate leak (PL) and vessel leak (VL). Interestingly, LFL and PL appear to associate with brain swelling and death, whereas vessel leak and CNP appear to associate with neurological sequelae in survivors28. This may suggest that the rapid accumulation of retinal haemorrhages (seen as evolving LFLs on FA) indicates multiple focal breaks in the BRB, which is likely to be mirrored in the BBB and brain. This leakage of blood cells and large proteins is concurrent with a rapid fluid egress, a possible mechanism for brain swelling and death.
VL was histologically associated with perivascular fibrinogen on histopathological assessment in the same study28. It was also frequently related to areas of CNP and together VL and CNP were associated with neurological sequelae. The authors conclude that sequestration of pRBCs results in immunological dysregulation with BRB disruption and patchy ischaemia, resulting in vasogenic and cytotoxic oedema, a pathological process distinct from that seen in LFL. The association with neurological sequelae and not death suggests that, in the brain, this may result in significantly less fluid egress than LFL, thus rendering this BBB compromise more survivable, albeit with neurological injury resulting from ischaemia and reperfusion.
Barrera et al. retrospectively analysed a large, prospectively collected dataset of Malawian children with CM, using clinical, photographic, angiographic and pathological analysis to establish whether sequestration of pRBCs is clinically visible in the retina and whether this correlates with outcome22. IVFDs in the venous circulation were demonstrated in an overwhelming majority of cases (98.3%) on review of the angiographic images. Though much less frequent, there was an increased risk of death when IVFDs affected the arteriolar circulation. In addition, detailed pathological analysis and cross-referencing of angiographic imaging demonstrate that intravascular filling defects are areas of parasite sequestration. Where parasites are sequestered there are marked changes in the retinal neurovascular unit with endothelial cell dysfunction alongside pericyte dysfunction and loss. Changes in pericyte function were paralleled in the brain in an unpublished small sub-analysis by the same authors. These findings add further weight to the hypothesis that BBB breakdown secondary to parasite sequestration leads to neurological injury. The mechanisms underpinning which children will die and which survive remain unclear, but may be related to host-parasite interactions and the host immune response.
Optical Coherence Tomography (OCT)
Optical coherence tomography (OCT) can be used to generate high-resolution cross-sectional images of the retina and optic nerve in-vivo. A beam of near-infrared light is split into a probe beam and reference beam, the former of which is reflected from the tissue which is being analysed and the latter from a mirror. The beams are recombined and the interference pattern is interpreted by an interferometer. Multiple data points over a 2mm distance are integrated, producing an image analogous to an in-vivo histopathological section. A spectral-domain OCT (SD-OCT) is illustrated in Figure 4. OCT facilitates the detection of subtle and sub-clinical macular oedema and other changes to retinal integrity, as well as helping to differentiate causes of optic nerve head swelling. We identified only one paper describing changes on OCT in human CM29. A further study uses the technology in a murine model of CM30.
Tu et al. describe several novel features on OCT (see Figure 4)29. Firstly, they identified hyperreflective areas (HRAs) affecting the inner retinal layers which, when large, colocalise to areas of retinal whitening and CNP on fundus photography and FA, respectively. The outer layers are spared as their blood supply is derived from the choroid, in which there is little to no sequestration. A hyperreflective signal from the inner retinal layers is seen in retinal artery occlusion and gives support to the theory that hyperreflectivity of the inner retina results from cytotoxic oedema secondary to tissue hypoxia in CM.
Cotton wool spots were visible in the retinal nerve fibre layer (RNFL) as raised and well-defined lesions with lower signal density when compared to HRAs29. They were much more common on OCT than have previously been appreciated clinically (37 vs. 5%), suggesting clinical misclassification as retinal whitening18,29. Cotton wool spots are caused by axonal injury in the RNFL, often secondary to ischaemia. This has been demonstrated in the paediatric retina in CM by increased levels of β-amyloid precursor protein in the RNFL during histopathological assessment7.
The presence of hyperreflective vessels and hyperreflective capillaries in the inner retinal layers is unique to CM. This provides another potential example of in-vivo visualisation of sequestered pRBCs29. However, it is not yet known how hyperreflectivity of vessels relates to visible vessel abnormalities and their histopathological correlates.
A further study assessing the retinal microvasculature in a murine model of CM used OCT in addition to confocal scanning laser ophthalmoscopy and histopathological analysis showed colocalization of regions of interest on OCT with parasite sequestration in the blood vessels, supporting the conclusions reached by Tu et al. that vessel hyperreflectivity represents sequestered pRBCs30.
OCT identified cystoid macular oedema in approximately 10% of patients, significantly less than has been observed on histopathology7,29. Patients that die and go on to autopsy are more likely to have severe disease, which could account for this difference.
Finally, longitudinal analysis of OCTs from survivors showed that HRAs develop into areas of retinal thinning by one month, which is more pronounced at one year29. This may be reflected in the brain, where significant brain atrophy is seen in those with neurodevelopmental deficits on magnetic resonance imaging (MRI) at one month31.
These OCT findings have not yet been correlated with clinical outcome. However, OCT shows promise as a bedside tool for the assessment of CM, as well as potentially aiding our understanding of the pathogenesis of the disease.
Automated Detection of Retinal Lesions
The approaches to automated image analysis can be loosely grouped into two major strategies: traditional computer vision and deep learning. Simply, traditional computer vision (TCV) uses algorithms based on functions such as colour thresholding and pixel counting to perform functions such as segmentation and detection of saliency (standing out against the background). It requires significant processing power but requires fewer lines of code than machine learning techniques. TCV can be computationally slow. AI in the form of deep learning is an important and growing area of retinal image analysis32. Traditional image analysis is time-consuming and costly, requiring trained graders to assess images and rigorous quality control processes. AI has the potential to reduce the need for human graders, freeing them up for quality control and assessing images which are equivocal or difficult to grade using algorithms. While the mathematics of AI is beyond the scope of this review, some key concepts are summarised below33.
Deep Learning
The rise of deep learning is based on the development of convolutional neural networks. A model is developed and trained using a bank of images, such that it can begin to recognize intrinsic features within an image without additional expert input and to make decisions based on those features. After training from a training dataset, deep learning models can be applied to a test dataset to further evaluate the models’ performance. Several approaches to deep learning have been trialled in MR detection.
Supervised vs. Unsupervised vs. Semi-Supervised Learning
Supervised learning uses a set of labelled training data to teach a model to recognize features of the same category’s data. A supervised model is usually more accurate than an unsupervised model, but the start-up cost is significant, requiring a large dataset and human intervention for labelling.
Unsupervised learning models cluster and analyse data on their own, to identify patterns within the data and thus summarize the patterns’ characteristics and then recognize them.
The key difference between unsupervised and supervised learning is the use of labelled data. A semi-supervised or weakly-supervised model uses a mixture of labelled and unlabelled data to assist a model benefiting from both labelled and unlabelled information and features.
Transfer Learning
Transfer learning utilizes a model pre-trained with a large amount of data similar to the data the new model will analyse and then fine-tunes it using a small amount of purpose-specific training data. This process significantly reduces the computational cost of training a model34. Transfer learning for detection of MR used a model trained to identify diabetic retinopathy and adapted it with a small MR training dataset34.
Most papers on imaging in MR relate to AI analysis. This is likely because models can be developed on pre-existing datasets, without the need to recruit further patients. Additionally, many of the features of MR on colour fundus imaging and FA are present in other retinopathies, most notably diabetic retinopathy. Accordingly, there are researchers developing algorithms which can detect lesions in both MR and diabetic retinopathy. This generalisability increases the number of researchers interested and capable of tackling the issue, as well as increasing access to funding.
We identified 21 papers which reported findings on the automated detection of retinal lesions on either colour fundus photography or FA. The results of these tables are summarized in Table 3. Metrics used in the evaluation of AI models are summarised in Figure 5. Dice coefficient (DC) is a metric that is frequently applied in image segmentation analysis to compare pixels or regions identified by a model with those identified by one or more experts. Because children without MR should be thoroughly worked-up for alternative causes of coma, most algorithms were optimised for high specificity, rather than sensitivity. In addition to sensitivity and specificity, the receiver operating curve (ROC) is also used, based on which area under curve (AUC) can be computed. AUC tends to be more informative than accuracy when the number of cases is imbalanced between different groups.
Image analysis is not independent of image acquisition and, while the results of AI analysis are extremely impressive, it should be noted that current camera technologies used in MR studies do not capture the periphery sufficiently well to accurately grade MR. Additionally, the algorithms with the best performance were often trained and tested using high-quality images captured with expensive cameras, some of which are table-mounted. Table-mounted cameras are not practical for capturing images in comatose children and the high expense of advanced cameras is a barrier to their use outside of the research environment in LMICs. While further efforts are made to increase the sensitivity and specificity of AI image analysis and reduce analysis time, parallel research should concentrate on the production of a low-cost widefield camera.
Other Technologies
Beare et al. hypothesised that outcome in CM could correlate with changes in optic nerve head blood flow and tested their hypothesis using laser Doppler flowmetry, which uses the principle of the Doppler effect in scattered laser light to quantify movement of erythrocytes35. They demonstrated an increased blood volume in children with papilloedema but did not show any correlation between blood-flow and outcome. The study was limited by the lack of a suitable control group or normative data in children.
We also identified two papers which used less common or novel imaging techniques to evaluate the retinas of mice infected with P. bergei. The mouse model of cerebral malaria is well-characterised and has been used extensively in research into the pathophysiological mechanisms underlying CM36.
Hyperspectral imaging is an adaption of fundus photography that uses computerised analysis of spectral reflections from the retina to quantify blood oxygenation in the vasculature. In murine CM reduced oxygenation was observed with this technique, possibly indicating parasitic haematophagy37. Though this finding may illustrate an important process in malarial pathogenesis its relation to malarial retinopathy remains unclear, and the technique does not appear to have been used in human subjects with malaria yet.
Finally, laser speckle imaging has been used to demonstrate changes to blood flow in mouse retinas using camera-phone technology38. Laser speckle imaging detects blurring in the speckle pattern (noise) created by interference in light emitted from a coherent laser light source. As components of the sample move this interference pattern changes and an image can be generated. The technique is non-contact and non-invasive but does not provide absolute values for blood flow. Sequential imaging would be required to show changes in blood flow which may limit utility in deteriorating CM patients.