View source for CNN for Enhanced Glaucoma Detection By Avijit Singh

By Avijit Singh

A combined convolutional neural network and a recurrent neural network was developed that not only extracts the spatial features in a fundus image but also the temporal features embedded in a fundus video. A fundus camera or a retinal camera is a specialized low power microscope with an attached camera that is designed to photograph the interior surface of the eye: the retina, retinal vascularity, optic disk, macula, and the posterior pole. The fundus is the combination of the retina and the posterior pole all together. Fundus imaging is the process where the reflected light reflected is used to form a two-dimensional representation of the three-dimensional retina or the fundus (By changing dimensionality, this is similar to what we did in the Neural Network MatLab project).


== What is Glaucoma? ==

Glaucoma is a leading cause of blindness worldwide. It is an eye disease that results in damage to the optic nerve and vision loss because of the buildup of intraocular pressure (IOP). The most common type of glaucoma is open-angle glaucoma where the drainage angle for fluid in the eye remains open. Open-angle glaucoma develops slowly over time and there is no pain associated with it, but peripheral vision may begin to decrease, followed by central vision and eventually blindness if left untreated. Closed-angle glaucoma is another less common type of glaucoma. Closed-angle glaucoma can present itself gradually or suddenly. If sudden, there may be severe eye pain, blurred vision, redness of the eye, and nausea. Vision loss due to glaucoma is permanent so the need to detect glaucoma early is of utmost importance.


== Risk and Treatments ==

Glaucoma’s risk increases with age, high pressure in the eye, and with a family history of glaucoma. A normal eye pressure falls between 10 and 21 mm Hg with the average eye pressure being 15 mm Hg. Someone with glaucoma may have an eye pressure of above 21 mm Hg. If glaucoma is detected and treated early, it is possible to slow the progression of the disease with medication, laser treatment, or even surgery. In order to treat glaucoma, the pressure of the eye has to decrease.


There are about 80 million people worldwide with glaucoma and about 3 million Americans are affected by glaucoma. Glaucoma is the second leading cause of blindness behind cataracts. With a Convolutional Neural Network that is able to detect glaucoma early, many people could have their vision saved since vision loss caused by glaucoma is permanent. With a CNN that is widely used, images of a patient’s fundus could be analyzed effectively and detect glaucoma early which would lead to early treatment.

== The Convolutional Neural Network ==
A Convolutional Neural Network (CNN) is a Deep Learning algorithm which can take in an input image, assign importance to various aspects/objects in the image, and be able to differentiate between one from the other. CNNs are a specialized type of artificial neural networks that use convolution in place of general matrix multiplication in at least one of their layers. Convolution is a mathematical tool for combining two signals to form a third signal which is very important in Digital Signal Processing. Digital Signal Processing takes real world signals like images, sounds, and videos that have been digitized. Once they have been digitized Digital Signal Processors can mathematically manipulate them.

The Convolutional Neural Network used Fundus photographs to provide a 30-degree image of the optic nerve head, which is the structure in the posterior of the fundus that allows the exit of retinal cell axons and the entry and exit of blood vessels. The 30-degree image provides data for the network to quantify a number of glaucoma specific morphological features like neuroretinal rim loss.


[[File: Glaucoma Progression.png|350px|Image: 350 pixels]]

But because the image may not take into account moving blood, fundus videos provide the network to analyze the videos in search of markers associated with blood flow. The Computer Aided Diagnosis systems played a role in making sure the CNN was accurate, reliable, and a fast diagnosis of glaucoma. The CNNs can facilitate autonomous classification based on features from thousands of fundus images. The specificity and sensitivity of these models range between 85 to 95%.

The CNN extracted spatial features from retinal images. Two different deep learning architectures were in the study: VGG16 and ResNet50. They have been widely used for medical image classification tasks. They both have similar architecture and consist of convolutional layers, pooling layers, and a final fully connected layer that produces a label for the input image.

== VGG16 == 

A VGG16 is a convolutional neural network that is 16 layers deep. There are in total about 138 million different parameters. You can load a pre-trained version of the network trained on more than a million images from the ImageNet database. The pre-trained network can classify images into 1000 object categories and is especially useful in medical classifications. It can be used for disease diagnosis using medical imaging like the x-ray or MRI.

[[File: VGG16.jpg]]

== ResNet50 ==

The ResNet50 is a CNN that is 50 layers deep and you can load a pre-trained version of the network trained on more than a million images from the ImageNet database. The pre-trained network can classify images into 1000 object categories.

[[File: ResNet50.png]]


== Recurrent Neural Networks ==

The Recurrent Neural Networks (RNN) extracted temporal features from retinal videos like the fundus videos. They are specifically designed to identify patterns in sequences of data and images. As RNNs run the same task for every element of a sequence, they are called recurrent. The output is dependent on previous computations. The most popular RNN is Long Short Term Memory (LSTM) which was proposed by Hochreiter and Schmidhuber. LSTM was made because networks learning to store information over extended periods of time with recurrent back propagation takes a very long time, most because of decaying error backflow meaning not much learning is taking place after a while. LSTM can bridge the time lags leading to more successful runs and for the network to learn much faster. LSTM solves complex tasks that haven’t been solved by over network algorithms!

== Combining CNN and RNN ==

The CNNs of VGG16 and ResNet50 were combined with the RNN LSTM to extract spatial and temporal features. Each fundus video is converted into images which is then passed onto the CNN to extract spatial information. Those outputs are then put into the LSTM to identify temporal features within the images.

[[File: Combination.png|300px|Image: 300 pixels]]

== Network Training ==

Training was done with transfer learning. Transfer learning is done to initialize the model weights based on large image datasets. To train the models, the datasets were randomly divided into three subsets: training, testing, and validation. Also, the Local Interpretable Model-Agnostic Explanations (LIME) algorithm was used. LIME helps to visualize sections of the fundus image that the model is using to make a final prediction.

== Results ==

By combining VGG16 and LSTM, the highest performance was achieved. The combined networks were able to correctly identify eyes with Glaucoma and healthy eyes. The networks were able to visualize the differences! As more Epochs were run, the accuracy of identification greatly increased.

[[File: EpochsofGlaucoma.png|300px|Image:300 pixels]]


These different networks have been developed which can correctly identify fundus images but they are not yet used in clinical settings because of false positive rates and low precision but with the combined network, that may change.

By combining both CNNs and RNNs, this was the first time that proved that both these networks combined could extract both spatial and temporal features. And by extracting both these features, this significantly improved the detection of glaucoma. This was the first solid evidence that by using video and image classification a highly accurate AI-based approach for glaucoma can be developed.

== Limitations ==

In this particular network, the images were from the same race. Because of this, there may be different results when the network is used on other races. The network will have to be trained with all types of demographics, Height and weight will both play roles in the fundus images because cholesterol rates can alter what arteries or veins may look like.

== References ==
References
“A Beginner’s Guide to Digital Signal Processing (DSP) | Design Center | Analog Devices.” Accessed November 18, 2022. https://www.analog.com/en/design-center/landing-pages/001/beginners-guide-to-dsp.html.
“A Combined Convolutional and Recurrent Neural Network for Enhanced Glaucoma Detection | Scientific Reports.” Accessed November 18, 2022. https://www.nature.com/articles/s41598-021-81554-4#ref-CR20.
Brownlee, Jason. “A Gentle Introduction to Transfer Learning for Deep Learning.” MachineLearningMastery.Com (blog), December 19, 2017. https://machinelearningmastery.com/transfer-learning-for-deep-learning/.
American Academy of Ophthalmology. “Fundus,” January 14, 2020. https://www.aao.org/eye-health/anatomy/fundus.
Hochreiter, Sepp, and Jürgen Schmidhuber. “Long Short-Term Memory.” Neural Computation 9, no. 8 (November 15, 1997): 1735–80. https://doi.org/10.1162/neco.1997.9.8.1735.
Khazaeni, Babak, and Leila Khazaeni. “Acute Closed Angle Glaucoma.” In StatPearls. Treasure Island (FL): StatPearls Publishing, 2022. http://www.ncbi.nlm.nih.gov/books/NBK430857/.
Kwon, Young H., John H. Fingert, Markus H. Kuehn, and Wallace L.M. Alward. “Primary Open-Angle Glaucoma.” The New England Journal of Medicine 360, no. 11 (March 12, 2009): 1113–24. https://doi.org/10.1056/NEJMra0804630.
“Lungs Disease Prediction Using Medical Imaging with Implementation of VGG, Resnet and Convolutional Neural Network | by Nitish Raj Pathak | Analytics Vidhya | Medium.” Accessed November 18, 2022. https://medium.com/analytics-vidhya/lungs-disease-prediction-using-medical-imaging-with-implementation-of-vgg-resnet-and-183e73b85df9.
Machiele, Ryan, Mahsaw Motlagh, and Bhupendra C. Patel. “Intraocular Pressure.” In StatPearls. Treasure Island (FL): StatPearls Publishing, 2022. http://www.ncbi.nlm.nih.gov/books/NBK532237/.
MD, Robert N. Weinreb. “Glaucoma Worldwide: A Growing Concern | Glaucoma.Org,” March 23, 2018. https://glaucoma.org/glaucoma-worldwide-a-growing-concern/.
Saha, Sumit. “A Comprehensive Guide to Convolutional Neural Networks — the ELI5 Way.” Medium, November 16, 2022. https://towardsdatascience.com/a-comprehensive-guide-to-convolutional-neural-networks-the-eli5-way-3bd2b1164a53.
“VGG-16 Convolutional Neural Network - MATLAB Vgg16.” Accessed November 18, 2022. https://www.mathworks.com/help/deeplearning/ref/vgg16.html.
American Academy of Ophthalmology. “What Is Glaucoma? Symptoms, Causes, Diagnosis, Treatment,” October 28, 2022. https://www.aao.org/eye-health/diseases/what-is-glaucoma.