Transfer Learning from Non-Medical Images to Medical Images Using Deep Learning Algorithms

Nooshin Osmani, Sorayya Rezayi, Erfan Esmaeeli, Afsaneh Karimi



Introduction: Machine learning, especially deep convolutional neural networks (DCNNs), is a popular method for computerizing medical image analysis. This study aimed to develop DCNN models for histopathology image classification utilizing transfer learning.

Material and Methods: We utilized 16 different pre-trained DCNNs to analyze the histopathology images from the animal diagnostic laboratory (ADL) database. During the image preprocessing stage, we applied two methods. The first method involved subtracting the mean of ImageNet images from all images. The second method involved subtracting the mean of histopathology training images from all images. Next, in the 16 pre-trained networks, feature extraction was done from their final six layers, and the features extracted from each layer were fed separately into the linear and non-linear support vector machine (SVM) for classification.

Results: The results obtained from the ADL database show that the classification rate in lung tissue images is much better than that of the kidney and spleen. For example, the lowest detection rate in non-linear SVM for lung tissue is 14.96%, almost close to the highest accuracy in kidney and spleen tissue. The classification accuracy of the spleen images is better than that of the kidneys, with only a slight difference. In linear SVM on lung images, ResNet101 obtained the most accurate result with a value of 99.56%, followed by ResNet50, ResNet152, VGG_16, and VGG_19. In non-linear SVM on lung tissue images, the ResNet101 network with 99.65% and ResNet50 with 99.21%, followed by ResNet152, VGG_16, and VGG_19 obtained the highest detection rate.

Conclusion: The classification results obtained from different methods on the ADL (including kidney, spleen, and lung histopathology images) database, confirmed the validity of transferring knowledge between non-medical and medical histopathology images. Additionally, it demonstrates the success of combining classifiers trained on deep features. This research obtained higher accuracy in the ADL database than the works done.


Machine Learning; Transfer Learning; Support Vector Machine;


Cooper M, Ji Z, Krishnan RG. Machine learning in computational histopathology: Challenges and opportunities. Genes Chromosomes Cancer. 2023; 62(9): 540-56. PMID: 37314068 DOI: 10.1002/gcc.23177

Chitra B, Kumar S. Recent advancement in cervical cancer diagnosis for automated screening: A detailed review. Journal of Ambient Intelligence and Humanized Computing. 2022; 13: 251–69.

Robbins P, Pinder S, De Klerk N, Dawkins H, Harvey J, Sterrett G, et al. Histological grading of breast carcinomas: A study of interobserver agreement. Hum Pathol. 1995; 26(8): 873-9. PMID: 7635449 DOI: 10.1016/0046-8177(95)90010-1

Ho C, Zhao Z, Chen XF, Sauer J, Saraf SA, Jialdasani R, et al. A promising deep learning-assistive algorithm for histopathological screening of colorectal cancer. Sci Rep. 2022; 12(1): 2222. PMID: 35140318 DOI: 10.1038/s41598-022-06264-x

Rodenburg B. Deep learning in histopathology research paper business analytics. Semantic Scholar; 2016.

LeCun Y, Bengio Y, Hinton G. Deep learning. Nature. 2015; 521(7553): 436-44. PMID: 26017442 DOI: 10.1038/nature14539

Gulshan V, Peng L, Coram M, Stumpe MC, Wu D, Narayanaswamy A, et al. Development and validation of a deep learning algorithm for detection of diabetic retinopathy in retinal fundus photographs. JAMA. 2016; 316(22): 2402-10. PMID: 27898976 DOI: 10.1001/jama.2016.17216

Ehteshami Bejnordi B, Veta M, Van Diest PJ, Van Ginneken B, Karssemeijer N, Litjens G, et al. Diagnostic assessment of deep learning algorithms for detection of lymph node metastases in women with breast cancer. JAMA. 2017; 318(22): 2199-210. PMID: 29234806 DOI: 10.1001/jama.2017.14585

Liu Y, Kohlberger T, Norouzi M, Dahl GE, Smith JL, Mohtashamian A, et al. Artificial intelligence–based breast cancer nodal metastasis detection: Insights into the black box for pathologists. Arch Pathol Lab Med. 2019; 143(7): 859-68. PMID: 30295070 DOI: 10.5858/arpa.2018-0147-OA

Nagpal K, Foote D, Liu Y, Chen P-HC, Wulczyn E, Tan F, et al. Development and validation of a deep learning algorithm for improving Gleason scoring of prostate cancer. NPJ Digit Med. 2019; 2: 48. PMID: 31304394 DOI: 10.1038/s41746-019-0112-2

Krizhevsky A, Sutskever I, Hinton GE. Imagenet classification with deep convolutional neural networks. Communications of the ACM. 2017; 60(6): 84-90.

Yildirim O, San Tan R, Acharya UR. An efficient compression of ECG signals using deep convolutional autoencoders. Cognitive Systems Research. 2018; 52: 198-211.

Yildirim Ö, Pławiak P, Tan R-S, Acharya UR. Arrhythmia detection using deep convolutional neural network with long duration ECG signals. Comput Biol Med. 2018; 102: 411-20. PMID: 30245122 DOI: 10.1016/j.compbiomed.2018.09.009

Oh SL, Ng EY, San Tan R, Acharya UR. Automated beat-wise arrhythmia diagnosis using modified U-net on extended electrocardiographic recordings with heterogeneous arrhythmia types. Comput Biol Med. 2019; 105: 92-101. PMID: 30599317 DOI: 10.1016/j.compbiomed.2018.12.012

Książek W, Abdar M, Acharya UR, Pławiak P. A novel machine learning approach for early detection of hepatocellular carcinoma patients. Cognitive Systems Research. 2019; 54: 116-27.

Yildirim O, Talo M, Ay B, Baloglu UB, Aydin G, Acharya UR. Automated detection of diabetic subject using pre-trained 2D-CNN models with frequency spectrum images extracted from heart rate signals. Comput Biol Med. 2019; 113: 103387. PMID: 31421276 DOI: 10.1016/j.compbiomed.2019.103387

Talo M, Yildirim O, Baloglu UB, Aydin G, Acharya UR. Convolutional neural networks for multi-class brain disease detection using MRI images. Comput Med Imaging Graph. 2019; 78: 101673. PMID: 31635910 DOI: 10.1016/j.compmedimag.2019.101673

Baloglu UB, Talo M, Yildirim O, San Tan R, Acharya UR. Classification of myocardial infarction with multi-lead ECG signals and deep CNN. Pattern Recognition Letters. 2019; 122: 23-30.

Safdari R, Kazemi Arpanahi H, Langarizadeh M, Ghazisaiedi M, Dargahi H, Zendehdel K. Design a fuzzy rule-based expert system to aid earlier diagnosis of gastric cancer. Acta Inform Med. 2018; 26(1): 19-23. PMID: 29719308 DOI: 10.5455/aim.2018.26.19-23

Khazaee Z, Langarizadeh M, Shiri Ahmadabadi ME. Developing an artificial intelligence model for tumor grading and classification, based on MRI sequences of human brain gliomas. International Journal of Cancer Management. 2022; 15(1): e120638.

Saraei M, Liu S. Attention-based deep learning approaches in brain tumor image analysis: A mini review. Frontiers in Health Informatics. 2023; 12: 164.

Silveira EC, Corrêa CFS. Recognition of epileptic seizures in EEG records: A transfer learning approach. Frontiers in Health Informatics. 2021; 10: 61.

Saeedi S, Maghooli K, Amirazodi S, Rezayi S. Towards a better diagnosis of prostate cancer: Application of machine learning algorithms. Frontiers in Health Informatics. 2022; 11: 119.

Talo M, Baloglu UB, Yildirim Ö, Acharya UR. Application of deep transfer learning for automated brain abnormality classification using MR images. Cognitive Systems Research. 2019; 54: 176-88.

Yan R, Ren F, Wang Z, Wang L, Zhang T, Liu Y, et al. Breast cancer histopathological image classification using a hybrid deep neural network. Methods. 2020; 173: 52-60. PMID: 31212016 DOI: 10.1016/j.ymeth.2019.06.014

Pashaei A, Sajedi H, Jazayeri N. Brain tumor classification via convolutional neural network and extreme learning machines. International Conference on Computer and Knowledge Engineering. IEEE; 2018.

Han Z, Wei B, Zheng Y, Yin Y, Li K, Li S. Breast cancer multi-classification from histopathological images with structured deep learning model. Sci Rep. 2017; 7(1): 4172. PMID: 28646155 DOI: 10.1038/s41598-017-04075-z

Phaye SSR, Sikka A, Dhall A, Bathula D. Dense and diverse capsule networks: Making the capsules learn better. arXiv Preprint; 2018.

Saha M, Chakraborty C, Racoceanu D. Efficient deep learning model for mitosis detection using breast histopathology images. Comput Med Imaging Graph. 2018; 64: 29-40. PMID: 29409716 DOI: 10.1016/j.compmedimag.2017.12.001

Deng J, Dong W, Socher R, Li L-J, Li K, Fei-Fei L. ImageNet: A large-scale hierarchical image database. IEEE Conference on Computer Vision and Pattern Recognition. IEEE; 2009.

Fang J, Xu X, Liu H, Sun F. Local receptive field based extreme learning machine with three channels for histopathological image classification. International Journal of Machine Learning and Cybernetics. 2019; 10(6): 1437-47.

He K, Zhang X, Ren S, Sun J. Deep residual learning for image recognition. IEEE Conference on Computer Vision and Pattern Recognition. IEEE; 2016.

Szegedy C, Liu W, Jia Y, Sermanet P, Reed S, Anguelov D, et al. Going deeper with convolutions. IEEE Conference on Computer Vision and Pattern Recognition. IEEE; 2015.

Deng J, Berg A, Satheesh S, Su H, Khosla A, Fei-Fei L. ImageNet large scale visual recognition challenge [Internet]. 2012 [cited: 22 Agu 2023]. Available from:

Chatfield K, Simonyan K, Vedaldi A, Zisserman A. Return of the devil in the details: Delving deep into convolutional nets. arXiv Preprint; 2014.

Clement D, Agu E, Suleiman MA, Obayemi J, Adeshina S, Soboyejo W. Multi-class breast cancer histopathological image classification using multi-scale pooled image feature representation (MPIFR) and one-versus-one support vector machines. Applied Sciences. 2023; 13(1): 156.

Sahu Y, Tripathi A, Gupta RK, Gautam P, Pateriya R, Gupta A. A CNN-SVM based computer aided diagnosis of breast cancer using histogram K-means segmentation technique. Multimedia Tools and Applications. 2023; 82(9): 14055-75.

Al-Haija QA, Adebanjo A. Breast cancer diagnosis in histopathological images using ResNet-50 convolutional neural network. IEEE International IOT, Electronics and Mechatronics Conference. IEEE; 2020.

Dabeer S, Khan MM, Islam S. Cancer diagnosis in histopathological image: CNN based approach. Informatics in Medicine Unlocked. 2019; 16: 100231.

Mahmud MI, Mamun M, Abdelgawad A. A deep analysis of transfer learning based breast cancer detection using histopathology images. International Conference on Signal Processing and Integrated Networks. IEEE; 2023.

Pal R, Saraswat M. Enhanced bag of features using alexnet and improved biogeography-based optimization for histopathological image analysis. International Conference on Contemporary Computing. IEEE; 2018.



  • There are currently no refbacks.