Performance Comparison of Support Vector Machine and Decision Trees for the Classification of Handwritten Digits

Handwritten digit recognition remains one of the fundamental applications of machine learning and pattern recognition due to its widespread use in banking, postal services, document processing and intelligent information systems. Although deep learning approaches have demonstrated remarkable performance, classical machine learning algorithms such as Support Vector Machine (SVM) and Decision Tree (DT) remain attractive because of their simplicity, computational efficiency and interpretability. This study evaluates and compares the performance of Support Vector Machine and Decision Tree classifiers for handwritten digit recognition using the Modified National Institute of Standards and Technology (MNIST) dataset. The dataset comprises 70,000 grayscale images of handwritten digits (0–9), each with a resolution of 28 × 28 pixels. Image preprocessing involved normalization of pixel values and standard data preparation before model implementation. Both classifiers were implemented using Python and the Scikit-learn library under identical experimental conditions, and model performance was evaluated using accuracy, precision, recall and F1-score. Experimental results showed that the Support Vector Machine outperformed the Decision Tree across all evaluation metrics. The SVM achieved an accuracy of 94.32%, precision of 94.41%, recall of 94.32%, and an F1-score of 94.35%, whereas the Decision Tree recorded 87.36% accuracy, 87.42% precision, 87.36% recall and 87.35% F1-score. The findings indicate that SVM provides superior classification performance for handwritten digit recognition, while Decision Tree offers faster implementation and greater interpretability. The study concludes that SVM is more suitable for applications requiring high recognition accuracy, whereas Decision Tree remains appropriate for applications where computational simplicity and model transparency are prioritized

Handwritten Digit Recognition, Support Vector Machine, Decision Tree, MNIST Dataset, Machine Learning, Image Classification

1. Bishop, C. M. (2006). Pattern Recognition and Machine Learning. Springer. [Google Scholar] [Crossref]

2. Breiman, L., Friedman, J. H., Olshen, R. A., & Stone, C. J. (1984). Classification and Regression Trees. Wadsworth International Group. [Google Scholar] [Crossref]

3. Cortes, C., & Vapnik, V. (1995). Support-vector networks. Machine Learning, 20(3), 273–297. [Google Scholar] [Crossref]

4. Decoste, D., & Schölkopf, B. (2002). Training invariant support vector machines. Machine Learning, 46(1–3), 161–190. [Google Scholar] [Crossref]

5. Demšar, J. (2006). Statistical comparisons of classifiers over multiple datasets. Journal of Machine Learning Research, 7, 1–30. [Google Scholar] [Crossref]

6. Duda, R. O., Hart, P. E., & Stork, D. G. (2001). Pattern Classification (2nd ed.). John Wiley & Sons. [Google Scholar] [Crossref]

7. Géron, A. (2022). Hands-On Machine Learning with Scikit-Learn, Keras and TensorFlow (3rd ed.). O'Reilly Media. [Google Scholar] [Crossref]

8. Goodfellow, I., Bengio, Y., & Courville, A. (2016). Deep Learning. MIT Press. [Google Scholar] [Crossref]

9. Han, J., Kamber, M., & Pei, J. (2012). Data Mining: Concepts and Techniques (3rd ed.). Morgan Kaufmann. [Google Scholar] [Crossref]

10. Kotsiantis, S. B. (2013). Decision trees: A recent overview. Artificial Intelligence Review, 39(4), 261–283. [Google Scholar] [Crossref]

11. LeCun, Y., Bottou, L., Bengio, Y., & Haffner, P. (1998). Gradient-based learning applied to document recognition. Proceedings of the IEEE, 86(11), 2278–2324. [Google Scholar] [Crossref]

12. LeCun, Y., Bengio, Y., & Hinton, G. (2015). Deep learning. Nature, 521(7553), 436–444. [Google Scholar] [Crossref]

13. Mitchell, T. M. (1997). Machine Learning. McGraw-Hill. [Google Scholar] [Crossref]

14. Moronkeji, I. O., Olabiyisi, S. O., & Adedeji, O. T. (2026). A comparative study of selected chaos maps to improve particle swarm optimisation in image segmentation tasks. EIRA Journal of Multidisciplinary Research and Development, 2(4). 17-26. [Google Scholar] [Crossref]

15. Murphy, K. P. (2022). Probabilistic Machine Learning: An Introduction. MIT Press. [Google Scholar] [Crossref]

16. Okunlola, S. O., Baale, A. A., & Olabiyisi, S. O. (2026). Performance Evaluation of Selected Chaos-Enhanced Particle Swarm Optimisation for Image Segmentation. Engineering and Technology Journal, 11(7), 10904–10911. https://doi.org/10.47191/etj/v11i07.13 [Google Scholar] [Crossref]

17. Olatunji, Babatunde Lekan, Olabiyisi, Stephen Olatunde, Oyeleye, Christopher Akinwale and Omotade, Adedotun Lawrence, 2025. "An Enhanced Chicken Swarm Optimization Algorithm Using Gaussian and Tent Chaotic Map Functions," International Journal of Research and Innovation in Applied Science, International Journal of Research and Innovation in Applied Science (IJRIAS), vol. 10(7), pages 653-664. [Google Scholar] [Crossref]

18. Patel, H. R., & Thakore, D. G. (2013). A survey on feature extraction techniques for image classification. International Journal of Computer Applications, 70(9), 1–7. [Google Scholar] [Crossref]

19. Pedregosa, F., Varoquaux, G., Gramfort, A., et al. (2011). Scikit-learn: Machine learning in Python. Journal of Machine Learning Research, 12, 2825–2830. [Google Scholar] [Crossref]

20. Python Software Foundation. (2024). Python Language Reference. https://www.python.org [Google Scholar] [Crossref]

21. Scikit-learn Developers. (2024). Scikit-learn User Guide. https://scikit-learn.org [Google Scholar] [Crossref]

22. Vapnik, V. N. (1998). Statistical Learning Theory. John Wiley & Sons. [Google Scholar] [Crossref]

23. Zhang, C., & Ma, Y. (2012). Ensemble Machine Learning: Methods and Applications. Springer. [Google Scholar] [Crossref]

• Enhancing Formation Control of Multi Agent Systems Using Ann Based Technique
• Improving Sliding Mode Control with Chattering Reduction using Fuzzy Based Technique
• Cooking Quality, Fasting Blood Glucose, Glycemic Index and Load of High–Fiber Noodles Made from Wheat, Tiger Nut Residue and Cassava Flour Blends
• Matrix Rhythm Therapy Versus Interferential Therapy Combined with Lumbar Stabilization Exercises in Chronic Non-Specific Low Back Pain: A Randomized Comparative Trial
• Formulation and Sensory Evaluation of Functional Cake Prepared from Sweet Potato Powder