Performance Evaluation of Naive Bayes and SVM in Classifying Public Opinion toward Game-Based Learning Policy

Septi Dwi  Supriati; Hanifah  Permatasari; Vihi  Atina

doi:10.32664/smatika.v16i02.2319

Authors

Septi Dwi Supriati Universitas Duta Bangsa
Hanifah Permatasari Universitas Duta Bangsa
Vihi Atina Universitas Duta Bangsa

DOI:

https://doi.org/10.32664/smatika.v16i02.2319

Keywords:

Edublox, Naïve Bayes, Roblox, SVM, TF-IDF

Abstract

This study aims to analyze public sentiment toward the EDUBLOX program on Instagram comments using the Naive Bayes and Support Vector Machine (SVM) algorithms. The research process consisted of data collection, text preprocessing, manual sentiment labeling, feature extraction using TF-IDF, model training, and performance evaluation. The labeling process was conducted manually by two independent annotators based on predefined sentiment guidelines, and annotation reliability was evaluated using Cohen’s Kappa coefficient. The obtained Cohen’s Kappa value was (\kappa = 0.7907), indicating substantial agreement and good consistency between annotators. The dataset was divided into training and testing data using an 80:20 ratio. The evaluation process used confusion matrix metrics, including accuracy, precision, recall, and F1-score, while the McNemar test was applied to determine whether the performance difference between the two models was statistically significant. The results showed that the SVM model achieved a testing accuracy of 76.67%, marginally outperforming the Naive Bayes model with a testing accuracy of 75.29%. In addition, SVM demonstrated slightly better precision, recall, and F1-score values compared to Naive Bayes. However, the McNemar test produced a p-value of 0.1366, indicating that the performance difference between the two algorithms was not statistically significant. Therefore, both models can be considered to have relatively comparable classification capabilities, although SVM showed a slight numerical advantage in sentiment classification performance on Instagram comments related to the EDUBLOX program.

References

[1] K. Imroatul, N. N. Alfarah, N. Oktaviani, V. N. Zumzumi, D. A. Zahra, and T. Rohman, “Game Based Learning sebagai Pendekatan Deep Learning dalam Pembelajaran PAI & BP di SMP N 2 Bojong Game Based Learning as a Deep Learning Approach in PAI & BP Learning at,” vol. 1, no. 1, pp. 22–29, 2025.

[2] A. B. Setiawan, I. Aknuranda, and A. R. Perdanakusuma, “Alasan Penurunan Popularitas dari Video Game Free Fire: Analisis Kasus dari Perspektif Pemain Game,” vol. 9, no. 3, pp. 1–12, 2025.

[3] K. J. Dermawan, N. A. Sabrina, G. F. Michelle, and A. T. Ayunda, “Sentiment Analysis of Roblox Genres on TikTok: Roleplay, RPG, and Horror using Naïve Bayes,” vol. 5, no. 1, pp. 425–438, 2026, doi: 10.51903/pkbfwq48.

[4] A. Bakir, S. Sulaiman, S. Abdullah, and S. Garfan, “Enhancing Sentiment Classification Accuracy Through Pre-Processing In Educational Text Messenger Data,” vol. 8, no. 2, 2025, doi: 10.32877/bt.v8i2.3377.

[5] L. Rhomaningtias, A. Khairunisa, S. S. M. Wara, and K. M. Hindrayani, “Analisis Sentimen Ulasan Aplikasi Smile Indonesia Menggunakan Metode Naive Bayes dan Support Vector Machine (SVM),” HOAQ (High Educ. Organ. Arch. Qual. J. Teknol. Informasi), vol. 16, no. 1, pp. 79–91, 2025, doi: 10.52972/hoaq.vol16no1.p79-91.

[6] Y. A. Wahid and D. D. Andayani, “Performance Comparison of SVM and Naïve Bayes for Indonesian-Language Sentiment Analysis on Free Fire Reviews Using TF-IDF and SMOTE,” Intell. Syst., vol. 6, no. 4, pp. 674–689, 2025, doi: 10.59562/jessi.v6i4.10818.

[7] N. Nurhikmah, R. S, and N. Nurdin, “Literature Review: Media Game Edukasi Interaktif dalam Pembelajaran Matematika,” J. Educ. Res., vol. 5, no. 4, pp. 4382–4390, 2024, doi: 10.37985/jer.v5i4.1573.

[8] B. Setiadi, E. Purwanto, and H. Permatasari, “Optimisasi Klasifikasi Sentimen Pada Review Hotel Bahasa Inggris Dengan Model Roberta Twitter,” vol. 7, no. 2, pp. 70–79, 2024, doi: 10.31598/sintechjournal.v7i2.1547.

[9] N. Anisa, “Penerapan Lexicon Based Untuk Analisis Sentimen pada Game PUBG dengan Ekstraksi Fitur TF-IDF,” J. Pract. Comput. Sci., vol. 5, no. 1, pp. 35–41, 2025, doi: 10.37366/jpcs.v5i1.5826.

[10] M. H. H. Aly, “Klasifikasi Diabetes Menggunakan Algoritma Support Vector Machine Radial Basis Function,” J. Tek. Inform. dan Teknol. Inf., vol. 4, no. 1, pp. 28–38, 2024, doi: 10.55606/jutiti.v4i1.3420.

[11] A. I. Tanggraeni and M. N. N. Sitokdana, “Analisis Sentimen Aplikasi E-Government pada Google Play Menggunakan Algoritma Naïve Bayes,” JATISI (Jurnal Tek. Inform. dan Sist. Informasi), vol. 9, no. 2, pp. 785–795, 2022, doi: 10.35957/jatisi.v9i2.1835.

[12] E. Martantoh and N. Yanih, “Implementasi Metode Naïve Bayes Untuk Klasifikasi Karakteristik Kepribadian Siswa di Sekolah MTS Darussa’adah Menggunakan PHP MySQL,” J. Teknol. Sist. Inf., vol. 3, no. 2, pp. 166–175, 2022, doi: 10.35957/jtsi.v3i2.2896.

[13] O. Alotaibi and E. Pardede, “Cleaning Big Data Streams: A Systematic Literature Review,” Technologies, pp. 1–24, 2023, doi: 10.3390/technologies11040101.

[14] T. I. Alfawas, A. Rahim, and R. Rudiman, “Penerapan Fitur Ekstraksi TF-IDF untuk Analisis Sentimen Ulasan Game Bus Simulator Indonesia dengan Algoritma Naive Bayes,” Innov. J. Soc. Sci. Res., vol. 4, no. 5, pp. 3177–3193, 2024, doi: 10.31004/innovative.v4i5.13975.

[15] N. S. Fauziah and R. D. Dana, “Implementasi Algoritma Naive Bayes dalam Klasifikasi Status Kesejahteraan Masyarakat Desa Gunungsari,” Blend Sains J. Tek., vol. 1, no. 4, pp. 295–305, 2023, doi: 10.56211/blendsains.v1i4.234.

[16] S. Bumbungan, Kusrini, and Kusnawi, “Penerapan Particle Swarm Optimization (PSO) dalam Pemilihan Parameter Secara Otomatis pada Support Vector Machine (SVM) untuk Prediksi Kelulusan Mahasiswa Politeknik Amamapare Timika,” J. Tek. AMATA, vol. 4, no. 1, pp. 81–93, 2022, doi: 10.55334/jtam.v4i1.77.

[17] A. N. Maulana, W. M. Y. bin Wan Yaacob, and F. Felawati, “Analyzing Public Sentiment on the Proposal to Return Regional Head Elections to DPRD on Platform X Using the C4.5 Algorithm,” J. Inf. Syst. Informatics, vol. 8, no. 2, pp. 1433–1450, 2026.

[18] N. H. Zabni and M. Furqan, “Sentiment Analysis on Teacher Salary Policy in Indonesia 2025 Using Support Vector Machine: A Case Study on Instagram Data,” J. Comput. Networks, Archit. High Perform. Comput., vol. 7, no. 3, 2025.

[19] Mukarramah, “Performance Comparison of Support Vector Machine (SVM) with Linear Kernel and Polynomial Kernel for Multiclass Sentiment Analysis on Twitter,” ILKOM J. Ilm., 2021.

[20] R. L. Mulianingrum and E. Y. Hidayat, “Comparative Performance of SVM and BERT-Base Using Hybrid Preprocessing for Fast Fashion Sentiment Analysis,” J. Appl. Informatics Comput., vol. 9, no. 6, pp. 3464–3478, 2025.

[21] C. Naya and Ermanto, “Analisis Sentimen Publik Terhadap Progres Pembangunan IKN di TikTok Menggunakan Naïve Bayes dan SVM,” Bull. Comput. Sci. Res., vol. 6, no. 2, 2026.

[22] M. Iqbal, “Analysis Of Indonesian People’s Sentiment Towards 2024 Presidential Candidates On Social Media Using Naïve Bayes Classifier and Support Vector Machine,” Build. Informatics, Technol. Sci., 2024.

[23] M. Jamil, Hadiyanto, and R. Sanjaya, “Sentiment Analysis: Classifying Public Comments on YouTube in Disaster Management Simulation in Indonesia Using Naïve Bayes and Support Vector Machine,” Ingenierie des Systemes d’Information, vol. 29, no. 2, pp. 437–446, 2024.