Implementation of Naive Bayes Method and Natural Language Processing in Web-Based Online Hoax News Detection System

Mohammad Ainur Rofiqi; Ahmad Heru Mujianto

doi:10.32664/smatika.v16i02.2338

Authors

Mohammad Ainur Rofiqi Universitas Hasyim Asy`ari Tebuireng Jombang, Indonesia
Ahmad Heru Mujianto Universitas Hasyim Asy’ari Tebuireng Jombang, Indonesia

DOI:

https://doi.org/10.32664/smatika.v16i02.2338

Keywords:

Hoax Detection, Naive Bayes, Natural Language Processing, TF-IDF, Web-Based System

Abstract

The rapid growth of digital media in Indonesia has accelerated the spread of hoax news, which poses serious threats to public trust and social stability. Based on data from the Ministry of Communication and Informatics of the Republic of Indonesia, a total of 12,547 hoax contents were identified from 2018 to 2023, while 1,923 new hoax contents emerged in 2024 alone. This research aims to design and build a web-based hoax news detection system by implementing the Multinomial Naive Bayes algorithm combined with Natural Language Processing (NLP) techniques. The system processes text through five NLP stages: sentence splitting, case folding, tokenizing, stopword removal, and stemming using PySastrawi. Feature weighting is performed using TF-IDF (Term Frequency–Inverse Document Frequency), and classification is executed using the Multinomial Naive Bayes algorithm enhanced with Laplace Smoothing and Log Posterior Probability. The output is converted using the Softmax function to produce probability percentages for each sentence. A manual calculation simulation using 10 training sentences and one test narrative containing four sentences was conducted to verify the algorithm. The system successfully classified the test narrative as hoax with a probability of 62.00% hoax and 38.00% non-hoax, consistent with the actual label. The system was evaluated using a Confusion Matrix on a 50-sentence test dataset, achieving Accuracy of 90.00%, Precision of 91.67%, Recall of 88.00%, and F1-Score of 89.80%. The resulting system provides a practical tool for the public to verify the credibility of news information in the digital era.

Author Biography

Ahmad Heru Mujianto, Universitas Hasyim Asy’ari Tebuireng Jombang, Indonesia

Department of Informatics Engineering, Universitas Hasyim Asy’ari Tebuireng Jombang.

References

[1] Asosiasi Penyelenggara Jasa Internet Indonesia, “Jumlah pengguna internet Indonesia tembus 221 juta orang,” 2024.

[2] Kementerian Komunikasi dan Informatika Republik Indonesia, “Siaran Pers No. 02/HM/KOMINFO/01/2024 tentang Hingga Akhir Tahun 2023, Kominfo Tangani 12.547 Isu Hoaks,” 2024.

[3] Kementerian Komunikasi dan Digital Republik Indonesia, “Komdigi Identifikasi 1.923 Konten Hoaks Sepanjang Tahun 2024,” 2025.

[4] H. Allcott and M. Gentzkow, “Social Media and Fake News in the 2016 Election,” J. Econ. Perspect., vol. 31, no. 2, pp. 211–236, 2017, doi: 10.1257/jep.31.2.211.

[5] S. Vosoughi, D. Roy, and S. Aral, “The spread of true and false news online,” Science (80-. )., vol. 359, no. 6380, pp. 1146–1151, 2018, doi: 10.1126/science.aap9559.

[6] C. Pelau, M. I. Pop, M. Stanescu, and G. Sanda, “The Breaking News Effect and Its Impact on the Credibility and Trust in Information Posted on Social Media,” Electronics, vol. 12, no. 2, 2023, doi: 10.3390/electronics12020423.

[7] S. H. Daulay, D. N. Aulia, and N. A. Zahra, “Framing The Lie: A Linguistic Analysis of Viral Fake News Discourse,” BASIS J. Bhs. dan Sastra Ingg., vol. 12, no. 2, pp. 253–264, 2025, doi: 10.33884/basisupb.v12i2.10000.

[8] R. R. Sani, Y. A. Pratiwi, S. Winarno, E. D. Udayanti, and F. Al Zami, “Analisis Perbandingan Algoritma Naive Bayes Classifier dan Support Vector Machine untuk Klasifikasi Hoax pada Berita Online Indonesia,” J. Masy. Inform., vol. 13, no. 2, 2022, doi: 10.14710/jmasif.13.2.47983.

[9] N. E. Febriyanty, M. A. Hariyadi, and C. Crysdian, “Hoax Detection News Using Naive Bayes and Support Vector Machine Algorithm,” Int. J. Adv. Data Inf. Syst., vol. 4, no. 2, pp. 191–200, 2023, doi: 10.25008/ijadis.v4i2.1306.

[10] G. Airlangga, “Comparative Analysis of Machine Learning Algorithms for Detecting Fake News: Efficacy and Accuracy in the Modern Information Ecosystem,” J. Comput. Networks, Archit. High Perform. Comput., vol. 6, no. 1, pp. 354–363, 2024, doi: 10.47709/cnahpc.v6i1.3466.

[11] W. Hidayat, J. Ong, H. Irsyad, and A. Rahman, “Ekstrasi Berita Hoax Pada Turn Back Hoax Berbasis Pendekatan TF-IDF & Cosine Similarity,” J. Ilm. Comput. Insight, vol. 7, no. 2, 2025, doi: 10.30651/comp_insight.v7i2.26678.

[12] M. F. Ramadhan, “Klasifikasi Topik dan Sentimen Judul Berita dengan Augmentasi dan TF-IDF,” RIGGS J. Artif. Intell. Digit. Bus., vol. 4, no. 2, pp. 6732–6741, 2025, doi: 10.31004/riggs.v4i2.1692.

[13] T. F. Mustafa and H. Alfianti, “Klasifikasi Berita Palsu Berbahasa Indonesia Menggunakan Algoritma Naive Bayes Berbasis Web,” J. Sains Inform. Terap., 2025, [Online]. Available: https://doi.org/10.62357/jsit.v4i3.564

[14] M. F. Ansyori and A. H. Mujianto, “Penerapan Natural Language Processing (NLP) dengan Metode Cosine Similarity pada Sistem E-Monev untuk Pencarian Program Pembangunan Daerah,” J. Software, Hardw. Inf. Technol., vol. 5, no. 2, pp. 84–102, 2025, doi: 10.24252/shift.v5i2.183.

[15] D. Rifaldi and others, “Evaluasi Sentimen Pengguna ChatGPT Menggunakan Naive Bayes: Tinjauan dari Confusion Matrix dan Classification Report,” J. Ris. Sist. dan Teknol. Inf., vol. 3, no. 2, pp. 81–89, 2025, [Online]. Available: https://doi.org/10.30787/restia.v3i2.1990

[16] O. N. Cahyani and F. Budiman, “Performa Logistic Regression dan Naive Bayes dalam Klasifikasi Berita Hoax di Indonesia,” Edumatic J. Pendidik. Inform., vol. 9, no. 1, pp. 60–68, 2025, doi: 10.29408/edumatic.v9i1.28987.

[17] A. Fardhina, R. M. Siregar, M. R. W. Br Sibarani, I. C. Br Ginting, and A. Pratama, “Sistem Deteksi Berita Hoaks berbasis Algoritma Natural Language Processing (NLP) menggunakan BERT,” J. Manaj. Inform. Sist. Inf. Dan Teknol. Komput., vol. 4, no. 1, pp. 450–461, 2025, doi: 10.70247/jumistik.v4i1.156.

[18] B. Wilie et al., “IndoNLU: Benchmark and Resources for Evaluating Indonesian Natural Language Understanding,” in Proc. 1st Conf. of the Asia-Pacific Chapter of the Association for Computational Linguistics, 2020, pp. 843–857. doi: 10.18653/v1/2020.aacl-main.85.

[19] J. Devlin, M. W. Chang, K. Lee, and K. Toutanova, “BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding,” in Proc. 2019 Conf. of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT), 2019, pp. 4171–4186. doi: 10.18653/v1/N19-1423.

[20] R. Fernando, Y. D. Proboningrum, S. D. Supriati, and Nurmalitasari, "NLP Implementation for AI Generated Text Detection (ChatGPT) Using Naive Bayes Method," J-INTECH (Journal of Information and Technology), vol. 13, no. 2, 2025, doi: 10.32664/j-intech.v13i02.2026.