Usability Evaluation of RAD-Based Production Monitoring Systems in Automotive Manufacturing Using System Usability Scale Method

Tiaraningsih; Agung Nugroho; Karsito

doi:10.32664/smatika.v16i02.2353

Authors

Tiaraningsih Universitas Pelita Bangsa
Agung Nugroho Universitas Pelita Bangsa
Karsito Universitas Pelita Bangsa

DOI:

https://doi.org/10.32664/smatika.v16i02.2353

Keywords:

Automotive Manufacturing, Production Monitoring, RAD, System Usability Scale, User Experience

Abstract

The rapid adoption of digital monitoring tools in the automotive sector often prioritizes technical deployment speed over user-centric design, leading to potential operational inefficiencies. This study evaluates the usability of a production monitoring system developed through Rapid Application Development (RAD) at an automotive manufacturing plant. Utilizing a purposive sampling of 20 active users comprising 15 shop-floor operators and 5 supervisors. This research employs the System Usability Scale (SUS) to quantify user experience post-implementation. Data integrity was rigorously verified through SPSS, yielding a Cronbach’s Alpha of 0.842, which confirms the high reliability of the evaluation instrument. The findings reveal an overall average SUS score of 71.25, categorized as Acceptable with a Good adjective rating and a Grade C classification. Despite this favorable total score, a notable disparity exists between user groups; supervisors rated the system at 74.0, while operators, who manage high-frequency data entry, provided a lower average of 68.5. This result underscores a usability friction inherent in rapidly developed systems that may hinder long-term industrial sustainability. The study concludes that while the RAD-based architecture meets professional functional standards, iterative refinements focusing on reducing interface complexity for operators are essential to minimize human error and optimize production performance.

References

[1] G. P. F. Kristianto and Y. N. Ulfia, “Framework for Digital Transformation in Industry 4.0: Insights Data Driven Analysis in the Indonesia Manufacture Sector,” J. Adv. Inf. Ind. Technol., vol. 7, no. 1, pp. 45–60, 2025, doi: 10.52435/jaiit.v7i1.630.

[2] I. Y. D. Atma and Insan, “Penerapan Model Rapid Application Development Dalam Perancangan Aplikasi Monitoring Bahan Baku Produksi UMKM,” J. Sains Inform. Terap., vol. 4, no. 3, pp. 465–472, 2025, doi: 10.62357/jsit.v4i3.734.

[3] H. H. Farzaneh, “Usability Evaluation of Software Tools for Engineering Design,” in Proceedings of the Design Society International Conference on Engineering Design, 2019, pp. 1303–1312. doi: 10.1017/dsi.2019.136.

[4] A. A.-A. Yasser Omar Abdallah, Essam Shehab, “Digital Transformation Challenges in the Manufacturing Industry,” in 18th International Conference in Manufacturing Research ICMR 2021, 2021, pp. 9–14. doi: https://doi.org/10.3233/ATDE210004.

[5] B. R. Pressman, R. S., & Maxim, Software Engineering: A Practitioner’s Approach (8th ed). Amerika Serikat: McGraw-Hill Education, 2015.

[6] L. E. P. Jayanti, “Evaluasi Usability pada Aplikasi Mobile JKN Menggunakan Metode System Usability Scale (SUS),” J. Nas. Komputasi dan Teknol. Inf., vol. 8, no. 2, pp. 839–846, 2025, doi: 10.32672/jnkti.v8i2.8942.

[7] E. R. D. C. Dewi, “Analisis Beban Kerja Mental Operator Produksi Menggunakan Pendekatan Ergonomi Kognitif di Pabrik Kelapa Sawit PT. X,” J. Tek. Mesin dan Ind., vol. 5, no. 1, pp. 33–40, 2026, doi: 10.55331/jutmi.v5i1.84.

[8] J. Brooke, “SUS: A Retrospective,” J. Usability Stud., vol. 8, no. 2, pp. 29–40, 2013.

[9] A. Bangor, P. Kortum, and J. T. Miller, “An Empirical Evaluation of the System Usability Scale,” Int. J. Hum. Comput. Interact., vol. 24, no. 6, pp. 574–594, 2008, doi: 10.1080/10447310802205776.

[10] N. P. R. Anrahvi, “Penerapan Metode System Usability Scale (SUS) dalam Mengukur Kepuasan Mahasiswa terhadap Website Direktori Akademik,” Indones. J. Bus. Econ. Manag., vol. 3, no. 2, pp. 74–80, 2024, doi: 10.57152/ijbem.v3i2.2020.

[11] P. Neumann, S. Winkelhaus, E. H. Grosse, and C. H. Glock, “Industry 4.0 and the Human Factor -- A Systems Framework and Analysis Methodology for Successful Development,” Int. J. Prod. Econ., vol. 233, p. 107992, 2021, doi: 10.1016/j.ijpe.2020.107992.

[12] J. Lazar, J. H. Feng, and H. Hochheiser, Research Methods in Human-Computer Interaction. Morgan Kaufmann, 2017. doi: 10.1016/B978-0-12-805390-4.00001-4.

[13] F. Rahman, “Digital Transformation in The Small To Medium Industries Automotive Components In Indonesia,” Int. J. Soc. Sci., vol. 3, no. 4, pp. 445–458, 2023, doi: 10.53625/ijss.v3i4.6994.

[14] W. U. Sari, “Analisis Usability Pada Sistem Informasi Akademik Mahasiswa Universitas Muhammadiyah Riau Menggunakan Metode System Usability Scale (SUS),” J. Penelit. Saintek, vol. 25, no. 2, pp. 78–87, 2020, doi: 10.21831/jps.v25i2.28763.

[15] J. Nielsen, “Why You Only Need to Test with 5 Users,” 1995. [Online]. Available:https://www.nngroup.com/articles/why-you-only-need-to-test-with-5-users/

[16] J. R. Lewis, “Sample Sizes for Usability Studies: Additional Considerations,” Hum. Factors, vol. 36, no. 2, pp. 368–378, 1994, doi: 10.1177/001872089403600215.

[17] et al. Rahma Yani, “System Usability Scale in Information System Application Development Using Systematic Mapping Study,” IJATIS Indones. J. Appl. Technol. Innov. Sci., vol. 2, no. 2, pp. 130–140, 2025, doi: https://doi.org/10.57152/ijatis.v2i2.2275.

[18] N. Huda, “Implementasi Metode Usability Testing Dengan System Usability Scale Dalam Penilaian Website RS Siloam Palembang,” KLIK Kumpul. J. Ilmu Komput., vol. 6, no. 1, pp. 36–47, 2019.

[19] E. Kaasinen and others, “Empowering and Engaging Industrial Workers with Operator 4.0 Solutions,” Comput. Ind. Eng., vol. 139, p. 105678, 2020, doi: 10.1016/j.cie.2019.01.052.

[20] A. B. Phil Kortum, “Usability Ratings for Everyday Products Measured With the System Usability Scale (SUS),” Int. J. Hum. Comput. Interact., vol. 29, no. 2, pp. 67–76, 2012, doi: 10.1080/10447318.2012.681221.

[21] A. Bangor, P. Kortum, and J. Miller, “Determining What Individual SUS Scores Mean: Adding an Adjective Rating Scale,” J. User Exp., vol. 4, no. 3, pp. 114–123, 2009.

[22] M.-P. Pacaux-Lemoine, Q. Berdal, and C. Guérin, “Designing Human--System Cooperation in Industry 4.0 with Cognitive Work Analysis: A First Evaluation,” Cogn. Technol. Work, vol. 24, no. 1, 2022, doi: 10.1007/s10111-021-00667-y.

[23] L. Repokari and M. Nieminen, “Usability Evaluation of a Shop-floor Information Support System,” in 7th International Conference on Human Aspects of Advanced Manufacturing: Agility and Hybrid Automation, 2000.

[24] C. Li and H. Zhu, “A Systematic Review of Human-Aware Systems in Manufacturing and Logistics: Overcoming Obstacles With Advanced Modeling Techniques,” Eng. Reports, vol. 7, no. 12, 2025, doi: 10.1002/eng2.70513.

[25] M. U. Mojumder and R. A. R., “Human-Machine Interfaces in Industrial Systems: Enhancing Safety and Throughput in Semi-Automated Facilities,” Am. J. Interdiscip. Stud., vol. 4, no. 1, pp. 1–26, 2023, doi: 10.63125/s2qa0125.

[26] C. A. Alejandro Chacón, Pere Ponsa, “Cognitive Interaction Analysis in Human–Robot Collaboration Using an Assembly Task,” Electronics, vol. 10, no. 11, p. 1317, 2021, doi: 10.3390/electronics10111317.

[27] E. Vilar, F. Rebelo, and P. Noriega, “Human Factors and Ergonomics in Manufacturing and Service Industries,” Hum. Factors Ergon. Manuf., 2012.

[28] L. Gualtieri, E. Rauch, and R. Vidoni, “Emerging Research Fields in Safety and Ergonomics in Industrial Collaborative Robotics: A Systematic Literature Review,” Robot. Comput. Integr. Manuf., vol. 67, p. 101998, 2021, doi: 10.1016/j.rcim.2020.101998.

[29] M. Chiara Leva, M. Demichela, L. Comberti, and A. Caimo, “Human performance in manufacturing tasks: Optimization and assessment of required workload and capabilities,” Saf. Sci., vol. 154, p. 105838, 2022, doi: https://doi.org/10.1016/j.ssci.2022.105838.

[30] I. Val, S., & García, “Overall Equipment Effectiveness for Elevators (OEEE) in Industry 4.0: Conceptual Framework and Indicators,” 2025, vol. 9, no. 227, 6AD, doi: https://doi.org/10.3390/eng6090227.