Analisis Sentimen Evaluasi Reaksi E-Learning Menggunakan Algorima Naïve Bayes, Support Vector Machine Dan Deep Learning

Nurul Firdausy; Imam Yuadi; Ira Puspitasari

doi:10.33633/tc.v22i3.8160

Analisis Sentimen Evaluasi Reaksi E-Learning Menggunakan Algorima Naïve Bayes, Support Vector Machine Dan Deep Learning

Nurul Firdausy, Imam Yuadi, Ira Puspitasari

Abstract

Evaluasi reaksi atau evaluasi kepuasan merupakan bentuk evaluasi paling umum digunakan dalam pelatihan karena kemudahan dan sifatnya yang lekat dengan pelatihan. Meskipun mengandung wawasan yang dapat bernilai dalam peningkatan kualitas penyelenggaraan pelatihan, namun penelitian terkait reaksi peserta masih sangat terbatas. Penelitian ini bertujuan melakukan analisis sentiment terhadap evaluasi reaksi peserta e-learning menggunakan algoritma Naïve Bayes, Support Vector Machine dan Deep Learning. Reaksi peserta berupa komentar diklasifikasikan ke dalam kategori apresiasi, saran dan kritik. Hasil penelitian menunjukkan model Naïve Bayes memiliki kinerja yang lebih baik dibandingkan SVM dan Deep Learning dalam prediksi sentimen komentar peserta dengan tingkat akurasi, presisi dan recall masing-masing sebesar 82,54%, 68,08% dan 69,81%. Prediksi sentiment reaksi peserta menggunakan model Naïve Bayes diperoleh hasil 70% berupa apresiasi, 16% berupa saran dan 14% merupakan kritik. Penelitian ini memberikan kontribusi praktis analisis evaluasi reaksi pelatihan dan menambah literatur implementasi text mining pada domain human resource analytics.

Keywords

Analisis Sentimen; Naïve Bayes; Support Vector Machine; Deep Learning; Evaluasi Pelatihan

Full Text:

PDF

References

M. A. Memon, R. Sallaeh, M. N. R. Baharom, S. Md Nordin, and H. Ting, “The relationship between training satisfaction, organisational citizenship behaviour, and turnover intention: A PLS-SEM approach,” Journal of Organizational Effectiveness, vol. 4, no. 3, pp. 267–290, Sep. 2017, doi: 10.1108/JOEPP-03-2017-0025.

N. P. Z. Thwin, Y. N. Soe, A. Bhaumik, and Midhunchakkarayarthy, “Impact Of Training And Development On Employee Performance: A Study Of Myanmar Internet Service Provider Industry,” Proceedings on Engineering Sciences, vol. 4, no. 3, pp. 291–300, 2022, doi: 10.24874/PES04.03.007.

K. Nama, B. Daweti, M. Lourens, and T. Chikukwa, “The impact of training and development on employee performance and service delivery at a local municipality in South Africa,” Problems and Perspectives in Management, vol. 20, no. 4, pp. 42–51, 2022, doi: 10.21511/ppm.20(4).2022.04.

S. Aishwarya, M. Ayishamillath, and K. M. Ali, “Analyzing the impact of learning and development on employee engagement in information technology sector,” International Journal of Advanced Science and Technology, vol. 28, no. 19, pp. 1272–1282, 2019.

H. Aguinis and K. Kraiger, “Benefits of training and development for individuals and teams, organizations, and society,” Annual Review of Psychology, vol. 60. pp. 451–474, Jan. 2009. doi: 10.1146/annurev.psych.60.110707.163505.

A. Alsalamah and C. Callinan, “The Kirkpatrick model for training evaluation: bibliometric analysis after 60 years (1959–2020),” Industrial and Commercial Training, vol. 54, no. 1. Emerald Group Holdings Ltd., pp. 36–63, 2021. doi: 10.1108/ICT-12-2020-0115.

N. H. Chia et al., “Harnessing power of simulation training effectiveness with Kirkpatrick model in emergency surgical airway procedures,” Heliyon, vol. 8, no. 10, Oct. 2022, doi: 10.1016/j.heliyon.2022.e10886.

D. Eseryel, “Approaches to Evaluation of Training: Theory & Practice,” Educational Technology & Society, vol. 5, no. 2, p. 2002.

A. Alsalamah and C. Callinan, “Adaptation of Kirkpatrick’s four-level model of training criteria to evaluate training programmes for head teachers,” Educ Sci (Basel), vol. 11, no. 3, Mar. 2021, doi: 10.3390/educsci11030116.

S. Kim, S. Park, J. Lavelle, M. Kim, and S. Chaudhuri, “Revisiting trainee reactions: A multilevel analysis of the nomological network,” Hum Resour Dev Q, vol. 31, no. 2, pp. 173–191, Jun. 2020, doi: 10.1002/hrdq.21383.

K. G. Brown, “An examination of the structure and nomological network of trainee reactions: A closer look at ‘smile sheets,’” Journal of Applied Psychology, vol. 90, no. 5, pp. 991–1001, Sep. 2005, doi: 10.1037/0021-9010.90.5.991.

W. Arthur, W. Bennett, P. S. Edens, and S. T. Bell, “Effectiveness of training in organizations: A meta-analysis of design and evaluation features,” Journal of Applied Psychology, vol. 88, no. 2, pp. 234–245, Apr. 2003, doi: 10.1037/0021-9010.88.2.234.

R. P. Harman, J. Kemp Ellington, E. A. Surface, and L. F. Thompson, “Exploring qualitative training reactions: Individual and contextual influences on trainee commenting,” Journal of Applied Psychology, vol. 100, no. 3, pp. 894–916, May 2015, doi: 10.1037/a0038380.

T. Sitzmann, K. G. Brown, W. J. Casper, K. Ely, and R. D. Zimmerman, “A Review and Meta-Analysis of the Nomological Network of Trainee Reactions,” Journal of Applied Psychology, vol. 93, no. 2, pp. 280–295, Mar. 2008, doi: 10.1037/0021-9010.93.2.280.

A. Giangreco, A. Sebastiano, and R. Peccei, “Trainees’ reactions to training: An analysis of the factors affecting overall satisfaction with training,” International Journal of Human Resource Management, vol. 20, no. 1, pp. 96–111, 2009, doi: 10.1080/09585190802528417.

K. Kraiger, “Decision-based evaluation,” 2002. [Online]. Available: https://www.researchgate.net/publication/281601028

D. R. Glerum, D. L. Joseph, A. F. McKenny, and B. A. Fritzsche, “The trainer matters: Cross-classified models of trainee reactions.,” Journal of Applied Psychology, vol. 106, no. 2, pp. 281–299, 2021, doi: 10.1037/apl0000503.

R. P. K. Andoh, E. C. Annan-Prah, E. A. Owusu, and P. M. Agyei, “Trainees’ aversions of employee training programs,” European Journal of Training and Development, 2022, doi: 10.1108/EJTD-02-2022-0022.

A. Gegenfurtner, A. Zitt, and C. Ebner, “Evaluating webinar-based training: a mixed methods study of trainee reactions toward digital web conferencing,” Int J Train Dev, vol. 24, no. 1, pp. 5–21, 2020, doi: 10.1111/ijtd.12167.

C. M. Govender, C. M. Mabaso, and K. Mehale, “SA Journal of Human Resource Management,” 2021, doi: 10.4102/sajhrm.

F. Vencovský, “Service Quality Evaluation Using Text Mining: A Systematic Literature Review,” in Lecture Notes in Business Information Processing, Springer Science and Business Media Deutschland GmbH, 2020, pp. 159–173. doi: 10.1007/978-3-030-61140-8_11.

F. V. Ordenes, B. Theodoulidis, J. Burton, T. Gruber, and M. Zaki, “Analyzing Customer Experience Feedback Using Text Mining: A Linguistics-Based Approach,” J Serv Res, vol. 17, no. 3, pp. 278–295, 2014, doi: 10.1177/1094670514524625.

A. Firdaus and W. I. Firdaus, “Text Mining Dan Pola Algoritma Dalam Penyelesaian Masalah Informasi : (Sebuah Ulasan),” 2021.

A. Gupta, V. Dengre, H. A. Kheruwala, and M. Shah, “Comprehensive review of text-mining applications in finance,” Financial Innovation, vol. 6, no. 1. Springer Science and Business Media Deutschland GmbH, Dec. 01, 2020. doi: 10.1186/s40854-020-00205-1.

S. Pawar, N. Ramrakhiyani, G. K. Palshikar, and S. Hingmire, Natural Language Processing and Information Systems, vol. 9103. in Lecture Notes in Computer Science, vol. 9103. Cham: Springer International Publishing, 2015. doi: 10.1007/978-3-319-19581-0.

L. P. Hung and S. Alias, “Beyond Sentiment Analysis: A Review of Recent Trends in Text Based Sentiment Analysis and Emotion Detection,” Journal of Advanced Computational Intelligence and Intelligent Informatics, vol. 27, no. 1. Fuji Technology Press, pp. 84–95, Jan. 01, 2023. doi: 10.20965/jaciii.2023.p0084.

J. Liu, S. Hu, F. Mehraliyev, and H. Liu, “Text classification in tourism and hospitality – a deep learning perspective,” International Journal of Contemporary Hospitality Management. Emerald Publishing, 2023. doi: 10.1108/IJCHM-07-2022-0913.

N. A. K. M. Haris, S. Mutalib, A. M. A. Malik, S. Abdul-Rahman, and S. N. K. Kamarudin, “Sentiment classification from reviews for tourism analytics,” International Journal of Advances in Intelligent Informatics, vol. 9, no. 1, pp. 108–120, Mar. 2023, doi: 10.26555/ijain.v9i1.1076.

D. A. Nurdeni, I. Budi, and A. B. Santoso, “Sentiment Analysis on Covid19 Vaccines in Indonesia: From the Perspective of Sinovac and Pfizer,” in 3rd 2021 East Indonesia Conference on Computer and Information Technology, EIConCIT 2021, Institute of Electrical and Electronics Engineers Inc., Apr. 2021, pp. 122–127. doi: 10.1109/EIConCIT50028.2021.9431852.

R. Ardianto, T. Rivanie, Y. Alkhalifi, F. Septia Nugraha, and W. Gata, “SENTIMENT ANALYSIS ON E-SPORTS FOR EDUCATION CURRICULUM USING NAIVE BAYES AND SUPPORT VECTOR MACHINE,” 2020.

S. P. Kristanto, L. Hakim, D. Yusuf, E. S. Haq, and A. R. Asyari, “Classification of Public Opinion on Vaccine Administration Using Convolutional Neural Network,” in 2022 5th International Conference on Vocational Education and Electrical Engineering: The Future of Electrical Engineering, Informatics, and Educational Technology Through the Freedom of Study in the Post-Pandemic Era, ICVEE 2022 - Proceeding, Institute of Electrical and Electronics Engineers Inc., 2022, pp. 65–71. doi: 10.1109/ICVEE57061.2022.9930412.

R. Haque, N. Islam, M. Tasneem, and A. K. Das, “Multi-class sentiment classification on Bengali social media comments using machine learning,” International Journal of Cognitive Computing in Engineering, vol. 4, pp. 21–35, Jun. 2023, doi: 10.1016/j.ijcce.2023.01.001.

M. Zhang, B. Fan, N. Zhang, W. Wang, and W. Fan, “Mining product innovation ideas from online reviews,” Inf Process Manag, vol. 58, no. 1, Jan. 2021, doi: 10.1016/j.ipm.2020.102389.

F. Martinez-Plumed et al., “CRISP-DM Twenty Years Later: From Data Mining Processes to Data Science Trajectories,” IEEE Trans Knowl Data Eng, vol. 33, no. 8, pp. 3048–3061, Aug. 2021, doi: 10.1109/TKDE.2019.2962680.

W. Y. Ayele, “Adapting CRISP-DM for Idea Mining A Data Mining Process for Generating Ideas Using a Textual Dataset,” 2020. [Online]. Available: www.ijacsa.thesai.org

A. Azevedo and M. F. Santos, “KDD, SEMMA AND CRISP-DM: A PARALLEL OVERVIEW,” 2008. Accessed: Mar. 16, 2023. [Online]. Available: https://recipp.ipp.pt/handle/10400.22/136

M. Farid Naufal, A. Fernando Susanto, C. Nathaneil Kansil, S. Huda, and K. kunci, “Analisis Perbandingan Algoritma Machine Learning untuk Prediksi Potensi Hilangnya Nasabah Bank Application of Machine Learning to Predict Potential Loss of Bank Customer.”

DOI: https://doi.org/10.33633/tc.v22i3.8160