Opinion Mining on Chat GPT based on Twitter Users

Muhammad Rikza Nashrulloh, Indri Tri Julianto, Rifky Khoerul Muzaky

Abstract


The presence of Chatbots can assist humans in their everyday lives. Chat GPT is one of the commonly used Chatbots that humans rely on to support their work, serve as an assistant, or even create artistic works or writings. The purpose of this research is to investigate opinions regarding the presence of Chat GPT. This Opinion Mining method is conducted by crawling data from Twitter, which can be categorized into three opinions: Positive, Negative, or Neutral. To calculate the accuracy level of the model created, two algorithms, Naïve Bayes and K-Nearest Neighbour, are compared. The model validation process utilizes K-Fold Cross Validation by varying the value of k (k=2, k=4, k=6, k=8, and k=10) and different sampling methods, namely Linear, Shuffled, and Stratified, to obtain optimal accuracy values. The research results indicate that the K-Nearest Neighbour Algorithm achieves the highest accuracy value of 92.40%. Based on this comparison, the K-Nearest Neighbour Algorithm is deemed suitable for modeling Opinion Mining of Chat GPT. The distribution of Twitter users' opinion percentages regarding Chat GPT is as follows: Positive 9.4%, Negative 1.4%, and Neutral 89%. Neutral opinions dominate the results of the conducted Opinion Mining.

Keyword : chat GPT, opinion mining, twitter


Full Text:

PDF

References


X. Deng and Z. Yu, “A Meta-Analysis and Systematic Review of the Effect of Chatbot Technology Use in Sustainable Education,” Sustainability, vol. 15, no. 4, pp. 1–19, Feb. 2023, doi: 10.3390/su15042940.

V. Taecharungroj, “‘What Can ChatGPT Do?’ Analyzing Early Reactions to the Innovative AI Chatbot on Twitter,” Big Data Cogn. Comput., vol. 7, no. 1, pp. 1–10, 2023, doi: 10.3390/bdcc7010035.

J. Y. Lee, “Can an artificial intelligence chatbot be the author of a scholarly article?,” J. Educ. Eval. Health Prof., vol. 20, no. 1, p. 6, 2023, doi: 10.3352/jeehp.2022.20.6.

S. Singh, S. Tiwari, and P. K. Yadav, “Chat GPT : Exploring The Capabilities And Limitations Of A Large Language,” Int. J. Eng. Appl. Sci. Technol., vol. 7, no. 12, pp. 111–115, 2023.

M. Dowling and B. Lucey, “ChatGPT for (Finance) research: The Bananarama Conjecture,” Financ. Res. Lett., no. 103662, pp. 1–20, 2023, doi: 10.1016/j.frl.2023.103662.

I. T. Julianto, D. Kurniadi, Y. Septiana, and A. Sutedi, “Alternative Text Pre-Processing using Chat GPT Open AI,” Janapati, vol. 12, no. 1, pp. 67–77, 2023, [Online]. Available: https://wjaets.com/content/artificial-intelligence-ai-based-chatbot-study-chatgpt-google-ai-bard-and-baidu-ai.

O. P. Zusrotun, A. C. Murti, and R. Fiati, “Sentimen Analisis Belajar Online Di Twitter Menggunakan Naïve Bayes,” JANAPATI, vol. 11, no. 3, pp. 310–320, 2022.

B. Kurniawan Rachmat, A. Suwarisman, I. Afriyanti, A. Wahyudi, and D. D. Saputra, “Analisis Sentimen Complain dan Bukan Complain pada Twitter Telkomsel dengan SMOTE dan Naïve Bayes,” J. Teknol. Inf. dan Komunikasi), vol. 7, no. 1, pp. 107–113, 2023, [Online]. Available: https://doi.org/10.35870/jti.

C. Messaoudi, Z. Guessoum, and L. ben Romdhane, “A Deep Learning Model for Opinion mining in Twitter Combining Text and Emojis,” Procedia Comput. Sci., vol. 207, no. Kes, pp. 2628–2637, 2022, doi: 10.1016/j.procs.2022.09.321.

A. H. Anshor and A. Safuwan, “Analisis Sentimen Opini Warganet Twitter Terhadap Tes Screening Genose Pendeteksi Virus Covid-19 Menggunakan Metode Naïve Bayes Berbasis Particle Swarm Optimization,” JINTEKS (Jurnal Inform. Teknol. dan Sains), vol. 5, no. 1, pp. 170–178, 2023.

L. R. Dharmawan, I. Arwani, and D. E. Ratnawati, “Analisis Sentimen pada Sosial Media Twitter Terhadap Layanan Sistem Informasi Akademik Mahasiswa Universitas Brawijaya dengan Metode K- Nearest Neighbor,” J. Pengemb. Teknol. Inf. dan Ilmu Komput., vol. 4, no. 3, pp. 959–965, 2020, [Online]. Available: http://j-ptiik.ub.ac.id/index.php/j-ptiik/article/view/7099.

E. D. Nurhazizah and I. Puspitasari, “Opinion Mining Fungsi KPI (Key Performance Indikator) Dengan Algoritma Naïve Bayes Clasifier Dan Support Vector Machine (SVM),” J. Cahaya Mandalika, vol. 3, no. 2, pp. 290–302, 2023.

Z. Azhari, L. Efrizoni, W. Agustin, and R. Yanti, “Opinion Mining menggunakan Algoritma Deep Learning untuk Menganalisis Penggunaan Aplikasi Jamsostek Mobile,” Indones. J. Comput. Sci., vol. 12, no. 2, pp. 666–678, 2023.

H. Andriana, S. S. Hilabi, and A. Hananto, “Penerapan Metode K-Nearest Neighbor pada Sentimen Analisis Pengguna Twitter Terhadap KTT G20 di Indonesia,” JURIKOM (Jurnal Ris. Komputer), vol. 10, no. 1, pp. 60–67, 2023, doi: 10.30865/jurikom.v10i1.5427.

A. Fauzy, Metode Sampling. Tanggerang Selatan: Universitas Terbuka, 2019.

A. A. Abdillah, A. Azwardi, and I. Wahyudi, “Optimasi Linear Sampling dan Information Gain pada Algoritma Decision Tree untuk Diagnosis Penyakit Diabetes,” Multinetics, vol. 7, no. 1, pp. 21–29, 2021.

A. Bisri and M. Man, “Machine Learning Algorithms Based on Sampling Techniques for Raisin Grains Classification,” Int. J. Informatics Vis., vol. 7, no. 1, pp. 7–14, 2023, doi: 10.30630/joiv.7.1.970.

A. Pebdika, R. Herdiana, and D. Solihudin, “Klasifikasi Menggunakan Metode Naive Bayes Untuk Menentukan Calon Penerima PIP,” JATI (Jurnal Mhs. Tek. Inform., vol. 7, no. 1, pp. 452–458, 2023.

M. K. Insan, U. Hayati, and O. Nurdiawan, “Analisis Sentimen Aplikasi Brimo Pada Ulasan Pengguna Di Google Play Menggunakan Algoritma Naive Bayes,” JATI (Jurnal Mhs. Tek. Inform., vol. 7, no. 1, pp. 478–483, 2023.

S. R. Cholil, T. Handayani, R. Prathivi, and T. Ardianita, “Implementasi Algoritma Klasifikasi K-Nearest Neighbor (KNN) Untuk Klasifikasi Seleksi Penerima Beasiswa,” IJCIT (Indonesian J. Comput. Inf. Technol., vol. 6, no. 2, pp. 118–127, 2021.

A. D. Savitri, F. A. Bachtiar, and N. Y. Setiawan, “Segmentasi Pelanggan Menggunakan Metode K-Means Clustering Berdasarkan Model RFM Pada Klinik Kecantikan (Studi Kasus : Belle Crown Malang),” J. Pengemb. Teknol. Inf. dan Ilmu Komput. Univ. Brawijaya, vol. 2, no. 9, pp. 2957–2966, 2018.

Junadhi, Agustin, M. Rifqi, and M. K. Anam, “Sentiment Analysis Of Online Lectures Using K-Nearest Neighbors Based On Feature Selection,” Janapati, vol. 11, no. 3, pp. 216–225, 2022.

M. Dennis, F. Zoromi, and M. K. Anam, “Penerapan Algoritma Naïve Bayes Untuk Pengelompokkan Predikat Peserta Uji Kemahiran Berbahasa Indonesia,” J. Media Inform. Budidarma, vol. 6, no. 2, pp. 1183–1190, 2022, doi: 10.30865/mib.v6i2.3956.

S. Khairunnisa, A. Adiwijaya, and S. Al Faraby, “Pengaruh Text Preprocessing terhadap Analisis Sentimen Komentar Masyarakat pada Media Sosial Twitter (Studi Kasus Pandemi COVID-19),” J. Media Inform. Budidarma, vol. 5, no. 2, pp. 406–414, 2021, doi: 10.30865/mib.v5i2.2835.

I. T. Julianto, D. Kurniadi, M. R. Nashrulloh, and A. Mulyani, “Comparison Of Classification Algorithm And Feature Selection in Bitcoin Sentiment Analysis,” JUTIF, vol. 3, no. 3, pp. 739–744, 2022.

D. S. Utami and A. Erfina, “Analisis Sentimen Pinjaman Online di Twitter Menggunakan Algoritma Support Vector Machine (SVM),” SISMATIK (Seminar Nas. Sist. Inf. dan Manaj. Inform., vol. 1, no. 1, pp. 299–305, 2021.

O. Abiola, A. Abayomi-Alli, O. A. Tale, S. Misra, and O. Abayomi-Alli, “Sentiment analysis of COVID-19 tweets from selected hashtags in Nigeria using VADER and Text Blob analyser,” J. Electr. Syst. Inf. Technol., vol. 10, no. 1, 2023, doi: 10.1186/s43067-023-00070-9.

D. Abimanyu, E. Budianita, E. P. Cynthia, F. Yanto, and Y. Yusra, “Analisis Sentimen Akun Twitter Apex Legends Menggunakan VADER,” J. Nas. Komputasi dan Teknol. Inf., vol. 5, no. 3, pp. 423–431, 2022, doi: 10.32672/jnkti.v5i3.4382.

M. Murali, B. Duraisamy, and J. Vankara, “Measurement : Sensors Independent component support vector regressive deep learning for sentiment classification,” Meas. Sensors, vol. 26, no. December 2022, pp. 1–8, 2023, doi: 10.1016/j.measen.2023.100678.

I. T. Julianto, D. Kurniadi, M. R. Nashrulloh, and A. Mulyani, “Twitter Social Media Sentiment Analysis Against Bitcoin Cryptocurrency Trends Using Rapidminer,” J. Tek. Inform., vol. 3, no. 5, pp. 1183–1187, 2022.

I. T. Julianto, “Analisis Sentimen Terhadap Sistem Informasi Akademik Institut Teknologi Garut,” J. Algoritm., vol. 19, no. 1, pp. 449–456, 2022, doi: 10.33364/algoritma/v.19-1.1112.

N. Alnuaimi, A. Almemari, M. Madine, K. Salah, H. Al Breiki, and R. Jayaraman, “NFT Certificates and Proof of Delivery for Fine Jewelry and Gemstones,” IEEE Access, vol. 10, pp. 101263–101275, 2022, doi: 10.1109/ACCESS.2022.3208698.

A. K. Fauziyyah and D. H. Gautama, “Analisis Sentimen Pandemi Covid19 Pada Streaming Twitter Dengan Text Mining Python,” J. Ilm. SINUS, vol. 18, no. 2, pp. 31–42, 2020, doi: 10.30646/sinus.v18i2.491.

J. Sangeetha and U. Kumaran, “A hybrid optimization algorithm using BiLSTM structure for sentiment analysis,” Meas. Sensors, vol. 25, no. December 2022, pp. 1–7, 2023, doi: 10.1016/j.measen.2022.100619.

A. E. Budiman and A. Widjaja, “Analisis Pengaruh Teks Preprocessing Terhadap Deteksi Plagiarisme Pada Dokumen Tugas Akhir,” J. Tek. Inform. dan Sist. Inf., vol. 6, no. 3, pp. 475–488, 2020, doi: 10.28932/jutisi.v6i3.2892.

L. K. Harsono, Y. Alkhalifi, Nurajijah, and W. Gata, “Analisis Sentimen Stakeholder atas Layanan haiDJPb pada Media Sosial Twitter Dengan Menggunakan Metode Support Vector Machine dan Naïve Bayes,” J. Ilmu-ilmu Inform. dan Manaj., vol. 14, no. 1, pp. 36–44, 2020.

A. Ahmad and W. Gata, “Sentimen Analisis Masyarakat Indonesia di Twitter Terkait Metaverse dengan Algoritma Support Vector Machine,” J. JTIK (Jurnal Teknol. Inf. dan Komunikasi), vol. 6, no. 4, pp. 548–555, 2022, doi: 10.35870/jtik.v6i4.569.

G. Feng, M. Fan, and Y. Chen, “Analysis and Prediction of Students’ Academic Performance Based on Educational Data Mining,” IEEE Access, vol. 10, pp. 19558–19571, 2022, doi: 10.1109/ACCESS.2022.3151652.

I. T. Julianto, D. Kurniadi, and F. M. Khoiriyyah, “Price Prediction of Non-Fungible Tokens (NFTs) using Data Mining Prediction Algorithm,” in 2023 International Conference on Computer Science, Information Technology and Engineering (ICCoSITE), Feb. 2023, pp. 633–637, doi: 10.1109/ICCoSITE57641.2023.10127679.

Yunitasari, H. S. Hopipah, and R. Mayasari, “Optimasi Backward Elimination untuk Klasifikasi Kepuasan Pelanggan Menggunakan Algoritme k-nearest neighbor (k-NN) and Naive Bayes,” Technomedia J., vol. 6, no. 1, pp. 99–110, 2021, doi: 10.33050/tmj.v6i1.1531.

D. Nurlaela, “Penerapan Adaboost untuk Meningkatkan Akurasi Naive Bayes Pada Prediksi Pendapatan Penjualan Film,” Inti Nusa Mandiri, vol. 14, no. 2, pp. 181–188, 2020.

Y. Septiana, Y. H. Agustin, M. Nashir Mudzakir, A. Mulyani, D. D. Siti Fatimah, and I. Tri Julianto, “Implementation of Classification Algorithm C4.5 in Determining the Emergency Patient in the Maternity Hospital Queue System,” in 2023 International Conference on Computer Science, Information Technology and Engineering (ICCoSITE), Feb. 2023, pp. 790–794, doi: 10.1109/ICCoSITE57641.2023.10127842.

D. Normawati and D. P. Ismi, “K-Fold Cross Validation for Selection of Cardiovascular Disease Diagnosis Features by Applying Rule-Based Datamining,” Signal Image Process. Lett., vol. 1, no. 2, pp. 23–35, 2019, doi: 10.31763/simple.v1i2.3.




DOI: https://doi.org/10.33633/jais.v8i2.8399

Article Metrics

Abstract view : 225 times
PDF - 130 times

Refbacks

  • There are currently no refbacks.


Flag Counter

 

 

 

 

Journal of Applied Intelligent System (e-ISSN : 2502-9401, p-ISSN : 2503-0493) is published by Department of Informatics Universitas Dian Nuswantoro Semarang and IndoCEISS.

  

 

Journal of Applied Intelligent System indexed by :


This journal is under licensed of Creative Commons Attribution 4.0 International License.

Visitor Stats