Opinion Mining on Chat GPT based on Twitter Users

Authors

DOI:

https://doi.org/10.33633/jais.v8i2.8399

Abstract

The presence of Chatbots can assist humans in their everyday lives. Chat GPT is one of the commonly used Chatbots that humans rely on to support their work, serve as an assistant, or even create artistic works or writings. The purpose of this research is to investigate opinions regarding the presence of Chat GPT. This Opinion Mining method is conducted by crawling data from Twitter, which can be categorized into three opinions: Positive, Negative, or Neutral. To calculate the accuracy level of the model created, two algorithms, Naïve Bayes and K-Nearest Neighbour, are compared. The model validation process utilizes K-Fold Cross Validation by varying the value of k (k=2, k=4, k=6, k=8, and k=10) and different sampling methods, namely Linear, Shuffled, and Stratified, to obtain optimal accuracy values. The research results indicate that the K-Nearest Neighbour Algorithm achieves the highest accuracy value of 92.40%. Based on this comparison, the K-Nearest Neighbour Algorithm is deemed suitable for modeling Opinion Mining of Chat GPT. The distribution of Twitter users' opinion percentages regarding Chat GPT is as follows: Positive 9.4%, Negative 1.4%, and Neutral 89%. Neutral opinions dominate the results of the conducted Opinion Mining.Keyword : chat GPT, opinion mining, twitter

References

X. Deng and Z. Yu, “A Meta-Analysis and Systematic Review of the Effect of Chatbot Technology Use in Sustainable Education,” Sustainability, vol. 15, no. 4, pp. 1–19, Feb. 2023, doi: 10.3390/su15042940.

V. Taecharungroj, “‘What Can ChatGPT Do?’ Analyzing Early Reactions to the Innovative AI Chatbot on Twitter,” Big Data Cogn. Comput., vol. 7, no. 1, pp. 1–10, 2023, doi: 10.3390/bdcc7010035.

J. Y. Lee, “Can an artificial intelligence chatbot be the author of a scholarly article?,” J. Educ. Eval. Health Prof., vol. 20, no. 1, p. 6, 2023, doi: 10.3352/jeehp.2022.20.6.

S. Singh, S. Tiwari, and P. K. Yadav, “Chat GPT : Exploring The Capabilities And Limitations Of A Large Language,” Int. J. Eng. Appl. Sci. Technol., vol. 7, no. 12, pp. 111–115, 2023.

M. Dowling and B. Lucey, “ChatGPT for (Finance) research: The Bananarama Conjecture,” Financ. Res. Lett., no. 103662, pp. 1–20, 2023, doi: 10.1016/j.frl.2023.103662.

I. T. Julianto, D. Kurniadi, Y. Septiana, and A. Sutedi, “Alternative Text Pre-Processing using Chat GPT Open AI,” Janapati, vol. 12, no. 1, pp. 67–77, 2023, [Online]. Available: https://wjaets.com/content/artificial-intelligence-ai-based-chatbot-study-chatgpt-google-ai-bard-and-baidu-ai.

O. P. Zusrotun, A. C. Murti, and R. Fiati, “Sentimen Analisis Belajar Online Di Twitter Menggunakan Naïve Bayes,” JANAPATI, vol. 11, no. 3, pp. 310–320, 2022.

B. Kurniawan Rachmat, A. Suwarisman, I. Afriyanti, A. Wahyudi, and D. D. Saputra, “Analisis Sentimen Complain dan Bukan Complain pada Twitter Telkomsel dengan SMOTE dan Naïve Bayes,” J. Teknol. Inf. dan Komunikasi), vol. 7, no. 1, pp. 107–113, 2023, [Online]. Available: https://doi.org/10.35870/jti.

C. Messaoudi, Z. Guessoum, and L. ben Romdhane, “A Deep Learning Model for Opinion mining in Twitter Combining Text and Emojis,” Procedia Comput. Sci., vol. 207, no. Kes, pp. 2628–2637, 2022, doi: 10.1016/j.procs.2022.09.321.

A. H. Anshor and A. Safuwan, “Analisis Sentimen Opini Warganet Twitter Terhadap Tes Screening Genose Pendeteksi Virus Covid-19 Menggunakan Metode Naïve Bayes Berbasis Particle Swarm Optimization,” JINTEKS (Jurnal Inform. Teknol. dan Sains), vol. 5, no. 1, pp. 170–178, 2023.

L. R. Dharmawan, I. Arwani, and D. E. Ratnawati, “Analisis Sentimen pada Sosial Media Twitter Terhadap Layanan Sistem Informasi Akademik Mahasiswa Universitas Brawijaya dengan Metode K- Nearest Neighbor,” J. Pengemb. Teknol. Inf. dan Ilmu Komput., vol. 4, no. 3, pp. 959–965, 2020, [Online]. Available: http://j-ptiik.ub.ac.id/index.php/j-ptiik/article/view/7099.

E. D. Nurhazizah and I. Puspitasari, “Opinion Mining Fungsi KPI (Key Performance Indikator) Dengan Algoritma Naïve Bayes Clasifier Dan Support Vector Machine (SVM),” J. Cahaya Mandalika, vol. 3, no. 2, pp. 290–302, 2023.

Z. Azhari, L. Efrizoni, W. Agustin, and R. Yanti, “Opinion Mining menggunakan Algoritma Deep Learning untuk Menganalisis Penggunaan Aplikasi Jamsostek Mobile,” Indones. J. Comput. Sci., vol. 12, no. 2, pp. 666–678, 2023.

H. Andriana, S. S. Hilabi, and A. Hananto, “Penerapan Metode K-Nearest Neighbor pada Sentimen Analisis Pengguna Twitter Terhadap KTT G20 di Indonesia,” JURIKOM (Jurnal Ris. Komputer), vol. 10, no. 1, pp. 60–67, 2023, doi: 10.30865/jurikom.v10i1.5427.

A. Fauzy, Metode Sampling. Tanggerang Selatan: Universitas Terbuka, 2019.

A. A. Abdillah, A. Azwardi, and I. Wahyudi, “Optimasi Linear Sampling dan Information Gain pada Algoritma Decision Tree untuk Diagnosis Penyakit Diabetes,” Multinetics, vol. 7, no. 1, pp. 21–29, 2021.

A. Bisri and M. Man, “Machine Learning Algorithms Based on Sampling Techniques for Raisin Grains Classification,” Int. J. Informatics Vis., vol. 7, no. 1, pp. 7–14, 2023, doi: 10.30630/joiv.7.1.970.

A. Pebdika, R. Herdiana, and D. Solihudin, “Klasifikasi Menggunakan Metode Naive Bayes Untuk Menentukan Calon Penerima PIP,” JATI (Jurnal Mhs. Tek. Inform., vol. 7, no. 1, pp. 452–458, 2023.

M. K. Insan, U. Hayati, and O. Nurdiawan, “Analisis Sentimen Aplikasi Brimo Pada Ulasan Pengguna Di Google Play Menggunakan Algoritma Naive Bayes,” JATI (Jurnal Mhs. Tek. Inform., vol. 7, no. 1, pp. 478–483, 2023.

S. R. Cholil, T. Handayani, R. Prathivi, and T. Ardianita, “Implementasi Algoritma Klasifikasi K-Nearest Neighbor (KNN) Untuk Klasifikasi Seleksi Penerima Beasiswa,” IJCIT (Indonesian J. Comput. Inf. Technol., vol. 6, no. 2, pp. 118–127, 2021.

A. D. Savitri, F. A. Bachtiar, and N. Y. Setiawan, “Segmentasi Pelanggan Menggunakan Metode K-Means Clustering Berdasarkan Model RFM Pada Klinik Kecantikan (Studi Kasus : Belle Crown Malang),” J. Pengemb. Teknol. Inf. dan Ilmu Komput. Univ. Brawijaya, vol. 2, no. 9, pp. 2957–2966, 2018.

Junadhi, Agustin, M. Rifqi, and M. K. Anam, “Sentiment Analysis Of Online Lectures Using K-Nearest Neighbors Based On Feature Selection,” Janapati, vol. 11, no. 3, pp. 216–225, 2022.

M. Dennis, F. Zoromi, and M. K. Anam, “Penerapan Algoritma Naïve Bayes Untuk Pengelompokkan Predikat Peserta Uji Kemahiran Berbahasa Indonesia,” J. Media Inform. Budidarma, vol. 6, no. 2, pp. 1183–1190, 2022, doi: 10.30865/mib.v6i2.3956.

S. Khairunnisa, A. Adiwijaya, and S. Al Faraby, “Pengaruh Text Preprocessing terhadap Analisis Sentimen Komentar Masyarakat pada Media Sosial Twitter (Studi Kasus Pandemi COVID-19),” J. Media Inform. Budidarma, vol. 5, no. 2, pp. 406–414, 2021, doi: 10.30865/mib.v5i2.2835.

I. T. Julianto, D. Kurniadi, M. R. Nashrulloh, and A. Mulyani, “Comparison Of Classification Algorithm And Feature Selection in Bitcoin Sentiment Analysis,” JUTIF, vol. 3, no. 3, pp. 739–744, 2022.

D. S. Utami and A. Erfina, “Analisis Sentimen Pinjaman Online di Twitter Menggunakan Algoritma Support Vector Machine (SVM),” SISMATIK (Seminar Nas. Sist. Inf. dan Manaj. Inform., vol. 1, no. 1, pp. 299–305, 2021.

O. Abiola, A. Abayomi-Alli, O. A. Tale, S. Misra, and O. Abayomi-Alli, “Sentiment analysis of COVID-19 tweets from selected hashtags in Nigeria using VADER and Text Blob analyser,” J. Electr. Syst. Inf. Technol., vol. 10, no. 1, 2023, doi: 10.1186/s43067-023-00070-9.

D. Abimanyu, E. Budianita, E. P. Cynthia, F. Yanto, and Y. Yusra, “Analisis Sentimen Akun Twitter Apex Legends Menggunakan VADER,” J. Nas. Komputasi dan Teknol. Inf., vol. 5, no. 3, pp. 423–431, 2022, doi: 10.32672/jnkti.v5i3.4382.

M. Murali, B. Duraisamy, and J. Vankara, “Measurement : Sensors Independent component support vector regressive deep learning for sentiment classification,” Meas. Sensors, vol. 26, no. December 2022, pp. 1–8, 2023, doi: 10.1016/j.measen.2023.100678.

I. T. Julianto, D. Kurniadi, M. R. Nashrulloh, and A. Mulyani, “Twitter Social Media Sentiment Analysis Against Bitcoin Cryptocurrency Trends Using Rapidminer,” J. Tek. Inform., vol. 3, no. 5, pp. 1183–1187, 2022.

I. T. Julianto, “Analisis Sentimen Terhadap Sistem Informasi Akademik Institut Teknologi Garut,” J. Algoritm., vol. 19, no. 1, pp. 449–456, 2022, doi: 10.33364/algoritma/v.19-1.1112.

N. Alnuaimi, A. Almemari, M. Madine, K. Salah, H. Al Breiki, and R. Jayaraman, “NFT Certificates and Proof of Delivery for Fine Jewelry and Gemstones,” IEEE Access, vol. 10, pp. 101263–101275, 2022, doi: 10.1109/ACCESS.2022.3208698.

A. K. Fauziyyah and D. H. Gautama, “Analisis Sentimen Pandemi Covid19 Pada Streaming Twitter Dengan Text Mining Python,” J. Ilm. SINUS, vol. 18, no. 2, pp. 31–42, 2020, doi: 10.30646/sinus.v18i2.491.

J. Sangeetha and U. Kumaran, “A hybrid optimization algorithm using BiLSTM structure for sentiment analysis,” Meas. Sensors, vol. 25, no. December 2022, pp. 1–7, 2023, doi: 10.1016/j.measen.2022.100619.

A. E. Budiman and A. Widjaja, “Analisis Pengaruh Teks Preprocessing Terhadap Deteksi Plagiarisme Pada Dokumen Tugas Akhir,” J. Tek. Inform. dan Sist. Inf., vol. 6, no. 3, pp. 475–488, 2020, doi: 10.28932/jutisi.v6i3.2892.

L. K. Harsono, Y. Alkhalifi, Nurajijah, and W. Gata, “Analisis Sentimen Stakeholder atas Layanan haiDJPb pada Media Sosial Twitter Dengan Menggunakan Metode Support Vector Machine dan Naïve Bayes,” J. Ilmu-ilmu Inform. dan Manaj., vol. 14, no. 1, pp. 36–44, 2020.

A. Ahmad and W. Gata, “Sentimen Analisis Masyarakat Indonesia di Twitter Terkait Metaverse dengan Algoritma Support Vector Machine,” J. JTIK (Jurnal Teknol. Inf. dan Komunikasi), vol. 6, no. 4, pp. 548–555, 2022, doi: 10.35870/jtik.v6i4.569.

G. Feng, M. Fan, and Y. Chen, “Analysis and Prediction of Students’ Academic Performance Based on Educational Data Mining,” IEEE Access, vol. 10, pp. 19558–19571, 2022, doi: 10.1109/ACCESS.2022.3151652.

I. T. Julianto, D. Kurniadi, and F. M. Khoiriyyah, “Price Prediction of Non-Fungible Tokens (NFTs) using Data Mining Prediction Algorithm,” in 2023 International Conference on Computer Science, Information Technology and Engineering (ICCoSITE), Feb. 2023, pp. 633–637, doi: 10.1109/ICCoSITE57641.2023.10127679.

Yunitasari, H. S. Hopipah, and R. Mayasari, “Optimasi Backward Elimination untuk Klasifikasi Kepuasan Pelanggan Menggunakan Algoritme k-nearest neighbor (k-NN) and Naive Bayes,” Technomedia J., vol. 6, no. 1, pp. 99–110, 2021, doi: 10.33050/tmj.v6i1.1531.

D. Nurlaela, “Penerapan Adaboost untuk Meningkatkan Akurasi Naive Bayes Pada Prediksi Pendapatan Penjualan Film,” Inti Nusa Mandiri, vol. 14, no. 2, pp. 181–188, 2020.

Y. Septiana, Y. H. Agustin, M. Nashir Mudzakir, A. Mulyani, D. D. Siti Fatimah, and I. Tri Julianto, “Implementation of Classification Algorithm C4.5 in Determining the Emergency Patient in the Maternity Hospital Queue System,” in 2023 International Conference on Computer Science, Information Technology and Engineering (ICCoSITE), Feb. 2023, pp. 790–794, doi: 10.1109/ICCoSITE57641.2023.10127842.

D. Normawati and D. P. Ismi, “K-Fold Cross Validation for Selection of Cardiovascular Disease Diagnosis Features by Applying Rule-Based Datamining,” Signal Image Process. Lett., vol. 1, no. 2, pp. 23–35, 2019, doi: 10.31763/simple.v1i2.3.

Downloads

Published

2023-07-31

Issue

Vol. 8 No. 2 (2023): Journal of Applied Intelligent System

Section

Articles

License

 
Authors who publish with this journal agree to the following terms:

  1. Authors retain copyright and grant the journal right of first publication with the work simultaneously licensed under a Creative Commons Attribution License that allows others to share the work with an acknowledgment of the work's authorship and initial publication in this journal.
  2. Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgment of its initial publication in this journal.
  3. Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work (See The Effect of Open Access).