Student poetry translations with and without the use of AI
An evaluation of translation quality
DOI:
https://doi.org/10.33633/lite.v21i2.14336Keywords:
artificial intelligence, poeticness, poetry translation, translation qualityAbstract
Poetry translation presents a unique challenge, as it requires maintaining a delicate balance between semantic fidelity and the preservation of the aesthetic qualities of the target text. This study aims to evaluate the quality of poetry translations produced by university students both with and without the assistance of artificial intelligence (AI), using four assessment criteria based on Nababan's (2012) framework: accuracy, acceptability, readability, and poeticness. The research employed a descriptive quantitative approach with an evaluation instrument using a three-point scale, where a score of 3 indicates high quality, 2 denotes moderate quality, and 1 reflects low quality. Data were obtained from poetry translations evaluated by experienced raters using a standardized rubric, with results analyzed in terms of frequency distribution and percentages. Findings reveal that in AI-assisted translations, the highest score attainment was observed in readability (52.76%), followed by acceptability (51.14%), accuracy (50.39%), and poeticness (24.34%) as the lowest. In contrast, in translations without AI, the highest score attainment was also found in readability (34.76%), followed by accuracy (33.06%), acceptability (32.43%), and poeticness (12.33%). This comparison indicates that AI use consistently enhances accuracy, acceptability, and readability scores, yet shows limited capacity to improve poeticness significantly. This aspect requires creativity, stylistic sensitivity, and nuanced linguistic judgment beyond the current capabilities of AI systems.References
Al Mudarra, A. (2025). The future of human translators in the era of artificial intelligence translation technologies. Journal of Language Teaching and Research, 16(4), 1213–1224. https://doi.org/10.17507/jltr.1604.16
Alafnan, M. A., & Alshakhs, T. (2025). Bridging linguistic and cultural nuances: A comparative study of human and AI translations of Arabic dialect Poetry. Advances in Artificial Intelligence and Machine Learning, 5(1), 3236–3260. https://doi.org/10.54364/AAIML.2025.51186
Alkhofi, A. (2025). Man vs. machine: can AI outperform ESL student translations? Frontiers in Artificial Intelligence, 8. https://doi.org/10.3389/frai.2025.1624754
Amaro, V., & Zhang, X. (2025). Intercultural interfaces: Artificial intelligence and the challenges of cultural sensitivity. E-Revista de Estudos Interculturais, 13.
Belhassen, S., Hakami, A., Alzobidy, S., & Hamda, A. (2025). Navigating the complexities of AI-driven literary translation: Challenges and perspectives across diverse user groups.
Chun Yin, W. (2024). Exploring creativity in ChatGPT and human translated literature. In Applying Technology to Language and Translation (pp. 117–140). Routledge. https://doi.org/10.4324/9781003399261-9
Dastjerdi, H. V., Hakimshafaaii, H., & Jannesaari, Z. (2008). Translation of poetry: Towards a practical model for translation analysis and assessment of poetic discourse. Journal of Universal Language, 9(1), 7–40. https://doi.org/10.22425/jul.2008.9.1.7
He, S., Ibrahim, N. A., & Kang, M. S. (2025). Bridging website communication and language: Translation quality assessment of Chinese medical texts on Malaysian medical tourism websites. Theory and Practice in Language Studies, 15(6), 1938–1948. https://doi.org/10.17507/tpls.1506.22
Hemmat, A. (2021). Hermeneutical translation of classics and cultures: The case of the I Ching and China’s inter-civilizational dialogue. Comparative Literature: East and West, 5(1), 15–28. https://doi.org/10.1080/25723618.2021.1940465
Hidalgo de Torralba Padrón, C. (2023). Artificial intelligence in translation and interpreting. 3(11), 70–76.
Karaban, V., & Karaban, A. (2024). AI-translated poetry: Ivan Franko’s poems in GPT-3.5-driven machine and human-produced translations. Forum for Linguistic Studies, 6(1). https://doi.org/10.59400/fls.v6i1.1994
Lefevere, A. (2016). Translation, rewriting, and the manipulation of literary fame. In Translation, Rewriting, and the Manipulation of Literary Fame. https://doi.org/10.4324/9781315458496
Lu, P., & Xu, F. (2025). The quality optimization of English–Chinese machine translation based on deep neural networks. Discover Artificial Intelligence, 5(1). https://doi.org/10.1007/s44163-025-00319-4
Manapbayeva, Z., Zaurbekova, G., Ayazbekova, K., Kazezova, A., & Pirmanova, K. (2024a). AI in literary translation: ChatGPT-4 vs. professional human translation of Abai’s poem ‘Spring’’.’ In S. E.E. (Ed.), Procedia Computer Science (Vol. 251, pp. 526–531). Elsevier B.V. https://doi.org/10.1016/j.procs.2024.11.143
McDonald, S. V. (2020). Accuracy, readability, and acceptability in translation. Applied Translation, 14(2), 21–29.
Nababan, H., Nababan, M. R., & Santosa, R. (2018). Translation techniques and their impact on the readability of translated bible stories for children. Humanus: Jurnal Ilmiah Ilmu-Ilmu Humaniora, 17(2), 212–222.
Nababan, M., & Nuraeni, A. (2012). Pengembangan model penilaian kualitas terjemahan. Kajian Linguistik dan Sastra, 24(1), 39–57.
Nababan, M. R., & Santosa, R. (2025). Translating for young minds: Techniques to ensure accuracy and acceptability in children’s Bible stories. Script Journal: Journal of Linguistics and English Teaching, 10(1), 94–107.
Naeem, A., ur Rehman, A. S., & Rasheed, A. (2025). Evaluating cultural adaptation in AI translations: A framework and implications for literary works. In AI Applications for English Language Learning (pp. 223–251). IGI Global. https://doi.org/10.4018/979-8-3693-9077-1.ch010
Peng, K. (2025). Human v.s. AI: A comparative study between MTI student translators and AI engines. In T. Hao, J. G. Wu, X. Luo, Y. Sun, Y. Mu, S. Ge, & W. Xie (Eds.), Lecture Notes in Computer Science: Vol. 15589 LNCS (pp. 169–180). Springer Science and Business Media Deutschland GmbH. https://doi.org/10.1007/978-981-96-4407-0_14
Pudjiati, D., Mulatsih, M. V. E., Ilham, I., Darta, D. M. S., & Suhel, F. R. (2024). Poetic translation and students’ appreciation through human translator and machine system. LLT Journal: A Journal on Language and Language Teaching, 27(2), 1049–1061. https://doi.org/10.24071/llt.v27i2.9299
Saehu, A., & Hkikmat, M. M. (2025). The quality and accuracy of AI-generated translation in translating communication based topics: Bringing translation quality assessments into practices. In Role of AI in Translation and Interpretation (pp. 237–265). IGI Global. https://doi.org/10.4018/979-8-3373-0060-3.ch009
Sahari, Y., Al-Kadi, A. M. T., & Ali, J. K. M. (2023). A cross sectional study of ChatGPT in translation: Magnitude of use, attitudes, and uncertainties. Journal of Psycholinguistic Research, 52(6), 2937–2954. https://doi.org/10.1007/s10936-023-10031-y
Yin, W. C. (2024). Exploring creativity in ChatGPT and human translated literature: A case study of The Old Man and the Sea in Chinese. In Applying Technology to Language and Translation (pp. 117–140). Taylor and Francis. https://doi.org/10.4324/9781003399261-9
Yuxiu, Y. (2024). Application of translation technology based on AI in translation teaching. Systems and Soft Computing, 6(January), 200072. https://doi.org/10.1016/j.sasc.2024.200072
Zhao, W., Huang, S., & Yan, L. (2024). ChatGPT and the future of translators: Overview of the application of interactive AI in English translation teaching. 2024 4th International Conference on Computer Communication and Artificial Intelligence, CCAI 2024, 303–307. https://doi.org/10.1109/CCAI61966.2024.10602989
Downloads
Additional Files
Published
How to Cite
Issue
Section
License
Copyright (c) 2025 Prof. Dr. Issy Yuliasri, M.Pd., Bambang Purwanto, S.S., M.Hum., Muhammad Zaki Pahrul Hadi, Agung Dwi Nurcahyo, Dr. Pryla Rochmahwati, M.Pd., Dr. Muhammad Rifqi, S.S., M.Pd., Kusuma Arum Diana Kumara, Fithriyatul Ma’sumah
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
Authors of LITE: Jurnal Bahasa, Sastra, dan Budaya must agree to the following terms:
- Authors retain copyright and grant the journal the right of first publication with the work simultaneously licensed under a Creative Commons Attribution-ShareAlike 4.0 International License that allows others to share the work with an acknowledgment of the work's authorship and initial publication in this journal.
- Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgment of its initial publication in this journal.
- Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) before and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work (See The Effect of Open Access).
