A Comparative Transitivity Analysis of Human vs. Artificial Intelligence-Based Translations: An Evaluation of an Abstract from Security Studies

Abstract

This study aims to compare and assess the system of transitivity analysis in translations produced by AI-based translation systems (ChatGPT and Google Translate) and human translation, examining their performance in translating a research abstract. To identify the transitivity process, Halliday’s transitivity system theory was adopted, highlighting critical discourse analysis. Due to the nature of data, a quantitative method was employed. Only material, relational, and verbal processes have been involved in the analysis. The findings revealed that ChatGPT Translation outperformed the other methods in highlighting material processes, while Human Translation was more effective for relational processes. Performance across all methods was largely consistent in the verbal category. In addition, in terms of quality, the identical n-gram counts across Human Translation, ChatGPT Translation, and Google Translate indicate that all three versions follow a similar pattern concerning linguistic structure. The BLEU score of 0.988 in this evaluation demonstrates high-quality machine translations, closely aligning with human reference. Future studies may address large number of abstracts that probably incorporate more process types with variable evaluation to gain more comprehensive understanding.