Artificial Intelligence or Human: The Use of ChatGPT in the Academic Translation for Religious Texts

Abstract

This article presents an empirical study investigating the ability of ChatGPT to translate religious texts involved in academic writing. Specifically, religious texts are not easy to translate even for human proficient translators. Three Arabic academic abstracts, religion-oriented, were involved in our study taken from AlQalam Journal which is published in Arabic and English. They have ready translations by the language adviser of the journal. The Arabic abstracts were first translated into English by ChatGPT. We then analyzed and compared ChatGPT translation to the human translation in terms of semantics and syntax. The former is represented by academic inappropriateness, failing to capture religious nuances, redundancy, translation gaps, and englishization, while the latter by first-person pronouns, to-infinitival vs. gerund use, passive versus active voice, full relative clauses, and indefiniteness. The findings revealed that ChatGPT’s performance is still far from human translation, and that ChatGPT translation needs significant human postediting. The study concludes with substantial recommendations for researchers and ChatGPT developers.

Introduction

The Neural Networking Algorithms (NNAs) have entered Artificial Intelligence (AI) sphere, leading to outstanding results. AI today creates much fear of what is going to happen in the future due to the fast and vast development of AI tools. It is not certain though, up to now, whether this could be true in the field of translation. NNAs are programmed to work in such a way that gives them the human brain power, which will, if true, be an incredible advancement in AI technology. However, it has been found in the literature that AI translation tools do not understand what is beyond text or the underlying meaning of a text, specifically a text that is embedded within culture such as poetry (Alowedi & Al-Ahdal, 2023), religious texts (Banat & Abu Adla, 2023).

Given the current high expectations about AI, this study first aims to examine whether ChatGPT can capture the Islamic religious nuances in religion-based expressions, beliefs, practices, prayers, and rituals involved in academic writing (Qarabesh et al., 2023). It focuses on such a type of text because it is widely argued that translating culture- and religion-based texts is one of the most difficult translation tasks encountered by human translators (see Katan, 2015; Qarabesh et al., 2023; Shormani, 2020; Sirriyya, 2009). Our second aim is to examine where ChatGPT fails to grasp Islamic nuances involved in academic writing. These nuances are aspects that are beyond actual text or wording. The paucity of studies on such a topic makes this study significant and relevant to the field as it serves to fill this gap. It tackles an important topic in the current hot debate about AI models’ power. It provides AI scientists and developers with valuable insights into how these models including ChatGPT can be developed by training them on massive (internet) data involving academic translation of Islamic rituals, prayers, repertoire, beliefs, traditions, nuances, and peculiarities. It also provides translators with valuable insights, underscoring the value of the postediting process given the fact that translation in AI world becomes a process where the translator’s role is just to postedit AI tools’ output.

This article is set up as follows. The section ChatGPT: Fast and Vast Development lays out the fast and vast development ChatGPT has witnessed. The section Translation and ChatGPT sheds light on how ChatGPT has been developed as a translation tool, spelling out the relationship between religious text, translation, and ChatGPT. The section The Present Study articulates the methodology of the study in terms of the approach employed, the data involved and the methods of analysis followed. The section Conclusion and Recommendations presents the results and discussion of the study. Section 6 concludes the paper and provides recommendations to AI specialists, specifically ChatGPT developers. The section Limitations and Further Research outlines the study limitations and suggestions for further research.

ChatGPT: Fast and Vast Development

AI or machine translation (MT) was first introduced into the field of translation in the 20th century, and its main aim was to replace human translators. It has been developed vastly since then (see e.g., Brown et al., 1993; Denkowski, 2015; Ketpun & Sripetpun, 2016; Tongpoon-Patanasorn & Griffith, 2020). MT started as Rule-based Machine Translation (RMT), Statistical Machine Translation (SMT), and finally Neural Machine Translation (NMT; Koehn et al., 2003; Zhang & Torres-Hostench, 2022; Dwivedi et al., 2023; Cotton et al., 2023). In RMT version, the method Google Translate first adopted was phrase-based method in which a phrase from SL is translated into TL, that is, English. However, this method was not successful, at least at the beginning but it has been improving and many modifications have been introduced into Google Translate’s working mechanism until reaching its present status utilizing NNAs. With the fast and vast developments AI has recently witnessed, these AI models including ChatGPT are expected to translate more accurately and appropriately.

The term ChatGPT is made of “Chat-Generative Pre-Trained Transformer.” ChatGPT is defined as “an intelligent chatting machine developed by OpenAI upon the InstructGPT, which is trained to follow an instruction in a prompt and provide a detailed response” (Jiao et al., 2023, p. 1, see also Ouyang et al., 2022). It is an AI model whose functionality depends on internet, meant not only for translation. Rather, it is used for several purposes and in many other fields such as creating content and debugging. OpenAI started developing GPT in 2018 (see e.g., Radford et al., 2019), and launched ChatGPT in 2022, as one of OpenAI’s most influential models of language learning (Dergaa et al., 2023). GPT has been developed in several stages, resulting in several models, namely GPT-2 introduced in 2019, GPT-3 in 2020, and GPT-4 in 2022 (see e.g., Dergaa et al., 2023; Jiao et al., 2023). Massive amounts of internet data have been trained on which enable ChatGPT to generate responses that appear to a great extent similar to human responses and in various and several fields and registers. According to Dergaa et al. (2023, p. 616), OpenAI has launched ChatGPT, “enhancing its ability to handle a wider range of queries and provide more accurate, relevant, and helpful responses to users” (see also van Dis et al., 2023; Dwivedi et al., 2023).

Such improvements of ChatGPT have been noted in several aspects like size of stored and training data, language “abilities,” expanded knowledge base, and higher accuracy and precision, which lead to what ChatGPT is all about today. These developments result in developing conversational AI models including Google’s Bidirectional Encoder Representations (Dergaa et al., 2023, see also Yogesh et al., 2023). Dergaa et al., for instance, argue that ChatGPT can write high quality essays for students, summarizing academic articles and answering “questions well enough to pass medical exams, and generate helpful computer codes”(p. 615). They also argue that ChatGPT can also write academic abstracts even highly qualified scholars do not find different from similar ones prepared by humans.

As for translation, several studies view ChatGPT as a translation tool paralleling other CAT such as Google Translate, Microsoft Translator or any other Internet-based translation models. In this sense, ChatGPT could be defined as an internet-based AI model. Though it was designed for translation purposes, it has been used for several other purposes. It consists of “two deep neural networks, namely the encoder and decoder” (Siu, 2023, p. 2). The former processes the source text and converts it into an internal representation, while the latter generates texts in the target language (TL) based on the prediction of “each subsequent target word” (Siu, 2023, p. 2).

Figure 1 presents the transformers ChatGPT consists of. These are called transformer blocks. From bottom-up, the first is Text and Position Embed. The second is Masked Multi Self Attention, and so on until reaching the Text Prediction and Task Classifier transformers, as is clear in Figure 1.

Figure 1.

ChatGPT Architecture.

Translation and ChatGPT

Translation is regarded as a human endeavor through which sciences, ideologies, experiences, knowledge, cultural elements, values, and all human activities are passed down from one generation to another (see Shormani, 2020, 2024). The task of defining translation has spanned over two centuries now, with various scholars offering definitions from different angles. These include linguistic perspectives (Catford, 1965), cultural viewpoints (Katan, 2015; Newmark, 1988; Nida, 1964; Shormani, 2020), and notions of equivalence (Nida, 1964), among other definitions and scholars. A century or so ago, Jakobson (1959, p. 233) defines translation as “an interpretation of verbal signs by means of some other language.”Catford (1965) defines translation as a human activity of much significance today. It has also attracted the interest of linguists, translators, and language teachers alike, and even mathematicians and engineers. In this definition, we notice that translation is important to all fields. Newmark (1988) defines translation more cautiously as rendering the meaning of a source text into the target text as intended by the author of the SL text. In this definition, however, we notice that Newmark is trying to include the author of the original text and his/her intentions.

Today, translation is viewed as more than simply what these theorists have argued for. Translation is considered to be more than brigading the culture gaps; it is a process whereby cultures and peoples (and texts) negotiate (Bassnett, 2014, see also Shormani, 2024). It is also a process where translators help mediate how several types of translation are performed. Translation aims to make peoples and nations get benefits from the experiences of other peoples. It also aims to bridge gaps between peoples worldwide. In this age of globalization, translation becomes a very essential process in several human endeavors such as intercultural communication, business, and trade. Additionally, the human role in translation becomes ever less than before. Translation now enters a new world, a technological world. It is no longer a difficult task. Translation as both process and product has made use of internet and AI models and tools. Several web applications have been developed and used worldwide. To mention just examples, consider Google Translate, Microsoft Translate, WordLingo, Babylon, all types of Computer-assisted translation (CAT) tools, and finally ChatGPT, let alone hundreds (perhaps thousands) of smartphone translation apps. The development translation has witnessed with the help of AI is tremendous. Huge data from any language can be translated in seconds. This is due to the fact that AI outlets like ChatGPT have been trained on massive internet data through artificial algorithmic neurons, which enable these outlets to translate any amount of written texts in seconds, whatever the language is (see e.g., Dergaa et al., 2023).

ChatGPT’s Translation Working Mechanism

Given that ChatGPT is trained on massive internet data, there should be a mechanism in which it works. Details aside, Figures 2 and 3 display ChatGPT’s working mechanism in translation. Figure 2 presents a Template showing a translation sample from Chinese into Romanian.

Figure 2.

Translation template.

Figure 3.

Translation prompt.

In Figure 2, there are two languages, where translation is from Chinese to Romanian and ChatGPT generates sentences for Chinese ⇔ Romanian. [Ro] represents the sentence in Romanian whereas [Zh] stands for the Chinese sentence.

Consider now Figure 3 which presents ChatGPT translation prompt, viz., input prompt, entered by a human “interlocutor.”

As is clear in Figure 3, there are “role,”“system” and “content.” These determine what ChatGPT is going to do. The “Request” is to “provide” the Target Generated Text [TGT] translation for the following sentence, which will be entered by the “user.” Requests include “Please provide the [TGT] translation for the following sentence step by step” to extract step-by-step translation. (Jiao et al., 2023, p. 7).

Many scholars (see e.g., Hendy et al., 2023; Jiao et al., 2023; Peng et al., 2023) have contended that ChatGPT’s performance is much better than other CAT tools like Google Translate and Microsoft Translator. Thus, ChatGPT’s success or failure might depend on the type of text. For example, ChatGPT might not be that successful in translating texts that are culture-based like proverbs, poetry or religion-based expressions like Islamic greetings, rituals, beliefs, and peculiarities, as in this study.

For these purposes, among others, and for easy access, AI tools such as Machine Translation, Google Translate, start to invade translation trying to replace human translators. Nowadays, translation is seen as a process resulting from Machine-human cooperation/collaboration. As has been stated earlier, AI plays a great role in non-human translation industry. These non-human translation machines have been developing dramatically vastly, specifically with the fast and vast development AI has undergone in recent years. However, AI-based translation tools have succeeded in technical translation, and nonliteral topics, in general. However, considering religious and cultural expressions and texts such as proverbs, idioms and idiomatic expressions, these seem to be difficult for CAT tools, in particular and AI tools like ChatGPT in general.

Translating Religious and Academic Texts

In this section, we briefly sketch translating religious-academic texts. To start with, religious texts have been viewed as not easy to translate, due perhaps to their idiosyncratic and metaphorical nature and differences between religions and cultures, viz. Source Language (SL), and TL cultures. Thus, these expressions are not easy to translate even by human proficient translators. Like culture-based texts, religious texts are also embedded in culture; these texts are heterogeneous in nature, including several religious rituals such as “divinity, jurisprudence, ethics, history, etc. …[and] sanctity” (Sirriyya, 2009, p. 15). For religious texts, there is more than one aspect involved, which is not easy to translate by humans, let alone AI models. Thus, if these texts are not easy to translate by human translators, it is expected that ChatGPT may not capture this very component of religious texts (cf. Charlesworth, 2012, Shormani, 2024).

In translating religious texts, two types of equivalence, viz. Dynamic Equivalence and Formal Equivalence were proposed by Nida and Charles (1969). In the former, the text produced in the TL should be as natural as possible to the SL, and convey both meaning and style. However, the latter aims to convey the “contextual meaning” from the SL to the TL. Nida (1964) considers formal equivalence or correspondence to be a substantial factor for translation to be successful. He believes that structure sometimes is more important than wording. Nida (1964, p. 57) gives us a very interesting example mirroring how this is important in several examples like the noun phrase old man. He opines that “the tolal meaning of the phrase is not signaled by the referential or emotive value of the isolated words old and man, but a part of the meaning is derived from the constructions itself.” He also adds that “the combination of attributive adjective and the noun head also possesses a meaning, namely that the first element qualifies the second. These constructions also include gray house, beautiful fur and tall tree labeling such phrases as a “quantifier-head phrase.” Thus, if such importance was given to just a simple noun phrase, what about complex phrases or sentences, and how syntactic properties are very important in rendering the meaning (see also Shormani, 2024).

As a translation approach, Nida proposes dynamic equivalence for its being capable of reproducing the overall effect and meaning of the source text in the TL (see also Goodwin, 2013). It entails conveying the original text’s intended meaning and effect, rather than relying on a literal or word-for-word (WFW) translation, thus giving naturalness of the TL, both language and culture, to the SL text. However, formal equivalence prioritizes a more literal and WFW translation. The emphasis is on closely mirroring the original text’s structure, syntax, and vocabulary, even if this comes at the expense of naturalness or idiomatic expression in the target language. The goal is to provide a faithful representation of the source text’s linguistic form and maintain its cultural and historical context. These two approaches represent a spectrum of translation strategies with dynamic equivalence emphasizing the target language’s naturalness and readability. However, formal equivalence stresses fidelity to the source text’s linguistic form. Translators often employ a combination of these approaches depending on the specific translation task, target audience, and the purpose of the translation.

As for translating academic texts, there are several studies investigating this topic. We will review five studies here. The first is Kovacs (2019) who argues that translating academic texts is “challenging in several respects, as it is an accurate, standardized, normative language form, the use of which requires thorough knowledge and experience from the part of the translation” (p. 28). Kovacs studies the translator trainees’ awareness of the characteristics of academic texts and how they could encounter these difficulties. She concludes that the trainees committed several serious types of errors, recommending that academic writing and translation require us to raise language skills to the necessary level. She adds that it is also necessary to teach “the characteristics of academic style” of these texts before translating them (p. 28).

The second study was conducted by Kiriman and Kapu (2023), investigating the strategies used by translators in translating academic texts, specifically academic abstracts. It draws on some problems encountered by translators and seeks some solutions. They argue that translating academic texts becomes inevitable in translation industry. Thesis abstracts are usually written in one language and are necessary to be translated into another language in such cases as in Bahasa Indonesia and translating them into English, for instance, or vice versa. They find a number of translation solutions including how to copy structures, change perspectives and density compensation, correspond culturally, and tailor texts.

The third study we will review here was conducted by Shleykina and Junnier (2021), who studied translating authorial presence with reference to first-person pronouns in research article abstracts. They studied cross-linguistic variation in academic discourse and how first-person pronouns are translated in three sub-corpora of academic abstracts. Their analysis was in terms of three factors, namely (a) abstracts written in Russian; (b) English versions of these abstracts, and (c) abstracts written by biologists in English published in the same journals. They conclude that L1 transfer of linguistic code and author agency and presence dominate research sphere.

The fourth study to review here was conducted by Tongpoon-Patanasorn and Griffith (2020). This study is somehow close to ours in subject matter and in using an AI model, viz. Google Translate. They translated 54 Thai thesis abstracts selected from several Humanities and Social Sciences disciplines. They conclude that academic writing requires a different register, a specific language that is used in academic theses, and in academic writing in general. They also found that Google Translate involves several types of errors, the most frequent of which are those pertaining to the use of academic terms, punctuation, capitalization, and fragmentation.

The final study to be reviewed here was conducted by Banat and Abu Adla (2023). The authors conducted a qualitative study to examine GPT-3’s ability to translate religious texts. They found that BERT model’s accuracy was 83%. They also found that in general GPT-3 can produce “understandable translations,” but it cannot capture the cultural nuances. They conclude that ChatGPT needs further development to overcome the cultural as well as religious challenges, recommending further research in this aspect. Based on this account of previous studies, this study strives to answer three major questions:

1. To what extent can ChatGPT translate Arabic religious texts into English?

2. Can ChatGPT translate religious concepts found in academic writing?

3. To what extent does ChatGPT deviate from human translation, and how can this deviation be categorized?

The Present Study

Study Data

Recall that this study aims to examine to what extent ChatGPT can translate Arabic academic-religious texts into English, and grasp religion- and culture-based concepts in our data. The sample of the study contains three abstracts of religious papers published in Arabic and English by AlQalam Journal:

1. رفع الملامة عن أبي شامة تأليف العلامة محمد بن عبد القادر الأهدل (ت1326هـ): دراسة وتحقيق

المنجيات والمهلكات وطرق علاجها من سورة الحجرات

“The Munjiat and Muhlikat and ways of handling them from Surat Al-Hujrat” (henceforth, MAHS, see here).

أحاديث الشؤم ومسالك العلماء فيها: دراسة عقدية

“Hadiths of pessimism and scholars’ paths in them: a doctrinal study” (henceforth, HPAS, see here).

These three academic abstracts were selected for the following reasons: (i) they are published recently, 2023, (ii) they involve texts deeply rooted in Islamic religiousness, from the Holy Qur’an and Hadith, (iii) they are suitable for micro analysis as involving several linguistic deviations, and (iv) they have ready translations.

Methods of Analysis

This study is qualitative in nature. For the scope and purpose of this study, we employed comparative and analytical methods, adopting micro analysis appropriate for analyzing linguistic data in terms of syntactic and semantic features/properties (cf. Palumbo, 2009). Since the abstracts involved in the study have ready English translations by the language adviser of the journal, they were again translated by ChatGPT, and then checked for syntax and semantic aspects. After the ChatGPT translation, we tabulated both human and ChatGPT translations (see Tables, 1–3), to compare both sets of translations, and then analyze them in terms of semantics and syntax. As for semantics, we focus on issues related semantic deviations where ChatGPT fails to provide faithful semantic renderings. The same procedure was applied with regard to syntactic analysis. These issues among others will be the focus of the section to follow.

Table 1.

First Article Abstract (RAAS).

Arabic text	ChatGPT translation	Human translation
ملخص البحث	Summary of the research	Abstract
رفع الملامة عن أبي شامة تأليف العلامة محمد بن عبد القادر الأهدل (ت1326هـ): دراسة وتحقيق	Clearing the Blame from Abu Shama by the scholar Muhammad bin Abdul Qadir Al-Ahdel (1326 AH). It is a study and investigation	“Raf’ Almalama ‘an Abi Shamah” by Mohammed Ibn Abdulqadir AlAhdal (D. 1326 AH): a study and investigation”
يهدف البحث الذي بين أيدينا إلى دراسة وتحقيق رسالة مخطوطة	The research at hand aims to study and verify a manuscript	“This article aims to study the manuscript”
بعنوان: (رفع الملامة عن أبي شامة) تأليف: العلامة محمد بن عبد القادر الأهدل (ت1326هـ).	entitled “Rafa Al-Malamah ‘an Abi Shammah” by the scholar Muhammad bin Abdul Qadir Al-Ahdal (d. 1326 AH).	“entitled “Raf’ Almalama ‘an Abi Shamah” by Mohammed Ibn Abdulqadir AlAhdal (D.1326AH”
من خلال إخراج النص إخراجًا سليمًا،	The text is to be accurately extracted,	“It does so by producing the text accurately,”
وإعطائه حقه من التوثيق والضبط والدراسة والتعليق.	authenticated, studied, and commented upon.	“via documenting, studying and commenting on it.”
تبحث هذه الرسالة في علم مشكل القرآن،	The text delves into the science of the complex verses of the Qur’an,	“The study investigates the Ayats, which involve ambiguity,”
أي: الآيات القرآنية التي التبس معناها	that is, the Qur’anic verses with obscure meanings
واشتبه على الناس،	that are often misunderstood by people	“and whose intentions were not clear to people”
فلم يُعرف المراد منها إلا بالطلب والتأمل.	that is, the Qur’anic verses with obscure meanings that are often misunderstood by people and require further inquiry and contemplation to understand their intended meaning.	“unless being carefully investigated.”
وفي هذه الرسالة قام المؤلف بالانتصار بالحجة لأبي شامة في تأويله لأحد المواضع المشكلة في القرآن الكريم،	In this text, the author defends Abu Shammah’s interpretation of a problematic verse in the Qur’an	“Thus, the study successfully defends Abu Shama’s interpretation of an ambiguous position in the Holy Qur’an,”
ردًّا على العلامة محمد بن يوسف جَدِّي	and refutes the claims made by the scholar Muhammad bin Yusuf Jaddi	“refuting the claims by Mohammed Ibn Yusuf Jaddi”
الذي ادَّعى أنَّ تأويله غير كافٍ.	that his interpretation was insufficient.	“that his interpretation is insufficient.”
وعرض المؤلف في رسالته بأسلوب دقيق موجز غير مخل الإجابة عن ذلك الإشكال.	The author presents a concise, accurate, and thorough explanation of this issue,	“The author presents a concise and accurate response to this issue,”
معتمدًا على القواعد الأصولية، والعقدية، والأدلة العقلية، والنقولات الأصلية التي تجلي ذلك الموضوع بشكل واضح وبَيِّن	relying on fundamental principles, beliefs, rational evidence, and original sources that clarify the matter.	“relying on foundational principles, doctrine, rational evidence, and original scholarly opinions, which elucidate the subject under study”
وقد اعتمدت في هذا البحث المنهج التاريخي، والمنهج الوصفي التحليلي.	This research adopts a historical and descriptive-analytical approach	“The study employs the historical, descriptive and analytical approac"”
واقتضت طبيعة البحث	—	—
أن أقسمه إلى ثلاثة أقسام	It is divided into three parts	“The study is divided into three sections”
جعلت القسم الأول لدراسة المؤلف	the first part focuses on the author’s background	“the first section studies the author"
والقسم الثاني لدراسة الرسالة المخطوطة	the second part discusses the manuscript itself	“the second section studies the manuscript”
والقسم الثالث للنص المحقق	and the third part focuses on the verified text	“and the third section presents the verified text”
وفي الخاتمة لخصت أهم النتائج والتوصيات، وكان من أهمها:	The conclusion summarizes the most important findings and recommendations, including:	“It concludes with the following findings: ”
عنيت الرسالة بتوجيه أحد المواضع المشكلة في القرآن الكريم،	The manuscript addresses a problematic verse in the Qur’an	“The manuscript tackles an ambiguous position in the Holy Qur’an,”
وكشفت عن أسرار القرآن في تعبيره ونظمه.	and reveals the secrets of its expression and organization.	“and reveals the secrets of the Qur’an in its expression and organization.”

Table 2.

Second Article Abstract (MAHS).

Arabic text	ChatGPT translation	Human translation
ملخص البحث	Summary of the research	Abstract
المنجيات والمهلكات وطرق علاجها من سورة الحجرات	Types of things that bring salvation and destruction, and ways to address them according to Surah Al-Hujurat	“The Munjiat and Muhlikat and ways of handling them from Surat Al-Hujrat”
يهدف البحث إلى جمع الآداب والأوامر في سورة الحجرات في مكان واحد	This study aims to gather the manners and commands mentioned in Surah Al-Hujurat in one place	“This article aims to examine the morals and orders in Surat Al-Hujrat in a single work”
وتفسيرها موضوعيًّا	to objectively interpret them	“objectively interpreting them gerund”
والاستفادة منها	benefit from them	“and how to benefit from them”
والتعرف على آراء المفسرين وأقوالهم	and recognize the opinions of the interpreters and their statements regarding the topic of the verses	“It also aims to clarify these provisions”
خاصة المنهي عنها	especially those that are prohibited	“specifically those forbidden”
لتجنب الوقوع فيها	to avoid falling into them	“to avoid committing them”
واستخدم الباحث المنهج الاستقرائي الموضوعي	The researcher used an inductive and objective method	“The study employs the inductive objective approach”
وذلك باستقراء الآيات في سورة الحجرات	by examining the verses in Surah Al-Hujurat	“through deducing the verses in Surat Al-Hujrat”
والتي تتحدث عن هذه الآداب	that speak about these manners	“which tackle these morals and orders”
وجمع المعلومات حولها	and gathering information about them	“It collects the data about the issues in question”
من أمهات كتب التفسير	from the sources and references of interpretation books	“from different reliable interpretation references and books”
والمنهج الوصفي	—	“the descriptive approach”
وذلك بنقل كلام المفسرين عن الموضوع	—	“through taking the interpreters’ opinions”
والتعليق عليها	The researcher also commented on them	“commenting on such opinions”
ما أمكن ذلك	—	“where possible”
وخلص الباحث إلى عدة نتائج، من أهمها:	and drew several conclusions, including:	“The study concludes with a number of findings including:”
اشتملت سورة الحجرات على مجموعة من الأوامر	Surah Al-Hujurat includes a set of commands	“Surat Al-Hujrat contains a number of orders”
التي يجب على المسلم أن يلتزم بها	that Muslims should adhere to	“that Muslim should abide by”
المنجيات المذكورة في السورة	The types of things that bring salvation mentioned in the Surah are	“The Munjiat mentioned in Surat Al-Hujrat are:”
الوقوف عند أمر الله تعالى وأمر رسوله صلى الله عليه وسلم،	standing firm in obedience to Allah and His Messenger	“abiding by Allah’s and His Prophet’s (PBUH) orders”
والتثبت عند سماع الأخبار ونقلها،	verifying news before conveying it	“making sure of what is heard and delivering it”
والحفاظ على الرابطة الإيمانية بالإصلاح بين المتخاصمين	maintaining the Islamic bond of brotherhood and reconciliation between disputing parties	“maintaining faith bond in reconciliation”
وبيان أن معيار التفاضل بين الناس هو التقوى	and emphasizing that the criterion of distinction between people is based on piety	“It is also emphasized that piety is the only means of preference”
المهلكات المذكورة في السورة فهي	The types of things that lead to destruction mentioned in the Surah are	“The Muhlikat mentioned in Surat Al-Hujrat are:”
السخرية والاستهزاء، والهمز واللمز والتنابز بالألقاب، وسوء الظن، والتجسس، والغيبة.	mockery and ridicule, backbiting and slandering, suspicion, spying, and gossiping.	“ridicule, mockery, gossip, mistrust, spying, and backbiting”

Table 3.

Third Article Abstract (HPAS).

Arabic text	ChatGPT translation	Human translation
ملخص البحث	Summary of the research	Abstract
أحاديث الشؤم ومسالك العلماء فيها: دراسة عقدية	The narrations about superstition and the approaches of scholars toward them constitute a theological study.	“Hadiths of pessimism and scholars’ paths in them: a doctrinal study”
يهدف هذه البحث للتعرف على مفهوم الشؤم	This research aims to understand the concept of superstition	“This article aims to identify the concept of pessimism”
من حيث اللغة والاصطلاح	in terms of language and terminology	“linguistically and terminologically,”
والوقوف على الأحاديث الواردة فيها من كتب الصحاح والسنن والمسانيد	and to examine the narrations related to it in the books of authentic traditions, collections, and chains of narrators.	“studying the Hadiths about it from Hadith Śahīhs, Sunan and Masānīd”
والتي يُرى في ظاهرها التعارض	These narrations appear to have conflicting aspects	“which appear contradicted.”
لأن منها: ما جاء في إثبات الشؤم والطيرة، ومنها:	as some of them affirm superstition and omens,	“This is because some of them prove evil omen and pessimism”
ما جاء في نفيهما،	while others negate them.	“and some others disapprove these concepts,”
فاختلفت لأجله أنظار العلماء في التوفيق بينها	This has led scholars to differ in reconciling between these narrations.	“which has led to non-consensus among scholars”
فكان هذا البحث جامعاً لمسالك العلماء في التعامل مع هذه الأحاديث الواردة في هذا الباب	Therefore, this research collects the approaches of scholars in dealing with these narrations	“This study serves to highlight the scholars’ paths in dealing with these Hadiths”
والتي تلخصت في أربعة مسالك واتجاهات	and summarizes them into four paths and directions	“which are summarized in four paths and directions”
والتي يتبيَّن من خلالها التعرف على معاني الأحاديث الواردة في هذا الباب	through which the meanings of the narrations in this topic can be understood	“through which the meanings of the Hadiths were explained”
وشرح جملها،	their phrases explained,	-----------
والاطلاع على أقوال الأئمة والعلماء فيها	and the opinions of scholars and imams in this regard explored.	“and scholars’ opinions were clarified”
ثم خُتم هذا البحث بذكر أهم النتائج فيه	Finally, this research concludes by mentioning its most important findings	“The study concludes with a number of findings”
والتي تلخَّصت في خمس نقاط رئيسية:	which can be summarized in five key points:	“the most important of which are:”
أنَّ مفهوم الشؤم والطيرة مأخوذ من فعل أهل الجاهلية،	Firstly, the concept of superstition and omens is derived from the practices of the pre-Islamic era.	“The concept of evil omen and pessimism is taken from pre-Islamic paganist life.”
الثانية: أنَّ فعل التشاؤم التطير شرك؛	Secondly, the act of superstition and seeking omens is considered a form of polytheism	“Pessimism and evil omen are considered polytheism”
لما يصحبه من اعتقاد فاعلوه من جلب النفع ودفع الضر من غير الله تعالى	due to the belief of those who engage in it that it brings benefit or wards off harm independently of Allah.	“due to believing that its wrongdoers bring benefit and save people from harm.”
الثالثة: التعرف على الأحاديث الواردة في الشؤم والطيرة،	Thirdly, recognizing the narrations related to superstition and omens reveals	“Identifying the Hadiths about the evil omen and pessimism”
وأنها ليست على اتجاه واحد،	that they are not consistent in one direction	“as they are not similar”
بيان تعامل العلماء مع ذلك الاختلاف	Fourthly, clarifying how scholars deal with the variations	“Explaining the scholars’ dealings with the contradiction”
الوارد في أحاديث الشؤم والطيرة،	found in the narrations about superstition and omens.	“contained in the Hadiths of evil omen and pessimism”
الخامسة: ذكر الاتجاه الراجح عند العلماء.	Fifthly, mentioning the prevailing opinion among scholars in this matter.	“Stating the most preferable opinion of scholars.”

Results and discussion

In this section, we first present the results we have come up with and then discuss them in details.

Results

The results are first tabulated in terms of Arabic text, ChatGPT translation, and Human translation in the following Tables:

Discussion

As we have alluded to above, our analysis is based on the criteria below:

1. Semantics

2.   b. Failing to capture religious nuances

3.   c. Redundancy

4.   d. Translation gaps

5.   e. Englishization

Syntax

  a. First-person pronouns

  b. To-infinitival vs. gerund use

  c. Passive vs. active voice

  d. Full relative clauses

  e. Indefiniteness

Semantics

As for semantics, we consider issues like how much ChatGPT translation deviates from human translation in terms of the criteria outlined in (1) above.

Academic inappropriateness: What we mean by this category is translations that are academically obscure/inappropriate. The first case in point here is translating the phrase “ملخص البحث” as “the research summary,” in the three abstracts (see Tables 1–3). However, Human translator translates it as Abstract, which is the common academic usage. ChatGPT translation seems to be Word-For-Word (WFW), while the human translator relies on a content translation (e.g., Bassnett, 2014; Newmark, 1988), a translation which is academic in nature.

If we look also at the translation of “تبحث هذه الرسالة في علم مشكل القرآن،,” we find that there are two problematic issues: the first concerns translating the verb “تبحث.” ChatGPT translates it as “delves” while human translates it as “investigates.” Once more, human translation seems to be academic while ChatGPT is WFW. The verb “delve” is not as appropriate as “investigate” here. ChatGPT translates the term “الرسالة” as “the text” while human as “study,” which is more appropriate and accurate from a research perspective.

Thus, the academic field is a crucial and independent register that needs to be paid much attention by human translators in the postediting process. Academic writing including scientific and research articles is different from any other type of writing. It has its own distinct rules, mechanics and procedures, and ChatGPT failed to grasp this specific register. In this aspect, our study agrees with Tongpoon-Patanasorn and Griffith’s (2020) study, though the AI model is different. While they used Google Translate, we utilized ChatGPT.

Failing to capture religious nuances: The second category is ChatGPT failure to capture religious nuances, sacred expressions, ritual words/phrases. To provide an example, consider how ChatGPT translates the sentence “لتجنب الوقوع فيها.” It translates it as “to avoid falling into them.” However, human translates it as “to avoid committing them.” As a religious term “الوقوع,” means “committing,” and not “falling” in this context. The Arabic term “فيها” refers to bad deeds, that is, sins, and hence the suitable English term to translate “الوقوع” as “committing.” Another aspect is translating the Arabic term “أمهات”; while ChatGPT translates it as “sources and references,” human translates it as “reliable.” In religious discourse, “أمهات” concerns reliable references/books.

The word “رسوله” is also intriguing, while ChatGPT chose a general word: “messenger,” human translation picked a word that conveys a more specific meaning for being a messenger of God, which is “prophet.” Another example is ChatGPT translation of the term "احاديث" which was translated as “narrations” throughout this abstract. This could be attributed, among other things, to the idea that ChatGPT does not recognize the plural of “حديث,” which is “أحاديث” ‘Hadiths’ and takes it as an ordinary term, not a religious one. This is also almost similar to the translation of the Arabic term “نفيهما.” It translates it as “negate them,” which is not appropriate, simply because these are not sentences, but rather concepts and notions, occurring in Hadiths.

Redundancy: We mean by this category translations that are not precise and concise. This is clear if we consider ChatGPT translation of “واشتبه على الناس، فلم يُعرف المراد منها إلا بالطلب والتأمل” which has been translated as “that are often misunderstood by people. That is, the Qur’anic verses with obscure meanings that are often misunderstood by people and require further inquiry and contemplation to understand their intended meaning.” This translation involves many unneeded details, while human translation does so as “and whose intentions were not clear to people, unless being carefully investigated,” something precise and concise. This cannot be considered “translation by addition” by ChatGPT, as it exceeds the translation purpose here, and it is also challenging to follow, specifically in translating this type of text. Furthermore, the expression “that is, the Qur’anic verses with obscure meanings” which was repeated by ChatGPT twice in two subsequent boxes (see Table 1). Human translation captured this and hence left one, viz, the second box empty. Thus, the best way to describe this is redundancy.

Translation gaps: What we mean by this category is words/phrases/sentences left untranslated by ChatGPT, though translation here cannot be dispensed with. There are several examples of this sort. For example, the Arabic phrase “والمنهج الوصفي” in Table 2, which ChatGPT left untranslated. The importance of translating this phrase ensues from the fact it is an academic approach employed in the study, and if left untranslated then, there would be something important missing. Another example would be “وذلك بنقل كلام المفسرين عن الموضوع,” which again was left untranslated by ChatGPT, though it is a complete sentence.

Another example is related to what Muslims glorify. For example, Muslims glorify the Qur’an and often (if not always) co-utter the word “Holy” with the “Qur’an” “القرآن الكريم.” The word “كريم” is mentioned in the Arabic text, but the English translation of ChatGPT ignores the word completely and uses “Qur’an” only without this adjective. A further example of a religious aspect ChatGPT ignores is “صلى الله عليه وسلم” “Peace be upon him”)see Table 2). In Islam, when the prophet Mohammed is mentioned, his name ought to be followed by this sentence to differentiate him from any ordinary person, but ChatGPT ignores the whole sentence, which was mentioned in the human translation.

Englishization: This category involves positions where translations rendered may not be appropriate, and the best translation of which is to englishize the Arabic terms. We will just give some examples for this category. To begin with, consider the translation of the Arabic title of MAHS: “المنجيات والمهلكات وطرق علاجها من سورة الحجرات.” ChatGPT translates it as “Types of things that bring salvation and destruction, and ways to address them according to Surah Al-Hujurat” while human translates it as “The Munjiat and Muhlikat and ways of handling them from Surat Al-Hujrat.” In other words, while ChatGPT translates it into English words, human translates it by englishization. In human translation, we find englishization of the terms “Munjiat and Muhlikat,” that is, entering these two words into English language.

Language contact, say, between English and Arabic has resulted in many Arabic religious terms entering the English lexicon such as subhan Allah, Ayat, Surat, Allah, maasha’allah, Hadith, among others, becoming English words/phrase now (see also Shormani, 2024). Yet, ChatGPT fails to capture some religion-based terms, such as “الصحاح والسنن والمسانيد,” which were translated as “authentic traditions, collections, and chains of narrators,” which are not only inappropriate but also incorrect. Human translation seems to be more specific, accurate, and appropriate in rendering these terms as “Śahīhs, Sunan and Masānīd.” The term Śahīhs means a book narrated by Al-Bukhari and Muslim, two very influential figures in the world of Hadith. Sunan refers to the sayings and deeds of the Prophet Mohammed’s (PBUH), and His companions, and Masānīd refers to those trustworthy people who transmit Hadiths to generations.

Syntax

Syntax and linguistics in general are a crucial aspect in translation (see e.g., Catford, 1965; Nida, 1964; Nida & Charles 1969). In this section, we examine whether syntax and syntactic notions are maintained by ChatGPT translation. Syntactically, there are several issues that we want to discuss here regarding the difference between ChatGPT and human translations. However, due to space we will just examine some aspects which have several examples, and to what extent ChatGPT maintains the syntax of the SL, and how it deviates from that of the TL. Thus, our discussion is confined to first-person pronouns, to-infinitival vs. gerund use, full relative clauses, passive vs. active voice and indefiniteness.

First-person pronouns: ChatGPT seems to ignore first-person pronouns. A case in point here is translating “أن أقسمه إلى ثلاثة أقسام,” which is written with a singular pronoun, but ChatGPT translates it as “It is divided into three parts” with a passive voice to avoid the singular pronoun. ChatGPT may have been made aware of the traditional way, that is, not using the pronoun I or We, but using “the research/researcher” instead. The same thing can be said about “جعلت” where the singular pronoun is embedded in the sentence, but it was translated as “the first part focuses on,” and again “اعتمدت” was translated to “This research adopts,” where the first singular pronoun I is not used. Here, ChatGPT translation does not deviate much from human translation (Table 1).

To-infinitival vs. gerund use: ChatGPT also tends to use to-infinitival constructions more than gerund such as “to objectively interpret them” in translating “وتفسيرها موضوعيًّا,” which is a gerund (see e.g., Shormani, 2013). Human translation of the same Arabic phrase is “objectively interpreting them” where gerund, but not to-infinitival, is used to maintain the SL syntax. The following sentences have also several gerunds, where all of them are again rendered as to-infinitive: “والاستفادة منها” to “benefit from them,” “والتعرف على آراء المفسرين وأقوالهم” to “and recognize the opinions of the interpreters and their statements.” A further example is observed in translating “يهدف البحث إلى جمع الآداب” as “This study aims to gather the manners,” where “جمع” was translated as “to gather.”

Full relative clauses: A reduced relative clause is a clause whose relative pronoun is not explicitly mentioned, where the relative pronoun which/that and the verb be are elided (see e.g., Shormani, 2013). ChatGPT seems to use full relative clauses, such as translating the Arabic adverbial phrase “خاصة المنهي عنها” as “especially those that are prohibited,” while the human translation prefers reduced relative clauses (specifically those forbidden). Another example is translating “المنجيات المذكورة” as “The types of things that bring salvation mentioned,” as well as translating “المهلكات المذكورة” as “The types of things that lead to destruction” (see Table 2). In all these cases, we find that while ChatGPT prefers full relative clauses, human prefers reduced relative clauses.

Passive versus active voice: In addition, ChatGPT prefers passive structures rather than active ones. An example of this is translating “من خلال إخراج النص إخراجًا سليمًا، “ by ChatGPT as “The text is to be accurately extracted” while Human translates it as “It does so by producing the text accurately.” Although the translation produced by both ChatGPT and Human is appropriate and accurate, it is recommended that a target text produces an equivalent syntactic structure (see Catford, 1965; Goodwin, 2013; Nida, 1964; Nida & Charles, 1969), and here the source text is in active voice.

Another example of preferring passive is translating the Arabic terms “وإعطائه حقه من التوثيق والضبط والدراسة والتعليق” as “authenticated, studied, and commented upon,” respectively, where we find four active Arabic terms are translated into passive ones. There are several examples of such usage, where ChatGPT might use the passive and active structure to translate the same verb. Consider the Arabic verb “تلخصت” which could be categorized as an unaccusative/ergative verb from the triliteral root ل خ ص “to summarize,” but not passive (e.g., Shormani, 2017). The Arabic verb تلخصت was translated as “summarizes” in one instance and as “can be summarized” in another. The first translation is active, specifically transitive/accusative, and the second is passive.

A further example is that ChatGPT translates “وشرح جملها،” as “their phrases explained.” Although the SL text is an active sentence, it was translated as passive. The absence of the verb be is excluded by ChatGPT along with others following it, specifically in “and the opinions of scholars and Imams in this regard explored,” which is another use of passive instead of active. The be was mentioned only once in “the meanings of the narrations in this topic can be understood,” which precedes “their phrases explained.” However, these sentences were translated as active sentences by human translation (see Table 3).

Indefiniteness: ChatGPT seems to favor indefiniteness of the SL text. To give an example, consider “رسالة مخطوطة” which has been translated by ChatGPT as “a manuscript” while human translates it as “the manuscript.” Here, it seems that human considers the context in which this Arabic phrase occurs. A further example that could be taken as favoring indefiniteness by ChatGPT is translating definite nouns as indefinite ones. For example, ChatGPT translates definite nouns such as “وقد اعتمدت في هذا البحث المنهج التاريخي، والمنهج الوصفي التحليلي” as “This research adopts a historical and descriptive-analytical approach,” where all the definite nouns were translated as indefinite ones, while human translates them as “the historical, descriptive and analytical approach” (see Table 1).

To recapitulate, it seems that ChatGPT rendering deviates considerably from both the syntax and semantics of the SL. We think that the deviations observed in ChatGPT translation may be ascribed to the fact that the training data are still not enough, that is, the training data do not include such Islamic nuances and peculiarities. Hence, ChatGPT fails to transfer the intended meaning and sense. As for syntax, in Abstract 1 the author of the Arabic Abstract intends to express a subtitle, that is, “دراسة وتحقيق,” which is a phrase, but ChatGPT translates it as a sentence. According to Nida (1964) and Nida and Charles (1969), this rendering violates what they term as formal equivalence or formal correspondence as opposed to dynamic equivalence (see also Goodwin, 2013). In terms of semantic equivalence, Nida proposes dynamic equivalence emphasizing the TL text’s naturalness and readability, as in translating “ملخص البحث” as “the research summary,” where TL naturalness is not maintained.

Conclusion and Recommendations

As we have seen in the course of this article, most religious concepts, beliefs, rituals, and respected entities were difficult to translate by ChatGPT. A religious expression may have more than one interpretation due to being subject to various interpretations within their respective religious communities. It seems that ChatGPT cannot select the most appropriate interpretation while maintaining a balance between both SL and TL text’s faithfulness and the needs of the target audience. An important aspect of religious texts that should be addressed here is that these texts often deal with abstract and metaphysical concepts that are challenging to express in concrete terms. These are what we may call transcendence and spiritual concepts (Sirriyya, 2009). Thus, translating these concepts accurately while preserving their depth and spiritual significance can be a difficult task for ChatGPT. These findings are in line with those of previous studies (e.g., Banat & Abu Adla, 2023).

Religious texts such as the Bible and the Holy Qur’an involve complex and classical language, idiomatic expressions, and specialized terminology. Translating these texts requires a deep understanding of the original language, its historical context, and cultural/religious nuances. Additionally, religious texts are deeply rooted in specific cultural and historical contexts. They involve using figurative language such as metaphors, religious symbols that may not have direct equivalents in the TL or its culture. Translators are needed to bridge the cultural and historical gaps left in ChatGPT translation to convey the intended meaning accurately. In this line of thought, it should be emphasized that Islamic texts are considered sacred by Muslims and are believed to originate from divine inspiration or revelation.

Another aspect where ChatGPT translation needs postediting concerns academic disciplines. Given the ChatGPT’s flaws concerning the academic aspects such as translating “ملخص البحث” as “the research summary,” which should be translated as “abstract,” human translators need to check these aspects, and postedit ChatGPT translation to end up with accurate and appropriate translation. According to a number of scholars (e.g., Kovacs, 2019; Kiriman & Kapu, 2023; Shleykina & Junnier, 2021; Tongpoon-Patanasorn & Griffith, 2020), the academic field is a crucial and independent register that needs to be paid much attention to by human translators in the translation process. Academic writing including scientific and research articles, Theses, and Dissertations, is different from any other type of writing. It has its own distinct rules, mechanics and procedures. Our findings in this juncture can be contrasted with those reached by Shleykina and Junnier (2021), and Tongpoon-Patanasorn and Griffith (2020).

Given these problems, we propose that ChatGPT translation needs human translators in what is so called “postediting” (see e.g., de Almeida & O’Brien, 2010; Groves & Schmidtke, 2009; Krings, 2001; O’Brien, 2007, 2012).O’Brien (2012, pp. 197–198), for instance, defines postediting as “the correction of raw machine-translated output by a human translator according to specific guidelines and quality criteria.” In this way, there would be a “complementation” between ChatGPT and Human in translation activity (see also Shormani, 2024). Given the flaws we have observed in this study, human postediting seems essential, enabling ChatGPT to translate religious texts involved in academic writing properly, hence minimalizing nonsense and incorrectness (see also Cotton et al., 2023). Thus, translators are needed to postedit ChatGPT output, reconsidering these issues to transfer both meaning and sense of the religious and cultural nuances of the SL text into the TL.

In addition, developers of ChatGPT may also incorporate into its training data Islamic religious and ritual practices, historical events and cultural aspects in an attempt to conceptualize ChatGPT’s performance. This is due to the fact that Islamic religion-based expressions are not separable from Islamic culture, thus giving ChatGPT enough room to familiarize itself with such data in the training process. ChatGPT developers could collaborate with religion scholars, jurisprudence scholars, interpreters of the Holy Qur’an, or the Bible scholars, providing valuable insights into the religious context, interpretations, and nuances essential for accurate religious translation. Their expertise can help guide the development and evaluation of the AI model’s religious translation “abilities.” Moreover, ChatGPT specialists should ensure that ChatGPT respects the sacredness, sensitivity, and cultural diversity related to religious texts, to avoid generating inappropriate translations. This was noted in our study, in that ChatGPT renders “القرإن الكريم” as “the Qur’an” and not as “the Holy Qur’an,” as well as “الرسول صلى الله عليه وسلم,” which was translated as “the prophet Mohammed” without mentioning “peace be upon Him,” among others. Finally, ChatGPT developers need to consider the englishization, inconsistency, redundancy, etc. in ChatGPT translation so that it can produce precise and concise translations better than what it does now.

Limitations and Further Research

Although the study has come up with interesting findings, it has certain limitations. The first limitation concerns tackling only academic “abstracts” involving religious texts. A broader study could include full articles dealing with religious and academic texts to understand the nature of insufficiency in ChatGPT’s working mechanism and training data so that ChatGPT developers target the deficits in translating religious and academic texts. Another limitation is related to the number of texts and journals: the study involves only three abstracts and one journal. A comprehensive study could include more abstracts/articles and more than one journal to broaden its scope and purpose. This study is also limited to qualitative analysis, that is, qualitatively discussing the difference between ChatGPT and human translations, and the type of postediting required. A more detailed study could involve quantitative analysis as well, providing statistical data of the percentage of the difference between ChatGPT and Human translations and the exact amount of postediting needed. This will help ChatGPT developers pinpoint the areas and, more importantly, the type of data that ChatGPT needs to be trained on at least concerning translating academic and religious texts. A final limitation that could be stated here concerns conducting only micro-level analysis, that is, linguistic analysis, focusing mainly on syntax and semantics. A more elaborated study could involve mezzo- and macro-levels, deeply probing the historical, cultural, and social factors affecting ChatGPT translation.