From hyperbole to neutrality: A linguistic analysis of planner- and LLM-generated summaries of public submissions

Abstract

Large language models (LLMs) are increasingly applied to text summarisation tasks in many professional domains, yet their use in urban planning remains underexplored. This paper investigates how planner-written summaries of public submissions compare with LLM-generated summaries, with a focus on linguistic characteristics, tone and semantic similarity. Drawing on 7932 public submissions to a housing reform discussion paper, we analysed a planner-written summary, and 25 summaries generated using ChatPDF (OpenAI GPT-4o). Five prompt engineering methods were used to generate the summaries. Evaluation methods combined syntactic and semantic analysis and automated evaluation metrics. Semi-structured reflections from planning officers supplemented the analysis. Results demonstrate that LLM summaries are more difficult to read and lexical diversity is significantly more complex than planner-written summaries. LLMs tend to incorporate emotive sentiment from submissions, whereas planners use more neutral language focused on planning impacts. Automated metrics revealed poor n-gram overlap but good semantic similarity. The findings suggest that LLMs can effectively capture meaning from text but struggle with interpretive neutrality and planning contexts.

Keywords

large language models text summarisation evaluation planning practice

Introduction

Large language models (LLMs) have drawn significant societal attention with their functionality applicable to everyday tasks like knowledge retrieval, text summarisation, question answering and text generation. These models are trained on large datasets and store a substantial amount of knowledge implicitly in their parameters (Petroni et al., 2019). In urban planning, emerging interest in text summarisation has focused on evaluating plans and analysing public feedback (Brinkley and Stahmer, 2024; Fu 2024). However, although LLMs exhibit impressive text summarisation capabilities, they are prone to overgeneralising, can misinterpret nuanced planning jargon (Fu et al., 2025a) and tend to produce homogenous output (Rudko and Bashirpour Bonab, 2025). It is also acknowledged that LLMs face significant challenges in knowledge-intensive tasks (Petroni et al., 2019). Hence, there are open questions regarding the application of LLMs in urban planning processes (Sanchez et al., 2023; Fu et al., 2025b). Thus, researchers must engage planners in the development and testing of LLMs to improve planners’ knowledge and perceptions of LLM usefulness for planning practice (Sanchez et al., 2023).

The aim of this paper is to contribute to current literature on LLMs in urban planning with a focus on the following research questions:

• What are the syntactic features, readability and lexical diversity of a planner-written public submissions summary and LLM-generated summaries?

• How semantically similar are planner and LLM-generated summaries?

To address these questions, this paper uses linguistic analysis to investigate the similarities and differences between the planner and LLM-generated public submissions summaries. By analysing linguistic attributes such as word groupings, word usage, sentence lengths and unique phrases, this study seeks to identify key differences between the text summaries. Five prompt engineering techniques are used to examine both linguistic and semantic characteristics. Each prompt was run 5 times to assess variation in the output (n = 25). Semi-structured interviews were conducted with planning staff who wrote the public submissions summary to gain insights into how public submission reports are written by planners.

It is important to undertake evaluations of LLM-generated texts as what may at first appear fluent, coherent and on topic may not be on closer inspection (Bender et al., 2021). Coherent text is in the eye of the beholder, given human understanding and communication depend on our recognition of who is taking part in the dialogue, their intentions and context, ‘as such, human communication relies on the interpretation of implicit meaning conveyed between individuals’ (Bender et al., 2021:616). On the contrary, LLMs have no concept of the readers mind, their intent or context (Bender et al., 2021). However, the latest LLMs can produce text with ‘a level of fluency and coherence that is nothing short of stunning’ (Arkoudas, 2023:1). But we must be aware that our perception of text is influenced by our linguistic competence and our tendency to interpret communicative acts as conveying coherent meaning and intent (Bender et al., 2021).

Drawing on 7932 public submissions in response to the public exhibition of a discussion paper: Changes to create low- and mid-rise housing by the Department of Planning and Environment, in Sydney, Australia in December 2023, we use ChatPDF to generate a summary of the 10 most common issues raised in the public submissions. ChatPDF is a user interface that acts like a wrapper around an underlying LLM, in this case OpenAI’s GPT-4o. The remainder of this paper refers to LLM interactions and output as GPT-4o.

Summaries of public submissions are a key element of planning ‘staff’ reports, and writing staff reports is a core skill in planning practice (Meck and Morris, 2004). However, staff reports and how planners write them have received almost no attention in planning research and literature (Johnson and Lyles, 2016). The literature that does exist focuses on the format and structure of reports, with less attention paid to writing style. Staff reports are the primary communication channel between planners, decision-makers and the public (Johnson and Lyles, 2016). Moreover, staff reports are important as they are both an administrative record and support legislative decisions. A poorly written report can be the basis for legal action (Higgins, 2006). Writing in plain English provides some insights and guidance for planners to improve their report writing by using short sentences, using verbs instead of nouns, use of conjunctions and positive language (Macris, 2000). The Australian Government Writing Handbook advises report writing should maintain a clear, factual and objective voice, should avoid repeating emotional or non-actionable commentary and should not critique or praise policy (Australian Government, 2024).

This paper will firstly provide an overview of text summarisation, LLM text detection, prompt engineering and then outline the case study. The paper then draws on qualitative syntactic and semantic analysis to explore the topics raised by the public submissions. The concluding section provides a discussion of the findings in relation to the research questions and suggestions for further research. Some conclusions follow.

Large language models use in urban planning

Emerging research on LLM and urban planning has predominantly focused on spatial analysis. There are hybrid spatial and textual approaches emerging (Quan and Lee, 2025), but scholars have not paid much attention to how LLMs can process, synthesise and interpret the large amounts of textual content generated and maintained by urban planning (Andrews et al., 2022). To this end, LLMs are emerging as a tool to aid urban planners with processing big textual data. For example, evaluating plans (Fu et al., 2024a), summarising public feedback (Fu et al. 2024b, 2025a) and planning text classification tasks (Lin et al., 2025). Thus, Fu et al. (2025b) argue that LLMs offer new opportunities to assist urban planners by addressing gaps in text analysis. Moreover, Fu et al. (2025b) have put forward AI and urban planning use cases that they envisage as the start of a toolbox for professional planners and mapped out a research pathway to analyse how LLMs can be leveraged to improve text rich planning processes.

Text summarisation

Given increasing interest in LLMs which are widely used in the form of interactive conversations (Ouyang et al., 2022), it is important to evaluate text generated by generative models. The most popular LLMs have not been solely pretrained in an unsupervised next-token prediction manner but are also fine-tuned using large amounts of human-written text, known as Reinforcement Learning with Human Feedback (RLHF), to improve the quality of the text generated and more closely align with human expectations and preferences (Arkoudas, 2023; Herbold et al., 2023). Hence, to improve the quality of text generated by LLMs, comparisons with human written text are necessary. When the literature exploring text summarisation by LLMs is examined, many conclude that there has been significant progress, but challenges persist, such as factual consistency, handling large inputs and linguistic quality (Arkoudas, 2023; Basyal and Sanghvi, 2023). Studies that have compared LLM-generated and human generated text have employed a mix of linguistic tools and human evaluations (Zhang et al., 2025).

The primary objective of text summarisation is to generate concise and informative summaries that preserve the core information of a source document, facilitate efficient understanding and support decision-making (Kotkar et al., 2024). Text summarisation approaches used by LLMs are broadly categorised as extractive and abstractive techniques. Extractive summarisation involves maintaining the original wording by assigning importance scores for key sentences or phrases based on factors like word frequency and position in the original text (Kotkar et al., 2024; Supriyono et al., 2024). With the emergence of pre-trained LLMs, abstractive summarisation can be used to generate entirely new sentences that encapsulate the meaning of the source text (Kotkar et al., 2024; Supriyono et al., 2024). The abstractive technique can produce more concise and fluent summaries but is prone to factual errors (Supriyono et al., 2024). ChatGPT advises that it uses:

‘an abstractive approach — unless you specifically ask for an extractive summary. That means I rephrase and condense the information rather than just copy parts of the text (OpenAI, 2025).

The large pretrained LLMs using the transformer architecture have improved text summarisation by providing language models capable of capturing long range word dependencies within texts (Bender et al., 2021; Jurafsky and Martin, 2026). This allows LLMs to produce text that not only seems to be fluent but also coherent over entire paragraphs (Bender et al., 2021).

Detecting LLM-generated text

With the widespread use of LLMs in society, a significant amount of research has focused its attention on detecting LLM-generated text. However, the diversity of textual use by humans and a growing number of LLMs make it difficult to accurately detect LLM-generated text. Notwithstanding, LLM-generated text detection often employs linguistic features such as syntactic patterns, readability and lexical diversity to distinguish LLM-generated content from human-written text. This case study is aware of which texts are generated by LLMs, but research into detecting LLM-generated text offers insights into the syntactic features that have been identified. For instance, Muñoz-Ortiz et al. (2024) note LLM-generated text exhibits low variance in sentence length and relies on simpler sentence structures compared to human texts, which highlights differences in readability and syntactic complexity. Moreover, Imperial and Madabushi (2023) found that LLMs tend to use simpler sentence structures and vocabulary, resulting in higher readability scores and Culda et al. (2025) observed human-authored texts exhibit significant variability, ranging from simple to complex sentences, influenced by linguistic and contextual factors. Given research uses a wide range of human authored texts, there are contradictions in the literature. For instance, Reviriego et al. (2023) found LLM-generated texts exhibit lower lexical diversity and fewer unique words than human texts, while Herbold et al. (2023) found LLMs exhibit high lexical diversity but lack depth.

Prompt engineering

For Liu et al. (2025), there is a critical gap between the theories of prompt engineering and its practical applicability to domains such as urban planning. Prompt engineering is a set of instructions that guide LLM behaviour and has emerged as a critical factor in determining the quality and relevance of LLM-generated text (Liu et al., 2025). Thus, the focus on engineered prompts is to yield more accurate and desirable responses. Schulhoff et al. (2024) surveyed 58 prompting techniques, but the most relevant to this case study include role or persona prompting which assigns a specific role in the prompt, for example, ‘act like a travel writer’. Tone & style prompting specifies a desired writing style in the prompt to shape the output. Policy-aware prompting provides relevant background information and/or instructions. Lastly, few-shot prompting provides an LLM with several examples as guidance. The policy-aware and few-shot prompting techniques achieve more desirable outputs which would otherwise require updates to the LLM parameters (Schulhoff et al., 2024).

Automated evaluation of LLM output

There is significant effort expended on evaluating LLM outputs, and it is an ongoing challenge. For example, Basyal and Sanghvi (2023) conducted a comparative study of LLMs that illustrated their varying effectiveness using the CNN/Daily Mail benchmark datasets and calculating evaluation metrics such as BLEU (Papineni et al., 2002), ROUGE (Lin, 2004) and BERTScore (Zhang et al., 2019). Traditional automatic evaluation metrics like BLEU and ROUGE evaluate the presence of n-grams and n-gram overlap with human-written text and BERTScore measures the similarity between LLM-generated summaries and human-written texts using word embeddings from the BERT language model. While BLEU and ROUGE are consistently used, these measures may fail to capture semantic qualities (Basyal and Sanghvi, 2023). Thus, Nguyen et al. (2024) review of automatic evaluation methods highlighted the importance of using multiple evaluation measures including clarity, accuracy, coverage and overall quality to measure long text outputs. Human evaluations remain the gold standard for evaluating LLM-generated text (Sai et al., 2023). However, human evaluation is time-consuming and resource-intensive, thus making automatic evaluations more cost effective for large scale studies, but issues remain, as many automatic evaluation metrics show weak correlation with human evaluations (Nguyen et al., 2024).

Case study

A discussion paper outlining proposed policy and legislative changes to promote low- and mid-rise housing was publicly exhibited between 15 December 2023 and 23 February 2024 (DPHI, 2025b). The discussion paper outlined the high-level policy directions and their intent to encourage more low- and mid-rise housing in well-located areas across the state of New South Wales (NSW), Australia. The primary intent of the policy was to provide housing choice in locations with access to transport and services. The policy context is the NSW Government’s commitments in the National Housing Accord (Australian Government, 2022). Under the Housing Accord agreement, all levels of government agreed to an aspirational target to build 1.2 million new well-located dwellings over 5 years from mid-2024 across Australia (DPHI, 2025a). Well-located areas must have walkable access to a range of shops and services, good public transport, infrastructure and amenities (DPHI, 2025b). The discussion paper indicated that these areas would be based around town centres and train stations which would be the subject of consultation and further analysis (DPHI, 2025b).

The key policy moves outlined in the discussion paper included the following:

• permit dual occupancies in all low-density residential zones;

• permit terraces, townhouses and 2-storey apartment blocks in low-density residential zones within 800 m walking distance of town centres and train stations;

• permit 6-storey apartment blocks in all medium-density residential zones within 800 m walking distance of town centres and train stations;

• introduce ‘non-refusal’ standards designed to enable these housing types.

After public exhibition, the policy was reviewed and implemented in two stages in July 2024 and February 2025. While the housing types were largely implemented as exhibited, the geographic scope of the project was reduced by excluding areas at risk from flooding and bushfire hazards and removing business zones from the scope of the policy (DPHI, 2025b).

Methods and materials

This study aims to examine the linguistic features, readability, lexical diversity and semantic similarities and differences between key issue summaries written by urban planners and generated by GPT-4o. To this end, this study employs a predominantly qualitative research methodology. Qualitative research uses inductive methods that seek to draw on general principles and theories for investigations. It does not use statistical tools, but tables and basic calculations are used to analyse textual data (Denzin and Lincoln, 1998).

Participants

The participants consisted of 4 planning officers employed by the NSW Department of Planning, Housing and Infrastructure. The planning officers’ industry experience ranged between 3 and 10 years. All participants had worked on the collection and analysis of public submissions for the low- and mid-rise housing policy. The planning officers were given a basic overview of how the GPT-4o text summaries were generated and asked to provide comments without any leading questions. Participation was voluntary, and all comments collected were anonymously recorded.

Dataset

The dataset consists of human-generated texts in the form of public submissions that predominantly resemble a letter format. While most people only submit a single submission, some people may submit multiple. Submissions can also be co-authored by neighbourhood groups. The data also contains submissions from state government agencies, local municipalities and industry groups representing various aspects of the property development industry. The dataset comprises 7932 submissions (1,822,023 words) in response to the discussion paper. The average length of submissions is 230 words. The submissions vary in length from 5 to 4000 words.

OpenAI’s GPT-4o, accessed via the ChatPDF wrapper, was utilised to create summaries of the 10 most frequently mentioned issues in public submissions. This concept is known as the Retrieval Augmented Generation (RAG) method. In basic terms, the PDF document is parsed to extract text content. The text is then split or chunked into segments of up to 1000 words, converted to vector embeddings and stored in a vector database along with the original text chunks (Parthasarathy et al., 2024). When we submit a prompt, ChatPDF performs a similarity search to find the most relevant text chunks from the PDF document, and the text chucks are then provided as context to GPT-4o which generates the output (Parthasarathy et al., 2024). To ensure factual responses and minimise randomness, ChatPDF sets the temperature to 0.0. Moreover, to restrict randomness, ChatPDF sets the Top_p to 1.0 and Top_k to 40. These parameters combined with a very low temperature force GPT-4o to consider the entire probability distribution but use the most probable words in its output (chatpdf.com).

To evaluate a range of GPT-4o outputs, this study implemented 5 prompt strategies starting with a simple baseline prompt to ‘Find the 10 most common issues in the pdf’ and progressed through to an advanced few-shot prompt. This approach provides an opportunity to evaluate how GPT-4o responds to increasing levels of instructions. Each prompt builds upon the previous prompt. After an initial review, the planning officers noted, ‘we don’t address consultation in submissions reports, we are consulting’ (Planning Officer, 2024) and provided feedback on the use of negative phrases:

‘the AI program is pretty accurate in summarising the issues raised. Only thing is it really reflects the negativity of the submissions by using words like perceived failure, widespread apprehension, raised alarms’ (Planning Officer, 2024).

To address this feedback, the policy-aware prompt provided planning context as well as common planning issues raised in submissions. The tone & style prompt instructed GPT-4o to maintain a neutral, informative tone and use transitional phrases to guide the reader. GPT-4o was directed to exclude both ‘lack of community consultation’ and ‘failure to address broader housing issues’ from its output, since community consultation is occurring and wider housing concerns are not actionable. All prompts can be found in Appendix A.

Analysis and evaluation

To compare the syntactic and semantic features of the planner-written and GPT-4o summaries, text preprocessing was performed first. Text preprocessing included cleaning the text of special characters, punctuation marks, numbers and stop words (frequently occurring words such as ‘the’, ‘a’, ‘and’ and ‘but’). During preprocessing, the features of the text are also extracted. In this step, the number of sentences, unique words, stop words, sentence lengths and word length values are extracted. Lastly, bigrams and trigrams are generated by tokenising (splitting into individual words) the words in the clean text, dividing them into consecutive pairs of tokens and counting the number of times bigrams and trigrams appear in the text. This approach identifies word pairings in the text to gain an understanding of relationships between words and word patterns in the text (Schonlau et al., 2017). This study calculated the frequencies for bigrams and trigrams, and the 10 highest frequencies are listed and discussed. All these steps were performed using basic Python scripts, and word clouds were created using Nvivo.

No text preprocessing is required for calculating automatic evaluation metrics. Evaluating open-ended text generated by LLMs is challenging because there is often no single correct answer. The best evaluation approach typically involves a combination of human evaluation and manual or automatic metrics (Davies, 2025). Automated evaluation of open-ended text is undertaken using reference-based methods which seek to compare the generated text to one or more human written reference texts that are considered correct or high-quality (Davies, 2025).

The Bilingual Evaluation Understudy (BLEU) is a common metric used to evaluate the quality of machine translations from one language to another language by measuring overlapping n-grams (sequences of words). BLEU calculates a score between 0 and 1. The more n-grams that overlap, the higher the score (Papineni et al., 2002). Recall-Oriented Understudy for Gisting Evaluation (ROUGE) comprises evaluation metrics used to evaluate sentence generation by calculating the F1-score for overlapping n-grams between human-written and LLM-generated texts. The Metric for Evaluation of Translation with Explicit Ordering (METEOR) was developed to align user and LLM-generated responses using token matching, paraphrasing and synonyms. The METEOR method goes beyond the word matching methods used by BLEU and ROUGE by using stemmed matches that remove common suffixes, for example, computers maps to both computers and computer and recognises words that are synonyms of each other using the WordNet database, that is, quick and fast are considered to have similar meanings (Banerjee and Lavie, 2005).

To evaluate readability, the scores for Flesch-Kincaid Grade Level (Kincaid et al., 1975), Flesch Reading Ease (Flesch, 1948) and Measure of Textual Lexical Diversity (MTLD) (McCarthy and Jarvis, 2010) were calculated for all prompt output. Flesch-Kincaid Grade Level (Kincaid et al., 1975) is a readability score indicating the U.S. school grade needed to understand a text. For example, a score from 9 to 12 indicates High School and 13+ indicates college-level education. The Flesch Reading Ease (Flesch, 1948) score is a readability score between 0 and 100 which measures how easy text is to understand, based on average sentence length and syllables per word. The MTLD scores (McCarthy and Jarvis, 2010) measure the range of lexical diversity which is an indicator of how complex and difficult a passage of text is to read.

Early attempts to evaluate machine-generated text were motivated by finding automated methods to remove the reliance on expensive and time-consuming human evaluations that cannot be reused (Papineni et al., 2002). With the advent of LLMs that can produce longer original content in response to human prompts, evaluation against human-written references has become increasingly difficult. In response, a recent method known as BERTScore computes a similarity score for each token in the prediction sentence with each token in the reference sentence. Instead of exact matches, BERTScore computes similarity using cosine similarity of contextual embeddings in the BERT model based on surrounding words (Zhang et al., 2019). BERTScore computes similarity scores at the word level before aggregating and assigning an importance-weighting (Zhang et al., 2019). The BLEU, ROUGE, METEOR, Flesch-Kincaid, MTLD and BERTScore metrics were implemented in Python scripts or use the Hugging Face Evaluate library (huggingface.co/docs/evaluate/en/index).

Results

This section presents the results with respect to the following research questions: What are the syntactic features, readability and lexical diversity of a planner-written public submissions summary and LLM-generated summaries? And how semantically similar are planner and LLM-generated summaries by examining the syntactic and semantic features of the LLM summaries.

Syntactic features of the planner-written and AI-generated summaries

At the syntactic level, analyses were conducted using the Python NLTK library to compare the planner and LLM-generated summaries to reveal the characteristic features of the texts. Common attributes of the text summaries such as total number of words and sentences, conjunctions, total number of unique words, stop words and readability scores are presented in Table 1. The total number of sentences is highest in the planner’s summary. GPT-4o prefers to summarise each issue with a single sentence but improves when instructed. GPT-4o consistently uses more unique words than the planners. Conversely, the planners used more stop words than all LLM prompts. The total number of words and conjunctions varies for each prompt but generally aligns with the planner’s summary for the few-shot prompt.

Table 1.

Features of the text summaries by prompt.

Features	Planner	Baseline		Persona		Policy aware		Tone & Style		Few Shot
Features	Planner	Mean	SD	Mean	SD	Mean	SD	Mean	SD	Mean	SD
Total number of words*	356	306.6	18.5	325.6	42.8	352.2	18.1	353.2	36.4	351.2	18.1
Number of sentences	21	10.6	1.3	14.4	3.4	12	1.4	16.6	4.0	19	1.0
Number of unique words	166	191	11.4	195	25.3	212	11.7	213.2	16.2	205	8.4
Number of stop words**	125	85	10.3	90.8	14.5	97.8	11.4	95	12.7	95.8	5.8
Number of conjunctions	30	32.6	3.3	28.8	10.7	31.2	2.8	25.4	4.8	27.2	2.8
Flesch-Kincaid (Grade level)	12.8	20.1	1.3	17.6	1.2	20.6	1.6	18.1	1.2	17.1	0.7
Flesch (Reading Ease Score)	30.6	1.3	2.9	4.2	4.6	0.2	0.4	1.2	1.8	2	2.5
Measure of Textual Lexical Diversity	67.4	170.5	15.7	163.2	15.1	174.8	26.1	189.8	35.9	163.4	9

SD, standard deviation.

*Including issue headings.

**standard NLTK list.

The Flesch-Kincaid Grade Level, Flesch Reading Ease and MTLD scores, which measure readability based on sentence length and syllable count, indicate that the planner’s summary is suitable for high school students, whereas the GPT-4o summaries are appropriate for university graduates. The Flesch Reading Ease suggests the planner’s summary has university-level readability (difficult) while the GPT-4o summaries have an ‘expert’ readability level or very difficult to read. Furthermore, the MTLD score that uses unique words to measure how complex or varied a text’s vocabulary is shows the GPT-4o summaries are significantly more complex to read than the planner summary.

Text-based features were also examined. For each summary, the percentage of unique words and stop words, and the average length of words and sentences were calculated for review (Table 2).

Table 2.

Percentages for features by prompt.

Features	Planner	Baseline		Persona		Policy aware		Tone & Style		Few Shot
Features	Planner	Mean	SD	Mean	SD	Mean	SD	Mean	SD	Mean	SD
Unique word	46.6	62.3	0.6	59.9	1.6	60.3	3.5	60.5	2.4	58.4	2.9
Stopwords	35.1	27.7	1.9	27.8	1.5	27.7	2.0	26.8	1.0	27.3	1.2
Conjunctions	8.4	10.6	0.9	8.6	2.5	8.9	0.6	7.2	0.8	7.7	0.6
Word lengths (characters)	5.8	6.6	0.2	6.6	0.1	6.7	0.1	6.6	0.0	6.8	0.1
Sentence lengths (word)	16.8	29.0	1.9	22.8	3.9	29.6	5.0	22.1	4.5	18.5	1.1

SD, standard deviation.

The percentage of unique words is consistently higher for the GPT-4o summaries. Conversely, stop word usage as a percentage is highest in the planner-written summary. When sentence lengths are analysed by word and character, the GPT-4o summaries consistently contain longer sentences. The analysis of the sentences shows the planner’s summary utilises more function words like conjunctions and stop words. The higher usage of function words is typical of human-generated text. Ludwig et al. (2013) note function words in English text can account for up to 55% of word usage (Ludwig et al., 2013). Moreover, the high usage of conjunctions in the summary texts reflects the writing structure of listing multiple issues connected to planning and policy matters. The few-shot prompt provides overall improvement but still generates more complex sentences than the planner’s summary.

These findings show distinct syntactic differences between planner-written and LLM-generated texts. The use of function words and the use of simpler, more accessible language are important distinguishing features in terms of syntax. Syntactic complexity and readability were assessed using various measures. When evaluated from these perspectives, this analysis indicates that planners exhibit a simpler language structure in their writing, which leads to a more digestible summary.

Semantic features of the planner-written and LLM-generated summaries

N-grams and word clouds can reveal semantic differences between the human-written and LLM-generated summaries. To evaluate the dataset from a semantic perspective, n-grams were used. The frequencies of bigrams (pairs of consecutive words) and trigrams (group of three consecutive words) in the public submissions, planner and LLM-generated text were extracted, and the 10 most frequent bigrams and trigrams were identified (see detailed bigram and trigram tables in Appendix B). All summaries identified ‘public transport’, ‘traffic congestion’, ‘local’ or ‘neighourhood character’ and ‘housing affordability’ as high-frequency bigrams. This indicates that regardless of the prompting method, the key concerns generally remain consistent. All GPT-4o bigrams list ‘green space’. However, this is not a high-frequency bigram in the submissions or the planner summary. The planners focused on ‘population density’ while the GPT-4o summaries frequently used ‘development density’. This reflects the strategic nature of the low- and mid-rise reforms. Future development would be subject to assessment through a permit application and was not mentioned by the planners in their summary. This leads to a key difference in the analytical tone displayed by the planner summary which used bigrams like ‘would lead’ and ‘proposal would’ to discuss potential material impacts. On the other hand, the GPT-4o summaries repeatedly used ‘many submissions’ and ‘express concern’ to summarise sentiment in the submissions. The trigrams identify frequently repeated negative expressions in the GPT-4o summaries, except the tone & style prompt, such as ‘widespread worry’, ‘express concerns’, ‘criticise’, ‘resistance’ and ‘reluctance’. Although these expressions are not listed as high-frequency trigrams in the submissions, they are regularly used in GPT-4o summaries, while the planner’s summary conveys the intent of the submissions with more neutral language. Planners noted their writing must be ‘as simple as possible, no emotive or effusive language’ (Planning Officer, 2024), and our writing is ‘meant to be neutral in the sense that when the facts are presented…it’s clear to the reader’ (Planning Officer, 2024). Example summary pairs can be found in Appendix C.

Word clouds allow the evaluation of the most prominent words in the summaries. In this analysis, the word clouds reinforce the n-gram findings by highlighting the repeated use of the terms ‘submissions’, ‘’increased’ and ‘community’ by GPT-4o regardless of the prompt technique (Figure 1). The GPT-4o summaries also use ‘transport’, ‘infrastructure’ and ‘planning’ while the planners frequently used terms such as ‘housing’, ‘density’ and ‘character’. The GPT-4o summaries identify valid topics through frequently used words, but different prompts extracted different topics such as ‘environment’ ‘green space’ and ‘planning’ which highlights the stochastic nature of LLM-generated text (Figure 1).

Automatic evaluation

This section evaluates and interprets the results of applying automated metrics to the planner and GPT-4o-generated summaries, and box plots are used to investigate the output variation. Throughout this section, the planner summary is referred to as the reference and the GPT-4o summaries are referred to as predictions.

The closer a BLEU score is to 1, the closer the match between the reference and prediction texts (Papineni et al., 2002). The BLEU scores in Table 3 indicate a poor translation, as there is limited word overlap between the planner and GPT-4o summaries. However, it has been observed that the BLEU score mainly measures unigram overlap, without considering aspects like meaning, fluency or grammatical accuracy. ROUGE provides 3 scores based on overlapping of n-grams. ROUGE-1 score represents unigram, and ROUGE-2 represents bigram overlaps in the reference summary and prediction summaries. ROUGE-L measures the longest common word sequence. The ROUGE scores are shown in Table 3. A good ROUGE score is 0.5 or above. It is noted the ROUGE-2 scores are smaller than ROUGE-1 and ROUGE-L which highlights a lack of bigram overlap between the summaries. The BLEU and ROUGE scores confirm that while there is a small number of similar words used in the planner and GPT-4o summaries, there is a small overlap for word pairs. These results concur with the bigram and trigram analysis in the previous section but provide an overall score that indicates the summaries have a poor relationship in terms of overlapping words. Moreover, the METEOR score shown in Table 3 also suggests there is a poor word overlap between the planner and GPT-4o summaries. A good METEOR score is 0.6 or above. Although METEOR is a more sophisticated approach to identify word overlap, it is based on unigrams and achieves a low score.

Table 3.

BLEU, ROUGE and METEOR scores.

Prompt	BLEU		ROUGE-1		ROUGE-2		ROUGE-L		METEOR
Prompt	Mean	SD	Mean	SD	Mean	SD	Mean	SD	Mean	SD
Baseline	0.208	0.016	0.241	0.015	0.021	0.004	0.168	0.010	0.160	0.018
Persona	0.204	0.026	0.243	0.020	0.025	0.005	0.174	0.014	0.169	0.030
Policy Aware	0.225	0.014	0.279	0.014	0.039	0.009	0.191	0.01	0.199	0.012
Tone & Style	0.200	0.023	0.252	0.030	0.033	0.008	0.169	0.015	0.183	0.029
Few Shot	0.250	0.015	0.297	0.020	0.054	0.01	0.205	0.011	0.227	0.027

SD, standard deviation.

The poor metrics calculated by BLEU, ROUGE and METEOR raise the question of how many words are present in both the planner and GPT-4o summaries. Given the summaries are generated from the same public submissions, it could be assumed that it would be significant. However, there are only an average of 40 words that appear in both the planner and GPT-4o summaries. This represents between 9 and 14% of all words used in the summaries. It is observed that the few-shot prompt summaries share the most common words with the planner summary and achieved the highest scores across all metrics.

Noting the poor performance of the traditional metrics, BERTScore, which measures semantic similarity using contextual representations, was applied to the prediction summaries. The BERTScores in Table 4 show the mean scores for the GPT-4o summaries. Higher scores indicate a stronger match between the meaning and context of the reference and prediction texts (Hanna and Bojar, 2021). A BERTScore of 0.85 or more indicates a strong semantic match between the reference and prediction texts. Scores between 0.70 and 0.85 indicate a good level of semantic similarity and scores below 0.70 indicate a weak semantic similarity (Hanna and Bojar, 2021). The mean F1 scores ranging from 0.797 to 0.820 suggest there is a good level of semantic similarity between the planner and GPT-4o summaries. The BERTScores improved in line with the complexity of the prompts, except for the tone & style prompt. The tone & style prompt only introduced writing instructions but resulted in GPT-4o producing a longer summary with the largest number of unique words. The MTLD score also suggested the text generated by the tone & style prompt was the most complex to read. The few-shot prompt which used the full array of prompt instructions achieved the highest BERTScore.

Table 4.

BERTScores.

Prompt	BERT (P)		BERT (R)		BERT (F1)
Prompt	Mean	SD	Mean	SD	Mean	SD
Baseline	0.795	0.004	0.798	0.004	0.797	0.004
Persona	0.796	0.007	0.800	0.007	0.798	0.007
Policy Aware	0.800	0.004	0.813	0.002	0.806	0.002
Tone & Style	0.794	0.004	0.803	0.008	0.798	0.006
Few Shot	0.815	0.004	0.825	0.003	0.820	0.003

SD, standard deviation.

Prompt and evaluation metric correlation

To understand if there are any statistical relationships between the 5 prompts and the evaluation metrics, Z-scores (Yilmaz, 2017) were calculated for each prompt and evaluation metric categories: Readability/Diversity, BLEU/ROUGE/METEOR and BERT Scores. The heatmap in Figure 2 shows the basic baseline and persona prompts were negatively correlated with almost all evaluation metrics and the policy-aware prompt achieved positive but low correlations for all evaluation metrics. The tone & style prompt, which introduced writing and grammatical instructions to guide the reader, achieved the highest correlation with the readability and MTLD scores but negative correlation with the other metrics. Conversely, the few-shot prompt has a negative correlation with the Readability/Diversity metrics but achieved the highest correlations with BLEU/ROUGE/METEOR and BERT scores. Hence, the few-shot prompt generated text more semantically aligned with the planner’s summary.

Variance in text generation

Language models produce textual output that is non-deterministic. This phenomenon produces variation in text generation even when the same prompt and parameters are used and is the result of the stochastic nature of next token (word) generation by LLMs (Fu and Li, 2025; Smith et al., 2025). To measure word choice and sentence variation, each prompt was run 5 times to assess how variable the text generation is. Figure 3 uses box plots to show the variations over 5 runs for all automated evaluations. The key observation is the few-shot prompt not only achieves a consistently higher evaluation scores, but it also reduces the text variability, except for the Read Ease score. Moreover, there is higher variability and inconsistency for persona, policy-aware and tone & style prompts, which highlights how LLMs can be sensitive to changes in prompts. In this instance, the metrics improved as each prompt provided additional instructions. However, when GPT-4o was prompted with the tone & style instruction to keep a neutral tone and include transitional phrases, almost all the metrics decreased. When training examples were added via the few-shot prompt, GPT-4o achieved the best results. Detailed results can be found in Appendix D.

Discussion

The aim of this paper was to investigate the syntactic features, readability, lexical diversity and semantic similarities of a planner-written public submissions summary and LLM-generated summaries. At first glance, the GPT-4o summaries are readable and were judged to be ‘pretty accurate in summarising the issues raised’ (Planning Officer, 2024). However, when viewed through a linguistic lens, the summaries are substantially different. The planner-written summary exhibits linguistic features and lexical structures that produce more readable text, while the GPT-4o summaries are more complex and less readable. Moreover, the GPT-4o summaries include emotive or hyperbolic language found in public submissions, whereas planners use more neutral language to assess the issues raised by the public. In contrast, the summaries have good semantic similarities, which demonstrates that the GPT-4o can capture the meaning of the submissions but generates summary text using different syntactic and lexical features.

The manual process of summarising public submissions is a labour-intensive, time-consuming and costly exercise, particularly when planners receive thousands of submissions, such as this case study. Hence, utilising LLMs to generate draft submission summaries could reduce the workload of planners and efficiently capture the spectrum of issues raised in the submissions (Fu et al., 2024b). Automated submission summarisation should not be viewed as a pathway to replacing planners, rather a support tool (Fu et al., 2025b), where planners remain in the loop (Mosqueira-Rey et al., 2023) to ensure summaries are readable, neutral and align with the planning context. Fu et al. (2024b) have shown that LLM-generated text may contain inherent biases from training data, a tendency to overgeneralise and overlook nuanced details. This case study extends these findings to include LLM-generated text is linguistic different to planner-written text. Hence, planners should not only be aware of overgeneralisation, omissions and hallucinations but also critically review the readability of LLM-generated text.

This study employed a combination of five prompt engineering methods and RAG to generate the results. There are limitations to using the basic RAG approach undertaken for this case study. In particular, the RAG method operates on fragments of the original text that it decides may contain keywords relevant to the prompt (Parthasarathy et al., 2024). As such, the RAG approach is not generating outputs based on the entire corpus and is limited by the single-pass retrieval system (Kamalipour et al., 2026). This RAG approach provided poor results using basic prompts, but when combined with the few-shot prompt did improve on all evaluation metrics and reduced text variability. The use of RAG is quickly evolving with the emergence of agentic LLM approaches (Fu and Li, 2025). Future research should focus on emerging agentic RAG architectures which use autonomous agents to orchestrate information retrieval and text generation, where factual grounding may be improved by self-refining queries and evaluating retrieved information (Kamalipour et al., 2026). Reducing text variability is critically important for improving planners’ trust in LLM-generated text.

While the search for automated metrics to evaluate LLM output continues, human evaluations remain the gold standard due to domain expertise (Fu and Li 2025; Sai et al., 2023) and understanding of local context beyond the written submissions (Sanchez, 2025). Human evaluation is inherently resource-intensive and time-consuming, compared to cost-effective and replicable automated metrics. Nguyen et al. (2024) emphasise that the evaluation of LLM-generated text requires a multi-dimensional approach including clarity, accuracy, coverage and quality. However, automated metrics continue to show a weak correlation with human evaluations. This case study sought to establish a baseline by testing commonly used automated metrics. The findings confirm that legacy metrics like BLEU, ROUGE and METEOR provided little insights for this case study. BERTScore offers an improved metric, though it fails to measure nuances at the linguistic level. For the foreseeable future, planners must evaluate any LLM-generated outputs, rather than rely on automated metrics. Fu and Li (2025) advocate for a hybrid approach where an LLM assists the planner with coding and evaluation, which may reduce time and costs of the summarisation process. There is also the potential for planners to lose knowledge and understanding of the submission corpus if they do not remain in the loop. Planners are taught to look for and assess material impacts. If they do not engage with the written submissions, they will not gain an understanding of how many times issues and concerns were raised and how strongly worded the concerns were. Notwithstanding, using LLMs to summarise submissions is a positive step towards efficiency and productivity for a common labour-intensive planning process.

Conclusion

The integration of LLMs into planning practice promises significant efficiency and productivity gains for processing large amounts of text generated by public consultation, but critical linguistic and interpretive issues remain. This study reveals GPT-4o effectively captures the semantic meaning of submissions, evidenced by good BERTScore results between 0.797 and 0.820. However, GPT-4o did not replicate the professional neutrality and readability of planner-written text in this instance. Quantitatively, GPT-4o summaries are more complex, requiring university-level literacy compared to the high school level literacy required to read planner-written summaries. Moreover, the GPT-4o summaries only share between 9 and 14% of the vocabulary used by planners. Advanced prompting is essential for higher quality LLM text generation. In this case, few-shot prompts achieved the highest correlation with most evaluation metrics and reduced text variability across multiple runs, whereas the baseline and persona prompts performed poorly. Consequently, the methodology used for this case study could reliably support data sorting and broad issue identification for large-scale consultation, but it should function as a support tool with planners-in-the-loop. Planners must manually review drafts to ensure emotive or hyperbolic language is not inherited from submissions and rephrase sentences to ensure material planning impacts are discussed with a neutral informative tone. Future research should experiment with the multi-agent LLM approach to focus agents on producing a neutral assessment of material impacts with less focus on reproducing sentiment in public submission.

Supplemental material

Supplemental material - From hyperbole to neutrality: A linguistic analysis of planner- and LLM-generated summaries of public submissions

Supplemental material for From hyperbole to neutrality: A linguistic analysis of planner- and LLM-generated summaries of public submissions by Wayne Williamson in Environment and Planning B: Urban Analytics and City Science

Footnotes

ORCID iD

Wayne Williamson

Funding

The authors received no financial support for the research, authorship, and/or publication of this article.

Declaration of conflicting interests

The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Data Availability Statement

Data sharing not applicable to this article as no datasets were generated or analysed during the current study.*

Supplemental material

Supplemental material for this article is available online.

Author biography

Wayne Williamson is a Senior Research Associate at the University of New South Wales’ City Futures Research Centre. His research interests include the adoption and use of various digital technologies in planning practice, planning discourse in the media and social networks.

References

Andrews

Cooke

Gomez

, et al. (2022) AI in Planning: Opportunities and Challenges and How to Prepare. American Planning Association.

Arkoudas

(2023) ChatGPT is no Stochastic Parrot. But it also claims that 1 is greater than 1. Philosophy & Technology 36: 1–29. https://doi.org/10.1007/s13347-023-00619-6

Australian Government (2022) National Housing Accord 2022. Available at: https://ministers.treasury.gov.au/sites/ministers.treasury.gov.au/files/2022-10/national-housing-accord-2022.pdf (accessed 20 July).

Australian Government (2024) Government Writing Handbook. Available at: https://www.stylemanual.gov.au/sites/default/files/2024-11/APSC_Government_Writing_Handbook.pdf (accessed 3 December).

Banerjee

Lavie

(2005) METEOR: an automatic metric for MT evaluation with improved correlation with human judgments. In: Proceedings of the ACL Workshop on Intrinsic and Extrinsic Evaluation Measures for Machine Translation and/or Summarization, Ann Arbor, Michigan, 5-9 June: Association for Computational Linguistics, pp. 65–72.

Basyal

Sanghvi

(2023) Text summarization using large language models: a comparative study of MPT-7b-instruct, Falcon-7b-instruct, and OpenAI Chat-GPT models. arXiv , preprint arxiv.org/abs/2310.10449.

Bender

Gebru

McMillan-Major

, et al. (2021) On the dangers of stochastic parrots: can language models be too big? In: Proceedings of the Conference on Fairness, Accountability, and Transparency, Virtual Event, 3–10 March. Association for Computing Machinery.

Brinkley

Stahmer

(2024) What is in a plan? Using natural language processing to read 461 California city general plans. Journal of Planning Education and Research 44(2): 632–648. https://doi.org/10.1177/0739456x21995890

Culda

Nerişanu

Cristescu

, et al. (2025) Comparative linguistic analysis framework of human-written vs. machine-generated text. Connection Science 37(1): 2507183. https://doi.org/10.1080/09540091.2025.2507183

10.

Davies

(2025) LLM evaluation: metrics, frameworks, and best practices, weights & biases. Available at: https://wandb.ai/onlineinference/genai-research/reports/LLM-evaluation-Metrics-frameworks-and-best-practices--VmlldzoxMTMxNjQ4NA (accessed 15 July).

11.

Denzin

Lincoln

(1998) Strategies of Qualitative Inquiry. Sage.

12.

Department of Planning, Housing and Infrastructure (DPHI) (2025a) Housing Targets. Available at: https://www.planning.nsw.gov.au/policy-and-legislation/housing/housing-targets (accessed 20 July).

13.

Department of Planning, Housing and Infrastructure (DPHI) (2025b) Low and mid-rise housing policy. Available at: https://www.planning.nsw.gov.au/policy-and-legislation/housing/low-and-mid-rise-housing-policy (accessed 20 July).

14.

Flesch

(1948) A new readability yardstick. Journal of Applied Psychology 32: 221–233. https://doi.org/10.1037/h0057532

15.

(2024) Natural language processing in urban planning: a research agenda. Journal of Planning Literature 39(3): 395–407. https://doi.org/10.1177/08854122241229571

16.

(2025) Automating plan evaluation using agentic large language models. Journal of Planning Education and Research 0739456X251372082. Available at: https://doi.org/10.1177/0739456X251372082, In press.

17.

Wang

(2024a) Can ChatGPT evaluate plans? Journal of the American Planning Association 90(3): 525–536. https://doi.org/10.1080/01944363.2023.2271893

18.

Sanchez

, et al. (2024b) Deciphering public voices in the digital era. Journal of the American Planning Association 90(4): 728–741. https://doi.org/10.1080/01944363.2024.2309259

19.

Brinkley

Sanchez

, et al. (2025a) Text mining public feedback on urban densification plan change in Hamilton, New Zealand. Environment and Planning B: Urban Analysis and City Science 52(3): 646–666.

20.

Quan

, et al. (2025b) Large language models in urban planning. Nature Cities 2: 585–592. https://doi.org/10.1038/s44284-025-00261-7

21.

Hanna

Bojar

(2021) A fine-grained analysis of BERTScore. In: Proceedings of the Sixth Conference on Machine Translation, Virtual Event, 10-11 November. Association for Computational Linguistics, pp. 507–517.

22.

Herbold

Hautli-Janisz

Heuer

, et al. (2023) A large-scale comparison of human-written versus ChatGPT-Generated essays. Scientific Reports 13(1): 18617. https://doi.org/10.1038/s41598-023-45644-9

23.

Higgins

(2006) An Ounce of Prevention: Best Practices for Making Informed Land Use Decisions. Institute for local Government.

24.

Imperial

Madabushi

(2023) Flesch or fumble? Evaluating readability standard alignment of instruction-tuned language models. (arXiv:2309.05454). arXiv . https://doi.org/10.48550/arXiv.2309.05454

25.

Johnson

Lyles

(2016) The unexamined staff report: results from an evaluation of a national sample. Journal of the American Planning Association 82(1): 22–36. https://doi.org/10.1080/01944363.2015.1109471

26.

Jurafsky

Martin

(2026) Speech and Language Processing: An Introduction to Natural Language Processing, Computational Linguistics, and Speech Recognition with Language Models, 3rd edition (January 2026). Available at: https://web.stanford.edu/∼jurafsky/slp3 (accessed on 7 March).

27.

Kamalipour

Asadi

Chimeh

(2026) From vectors to knowledge graphs: a comprehensive analysis of modern retrieval-augmented generation architectures. Computer Science Review 61: 100925. https://doi.org/10.1016/j.cosrev.2026.100925

28.

Kincaid

Fishburne

Rodgers

, et al. (1975) Derivation of New Readability Formulas (Automated Readability Index, Fog Count and Flesch Reading Ease Formula) for Navy Enlisted Personnel. Naval Technical Training Command.

29.

Kotkar

Mahadik

, et al. (2024) Comparative analysis of transformer-based large language models (LLMs) for text summarization Proceedings of the 1st International Conference on Advanced Computing and Emerging Technologies. Ghaziabad, India, 23-24 August.

30.

Lin

(2004) ROUGE: a package for automatic evaluation of summaries. In: Proceedings of Text Summarization Branches out. Association for Computational Linguistics, pp. 74–81.

31.

Lin

Thackway

Soundararaj

, et al. (2025) Transforming urban planning through machine learning: a study on planning application classification using natural language processing. Environment and Planning B: Urban Analytics and City Science 53: 11–31. https://doi.org/10.1177/23998083251369142

32.

Liu

Yigitcanlar

Browne

, et al. (2025) Prompts for Planning-AI integration: LLM prompt design for supporting sustainable urban development. Journal of Open Innovation: Technology, Market, and Complexity 11(4): 100666. https://doi.org/10.1016/j.joitmc.2025.100666

33.

Ludwig

de Ruyter

Friedman

, et al. (2013) More than words: the influence of affective content and linguistic style matches in online reviews on conversion rates. Journal of Marketing 77(1): 87–103. https://doi.org/10.1509/jm.11.0560

34.

Macris

(2000) Planning in Plain English: Writing Tips for Urban and Environmental Planners. Routledge.

35.

McCarthy

Jarvis

(2010) MTLD, vocd-D, and HD-D: a validation study of sophisticated approaches to lexical diversity assessment. Behavior Research Methods 42(2): 381–392. https://doi.org/10.3758/BRM.42.2.381

36.

Meck

Morris

(2004) Formatting and writing the staff report. Zoning Practice 21(10): 1–7.

37.

Mosqueira-Rey

Hernández-Pereira

Alonso-Ríos

, et al. (2023) Human-in-the-loop machine learning: a state of the art. Artificial Intelligence Review 56: 3005–3054. https://doi.org/10.1007/s10462-022-10246-w

38.

Muñoz-Ortiz

Gomez-Rodriguez

Vilares

(2024) Contrasting linguistic patterns in human and LLM-generated news text. Artificial Intelligence Review 57: 2–28. https://doi.org/10.1007/s10462-024-10903-2

39.

Nguyen

Chen

Pobbathi

, et al. (2024) A comparative study of quality evaluation methods for text summarization. arXiv, preprint arXiv:2407.00747v1 .

40.

OpenAI . (2025). ChatGPT [Large language model]. https://chat.openai.com

41.

Ouyang

Jiang

, et al. (2022) Training language models to follow instructions with human feedback. Advances in Neural Information Processing Systems 35: 27730–27744.

42.

Papineni

Roukos

Ward

, et al. (2002) BLEU: a method for automatic evaluation of machine translation. In: Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics, Philadelphia, July, pp. 311–318.

43.

Parthasarathy

Zafar

Khan

, et al. (2024) The ultimate guide to fine-tuning LLMs from basics to breakthroughs: an exhaustive review of technologies, research, best practices, applied research challenges and opportunities. arXiv, preprint arXiv:2408.13296v1 .

44.

Petroni

Piktus

Fan

, et al. (2019) Language models as knowledge bases? In: Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), Hong Kong, November, pp. 343–353.

45.

Planning Officer (2024) Planning officer interviews with author. August-September 2024.

46.

Quan

Lee

(2025) Enhancing participatory planning with ChatGPT-assisted planning support systems: a hypothetical case study in Seoul. International Journal of Urban Sciences 29(1): 89–122. https://doi.org/10.1080/12265934.2025.2462823

47.

Reviriego

Conde

Merino-Gómez

, et al. (2023) Playing with words: comparing the vocabulary and lexical richness of ChatGPT and humans. (arXiv:2308.07462). arXiv . https://doi.org/10.48550/arXiv.2308.07462

48.

Rudko

Bashirpour Bonab

(2025) ChatGPT is incredible (at being average). Ethics and Information Technology 27(36): 1–16. https://doi.org/10.1007/s10676-025-09845-2

49.

Sai

Mohankumar

Khapra

(2023) A survey of evaluation metrics used for NLG systems. ACM Computer Surveys 55(2): 1–39. https://doi.org/10.1145/3485766

50.

Sanchez

(2025) Artificial Intelligence for Urban Planning. Routledge.

51.

Sanchez

Shumway

Gordner

, et al. (2023) The prospects of artificial intelligence in urban planning. International Journal of Urban Sciences 27(2): 179–194. https://doi.org/10.1080/12265934.2022.2102538

52.

Schonlau

Guenther

Sucholutsky

(2017) Text mining with n-gram variables. The Stata Journal 17(4): 866–881. https://doi.org/10.1177/1536867x1801700406

53.

Schulhoff

Ilie

Balepur

, et al. (2024) The prompt report: a systematic survey of prompting techniques. arXiv preprint arXiv:2406.06608 . https://doi.org/10.48550/arXiv.2406.06608

54.

Smith

Bouadjenek

Kheya

, et al. (2025) A comprehensive analysis of large language model outputs: similarity, diversity, and bias. arXiv, preprint arXiv: 2505.09056 .

55.

Supriyono

WAP

Wibawa

Suyono , et al. (2024) A survey of text summarization: techniques, evaluation and challenges. Natural Language Processing Journal 7: 100070. https://doi.org/10.1016/j.nlp.2024.100070

56.

Yilmaz

(2017) Z Score. In: Allen

(ed) The SAGE Encyclopedia of Communication Research Methods. Sage.

57.

Zhang

Kishore

, et al. (2019) Bertscore: evaluating text generation with bert. arXiv, preprint arXiv:1904.09675v3 .

58.

Zhang

Jin

Meng

, et al. (2025) A comprehensive survey on automatic text summarization with exploration of LLM-based methods. arXiv, preprint arXiv:2403.02901v2 .

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.46 MB