Sage Journals: Discover world-class research

Abstract

The rationale for this article is that often, decision-makers in waste management (wm) tend to neglect goals and confuse them with means like circular economy or waste hierarchy. Because clear goals are crucial for developing effective wm strategies, the objectives of this mini review are (1) to clarify wm goals in a historical context by a literature review, (2) to investigate how (a) these goals have been observed in general scientific publishing and (b) specifically in Waste Management and Research (WM&R) and (3) to recommend measures for better consideration of wm goals by the publication sector. Based on general as well as specific bibliographic analyses of databases in Scopus and Google Scholar, the study confirms that little attention was given to wm goals in scientific publishing. For instance, during the first 40 years of WM&R, 63 publications and eight editorials were found containing terms related to wm goals, but only 14 respectively and eight explicitly discuss wm goals. We recommend focussing more on wm goals. Editors, authors, reviewers and professional associations in the field of wm should become aware of this challenge and react. If WM&R decides to become a strong platform for the issue wm goals, it will be in a unique selling proposition and more authors, articles and readers are likely to result. This article aims at setting a starting signal for such an endeavour.

Keywords

Goals of waste management Waste Management & Research literature review bibliographic analysis waste management objectives waste management strategy effective waste management

Introduction

The motivation for this article is that goals and means of waste management (wm) are often confused: The main wm goals are protection of humans and the environment, and conservation of resources (see below). The means to reach these goals are manifold and range from circular economy strategies to waste-to-energy (WTE), composting and safe disposal such as landfilling without after care.

The distinction between goals and means is crucial for the effectiveness of overall wm. Examples include the so-called waste hierarchy, or the circular economy strategy. Both are well suited means to cope with specific challenges of wm. But they cannot serve as ubiquitous and generally valid goals for wm. Legislation that is based on the waste hierarchy requires many exemptions from the hierarchy principle: Reuse is not always better than lower-ranked recycling, as illustrated by products with high emissions that can be replaced by clean products. Disposal of wastes in a WTE plant can be superior to higher-ranked recycling if wastes hold legacy substances that require complete (>99.99%) destruction. Equally, in a circular economy, waste plastics containing toxic chemicals that are out phased or even banned today are not suitable for recycling and must be removed from cycles.

Based on experience, the authors are convinced, that decisions which are primarily founded on the wm goals lead to more cost-effective solutions, protecting humans and conserving the environment and resources better than strategies focussing on specific means.

Therefore, the novelty and objectives of this article are the following:

To discuss the evolution of wm goals over time (evolution of wm goals).

To critically review how such goals have been considered in (a) all articles of the Scopus and Google Scholar database as well as in (b) the journal WM&R during the last 40 years (publishing wm goals).

To recommend measures and show the benefit how journals such as WM&R can focus more strongly on wm goals by incorporating them in their editorial policy (conclusions and recommendations).

The focus on WM&R is justified by the fact that this journal has been one of the very first journals directed specifically towards the topic of wm. It was started in January 1983 by the International Solid Waste and Public Cleansing Association as their official journal and replaced the former International Solid Waste Association (ISWA) journal. Since 1983, we counted 40 volumes of WM&R, containing 288 editorials by 178 editors and around 3600 articles by thousands of authors.

Since its first issue edited by R. B. Dean, T. H. Christensen and J. A. Hansen (Dean et al., 1983), WM&R aimed at being a leader in the exchange of information in practice and theory of wm. On the occasion of the 40th anniversary, it seems timely to explore how the journal approached the issue of wm goals, to discuss ways to improve its performance regarding wm goals and to propose ways to incorporate goal-oriented wm in future WM&R activities and issues. With the large and authoritative ISWA in the background of the journal, WM&R has the power and responsibility to take part actively in the definition and implementation of a goal-oriented wm.

Materials and methods

To address the three topics of this article, the following procedures have been chosen. Firstly, a working hypothesis for wm goals is established based on existing literature. Then, the results are used for bibliographic analyses how the wm community took these goals into account by their publication practice. Finally, based on these findings, recommendations are made to improve the consideration of wm goals in publishing. The review step of this study partly follows the Prisma protocol (Liberati et al., 2009).

Evolution of wm goals

In a first step, a literature search was performed to investigate the development of wm goals over time. Starting point was the seminal book ‘Unser Abfall aller Zeiten – eine Kulturgeschichte der Städtereinigung’ (‘Our waste ever – a cultural history of urban cleansing’) by G. Hösel (Hösel, 1990). This book covers the development of waste disposal over the last 6000 years. Hösel pointed out the various wm objectives of selected cultures throughout the course of history. He presented close to 700 literature citations which serve as an excellent source for further studies. These citations have been used on a case-by-case basis. In addition, our research work on wm goals published over the last 30 years and cited below has been exploited. The result of this first step was a comprehensive working hypothesis for wm goals.

Publishing wm goals

As a second step, bibliographic analyses were conducted to investigate how wm goals have been observed in publishing. To begin with, databases of Scopus and Google were screened according to selected search terms (see below). Later, the databases were specifically narrowed down to the journal WM&R. The focus was laid on WM&R because it is a major journal in its field, and the present article is a contribution to the 40th anniversary of WM&R.

Key search terms were selected based on the results of the first step, the definition of wm goals. Because keywords given to an article by the respective authors are sometimes of little importance for the content of this article, we first reassigned keywords to all articles according to our objectives and the content of the articles. Next, these new keywords were matched with the three topics: ‘protection of human health and environment’, ‘resource conservation’ and ‘no export in time and space’. The relevance of a keyword in an article was determined according to the direct importance of the keyword for the selected wm goals. In addition, the general content of the article was also considered for weighting the keyword, for example, articles on final sinks in most cases focus also on ‘no export in time’. Hence, relevance was always determined according to keywords and content of a article, with a limit of being relevant or not at a relevance level of 3.

All publications contained in the databases of Scopus and Google Scholar were screened for articles that contain the following search requests: (SRCTITLE (waste AND management AND research)) AND ((editorial)) AND (-predefined keyword-), whereas the predefined keyword was one of the following: ‘goals of waste management’, ‘waste management goals’, ‘targets of waste management’, ‘waste management targets’, ‘objectives of waste management’ or ‘waste management objectives’. The results revealed which goal-related terms are most often mentioned when wm goals are discussed in the general scientific literature. A comparison of the results of step 1 and 2 allowed to narrow down the broad range of goals used, and to decide which goal definition to use for the work with WM&R. Applying a critical review, we primarily focussed on editorials as they have been published in this journal. Editorials have been written by editors as well as by other, usually invited authors. To identify if and how wm goals have been considered in WM&R, the following procedure has been applied:

Firstly, all articles and editorials published by WM&R since its beginning in 1983 were screened by Scopus and Google Scholar for the above-named search requests. Furthermore, using all 40 issues of WM&R, a list of all 288 editorials of the last 40 years was produced and three to five (sometimes more) keywords were allocated to each editorial. On the one hand, the keywords were defined according to the content of the articles. On the other, they were chosen according to the goals deduced above. As this procedure resulted in more than 480 different keywords, they have been allocated to 16 thematic categories that had been defined beforehand. A list of all keywords and their related thematic categories is given in Supplemental Table S1. The categories were sorted after their potential relevance to the previously defined wm goals. In the last step, the actual relevance with respect to wm goals was evaluated according to two questions: To which goal or goals of wm does the respective editorial refer to? And in which relevant way does the content relate to the goals?

Relevant to which goal?

Protection of human health and environment

Resource conservation

No export in time and space

Scoring of relevance:

Content relates to goal and authors clearly denote it as a goal of wm

Content relates to goal, but authors do not clearly denote it as a goal of wm

Content relates to goals in a wider sense only

Weak or indirect relation to goals

No relation to goals of wm

Conclusions and recommendations

The recommendations have been deduced from the insights gained during steps (1) and (2) as well as from our earlier studies listed in the literature section. They represent the conclusions of the authors and are thus subject for debate. In fact, we hope that this article will start a discussion about the value of a goal-oriented wm, which distinguishes between aims and means.

Although a major part of this article is based on specific experience with WM&R, we address editors and authors of all journals in the field of wm. We aim at presenting the advantage of using a goal-oriented approach for the management of wastes and resources. The main purpose is to encourage the incorporation of wm goals in editorial policy. Editors and authors are invited to consider wm goals in their future publication activities.

Results and discussion

For the presentation of the results, the same structure has been kept as for the former sections: (1) ‘Evolution of wm goals’, (2) ‘Publishing wm goals’, particularly WM&R and (3) ‘Conclusions and recommendations’. The latter are divided into a general section for publishing in the field of wm and into a specific section for focussing specifically on WM&R and its parent organisation ISWA. This section may serve as a reference for similar journals with related professional associations in the background.

Evolution of wm goals

In Hösel (1990), practice and evolution of wm is presented from early civilisation to the state of the art at the end of the 20th century. In this book about wm phenomenology, goals are not explicitly discussed, but they serve as the implicit background for understanding why certain concepts and measures have been chosen. In another complementary approach, the metaphor of the Metabolism of the Anthroposphere (Baccini and Brunner, 2012) has been used for understanding wm as a function of the physiological and cultural requirements of human settlements. Or, in simpler words ‘what goes in must come out’ is true not only for all living things and thus also human beings, but for human settlements, too. Hence, the two parameters per capita consumption and population density decide how much waste is being produced per area. Based on these two historical and physiological descriptions of wm, the evolution of wm goals is described in the following sections.

Protection of human health

For ancient nomadic people, wastes were not much of a problem due to the extremely low population density. The scarcity of food for humans and animals was the dominant limit for development. Wastes did not play a crucial role due to their small amount: According to Baccini and Brunner, 2012, a prehistoric person needed in total for food, shelter, clothing, etc. about 6 tonnes of material per year. The input into the household metabolism consisted mainly of food. The resulting output amounted to 5.1 tonnes c.year⁻¹ off gas from breathing, 0.8 tonnes c.year⁻¹ excreta (faeces and urine) from food digestion and 0.1 tonne c.year⁻¹ wastes from tools, cloths and others. Hence, there was a minor amount of main biogenic material that had to be disposed of. Due to the composition of the input, most of these wastes could easily and quickly be metabolised to CO₂ and water. The remaining inorganic material did not regionally accumulate because the input was in equilibrium with the output. A hygienic problem was unlikely if dilution of microorganisms in water and soil was high enough. Hence, the main wm means of such prehistoric societies were degradation and dilution of wastes by natural processes. The main (implicit) wm goal was the protection of humans.

With the development from nomadic to sedentary cultures, two changes in metabolism came along: The physiological input–output of individuals stayed about the same. But the requirements for the production of food and – new – for shelter (housing and cloths) became more material-intensive (Baccini and Brunner, 2012). As a result, wastes changed in quality, and the amount of waste increased. But the most severe change was the regional accumulation of nondegradable waste. Many early settlements took advantage of rivers and lakes as means to dilute and transport waste downstream, thus preventing an accumulation problem within their habitat. Some cultures like pile-dwellers built their houses on swamps and marshlands. Wastes were disposed of below the dwellings, enabling existing ecosystems to rapidly degrade biogenic wastes. Again, as for nomads, the protection of human health from microorganisms and other nuisances was the main implicit goal of these early settlers.

During the last 10,000 years, with the development of cities, infrastructures, new materials and a corresponding growth in material turnover, wm became an increasing problem for large and expanding residential and urban areas (Baccini and Brunner, 2012). In particular, to ensure human health, early civilisations invented ingenious measures such as controlled sewage transport and disposal (Hösel, 1990). The main goal was still protection of human health, and thus wm policy aimed at removing wastes as fast as possible from human settlement. Removal remained the key strategy throughout several millennia, with resources recovery as a second option for those wastes that constitute an important resource like building wastes. Particularly in times of economic crisis after disasters such as earthquakes, floods or wars, waste materials of destroyed buildings were in high demand. In a cost balance between primary and secondary resources, the later were often advantageous because the expensive primary extraction step was not necessary anymore. The first recycling activities were not aiming at resource conservation, but at economic advantages.

The situation changed towards the end of the 17th century when epidemic plagues regularly hit many continents. Emerging scientific knowledge about the nature, development and transfer of diseases was the base for a new wm. A broad bundle of measures was designed to fight the transmission of pest, cholera, typhus and others. Open canals and pits were cleared, roads were cleaned and household wastes as well as sewage were disposed of in a controlled way at regular intervals. Often, such measures were taken by the authorities to maintain and improve the health of the workforce because it became clear that provision of personal hygiene and medical care alone were not enough to eliminate plagues (Ossberger, 1997). Thus, the protection of human health became established as the premier goal of wm.

The industrial revolution from approximately 1760 to 1840 started a new age of exceptionally large material turnover. The industrial processes were often accompanied by severe emissions and wastes, which were not regulated at the time. Mining activities – that actually started already before the Roman empire – became large scale and resulted in many valuable materials but even more wastes (Sörme et al., 2001). A substantial fraction of this new material was used in long-living infrastructure, for example, for transportation, manufacturing and supply, communication and housing. The corresponding wastes of this stock of long-living goods turned up about 50 to 200 years later; some of it must still be managed today, either as a threat to human health (e.g. remains of the early coal and town gas age) or as a secondary resource (waste from building material).

Protection of the environment

It is interesting to note that the industrial revolution also triggered some of the first environmental legislation. Beginning in 1833, so-called Factory Acts aimed at improving the harsh working conditions of the labour force in industry. Exceptionally long working hours and unfavourable working conditions were no longer accepted, and thus regulated by law in several civilised countries (e.g. see the Swiss Factory Law of 1877; Studer, 2022). In addition to the focus on people’s health, some countries extended the protection to the surroundings of factories, asking owners to reduce negative impacts on the environment. In most cases, this general requirement overexerted the knowledge and capacity of entrepreneurs as well as authorities, but it was an important first official statement that industrial activities cause environmental pollution, and that the polluter has the responsibility to prevent and reduce emissions. Thus, it may be concluded that the polluter pays principle is more than 150 years old. Hence, the extension of the goal protection of human health to protection of human health and environment can be traced back to the middle of the 19th century.

The increase in material turnover of the anthropogenic metabolism from about 6 t c.y⁻¹ to more than 200 t c.y⁻¹, and the more than 1000 times higher population density in urban areas had severe implications on humans and the environment. One of the most significant effects is the substantial impact of liquid, solid and gaseous waste flows on water, soil and air quality. During the past 10,000 years, for many substances, the anthropogenic loadings per unit of soil, water and air have increased more than 10,000-fold. Not all natural processes can cope with such high substance flows. For nondegradable, persistent substances that accumulate in specific environmental compartments, severe problems arose, in particular due to loadings that originate from waste flows and their treatment.

On a pure volume or mass base, wm poses little environmental challenges. The exploitation of materials from the earth crust (construction materials, ores, fossil fuels and others) create large holes in the crust. As long as the excavated materials are serving a purpose in buildings and infrastructure, or are consumed as energy carriers, these holes will stay empty and cannot be filled, the amount of resulting waste is too small. The disposal problem is thus not caused by the sheer mass of material but by the composition and quality: The use of materials results in the generation of waste and in dissipation (distribution of substances in a manner that does not allow its collection). Because modern materials hold many xenobiotic and hazardous substances, such substances are dissipating on a large scale, too. While dissipation is responsible for rising concentrations in global environmental compartments, the disposal of wastes often results in specific local and regional loadings. If not done properly, during waste treatment, hazardous substances in these wastes are released to the environment. Examples comprise the contamination of groundwater by open dump and landfills; air pollution and resulting soil pollution due to waste incineration without the necessary air pollution control; soil contamination by the application of refuse-derived compost and others. During the second half of the 20th century, these problems became visible and pressing, and protection of the environment advanced to the premier goal of waste legislation in many countries.

Conservation of resources

During the last two centuries, the huge and general increase in resource extraction and synthesis changed the anthropogenic as well as natural metabolism. A report by the Club of Rome in 1972 on the Limits to Growth presented the link between resource use and environmental degradation; it had a tremendous impact on wm (Meadows et al., 1972). Based on a global economic model of William Forrester, the authors showed prospective scarcities for many crucial elements, restricting global development and welfare if no countermeasures are taken. They promoted limiting growth of resource use and suggested recycling. While the issue of scarcity remained controversial (cf. the bet between Julian Simon and Paul Ehrlich about the price of commodities in the face of population explosion, which Simon won (Sabin, 2013)), the view that recycling of waste can reduce the amount of primary resources needed was widely accepted. Hence, conservation of resources become a prime goal of wm during the past 50 years. In fact, to use wastes to produce new goods has been a long-practised strategy to overcome material shortages during times of war, poverty or high uncertainty (e.g. Klinglmair and Fellner, 2010). The new policy of a circular economy is the culmination of a worldwide effort to make best use of valuable resources and to protect the environment.

Sustainability, or no export in time and space

In 1987, the United Nations (UN) Commission for Environment and Development published the so-called Brundtland-Report Our common future (UN, 1987). This report acknowledges that the interaction between humans and the planet they live on has profoundly changed. Nowadays, we have the power to radically alter planetary systems, endangering the atmosphere, soils, waters, the biosphere and their relationships with each other. The UN Commission says, that ‘The rate of change is outstripping the ability of scientific disciplines and our current capabilities to assess and advise’. Thus, humankind is asked to consider the needs of future as well as neighbouring generations, ensuring that everybody can take part in the use of resources in a healthy environment.

The Brundtland report had a distinct impact on wm in as far as the two components time and space got into the game: Sustainable wm may not postpone waste-derived problems into the future. Such issues must be solved by the generation that creates the problem. Hence, landfilling requiring long aftercare (e.g. >50 years) for leachates, air pollution and soil stability is not sustainable and must be replaced by waste recycling, pretreatment, and after-care-free disposal practices (including after-care-free landfilling). The same is true for recycling, composting, incineration and others. A sustainable wm requires that only two products are being produced: healthy and environmentally sound secondary resources that are ready for a market, and goods and emissions that are compatible with natural material flows and do not require aftercare. In some national legislations, such wm goals are explicitly mentioned, for example, the Austrian Federal Act on Waste Management (Bundesgesetz über eine nachhaltige Abfallwirtschaft AWG 2002) states explicitly that ‘waste treatment may only produce products that when disposed of do not compromise future generations’ (Bundesministerium für Verkehr Innovation und Technologie, 2002).

Similarly, to the no export in time requirement comes a no export in space. While autonomy is not a general goal per se, it is important that waste transports serve mainly the first two wm goals, and not just economic criteria neglecting long-term emissions. To protect low-income countries with less effective mechanisms for pollution control, waste exports became regulated in 1989 by the Basel Convention that controls the transboundary movement of hazardous waste (United Nations, 2020).

Other goals

In addition to the goals discussed above, other goals have been cited in the literature. An overview of definitions in journals and official documents shows a wide range. Often the denotations are ambiguous. For instance, in the 1970s, the goals of the German wm strategy were first reduction of the mass of wastes, second reuse and valorisation of wastes, and third to properly dispose of wastes according to the rules (Deutscher Bundestag, 1976). Protection of health and environment were not the focal point. The Waste Framework directive of the European Union (Directive 2008/98/EC of the European Parliament and of the Council of 19 November 2008 on waste and repealing certain Directives, 2008) requires ‘measures to protect the environment and human health by preventing or reducing the generation of waste, the adverse impacts of the generation and management of waste and by reducing overall impacts of resource use and improving the efficiency of such use, which are crucial for the transition to a circular economy and for guaranteeing the Union’s long-term competitiveness.’ To reach these goals, the Directive states a Waste Hierarchy of: (1) prevention; (2) preparing for reuse; (3) recycling; (4) other recovery, for example, energy recovery and (5) disposal.

In the Global Waste Management Outlook (Wilson and Carpinteria Rogero, 2015), the following operational goals are quoted for wm: By 2020, ensure access for all to adequate, safe, and affordable solid waste collection services, and eliminate uncontrolled dumping and open burning. By 2030, ensure sustainable and environmentally sound management of all wastes, particularly hazardous wastes, and substantially reduce waste generation through prevention and the 3Rs (reduce, reuse, recycle), thereby creating green jobs. Also, reduce by 50% global per capita food waste at the retail and consumer levels and reduce food losses in the supply chain.

The results of this first literature search suggests that different kinds or levels of goals can be discerned: General goals of universal validity, more specific goals that are often subgoals of the general goals and means as well as principles that are used as surrogates for goals. To illustrate this finding, the following example may serve: According to Wilson and Velis (2015), the five Global Waste Management Goals have been defined as: (1) to ensure access for all to adequate, safe and affordable solid waste collection services by 2020; (2) to stop uncontrolled dumping and open burning; (3) to achieve sustainable and environmentally sound management of all wastes, particularly hazardous wastes by 2030; (4) to ensure by 2030 a substantial reduction in waste generation through prevention and the 3Rs (reduce, reuse, recycle) and (5) cutting by a half per capita global food waste at the retail and consumer levels, and reducing food losses in the supply chain.

These five specific goals are actually subgoals of the general goals protection of human health and the environment, and conservation of resources. Again, they are also means. They are more precise including timeframes, stating specific waste categories (food waste, hazard waste, etc.), and addressing specific problems (uncontrolled dumping and open burning). They serve particularly well for low-income countries. The drawback is, that for becoming effective, the five subgoals need to be associated with priorities.

In addition, the following two conclusions can be drawn from the literature work: Firstly, the topic goals of wm is often treated ambiguously. There is rarely a clear distinction between goals on a strategic level and means – or subgoals – on an operational level. This endangers the overall effectiveness of wm. Clear goals and proper means are two different pair of shoes and should not be mixed up. Secondly, most information such as technical and scientific articles or legislation calls: (1) the protection of human health and environment as the main goal of wm, followed by (2) conservation of resources and (3) sustainability as a general notion. The latter often includes after carefree wm with no export of waste-related issues such as costs or emissions in time and space.

Publishing wm goals

The search using Scopus general database showed limited results only because merely titles, keywords and abstracts are searched by the engine. In total, 112 scientific works were found (see Supplemental Table S2), only nine of them address concrete wm goals. They name protection of the environment (Greene and Tonjes, 2014; Sharma et al., 1995) and human health, as well as resource conservation (Allesch and Brunner, 2016; Brunner and Fellner, 2007; Döberl and Huber, 2002; Markic et al., 2019; Pongracz et al., 2004; Stanisavljevic et al., 2015). Only Hahn (2000) mentioned ‘the sustainable disposal of waste substances by preventing the bioavailability to the environment of the substances contained in the waste’.

To generate a more comprehensive and meaningful outcome, the same search was performed by applying the search engine Google Scholar on the Google database, which according to Gusenbauer (2018) holds 389,000,000 articles, and most likely more today. This search clearly resulted in more matches, as the full texts are screened too (see Supplemental Table S3). For all keywords, a total of around 2000 publications has been found. Of them, around 1400 were published in the last 5 years. Articles that matched with the search term ‘goals of waste management’, really focussed on wm goals. Examples include Brunner (2013), and others discussing protection of human health and environment, conservation of resources and sometimes also aftercare (Khudyakova and Lyaskovskaya, 2021; Stanisavljevic and Brunner, 2019, 2020, 2021. Many articles name sustainability (Alkarimiah et al., 2022) and waste minimisation (e.g. Borshchuk and Vasylyshyn, 2021), as well as recycling (e.g. Muturi, 2021; Quicker et al., 2020) as wm goals.

Using the search term ‘waste management goals’, articles were found that first of all deal with policymaking and strategies to reach wm goals (e.g. Holden, 2022), especially for local authorities and specific waste stream management (e.g. Prata et al., 2022; Todor, 2021).

‘Targets of waste management’ as well as ‘waste management targets’ as search term resulted often in articles about special waste stream treatment (e.g. Thomas et al., 2022), and implementation of regional and EU wm targets (e.g. Oskarsson et al., 2022; Sebastian and Louis, 2021). Similarly, the wording wm objectives seems to be a common expression for policymaking, especially dealing with local authorities (‘. . .to reach the waste management objectives of. . .’; e.g. Batalhao et al., 2021).

The expression waste management objectives also gives results in the direction of concepts like Waste Hierarchy, (e.g. Roman, 2021), 3R principle (e.g. Usman et al., 2021), Circular Economy (e.g. Allevi et al., 2021) and Zero waste (e.g. Alkarimiah et al., 2022). Also named are concrete goals like protection of human health and environment (e.g. Kosoe et al., 2021), resource conservation (e.g. Suhag, 2021) and no export in space and time (e.g. Hettiaratchi et al., 2021). Ragossnig and Schneider (2017), named recycling as main goal of wm, and Ragossnig (2017) also named closed material loops and circular economy.

After this general investigation, a specific search limited to articles published in WM&R only has been performed on Google Scholar. This search resulted in 63 matches (see Supplemental Table S4). Only seven of them had a relevant content regarding wm goals. Those are mainly conservation of resources, protection of human health and no export in space and time (e.g. Agamuthu and Khidzir, 2009; Rechberger, 2001; Wilson and Velis, 2015). Ragossnig (2020) named maximising material recycling while achieving clean material loops in his editorial for WM&R. It is noteworthy, that none of these publications was written before the year 2001.

Furthermore, all editorials of WM&R have been screened for goal-oriented keywords because the direction and focus of a scientific journal is eminently shaped by their editors. All 288 issues since 1983 have been evaluated regarding their relevance to wm goals (as described in the methodical part). To do so, all editorials have been summarised by 3–5 (sometimes more) keywords (see Supplemental Table S1), which have subsequently been summarised into 16 categories of keywords (see Figure 1). This resulted in a total of 575 entries of keyword categories.

Figure 1.

Frequency of each keyword category in 288 editorials of WM&R.

The treatment of special waste types and treatment processes (WTE, composting, plastic recycling, etc.) was addressed most often (109 namings or 19% of all entries), followed by topics about waste policy and development (such as wm strategies, policymaking, etc.), which was mentioned 79 times (or 14%). Recycling and resource recovery was in the focus of 78 editorials (13%), whereas 61 times, no specific scientific topic related to wm was addressed (11%). Almost 10% of all editorials include environmental impacts of wm. Human health issues, which are also important wm goals, were considered in only about 2% of all editorials. Even lower matches were found for the topics waste exports, goals of wm and final sinks and aftercare, which all strongly relate to the goals of wm (see below).

Furthermore, it was examined how strongly each editorial focuses on one of the defined wm goals. Thereto, relevance scores from 1 to 5 (5 indicating the highest relevance), have been assigned to each editorial. For each keyword category, a relevance score regarding wm goals has been assessed (remember the description in Material and Methods – Review of Goals in WM&R).

The results show that of all categories investigated, 13 are of greater relevance for wm goals, having an average rating of maximum 3 (topic relates in a wider sense to wm goals) (see Table 1).

Table 1.

Wm goals and related topics are rarely discussed in WM&R editorials. Most of the 575 editorials are assigned to 2–3 categories. Bold frames indicate high relevance to wm goals (keyword categories with relevance score >3).

Keyword categories	Frequency [%]	Average relevance score (regarding wm goals)
Goals of waste management	1.22	5.0
Final sinks and aftercare	1.22	3.9
Waste export	1.39	3.6
Human health	2.26	3.4
Recycling and resource recovery	13.57	3.2
Informal sector	3.48	3.2
Environmental impacts	9.39	2.9
Landfills	3.48	2.9
Waste treatment processes	18.96	2.8
Waste policy and development	13.74	2.6
Data management	5.57	2.6
Economic aspects	5.57	2.4
Developing countries	4.17	2.4
Sustainability	2.09	2.3
Social aspects and public perception	3.30	2.2
No specific scientific topic	10.61	1.2

The keyword category ‘Goals of waste management’ relates to those editorials, which clearly name wm goals and declare them to be goals of wm. Hence, the average scoring for this category is 5 (highest relevance). Only 1.22% (seven editorials) refers to this category. Clearly, WM&R topics that deal with final sinks and aftercare are of great relevance to the topics of WM because in most cases they discuss long-term environmental challenges, and name this topic as an important goal of wm. The keyword category with the 3rd highest relevance score represents ‘waste exports’. Editorials dedicated to this topic mostly discuss this issue broadly but do not mention this to be a relevant goal of wm.

The question about human health often comes together in the context of informal wm (health threat for waste pickers and collectors in the informal sector) and worker’s health at dumpsites or in the collection systems and relates of course directly to the goals of wm.

Editorials that deal with recycling and resource recovery in general often focus on methods (such as nutrient recovery, WTE and others), but rarely say specific goals and targets to follow (relevance score of 3.2).

The topic sustainability should be relevant for wm goals, but in reality, it is often used in an overly broad context and with unclear denotation.

Out of 575 entries, 133 (or 23%) have been referred to the above named more relevant categories (score of relevance >3). These topics were named alone or in combination with other topics in the respective articles. Less than one-quarter of all entries refer to a category with high relevance for wm goals!

Figure 2 shows the distribution of editorials in view of their relevance to wm goals. Out of 288 editorials, seven (2.4%) of them deal with one or more wm goals, naming it explicitly and announcing it to be a goal of wm. About 57 (20%) editorials have been considered to deal with one or more wm goals, naming it clearly but not associating it with a goal of wm. In total, 75 (26%) editorials have been considered to deal with one or more wm goals, matching the topic broadly. The content of 51 (18%) editorials could be interpreted to refer to one or more wm goals. The majority of 98 (34%) of all editorials do not have any relation at all to wm goals.

Figure 2.

Frequency of editorials in WM&R according to their relevance to wm goals. 5: most relevant to wm goals, 1: no relevance to wm goals. Total number of WM&R editorials screened 288.

Our analysis reveals that wm goals become more relevant over time (Figure 3). The number of WM&R editorials focussing on goal topics increases. However, editorials addressing goals and clearly pointing out their importance are a phenomenon of the past 12 years. The first editorial published in that specific context, with a scoring of 5, was ‘Clean cycles and safe final sinks’ by Brunner (2010).

Figure 3.

Development of relevance for 40 years of WM&R editorials with respect to wm goals. The boxes show 50% of the editorials written in the respective years, and the whiskers include the other 50%. The trend line (dark line) is based on the average relevance score of the editorials from each volume from 1983 to 2023.

Which wm goals are addressed most often in WM&R editorials? Table 2 shows, to which goals the editorials refer to, and if, to what extent they do so. Out of 379 entries (some editorials refer to just one wm goal, some to all three and some do not refer to goals at all), 214 times (or in 56% of all entries) a wm goal was named in a relevant way, at least the topic was broadly matched in the respective editorial (scored with 3 and higher). In total, 67 times (or 18% of all editorials) goals could be interpreted from the context (scored with 2), and 98 times (or 26% of all editorials), no goal was matched at all (scored with 1).

Table 2.

Frequency of wm goals in WM&R’s editorials, and frequency of the respective relevance score 5 to 1 (with 5 showing the highest relevance), in %.

Waste management goal	Frequency of naming (%)	Frequency of editorials with relevance score of
Waste management goal	Frequency of naming (%)	5	4	3	2	1
Protection of human health and environment	37.2	1.8%	8.7%	17.2%	9.5%	0.0%
Resource conservation	30.3	1.8%	9.5%	11.6%	7.4%	0.0%
No export in time and space	6.6	1.3%	2.1%	2.4%	0.8%	0.0%
No relation to any goal	25.9	0.0%	0.0%	0.0%	0.0%	25.9%

The most frequently addressed topic was protection of human health and environment (37%), directly followed by resource conservation (30%). This finding also copes with the aims and scope of WM&R that ‘The challenge to society is to minimise these impacts, maintain an acceptable quality of the environment and sustain the quality of life and resource supplies for future generations’ (Sage Journals, 2022).

The topic of sustainability, as expressed by the goal ‘no export of wm problems in space and time’, was addressed less frequently (<7%). It appears that the concretisation of sustainability as a wm goal is more difficult. While the topic of waste exports should have become more popular due to the Basel Convention 1989 (United Nations, 2020), which regulates the transboundary movement of hazardous waste and the political weight of other international organisations, the no export in time is rarely discussed. The Basel Convention has already been established in 1989. Nevertheless, the number of articles published annually and focussing on waste exports increased only lately. When searching the total database of Google Scholar, entering ‘waste export’ OR ‘export of waste’ OR ‘waste transfer’ OR ‘transfer of waste’ the results increased from 31 articles in 1980 to 105 in 1990, 407 in 2000, 572 in 2010 and 1270 in 2022. Most of the export-related articles focus on plastic waste export and hence are strongly linked to China’s plastic import ban of 2018. The results for WM&R are similar, only two articles containing these search terms were published in 2020, four in 2021 and 10 in 2022. This shows that WM&R is slowly starting to pay attention to the export issue.

To solve waste-related issues here and now makes sense from an ethical point of view. From an economic standpoint, and under the condition that all other goals are met, it may be more effective to resolve waste problems in a least cost way. This may be in a distant place where boundary conditions are more favourable (climate, geology, etc.), or in the future when novel technologies allow more effective treatment. It seems timely to start such discussions, considering pros and cons of a no export in time and space goal. Particularly because sustainability is such a broad issue, such a discussion must include stakeholders from many fields such as social, political, economic and technological science.

Conclusions and recommendations

To define and follow distinct and explicit goals is an important prerequisite for cost-effective sustainable wm. Today, although goals exist and several countries state goals in their legislation, universal and internationally accepted wm goals have not been agreed upon yet. Hence, when defining strategies and principles for implementation of measures, an indispensable basis is missing. This is a drawback for present and future development in all fields of wm such as collection, recycling, treatment, WTE and final disposal. In our view, because of the lack of universal goals, global management practice follows various pathways that are sometimes determined by objectives originating beyond wm.

The results of this bibliographic review show that various wm goals are mentioned in the applied and scientific literature. Most often, protection of human health, protection of the environment, conservation of resources and a vague sustainability are cited as goals which are never disputed. However, it is rare that a specific focus of an article has been put on wm goals only. For instance, wm goals have not been in the focus of WM&R yet. Although editors and authors are aware of the topic, an in-depth debate over the benefit of a goal-oriented wm strategy cannot be discerned yet. A definition of goals that is generally accepted is still missing, and the fact that some of the published goals are contradicting each other is rarely discussed.

General recommendations

For improvement of wm on a strategic as well as a practical level, we propose:

To build awareness for the importance of wm goals. Case studies allow comparing the effectiveness of different wm strategies and bring to light the advantage of a goal-oriented approach (cf. Allesch and Brunner, 2016). Stakeholders must know the main wm goals when they decide about the organisation of a circular economy or the establishment of a waste hierarchy.

To define generally valid wm goals on the base of a broad discussion about wm goals. These covers (a) universally valid goals that form the base of wm and that are to be applied ubiquitously and (b) appropriate subgoals, principles and means that are required due to specific economic, cultural, technological or climatic conditions of a region. While step (a) is more of a global consensus finding process among the various stakeholders, step (b) serves for differentiation and individual operationalisation of the general goals by the regional wm community concerned.

The engagement of public and private stakeholders in these discussions is indispensable and should comprise international organisations (ISWA, International Waste Working Group, Air & Waste Management Association, European Waste Management Association, Zero Waste Europe, etc.) as well as national bodies regulating wm, key industries from the waste sector, Non-Governmental Organisations and others.

3. Based on step (2), to incorporate the generally valid, global wm goals in the bylaws, statutes and articles of recognised international institutions such as WHO, UN, ISWA and others.

4. Based on steps (2) and (3), to discuss, define and recommend auditing methods that allow measuring if wm goals are reached (e.g. Verein Deutscher Ingenieure (VDI) 2016 – Guideline 3925 Methods for evaluation of waste treatment processes (VDI, 2016)).

Recommendations for publishing

In general, publishers of wm literature should make wm goals clear to editors, authors, reviewers and readers. Awareness for a goal-oriented approach is important and can be promoted if editorial policies make sure that: (1) journals state wm goals as one of their prime general target; (2) editors address the topic of goals once or twice a year in their editorial, presenting the state of the art, showing progress in definition, consensus finding and implementation and discussing critically which goals change or remain the same. In addition, if trends become visible that wm goals are compromised by new wm strategies or practices, editors should set off alarm bells; (3) journal guidelines ask authors to explicitly state in their letter to the publisher how their article relates to or even supports wm goals quoted in the respective journal and (4) guidelines require reviewers to check for wm goal orientation of manuscripts.

The broad discussion about wm goals suggested above should also be promoted, followed and presented by the relevant journals in the field. The inclusion of discussions and their results in editorials and articles will improve awareness for the goal issue, particularly if editors promote articles that focus on past, present and future goals and their development.

While it is comparatively straight forward to address the limited numbers of editors in the field of wm, the real challenge are the many reviewers who actually control the quality of scientific journals (Brunner and Cossu, 2015). It is of utmost importance that they know and assert the difference between wm goals and means, and that they point out corresponding deficits in articles.

Specific recommendations for WM&R

The bibliographic investigation shows that WM&R has treated the issue of wm goals so far mainly implicitly. Wm goals exist, but they are little known, and sometimes disregarded. Distinct editorials, articles and reviews on general goals have not been published yet. The same is true for competing journals. If in future WM&R decides to become a strong platform for the issue wm goals, this will be a unique selling proposition. If well-advertised, more authors, articles, readers and journal editions are likely to result. Goal-oriented, effective wm will be promoted. The current article might present a proper starting point for such an endeavour.

The journal WM&R is in a special situation because it is the publication organ of a large global institution, the ISWA. A strong asset of ISWA are the 10 working groups that are focussing on individual wm topics. If within ISWA a new working group on wm goals is formed, a fast information transfer from the working group to the readers of WM&R becomes possible. We thus recommend forming a new temporary working group for discussion and streamlining ISWA policy on goals ensuring that both ISWA and WM&R follow the same goals. In addition, a common guideline on goal-oriented wm could serve all working groups as a directive for their activities. ISWA and WM&R could also play a leading role in the transfer of goal-related knowhow from academia and practice to regulatory bodies.

After 40 years of active publishing in the field of wm, it is timely for WM&R to focus on the foundation of its science, engineering and practice the wm goals. ISWA, publisher, editors, authors and reviewers are asked to pay more attention to the goals and to encourage authors and readers of WM&R to focus more on this core issue of our business.

Supplemental Material

sj-docx-1-wmr-10.1177_0734242X231172104 – Supplemental material for Improving waste management by focussing on goals: A mini review of the publication sector

Supplemental material, sj-docx-1-wmr-10.1177_0734242X231172104 for Improving waste management by focussing on goals: A mini review of the publication sector by Paul H Brunner, Astrid Unterberger and Johann Fellner in Waste Management & Research

Supplemental Material

sj-docx-2-wmr-10.1177_0734242X231172104 – Supplemental material for Improving waste management by focussing on goals: A mini review of the publication sector

Supplemental material, sj-docx-2-wmr-10.1177_0734242X231172104 for Improving waste management by focussing on goals: A mini review of the publication sector by Paul H Brunner, Astrid Unterberger and Johann Fellner in Waste Management & Research

Supplemental Material

sj-docx-3-wmr-10.1177_0734242X231172104 – Supplemental material for Improving waste management by focussing on goals: A mini review of the publication sector

Supplemental material, sj-docx-3-wmr-10.1177_0734242X231172104 for Improving waste management by focussing on goals: A mini review of the publication sector by Paul H Brunner, Astrid Unterberger and Johann Fellner in Waste Management & Research

Supplemental Material

sj-docx-4-wmr-10.1177_0734242X231172104 – Supplemental material for Improving waste management by focussing on goals: A mini review of the publication sector

Supplemental material, sj-docx-4-wmr-10.1177_0734242X231172104 for Improving waste management by focussing on goals: A mini review of the publication sector by Paul H Brunner, Astrid Unterberger and Johann Fellner in Waste Management & Research

Footnotes

Acknowledgements

The authors are grateful to David Laner for his valuable comments on an early draft of this article.

Declaration of conflicting interests

The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The authors received no financial support for the research, authorship, and/or publication of this article.

ORCID iDs

Paul H Brunner

Johann Fellner

Supplemental material

Supplemental material for this article is available online.

References

Agamuthu

Khidzir

(2009) Drivers of sustainable waste management in Asia. Waste Management & Research 27: 625–633.

Alkarimiah

Makhtar

Aziz

(2022) Energy recovery from solid waste. Solid Waste Engineering and Management 3: 231–297.

Allesch

Brunner

(2016) Benchmarking- Austrian waste management: Are the objectives of waste management achieved? Österreichische Wasser- und Abfallwirtschaft 68: 405–414.

Allevi

Gnudi

Konnov

, et al. (2021) Municipal solid waste management in circular economy: A sequential optimization model. Energy Economics 100: 105–383.

Baccini

Brunner

(2012) Metabolism of the Anthroposphere: Analysis, Evaluation, Design. Cambridge, MA: MIT Press.

Batalhao

Denilson

Cristina

, et al. (2021) Waste Management in Brazil. Waste Management Policy and Practices in BRICS Nations. Boca Raton: CRC Press.

Borshchuk

Vasylyshyn

(2021) Theoretical and applied principles of the formation of waste management systems. Economics and Education 6: 31–35.

Brunner

(2010) Clean cycles and safe final sinks. Waste Management & Research 28: 575–576.

Brunner

(2013) Cycles, spirals, and linear flows. Waste Management & Research 31: 1–2.

10.

Brunner

Cossu

(2015) Reviewers: The ultimate resource for scientific publishing, Waste Management 35: 1–2.

11.

Brunner

Fellner

(2007) Setting priorities for waste management strategies in developing countries. Waste Management & Research 25: 234–240.

12.

Bundesministerium für Verkehr Innovation und Technologie (2002) Bundesgesetz über eine nachhaltige Abfallwirtschaft (Abfallwirtschaftsgesetz 2002 – AWG 2002), Vienna, Austria.

13.

Dean

Christensen

Hansen

(1983) Editorial. Waste Management & Research 1: 1–2.

14.

Deutscher Bundestag (1976) Abfallwirtschaftsprogramm 75 der Bundesregierung, Drucksache 7/4826 Bonn-Bad Godesberg 1: Dr Hans Heger.

15.

Directive 2008/98/EC of the European Parliament and of the Council of 19 November 2008 on waste and repealing certain Directives (2008) Official Journal of the European Union. L312: 3–30.

16.

Döberl

Huber

(2002) Long-term assessment of waste management options – a new, integrated and goal-oriented approach. Waste Management & Research 20: 311–327.

17.

Greene

Tonjes

(2014) Quantitative assessments of municipal waste management systems: Using different indicators to compare and rank programs in New York State. Waste Management 34: 825–836.

18.

Gusenbauer

(2018) Google Scholar to overshadow them all? Comparing the sizes of twelve academic search engines and bibliographic databases. Scientometrics 118: 177–214.

19.

Hahn

(2000) Future demands on waste management and current legislation. Wasser und Boden 52: 4–6.

20.

Hettiaratchi

Jayasinghe

Yarandi

, et al. (2021) Innovative practices to maximize resource recovery and minimize greenhouse gas emissions from landfill waste cells: Historical and recent developments Journal of the Indian Institute of Science 131: 537–556.

21.

Holden

(2022) The inverted waste pyramid as a policy tool to Unite SDG 11 and SDG 12 waste management targets SDG 11 In: Ramani

Hettiarachchi

(eds) Sustainable cities and communities, London: Routledge India.

22.

Hösel

(1990) Unser Abfall aller Zeiten. München: Jehle Verlag.

23.

Khudyakova

Lyaskovskaya

(2021) Improving the sustainability of regional development in the context of waste management. Sustainability 13: 1755.

24.

Klinglmair

Fellner

(2010) Urban mining in times of raw material shortage: Copper management in Austria during World War I. Journal of Industrial Ecology 144: 666–679.

25.

Kosoe

Osumanu

Darko

(2021) Connecting solid waste management to sustainable urban development in Africa. In: Addaney M and Cobbinah P (eds) Sustainable urban futures in Africa, New York: Routledge.

26.

Liberati

Altman

Tetzlaff

, et al. (2009) The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate healthcare interventions: Explanation and elaboration BMJ 339: b2700.

27.

Markic

Carapina

Bjelic

(2019) Using material flow analysis for waste management planning. Polish Journal of Environmental Studies 28: 255–265.

28.

Meadows

Randers

, et al. (1972) The Limits to Growth: A Report to The Club of Rome. Washington, DC: Potomac Associates – Universe Books.

29.

Muturi

(2021) Influence of stakeholders participation and management of solid waste disposal. A critical literature review. Journal of Environment 1: 14–29.

30.

Oskarsson

Agnarsson

Davidsdottir

(2022) Waste management in Iceland: Challenges and costs related to achieving the EU municipal solid waste targets. Waste Management 151: 131–141.

31.

Ossberger

(1997) Geschichte der Abfallwirtschaft in Österreich, Diploma Thesis: TU Vienna.

32.

Pongracz

Phillips

Keiski

(2004) Evolving the theory of waste management: defining key concepts. In: Popov

Itoh

(eds), Waste Management and the Environment II. The Waste Minimization and Resources Optimization Conference. Oulu: Oulu University Press, pp. 471–480.

33.

Prata

Patricio Silva

Durate

, et al. (2022) The road to sustainable use and waste management of plastics in Portugal. Frontiers of Environmental Science and Engineering 16: 5.

34.

Quicker

Grosso

Cosonni

(2020) The zero waste utopia and the role of waste-to-energy. Waste Management & Research 38: 481–484.

35.

Ragossnig

(2017) What is the real scope of waste management? Waste Management & Research 35: 1097–1098.

36.

Ragossnig

(2020) Construction and demolition waste – Major challenges ahead! Waste Management & Research 38: 345–346.

37.

Ragossnig

Schneider

(2017) What is the right level of recycling of plastic waste? Waste Management & Research 35: 129–131.

38.

Rechberger

(2001) The use of statistical entropy to evaluate the utilisation of incinerator ashes for the production of cement. Waste Management & Research 19: 262–268.

39.

Roman

(2021) Essential aspects of transition to a circular economy from waste management perspective. The USV Annals of Economics and Public Administration 2: 14–20.

40.

Sabin

(2013) Bet - Paul Ehrlich, Julian Simon, and Our Gamble Over Earth’s Future New Haven, CT: Yale University Press.

41.

Sage Journals (2022) Available at: https://journalssagepubcom/aims-scope/WMR (accessed 1 October 2022).

42.

Sebastian

Louis

(2021) Understanding waste management at airports: A study on current practices and challenges based on literature review. Renewable and Sustainable Energy Reviews 147: 1–19.

43.

Sharma

Gupta

Sushil

(1995) The objectives of waste management in India: A futures inquiry. Technological Forecasting and Social Change 48: 285–309.

44.

Sörme

Bergbäck

Lohm

(2001) Century perspective of heavy metal use in urban areas. A case study in Stockholm. Water, Air and Soil Pollution: Focus 1: 197–211.

45.

Stanisavljevic

Brunner

(2019) Quantity AND quality: New priorities for waste management. Waste Management & Research 37: 665–666.

46.

Stanisavljevic

Brunner

(2020) The user – beneficiary or victim of modern waste management systems? Waste Management & Research 38: 597–598.

47.

Stanisavljevic

Brunner

(2021) Megacities need both: Circular economy and final sinks! Waste Management & Research 39: 437–439.

48.

Stanisavljevic

Vujovic

Zivancev

(2015) Application of material flow analysis as a decision support tool for waste management in small municipalities- case study of Serbia. Waste Management & Research 33: 550–560.

49.

Studer

(2022) Fabrikgesetze Historisches Lexikon der Schweiz (HLS), vers 06082021: https://hls-dhs-dssch/de/articles/013804/2021-08-06/

50.

Suhag

(2021) Solid waste management Basic concepts in environmental technology. Boca Raton: CRC Press.

51.

Thomas

Osinska

Slosarczyk

(2022) Long-term behaviour of cement mortars based on municipal solid waste slag and natural zeolite—a comprehensive physico-mechanical, structural and chemical assessment. Materials 15: 1–16.

52.

Todor

(2021) Municipal waste management. In: Todor

Helepciuc

(eds) Europeanization of Environmental Policies and their Limitations, Cham: Springer.

53.

UN (1987) Our Common Future – Brundtland Report. World Commission on Environment and Development (WCED), New York, NY: United Nations.

54.

United Nations (2020) Basel convention on the control of transboundary movements of hazardous wastes and their disposal United Nations, Geneva: UN Environment Program, Basel Convention (UNEP/BRS/2014/3/Rev2).

55.

Usman

Lamangida

Akhbar

(2021) Implementation of waste management policy with 3R principles (Reduce, Reuse, Recycle) in the Gorontalo City. Journal of Public Representative and Society Provision 1: 29–37.

56.

VDI (2016) Methods for evaluation of waste treatment processes, Guiding document 3925. Düsseldorf, Germany: Verein Deutscher Ingenieure e.V., p. 163.

57.

Wilson

Carpintero Rogero

(2015) Waste management as a political priority In: Wilson D (eds) Global Waste Management Outlook, Chapter 1. Paris & Vienna: United Nations Environmental Programme (UNEP) and International Solid Waste Association (ISWA).

58.

Wilson

Velis

(2015) Waste management – still a global challenge in the 21st century: An evidence-based call for action. Waste Management & Research 33: 1049–1051.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.03 MB

0.02 MB