Sage Journals: Discover world-class research

Abstract

Background

Fetal growth restriction, i.e. the restriction of genetically predetermined growth potential due to placental dysfunction, is a major cause of neonatal morbidity and mortality. The consequences of inadequate fetal growth can be life-long, but the risks can be reduced substantially if the condition is identified prenatally. Currently, screening strategies are based on ultrasound detection of a small-for-gestational age fetus and do not take into account the underlying vascular pathology in the placenta. Measurement of maternal circulating angiogenic biomarkers placental growth factor, sFlt-1 (soluble FMS-like tyrosine kinase-1) are increasingly used in studies on fetal growth restriction as they reflect the pathophysiological process in the placenta. However, interpretation of the role of angiogenic biomarkers in prediction of fetal growth restriction is hampered by the varying design, population, timing, assay technique and cut-off values used in these studies.

Methods

We conducted a systematic-review in PubMed (MEDLINE), EMBASE (Ovid) and Cochrane to explore the predictive performance of maternal concentrations of placental growth factor, sFlt-1 and their ratio for fetal growth restriction and small-for-gestational age, at different gestational ages, and describe the longitudinal changes in biomarker concentrations and optimal discriminatory cut-off values.

Results

We included 26 studies with 2514 cases with small-for-gestational age, 27 cases of fetal growth restriction, 582 cases mixed small-for-gestational age/fetal growth restriction and 29,374 reference. The largest mean differences for the two biomarkers and their ratio were found after 26 weeks of gestational age and not in the first trimester. The ROC-AUC varied between 0.60 and 0.89 with sensitivity and specificity matching the different cut-off values or a preset false-positive rate of 10%.

Conclusions

Most of the studies did not make a distinction between small-for-gestational age and fetal growth restriction, and therefore the small-for-gestational age group consists of fetuses with growth restriction and fetuses that are constitutionally normal. The biomarkers can be a valuable screening tool for small-for-gestational age pregnancies, but unfortunately, there is not yet a clear cut-off value to use for screening. More research is needed to see if these biomarkers are sufficiently able to differentiate growth restriction on their own and how these biomarkers in combination with other relevant clinical and ultrasound parameters can be used in clinical routine diagnostics.

Keywords

Biochemical markers small-for-gestational age fetal growth restriction placenta

Introduction

It has long been acknowledged that small-for-gestational age (SGA) neonates are at increased risk of perinatal mortality and morbidity and have a higher risk of developing disabilities in childhood, and severe cardiovascular and metabolic diseases later in life.^1,2 The obvious reason is that, among the SGA neonates, a subgroup has suffered from intrauterine or fetal growth restriction (FGR), the rest being healthy and constitutionally small due to their demographic factors. Neonates with FGR have failed to reach their genetically predetermined growth potential due to placental dysfunction. The consequences of inadequate fetal growth can be life-long, but the risk of perinatal morbidity and mortality can be reduced substantially if the condition is identified prenatally.^1,3 An optimal screening strategy for the prediction of SGA and mainly those with fetal growth restriction is highly desirable.^4,5 Currently, screening strategies focus on antenatal ultrasound to detect small fetuses, defined as a fetal abdominal circumference or an estimated fetal weight (EFW) below the 10th percentile. However, this ultrasound-based strategy ignores the underlying pathological process in the placenta that leads to suboptimal supply of nutrients to the growing fetus. Abnormal vascular growth and transformation in the placenta can create a high vessel resistance network that has been associated with the development of fetal growth restriction and also pre-eclampsia.⁶

The vascular endothelial growth factor (VEGF) family plays a major role in regulating vasculogenesis and angiogenesis in early human placentation.⁷ Placental growth factor (PlGF) is a pro-angiogenic factor predominantly expressed in placental trophoblast and binds to VEGF-receptor or fms-like tyrosine-1 (Flt1), thereby enhancing the pro-angiogenic action of VEGF-A. Besides this membrane-bound receptor form, Flt-1 also exists in a soluble form (sFlt-1), which is secreted by the placenta and scavenges VEGF and PlGF, thereby lowering circulating concentrations of unbound VEGF and PlGF. Reduced placental perfusion and subsequent placental hypoxia are associated with increased production of sFlt-1 and decreased concentrations of PlGF in both human and animal models.⁸ This altered balance in pro-/anti-angiogenic factors is postulated to cause generalized endothelial dysfunction that is characteristic of pre-eclampsia which makes endothelial dysfunction a hallmark feature of pre-eclampsia. Fetal growth restriction has pathophysiological features in common with pre-eclampsia. Therefore, it was conceivable that measuring these placental biomarkers in maternal blood could be used in the prediction of pre-eclampsia and fetal growth restriction.^6,9

The serum sFlt-1/PlGF ratio has already been used in improving the clinical management for pre-eclampsia. Pre-eclampsia is defined as the new onset of hypertension and proteinuria after 20 weeks of gestation. This disorder is associated with a high risk of fetal growth restriction and maternal morbidity. A low sFlt-1/PlGF ratio was highly predictive for ruling out pre-eclampsia and avoiding unnecessary hospitalization with a significant economic impact.¹⁰ At this moment, there is no conclusive evidence that measuring maternal angiogenic biomarkers are (also) useful in the clinical management of fetal growth restriction. Despite many studies linking maternal angiogenic concentrations to the neonatal outcomes (mainly SGA),^3,11–16 there is no systematic review of the predictive performance of PlGF, sFlt-1 and sFlt-1/PlGF ratio at different trimesters of pregnancy.

The main purpose of this systematic review is to assess studies that evaluate the predictive performance of placental angiogenic biomarkers (PlGF, sFlt-1 and/or the sFlt-1/PlGF ratio) for SGA and fetal growth restriction. First, we will describe differences in serum biomarker concentrations between SGA and reference pregnancies at different gestational age (GA) periods. Next, we will present the longitudinal trajectories of PlGF, sFlt-1 and their ratio for SGA and reference pregnancies. And third, we evaluate the prognostic performance of these biomarkers for the prediction of SGA.

Methods

Search strategy and eligibility criteria

We performed a systematic search according to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses for Protocols 2015 (PRISMA-P 2015).¹⁷ The databases PubMed (NCBI), EMBASE (Ovid) and the Cochrane Library were searched for original studies investigating the biomarkers PlGF, sFlt-1 and/or the sFlt-1/PlGF ratio in SGA or FGR, from database inception until 27 July 2018. Medical subject headings (MeSH) and text words that we used are listed in Supplementary Table 1, which were combined with Boolean Operators: AND and OR. The included languages were English and Dutch. There was no restriction applied on the type of study. Exclusion criteria were studies where no full text was available, or those performed in multiple pregnancies, nonviable pregnancies, or the measurements being performed after the diagnosis of SGA/FGR was made, as this is in disagreement with the aim of the review. Studies were also excluded if they had no reference group without SGA, did not report absolute values, specific GA timing or the assay methods used of biomarker measurements. Additionally, the reference lists of the selected articles were screened to add all relevant studies. The study protocol was approved by the medical ethical committee of the Maastricht University Medical Centre (17–4-0.15.1/ab). All procedures were in accordance with institutional guidelines and adhered to the principles of the Declaration of Helsinki and Title 45, U.S. Code of Federal Regulations, Part 46, Protection of Human Subjects (revised 13 November 2001, effective 13 December 2001).

Study selection

An initial title and abstract review was performed by two independent reviewers (MH and AH), using Rayyan.¹⁸ A third reviewer aided in the exclusion of duplicates and ineligible papers after first selection (S.A.). Next, two investigators analysed the full-text articles to determine their eligibility in the review (M.H. and A.H.) Any disagreement was resolved through discussion. If consensus could not be achieved, a third reviewer (S.A.) was consulted.

Data extraction

Data from the eligible studies were extracted by two independent reviewers (M.H. and A.H.) using a predesigned data collection form. The data on first author, publication year, country, type of study design, characteristics of the population studied, sample size, GA at sampling, cut-offs used, sensitivity, specificity, area under the curve and assay type. If information was missing, corresponding authors were contacted by email.

Quality assessment

The methodological quality of the included studies was assessed using the Quality Assessment of Diagnostic Accuracy Studies-2 (QUADAS-2) tool.¹⁹ The risk of bias is judged in four main domains: (1) patient selection, (2) index test, (3) reference standard and (4) flow of patients in the study and timing of the index test and the reference standard. Concerns about the applicability of the test for clinical practice are also judged in the first three domains.

Data synthesis

In order to be able to quantitatively assess the relationship between SGA or FGR and the biochemical markers, we converted data given in median or multiple of the median (MoM) with IQR to mean with SD, using an online calculator of the method devised by Hozo et al.²⁰ Next, the data were categorized by GA at time of sampling. The categories made are: 9–14 weeks, 15–25 weeks, 26–35 weeks and ⩾36 weeks of GA, for both SGA and FGR definitions.

The primary outcome for each study was the mean difference in PlGF, sFlt-1 and their ratio between the SGA and reference group, reported with 95% confidence interval (CI).

The ratio between total variation due to heterogeneity and total variation (I² statistics) is presented as a measure for heterogeneity and was used to aid the decision between a fixed effect and random effects meta-analysis. I² can distinguish true heterogeneity from sampling variance and is expressed as a percentage. Sources of heterogeneity (assay of biomarker measurement, cut-off of SGA) and the difference between reference and SGA pregnancies could not be investigated by meta-regression analyses using a mixed-effects model because of the low amount of studies. Studies were pooled using inverse-variance weighting and were summarized using forest plots. The second outcome was the prognostic performance of these biomarkers and, if available, of the cut-off values to differentiate between SGA versus reference group. The last outcome was the presentation of the longitudinal levels of the PlGF, sFlt-1 and their ratio for SGA and reference pregnancies, with a difference in assay used. The meta-analyses were performed in R version 3.2.3 using the meta-package.

Results

Study and data selection

Our search identified a total of 593 studies (Figure 1). After first selection by title and abstract, duplicates were removed and 489 studies were screened based on the title and abstract, leaving 43 full-text articles to be analysed in detail. In this phase, studies were excluded because of the following reasons: no reference group without SGA (n = 9), no absolute values available (n = 5), no test method described (n = 1), no full-text available (n = 1), no specific defined GA range (n = 1).

Figure 1.

Flowchart summarizing study selection process.

The 26 included articles analysed were published between 2005 and 2018, from all over the world. A total of 2514 cases of SGA, 27 cases of FGR and 582 cases of mixed SGA/FGR (i.e. both FGR were identified during pregnancy and SGA) were included, in addition to a total of 29,374 ‘reference’ pregnancies used as controls in the different studies.

Study characteristics are presented in Table 1. Most studies employed the case–control design^{2,4,15,21–33} and 10 studies were prospective cohorts.^{1,3,13,14,16,34–38} The majority of studies recruited patients with SGA as outcome.^{1–4,13–16,21–23,25,26,28–38} The definition for SGA was lower than the 10th birthweight percentiles upper reference limit in 18 studies.^{3,4,13–16,21–23,25,26,29–32,34,35,37} Five studies used the cut-off a birthweight lower than the 5th percentile.^1,2,28,36,38 One study used a birthweight lower than the 3rd percentile.²⁴ One study used a birthweight lower than 2 standard deviations as cut-off value.³³ One study used FGR: estimated fetal weight < 10th and an umbilical artery PI > 95th as cut-off value.²⁷ Three studies described a population with pre-eclampsia as comorbidity. This differs from 1.3 to 18.4% PE in the study population and 2.1–5.0% in the reference group.^4,24,31

Table 1.

Characteristics of included studies.

Reference	Country	Study design	Cases (n)	Reference group (n)	Reference group	GA at samplingCase (weeks)	GA at samplingReference(weeks)	SGA/FGR	PE casesreference(n)	Assay kit
Bakalis et al¹	UK	Prospective cohort	469	9003	Normal pregnancy	32.1 (31.9–32.6)	32.2 (32.0–32.5)	SGA: BW <5th	0%	RocheDiagnostics
Berkane et al.²¹	France	Nested case–control	24	40	Normal pregnancy	26.7 ± 1.04	26.6 ± 0.96	SGA: BW <10th	0%	sFlt-1: R&D SystemsPlGF: Alere
Birdir et al.²	Germany	Case–control	20	65	Normal pregnancy	12.6 ± 0.6	12.6 ± 0.4	SGA: BW <5th	0%	Thermo Fisher
Birdir et al.¹³	Germany	Prospective cohort	93	676	Normal pregnancy	34.1 ± 2.5	34.6 ± 2.3	FGR: BW <5th a.umb PI >95th centile.SGA: BW <10th	0%	Thermo Fisher
Borras et al.²²	Spain	Case–control	14	28	Normal pregnancy	≤14	≤14	SGA: BW <10th	0%	Bender Medsystems
						15–28	15–28
						>28	>28
Boucoiran et al.³⁴	Canada	Prospective cohort	56	533	Normal pregnancy	12–18		SGA: BW <10th	0%	sFlt-1: Beckman-CoulterPlGF: Perkin Elmer
						24–26
Boutsikou et al.³⁵	Greece	Prospective cohort	25	15	AGA	37.9 ± 1.3	38.8 ± 1.1	SGA: BW <10th	0%	R&DSystems
Chaiworapongsa et al.²³	USA	Case–control	53	135	Normal pregnancy	36.9 (25–39.7)	37.6 (20–41.7)	SGA: BW <10th	0%	R&DSystems
Crovetto et al.²⁴	Spain	Nested case–control	462	702	Normal pregnancy	10 ± 0.9	10 ± 0.9	FGR or BW <3rd	85 (18.4%)183 (2.1%)	R&DSystems
Darling et al.³	Tanzania	Prospective cohort	60	363	AGA	22 (19.4–24.6)	22 (19.4–24.6)	SGA: BW <10th	0%	R&DSystems
Erez et al.²⁵	USA	Case control	145	201	Normal pregnancy	12.0 (10.7–13.7)	12.2 (11–13.4)	SGA: BW <10th	*	R&DSystems
						22.7 (21.3–23.9)	22.3 (21.1–23.7
Fadigas et al.³⁶	UK	Prospective cohort	158	3701	Normal pregnancy	36.3 (36.0–36.4)	36.1 (36.0–36.4)	SGA: BW <5th	0%	RocheDiagnostics
Fruscalzo et al.²⁶	Germany	Case–control	54	37	AGA	12.33 ± 1.44	12.47 ± 0.67	SGA: BW <10th	*	RocheDiagnostics
Furuta et al.³⁷	Japan	Prospective cohort	13	170	Normal pregnancy	25 (24–25)	24 (24–27)	SGA: BW <10th	0%	R&DSystems
Gaccioli et al.¹⁴	UK	Prospective cohort	84	3981	Normal pregnancy	28.1 (27.9–28.3)	28.3 (28.0–28.6)	SGA: BW <10th	0%	RocheDiagnostics
						36.0 (35.7–36.1)	36.1 (36–36.4)
Herraiz et al.²⁷	European multicentre	Case–control	27	171	Normal pregnancy	28.1 ±2.5	29.4 ±3.0	FGR: EFW <10th or a.umb PI >95th	0%	RocheDiagnostics
						35.3 ±1.0	37.2 ±2.2
Karagiannis et al.³⁸	UK	Prospective cohort	274	1869	Normal pregnancy	11–13	11–13	SGA: BW <5th	0%	R&DSystems
Rizos et al.¹⁵	Greece	Case–control	14	87	Normal pregnancy	12.7 ±0.5	12.5 ±0.5	SGA: BW <10th	0%	RocheDiagnostics
						22.0 ±1.3	21.7 ±0.9
						31.7 ±1.2	31.7 ±1.6
Savvidou et al.²⁸	UK	Case–control	15	42	Normal pregnancy	23–25	23–25	SGA: BW <5th	0%	R&DSystems
Shibata et al.²⁹	USA	Case–control	22	29	Normal pregnancy	39.1 ±1.6	40.0 ±1.0	SGA: BW <10th	0%	R&DSystems
Thadhani et al.³⁰	USA	Case control	40	80	Normal pregnancy	9.8 ± 1.6	10.6 ± 3.1	SGA: BW <10th	0%	R&DSystems
Triunfo et al.,³¹	Spain	Case–control	80	80	AGA	34.9 ±1.1	34.7 ±1.2	SGA: BW <10th	1 (1.3%)4 (5.0%)	R&DSystems
Triunfo et al.⁴	Spain	Nested case–control	46	92	AGA	9.9 ± 0.8	9.9 ± 1	SGA: BW <10th	6 (13.0%)2 (2.2%)	R&DSystems
						17.5 ± 0.7	17.5 ± 0.7
Valino et al.¹⁶	UK	Prospective cohort	822	7207	Normal pregnancy	32.1 (32.0–32.5)	32.2 (32.0–32.5)	SGA: BW <10th	0%	*
Wallner et al.³²	Austria	Case–control	15	16	Normal pregnancy	33	38	SGA: BW <10th	0%	R&DSystems
Wathen et al.³³	Finland	Case–control	11	51	Normal pregnancy	13.7 ± 0.5		SGA: BW ≤ 2SD	0%	R&DSystems
						19.2 ± 0.6

SGA: small-for-gestational-age; AGA: appropriate-for-gestational-age; FGR: fetal growth restriction; BW: birthweight; PE: pre-eclampsia; PlGF: placental growth factor; sFlt-1: soluble Fms-like tyrosine kinase-1; GA: gestational age; R&D Systems, enzyme-linked immunosorbent assay (ELISA); Roche Diagnostics, electrochemiluminescence immunometric assay. Beckman Coulter, chemiluminescence immunometric assay; Thermo Fisher, immunometric assay TRACE Kryptor; Perkin Elmer, Time-Resolved Fluorescence immunometric assay; Bender MedSystems, enzyme-linked immunosorbent assay (ELISA); Alere, Point-of-care fluorescence immunoassay.

*Not mentioned.

The included studies used a variety of assay platforms from various manufacturers – 15 used R&D systems,^{3,4,21,23–25,28–33,35,37,38} 6 studies used Roche Diagnostics,^{1,14,15,26,27,36} 2 used Thermo Fisher,^2,13 1 study Beckman-Coulter,³⁴ 1 study Alere,²¹ 1 study Bender Medsystems²² and 1 study gave no information about the used assay.

Quality assessment

The quality assessment of the included studies is depicted in Supplementary Figure 1. The majority of the studies were defined as good quality (n = 17). The other studies were either of moderate quality (n = 6) or low quality (n = 3). In more studies the information about patient selection and/or biomarkers assays was not provided clearly.

PlGF measurements during SGA and reference pregnancy

Forest plots of PlGF measurements during SGA and normal pregnancy are depicted in Figure 2. The PlGF forest plots were divided into four GA categories; biomarker measurements < 14 weeks (n = 8 studies), 15–25 weeks (n = 3), 26–35 weeks (n = 8) and > 36 weeks of GA (n = 3). In first and second trimesters, we found a minimum change between SGA and reference pregnancies (–5.2 [95% CI: –6.6 to –3.7 pg/mL] to –34.5 pg/mL [95% CI –69.4 to 0.4 pg/mL]). However, after 26 weeks, forest plots illustrate that SGA group had lower PlGF values compared with reference group (–203.5 pg/mL [95% CI: –246.0 to –160.9 pg/mL] to –124.1 [95% CI: –129.5 to –118.7]).

Figure 2.

Forest plot of PlGF measurements during SGA and reference pregnancy. The PLGF forest plot compared SGA with control (reference) group, divided into four GA categories; biomarker measurements < 14 weeks, 15–25 weeks, 26–35 weeks and > 36 weeks of GA. Only first author of each study is given.

sFlt-1 measurements during SGA and reference pregnancy

There was no significant mean difference during first (n = 8 studies) or second trimester (n = 6) between SGA and reference group (Figure 3). After 26 weeks, SGA group had significantly higher sFlt-1 values compared with the reference group (n = 8) with pooled mean difference of 307 pg/mL (95% CI: 9 to 605 pg/mL). The largest mean difference was found after 36 weeks (n = 5) with a pooled mean difference of 1152 pg/mL (95% CI: 317 to 1988 pg/mL).

Figure 3.

Forest plot of sFlt-1 measurements during SGA and reference pregnancy. The sFlt-1 forest plot compared SGA with control (reference) group, divided into four GA categories; biomarker measurements < 14 weeks, 15–25 weeks, 26–35 weeks and > 36 weeks of GA. Only first author of each study is given.

sFlt-1/PlGF ratio during SGA and reference pregnancy

The sFlt-1/PlGF measurement during SGA pregnancies compared with reference values are depicted in Figure 4. No significant mean difference was found between SGA and reference for the sFlt-1/PlGF ratio during different GA intervals: < 14 weeks, 15–25 weeks and 26–36 weeks (n = 5, 2 and 5 studies, respectively). The sFlt-1/PlGF ratio values were higher in the SGA group versus reference only after 36 weeks based on one study with a mean difference of 30.1 (95% CI: 24.8–35.4).

Figure 4.

Forest plot of sFlt-1/PlGF measurements during SGA and reference pregnancy. The sFlt-1/PlGF forest plot compared SGA with control (reference) group, divided into four GA categories; biomarker measurements < 14 weeks, 15–25 weeks, 26–35 weeks and > 36 weeks of GA. Only first author of each study is given.

PlGF longitudinal courses show the most significant differences between SGA and control pregnancies, which is in contrast with the sFLT-1 and ratio courses where differences between the two groups is minimal.

Longitudinal Course of PlGF, sFlt-1 and ratio during SGA pregnancy and reference

In Figures 5 to 7, the longitudinal course of PlGF, sFlt-1 and the ratio are presented for both the SGA and the reference group. In reference pregnancies, PlGF concentrations were low in the first trimester, followed by a steady increase starting from 12 weeks till a peak around 30 weeks, followed by a sharp drop until term. SGA pregnancies had a similar but blunter curve with consistently lower PlGF concentrations, mainly between 24 and 36 weeks. Concentrations of sFlt-1 in reference pregnancies showed a dual peak, around 16 weeks and again after 30 weeks until term. SGA pregnancies had a grossly similar curve with demonstrably higher sFlt-1 concentrations after 30 weeks until term. The sFlt-1/PlGF ratio in reference pregnancies showed a steady decrease throughout pregnancy with a nadir around 30 weeks, and a slight shallow increase thereafter until term. Similar to the sFlt-1 curve, SGA pregnancies had grossly a similar curve for the sFlt-1/PlGF ratio with an earlier and steeper rise from 28 weeks onwards.

Figure 5.

Longitudinal course of PlGF. SGA pregnancies in orange. Reference pregnancies in blue. The dots present the individual studies. The number of cases and controls included in the study influence the size of the dots. Pregnancy duration in weeks.

Figure 6.

Longitudinal course of sFlt-1. SGA pregnancies in orange. Reference pregnancies in blue. The dots present the individual studies. The number of cases and controls included in the study influence the size of the dots. Pregnancy duration in weeks.

Figure 7.

Longitudinal course of sFlt-1/PlGF ratio. SGA pregnancies in orange. Reference pregnancies in blue. The dots present the individual studies. The number of cases and controls included in the study influence the size of the dots. Pregnancy duration in weeks.

Prognostic performance of the biomarkers

In Table 2, studies reporting sensitivity, specificity, positive and negative predictive values and area under the receiver operating characteristic curve (AUC) for predicting FGR or SGA using PlGF (8 studies), sFlt-1 (4 studies), and sFlt-1/PlGF ratio (8 studies) were included. Pooled analysis of prediction parameters was not possible due to the different GA at sampling and different definitions used for SGA (birthweight <5th or <10th percentile) and FGR.

Table 2.

Prognostic performance of individual biomarkers for SGA or FGR.

Reference	GA at sampling (weeks)	Cut-off (pg/mL)	SGA/FGR	AUC (95% CI)	Sensitivity(%)(95% CI)	Specificity(%)(95% CI)	PPV(%)(95% CI)	NPV(%)(95% CI)
PlGF
Birdir et al.²	12.6 ± 0.6	<34.7	BW <5th	0.65 (0.51–0.80)	90 (68–99)	40 (28–53)	32 (20–45)	93 (75–99)
Birdir et al.¹³	34.1	<60,4	FGR	0.72 (0.59–0.86)	45 (24–68)	95 (93–97)	24 (13–40)	98 (97–99)
Boucoiran et al.³⁴	12–18	*	BW <10th	0.67 (0.57–0.77)	30 (19–44)	90 (87–92)	23 (14–34)	93 (91–95)
Crovetto et al.²⁴	10.0 ± 0.9	*	FGR¹ or BW <3rd	0.67 (0.63–0.70)	61 (57–66)	90 (88–92)	80 (76–84)	78 (75–81)
Furuta et al.³⁷	25.0	<500	BW <10th	0.76	100 (75–100)	54 (46–62)	14 (8–24)	100 (95–100)
Gaccioli et al.¹⁴	28	<252.7	BW <10th<37 weeks GA at delivery	0.89 (0.81–0.98)	81 (66–90)	91 (90–92)	5	100
	36	<104.4	BW <10th<37 weeks GA at delivery	0.75 (0.69–0.82)	43	91	7	99
Karagiannis et al.³⁸	11–13	*	BW <5th	*	37	90	15	97
Triunfo et al.⁴	24–26	*	BW <10th	0.73 (0.61–0.85)	23 (13–36)	90 (87–92)	18 (10–30)	92 (90–94)
	17.5 ± 0.7	*	BW <10th	0.77 (0.67–0.86)	52 (37–67)	90 (82–95)	73 (54–86)	79 (70–86)
sFlt-1
Birdir et al.¹³	34.1	>3369	FGR	0.67 (0.52–0.83)	59 (36–79)	82 (79–85)	9 (5–16)	98 (97–99)
Crovetto et al.²⁴	10.0 ± 0.9	*	FGR¹ or BW <3rd	0.60 (0.56–0.64)	50 (45–55)	90 (88–92)	77 (71–81)	73 (70–76)
Gaccioli et al.¹⁴	28	>2788	BW <10th<37 wks GA at delivery	0.74 (0.62–0.85)	46	90	3	100
	36	>6266	BW <10th<37 weeks GA at delivery	0.73 (0.66–0.81)	41	91	6	99
Triunfo et al.⁴	9.9 ± 1.0	*	BW <10th	0.61 (0.51–0.71)	20 (9–34)	90 (82–95)	50 (27–73)	69 (60–77)
	17.5 ± 0.7	*	BW <10th	0.69 (0.59–0.79)	35 (21–50)	90 (82–95)	64 (43–81)	73 (64–81)
sFlt-1/PlGF
Birdir et al.²	12.6 ± 0.6	>78,7	BW <5th	0.55 (0.38–0.70)	95 (75–100)	31 (20–43)	30 (19–43)	95 (74–99)
Birdir et al.¹³	34.1	>63,6	FGR	0.70 (0.55–0.86)	50 (28–72)	94 (92–96)	21 (12–35)	98 (97–99)
Boucoiran et al.³⁴	12–18	*	BW <10th	0.67 (0.57–0.78)	18 (9–30)	90 (87–92)	15 (8–26)	92 (89–94)
	24–26	*	BW <10th	0.70 (0.58–0.83)	21 (12–34)	90 (87–92)	17 (10–28)	92 (90–94)
Crovetto et al.²⁴	10.0 ± 0.9	*	FGR or BW <3rd	0.68 (0.65–0.72)	61 (57–66)	90 (88–92)	80 (76–84)	78 (75–81)
Furuta et al.³⁷	25.0	>4,1	BW <10th	0.61	46 (19–75)	93 (88–96)	33 (14–59)	96 (91–98)
Gaccioli et al.¹⁴	28	>5.78	BW <10th<37 weeks GA delivery	0.89 (0.82–0.97)	73 (52–88)	85 (84–87)	3 (2–5)	97 (95–98)
	36	>38		0.77 (0.70–0.84)	53 (40–67)	86 (84–87)	6 (4–8)	99 (98–99)
Herraiz et al.²⁷	<34	>33	FGR+ BW<10th	*	84 (60–96)	97 (94–99)	*	*
		>85	FGR+ BW<10th	*	58 (34–80)	*	*	*
	>34	>33	FGR+ BW<10th	*	75 (35–96)	79 (74–83)	23 (15–34)	97 (94–99)
		>110	FGR+ BW<10th	*	63 (25–91)	96 (91–97)	50 (33–67)	97 (94–98)
Triunfo et al.³¹	34.9 ± 1.1	*	BW<10th	0.63 (0.54–0.72)	30 (20–41)	90 (81–96)	75 (56–88)	56 (47–65)

GA of sampling in mean±SD, or min–max; cut-off value in pg/mL; GA: gestational age at time of sampling; SGA: small-for-gestational-age; FGR: fetal growth restriction; AUC: area-under the curve; PPV: positive predictive value; NPV: negative predictive value; Data presented as mean, with 95% confidence interval (if available).

*Data not available.

For PlGF only four of the eight studies reported cut-off values using different assays and for the two studies which used the Thermo Fisher assay also different cut-off values were used. The AUC values varied from 0.65 to 0.89, reflecting the wide range in cut-off values used: 34.7 (at 12 weeks)¹³ to 252 (at 28 weeks).¹⁴ For sFlt-1, two of the four studies reported cut-off values, and these two studies used different assay types. The AUC values vary from 0.61 to 0.74. For the sFlt-1/PlGF ratio, five out of eight studies reported cut-off values and sometimes more than one cut-off value was used.^14,27 The AUC values vary from 0.55 to 0.89.

Five studies used a combination of maternal, fetal and biochemical factors in the prediction of SGA. In the first trimester, a model using serum PlGF concentrations, in combination with maternal factors, mean arterial pressure (MAP), pregnancy-associated plasma protein-A and beta-human chorionic gonadotropin had a prediction rate of 73% for early SGA (requiring delivery before 37 weeks) and 46% for late SGA (delivering at term) with a FPR of 10%.³⁸ A model using a combination of sFlt-1/PlGF ratio and a priori risk for SGA, MAP and uterine artery pulsatility index (UtA-PI) had a prediction rate of 42% for SGA and 66% for FGR, with a FPR of 10%.²⁴

In the third trimester, serum PlGF concentrations were used in two models in combination with maternal factors and EFW to predict SGA with a higher sensitivity for early SGA (delivering at 32–36 weeks) compared with late SGA (delivering > 37 weeks) and for severe SGA (birthweight < 3rd percentile) compared to <10th percentile.^1,36 Using a combination of sFlt-1/PlGF ratio and EFW, the prediction rate for SGA was 45%, with a 10% FPR and AUC of 0.77 (95% CI: 0.70–0.85).³¹

Discussion

In this systematic review, we included 26 studies that investigated the values of measuring the angiogenic biomarkers PlGF, sFlt-1 and their ratio in pregnancies complicated by FGR or SGA. This is the first review to our knowledge that focused on studies describing biomarkers changes in SGA pregnancies. Using GA-specific intervals, the data analysis demonstrated that the mean differences between SGA and reference pregnancies were larger in serum concentration of PlGF than with sFlt-1 or their ratio, and were more profound during later stages of pregnancy (after 25 weeks) and not in the first trimester.

In the longitudinal course of biomarkers, there was a good discrimination between SGA and reference pregnancies especially during 24–36 weeks for PlGF, while for sFlt-1 and the sFlt-1/PlGF ratio, there was no discrimination between SGA and reference pregnancies as depicted by the overlapping confidence intervals. The amount of literature for sFlt-1 and the sFlt-1/PlGF ratio is limited and also heterogeneous which can result in a diminished discrimination between SGA and reference pregnancies for sFlt-1 and the sFlt-1/PlGF ratio. Furthermore, screening with biomarkers for SGA showed a moderate predictive performance with a wide range of AUC that varied from 0.65–0.89 for PlGF, 0.55–0.89 for sFlt-1 and the PlGF/sFlt-1 ratio, reflecting the heterogeneity in outcome definition, assay methods and timing of screening.

Angiogenic biomarkers, as non-invasive and readily available indicators of placental function, have made a large impact on the way researchers nowadays understand and approach placental dysfunction disorders.³⁹ Since the earlier publications describing lower maternal serum PlGF and higher sFlt-1 concentrations in women with pre-eclampsia,⁴⁰ these biomarkers have increasingly claimed a prominent place in the clinical management of pre-eclampsia. Using a cut-off value of 38, the sFlt-1/PlGF ratio as proposed is a reliable predictor for the imminent development of pre-eclampsia.^10,41

Since pre-eclampsia and FGR (fetal growth restriction) probably share a common underlying placental pathology,⁴² we aimed in this review that identifying cut-off values for these biomarkers this could be of use to differentiate between pregnancies with or without FGR. Only 3 out of 26 studies had patients included with pre-eclampsia. Because of this relative low percentage, we could not perform a reliable analysis without PE. An optimal discriminatory cut-off value could not be calculated from published studies, neither for the outcome of FGR nor for SGA due to the limited number of studies reporting cut-off values used (4 out of 8 studies) and their heterogeneous setting. In addition, pooled analysis of prediction parameters was also not possible due to the different assay methods, different GA at sampling and different definitions used for SGA (birthweight <5th or <10th percentile) and FGR. Benton et al.¹² showed that low PlGF concentrations (<5th percentile for GA) had a good capacity (AUC of 0.96) to discriminate fetuses with placental disease from those who are constitutionally small. Using a GA-adjusted percentiles for PlGF could, therefore, improve clinical management of pregnancies with SGA by identifying the subgroup with an underlying placental pathology and ultimately form the highest risk group.

We and others have demonstrated the added value of combining the measurement of placental biomarkers in the first trimester with clinical characteristics, Doppler parameters or fetal growth in the prediction of FGR.^38,43,44 The use of multiple parameters could increase the specificity and sensitivity of the screening, mainly because each biomarker reflect a different mechanistic pathway linked to the disease process. While abnormal uterine artery Doppler reflects the inadequate trophoblastic invasion of the maternal spiral arteries, abnormal serum concentrations of PlGF and sFlt-1 reflect the dysregulated secretory activity by placental trophoblasts,⁶ both well recognized mechanisms linked to the development of FGR. Earlier screening, before evident clinical recognition is possible, may have the potential to improve the clinical management and reduced morbidity for both mother and fetus. Consequently, appropriate planning of follow-up visits and the timely commencement of therapeutic interventions, such as Aspirin is possible, with recent large trials reporting great success in reversing the natural outcome of placenta dysfunction.⁴⁵

This review highlights the importance of meticulous consideration of methodological aspects of studies, that might affect the interpretation of results. The considerable heterogeneity between the studies included, as reflected by high I² values in the forest plot analysis, could have various sources. Worldwide, different fetal and neonatal growth charts are used for defining FGR and SGA with varying cut-off values. It is difficult to define FGR, the fetus does not reach its biological growth potential as a consequence of impaired placental function. It is usually defined by the statistical deviation of fetal size from a population-based reference, with a threshold at the 10th, 5th of 3rd centile. However this threshold is better usable for SGA, based on actual weight at birth. Due to the lack of this gold standard to define FGR based on the underlying pathophysiology, SGA is used as an inappropriate proxy, that represents a heterogeneous group of both constitutionally small and growth restricted fetuses.⁴⁶ Growth-restricted fetuses with a birth weight > 10th centile, may not reach their growth potential, but remain undiagnosed because of the birthweight, with an increased risk of adverse outcome. Recently, consensus definitions have been reached by Delphi procedure for both early-onset and late-onset FGR.⁴⁷ These new definitions of FGR are useful for a prospective study in the future, many of the biomarker studies in this review have been published before the new definition.

The heterogeneity could also be caused by varying study design and using different assays. The studies included in this review used different immunoassays for PlGF and sFlt-1. It is well known that standardization of immunoassays is difficult. A major problem in immunoassays is that the recognition of the analyte is determined by the reagent using a limited portion of the molecule.⁴⁸ Large differences in assay calibration can lead to misinterpretations of the clinical course when different assays are used.⁴⁹ For PlGF and sFlt-1 there is no reference method defined and there is no reference material, like a WHO standard available. The different antibodies which are used can have another specificity for the PlGF isoforms and to free and bound PlGF. In addition, the larger studies which included more than 100 SGA cases or controls were also based on R&D Systems and Roche assays. Although, there is no reference method or material available for PlGF and sFlt-1, it is described that the PlGF and sFlt-1 immunoassays of Roche Diagnostics are calibrated with recombinant human PIGF and sFlt–1 and standardized against the Quantikine PIGF and sFlt-1 ELISAs from R&D Systems.⁵⁰ The study of Schiettecatte et al., 2010 showed that there is good correlation between Roche and the Quantikine ELISA R&D Systems.⁵¹ The Kryptor/Thermo Fisher PlGF method is also calibrated with recombinant human PlGF and standardized against the Quantikine PlGF ELISA from R&D system. The Kryptor sFlt-1 method is standardized against the Roche method.⁵² The study of Helden et al., 2015 confirmed a good correlation between Kryptor and Roche assay.⁵³ So we can conclude that the results of the R&D system, Roche and Thermo Fisher assays are very similar because they are standardized against each other.⁵⁴

Conclusions

In conclusion, this review outlines the differences in angiogenic biomarker concentrations in SGA/FGR pregnancies, similar to earlier reviews on the related placental disorder of pre-eclampsia. Despite variation in study design and test methods, angiogenic biomarkers showed different mean levels and longitudinal trends in SGA pregnancies. Further research should focus on improving the prediction of at-risk SGA pregnancies by combining placental biomarkers with clinical characteristics and ultrasound measures of fetal growth velocity.

Supplemental Material

ACB882042 Supplemetal Material - Supplemental material for Role of sFlt-1 and PlGF in the screening of small-for-gestational age neonates during pregnancy: A systematic review

Supplemental material, ACB882042 Supplemetal Material for Role of sFlt-1 and PlGF in the screening of small-for-gestational age neonates during pregnancy: A systematic review by MLE Hendrix, JAP Bons, A van Haren, SMJ van Kuijk, WPTM van Doorn, DM Kimenai, O Bekers, MEA Spaanderman and S Al-Nasiry in Annals of Clinical Biochemistry

Footnotes

Acknowledgements

We acknowledge the advice and help from Bart de Vries, information specialist.

Declaration of conflicting interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

Ethical approval

Not applicable.

Guarantor

SAl-N.

Contributorship

All authors made a significant contribution to this study. All the authors have accepted responsibility of the entire content of this submitted manuscript and approved submission.

ORCID iD

MLE Hendrix

Supplemental material

Supplemental material for this article is available online.

References

Bakalis

, et al. Prediction of small-for-gestational-age neonates: screening by biophysical and biochemical markers at 30–34 weeks. Ultrasound Obstet Gynecol 2015; 46: 446–451.

Birdir

, et al. Impact of maternal serum levels of visfatin, AFP, PAPP-A, sFlt-1 and PlGF at 11–13 weeks gestation on small for gestational age births. J Matern Fetal Neonatal Med 2017; 30: 629–634.

Darling

, et al. Angiogenic and inflammatory biomarkers in midpregnancy and small-for-gestational-age outcomes in Tanzania. Am J Obstet Gynecol 2014; 211: 509.

Triunfo

, et al. Changes in uterine artery Doppler velocimetry and circulating angiogenic factors in the first half of pregnancies delivering a small-for-gestational-age neonate. Ultrasound Obstet Gynecol 2017; 49: 357–363.

Lindqvist

Molin

Does antenatal identification of small-for-gestational age fetuses significantly improve their outcome?

Ultrasound Obstet Gynecol 2005; 25: 258–264.

Vrachnis

, et al. Placental growth factor (PlGF): a key to optimizing fetal growth. J Matern Fetal Neonatal Med 2013; 26: 995–1002.

Shibuya

Vascular endothelial growth factor (VEGF) and its receptor (VEGFR) signaling in angiogenesis: a crucial target for anti- and pro-angiogenic therapies.

Genes Cancer 2011; 2: 1097–1105.

Chau

Hennessy

Makris

Placental growth factor and pre-eclampsia.

J Hum Hypertens 2017; 31: 782–786.

MacDonald

, et al. Assessing the sensitivity of placental growth factor and soluble fms-like tyrosine kinase 1 at 36 weeks' gestation to predict small-for-gestational-age infants or late-onset preeclampsia: a prospective nested case-control study. BMC Pregnancy Childbirth 2018; 18: 354.

10.

Caillon

, et al. Evaluation of sFlt-1/PlGF ratio for predicting and improving clinical management of pre-eclampsia: experience in a specialized perinatal care center. Ann Lab Med 2018; 38: 95–101.

11.

Asvold

, et al. Angiogenic factors in maternal circulation and the risk of severe fetal growth restriction. Am J Epidemiol 2011; 173: 630–639.

12.

Benton

, et al. Placental growth factor as a marker of fetal growth restriction caused by placental dysfunction. Placenta 2016; 42: 1–8.

13.

Birdir

, et al. Predictive value of sFlt-1, PlGF, sFlt-1/PlGF ratio and PAPP-A for late-onset preeclampsia and IUGR between 32 and 37weeks of pregnancy. Pregnancy Hypertens 2018; 12: 124–128.

14.

Gaccioli

, et al. Screening for fetal growth restriction using ultrasound and the sFLT1/PlGF ratio in nulliparous women: a prospective cohort study. Lancet Child Adolesc Health 2018; 2: 569–581.

15.

Rizos

, et al. Placental growth factor and soluble fms-like tyrosine kinase-1 are useful markers for the prediction of preeclampsia but not for small for gestational age neonates: a longitudinal study. Eur J Obstet Gynecol Reprod Biol 2013; 171: 225–230.

16.

Valino

, et al. Biophysical and biochemical markers at 30–34 weeks' gestation in the prediction of adverse perinatal outcome. Ultrasound Obstet Gynecol 2016; 47: 194–202.

17.

Moher

, et al. Preferred reporting items for systematic review and meta-analysis protocols (PRISMA-P) 2015 statement. Syst Rev 2015; 4: 1.

18.

Ouzzani

, et al. Rayyan – a web and mobile app for systematic reviews. Syst Rev 2016; 5: 210.

19.

Whiting

, et al. QUADAS-2: a revised tool for the quality assessment of diagnostic accuracy studies. Ann Intern Med 2011; 155: 529–536.

20.

Hozo

Djulbegovic

Hozo

Estimating the mean and variance from the median, range, and the size of a sample.

BMC Med Res Methodol 2005; 5: 13.

21.

Berkane

, et al. Abnormal steroidogenesis and aromatase activity in preeclampsia. Placenta 2018; 69: 40–49.

22.

Borras

, et al. Angiogenic growth factors in maternal and fetal serum in pregnancies complicated with intrauterine growth restriction. J Obstet Gynaecol 2014; 34: 218–220.

23.

Chaiworapongsa

, et al. The maternal plasma soluble vascular endothelial growth factor receptor-1 concentration is elevated in SGA and the magnitude of the increase relates to Doppler abnormalities in the maternal and fetal circulation. J Matern Fetal Neonatal Med 2008; 21: 25–40.

24.

Crovetto

, et al. Differential performance of first-trimester screening in predicting small-for-gestational-age neonate or fetal growth restriction. Ultrasound Obstet Gynecol 2017; 49: 349–356.

25.

Erez

, et al. The change in concentrations of angiogenic and anti-angiogenic factors in maternal plasma between the first and second trimesters in risk assessment for the subsequent development of preeclampsia and small-for-gestational age. J Matern Fetal Neonatal Med 2008; 21: 279–287.

26.

Fruscalzo

, et al. First trimester TTR-RBP4-ROH complex and angiogenic factors in the prediction of small for gestational age infant's outcome. Arch Gynecol Obstet 2017; 295: 1157–1165.

27.

Herraiz

, et al. Characterization of the soluble fms-like tyrosine kinase-1 to placental growth factor ratio in pregnancies complicated by fetal growth restriction. Obstet Gynecol 2014; 124(2 PART1): 265–273.

28.

Savvidou

, et al. Maternal serum concentration of soluble fms-like tyrosine kinase 1 and vascular endothelial growth factor in women with abnormal uterine artery Doppler and in those with fetal growth restriction. Am J Obstet Gynecol 2006; 195: 1668–1673.

29.

Shibata

, et al. Soluble fms-like tyrosine kinase 1 is increased in preeclampsia but not in normotensive pregnancies with small-for-gestational-age neonates: Relationship to circulating placental growth factor. J Clin Endocrinol Metab 2005; 90: 4895–4903.

30.

Thadhani

, et al. First trimester placental growth factor and soluble Fms-like tyrosine kinase 1 and risk for preeclampsia. J Clin Endocrinol Metab 2004; 89: 770–775.

31.

Triunfo

, et al. Angiogenic factors and Doppler evaluation in normally growing fetuses at routine third-trimester scan: prediction of subsequent low birth weight. Fetal Diagn Ther 2016; 40: 13–20.

32.

Wallner

, et al. Angiogenic growth factors in maternal and fetal serum in pregnancies complicated by intrauterine growth restriction. Clin Sci 2007; 112: 51–57.

33.

Wathen

, et al. Maternal serum-soluble vascular endothelial growth factor receptor-1 in early pregnancy ending in preeclampsia or intrauterine growth retardation. J Clin Endocrinol Metab 2006; 91: 180–184.

34.

Boucoiran

, et al. Risks for preeclampsia and small for gestational age: predictive values of placental growth factor, soluble fms-like tyrosine kinase-1, and inhibin A in singleton and multiple-gestation pregnancies. Am J Perinatol 2013; 30: 607–612.

35.

Boutsikou

, et al. Soluble vascular endothelial growth factor receptor-1 in intrauterine growth restricted fetuses and neonates. Early Hum Dev 2006; 82: 235–239.

36.

Fadigas

, et al. Prediction of small-for-gestational-age neonates: screening by placental growth factor and soluble fms-like tyrosine kinase-1 at 35–37 weeks. Ultrasound Obstet Gynecol 2015; 46: 191–197.

37.

Furuta

, et al. Serum placental growth factor and soluble fms-like tyrosine kinase 1 at mid-gestation in healthy women: association with small-for-gestational-age neonates. J Obstet Gynaecol Res 2017; 43: 1152–1158.

38.

Karagiannis

, et al. Prediction of small-for-gestation neonates from biophysical and biochemical markers at 11–13 weeks. Fetal Diagn Ther 2011; 29: 148–154.

39.

Stepan

, et al. Implementation of the sFlt-1/PlGF ratio for prediction and diagnosis of pre-eclampsia in singleton pregnancy: implications for clinical practice. Ultrasound Obstet Gynecol 2015; 45: 241–246.

40.

Maynard

, et al. Excess placental soluble fms-like tyrosine kinase 1 (sFlt1) may contribute to endothelial dysfunction, hypertension, and proteinuria in preeclampsia. J Clin Invest 2003; 111: 649–658.

41.

Zeisler

, et al. Predictive value of the sFlt-1:PlGF ratio in women with suspected preeclampsia. N Engl J Med 2016; 374: 13–22.

42.

Brosens

, et al. The "Great Obstetrical Syndromes" are associated with disorders of deep placentation. Am J Obstet Gynecol 2011; 204: 193–201.

43.

Crovetto

, et al. First-trimester screening with specific algorithms for early- and late-onset fetal growth restriction. Ultrasound Obstet Gynecol 2016; 48: 340–348.

44.

Hendrix

MLE

, et al. Reduced fetal growth velocities and the association with neonatal outcomes in appropriate-for-gestational-age neonates: a retrospective cohort study. BMC Pregnancy Childbirth 2019; 19: 31.

45.

Rolnik

, et al. Aspirin versus placebo in pregnancies at high risk for preterm preeclampsia. N Engl J Med 2017; 377: 613–622.

46.

Mlynarczyk

, et al. The clinical significance of an estimated fetal weight below the 10th percentile: a comparison of outcomes of <5th vs 5th–9th percentile. Am J Obstet Gynecol 2017; 217: 198 e1–198 e11.

47.

Gordijn

, et al. Consensus definition of fetal growth restriction: a Delphi procedure. Ultrasound Obstet Gynecol 2016; 48: 333–339.

48.

Masseyeff

RF.

Standardization of immunoassays.

Ann Ist Super Sanita 1991; 27: 427–436.

49.

Stenman

UH.

Immunoassay standardization: is it possible, who is responsible, who is capable?

Clin Chem 2001; 47: 815–820.

50.

Andersen

, et al. Diagnosis of preeclampsia with soluble fms-like tyrosine kinase 1/placental growth factor ratio: an inter-assay comparison. J Am Soc Hypertens 2015; 9: 86–96.

51.

Schiettecatte

, et al. Multicenter evaluation of the first automated Elecsys sFlt-1 and PlGF assays in normal pregnancies and preeclampsia. Clin Biochem 2010; 43: 768–770.

52.

Nucci

, et al. Maternal serum placental growth factor (PlGF) isoforms 1 and 2 at 11–13 weeks' gestation in normal and pathological pregnancies. Fetal Diagn Ther 2014; 36: 106–116.

53.

van Helden

Weiskirchen

Analytical evaluation of the novel soluble fms-like tyrosine kinase 1 and placental growth factor assays for the diagnosis of preeclampsia.

Clin Biochem 2015; 48: 1113–1119.

54.

McCarthy

, et al. Comparison of three commercially available placental growth factor-based tests in women with suspected preterm pre-eclampsia: the COMPARE study. Ultrasound Obstet Gynecol 2019; 53: 62–67.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.14 MB