Sage Journals: Discover world-class research

Abstract

Protealg^® is the dried and ground whole biomass of Galdieria sulphuraria, a microalga naturally rich in protein and phycocyanin, and contains carotenoids. It is expected that Protealg will be consumed as a food ingredient or supplement by the general public. The safety of Protealg was evaluated in an Ames bacterial reverse mutation test, in vitro mammalian cell micronucleus test, and 13-week oral toxicity study in rats at the dose levels of 500, 2000, and 5000 mg/kg/day. The 13-week toxicity study included a 4-week treatment-free period to evaluate recovery. A functional observation battery and ophthalmology examinations were performed, and hematology, blood biochemistry, thyroid hormone, and urinalysis parameters were determined. Seminology parameters and estrus cycle staging were investigated. A macroscopic necropsy examination was performed and tissues were examined microscopically. Protealg showed no evidence of mutagenicity or clastogenic activity. It was well tolerated by rats and no clinical findings indicative of toxicity were observed. There were no significant findings in the in-life or post-mortem investigations. In conclusion, no toxicity was observed after administration of Protealg at dose levels up to 5000 mg/kg/day to rats for 13 weeks, supporting the safety of Protealg for use as a high protein food supplement. The NOAEL was established as 5000 mg/kg/day. Allowing for a margin of exposure of 100-fold, the corresponding anticipated daily intake will be 50 mg/kg/day equivalent to 3.5 g/day for a 70 kg subject. Incorporation levels in various foods will be defined on the basis of appropriate exposure estimations using official recommendations.

Keywords

Microalgal biomass high-protein food supplement phycocyanin Ames test micronucleus test

Introduction

At the present time, 24 algal species are approved for human consumption in Europe. These include 3 microalgae, 8 brown algae, 11 red algae, and 2 green algae.¹ The microalgal species most widely used for human nutrition is Arthrospira platensis (commonly known as spirulina), a member of the prokaryotic Cyanobacteria phylum that obtain their energy through photosynthesis. Cyanobacteria take their name from their blue-green color, resulting from the abundant phycocyanin that they contain.

Spirulina is a traditional food source in parts of Africa and Mexico. It is consumed for its content of protein, vitamins, and minerals. Of the world production of spirulina, 50% is accounted for by China.² Compared to staple protein sources such as soy, the production of microalgae is very small indeed. If microalgae are to become a more widely used source of protein, increased production capacity will be needed.³

Spirulina is generally produced in open pond or raceway systems. This mode of production presents issues relating to control of contamination by bacteria and heavy metal elements. Contamination by other cyanobacteria is of particular concern as the contaminant organisms can produce microcystins that are potentially irritant to the gastrointestinal tract and/or hepatotoxic.^4,5 Some bioanalytical methods have been developed to detect these toxins in dietary products,⁶ but this precautionary approach does not prevent renewed contamination.⁷ A recent study using liquid chromatography with tandem mass spectrometry detection demonstrated the presence of microcystins in 34 products containing spirulina that were produced in China, which did not exceed safe levels, but could cause issues in the case of long-term chronic exposure.⁸

For these reasons, it would be of interest to identify alternative types of microalgae with good nutritive properties and produced using culture methods that avoid the known drawbacks of present spirulina culture methods. Protealg^® is produced by a standardized process in bioreactors and is therefore not subjected to seasonal changes, so the biomass composition is similar in each batch. Such novel culture methods must combine high productivity with scalability and a good safety profile.

We have been interested in a group of red algae, called Rhodophytes, which are mostly marine algae. These algae are characterized by the presence of a red pigment, phycoerythrin, in their photosynthesis system. Among the Rhodophytes, there is one subdivision, Cyanidiophytina, which is composed of unicellular red algae living in acidic hot springs.⁹

This subdivision contains the Cyanidiophycea class of algae that have a green-blue color similar to cyanobacteria.¹⁰ These algae have a high protein content and the ability to produce phycocyanin, which can be extracted and used as a natural blue food colorant.^11
–13 Cyanidiophyceae are extremophilic organisms that are able to withstand very acid pH levels (0.05–5) and very high temperatures (25–56°C). They are generally the only organisms present in their natural thermo-acidophilic habitats.^10,14

Cyanidiophyceae can be cultivated both as heterotrophs (where consumption of organic carbon substrate is provided by the culture medium) and as autotrophs (where light is used to capture CO₂ via photosynthesis).^15,16 As heterotrophs, these algae are suitable for production in bioreactors,^17
–19 where conditions, as for many other fermented food products, can be arranged for culture in the absence of any other microbial species

One species in this class, Galdieria sulphuraria (formerly known as Cyanidium caldarium), provides an interesting candidate as Galdieria biomass productivity is 50–100 times higher than that of spirulina.¹⁷ G. sulphuraria is a spherical organism with multilobed chloroplasts and an average cell size of 3–11 µm. The reproduction of Galdieria is vegetative by budding sporangia.

Heterotrophic growth of G. sulphuraria does not permit high production of pigments such as chlorophyll and phycocyanin.^15,20 Mixotrophic growth, which is also called photo-heterotrophy,¹⁸ provides a combination of good growth rate and phycocyanin production; the cells use organic carbon as an energy source,¹⁸ while the presence of light induces metabolic pathways associated with photosynthesis.²¹

A natural variant of G. sulphuraria selected in the dark can acquire the ability to produce chlorophyll and phycocyanin in heterotrophy.^22,15

Under the correct culture conditions, Galdieria produces a biomass that is rich in protein (>45% total protein by dry weight, as determined according to the Kjeldahl method with a total nitrogen conversion factor of 6.25), essential amino acids, and minerals.^12,13 Galdieria protein digestibility has been evaluated at 96% using an in vitro method adapted from Boisen and Fernandez.²³ The Protealg batches used in this work have a protein content of >70% and a phycocyanin content of 8%. The batches used in this study are representative of those produced in standardized production process. Less than 5% variation in protein content is typically observed between batches and the amino acid profile is stable.

Galdieria is grown on an organic substrate, then fermented and processed using a patented procedure under highly controlled conditions. The dried and ground microalgal biomass has a neutral flavor and is rich in protein and other nutritive elements, such as vitamins and minerals. It is therefore well suited for use as a food supplement. The phycocyanin content can be extracted from G. sulphuraria for use as a natural blue food colorant.

It is expected that Protealg will be consumed as a food ingredient or supplement by the general public. The maximum levels for human consumption will be determined using a safety margin of at least 100-fold (the current standard safety margin in the United States). This represents 50 mg/kg/day or 3.5 g/day for a 70 kg adult. Incorporation levels in various foods will be defined on the basis of appropriate exposure estimations, using official recommendations and tools such as the food additive intake model.

We present here a evaluation of the safety of Protealg the protein-rich dried and ground microalgal biomass prepared from mixotrophic culture of G. sulphuraria. Studies were performed to investigate the safety of Protealg for use as a food supplement. The aim of the present article is to describe the results of a 13-week oral rat toxicity study and short-term in vitro genotoxicity assays conducted with Protealg.

Methods

Test material

The item under consideration was Protealg, a dried and ground microalgal biomass, prepared from a unicellular eukaryotic microalga, G. sulphuraria (NCBI: txid130081; phylum: Rhodophyta, subphylum: Cyanidiophytina, class: Cyanidiophyceae, order: Cyanidiales, family: Galdieriaceae, genus: Galdieria).

The batch numbers of Protealg used in these studies were CIT170042 for the in vitro studies and CIT170405 for the 13-week toxicity study. Protealg was supplied as a biomass of dried and ground microalgae and presented a fine dark blue-green powder appearance. Purity was considered to be 100%.

The protein content of batch CIT170405 was 71.3 ± 2.2g/100 g (determined using the Kjeldahl method). Amino acid composition analysis of the biomass was carried out by Eurofins company using high performance liquid chromatography (HPLC), according to the European norms NF EN ISI 13903 and NF EN ISO 13904 for tryptophan (see Online Supplemental Table 6).

Phycocyanin content was 8.02 ± 0.23 g/100 g. Because of the cell wall structure of G. sulphuraria, the method used to determine phycocyanin content in Spirulina platensis ²⁴ could not be used. Culture samples of Galdieria were centrifuged at 10,000g for 10 min, washed in water and freeze dried, before being mechanically disrupted for 3 min at 30 Hz (using a Tissue Lyser II, Qiagen^®, Qiagen SAS, Courtaboeuf, France). The powder was resuspended in Tris-HCl buffer (pH 7.5) and incubated for 1 h at room temperature, before centrifugation for 10 min at 20,000g for separation of the phycocyanin content in the supernatant from the cell debris. Phycocyanin concentration was calculated according to the method in the work of Moon et al.²⁵

Before use, Protealg was suspended in water for injections (CDM Lavoisier, France; in vitro tests) or drinking water treated by reverse osmosis using an Elix 5 (Millipore SA, Molsheim, France; in vivo study). The suspension used in the in vivo study was green to dark green/almost black at the high dose and very viscous at higher concentrations.

An analysis certificate for the test item was provided by Eurofins, France, which is accredited for ISO/IEC 17025:2005 COFRAC 1-0287.

Bacterial reverse mutation (Ames) test

Protealg was tested in three independent experiments, with or without a metabolic activation system: S9 mix, prepared from a liver post mitochondrial fraction of rats induced with Aroclor 1254 (the S9 fraction was obtained from Moltox: Molecular Toxicology Inc., Boon, NC, USA). The direct plate incorporation method (first and third tests with or without S9 mix) or the preincubation method (second test with S9 mix; 60 min, 37°C) was used. The study protocol was based on OECD guideline 471.

Five strains of Salmonella typhimurium were used (obtained from culture collections: Public Health England, UK or Moltox): TA1535, TA1537, TA98, TA100, and TA102. Each bacterial tester strain was treated with at least five dose levels of the test item (three plates/dose level). The vehicle was water for injections.

Six known mutagens were used as positive controls to check the test system sensitivity: sodium azide, 9-aminoacridine, 2-nitrofluorene, 2-anthramine, and benzo(a)pyrene (dissolved in dimethylsulfoxide), and mitomycin C (dissolved in water).

After 48–72 h of incubation at 37°C, revertant colonies (numbers of revertants per plate) were scored using an automatic counter (Sorcerer Automatic Colony Counter software for acquisition of data and Ames Study Manager software for data management; Perceptive Instruments Ltd, UK).

Toxicity was evaluated by observation of a decrease in the number of revertant colonies and/or thinning of the bacterial lawn. The criteria for a positive result were a reproducible 2-fold increase (for TA98, TA100, and TA102 strains) or 3-fold increase (for TA1535 and TA1537 strains) in the mean number of revertant colonies when compared with the vehicle control and/or by evidence of a dose relationship.

In vitro mammalian cell micronucleus test

The in vitro mammalian cell micronucleus test was performed to investigate the genotoxic potential (clastogenic and/or aneugenic) of Protealg as measured by induction of an increase in the frequency of micronucleated cells in the mouse lymphoma cell line L5178Y TK^+/− (obtained from American Type Culture Collection, USA). Protealg (suspended in water for injections) was tested with and without a metabolic activation system, S9 mix (the S9 fraction was obtained from Moltox, Molecular Toxicology Inc.). In the presence of S9 mix, treatments for 3 h were followed by 24 h recovery. In the absence of S9 mix, treatments for 3 h were followed by 24 h recovery and treatments for 24 h were conducted without recovery. The study protocol was based on OECD guideline 487.

Depending on the presence or absence of precipitate and/or cytotoxicity at the end of the recovery period, Protealg-treated cultures from at least three dose levels, along with vehicle and positive control cultures, were spread on glass slides, air-dried and stained with 5% Giemsa for 15 min. Positive control treatments were with mitomycin C, colchicine, or cyclophosphamide. Slides were coded for blind scoring at analysis.

Cytotoxicity and micronucleus frequency were estimated for each treated cell culture.

Cytotoxicity: It was evaluated for each control or Protealg treatment by determining the decrease in cell population doubling (PD) when compared to the vehicle control, where PD was calculated as follows:

PD = [log (N/N0)] /log 2

Mean PD as % of control= \frac{Mean PD treated}{Mean PD vehicle control} ×100

Decrease in PD (%) = 100-Mean PD as % of control

Cytotoxicity is determined, as required by OECD Guideline 487, to avoid false positive results resulting from strong cytotoxicity. The selection of dose levels for microscope scoring of micronuclei is based on the Guideline criterion of PD >55%.

Micronuclei: The number of micronuclei was recorded by microscopic inspection of 1000 mononuclear L5178Y cells per culture. The number of cells with micronuclei and the number of micronuclei per cell were recorded separately for each culture to evaluate the genotoxic potential of the test item.

The test item was considered to show evidence of clastogenicity and/or aneugenicity if all three of the following criteria were met:

There was a statistically significant, dose-related increase in the frequency of micronucleated cells.

The frequency of micronucleated cells in each replicate culture was above the vehicle historical range for at least one dose level.

There was a statistically significant difference versus the vehicle control at one or more dose levels.

The results were considered to be clearly negative if none of the criteria for a positive response were met.

Toxicology study

Thirteen-week toxicity study—Experimental design

The potential toxicity of Protealg was evaluated following oral administration (by gavage) to Sprague-Dawley rats for 13 weeks, followed by a 4-week treatment-free period. The animals were treated twice daily (bis in die: bid) with a 4-h interval between the two administrations. Dose levels of 0, 500, 2000, and 5000 mg/kg/day were chosen based on the results of a preliminary dose range finding study at these dose levels in which Protealg was well tolerated (results not reported here).

Protealg was administered as a suspension in drinking water treated by reverse osmosis (using an Elix 5, Millipore SA), under a constant dosage volume of 10 mL/kg/administration. A control group (10 males and 10 females) received the vehicle only and three groups of 15 rats/sex received Protealg at 0, 250, 1000, and 2500 mg/kg/bid (see Table 1). At the end of the treatment period, five rats/sex from the control and high-dose groups were kept for a 4-week treatment-free period.

Table 1.

Dose groups in the 13-week toxicity study.

Group	Status	Rats	Dose level per administration (mg/kg/bid)	Total daily dose level (mg/kg)
1	Principal	10 males and 10 females	0	0
1	Recovery	5 males and 5 females	0	0
2	Principal	10 males and 10 females	250	500
3	Principal	10 males and 10 females	1000	2000
4	Principal	10 males and 10 females	2500	5000
4	Recovery	5 males and 5 females	2500	5000

The batch of Protealg tested in the present study (batch no. CIT_17_0405) resulted in very viscous dose formulations and treatment at 2500 mg/kg per administration was considered to be the maximum feasible dose; dose formulations became pasty and unsuitable for administration at concentrations above 250 mg/mL. Administration was performed twice daily, thus achieving the same total daily dose levels as were used in the preliminary study (5000 mg/kg/day). Due to the viscosity of the dose formulation, this was considered to be the maximum feasible dose.

The study protocol was based on OECD test guideline 408 and the “EFSA guidance on the preparation and presentation of an application for authorization of a novel food in the context of EU regulation 2015/2283.”

A subchronic study conducted for a period of at least 90 days is required by the “Guidance on the preparation and presentation of an application for authorization of a novel food in the context of regulation (EU) 2015/2283.” Only one species is required by the guidance. As permitted by the guideline, additional endocrine-related endpoints were included in the study design.

The concentrations of the dose formulations were checked in weeks 1, 4, 8, and 11 by the determination of total nitrogen using the Kjeldahl method, which was validated before dose formulation analysis. Total protein content was calculated as follows:

Total protein = total nitrogen x 6.25

Animals

Sprague-Dawley rats (Crl CD^® (SD) IGS BR, Cesarean Obtained, Barrier Sustained-Virus Antibody Free (COBS-VAF^®)) were obtained from Charles River Laboratories, Italy. The rats were 5 weeks old at receipt with mean body weights of 122 g (males) and 133 g (females). They were acclimated to the study conditions for 7 days before the study.

The animals were housed in twos or threes (same sex and group) in Tecniplast 2000P cages (2065 m²) containing autoclaved sawdust (SICSA, France) and two objects for environmental enrichment. SSNIFF R/M-H pelleted maintenance diet (SSNIFF Spezialdiäten GmbH, Germany) was distributed weekly (at least) and the animals had free access to bottles containing tap water (filtered with a 0.22 µm filter). Food was removed during periods of fasting, but the animals were not deprived of water.

Each animal was checked daily for mortality, morbidity, and clinical signs throughout the study. Detailed clinical examinations were performed once a week. Body weight and food consumption were recorded at least once a week during the study.

Ophthalmology

Ophthalmological examinations were performed once on all animals during the pretreatment period and on control and high-dose rats at the end of the treatment period. No ophthalmology examination was performed at the end of the treatment-free period as no treatment-related changes were observed at the end of the treatment period.

Functional observation battery

A functional observation battery of tests (FOB) was performed on all animals in study week 11. This evaluation included:

A detailed clinical examination (in the cage: touch escape; in the hand: fur appearance, salivation, lacrimation, piloerection, exophthalmos, reactivity to handling, pupil size; in the standard arena: grooming, palpebral closure, defecation, urination, tremors, twitches, convulsions, gait, hypo-/hyper-activity, posture, stereotypy, behavior, breathing, ataxia, and hypotonia).

The assessment of reactivity to manipulation and different stimuli (touch response, forelimb grip strength, pupillary reflex, visual stimulus response, auditory startle reflex, tail pinch response, righting reflex, and landing foot splay).

Rectal temperature was measured at the end of the observation period.

Motor activity was measured over a 60-min period.

The rats were randomized and observed in the cage, in the hand, and in the standard arena. No examination was performed at the end of the treatment-free period as no treatment-related changes were observed at the end of the treatment period.

Laboratory investigations

Hematology, blood biochemistry, urinalysis, and circulating thyroid hormone levels were investigated at the end of the treatment and treatment-free periods.

The animals were fasted overnight prior to the collection of blood samples (over appropriate anticoagulant) before the first daily treatment.

For urine collection, the animals were placed in individual metabolism cages for an overnight period of at least 14 h. The urine was collected onto thymol crystals.

Thyroid stimulating hormone (TSH) analysis was performed at the end of the treatment and treatment-free periods. Blood samples were taken into K₃EDTA tubes and Luminex xMAP^® technology (Biorad, Marnes-la-Coquette, France) was used for analysis. Microspheres coated with antibodies directed against TSH were color-coded with two fluorescent dyes. The TSH in rat plasma was captured by antibody-coupled beads, and then the beads were incubated with a biotinylated-coupled antibody which was revealed using a streptavidin-phycoerythrin (SAPE) conjugate. The microspheres specific to TSH were identified by the emission wavelengths of the beads and the analyte in the tested sample was detected and quantified with SAPE. Analyte concentrations were then determined by interpolating the fluorescence intensity (FI) values of the samples in a calibration curve in which the SAPE FI was plotted against the logarithm (base 10) of the TSH concentrations.

T3 and T4 concentrations were determined in plasma samples at the end of the treatment period. Solid phase extraction was performed using an Oasis HLB (1 cc, 10 mg). After evaporation under nitrogen at +35°C, the extract was reconstituted in 10 mM methanol/ammonium acetate buffer (25/75 v/v) and analyzed by reverse phase liquid chromatography with MS/MS detection (HPLC-MS/MS; Agilent) after electro-spray ionization (positive mode). T3 and T4 concentrations were not determined at the end of the treatment-free period as no changes were detected at the end of the treatment period.

The analytical methods used for the detection of TSH, T3, and T4 were all formally validated before use in the study.

Reproductive parameters

Seminology parameters (epididymal sperm motility and morphology; testicular and epididymal sperm count) were investigated at the end of the treatment period and a detailed histopathological examination of the testes was performed in control and high-dose male rats.

The estrus cycle stage was determined for 14 consecutive days (from a fresh vaginal lavage stained with methylene blue) over the last 2 weeks of the treatment (study weeks 12 and 13) and treatment-free (study weeks 16 and 17) periods.

Histology/pathology

A full macroscopic post-mortem examination was performed. Designated organs were weighed and tissues were preserved. A microscopic histopathology examination was performed on selected tissues (and on any macroscopic lesions) from control and high-dose animals euthanized at the end of the treatment period. As there were no test item-related findings, these examinations were not performed at the end of the treatment-free period.

Statistical analysis

Statistical analysis was performed on body weight, food consumption, seminology parameters, hematology, blood biochemistry, thyroid hormones, and urinalysis data (Citox software, version D.7) and on organ weights (PathData software, V6.2d2).

Guideline compliance, GLP compliance, and animal health regulations

Technical guidance

The bacterial reverse mutation study design was based on OECD guideline No. 471, adopted on July 21 1997,²⁶ and on Council Regulation (EC) No. 440/2008 of May 30, 2008 (Annex Part B, 13/14).

The in vitro mammalian cell micronucleus test study design was based on OECD guideline No. 487, adopted 26 September 2014.²⁷

The study design of the 13-week toxicity study was based on technical guidance in:

OECD Guideline No. 408, 21 September 1998²⁸ on Repeated Dose 90-day Oral Toxicity Study in Rodents.

EFSA Panel on Dietetic Products, Nutrition and Allergies, 2016. Guidance on the preparation and presentation of an application for authorisation of a novel food in the context of Regulation (EU) 2015/2283.

GLP compliance

The studies were all performed in compliance with the following principles of good laboratory practice:

OECD Principles of GLP (as revised in 1997),²⁶ ENV/MC/CHEM (98) 17.

Directive 2004/10/EC of the European Parliament and of the Council of 11 February 2004.

Annexe 2 à l’article D523-8 du code de l’environnement.

Animal health regulations

The 13-week toxicity study was conducted in compliance with the following animal health regulations:

Council Directive No. 2010/63/EU of 22 September 2010 on the protection of animals used for scientific purposes (French decree No. 2013-118 of 01 February 2013 on the protection of animals used for scientific purposes).

Results

Bacterial reverse mutation (Ames)

The mean number of revertant Salmonella colonies for the vehicle and positive controls met the acceptance criteria, demonstrating the validity of the study test (see Online Supplemental Table 1).

Based on the results of the preliminary test, the highest dose recommended by OECD guideline 471, namely 5000 µg/plate, was selected as the high dose for this study.

No precipitation of the test item was observed at any dose level. Blue coloration of the agar was observed at the high dose level of 5000 µg/plate (except for TA102 without S9 mix), but this coloration did not prevent scoring of the revertant colonies.

Moderate toxicity as demonstrated by thinning of the bacterial lawn (in strain TA102) or reduction in the mean number of revertant colonies (strain TA1537) was occasionally noted at higher dose levels without S9 mix. No toxicity was noted for the other tester strains or test conditions.

Protealg did not induce any noteworthy increase in the number of revertant colonies, in any strain, or under any test condition. It was therefore considered that Protealg did not demonstrate any mutagenic activity in the Ames bacterial reverse mutation test with five S. typhimurium strains, either in the presence or in the absence of a rat liver metabolizing system (S9).

In vitro mammalian cell micronucleus

The highest dose levels selected for microscopic scoring of the in vitro mammalian cell micronucleus test were based on the observed cytotoxicity test (see Online Supplemental Table 2). Following the criterion specified in OECD guideline 487, the highest dose level should aim to produce 55% cytotoxicity as measured by the PD value. Cytotoxicity (as shown by a decrease in mean PD) and the mean frequency of micronucleated cells in vehicle cultures met the acceptance criteria. Positive control cultures showed statistically significant increases in the frequency of micronucleated cells.

After treatment with Protealg, a blue-green coloration of the culture medium was observed at dose levels ≥300 µg/mL. The presence of precipitate (recorded on slides) prevented cell scoring at dose levels ≥400 µg/mL (first experiment without S9 mix; 3 h treatment), ≥800 µg/mL (all experiments with S9 mix) and ≥2000 µg/mL (first experiment without S9 mix; 24 h treatment).

Short treatments without S9 mix: For the short treatments without S9 mix (3 h treatment + 24 h recovery): The dose levels selected for analysis in the first experiment were 37.5, 75, and 150 µg/mL (this last induced a 13% decrease in PD). Severe cytotoxicity was induced at higher dose levels. The dose levels selected for analysis in the second experiment were 200, 250, and 300 µg/mL (this last induced a 45% decrease in PD). Again severe cytotoxicity was induced at higher dose levels. The dose levels selected for analysis in the third experiment were 75, 150, and 200 µg/mL (this last induced a 39% decrease in PD). Severe cytotoxicity was induced at higher dose levels.

Long treatment without S9 mix: For the long treatment without S9 mix (24 h treatment + 0 h recovery), the selected dose levels for analysis were 200, 400, and 600 µg/mL (this last induced a 45% decrease in PD). Severe cytotoxicity was induced at higher dose levels.

Treatments with S9 mix (3 h treatment + 24 h recovery): The dose levels selected for analysis were 100, 200, and 300 µg/mL (this last induced a 38% decrease in PD). Severe cytotoxicity was induced at higher dose levels.

For short and long treatments without or with S9 mix, even though the results did not attain the level of toxicity required by guidelines (a 55 ± 5% decrease in PD), despite the use of narrow dose levels spacings used, the overall results are considered to be sufficiently robust to allow a reliable interpretation.

Micronucleus frequency: No statistically significant increase in the frequency of micronucleated L5178Y TK^+/− mouse lymphoma cells was observed at any of the analyzed Protealg dose levels when compared to the vehicle, and no dose–response relationship was demonstrated by linear regression. Moreover, none of the analyzed dose levels (in either of the replicate cultures) were considered to show a frequency of micronucleated cells above the corresponding vehicle control historical range.

It was concluded that, under the experimental conditions of the study and based on the overall results of experiments conducted with and without S9 mix, the test item, Protealg, did not induce any chromosome damage or damage to the cell division apparatus, in cultured mammalian somatic cells, using L5178Y TK^+/− mouse lymphoma cells, as measured by the production of micronucleated cells.

Toxicology study

Formulation analysis

The Protealg concentrations in the dose formulations were within the accepted range (±15% of the nominal concentration), except for one low dose formulation in week 8 which was outside specification.

Clinical signs of toxicity

Protealg was well tolerated and there were no clinical signs indicative of toxicity and no unscheduled mortality during the study.

Food consumption and body weight

Lower mean body weight (see Figure 1(a)) was seen throughout the study in high-dose male rats when compared to the corresponding control, but without statistical significance. This was considered to fall within the physiological variations and was not considered to be adverse in view of the low magnitude (−5%) and lack of dose relationship.

Figure 1.

(a) Body weights over the treatment period (principal and recovery animals). (b) Food consumption over the treatment period (principal and recovery animals).

Lower food consumption (see Figure 1(b)) was also noted over the course of the study, with statistical significance on a few occasions, in male rats treated at the high dose of 5000 mg/kg/day. The lower food consumption in high-dose males may correlate with the lower body weight gain in these animals, but was not considered to be adverse in view of the low magnitude. Lower food consumption could have been caused by the viscosity of the Protealg suspensions leading to satiety.

No relevant differences in food consumption or bodyweight were recorded in any other group.

Ophthalmology

There were no ophthalmological findings at any dose level at the end of the treatment period. Consequently no examination was considered necessary at the end of the treatment-free period.

Functional observation battery

No relevant differences between control and Protealg-treated animals were recorded for the neurological, behavioral, or autonomic functions during the FOB (see Figure 2).

Figure 2.

Functional observation battery in study week 11.

Laboratory investigations

Among the hematology parameters at the end of the treatment period (see Table 2) statistically significant, shortened prothrombin time was noted in male rats treated at 2000 or 5000 mg/kg/day, and statistically significant prolongation of prothrombin time was also noted at all dose levels in female rats, but with a poor dose relationship. As this effect was not consistent in males and females, and as values were within the physiological range, a relationship to treatment with Protealg was considered to be unlikely.

Table 2.

13-week oral toxicity study: Hematology at end of treatment period.

	Males				Females
Dose level of Protealg (mg/kg bw/day)	0	500	2000	5000	0	500	2000	5000
White blood cells (G/L)	12.43	10.31^a	8.98^b	11.07	6.12	5.72	5.64	6.25
Red blood cells (T/L)	9.12	9.23	8.72	9.22	8.26	8.47	8.63	9.24^b
Hemoglobin (g/dL)	15.4	15.5	15.1	15.9	14.9	15.2	15.5	16.4^b
Packed cell volume (L/L)	0.48	0.49	0.47	0.49	0.45	0.46	0.47	0.51^b
Mean corpuscular volume (fL)	53.0	52.8	54.4	53.2	54.8	54.4	54.9	54.3
Mean corpuscular hemoglobin (pg)	16.9	16.8	17.3	17.2	18.1	18.0	17.9	17.8
Mean corpuscular hemoglobin concentration (g/dL)	31.9	31.8	31.8	32.4	33.0	33.0	32.6	32.8
Platelets (G/L)	943	951	902	929	826	866	954^b	838
Reticulocytes (%)	2.53	2.52	2.76	2.26	2.05	1.90	2.01	1.83
Neutrophils (G/L)	1.67	1.45	1.45	1.50	0.59	0.97^a	0.62	0.58
Eosinophils (G/L)	0.15	0.14	0.16	0.18	0.10	0.11	0.12	0.10
Basophils (G/L)	0.03	0.02	0.01	0.02	0.01	0.01	0.01	0.01
Lymphocytes (G/L)	10.12	8.32	6.99^b	8.97	5.17	4.42	4.68	5.31
Large unstained cells (G/L)	0.09	0.06	0.06	0.06	0.05	0.03	0.04	0.05
Monocytes (G/L)	0.36	0.32	0.31	0.34	0.20	0.18	0.17	0.20
Prothrombin time (s)	25.3	24.3	23.5^b	23.8^b	22.4	24.2^a	25.2^a	24.9^a

^a Significance level p < 0.05.

^b Significance level p < 0.01.

Statistically significant increases in mean red blood cell count (9.24T/L), hemoglobin concentration (16.4 g/dL), and packed cell volume (0.51L/L) were noted in females at the high dose of 5000 mg/kg/day when compared to controls at the end of the treatment period. The increased values were considered to be without consequence as they were within the historical control range (RBC: 6.82–10T/L, HB: 12.4–16.6 g/dL, and PCV: 0.38–0.54L/L). These increases were considered to be unrelated to the Protealg treatment as they were not observed in males and were strongly influenced by a single female with high RBC (11.41T/L), HB (21.3 g/dL), and PCV (0.70L/L) values indicative of hemoconcentration and suggestive of dehydration. But there were no correlated blood biochemistry changes to favor this hypothesis, no corresponding clinical signs were noted, and there were no associated necropsy findings.

Among the blood biochemistry parameters at the end of the treatment period (see Table 3), there were statistically significant, lower mean protein and albumin levels in female rats treated at 5000 mg/kg/day when compared to controls. These findings were considered to be related to the Protealg treatment, but were found to be reversible at the end of the treatment-free period. As the values remained within the respective physiological ranges, the decreases were considered to be non-adverse.

Table 3.

Thirteen-week oral toxicity study: Blood biochemistry at end of treatment period.

	Males				Females
Dose level (mg/kg bw/day)	0	50	2000	5000	0	500	2000	5000
Sodium (mmol/L)	143.8	144.2	144.0	144.6	142.8	142.4	142	142.7
Potassium (mmol/L)	3.88	4.06	3.93	3.78	3.71	3.63	3.59	3.49
Chloride (mmol/L)	107.5	107.6	107.8	108.0	107.7	108.0	108.2	108.1
Calcium (mmol/L)	2.61	2.59	2.57	2.57	2.61	2.57	2.57	2.53^a
Inorganic phosphorus (mmol/L)	2.14	2.05	2.04	2.09	1.64	1.60	1.59	1.65
Glucose (mmol/L)	7.47	7.96	7.80	7.27	6.81	6.99	6.41	6.11^a
Urea (mmol/L)	4.4	4.3	4.5	4.1	5.7	5.6	5.6	5.0
Creatinine (µmol/L)	32.68	33.45	34.26	31.61	36.66	34.73	35.76	34.98
Total bilirubin (µmol/L)	0.82	0.66	1.11	1.00	1.34	0.93	1.15	1.34
Proteins (g/L)	62.8	62.0	60.8	60.8	67.5	67.7	66.9	63.1^a
Albumin (g/L)	38	38	37	37	42	41	41	38^b
Albumin to globulin ratio	1.51	1.53	1.53	1.53	1.62	1.58	1.55	1.55
Cholesterol (mmol/L)	1.68	1.68	1.73	1.50	2.34	2.19	1.79^b	2.03
Triglycerides (mmol/L)	0.59	0.61	0.62	0.60	0.36	0.41	0.31	0.33
Alkaline phosphatases (IU/L)	192	205	219	229	112	113	135	130
Aspartate aminotransferase (IU/L)	83	80	80	78	95	108	99	163
Alanine aminotransferase (IU/L)	37	31	35	32	42	64	43	87

^a Significance level p < 0.05.

^b Significance level p < 0.01.

Statistically significant, slightly lower pH was recorded in the urine of males and females at 5000 mg/kg/day at the end of the treatment period (see Online Supplemental Table 3). This was considered to be non-adverse in view of the low amplitude and was found to be reversible at the end of the treatment-free period. The color of the urine was yellow to dark yellow and without evidence of blue coloration.

There were no differences in T3 and T4 levels between control and test item-treated animals at the end of the treatment period (see Table 4), so these parameters were not evaluated at the end of the treatment-free period.

Table 4.

Thyroid hormone values at the end of the treatment and treatment-free periods.

	Males				Females
Dose level (mg/kg/day)	0	500	2000	5000	0	500	2000	5000
	Treatment period
TSH (pg/mL)	1531 ±756.3	1557 ±1604.2	1999 ±1633.0	2787 ±1778.9	682 ±416.6	1300 ±1053.0	1326 ±885.8	1240 ±1382.5
T3	0.31	0.33	0.37	0.38	0.46	0.45	0.41	0.45
T4	46.05	46.58	49.39	43.17	26.12	29.65	23.84	27.72
	Treatment-free period
TSH (pg/mL)	2332 ±1342.2			3042 ±1256.3	680 ±535.7			551 ±319.4

±: Standard deviation. Historical data for males: 489–4485 and for females: 501–1278.

Slightly higher mean TSH concentrations were noted in males at 2000 and 5000 mg/kg/day, and in all test item-treated females (without any evident dose-relationship) at the end of the treatment period (see Table 4). None of these changes were statistically significant and mean values remained within the historical ranges. Individual variability was very high (see Figure 3) and there were one or two isolated elevated values in each treated group.

Figure 3.

TSH levels at the end of the treatment and treatment-free periods (with geometric mean). TSH: thyroid stimulating hormone.

At the end of the treatment-free period, mean TSH level remained higher in recovery males previously treated at 5000 mg/kg/day, when compared to the corresponding control value, but in females, the value was lower than the control value. In controls, mean TSH level was higher at the end of the treatment-free period than at the end of the treatment period.

As there were no effects on T3 or T4 levels at the end of the treatment period, the elevated outlying TSH values did not correspond to higher T3 or T4 values, and there were no changes in organ weights or histological findings in the pituitary, thyroid, or liver at the end of the treatment or treatment-free period. It was therefore considered that the increased TSH levels were without consequences and of no toxicological importance.

Reproductive parameters

There were no treatment-related effects on seminology parameters (epididymal sperm count, epididymal sperm morphology and motility, and testicular sperm count) or estrus cycles (see Table 5).

Table 5.

Reproductive parameters.

Dose level (mg/kg/day)	0	500	2000	5000
Males	Study week 13
% of motile epididymal sperm	96.5	95.1	99.1	98.1
% of morphologically normal epididymal sperm	97.3	97.9	97.9	96.6
Mean number of epididymal sperm (10⁶/cauda)^a	157	143	151	141
Mean number of epididymal sperm (10⁶/g cauda)^a	419	397	413	409
Mean number of testicular sperm heads (10⁶/g testis)^a	138	116^a	113^b	120
Daily sperm production rate (10⁶/g testis/day)	22.7	19.0^a	18.5^b	19.7^a
Females	Study weeks 12 and 13
Number of cycles	2.3	2.1	2.4	2.4
Cycle length (days)	4	4.1	4.2	3.9
Rats with normal cycles	9	9	10	7
	Study weeks 16 and 17
Number of cycles	2.5	–	–	2.8
Cycle length (days)	4.1	–	–	4.1
Rats with normal cycles	4	–	–	4

–: not performed.

^a Statistical significance level p < 0.05.

^b Statistical significance level p < 0.01.

The daily sperm production rate was slightly lower at all dose levels of Protealg, correlating with a slightly lower mean number of testicular sperm heads at 500 and 2000 mg/kg/day. This was not considered to be treatment related as there was no dose relationship, there were no microscopic findings in the testes or epididymides, and individual values were similar to the historical control values.

At the end of the treatment period (weeks 12 and 13), the mean numbers and duration of cycles in test item treated groups were similar to those of the control group. At 5000 mg/kg/day, 3/10 females had slightly longer diestrus and shorter metestrus stages, leading to a mean average cycle of less than 4 days, but overall, the mean numbers of cycle stages were similar in all groups. The only differences observed were of minor amplitude and/or not dose-related.

The mean numbers of normal cycles and the mean numbers of cycle stages were similar in females previously treated at 5000 mg/kg/day and in controls at the end of the treatment period (study weeks 16 and 17), and estrus cycle stage was similar in both groups (determined at microscopic examination).

Histology/pathology

There were no test item-related organ weight differences (see Online Supplemental Table 4) and no macroscopic or microscopic postmortem findings.

The organ weight differences were not considered to be test item related as they were not dose-related, and they were secondary to low body weights, without macroscopic/microscopic correlates and/or of minimal magnitude.

The macroscopic findings (Online Supplemental Table 5) in treated animals were also observed in control animals and/or correlated with historical control data for rats of this strain and age. They were thus considered to be incidental.

The microscopic findings were not considered to be test item related as they were consistent with spontaneously occurring findings in rats of this age, the findings were distributed randomly among the groups, or their appearance was similar to findings in controls.

There were no test item-related microscopic changes at the detailed examination of testes and epididymides.

At the end of the treatment free period, there were no test item-related gross findings and no microscopic examination was performed as there were no test item-related changes at the end of the treatment period.

Discussion

The safety of Protealg, a high protein microalgal biomass prepared from G. sulphuraria, for use as a food supplement was evaluated in a 13-week oral toxicity study in Sprague-Dawley rats, the Ames bacterial reverse mutation test, and the in vitro mammalian cell micronucleus test in L5178Y TK^+/− mouse lymphoma cells.

These studies were performed in compliance with good laboratory practices, and the design of the studies followed current technical guidance documents, namely OECD TG 408 for the 13-week oral toxicity study, OECD TG 471 for the Ames test, and OECD TG 487 for the in vitro mammalian cell micronucleus test.

In the genetic toxicology studies, the results were uniformly negative and it was concluded that there was no evidence of mutagenic properties in the Ames test or the in vitro micronucleus test. The testing of proteins in the Ames test may give rise to artefactual results resulting from “feeding” of the auxotrophic Salmonella tester strains by free histidine released from the proteinaceous test item. Despite a relatively elevated content of 0.8 g histidine/100 g Protealg (after hydrolysis), the values for revertant colony numbers were coherent with the historical control data range and there was no enhanced growth of the bacterial lawn. It was considered that Protealg did not give rise to such histidine artifacts.

In the 13-week oral toxicity study, administration was by the oral route, the intended route of exposure in humans. Protealg was administered by oral gavage as a suspension in purified water while control animals received saline solution. Because of the viscosity of the Protealg dose formulation, the daily dose was divided, and gavage was performed twice daily (bid). It was considered that the high dose level of Protealg (5000 mg/kg/day given as 2500 mg/kg/day bid) was the maximum feasible dose.

The results for the various parameters evaluated in the 13-week oral toxicity study were generally negative; there were no findings for clinical observations, FOB, ophthalmology, urinalysis, seminology, organ weight analysis, or histopathology. For other parameters, minor findings were present which were discounted as fortuitous findings with no harmful consequences. These included minor changes in body weight and food consumption, red blood cell values, blood coagulation (as measured by prothrombin time), thyroid parameters, and estrus cycle prolongation. Overall it can be concluded that administration of Protealg twice daily at a total daily dose of 5000 mg/kg/day (maximum feasible dose) did not result in any findings indicative of toxicity.

At the present time, studies on reproductive toxicity, neurotoxicity, and long term exposure have not been performed.

This conclusion supports the safety of Protealg for use as food ingredient or supplement by the general public. The daily dietary reference intake of protein for humans is 0.8 g/kg or 48 g for a 60 kg human subject (WHO, EFSA).²⁹ The dose levels used in the 13-week oral toxicity study (up to 5000 mg/kg body weight/day) exceed this intake level (800 mg/kg bodyweight/day) by a factor of more than 6-fold.

The absence of toxicity in the 13-week study at the maximum feasible dose level of 5000 mg/kg/day, which is 100-fold the maximum human dose (this is the current US standard safety margin and represents 50 mg/kg/day for a 70 kg adult), taken together with the negative findings of the Ames test and in vitro micronucleus test, support the safety of the intended use of Protealg, a high protein microalgal biomass prepared from G. sulphuraria, for use as a food ingredient or supplement.

Supplemental material

Supplementary_table_1_Ames_test - Safety evaluation of Galdieria high-protein microalgal biomass

Supplementary_table_1_Ames_test for Safety evaluation of Galdieria high-protein microalgal biomass by Virginie Modeste, Alizée Brient, Catherine Thirion-Delalande, Roy Forster, Corinne Aguenou, Hywel Griffiths and Olivier Cagnac in Toxicology Research and Application

Supplemental material

Supplementary_table_2_In_vitro_micronucleus_test - Safety evaluation of Galdieria high-protein microalgal biomass

Supplementary_table_2_In_vitro_micronucleus_test for Safety evaluation of Galdieria high-protein microalgal biomass by Virginie Modeste, Alizée Brient, Catherine Thirion-Delalande, Roy Forster, Corinne Aguenou, Hywel Griffiths and Olivier Cagnac in Toxicology Research and Application

Supplemental material

Supplementary_table_3_13_Week_Urinalysis - Safety evaluation of Galdieria high-protein microalgal biomass

Supplementary_table_3_13_Week_Urinalysis for Safety evaluation of Galdieria high-protein microalgal biomass by Virginie Modeste, Alizée Brient, Catherine Thirion-Delalande, Roy Forster, Corinne Aguenou, Hywel Griffiths and Olivier Cagnac in Toxicology Research and Application

Supplemental material

Supplementary_table_4_Organ_weights - Safety evaluation of Galdieria high-protein microalgal biomass

Supplementary_table_4_Organ_weights for Safety evaluation of Galdieria high-protein microalgal biomass by Virginie Modeste, Alizée Brient, Catherine Thirion-Delalande, Roy Forster, Corinne Aguenou, Hywel Griffiths and Olivier Cagnac in Toxicology Research and Application

Supplemental material

Supplementary_table_5_Macroscopic_findings - Safety evaluation of Galdieria high-protein microalgal biomass

Supplementary_table_5_Macroscopic_findings for Safety evaluation of Galdieria high-protein microalgal biomass by Virginie Modeste, Alizée Brient, Catherine Thirion-Delalande, Roy Forster, Corinne Aguenou, Hywel Griffiths and Olivier Cagnac in Toxicology Research and Application

Supplemental material

Supplementary_Table_6_Amino_acids_(002) - Safety evaluation of Galdieria high-protein microalgal biomass

Supplementary_Table_6_Amino_acids_(002) for Safety evaluation of Galdieria high-protein microalgal biomass by Virginie Modeste, Alizée Brient, Catherine Thirion-Delalande, Roy Forster, Corinne Aguenou, Hywel Griffiths and Olivier Cagnac in Toxicology Research and Application

Footnotes

Authors’ note

Authors Virginie Modeste, Alizée Brient, Catherine Thirion-Delalande, and Roy Forster are employees of Citoxlab France, which received financial support for the conduct of this research from Fermentalg. Authors Olivier Cagnac, Corinne Aguenou, and Hywel Griffiths are employees of Fermentalg.

Acknowledgement

The authors thank Joanna Moore, ELS (Citoxlab France), for assistance in preparing the manuscript.

Declaration of conflicting interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

Supplemental material

Supplemental material for this article is available online.

References

Centre d’étude et de Valorisation des Algues (CEVA). Réglementation algues alimentaires. Pleubian, France: CEVA, 22610, 2014.

Horowitz

, et al. Life cycle assessment of industrial scale production of spirulina tablets. Algal Res 2018; 34: 154–163.

Bleakley

Hayes

. Algal proteins: extraction, application, and challenges concerning production. Foods 2017; 6(5): E33.

García

de Vicente

Galán

. Microalgae, old sustainable food and fashion nutraceuticals. Microbiol Biotechnol 2017; 10(5): 1017–1024.

Drobac

Tokodi

Simeunović

, et al. Human exposure to cyanotoxins and their effects on health. Arh Hig Rada Toksikol 2013; 64(2): 119–130.

Manali

Arnraj

Kumar

, et al. Detection of microcystin producing cyanobacteria in Spirulina dietary supplements using multiplex HRM quantitative PCR. J Appl Phycol 2017; 29(3): 1279–1286.

Shimamatsu

. Mass production of Spirulina, an edible microalga. Hydrobiologia 2004; 512(1): 39–44.

Jiang

Chen

Liang

. Detection of the hepatotoxic microcystins in 36 kinds of cyanobacteria spirulina food products in China. Food Addit Contam A Chem Anal Control Expo Risk Assess 2008; 25(7): 885–894.

Doemel

Brock

. pH of very acid soils. Nature 1971; 229(5286): 574.

10.

Ciniglia

Yang

Pollio

, et al. Cyanidiophyceae in Iceland: plastid rbcL gene elucidates origin and dispersal of extremophilic Galdieria Sulphuraria and G. Maxima (Galdieriaceae, Rhodophyta). Phycologia 2014; 53(6): 542–551.

11.

Eriksen

. Production of phycocyanin – a pigment with applications in biology, biotechnology, foods and medicine. Appl Microbiol Biotechnol 2008; 80(1): 1–14.

12.

Cagnac

Lannig

Labro

. Novel method for the culture of unicellular red algae. 2017a. WO/2017/050917.

13.

Cagnac

Gehin

Champeau

. Method for culturing unicellular red algae (ura) with milk permeate. 2017b. WO/2017/093345.

14.

Schönknecht

Chen

W-H

Ternes

, et al. Gene transfer from bacteria and archaea facilitated evolution of an extremophilic eukaryote. Science 2013; 339(6124): 1207–1210.

15.

Gross

Schnarrenberger

. Heterotrophic growth of two strains of the acido-thermophilic red alga Galdieria sulphuraria . Plant Cell Physiol 1995; 36(4): 633–638.

16.

Oesterhelt

Schmaelzlin

Schmitt

, et al. Regulation of photosynthesis in the unicellular acidophilic red alga Galdieria sulphuraria . Plant J 2007; 51(3): 500–511.

17.

Schmidt

Wiebe

Eriksen

. Heterotrophic high cell-density fed-batch cultures of the phycocyanin-producing red alga Galdieria sulphuraria . Biotechnol Bioeng 2005; 90(1): 77–84.

18.

Sloth

Wiebe

Eriksen

. Accumulation of phycocyanin in heterotrophic and mixotrophic cultures of the acidophilic red alga Galdieria sulphuraria . Enzyme Microb Tech 2006; 38(1–2): 168–175.

19.

Graverholt

Eriksen

. Heterotrophic high-cell-density fed-batch and continuous-flow cultures of Galdieria sulphuraria and production of phycocyanin. Appl Microbiol Biot 2007; 77(1): 69–75.

20.

Schoenknecht

Chen

Ternes

, et al. Gene transfer from bacteria and archaea facilitated evolution of an extremophilic eukaryote. Science 2013; 339(6124): 1207–1210.

21.

Stadnichuk

Rakhimberdieva

Bolychevtseva

, et al. Inhibition by glucose of chlorophyll a and phycocyanobilin biosynthesis in the unicellular red alga Galdieria partita at the stage of coproporphyrinogen III formation. Plant Sci 1998; 136(1): 11–23.

22.

Troxler

Bogorad

. Studies on the formation of phycocyanin, porphyrins, and a blue phycobilin by wild-type and mutant strains of Cyanidium caldarium . Plant Physiol 1966; 41(3): 491–499.

23.

Boisen

Fernandez

. Prediction of the total tract digestibility of energy in feedstuffs and pig diets by in vitro analyses. Anim Feed Sci Tech 1997; 68(3–4): 277–286.

24.

Yoshikawa

Belay

. Single-laboratory validation of a method for the determination of c-phycocyanin and allophycocyanin in spirulina (Arthrospira) supplements and raw materials by spectrophotometry. J AOAC Int 2008; 91(3): 524–529.

25.

Moon

Mishra

Kim

, et al. Isolation and characterization of thermostable phycocyanin from Galdieria sulphuraria . Korean J Chem Eng 2014; 31(3): 490–495.

26.

OECD guideline 471 for testing of chemicals. Bacterial Reverse mutation test, adopted 21st July 1997.

27.

OECD guideline 487 for the testing of chemicals. In vitro mammalian cell micronucleus test, adopted 26th September 2014.

28.

OECD guideline 408 for the testing of chemicals. Repeated dose 90-day oral toxicity study in rodents, adopted 21st September 1998.

29.

WHO and EFSA recommended protein intake. https://ec.europa.eu/jrc/en/health-knowledge-gateway/promotion-prevention/nutrition/protein. (accessed 17 June 2019).

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.18 MB

0.35 MB

0.31 MB

0.26 MB

0.19 MB

0.29 MB