Abstract
Two different approaches can be used in phylogenomics: combined or separate analysis. In the first approach, different datasets are combined in a concatenated supermatrix. In the second, datasets are analyzed separately and the phylogenetic trees are then combined in a supertree. The supertree method is an interesting alternative to avoid missing data, since datasets that are analyzed separately do not need to represent identical taxa. However, the supertree approach and the corresponding consensus methods have been highly criticized for not providing valid phylogenetic hypotheses. In this study, congruence of trees estimated by consensus and supertree approaches were compared to model trees obtained from a combined analysis of complete mitochondrial sequences of 102 species representing 93 mammal families. The consensus methods produced poorly resolved consensus trees and did not perform well, except for the majority rule consensus with compatible groupings. The weighted supertree and matrix representation with parsimony methods performed equally well and were highly congruent with the model trees. The most similar supertree method was the least congruent with the model trees. We conclude that some of the methods tested are worth considering in a phylogenomic context.
Introduction
The phylogenomic era has brought a shift from single to multiple datasets (or genes) to study phylogenetic relationships. 1 While increasing the number of characters decreases stochastic errors, it also increases phylogenetic signal.2–4 However, phylogenomic studies present numerous methodological challenges (see review by Delsuc et al). 5 Two opposite views have been proposed as to how to incorporate the growing amount of data to infer evolutionary relationships. Whereas the combined approach (sensu de Queiroz) 6 concatenates different datasets in a supermatrix,7–10 the consensus approach (sensu de Queiroz) 6 analyzes datasets separately and the resulting trees are combined with a consensus11–13 or a supertree method.14–16 The pros and cons of these competing approaches have been debated at length in the literature.10,17–25 When the combined approach is used, the concatenation of numerous genes from different species often results in a supermatrix with missing data. Indeed, a taxon bias has been observed in genetic databases, with a large number of genes (or whole genome) sequenced for a few key species thus resulting in large supermatrices dominated by missing data.2,23,24,26,27
An approach that can be applied to deal with incomplete matrices is the supertree method.14–16,22 In the likely event that some gene sequences are not available for all species, it is possible to estimate a phylogenetic tree for each gene separately and then combine the resulting trees with a supertree approach. Whereas separate datasets may not contain identical sets of taxa (i.e. only overlapping sets), the resulting supertree includes all taxa. Numerous supertree methods have been developed, the most familiar being the matrix representation with parsimony (MRP).28–30 They can be defined as a generalization of consensus methods, which only applies to trees defined on an identical set of taxa.6,11,12,20,31 Interestingly, it is possible to compare performances of classical consensus methods with that of supertree methods in a consensus setting, i.e. when all datasets have identical taxa. 32 Since supertree methods are often developed from existing consensus methods, a setting that allows both types of methods to be directly compared is desirable to assess their relative accuracies.
The supertree strategy seems to be increasingly used in phylogenomics, where large amounts of data can be subdivided to facilitate phylogenetic analyses (i.e. divide-and-conquer strategy).22,33 Furthermore, supertree methods have been proposed as representing the optimal solution to reconstruct the Tree of Life.14,15,33,34 Consensus and supertree methods are similar in design, and both can be referred to as separate analyses, by opposition to a combined analysis (sensu de Queiroz) 6 where sequence datasets are concatenated in a single supermatrix. A heated debate between those in favor and those opposed to the use of consensus has been raging for the last decade.18–20,35–37 The same debate has recently been extended to supermatrices and supertrees.7,10,14,16,17,23,25,38–40
The mammalian phylogeny has been extensively studied and the analysis of different sources of data leads to congruent phylogenies (see review by Springer and Murphy, 41 Springer et al 42 and Wildman et al.) 43 Recent studies of mammal species have shown that mitochondrial phylogenies can be congruent to nuclear phylogenies when potential phylogenetic biases are removed. For example, additional taxa can be added to break long branches 44 and compositional bias and heterogeneity in substitution rates can be appropriately handle.44–46 Another alternative is to use different substitution models depending on the codon position in order to account for a compositional bias of nucleotides among species. 47 Numerous mitogenomic sequences of mammalian species are available and this data availability combined to accurate phylogenetic hypotheses is an ideal setting to test different phylogenetic approaches.
In this study, three different approaches that are commonly used in phylogenomics to analyze DNA sequence matrices were studied in a consensus setting (sensu Bininda-Emonds 32 ): (1) topological consensus methods, (2) topological supertree methods, and (3) weighted supertree methods that account for branch lengths. Congruence among these competing approaches was compared with respect to model trees that were obtained from a complete matrix of mitogenomic mammalian sequences (i.e. a phylogenetic tree obtained from concatenated gene sequences) of 102 species representing 93 mammal families.
Methods
Model tree
DNA sequence alignments
In February 2009, 96 mammal families had at least one species with a complete mitochondrial (mt) sequence in GenBank. In this study, one taxon per family was chosen. When more than one taxon was available, only one species was selected per family (with a few exceptions, see below) and the entire mt sequence was downloaded (see Appendix 1 for GenBank accession numbers). The twelve mitochondrial genes of the H-strand were aligned using ClustalX 2.0.10 48 and the alignment was further verified by eye in SeAl 2.0a11. Ambiguous sites and overlapping regions of the ATP6-ATP8 and NAD4-NAD4L were removed.
Stationarity of base frequencies across taxa was tested on the complete alignment using the chi-square test of homogeneity of base frequencies implemented in PAUP* 4.0. 49 A test of congruence among distance matrices (CADM) 50 was used to determine the congruence among the twelve mt genes using R 2.9.0,51,52 and Ape 2.3 package,53,54 with 9999 permutations for significance testing.
A well-supported tree, compatible with current consensus of mammal molecular phylogeny,41,55 was required to represent interfamilial mitogenomic relationships. However, systematic errors, mainly caused by reconstruction artifacts, can produce a biased tree topology. 5 Among potential systematic errors, heterogeneity in base composition47,56–59 and different evolutionary rates among species60–65 have often been cited as confounding factors affecting the inference of mammalian mitogenomic relationships. Indeed, preliminary analyses of our dataset revealed the presence of systematic biases and different strategies were used to reduce their effect.46,58 For one, the third codon position was removed since it evolves more rapidly, especially in the mitochondrial genome,56,66 and is often saturated for higher-level relationships. 47 Also, the first codon position of leucine (C and T) was recoded as pyrimidine (Y). Additionally, three problematic species were removed (Anomalurus sp., Anomaluridae, Erinaceus europaeus, Erinaceidae and Manis tetradactyla, Manidae). These species are known to be affected by either reduced or accelerated evolutionary rates which can lead to long-branch attraction or positional uncertainty due to short branches.46,62,66–68 Finally, nine extra species were added to break long branches 46 within the following six families: Chrysochloridae, Elephantidae, Macroscelidae, Procaviidae, Soricidae and Talpidae. Consequently, a total of 102 complete mt sequences, representing 93 mammalian families, were included (see Appendix 1).
Phylogenetic inference
Modeltest 3.7 was used to identify the best model of nucleotide substitution. 69 Both the hierarchical likelihood ratio tests (hLRTs) and Akaike information criterion (AIC) suggested a general time-reversible model (GTR)70–72 following a gamma distribution (Γ) 73 with invariant sites (I). The equilibrium frequencies of nucleotides A, C, G, and T were: gA = 0.3452, gC = 0.2054, gG = 0.0901, gT = 0.3593, the relative substitution rates were: rAC = 1.1083, rAG = 6.7749, rAT = 1.1934, rCG = 1.3020, rCT = 3.9717, rGT = 1.0000, and parameters α and I were 0.6762 and 0.4437 respectively. Phylogenetic trees were estimated using two different methods: maximum likelihood (ML)74,75 and Bayesian maximum likelihood (BML).76,77 ML analysis was performed with PhyML 3.0, 78 with a GTR + Γ + I model, where base frequencies, proportion of invariable sites and gamma shape distribution parameters were estimated from the data. The number of categories for the gamma distribution was set to six. A subtree pruning and regrafting (SPR) algorithm was selected, starting from a BioNJ tree, and ten additional random starting trees. Non-parametric bootstrap support (BS) was assessed using identical settings in PhyML for 100 replicates. BML was performed with MRBAYES 3.1.2 76 on a shared-memory multiprocessor computer (Altix 4700). Two MCMC analyses were run for 5,000,000 generations each, using the same GTR + Γ + I model. The Metropolis coupling used eight chains, starting from a random tree and eight swaps with Markov chains sampled every 100th generation, and with a burn-in of 10%. The majority-rule consensus tree and Bayesian posterior probabilities (BPP) were obtained from the tree distribution.
Model tree topology
The ML and BML tree topologies were identical, except at two nodes. These incongruent clades were represented by polytomies in order to render the ML and BML trees completely congruent (Fig. 1). This topology was used as the first model tree (MT1). A second model tree (MT2) was then constructed by collapsing all branches that were not compatible to the current molecular consensus of mammal phylogeny. In this second tree (Fig. 2), eight extra polytomies were added to the first model tree to ensure that all clades were compatible with recent molecular studies.46,66,79–82

First model tree (MT1) representing mitogenomic relationships among 93 mammalian families. Bootstrap values (BS) and Bayesian posterior probabilities (BPP) are indicated on branches (BS/BPP). Branches without values correspond to BS/BPP = 100/100.

Second model tree (MT2) representing mitogenomic relationships among mammalian families with eight extra polytomies added to MT1 to obtain a tree compatible with recent molecular studies. Bootstrap values (BS) and Bayesian posterior probabilities (BPP) are indicated on branches (BS/BPP). Branches without values correspond to BS/BPP = 100/100.
Consensus and Supertree Methods
Individual datasets
For the consensus and supertree methods, the twelve individual mt genes were analyzed separately. Stationarity of base frequencies across taxa was tested on each of the twelve datasets using the chi-square test of homogeneity of base frequencies implemented in PAUP* 4.0, with a Bonferroni correction for multiple tests. 83 Modeltest 3.7 was used to identify the best substitution model for each dataset. ML analyses were then performed on each dataset with PhyML 3.0, using the model suggested by the AIC criterion. Analytical parameters were identical to those described for the complete matrix analysis, except for the evolutionary model (listed in Table 1 for each gene). Individual gene trees are available upon request.
Statistical description of individual datasets (the twelve genes on the mitochondrial H-strand) and of concatenated datasets (ALL). L (bp): length of the gene in base pairs. No cst: number of constant sites in the alignment. No info: number of informative sites in the alignment. AIC: Model selected according to AIC criterion in Modeltest, which always included parameters G (a gamma distribution of substitution rates) and I (a proportion of invariable sites). χ2 (1, 2): chi square test for homogeneity of base frequencies across species on datasets with third codon position removed (P = 1.0 in every case). χ2 (1, 2, 3): chi square test on datasets with codon positions 1, 2 and 3 included. *Identifies significant values after a Bonferroni correction, P < 0.004 (i.e. 0.05/12).
GTR: General time reversible mode.170–72
TrN: Tamura-Nei model. 111
TVM: Tranversional model. 69
TIM: Transitional model. 69
Given that all twelve datasets included an identical number of taxa (n = 102), the comparison of consensus and supertree methods was performed in a consensus setting. 32 Therefore, even though we will maintain the use of “supertree” for methods that have been developed in a supertree context, all methods can be considered as consensus methods and can be divided into three categories: (1) consensus techniques based on topological relationships (topological consensus methods), (2) supertree techniques based on topological relationships (topological supertree methods), and (3) supertree techniques that take into account branch lengths (branch-length supertree methods).
Topological consensus methods
Four consensus methods were applied to combine the twelve independent gene trees in PAUP* 4.0: (1) strict, (2) majority rule (MR), (3) majority rule with compatible groupings (MRC), and (4) Adams consensus. The strict consensus only retains groups that are identical among all input trees.84,85 The majority rule consensus (MR) contains groups that are present in more than 50% of input trees,11,86 such that groups found in seven or more trees were kept. The second type of majority rule consensus (MRC) retains all compatible groupings below 50% of occurrence in addition to those above 50%. The Adams consensus presents groups that are nested within another without necessarily including identical taxa.87,88 Therefore, Adams consensus does not only propose monophyletic groups. A more complete description of these methods can be found in Swofford. 11
Topological supertree methods
Three different optimality criteria were used to construct supertrees (consensus) from the twelve independent gene trees in CLANN 3.0.2: 89 (1) matrix representation with parsimony (MRP), (2) most similar supertree (MSS), and (3) maximum splits fit (SFIT). In MRP, nodes present in each tree are coded into a binary matrix using the Baum and Ragan method.28–30 The binary matrix is then analyzed with a parsimony algorithm 90 using ten TBR searches and a random starting tree. The MSS method calculates the symmetric differences between each gene tree and the supertree and sums these differences to obtain a supertree score. 91 The optimal supertree is the one with the best score (smallest distance). For the SFIT method, the splits present in each gene tree and a candidate supertree are recorded and the supertree with the maximum split fit (sharing the greatest number of splits) is selected. 89 For MSS and SFIT, a SPR heuristic search using ten repetitions each starting with a different NJ tree was selected to search among all possible supertree topologies (default parameters in CLANN). For all of these methods, a strict consensus was used to combine equally optimal supertrees, if any.
Branch-length supertree methods
Three other optimality criteria, which take into account branch lengths of input trees, were also employed: (4) average consensus (AC), (5) unweighted super-distance matrix (SDM), and (6) weighted super-distance matrix (SDMw). These methods are implemented in the SDM program. 92 The AC criterion optimizes the sum-of-squared distances between each source tree and the consensus tree, by averaging the path-length distance matrices computed from each gene tree and then applying a least-squares algorithm to this average matrix. 93 SDM applies the same criterion, except that path-length distance matrices are first transformed so as to minimize the sum-of-squared distances among them. 92 The weighted version (SDMw) assigns a weight to each tree prior to computing the average matrix, based on the sequence length of the corresponding gene. All supertrees (consensus) were estimated using an ordinary least squares algorithm 94 in PHYLIP 3.68 95 with the FITCH program, using the jumble option (J = 10) which randomizes the input order of species for each run, and with global rearrangements allowed (SPR algorithm).
Distance metrics
Two dissimilarity measures were computed in PAUP* 4.0 to quantify the congruence between model trees (MT1 and MT2) versus consensus and supertrees. The symmetric-difference or partition metric (PM) counts the number of different splits in the trees being compared.96,97 PM was normalized by dividing each value by its maximal possible value (2n–6), where n is 102 taxa. The maximum agreement subtree index (D1) calculates the number of taxa that need to be pruned from the trees to obtain a congruent topology.98–100 Here again, normalized D1 are obtained by dividing each value by its maximum possible value (n–3), where n is 102 taxa. Rohlf's consensus information index (CII) 101 was also calculated on the consensus trees to measure their relative resolution (this index ranges from 0 when the consensus is a bush to 1 when the tree is fully resolved).
Results
Individual datasets
The length (L) of each of the twelve aligned mitochondrial genes varied from 90 to 1803bp, when all three codon positions were included. The homogeneity test of base frequencies indicated that seven out of the twelve datasets were heterogeneous (Table 1). However, when only the two first codon positions were considered, all datasets were homogeneous. Consequently, subsequent analyses were performed using alignments with only the first and second codon positions. The two optimality criteria (hLRTs and AIC) implemented in Modeltest suggested different models for some datasets. Indeed, whereas the hLRTs criterion proposed a GTR model for all datasets, AIC suggested varying models depending on the dataset, as listed in Table 1. The congruence among distance matrix test (CADM) suggested that all twelve datasets were congruent (Friedman's χ 2 = 44341.5, Kendall's W = 0.7175, P = 0.0001).
Topological consensus methods
Important differences were observed between PM and D1 and among topological consensus methods (Table 2). These results may be explained by the fact that some consensus methods were poorly resolved, i.e. CII = 0.02 for strict consensus, 0.10 for MR and 0.18 for Adams, compared to CII ranging from 0.53 to 1.0 for all other methods. The resolution level affects the congruence indices. When comparing a fully resolved tree to a bush, PM will be of 0.5 because only the clades in the fully resolved tree are contributing to the distance. On the other hand, D1, which calculates the number of taxa that have to be pruned from both trees to obtain identical topologies, will exhibit a very big value given that n–2 taxa need to be deleted for both topologies to be compatible. Because the majority rule consensus that included all compatible groupings (MRC) is more resolved than other classical consensus methods (CII = 0.94), it provided the best results and was the closest to model tree topologies (PM = 0.22–0.23 for MT1-MT2). The majority rule consensus (MR) was the second most congruent consensus method (PM = 0.30–0.25 for MT1-MT2), although much less resolved (CII = 0.10), and thus D1 was considerably increased (0.79–0.73 for MT1-MT2, compared to 0.39–0.46 for MRC).
Congruence of phylogenetic trees inferred from consensus and supertree methods (that ignore or consider branch lengths). CII: Rohlf's consensus information index (ranges from 0 to 1; 0 being a bush and 1 a fully resolved tree), PM: partition metric, D1: maximum agreement subtrees. Indices range from 0 to 1. MR: majority rule consensus, MRC: majority rule consensus with compatible groupings, MRP: matrix representation with parsimony, Mss: most similar supertree, SFIT: maximum splits fit, AC: average consensus, SDM: unweighted super-distance matrix, SDMw: weighted super-distance matrix.
Strict consensus of five most parsimonious trees.
Strict consensus of two equally optimal supertrees.
Strict consensus of 184 equally optimal supertrees.
Supertree methods
The topological supertree techniques suggested more than one optimal supertree: 184 (SFIT), five (MRP), and two (MSS) optimal supertrees. These supertrees were combined using a strict consensus supertree, and thus, were not fully resolved (CII = 0.53 to 0.98, compared to a value of 1.0 for the branch-length supertree methods). The least congruent method was MSS (PM = 0.54–0.56 for MT1-MT2; D1 = 0.62–0.60 for MT1-MT2). MRP and SFIT performed well (for MRP: PM = 0.23 for MT1 and MT2 and D1 = 0.47–0.43; and for SFIT: PM = 0.24–0.22 and D1 = 0.51–0.50).
Weighted supertree methods that take branch lengths into account performed relatively well (PM range from 0.22 to 0.30, and D1 ranging from 0.43 to 0.50) and proposed one optimal, fully resolved supertree. AC was slightly more accurate than both SDM and SDMw. The weighted version of SDM (i.e. SDMw) did not improve phylogenetic performance (PM increased slightly while D1 remained the same).
Discussion
A method commonly used in large-scale studies is the construction of supertrees from individual source trees. 14 Supertree methods combine trees that have overlapping taxon sets, whereas consensus methods summarize trees with identical taxon set. Both approaches have been extensively studied.6,18,32,92,102,103 They can be compared and tested in a consensus setting, where identical taxon sets are used.32,103–108
Among the consensus methods, the majority rule with compatible groupings (MRC) was the most congruent to MT1 (and second most congruent to MT2), when compared to all other consensus and supertree methods tested. Criticisms of consensus methods emphasized the poor resolution of consensus trees.9,18,36 However, MRC was well resolved (CII = 0.94), which may explain its performance relative to other topological consensus methods. Through simulations, Bininda-Emonds 32 has also observed that MRC provided the highest accuracy amongst consensus methods.
In line with previous studies, we confirmed that supertree techniques based on topological relationships did not offer a fully resolved consensus tree. In general, most supertree methods gave similar results (except for MSS, see below). This result was surprising, given that numerous studies have proposed that methods accounting for branch lengths should provide more accurate supertrees.106–108 However, Criscuolo et al 92 have shown that MRP (that do not account for branch lengths) and SDM (that do account for branch lengths) were equally accurate at low levels of missing data (i.e. 25% of deleted taxa), and that the benefit of accounting for branch lengths was only revealed at higher levels of missing data (e.g. 75% of deleted taxa). The consensus setting used in this study did not allow the investigation of the effect of missing data and therefore the difference in performance could have been seen in a supertree setting.
MSS was the least accurate of all supertree methods. Creevey and McInerney 89 have compared their MSS approach to the AC technique, but without branch lengths (i.e. by setting all branch lengths equal to one). The better results obtained with AC (and SDM) with respect to MSS, might suggest that branch lengths contain information different from topological relationships, when supertree methods are used. A similar result was also observed by Criscuolo et al. 92 As for the other supertree methods (SFIT and MRP), they provided similar results to techniques that use branch lengths. This result is consistent with studies that have shown that MRP is accurate under certain conditions.92,102,109,110 Through simulations, Bininda-Emonds and Sanderson 102 have observed that MRP provided accuracy values comparable to those obtained from a supermatrix analysis (and that accuracy was slightly increased when a weighted MRP was used). Among the supertree methods that account for branch lengths, SDM outperformed slightly SDMw. The distance matrices are weighted according to sequence lengths in SDMw, with trees inferred from longer sequences contributing more to the “super” distance values. Thus, biases will be amplified if they are associated with longer sequence datasets. This might explain why SDMw might not always provide the optimal solution. Also, AC was slightly more accurate than SDM, in contrast with Criscuolo et al 92 who showed the opposite under all conditions tested. Fitzpatrick et al 109 in a fungal study comparing AC, MRP and the supermatrix approach, reported that AC might be prone to long-branch attraction, but this was not the case here. Supertree methods that account for branch lengths may thus provide additional information, which could help resolve some least-resolved clades.
The results from this study demonstrate that most of the supertree methods tested were highly congruent with both model trees. Interestingly, the majority rule consensus with compatible clades was also highly congruent, which suggest that it represents an accurate and fast approach to summarize information obtained in separate analyses.
Footnotes
Disclosures
This manuscript has been read and approved by all authors. This paper is unique and is not under consideration by any other publication and has not been published elsewhere. The authors and peer reviewers of this paper report no conflicts of interest. The authors confirm that they have permission to reproduce any copyrighted material.
Acknowledgements
For the computational resources, we would like to thank Marie-Hélène Duplain for granting access to the Laboratoire Interfacultaires de Micro-Informatique de l'Université de Montréal, outside opening hours. We would also like to thank Antoine Lapointe and Daniel Stubbs for their programming skills and the RQCHP (Réseau Québécois de Calcul de Haute Performance) for granting access to its HPC facilities for the Bayesian analyses. Also, we greatly appreciated the help of Stéphane Guindon for running some of the ML analyses. This study was supported by a FESP (Faculté des Études Supérieures de l'Université de Montréal) scholarship to VC and by NSERC grant OGP0155251 to FJL.
