Abstract
The human brain is composed of multiple, discrete, functionally specialized regions that are interconnected to form large-scale distributed networks. Using advanced brain-imaging methods and machine-learning analytical approaches, recent studies have demonstrated that regional brain activity during the performance of various cognitive tasks can be accurately predicted from patterns of task-independent brain connectivity. In this review article, we first present evidence for the predictability of brain activity from structural connectivity (i.e., white matter connections) and functional connectivity (i.e., temporally synchronized task-free activations). We then discuss the implications of such predictions to clinical populations, such as patients diagnosed with psychiatric disorders or neurologic diseases, and to the study of brain–behavior associations. We conclude that connectivity may serve as an infrastructure that dictates brain activity, and we pinpoint several open questions and directions for future research.
Keywords
A fundamental premise in neuroscience holds that the function of any component of the brain—whether a single neuron, a neuronal population, or a cortical area—is largely defined by its connections (Friston 2004). It appears that our brains adhere to two basic principles of functional organization: 1) specialization of distinct areas to distinct cognitive functions or mental processes and 2) integration between these specialized areas, which is achieved through neural connectivity (Friston 2004). Over the past decade, investigations of brain connectivity in general and its relations to localized cognitive task activations in particular have offered unique insights into their complementary roles in shaping human behavior.
Connectivity patterns can be measured noninvasively with MRI, with two main approaches providing complementary findings: diffusion MRI exploits the geometrical traits of neuronal connections to reconstruct known anatomic tracts, referred to as structural connectivity, while the fMRI signal can be correlated across the brain to detect functionally defined brain networks, referred to as functional connectivity (Bassett and Sporns 2017). Structural connectivity indices often reflect the probability of connections or the number and length of streamlines among different brain areas. Functional connectivity measurements represent either the temporal synchronization between a predefined region of interest and the rest of the brain or whole-brain networks of areas that share similar patterns of spontaneous fluctuations in activity over time.
Brain connectivity calculated from imaging-derived measurements is commonly independent of specific task requirements. Unlike traditional fMRI studies, which introduce a task and measure the corresponding brain responses, participants in brain connectivity studies are usually not engaged in any particular cognitive function while being scanned. This holds for structural connectivity but importantly also for functional connectivity, which is widely assessed while participants are in an alleged “rest” state. Therefore, it remains an open question what can be learned from distributed patterns of brain connectivity on the localized brain activations that are associated with complex functions and cognition.
One way to address this question exploits machine-learning algorithms, which have revolutionized the analysis of neuroimaging data in the past two decades. Machine learning uses multivariate methods to make predictions about individual subjects and discover patterns in data (Smith and Nichols 2018). Generally, a model can be trained to relate between brain activity and connectivity in one set of participants and tested on another, thus providing predictions of brain activity from connectivity for unseen individuals (Figure 1). These two sets of participants, namely the training set and the test set, should be completely independent to avoid cross-contamination and overfitting of the model. To achieve that, cross-validation is widely used to iteratively divide a single data set into independent training and test subsamples. During training, the model is fed with a set of connectivity features that can be selected by using a priori knowledge or date-driven methods (e.g., principal component analysis). By learning the relations between these features and the observed task-activation maps, the model is able to generate predicted task-activation maps for novel participants based on their connectivity data only. In this article we review the rapidly growing literature on prediction of brain activity from connectivity. We present evidence from structural and fMRI studies that have employed machine-learning approaches to relate between connectivity measures and task-induced brain activity. We then discuss implications both practical (e.g., clinical) and theoretical (e.g., the role of connectivity as an infrastructure that dictates brain activity) and conclude with future directions and open questions (see Figure 2 for a theoretical framework and article overview).
Schematics of a general machine-learning approach for predicting brain activity from structural or functional connectivity. Brain connectivity can be measured noninvasively by using diffusion or functional MRI. Following image processing, structural or functional connectivity features are extracted and fed into a prediction model, trained to relate these features with observed task-activation maps in the training data. Last, the model is applied to unseen individuals to predict their task-activation maps from connectivity data.
Theoretical framework and article overview. Brain connectivity is depicted at the bottom of the pyramid, reflecting an infrastructure for brain activity and higher cognition. The right-hand side highlights topics that are discussed in this article in relation to each pyramid level. First, we review evidence for the predictability of brain activity from structural and functional connectivity. With regard to functional connectivity, in Box 1 we discuss resting-state versus naturalistic paradigms. Then, we consider the applicability of connectome-based prediction of brain activity to clinical populations (i.e., psychiatric disorders and neurologic diseases). At the highest level, we address the contribution of connectivity and connectivity-derived activation maps to the prediction of individual behavioral traits (e.g., cognitive or psychological). In Box 2 we highlight the possible advantages of complex connectome representations for brain–behavior association studies.
Predicting Brain Activity from Structural and Functional Connectivity
Over the past decade, an increasingly growing body of evidence has suggested that individual patterns of task-induced brain activity can be accurately predicted from task-free MRI measurements of brain connectivity. In a pioneering work by Saygin and others (2012), anatomic connectivity was used to predict functional activation to faces in the fusiform gyrus. The authors first computed the structural connection probability of each voxel in the fusiform gyrus with the rest of the brain and then trained a linear regression model to relate between these connectivity patterns and each voxel’s activity to the contrast of faces > scenes. The model was applied to unseen participants, resulting in predicted activations for each fusiform voxel that were strikingly similar to the actual (observed) fMRI activations (Figure 3A, left-hand side).
Predictions of task-activation maps from brain connectivity. The figure demonstrates high similarities between actual (top) and predicted (bottom) task-activation maps for representative participants and task contrasts. (A) Brain activity in response to visual categories (right: faces > objects; left: words > objects) was predicted from patterns of structural connectivity derived from diffusion MRI. Activity is depicted on the inferior surface of the hemisphere, color scaled in standardized units. Adapted with permission from Saygin and colleagues (Saygin and others 2012; Saygin and others 2016). (B) Brain activity in a motor task (left) and a working-memory task (right) was predicted from patterns of functional connectivity derived from resting-state fMRI. Activity is depicted on the lateral surface of the hemisphere, color scaled in z scores. Adapted with permission from Cole and others (2016) and Tavor and others (2016).
These seminal findings point to close associations between brain structure (in the form of connectivity) and function. Specifically, although the locations of face-selective voxels differ across the population, their extrinsic connections vary systematically with function in each individual (Saygin and others 2012). This approach for predicting of functional profiles from structural connectivity was later extended to predict brain activity in response to visual categories other than faces (e.g., bodies, objects, and scenes; Osher and others 2016; Wang and others 2017), as well as higher cognitive functions such as reinforcement learning (Smittenaar and others 2017). Besides prediction per se, it allowed the detection of particular anatomic connections that most strongly predict and therefore possibly define the neural mechanisms underlying specific functions (Osher and others 2016).
More direct evidence for the causal role of connectivity in instructing functional specialization comes from a longitudinal study that followed the structural and functional profiles of children aged 5 to 8 y (Saygin and others 2016). In this work, the location of the functionally defined visual word form area in each 8-year-old could be predicted from that child’s structural connectivity at age 5, suggesting that early-developing white matter connections may guide later functional specialization (Figure 3A, right-hand side).
An extensive analysis of structural disconnections in 1333 stroke cases recently revealed that the patterns of brain areas disconnected by strokes are significantly correlated with task-related fMRI activations derived from meta-analyses (Thiebaut de Schotten and others 2020). The close correspondence between disconnections following stroke and task-associated functional architecture support a mechanistic role of structural connectivity in regional brain function and enabled the development of an atlas mapping cognitive functions onto white matter tracts (Thiebaut de Schotten and others 2020).
Alongside the increased interest in the predictability of brain activity from anatomic connectivity, an emerging line of research focuses on the relations between brain activity and functional connectivity and the predictability of the former from the latter. The first demonstrations of a strong relationship between resting-state functional connectivity and task-evoked brain activity were based on identifying similarities between whole-brain network architectures across rest and tasks (Cole and others 2014; Smith and others 2009). Following these findings, several machine learning–inspired techniques were developed to explore the association between cognitive task activations and task-free network organization. For example, the use of resting-state functional connectivity features reflecting networks of synchronized activity, which were fed into a machine learning–based regression model, has been shown to provide accurate predictions of brain activations across various cognitive tasks (Tavor and others 2016). The model was trained to relate between task-independent predictors and task-evoked activation maps and was subsequently applied to unseen (out of sample) participants to predict their task activations (z score maps). The predicted activation maps were strikingly similar to the actual task-evoked maps, capturing individual differences in activity strength and topology (Figure 3B, right-hand side). The ability to produce such predictions has highlighted the central role of connectivity in the functional organization of the brain.
Using their method referred to as “activity flow mapping,” Cole and others (2016) were able to predict task activations across different cognitive domains by estimating task-evoked activity flow (the spread of activation amplitudes) over resting state–derived functional connectivity networks (Figure 3B, left-hand side). This method employs functional connectivity to map the flow of activity among brain regions and construct empirically derived network models, which simulate the way that task-evoked activations are generated. The task-evoked activation of a given brain region is predicted by summing the activity of all other brain regions, weighted by their connectivity with that region. Successful predictions support the cognitive relevance of resting-state functional networks, showing that a brain region’s response profile during task performance is at least partially governed by its intrinsic connectivity fingerprint (Ito and others 2022; Schultz and others 2022).
During the past few years, several efforts have been made to enhance the prediction of task activity from connectivity by employing more advanced approaches. These include alternative feature extraction routines (Dohmatob and others 2021; Harrison and others 2020; Zheng and others 2022) and modeling algorithms (Cohen and others 2020; Ngo and others 2021; Niu and others 2021; Tobyne and others 2018; Zheng and others 2022), as well as decoding-based methods, transforming spontaneous neural activity into task-related patterns (Liu and others 2022; Schuck and Niv 2019). Another line of investigation focuses on alternative task-free paradigms to calculate functional connectivity measures, other than resting-state fMRI (Box 1).
Is resting state the best state to measure functional connectivity?
Regardless of the choice of experimental paradigm and modeling approach, the predictability of task-evoked activity from task-free measures suggests that individual differences in brain activation are inherent, trait-like features rather than state-dependent features. These findings have had important implications to clinical research and to the study of brain–behavior associations. In the next two sections we first concern expanding the predictive capacity of brain activity from connectivity to clinical populations and then discuss the contribution of predicted task activations to study individual differences in cognitive traits.
Relating Brain Activity and Connectivity in Clinical Populations
The majority of studies predicting brain activity from structural connectivity (Osher and others 2016; Saygin and others 2012) or functional connectivity (Cole and others 2016; Tavor and others 2016) have employed data from healthy participants, such as those provided by the Human Connectome Project (Smith and others 2013) (Figure 4A). Still, one of the most intriguing applications of predicting task activity from connectivity is for neurologic and psychiatric patients. These populations are challenging in terms of data collection (e.g., most likely to suffer difficulties to engage in cognitively demanding tasks; Zhang and others 2021), making them ideal candidates to benefit from the prediction of task-evoked brain activity without the need to actually perform in-scanner tasks. Yet, the applicability of the prediction method in clinical settings is not straightforward, for two reasons: first, clinical imaging data are typically of lower quality than the state-of-the-art data of young healthy adults on which models are commonly developed; second, patients usually exhibit larger interindividual variability in brain activity than healthy controls (Parker Jones and others 2017).
Examples of actual (top) and predicted (bottom) task-activation maps in healthy and clinical populations. (A) Young healthy adults from the Human Connectome Project data set (Smith and others 2013). Brain activity is shown in a language task (MATH vs. STORY contrast). (B) Patients with schizophrenia. Brain activity is shown in a working-memory task (2-BACK vs. 0-BACK contrast). Adapted with permission from Tik and others (2021). (C) Developmental data including children and adolescents 5 to 18 y of age, provided by the Human Connectome Project (Bookheimer and others 2019). Brain activity is shown in an emotion task (FACES vs. SHAPES contrast). Unthresholded brain activity is depicted on the surface representation of the cortex; circles highlight areas of accurate predictions of individual-specific activation patterns.
Several examinations of the potential clinical implication of resting state–based predictions of task activations have focused on presurgical mapping in preparation for neurosurgery (Niu and others 2021; Parker Jones and others 2017). Such presurgical planning commonly involves task fMRI to identify brain areas associated with key cognitive functions, aiming to maximize tissue resection while minimizing postoperative neurologic deficits. In these works, brain activity in a language task (category fluency; Parker Jones and others 2017) or a motor task (hand movement; Niu and others 2021) was successfully predicted from resting-state functional connectivity features in presurgical patients diagnosed with operable brain tumors, temporal lobe epilepsy, or vascular lesions. The ability to infer about the functionality of brain areas from task-free fMRI rather than task fMRI may dramatically benefit the process of presurgical planning, save scanning-related resources, and reduce patients’ inconvenience. Furthermore, task activity in patients (whole-brain statistical maps) could be predicted with a model trained on healthy controls only, making it possible, in theory, to predict activity for additional task domains without the need to acquire task-fMRI training data in patients.
With a similar approach, resting-state functional connectivity extracted from healthy controls was shown to accurately predict individual variability in brain activity during a working memory task in patients with schizophrenia (Tik and others 2021; Figure 4B). As patients who have schizophrenia and psychiatric disorders in general may likely demonstrate incompliance with task fMRI, fatigue, and lack of motivation, the ability to predict their task-induced brain activity from task-free data is of immense value. Still, it should be noted that despite successful predictions in both populations, a model trained on healthy individuals performed better for unseen healthy individuals than for patients. Thus, further research efforts should focus on improving model generalizability across populations to make task-activation prediction truly applicable for clinical use.
On a theoretical perspective, the fact that “transfer learning” (i.e., training the model on healthy participants and testing on patients) was possible suggests that, despite major alterations in brain activity and connectivity in patients diagnosed with schizophrenia, the relations between the two remains unaltered. Investigation of which brain networks mostly contribute to prediction success may gain additional insights into activity-connectivity relations in health and disease (Tik and others 2021).
The notion that regional brain activations may be dictated by distributed processes, reflected in functional connectivity measures, suggests a mechanistic role of connectivity in the aberrations in brain activity that are widely reported in neurologic and psychiatric patients. Taking advantage of the activity flow mapping approach, alterations in brain activity associated with cognitive functions were predicted in patients with schizophrenia (Hearne and others 2021) and patients at risk of Alzheimer disease (Mill and others 2020). Interestingly, modeling the flow of healthy brain activity over resting-state functional connectivity derived from at-risk individuals resulted in predicted activation maps that were more similar to the actual maps of at-risk rather than healthy individuals. This finding supports functional connections as a mechanism underlying Alzheimer-related dysfunction, even in very early stages of the illness (Mill and others 2020).
Because resting-state fMRI does not require patients’ compliance in tasks, it can be applied not only to neurologic, psychiatric, cognitively impaired, or pediatric patients (see Figure 4C for a demonstration of task-activity prediction in children) but even to patients with disorders of consciousness (i.e., minimally conscious state, vegetative state, or coma; Zhang and others 2021). Several recent studies have assessed these populations and discovered unique patterns of resting-state functional connectivity (Demertzi and others 2019) that could be used to differentiate minimally conscious from unresponsive patients (Demertzi and others 2015). Furthermore, activations in a mental imagery task were predicted from resting-state data, discriminating between patients who are capable and uncapable of volitional task performance (Craig and others 2021). Importantly, though, this study did not predict individual task-activation maps but merely the ability or inability to engage in the task, as reflected by the presence or absence of task-induced brain activations. Future work should therefore examine the feasibility of task-activity prediction in patients with disorders of consciousness and its applicability for gaining better insights into the underlying neuropathology of these severe conditions, ultimately contributing to the development of a reliable biomarker of consciousness.
Predicting Brain Activity from Connectivity to Enhance Brain–Behavior Associations
The prediction of individual behavioral phenotypes from neuroimaging data is a rapidly growing field (Chen and others 2022; Dubois and Adolphs 2016; Finn and Bandettini 2021; Finn and others 2015; He and others 2022; Qi and others 2022; Shen and others 2017; Sripada and others 2020; Yeung and others 2022). Using machine-learning approaches, it is possible to map from individual brain activity patterns to the corresponding cognitive or psychological traits (Dubois and Adolphs 2016). For example, individual intelligence scores were accurately predicted from brain activations during a working-memory task (Greene and others 2018; Sripada and others 2020). This task yielded better predictions as compared with other tasks (e.g., motor, emotion) or no tasks (i.e., resting state), suggesting that a cognitively demanding environment may amplify individual differences and thereby improve predictive models of individual traits. However, task fMRI requires careful design, valuable scanner time, and participants’ cooperation to elicit the appropriate neural response.
In a recent work (Gal, Tik, and others 2022), we asked whether predicted task-activation maps, rather than the actual ones, could be used to further predict individual traits, thus offering a “bypass” relating brain to behavior while avoiding the use of time-consuming and often tedious fMRI tasks. We found that task-activation maps predicted from resting-state functional connectivity yielded more accurate predictions of individual traits, including cognitive abilities and personality dimensions, relative to predictions derived from the resting-state connectome directly. Thus, we suggest that the representation of connectivity as predicted task activity may serve as a novel approach for estimation of individual traits from a simple and effortless fMRI scan, without actually performing any task.
It remains an open question why predicted task-activation maps outperform actual maps in estimating individual traits. One possible explanation is that while actual task-activation maps contain the “true” brain activity as well as other transient factors (e.g., residual motion, scanning artifacts, attention level fluctuations), task-activation maps predicted from functional connectivity may capture the stable elements only and minimize the effect of transient factors.
The prediction of individual cognitive traits was further improved by combining multiple predicted task-activation maps rather than a single map (Gal, Tik, and others 2022) or by combining connectivity data acquired at multiple brain states rather than the resting state alone (McCormick and others 2022; see Box 1 for a discussion on alternative brain states to measure functional connectivity). Moreover, behavioral predictions were dramatically enhanced by transforming the resting-state connectome into a task-related connectome, which amplifies behaviorally relevant individual differences (Yoo and others 2022). This connectome-to-connectome transformation offered similar prediction performance with just a third of the number of participants needed when relying on resting-state data alone, thus providing a possible solution to the problem of limited sample sizes in brain–behavior association studies (Box 2). These works (Gal, Tik, and others 2022; Yoo and others 2022) therefore emphasize the potential of task-activity prediction from functional connectivity not only as a goal by itself (e.g., for noncompliant populations as discussed earlier) but also as a means to the goal of predicting individual traits. In other words, predicted brain activity, whether in the form of task-activity maps or task connectomes, may be manifested to reveal information about behavior and cognition on an individual basis.
Can the prediction of brain activity from connectivity aid brain-wide association studies?
Summary and Future Directions
In this article we review recent literature on the relations between brain activity and connectivity and the predictably of the former from the latter, using brain imaging measures and machine-learning approaches. We highlight the promise and potential of task-activity prediction from task-free connectivity for clinical uses such as presurgical planning and for studying the underlying mechanisms of the functional abnormalities in psychiatric and neurologic disorders. We then discuss the application of predicted brain activation to explore individual differences in cognitive and behavioral traits. Overall, current evidence points to a mechanistic role of brain connectivity in cognitive task-induced activations, as well as their behavioral manifestations in the form of cognitive abilities, emotional traits, and so on.
Intriguingly, the causal relationship between brain activity and connectivity may alternatively be reversed such that consistent co-activations during tasks could lead to more consistent shared fluctuations at rest (Gabard-Durnam and others 2016). In other words, brain areas that are independently at first active during task develop to form functional networks that can then be revealed in the absence of task (e.g., functional connectivity networks extracted from resting-state fMRI). This hypothesis is rooted in Hebbian learning principles (“those who fire together wire together”) and may be directly addressed in future studies aiming to delineate the direction of causality between brain activity and connectivity.
Regardless, studying brain connectivity seems to provide a window into brain function and cognition. This holds great potential for developing connectivity-based disease biomarkers. For example, based on the finding that predicted task-activity maps can be utilized to predict cognitive abilities (Gal, Coldham, and others 2022; Gal, Tik, and others 2022), future research may investigate the possibility of these maps to be used as a biomarker for cognitive decline. This may offer an unprecedented opportunity for early diagnosis of neurodegenerative disorders from a simple and effortless fMRI scan, while minimizing patients’ inconvenience and without them having to perform any cognitive task.
Both structural and functional connectivity has been shown to successfully predict task-induced brain activity. This raises the question whether structural and functional connectivity predicts unique variance in task activation and whether one modality may be more useful for prediction than the other. In a previous work (Tavor and others 2016), functional connectivity features and microstructural properties were incorporated into the prediction model. The microstructural properties included fractional anisotropy and the dot product between the principal diffusion direction and the cortical surface, which may be considered proxies of structural connectivity. Although structural features were exploited by the model, removing them did not affect prediction performance, suggesting that functional connectivity alone is sufficient to predict individual variability in task maps. Importantly, however, these structural features are not direct measures of structural connectivity, such as streamline count and average streamline length. Thus, a direct comparison of the predictability of task-activation maps from functional versus structural connectivity is yet to be performed. Interestingly, such a comparison was recently reported in the context of behavioral trait predictions (Qi and others 2022) showing that functional connectivity-based models gave the best prediction performance across all models and traits, especially in the cognitive domain. The authors suggested that behaviorally relevant information in structural and diffusion features might reflect a subset of the variance captured by functional connectivity.
Notably, there is an abundance of free parameters and methodological decisions to be made by researchers investigating the prediction of task-activity from task-free fMRI. A systematic comparison is lacking among all aspects of the prediction pipeline (i.e., input data, preprocessing, feature extraction, and prediction algorithms); thus, future work is still needed to determine an optimal prediction regime. Specifically, prediction performance is highly dependent on the preselection of connectivity features that are fed into the model. Features can be extracted with a data-driven, unsupervised approach such as principal component analysis or, alternatively, based on a priori knowledge of the target variable, such as which regions are involved during execution of the task to be predicted. Some machine-learning algorithms, such as LASSO or Elastic Net, automatically drop variables to leave just the most important ones (Smith and Nichols 2018). Thus, the selection of features and choice of algorithm add to the complexity of connectivity-based predictions of brain activity.
Additional open questions concern 1) the minimum data requirements for successful prediction (e.g., number of participants in the training and test sets, number of fMRI time points, acquisition parameters affecting data quality); 2) the influence of the content of naturalistic movie watching during scanning on subsequent prediction performance; 3) the potential of combining multiple input data types (e.g., functional and structural connectivity) to improve predictions; and 4) the generalizability of predictions across data sets, sites, or populations (i.e., training a model on one data set and testing on another). These lines of investigation should guide future experimental choices, such as what data to acquire (resting-state or naturalistic stimuli? what stimuli?), which features to select (how to determine their importance), and what model to fit. In fact, the answers to these questions most likely differ according to the experimental goal, prediction target, study population, data size and quality, and so on.
Besides improving prediction performance, further research is required to investigate what information in task-free connectivity characterizes the individual variations in task activity in a way that makes prediction possible. These could be “structural” features, driven from brain organization and connectivity, or “functional” features related to participants’ cognitive or emotional state during the resting-state scan. In any case, the ability to infer individualized functional patterns from task-free fMRI suggests that individual variability in brain activity may stem from interindividual differences in the functional connectome.
As computational neuroimaging continues to develop and “big data” consortia are collected worldwide, the prediction of brain activity is expected to become more and more robust and applicable in the upcoming years. By combining human neuroimaging with advanced artificial intelligence, this rapidly growing field offers not only practical implications but also exciting novel insights into the relations between brain activity and connectivity and the neural basis of human behavior.
Footnotes
Declaration of Conflicting Interests
The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was supported by the Israel Science Foundation (grant 1603/18).
