Abstract
There are many ways in which researchers ask participants about their personal goals or projects, yet findings are subsequently considered interchangeable. This study experimentally tested whether different ways of asking participants about their goals elicits different goals and impacts reports of goal progress. Undergraduate participants (N = 285) were assigned to one of three conditions (personal projects, personal goals, open-ended goals), listed an unlimited number of goals they were currently pursing, rated each goal on a series of goal characteristics, and six weeks later reported on their goal progress. Results indicated that participants reported significantly more goals in the personal project condition than in the other two conditions, and that these goals were rated as less difficult. Overall, the present study provides further insight into the effects of the elicitation methods employed in goal pursuit research.
Relevance Statement
Are different ways of eliciting personal goals equivalent? This experiment shows we can draw similar conclusions from studies that ask about goals in different ways, which can help build a unified field of research on personal goal pursuit.
Key Insights
We manipulated how participants were asked about personal goals. Multiple goal characteristics were assessed for each goal. Different methodologies did not elicit very different goals. Similar conclusions can be drawn from studies with different operationalizations.
Pursuing personal goals is a ubiquitous aspect of human nature, so it is no wonder that psychology researchers have investigated how goal pursuit occurs. But to what extent do the ways that researchers study goal pursuit shape the answers they obtain? Past research has used various definitions of personal goals, along with numerous constructs that are similar to goals (e.g., personal projects, personal strivings). Conclusions from studies using these distinct constructs have been used to generalize to goal pursuit more broadly, assuming that the specific definition provided to participants, or the way that the list of personal goals is elicited, does not matter. We test this assumption directly by experimentally examining whether different ways of asking participants about their goals would elicit different goals and impact reports of goal progress.
Goal Elicitation and the Jangle Problem
Research on personal goals and other constructs that can be considered similar to goals, such as personal projects (Little, 1983), has led to a potential jangle fallacy (Kelley, 1927), which refers to the erroneous belief that constructs are theoretically different just because they have different names. Goals have been defined as “a cognitive representation of a desired end state that a person is committed to attain” (Milyavskaya & Werner, 2018, p. 163). Personal projects, on the other hand, are defined as “a set of interrelated acts extending over time, which is intended to maintain or attain a state of affairs foreseen by the individual” (Little, 1983, p. 276). Based on these definitions, goals are what people
Beyond the conceptual definitions, many similarities can be found in the methods used to study these constructs. In both cases, participants are asked to think of the personal projects or personal goals that they are currently pursuing, engaged in, or are planning to pursue, and are asked to name between 3–15 goals/projects (e.g., Koestner et al., 2002; Little & Gee, 2007). Each goal or project is then rated on several characteristics. Indeed, some researchers use personal projects methodology to examine what they term personal goals (e.g., Salmela-Aro et al., 2007; Sheldon & Kasser, 1998). For example, Sheldon and Kasser (1998) report using Little's (1983) personal projects methodology to elicit goals for a study on personal goals and goal progress. In some cases, researchers do not even report the exact definition or wording that is used to elicit goals from participants, stating simply that participants were asked to name a certain number of personal goals (e.g., Riediger & Freund, 2004). Similarly, when reviewing past literature, authors typically combine findings from studies using personal project definitions with goal definitions (and include both in meta-analyses, e.g., Klug & Maier, 2015). There is currently no evidence, however, that supports such interchangeable use of these constructs. Is it the case that goals and personal projects, despite having different names and representing theoretically different constructs, are actually perceived in the same way by study participants? Or does the way researchers elicit goals from participants impact participants’ answers, and ultimately the conclusions that can be drawn from the research?
Furthermore, both definitions of goals and personal projects are specific definitions derived from researchers’ conceptualizations of goals. But how does this match up with what laypeople think about when asked about their goals? Do people spontaneously think of lofty, long-term goals? Or do they think about what they have to do in the near term? People's lay understanding of what is meant by goals is likely to color their answers to this question, but this has not been investigated. Specifically, does the detailed description frequently used in research on personal goals result in people providing a different set of goals than what they would otherwise think of and pursue in their daily life? We know that simply setting goals can lead to greater progress on those goals (Norcross et al., 2002); are studies where researchers provide a detailed description of goals inherently manipulating something by leading participants to write down more goals, or different goals that they would otherwise be thinking about, thereby getting them to ‘set’ new goals and increasing goal attainment? If research on goal pursuit hopes to be relevant to people's actual day-to-day goal pursuit, it needs to ensure that the way researchers ask about goals does not inadvertently lead participants to set more, or different goals than they would have otherwise.
Goal Characteristics
Each goal that a person pursues can be described along several dimensions that can characterize the nature of the goal (Austin & Vancouver, 1996). Here, we focused on four that have been consistently linked to goal progress, and thus are likely to be of particular interest to goal researchers: difficulty, specificity, commitment, and motivational quality.
Goal difficulty represents the standard of attainment, or how much effort would be required for the individual to accomplish the goal (Locke & Latham, 1990). Relatedly, goal specificity corresponds to the level of abstraction of a goal (Carver & Scheier, 1982). Specific goals (sometimes referred to as concrete goals; Fujita & MacGregor, 2012), in contrast to broad/abstract goals, frequently have an identifiable end-point, and require relatively fewer, concrete actions to attain. Personal projects have been proposed to be more context-dependent than personal goals, and focus specifically on actions carried out in the service of goals (Little, 2007, p. 37). If this theoretical distinction exists, participants should view their personal projects as more concrete and grounded than the likely loftier and aspirational goals, which may be less specific and/or more difficult. In a similar vein, participants asked about personal projects should indicate more projects (since they are expected to be smaller, more specific actions that are easier to accomplish).
Commitment can be defined as determination (Locke & Latham, 1990), or “how long an individual is willing to strive for a specific goal” (Austin & Vancouver, 1996). Some level of commitment distinguishes goals from fantasies or wishes (Gollwitzer, 1990). And greater goal commitment has been linked to greater effort, persistence, and performance (Koestner et al., 2002). Given that commitment is an integral aspect of goals, we expected that when participants are asked to spontaneously think of personal goals (without any definitions or further prompts), they would be more likely to think of those goals to which they are most committed, such that commitment would be highest in that condition (compared to, for example, thinking about all the personal projects that they are currently pursuing).
Finally, motivational quality refers to the reasons
In addition to these four characteristics, we examined goal progress and attainment. Progress concerns the amount of movement towards the desired end state (Carver & Scheier, 1982), whereas attainment is a dichotomous judgment of whether the goal was accomplished; this allows us to compute the number of goals that have been attained over a specific time period. If participants setting open-ended goals are indeed setting more abstract, difficult goals, they could be expected to attain fewer of these goals, despite subjectively feeling that they are generally making good progress on their goals. In contrast, personal projects, which focus on smaller, in-the-moment concerns, may elicit goals that can be accomplished over a shorter time frame, such that more of them could be completed within the six weeks of the current study.
Present Study
The present study examined whether instructions used to elicit goals/personal projects affect (1) the number of goals participants report; (2) the characteristics of the goals they report; and (3) subsequent self-reported progress on those goals. We hypothesized that asking about personal projects would elicit more goals and result in greater goal attainment, but that asking about goals in an open-ended manner would result in less specific, more difficult goals, that are greater in commitment and autonomous motivation. We had also pre-registered (https://osf.io/fjm7p) a secondary aim of this project: to examine to what extent self-reported Big Five personality traits correlated with individual differences in reports of goal characteristics and goal progress. Because this is beyond the scope of the current work (which is concerned with elicitation methods), we present these findings in the Supplementary Materials.
All hypotheses and planned analyses were pre-registered after data from wave 1 was collected, but before any of the data was cleaned or analyzed (see https://osf.io/fjm7p). Given the likelihood of null effects (i.e., it may not matter how we ask about goals), we supplemented frequentist analyses with Bayesian analyses that allow us to specify our confidence in the effects (null or otherwise). We report how we determined our sample size, all data exclusions, all manipulations, and all measures (either in the text or on OSF). All materials, data, syntax, and full output from the analyses can be found in the Supplementary Materials
1
Another (submitted) manuscript uses data from this study to examine the relation between depressive symptoms, goal characteristics, and goal attainment; there is no overlap in the results reported between the two manuscripts.
Method
Participants and Procedure
Ethical approval was obtained from the university REB. Participants were 285 undergraduate students (
Participants came into the lab and were randomly assigned to one of three goal elicitation conditions. They received an instructions sheet asking them to name all their current goals/personal projects in one of three ways (see the Supplementary Materials for exact wording):
Using the personal projects methodology, including a definition of personal projects (from Little & Gee, 2007).
Asking about personal goals, with a definition most commonly used in the personal goal literature (e.g., Koestner et al., 2008; Werner et al., 2016).
In an open-ended manner, without providing a definition of goals (“Everyone pursues goals. Please think of the goals you are currently pursuing and write them below.”).
A blank sheet was provided to participants to write down their goals/projects (there were no lines on the sheet, to avoid inadvertently suggesting to participants how many they ‘should’ be writing based on the number of blank spaces). After manually writing down all their goals/projects, participants entered their goals/projects one at a time into a computerized survey, rated each goal/project on several characteristics, and completed personality questionnaires (see OSF for full survey). Six weeks later, in an optional online survey, 122 participants (42%) reported their progress and attainment for each of the goals/projects
2
This follow-up was sent out in February–April 2020; the university was shut down for in-person classes due to COVID-19 on March 12, 2020. This likely contributed to the low response rate. To examine whether goal progress or attainment was impacted by COVID-related changes, we tested for differences in goal progress and attainment for those who responded before the date the university made the announcement (pre-COVID), and those who responded on or after that date(post-COVID). We also conducted a second analysis using a week after the announcement as the pre/post cut-off point. In both cases, there were (surprisingly) no differences in progress between those who completed the follow-up pre-COVID and post-COVID. We thus proceeded with the planned analyses of progress.
Measures
Number of Goals
Participants were given a blank page to write down as many personal goals/projects as they wanted. They were then asked to count the number of goals written and enter this number into the electronic survey; this was checked manually by research assistants, and when there was a discrepancy, the number written on the paper prevailed.
Goal Characteristics
For each nominated goal/project, participants rated each of the following characteristics using one face-valid item: specificity (“How would you define the current goal/project you are pursuing?” 1 =
Goal Progress and Attainment
This was assessed at the online follow-up 6 weeks after the initial in-lab study, and measured in two ways. First, for each goal/project, participants reported goal progress on one item (“I have made a lot of progress toward this goal”), rated on a 1 (
Results
Descriptive Statistics
After removing the three outliers who listed more than 20 goals, the average number of goals amongst participants was 7.5 (
Participants also reported being very committed to their goals. Motivation for goals was relatively more autonomous than controlled. To examine variability across goals, we computed the intraclass correlation coefficients (
Descriptive Statistics and Pearson Correlations With 95% Confidence Intervals of All Measured Study Variables (Collapsed Across Condition)
aHigher numbers = less specific (more broad).
*
Effects of Goal Elicitation
We first examined differences across the three conditions on the number of goals that participants set (see Table 2 for all comparisons across condition). As hypothesized, participants reported more goals in the personal projects condition than in the open-ended or personal goal conditions. A series of ANOVAs
3
Although we preregistered the use of a MANOVA, assumption of linear relationships among dependent variables were not met.
Differences in Key Goal Variables Across Elicitation Conditions
To check the robustness of these findings we conducted follow-up Bayesian analyses using JASP software
4
See Supplementary Materials for a detailed description of these analyses.
We next examined the variability of goal characteristics across the goals, both by looking at the intraclass correlations (
Additional Exploratory Analyses
Finally, in exploratory (non pre-registered) analyses, we examined whether responses are different for the first three or first four reported goals, compared to the average from all the goals reported, on each goal characteristic and goal attainment (using paired t-tests). This was done to examine the potential effects of asking participants to only list three or four goals, as is common in many studies of goal pursuit (e.g., Holding et al., 2019; Milyavskaya et al., 2015). As seen in Table 3, there were differences in specificity, commitment, controlled motivation, and progress. Supplementary analyses found no interaction with condition (except for commitment) – that is, the first few goals were different from the average of all the goals in the same way across conditions. Overall, these analyses showed that the first few goals that a person reports are more specific, higher on commitment, more controlled (but not more or less autonomous), and slightly more difficult (only for the first three, not the first four). People also report (slightly) more progress on the first few goals
5
Additional analyses comparing the first 3 (or 4) goals to the remainder of the goals (excluding the first 3 or 4) show similar results; see Supplementary Materials.
Differences in Goal Ratings Between all the Goals Compared to Either the First Three or First Four Goals (From Repeated-Samples t-Tests)
Discussion
The present study examined whether the way that goals are elicited makes a difference in the number and characteristics of goals that are reported, and in subsequent goal progress. In line with our hypothesis, we found that asking about personal projects led participants to report more goals than asking about personal goals using either the specific or open-ended formulation. These goals were also rated as less difficult, and participants in the personal project condition reported more commitment for their goals than the detailed goals condition (but not the open-ended goals). There were no other differences across conditions. Contrary to our hypotheses, the open-ended goal condition did not elicit less specific goals, and participants did not report more autonomous motivation or greater commitment for these goals. There were also no differences across condition in goal progress or goal attainment. Further Bayesian analyses suggested moderate support for the null hypotheses. Altogether, these findings suggest that these elicitation methods appear to result in mostly similar goals on most characteristics (other than difficulty). Unless difficulty or number of goals are of particular interest, research can likely continue to combine findings and draw conclusions across studies using different elicitations methodologies.
Overall, we found that participants reported a greater number of goals, which they rated as less difficult, in the personal projects condition. This fits with the theoretical understanding of personal projects as something that people are currently working on, which may include daily tasks that would not typically qualify as goals. It may also be that providing examples, and a longer definition, led participants to think more broadly about their day-to-day pursuits and record these as personal projects, even though they would not normally meet the definition of goals. We also did not examine the timescale of the goals that participants reported – that is, one key difference may have been in the extent to which participants in the different conditions set short-term versus long-term goals. However, given that more abstract goals typically take longer to attain, and conversely shorter-term goals could be expected to be more specific, combined with the lack of differences in specificity found the current study, it suggests that personal goals and personal projects do not elicit goals that are pursued on different timescales. Future research, however, needs to examine this more closely.
There were no differences between providing a detailed definition of goals and simply asking participants to think about their personal goals, which suggests that the detailed definition approximates what people generally think of goals. This is encouraging, as researchers’ definitions of a construct do not necessarily parallel lay understandings of this construct. In the case of goals, however, it seems that the definition frequently provided by researchers leads people (or at least undergraduate students) to think of the same thing as if they were simply asked to think of goals more generally. Note, however, that since the Bayes analyses showed only weak to moderate support for the null hypothesis (that there are truly no differences across conditions), further research should obtain additional evidence until a strong conclusion can be drawn, preferably with data from different populations.
One key difference across the conditions was the length of the descriptions of the goals provided to participants, with 196 words (including concrete examples) in the personal project condition, 92 words in the personal goals condition, and 41 words in the open answer condition. Although this may at first appear to be a confound, these descriptions represent the actual way in which these types of goals are typically elicited in the literature; any attempts at making the description the same length would be a departure from the elicitation procedure that is typically used to ask about that type of goal. Since the main aim of our study was to compare existing elicitation procedures, we chose to use existing wording rather than artificially shorten or lengthen it to equate them on length. Our results, showing that there are very few differences in the goals provided in response to these conditions, suggests that length of the text may not matter much either.
In exploratory analyses examining differences between variables computed across all goals and the first three or four goals that a person reported, we did find some differences. This suggests that contrary to the personal project elicitation, which typically asks for 10–15 projects (Little & Gee, 2007), asking about three or four goals as is typically done in goal research (e.g., Holding et al., 2017, 2019; Werner et al., 2016) results in a focus on goals that are particularly important and more likely to be attained. This is potentially important for generalizing to goal pursuit more broadly. However, the differences were relatively small (less than 0.2 on a 1–7 scale); it remains to be seen whether these differences actually affect inferences that can be drawn from research using different elicitations. Additionally, the average number of goals that participants indicated was 7.5, suggesting that the practice of asking participants to list 10–15 projects may artificially lead participants to report goals that they do not truly care about. The average number of goals reported in our study is similar to other research where participants were allowed to indicate as many or as few idiographic goals as they wanted (Monzani et al., 2015).
The present research focused on three different elicitations; however, researchers have conceptualized other similar constructs including personal concerns (Klinger & Cox, 2004), life tasks (Cantor et al., 1987), and personal strivings (Emmons, 1992). Research focusing on these distinct constructs all elicit and assess goals in slightly different ways (Lloyd et al., 2019; for a review see Negru, 2011). The present study, however, suggests that for at least some of these, there are limited differences in the resulting data. It may thus be that despite conceptual and methodological differences, a continued reliance on these different constructs and terminology only serves to fracture the literature, promulgating a further jingle jangle problem in research that ultimately assesses practically indistinguishable constructs. More efforts are needed to theoretically and empirically integrate these various constructs to arrive at a better understanding of goal pursuit.
Limitations
This study was conducted with first and second-year undergraduate students on a university campus in a western country; our sample is thus young and predominantly female (76%), white (57%), and educated. It is thus unknown if these results would generalize to other populations. Much of the research on goal pursuit has been conducted with undergraduates (Koestner et al., 2008; Milyavskaya et al., 2015; Sheldon & Elliot, 1999), although there is also a sizeable literature on goal pursuit in adolescents (Massey et al., 2008) and older adults (e.g., Bailly et al., 2012; Riediger et al., 2005). A second limitation is the high dropout rate (58%) and the timing of the follow-up, which coincided with closures of the university and of new restrictions due to the COVID-19 pandemic. This may have affected the drop-out rate and very likely altered goal pursuit (see Footnote 1). Future research could examine how goal pursuit changes in the face of unforeseen or unusual circumstances. In this study, the results for goal progress and attainment should thus be interpreted with caution. Furthermore, goal progress was only assessed at the follow-up, and not at baseline, so that we did not examine change in progress over time. Another limitation of the research concerns the limited number of goal characteristics examined in the present study. It may be that our conditions did elicit goals that differed in other ways that were not assessed here, such as time scale, urgency, or enjoyment.
A final limitation is that all of the goal characteristics were assessed with single items (one per characteristic for each goal), and aggregated across goals. Although this is the standard for research in both personal projects and goal pursuit research (Little & Gee, 2007; Sheldon & Kasser, 1998), and the items are all face valid, this is still an issue as far as measurement is concerned. Perhaps more importantly, since most of the variation in goal characteristics was on the goal level, it is unclear how useful or meaningful it is to aggregate them across goals (see Lüdtke & Trautwein, 2007). That is, although personal project research explicitly treats such projects as a level of personality (Little, 1989), and extensive research has examined goal pursuit as an individual difference (e.g., Sheldon & Elliot, 1999), goal pursuit is first and foremost goal-specific (see Milyavskaya & Werner, 2018). Aggregating such goal-specific characteristics on a between-person level can result in unreliable measurement (since there is little shared variance on the between-person level) among items. Increasing the number of items per characteristic, however, would not help with this problem (although assessing more goals can; Lüdtke & Trautwein, 2007). Overall, attempting to investigate individual differences, while forgetting the extensive within-person variability, can hamper efforts that seek to better understand personal goal pursuit.
Conclusions
This study found that although using the personal project methodology elicited more goals and less difficult goals than asking about personal goals, there were no differences in other goal characteristics. This suggests that how participants are asked about their goals may not matter very much, particularly in studies that only ask participants about three or four goals. Contrary to our hypotheses, results also suggest that both personal goal and personal project definitions align with lay-person views of goals, as both lead to similar reported goals. This suggests that researchers can probably draw similar conclusions from studies that ask about goal pursuit in different ways, providing further confidence in our ability to build a more unified field of research on personal goal pursuit.
Footnotes
Appendix
This research was supported by funding from the Social Sciences and Humanities Research Council of Canada (SSHRC) to Marina Milyavskaya.
Data Availability
For this article, data is freely available (see the Index of Supplementary Materials section).
Supplementary Materials
For this article, the following Supplementary Materials are available (for access see Index of Supplementary Materials below)
via the OSF repository
via the PsychArchives repository
Ethics Approval
This research was approved by the Research Ethics Board at Carleton University.
Author Note
Last three authors (KC, CR, and JY) contributed equally to this manuscript; order of authorship for last three authors is alphabetical.
The authors have declared that no competing interests exist.
Acknowledgements
The authors have no additional (i.e., non-financial) support to report.