Review of Psychological Statistics and Psychometrics Using Stata,by Scott A. Baldwin

Abstract

In this article, I review Psychological Statistics and Psychometrics Using Stata, by Scott A. Baldwin (2019, Stata Press).

1 Introduction

When I first started working for the UCLA Statistical Consulting Group, the website for which the group is so well known (https://stats.idre.ucla.edu/) was just being created. Much of the introductory material was spread across the site. When statistical consulting clients would come to the office for assistance, I would first show them where they could find the webpages that introduced Stata, then the webpages that introduced the statistical technique to be used, and finally some webpages that discussed the interpretation of the output. It soon became clear that this information needed to be consolidated into one place, as clients would often get lost while seeking the materials they needed. Those memories kept coming back to me as I was reading Psychological Statistics and Psychometrics Using Stata. Scott Baldwin (2019) has done an excellent job of putting into one book much of what data analysts need: a brief introduction to Stata, followed by a logical progression of analyses from simple to complex, with just enough theory to understand the concepts, and complete explanations of the Stata syntax and the output.

2 Content, strengths, and weaknesses

Chapters 1 and 2 are introductory chapters. Chapter 1 highlights the many advantages of using Stata. It also contains a nice discussion of some of the problems currently facing science and scientific writing, particularly with respect to p-values. This is a timely discussion, and Baldwin emphasizes the need for the data analyst to really think about each step of the research and analysis process, rather than simply following “the rules”. Chapter 2 gives a brief overview of the use of Stata. Baldwin gives suggestions for the organization of files to improve workflow and stresses the need for reproducible analyses.

Chapters 3 through 10 focus on various ways to analyze the relationships between variables. This part of the book is written in a way that not only helps readers learn the material for the first time but also is a good refresher for those who have forgotten key concepts or need clarification on a specific point.

Chapter 3 discusses linear regression with continuous predictors. Simulations are introduced in this chapter as a way to understand the concepts. This is a great idea: it allows data analysts to become familiar with the idea of simulations and how to run them in a context where most will already know the answer. These examples also set up the basics of programming simulations for later chapters, when they are used to address other types of questions. Baldwin covers the theory well and makes good use of graphs to illustrate the concepts. He also introduces several of the commands that make Stata such a powerful data analysis program, including the margins and lincom commands and the coeflegend option of the regress command. I particularly like that Baldwin shows all the graph code necessary to make the graphs ready for presentation because I find this to be an effective method for teaching the options needed for such graphs. In fact, it reminds me of Michael Mitchell’s (2012) book, A Visual Guide to Stata Graphics.

Chapter 4 covers the use of both continuous and categorical predictors. Most of the material focuses on the use of contrast variables, specifically, using dummy coding. Given the practical nature of the book, this is not a surprise; however, given the title of the book, it is a bit surprising that Baldwin neglects to mention effect coding. His example of a publication in which the authors failed to use contrast variables when needed (that is, treating a nominal variable as if it were continuous) is an eye-opening reminder not only of the need to learn how to correctly include nominal variables in regression models but also of the fallibility of the peer-review process. Chapter 5 is a short chapter that covers the t test and one-way ANOVA and sets up chapter 6, which covers factorial ANOVA. The many examples of the use of the contrast command and its options help data analysts see how this useful command can address many different types of research questions.

Chapter 7 introduces the analysis of repeated-measures data. All the example analyses are done with the data in long form using the mixed command. While I understand the rationale for this approach, I feel an opportunity is lost to show how to run such analyses with the data in wide form. It also seems very “un-psychology-like”: training students how to analyze repeated-measures data in wide form via a repeated-measures ANOVA is still a staple of many graduate psychology programs. On the other hand, there is a thoughtful section on the advantages of analyzing repeated-measures data using a multilevel model rather than a traditional repeated-measures ANOVA. This choice of analysis is in keeping with the theme of showing what is currently being done by researchers, rather than blindly adhering to traditional psychological teachings. I also find the discussion of covariance structures to be clear and helpful, which is a rare treat. The chapter finishes with a nice explanation of two important topics: degrees of freedom and why it is difficult to calculate in a multilevel model with unbalanced data, and heteroskedastic standard errors (which, in my opinion, should probably be used more than they are currently).

To me, chapter 8, which introduces power and sample-size calculations, feels out of place, coming as it does between two chapters that focus on the use of the mixed command. Nonetheless, it is a very good introduction to the subject. Like in the other chapters in this book, Baldwin does an excellent job of balancing the necessary theoretical discussion with examples and clear explanations of the Stata syntax needed to address the question at hand. He provides examples of both simulations and the power command. I particularly like the coverage of type M and S errors, which I do not think get enough attention. However, I wish Baldwin had included more examples of calculating power for other types of analyses besides z tests, t tests, and ANOVAs, especially via simulation. I also expected some discussion of power analyses for repeated-measures data, because that is the topic of the previous chapter.

Chapter 9 resumes the discussion of the use of the mixed command, this time for analyzing cross-sectional data. The chapter starts with a helpful discussion of the statistical and conceptual issues associated with clustered data. Too often, data analysts are told that clustering needs to be modeled but not told why that is necessary. Emphasis in this chapter is on interpretation of the output rather than attempting to cover the many different topics that are associated with multilevel models, such as centering, use of maximum likelihood versus restricted maximum likelihood, and model diagnostics. The discussion of Bayesian analyses is a welcome surprise. As in previous chapters, the use of graphs to illustrate concepts is great and so is the section on between- and within-cluster relationships.

Chapter 10 is what I expected to follow chapter 6, because it covers multilevel models for longitudinal data. As with the previous chapter, there are many topics that could have been covered, but Baldwin keeps it to the basics. He provides a solid foundation that makes it likely that data analysts will be able to assimilate knowledge from other sources with relative ease. The last part of the book is about measurement. On page 315, Baldwin quotes Andrew Gelman at https://statmodeling.stat.columbia.edu/2015/04/28/whats-important-thing-statistics-thats-not-textbooks/: “For my money, the #1 neglected topic in statistics is measurement.” (See also Gelman, Hill, and Vehtari [2021, chap. 2].) I could not agree more. Throughout these chapters, Baldwin emphasizes that decisions regarding model development are not clear-cut and should be based on a variety of criteria, not just a single test. However, I wish some attention had been given to the question of sample size in these chapters, as well as a discussion of how p-values are used (or not used) in the decision-making process of model development.

Chapter 11 uses confirmatory factor analysis to investigate issues of reliability. Only one-factor models are considered. This chapter follows the now-familiar pattern of theory, code, and explanation of output. There is a short but wonderful section on how to write the commands for bootstrapped standard errors. I especially like these types of examples because I feel that many data analysts want to do this but do not know how. I hope having a clear example to follow will increase their use in the published literature.

Chapter 12 uses exploratory factor analysis to investigate factor validity. Again, Baldwin covers the basics well and leaves deep discussions about a variety of related topics to other, well-chosen sources. This emphasis on the basics not only helps to keep the discussion focused; it also gives the analyst a sense of mastery of the topic (or at least the basics of the topic) and leaves them ready to learn about related topics when needed. I also appreciate the discussion of how the output from exploratory factor analysis is labeled differently in other statistical software programs, because I have found that difference in labeling to be a source of confusion for many data analysts.

Chapter 13 provides a brief introduction to the topic of measurement invariance, and it is the only chapter that I felt was rushed. This is, by far, the most technical chapter in the book. Still, Baldwin does a good job of explaining the logic of what needs to be done and provides examples with clear explanations of the syntax used.

3 Conclusion

This book deserves a prime spot on the shelf of most data analysts from a wide variety of disciplines. It is well written with an admirable balance of theory, practical examples, and explanations of both syntax and results. I do wish the title was a little more inclusive, because many data analysts trained in departments other than psychology will benefit greatly from this book. I also wish there was a chapter on models with noncontinuous outcomes. When I think of the types of models that are commonly run to investigate relationships between variables, that is the only major category missing from this book. Baldwin is clearly a gifted teacher and writer, and I hope that he will write more books.