Abstract
In this article, I review The Mata Book: A Book for Serious Programmers and Those Who Want to Be, by William Gould (2018, Stata Press).
Keywords
Mata is a modern self-compiling and platform-independent matrix-programming language that was released by StataCorp as part of Stata 9 in April 2005. As William Gould writes in his book, Mata is a serious programming language. Its code looks similar to code from other serious programming languages, such as C or C++, and it features powerful programming concepts, such as structures, classes, and pointers. Furthermore, Mata is compiled, which means that it executes much faster than a script language, such as Stata’s ado-code. 1 Despite being a serious programming language, Mata is not very difficult to learn and can be put into good practice by applied researchers who do not consider themselves to be “programmers”. Personally, I believe that most Stata users could profit from a bit of Mata knowledge. For example, Mata is also a great tool for processing text, not only for statistical computations. The way Mata is documented, however, may have set the bar too high for many users—at least until now.
Stata comes with a reference manual for Mata that provides all the details about Mata’s syntax and the various predefined Mata functions. This manual, however, is more like a lexicographic dictionary and contains little overarching material providing guidance to applied researchers on how to use Mata in practice and how to best approach specific programming problems. Although it is perfectly possible to learn Mata by using the manual only—at least for those who have some minimal programming experience— a didactically well-structured and problem-oriented textbook on Mata was definitely missing.
Though it took 13 years for this much needed book to surface, it finally has, and even better, it was written by the person who has the most inside knowledge about Mata: William Gould. The book’s language and explanations are in typical Gould style, well known to regular readers of his contributions to the Stata Journal, Statalist, or Stata Blog. The examples are didactically well structured and walk the reader through specific problems step by step without relying on much background knowledge—and the language is entertaining. “Can there be anything more boring than a textbook on a programming language?”, you might ask yourself. Believe it or not, I had a lot of fun reading this book. Admittedly, this might be due to my personal d´eformation professionnelle, but I am convinced that much of the enjoyment was also due to Gould’s style of exposition, which is highly instructive and entertaining at the same time. I hope that others will have a similar experience and that the book will make Mata more accessible to a wide range of Stata users.
So what is in the book? It has about 420 pages structured into nineteen chapters and four appendices. The first three chapters give a motivating introduction (chapter 1); briefly show how Mata can be used interactively, in a do-file, or in an ado-file (chapter 2); and provide a short tour of some of the functionality of Mata (chapter 3).
The next six chapters contain an in-depth treatment of the basic features of Mata. Experienced programmers will find many topics in these chapters that they are familiar with. Nonetheless, it makes sense for these users to skim through these chapters to ensure they pick up all aspects in which Mata provides unique features and differs from other languages. For readers who are not fluent programmers, closely studying these chapters will be essential for understanding the rest of the book. There is an immediate payoff: After working through these six chapters, one should be well equipped to program simple functions for a wide array of problems.
Chapter 4 explains how a Mata function is defined and introduces basic programming concepts, such as looping using
In chapter 9, a worked example illustrates how to package a Mata function as a do-file, an ado-file, or part of a Mata library. Most importantly, the chapter also introduces certification. The basic idea is that, alongside programming a Mata function, one should write code that tests the function and asserts that it does what it is supposed to do (that is, returns correct results). Depending on the situation, such test code can be kept within the file defining the Mata function or, more typically, as a separate do-file; the important thing is to have such code so that you can run the tests every time you make a change to the function. Furthermore, if you write the test file in parallel to developing the Mata code, it comes at not much additional cost.
The remaining chapters are on more advanced topics. Chapters 10 and 11 discuss structures. Structures are very useful for programming systems that include multiple functions. They are containers that allow you to collect objects of various types (including other structures) under a single name. This allows you to pass around a large set of objects among the relevant functions using just a single argument. Without the help of structures (or classes; see below), programming a large system would be a very tedious exercise. Chapter 10 first introduces structures and shows how they work in Mata. This includes a valuable discussion of using pointers with structures to conserve memory. Thereafter, chapter 11 illustrates the use of structures by implementing a system of functions for linear regression. The illustration in this chapter is remarkable because it is not just a didactical toy example. The result of the chapter is a high-quality and ready-to-use regression procedure. Gould also introduces a programming concept he calls “self-threading code”. The idea is to link the functions of the system in a clever way such that, for each result that may be requested (for example, the coefficient vector, the variance matrix, or the R-squared), the necessary computations are triggered automatically and, in particular, that no computation has to be done more than once.
Chapters 12 and 13 are on class programming. Classes are structures on steroids. Like structures, they are useful for programming larger systems, but they have additional features that make them much more powerful. For example, unlike structures, classes can contain functions. The first chapter again introduces the concept; the second chapter provides a worked example. The example is again linear regression; that is, the structure-based linear regression from chapter 11 is ported to a class system in chapter 13. Taken together, chapters 12 and 13 teach the basics of class programming in an accessible way. The applied example is geared toward programming statistical procedures. However, if you are interested, for example, in using Mata to program an agent-based model, chapter 12 should still give you all the elements you need.
In chapters 14 and 15, Gould shows how Stata macros can be employed in Mata code. The idea is to define Stata macros for various pieces of code (such as type declarations or constants) and then, in the Mata code, substitute the respective pieces by the macros. Stata macros just contain chunks of text, and the expanded macros will insert the respective text into the Mata code when a function is compiled; it is as if you actually typed the text. Personally, I find this approach extremely useful. It allows you to write more compact and much more readable code because meaningful names can be given to the different elements. It also avoids errors and inconsistencies and makes it easy to apply adjustments to a whole system by just changing the macro definitions. The approach is powerful and can even be taken further than presented in the book: Using Stata macros and loops, you can fully automate the generation of Mata code, which is useful, for example, if you need to write batches of functions that are very similar to each other.
The last three substantive chapters introduce associative arrays and show how they can be used to implement a system for sparse matrices. Chapter 16 briefly introduces Mata’s class for associative arrays. Associative arrays can have many uses. One obvious application is to use them to store sparse matrices (that is, very large matrices in which many cells are “empty”; such matrices often occur, for example, in social network analysis). The illustration provided in chapters 17 and 18, however, is not just about storing a sparse matrix without wasting memory. The main goal is to develop a system for efficient multiplication of sparse matrices. These two chapters are probably the most demanding ones in the book, but working them through will teach you many useful things. For example, you will learn more on how to use pointers (pointers are awesome; they can be very confusing, but you can do amazing things if you master them), and you will also learn how to take timings of your functions.
The final chapter, chapter 19, is just three pages long and gives a brief overview of Mata’s reference manual. It is followed by four appendices. The first appendix provides a detailed account on how to write new Stata commands based on Mata code and on the interface between Stata and Mata. The second appendix contains technical details about complex numbers in Mata. The third provides a brief overview of the differences between Mata and C. Finally, the fourth appendix is just a single page on how to use a pointer vector to define a three-dimensional array.
Although “The Mata Book” is a technical book on a technical issue, it is not a dry read. As mentioned, at least to me, Gould’s style of exposition is highly instructive while being entertaining. Reading is light, but at the same time, you get supplied with all the important information. Of course, there are some more difficult sections that you might have to read twice before fully understanding. But overall, I think Gould did a remarkable job of explaining even complicated concepts in an easy way that anyone can understand: he uses a direct and clear language and includes all intermediate steps necessary for comprehension, yet he does not hide details and complications that might be important for writing high quality code.
Furthermore, the book is much more than just an introduction to Mata. Apart from introducing the programming language and its functionality, the book also teaches you how to write clean and transparent, well-structured and robust code, including certification, and shows you various smaller and larger tricks, for example, on how to preserve precision or on how to assess and improve the speed of the computations. In my opinion, the book is a very valuable read for anyone who is at least mildly interested in acquiring some programming skills. And interested you should be! You can get a lot of mileage out of a little programming.