Mata Matters: Overflow,Underflow and the IEEE Floating-Point Format

Abstract

Mata is Stata's matrix language. The Mata Matters column shows how Mata can be used interactively to solve problems and as a programming language to add new features to Stata. In this quarter's column, we investigate underflow and overflow and then delve into the details of how floating-point numbers are stored in the IEEE 754 floating-point standard. We show how to test for overflow and underflow. We demonstrate how to use the %21x format to see underflow and the %16H, %16L, %8H, and %8L formats for displaying the byte content of doubles and floats.

Keywords

pr0038 underflow overflow denormalized number normalized number subnormal number double precision missing values IEEE 754 format binary hexadecimal

References

Cox

N. J.

2006. Stata tip 33: Sweet sixteen: Hexadecimal formats and precision problems. Stata Journal 6: 282–283.

Goldberg

1991. What every computer scientist should know about floating-point arithmetic. ACM Computing Surveys 23: 5–48. Available at http://docs.sun.com/source/806-3568/ncg-goldberg.html.

Gould

2006. Mata Matters: Precision. Stata Journal 6: 550–560.

IEEE. 1985. IEEE Standard 754–1985 for binary floating-point arithmetic. New York: Institute of Electrical and Electronics Engineers.

Knuth

D. E.

1998. The Art of Computer Programming. Volume 2: Seminumerical Algorithms. 3rd ed. Reading, MA: Addison–Wesley.

Severance

1998. An interview with the old man of floating-point. IEEE Computer 114–115. Available at http://www.cs.berkeley.edu/∼wkahan/ieee754status/754story.html.

Shea

B. L.

1989. A remark on Algorithm AS152: Cumulative hypergeometric probabilities. Applied Statistics 38: 199–204. Code available at http://lib.stat.cmu.edu/apstat/152.