Abstract
This research outlines the process of building a sample frame of US SMEs. The method starts with a list of patenting organizations and defines the boundaries of the population and subsequent frame using free to low-cost data sources, including search engines and websites. Generating high-quality data is of key importance throughout the process of building the frame and subsequent data collection; at the same time, there is too much data to curate by hand. Consequently, we turn to machine learning and other computational methods to apply a number of data matching, filtering, and cleaning routines. The results show that it is possible to generate a sample frame of innovative SMEs with reasonable accuracy for use in subsequent research: Our method provides data for 79% of the frame. We discuss implications for future work for researchers and NSIs alike and contend that the challenges associated with big data collections require not only new skillsets but also a new mode of collaboration.
Keywords
References
1.
Anagnostopoulos, I., C. Anagnostopoulos, V. Loumos, and E. Kayafas. 2004. “Classifying Web Pages Employing a Probabilistic Neural Network.” IEE Proceedings – Software 151 (3): 139. DOI: https://doi.org/10.1049/ip-sen:20040121.
2.
Andrianantoandro, E., S. Basu, D.K. Karig, and R. Weiss. 2006. “Synthetic Biology: New Engineering Rules for An Emerging Discipline.” Molecular Systems Biology 2. DOI: https://doi.org/10.1038/msb4100073.
3.
Arora, S.K., Y. Li, J. Youtie, and P. Shapira. 2015. “Using the Wayback Machine to Mine Websites in the Social Sciences: A Methodological Resource.” Journal of the Association for Information Science and Technology . DOI: https://doi.org/10.1002/asi.23503.
4.
Arora, S.K., A.L. Porter, J. Youtie, and P. Shapira. 2012. “Capturing New Developments in An Emerging Technology: An Updated Search Strategy for Identifying Nanotechnology Research Outputs.” Scientometrics 95 (1): 351–70. DOI: https://doi.org/10.1007/s11192-012-0903-6.
5.
Atkinson, R.D., and M. Lind. 2018. Big is Beautiful: Debunking the Myth of Small Business . MIT Press.
6.
Barcaroli, G., M. Scannapieco, and D. Summa. 2016. “On the Use of Internet As a Data Source for Official Statistics: A Strategy for Identifying Enterprises on the Web.” Rivista Italiana Di Economia, Demografia E Statistica 70 (4): 20–41.
7.
Baruch, Y., and B.C. Holtom. 2008. “Survey Response Rate Levels and Trends in Organizational Research.” Human Relations 61 (8): 1139–60. DOI: https://doi.org/10.1177/0018726708094863.
8.
Berardi, G., A. Esuli, T. Fagni, and F. Sebastiani. 2015. “Classifying Websites by Industry Sector.” In Proceedings of the 30th Annual ACM Symposium on Applied Computing – SAC ’15 , 1053–59. New York, New York, USA: ACM Press. DOI: https://doi.org/10.1145/2695664.2695722.
9.
Beręsewicz, M., R. Lehtonen, F. Reis, L. Di Consiglio, and M. Karlberg. 2018. “An Overview of Methods for Treating Selectivity in Big Data Sources.” Eurostat Statistical Working Papers. Luxembourg. DOI: https://doi.org/10.2785/312232 (accessed July 2020).
10.
Birch, D.G.W. 1987. Job Creation in America: How Our Smallest Companies Put the Most People to Work. University of Illinois at Urbana-Champaign.
11.
Blazquez, D., and J. Domenech. 2018. “Big Data Sources and Methods for Social and Economic Analyses.” Technological Forecasting and Social Change 130 (May): 99–113. DOI: https://doi.org/10.1016/j.techfore.2017.07.027.
12.
Blazquez, D., J. Domenech, and A. Debón. 2018. “Do Corporate Websites’ Changes Reflect Firms’ Survival?” Online Information Review 42 (6): 956–70. DOI: https://doi.org/10.1108/OIR-11-2016-0321.
13.
Broder, A. 2002. “A Taxonomy of Web Search.” ACM SIGIR Forum 36 (2): 3–10. DOI: https://doi.org/10.1145/792550.792552.
14.
Bruni, R., and G. Bianchi. 2020. “Website Categorization: A Formal Approach and Robustness Analysis in the Case of e-Commerce Detection.” Expert Systems With Applications 142: 113001. DOI: https://doi.org/10.1016/j.eswa.2019.113001.
15.
Buelens, B., P. Daas, J. Burger, M. Puts, and J. Van Den Brakel. 2014. “Selectivity of Big Data.” Available at: https://www.researchgate.net/profile/Bart_Buelens/publication/261436243_Selectivity_of_Big_data/links/00b49539806fa06b08000000/Selectivity-of-Big-data.pdf.
16.
Cohen, W.M., R.R. Nelson, and J.P. Walsh. 2000. “Protecting Their Intellectual Assets: Appropriability Conditions and Why U.S. Manufacturing Firms Patent (or Not).” 7552. Available at: http://www.nber.org/papers/w7552 (accessed February 2020).
17.
Connelly, R., C.J. Playford, V. Gayle, and C. Dibben. 2016. “The Role of Administrative Data in the Big Data Revolution in Social Science Research.” Social Science Research 59: 1–12. DOI: https://doi.org/10.1016/j.ssresearch.2016.04.015.
18.
Cortes, C., and V. Vapnik. 1995. “Support-Vector Networks.” Machine Learning 20 (3): 273–97. DOI: https://doi.org/10.1007/BF00994018.
19.
Coughlin, S.M. 2006. “Is the Patent Paradox a Result a Large Firm Perspective-Differential Value of Small Firm Patents Over Time Explains the Patent Paradox.” Santa Clara Computer & High Tech. LJ 23: 371. Available at: https://digitalcommons.law.scu.edu/cgi/viewcontent.cgi?referer=
https://scholar.google.com/&httpsredir=1&-article=1429&context=chtlj (accessed March 2019).
20.
Cruz, R.M.O., L.G. Hafemann, R. Sabourin, and G.D.C. Cavalcanti. 2018. “DESlib: A Dynamic Ensemble Selection Library in Python.” Available at: http://arxiv.org/abs/1802.04967 (accessed March 2020).
21.
Demirel, P., and M. Mazzucato. 2012. “Innovation and Firm Growth: Is R&D Worth It?” Industry & Innovation 19 (1): 45–62. DOI: https://doi.org/10.1080/13662716. 2012.649057.
22.
Dennis Jr., W.J. 2003. “Raising Response Rates in Mail Surveys of Small Business Owners: Results of An Experiment.” Journal of Small Business Management 41 (3): 278–95. DOI: https://doi.org/10.1111/1540-627X.00082.
23.
Einav, L., and J. Levin. 2014. “Economics in the Age of Big Data.” Science 346 (6210): 1243089–1243089. DOI: https://doi.org/10.1126/science.1243089.
24.
ESSnet Big Data. 2020. “WPC Enterprise Characteristics – ESSnet Big Data.” Available at: https://webgate.ec.europa.eu/fpfis/mwikis/essnetbigdata/index.php/WPC_Enterprise_characteristics (accessed March 2020).
25.
Fan, R.-E., K.-W. Chang, C.-J. Hsieh, X.-R. Wang, and C.-J. Lin. 2008. “LIBLINEAR: A Library for Large Linear Classification.” Journal of Machine Learning Research 9: 1871–74. DOI: https://dl.acm.org/doi/pdf/10.5555/1390681.1442794.
26.
Flanegin, F., S. Racic, and D. Rudd. 2011. “Accuracy and Cost of U.S. Financial Data.” Journal of Applied Business Research (JABR) 25 (6). DOI: https://doi.org/10.19030/-jabr.v25i6.994.
27.
Fortune. 2019a. “Fortune 500 j Fortune.” Available at: https://fortune.com/fortune500/ (accessed March 2020).
28.
Fortune. 2019b. “Fortune 500 j Fortune.” Available at: https://fortune.com/fortune500/ (accessed March 2020).
29.
Giesen, D., M. Vella, C.F. Brady, P. Brown, D. Ravindra, and A. Vaasen-Otten. 2018. “Response Burden Management for Establishment Surveys at Four National Statistical Institutes.” Journal of Official Statistics 34 (2): 397–418. DOI: https://doi.org/10.2478/jos-2018-0018.
30.
Glover, E.J, K. Tsioutsiouliklis, S. Lawrence, D.M. Pennock, and G.W. Flake. 2002. “Using Web Structure for Classifying and Describing Web Pages.” In Proceedings of the Eleventh International Conference on World Wide Web – WWW ’02 , 562. New York, New York, USA: ACM Press. DOI: https://doi.org/10.1145/511446.511520.
31.
Graber, M.A., and M. Weckmann. 2002. “Pharmaceutical Company Internet Sites As Sources of Information About Antidepressant Medications.” CNS Drugs 16 (6): 419–23. DOI: https://doi.org/10.2165/00023210-200216060-00005.
32.
Groves, R.M., and B.A. Harris-Kojetin, Eds. 2017. “Innovations in Federal Statistics.” Washington, D.C.: National Academies Press. DOI: https://doi.org/10.17226/24652.
33.
Hastie, T., S. Rosset, J. Zhu, and H. Zou. 2009. “Multi-Class AdaBoost.” Statistics and Its Interface 2 (3): 349–60. DOI: https://doi.org/10.4310/SII.2009.v2.n3.a8.
34.
Hicks, D., and D. Hegde. 2005. “Highly Innovative Small Firms in the Markets for Technology.” Research Policy 34 (5): 703–16. DOI: https://doi.org/10.1016/j.respol.2005.03.008.
35.
Hu, X., and R. Rousseau. 2015. “From a Word to a World: The Current Situation in the Interdisciplinary Field of Synthetic Biology.” PeerJ 3 (January): E728. DOI: https://doi.org/10.7717/peerj.728.
36.
Kanani, P.H, and A.K. McCallum. 2012. “Selecting Actions for Resource-Bounded Information Extraction Using Reinforcement Learning.” In Proceedings of the Fifth ACM International Conference on Web Search and Data Mining – WSDM ’12 , 253. New York, New York, USA: ACM Press. DOI: https://doi.org/10.1145/2124295.2124328.
37.
Kaplan, R.M., D.A. Chambers, and R.E. Glasgow. 2014. “Big Data and Large Sample Size: A Cautionary Note on the Potential for Bias.” Clinical and Translational Science 7 (4): 342–46. DOI: https://doi.org/10.1111/cts.12178.
38.
King, G. 2007. “An Introduction to the Dataverse Network As An Infrastructure for Data Sharing.” Sociological Methods & Research 36 (2): 173–99. DOI: https://doi.org/10.1177/0049124107306660.
39.
Kingma, D.P., and J. Ba. 2014. “Adam: A Method for Stochastic Optimization.” ArXiv Preprint ArXiv:1412.6980. Available at: http://arxiv.org/abs/1412.6980 (accessed February 2020).
40.
Kitchin, R. 2014. “Big Data, New Epistemologies and Paradigm Shifts.” Big Data & Society 1 (1): 205395171452848. DOI: https://doi.org/10.1177/2053951714528481.
41.
Lane, J. 2016. “Big Data for Public Policy: The Quadruple Helix.” Journal of Policy Analysis and Management 35 (3): 708–15. DOI: https://doi.org/10.1002/pam.21921.
42.
Lane, J. And S. Bertuzzi. 2011. “Measuring the Results of Science Investments.” Science 331 (6018): 678–80. DOI: https://doi.org/10.1126/science.1201865.
43.
Lanjouw, J.O., and M. Schankerman. 2004. “Protecting Intellectual Property Rights: Are Small Firms Handicapped?” The Journal of Law and Economics 47 (1): 45–74. DOI: https://doi.org/10.1086/380476.
44.
Lewandowski, D., and N. Höchstötter. 2008. “Web Searching: A Quality Measurement Perspective.” In Web Search: Multidisciplinary Perspectives , edited by Amanda Spink and Michael Zimmer, 309–40. Berlin, Heidelberg: Springer-Verlag. DOI: https://doi.org/10.1007/978-3-540-75829-7_16.
45.
Lewandowski, D. 2015. “Evaluating the Retrieval Effectiveness of Web Search Engines Using a Representative Query Sample.” Journal of the Association for Information Science and Technology 66 (9): 1763–75. DOI: https://doi.org/10.1002/asi.23304.
46.
Lin, C., Y.-A. Huang, and R. Stockdale. 2011. “Developing a B2B Web Site Effectiveness Model for SMEs.” Internet Research 21 (3): 304–25. DOI: https://doi.org/10.1108/10662241111139327.
47.
Lindič, J., M. Bavdaž, and H. Kovačič. 2012. “Higher Growth Through the Blue Ocean Strategy: Implications for Economic Policy.” Research Policy 41 (5): 928–38. DOI: https://doi.org/10.1016/j.respol.2012.02.010.
48.
Lundvall, B.-Å., and S. Borrás. 2005. “Science, Technology and Innovation Policy.” In The Oxford Handbook of Innovation , edited by Jan Fagerberg, David C. Mowery, and Richard R. Nelson: 599–631. Oxford University Press Oxford.
49.
MacFeely, S. 2019. “Big Data and Official Statistics.” In Big Data Governance and Perspectives in Knowledge Management, edited by Sheryl Kruger Strydom and Moses Strydom, 25–54. IGI Global. DOI: https://doi.org/10.4018/978-1-5225-7077-6.ch002.
50.
Mergel, I., R.K. Rethemeyer, and K. Isett. 2016. “Big Data in Public Affairs.” Public Administration Review 76 (6): 928–37. DOI: https://doi.org/10.1111/puar.12625.
51.
Mooney, R.J., and R. Bunescu. 2005. “Mining Knowledge from Text Using Information Extraction.” ACM SIGKDD Explorations Newsletter 7 (1): 3–10. DOI: https://doi.org/10.1145/1089815.1089817.
52.
Munari, F., and L. Toschi. 2014. “Running Ahead in the Nanotechnology Gold Rush. Strategic Patenting in Emerging Technologies.” Technological Forecasting and Social Change 83: 194–207. DOI: https://doi.org/10.1016/j.techfore.2013.07.002.
53.
Nathan, M., A. Rosso, T. Gatten, P. Majmudar, and A. Mitchell. 2013. “Measuring the UK’s Digital Economy With Big Data.” National Institute of Economic and Social Research London. Available at: https://www.niesr.ac.uk/sites/default/files/publications/SI024_GI_NIESR_Google_Report12.pdf (accessed February 2020).
54.
National Academies of Sciences, Engineering, and Medicine. 2018. “Sampling and Estimation.” In Reengineering the Census Bureau’s Annual Economic Surveys , edited by K.G. Abraham, C.F. Citro, G.D. White, and N.K. Kirkendall: 91–118. Washington, D.C.: National Academies Press. DOI: https://doi.org/10.17226/25098.
55.
OECD, and the Royal Society. 2010. Symposium on Opportunities and Challenges in the Emerging Field of Synthetic Biology . OECD Publishing. DOI: https://doi.org/10.1787/9789264086265-en.
56.
Oldham, P., S. Hall, and G. Burton. 2012. “Synthetic Biology: Mapping the Scientific Landscape.” Edited by J.A. Gilbert. PLoS ONE 7 (4): E34368. DOI: https://doi.org/10.1371/journal.pone.0034368.
57.
Oostrom, L., A. Walker, B. Staats, M. Slootbeek-van Laar, S. Azurduy, and B. Rooijakkers. 2016. “Measuring the Internet Economy in the Netherlands: A Big Data Analysis.” The Hague: Statistics Netherlands . Available at: https://www.nldigital.nl/wp-content/uploads/2016/10/measuring-the-internet-economy.pdf (accessed February 2020).
58.
PCAST. 2005. “The National Nanotechnology Initiative at Five Years.” Washington, D.C., U.S.A. Available at: https://www.nano.gov/sites/default/files/pub_resource/final_pcast_nano_report_for_web.pdf (accessed February 2019).
59.
Pedregosa, F., G. Varoquaux, A. Gramfort, V.M.B. Thirion, O. Grisel, M.Blondel, et al. 2011. “Scikit-Learn: Machine Learning in Python.” Journal of Machine Learning Research 12: 2825–30. Available at: https://www.jmlr.org/papers/volume12/pedregosa11a/pedregosa11a.pdf (accessed February 2019).
60.
Pew. 2018. “Internet/Broadband Fact Sheet.” Available at: http://www.pewinternet.org/fact-sheet/internet-broadband/ (accessed February 2019).
61.
Popp, D. 2016. “Economic Analysis of Scientific Publications and Implications for Energy Research and Development.” Nature Energy 1 (4): 16020. DOI: https://doi.org/10.1038/nenergy.2016.20.
62.
Popp, D. 2017. “From Science to Technology: The Value of Knowledge from Different Energy Research Institutions.” Research Policy 46 (9): 1580–94. DOI: https://doi.org/10.1016/j.respol.2017.07.011.
63.
Qi, X., and B.D. Davison. 2007. “Web Page Classification.” ACM Computing Surveys 41(2): 1–31. DOI: https://doi.org/10.1145/1459352.1459357.
64.
Raimbault, B., J.-P. Cointet, and P.-B. Joly. 2016. “Mapping the Emergence of Synthetic Biology.” Edited by Mark Isalan. PLOS ONE 11 (9): E0161522. DOI: https://doi.org/10.1371/journal.pone.0161522.
65.
Rajaraman, A., and J.D. Ullman. 2011. Mining of Massive Datasets . Cambridge University Press.
66.
Reimsbach-Kounatze, C. 2015. “The Proliferation of ‘Big Data’ And Implications for Official Statistics and Statistical Agencies: A Preliminary Analysis.” 245. OECD Digital Economy Papers. Paris. DOI: https://doi.org/10.1787/5js7t9wqzvg8-en.
67.
Rothwell, R. 1989. “Small Firms, Innovation and Industrial Change.” Small Business Economics 1 (1): 51–64. DOI: https://doi.org/10.1007/BF00389916.
68.
Safavian, S.R., and D. Landgrebe. 1991. “A Survey of Decision Tree Classifier Methodology.” IEEE Transactions on Systems, Man, and Cybernetics 21 (3): 660–74. DOI: https://doi.org/10.1109/21.97458.
69.
Särndal, C.-E., B. Swensson, and J. Wretman. 2003. Model Assisted Survey Sampling . New York: Springer-Verlag.
70.
Shapira, P., A. Gök, E. Klochikhin, and M. Sensier. 2013. “Probing ‘Green’ Industry Enterprises in the UK: A New Identification Approach.” Technological Forecasting and Social Change . DOI: https://doi.org/10.1016/j.techfore.2013.10.023.
71.
Sicilia, M.-A., E. García-Barriocanal, and S. Sánchez-Alonso. 2017. “Community Curation in Open Dataset Repositories: Insights from Zenodo.” Procedia Computer Science 106: 54–60. DOI: https://doi.org/10.1016/j.procs.2017.03.009.
72.
Simon, H.A. 1996. The Sciences of the Artificial . 3rd ed. Cambridge, MA: MIT Press.
73.
Sullivan, I., A. DeHaven, and D. Mellor. 2019. “Open and Reproducible Research on Open Science Framework.” Current Protocols Essential Laboratory Techniques 18 (1). DOI: https://doi.org/10.1002/cpet.32.
74.
Talan, D.M. 2016. “Opportunities and Challenges for Using Big Administrative Data.” Available at: https://www.bls.gov/osmr/research-papers/2016/pdf/st160100.pdf (accessed February 2019).
75.
Tambe, P. 2014. “Big Data Investment, Skills, and Firm Value.” Management Science 60 (6): 1452–69. DOI: https://doi.org/10.1287/mnsc.2014.1899.
76.
Teece, D.J., M. Peteraf, and S. Leih. 2016. “Dynamic Capabilities and Organizational Agility: Risk, Uncertainty, and Strategy in the Innovation Economy.” California Management Review 58 (4): 13–35. DOI: https://doi.org/10.1525/cmr.2016.58.4.13.
77.
Ten Bosch, O., D. Windmeijer, A. Van Delden, and G. Van Den Heuvel. 2018. “Web Scraping Meets Survey Design: Combining Forces.” In Big Data Meets Survey Science Conference, Barcelona, Spain . Available at: https://www.bigsurv18.org/conf18/uploads/73/61/20180820_BigSurv_WebscrapingMeetsSurveyDesign.pdf (accessed February 2020).
78.
Toumey, C. 2009. “Plenty of Room, Plenty of History.” Nature Nanotechnology 4 (12): 783–84. DOI: https://doi.org/10.1038/nnano.2009.357.
79.
U.K. Office for National Statistics. 2020. “Business enterprise research and development, UK: 2019.” Available at: https://www.ons.gov.uk/economy/governmentpublicsectorandtaxes/researchanddevelopmentexpenditure/bulletins/businessenterpriseresearchanddevelopment/2019 (accessed February 2020).
80.
U.S. Census Bureau. 2020a. “About Annual Business Survey.” Available at: https://www.census.gov/programs-surveys/abs/about.html (accessed March 2020).
81.
U.S. Census Bureau. 2020b. “About this Survey.” Available at: https://www.census.gov/-programs-surveys/abs/about.html.
82.
U.S. Energy Information Administration. 2018. “Available Energy Explained.” Available at: https://www.eia.gov/energyexplained/renewable-sources/incentives.php (accessed February 2019).
83.
U.S. Small Business Administration. 2018. “Frequently Asked Questions.” Available at: https://www.sba.gov/sites/default/files/advocacy/Frequently-Asked-Questions-Small-Business-2018.pdf (accessed February 2019).
84.
USPTO. 2019. “PatentsView.” Available at: https://www.patentsview.org/download/(accessed March 2020).
85.
USPTO. 2020. “US Patent Activity, CY 1790 to Present.” Available at: https://www.uspto.gov/web/offices/ac/ido/oeip/taf/h_counts.htm (accessed March 2020).
86.
Van Delden, A., D. Windmeijer, and O. Ten Bosch. 2019a. “Finding Enterprise Websites.” In European Establishment Statistics Workshop . Bilbao, Spain. Available at: https://www.researchgate.net/profile/Arnout_Delden/publication/336995371_Finding_enterprise_websites/links/5dbebf29299bf1a47b0f5669/Finding-enterprise-websites.pdf (accessed February 2020).
87.
Van Delden, A., D. Windmeijer, and O. Ten Bosch. 2019b. “Searching for Business Websites.” Centraal Bureau Voor De Statistiek. Available at: https://www.cbs.nl/-/media/_pdf/2020/01/searching-for-business-websites.pdf (accessed February 2020).
88.
Van Doren, D., S. Koenigstein, and T. Reiss. 2013. “The Development of Synthetic Biology: A Patent Analysis.” Systems and Synthetic Biology 7 (4): 209–20. DOI: https://doi.org/10.1007/s11693-013-9121-7.
89.
Wang, F., and L. Vaughan. 2014. “Firm Web Visibility and Its Business Value.” Internet Research 24 (3): 292–312. DOI: https://doi.org/10.1108/IntR-01-2013-0016.
90.
Williams, C.K.I., and C.E. Rasmussen. 2006. Gaussian Processes for Machine Learning . Cambridge, MA: MIT Press.
91.
Young, L.J., M. Hyman, and B.R. Rater. 2018. “Exploring a Big Data Approach to Building a List Frame for Urban Agriculture: A Pilot Study in the City of Baltimore.” Journal of Official Statistics 34 (2): 323–40. DOI: https://doi.org/10.2478/jos-2018-0015.