Quantitative Evaluation of Digital Economy Policy in Heilongjiang Province of China Based on the PMC-AE Index Model

Abstract

The digital economy has gradually become an important support for high-quality economic development, but the speed of digital economy development is affected by policies. A reasonable digital economy policy system has practical significance for the development of the digital economy. This study takes the digital economy policy text published by Heilongjiang Province from 2015 to 2022 as the research object, uses the method of text mining and content analysis, innovatively introduces AE automatic coding technology into the PMC index model, constructs the PMC-AE index model, and quantitatively evaluates and analyzes the policy text. The study found that the digital economy policy of Heilongjiang Province is at a good level as a whole, but there is still room for improvement. The digital economy policy of Heilongjiang Province is mainly based on guiding policies. The richness of policy types, participants, and incentives is insufficient. The limitation of policy timeliness is strong, and the cooperation of the issuing agencies is not sufficient. Based on the research results, it is suggested that policymakers should pay attention to the combination of macro and micro policies, promote multi-agent participation, enrich incentive measures, develop both policy guidance and supervision, improve the timeliness and consistency of policies, and strengthen the communication between the Issuing Departments. In addition, this study puts forward policy suggestions for specific government departments from four aspects of digital industrialization, industrial digitization, digital governance, and data value. This study not only provides policy recommendations for policymakers but also expands the field of policy evaluation.

Keywords

digital economy policy policy text PMC-AE index model text mining quantitative evaluation

Introduction

In recent years, the rise of a new generation of information technology such as big data, 5G, AI, and blockchain has led to a subversive change in the mode of social production, and many new formats and models have emerged (Ren et al., 2023). Digital technology has a positive impact on business innovation in emerging markets (Dana, Salamzadeh, Mortazavi, & Hadizadeh, 2022). And the digital economy plays a stronger role in promoting innovation for large enterprises (Q. Li et al., 2023). The digital economy has gradually become an important engine to promote economic development. To seize the development opportunity, countries around the world have successively issued digital economy-related policies, deployed digital economy development strategies, and supported the development of the digital economy. For example, the United States’ digital economy agenda, Japan’s e-Japan strategy, and Germany’s digital economy strategy 2025. In the context of the world’s unprecedented changes in a century, especially in the case of the outbreak of the new coronal pneumonia epidemic, the digital economy has played an important role in the process of economic development, showing great vitality and toughness.

Sustainability is the key element of the digital economy, and the development of the digital economy cannot be separated from the support of policies (Dana, Salamzadeh, Hadizadeh, Heydari, & Shamsoddin, 2022; Huang et al., 2020). China started late in the field of digital economy development. To promote the development speed of China’s digital economy, the Chinese government has given strong policy support. Since the 18th National Congress of the Communist Party of China, the Central Committee of the Communist Party of China has supported the comprehensive deepening of reform in Northeast China, further promoted the Revitalization Strategy of Northeast China, accelerated the adjustment of industrial structure, adapted to the development requirements of the new era, played the advantages of the digital economy, and assisted the economic revitalization of Northeast China.

According to the China Digital Economy Development Report (2022) released by the China Institute of Information and Communications, there are imbalances in the proportion of digital economy scale and growth rate among China’s provinces, and the development of the digital economy in Northeast China is relatively slow. Among the three provinces in Northeast China, Heilongjiang province lags behind the other two provinces in terms of the proportion of digital economy scale and growth rate. To promote the development of the digital economy in Heilongjiang Province of China, it is necessary to evaluate the existing digital economy policies. At the same time, objective policy evaluation is also crucial to the improvement of digital economy policies.

The current research streams on digital economy mainly include: novelty in science, structured digital economy, area growth, rationalization and contextualization, creativity, and entrepreneurship (Okpalaoka, 2023). However, scholars’ research on the digital economy policy text itself is relatively less. To fill this gap, this study takes Heilongjiang digital economy policy text as the research object, constructs the PMC-AE index model through text information mining, and carries out the quantitative evaluation of the policy text.

The research questions to be solved in this study are as follows: (1) what is the quality of the digital economy policy in Heilongjiang Province? (2) Whether there are deficiencies in the design of the digital economy policy of Heilongjiang Province in China, and how to solve the dilemma of slow development of the digital economy from the perspective of policy.

The main contributions of this study include: (1) The PMC-AE index model based on text mining is used to quantitatively analyze the digital economic policy text, and the method of digital economic policy evaluation is innovated; (2) Through the text evaluation of digital economy policy, this paper puts forward relevant policy suggestions, which can provide a reference for the formulation, adjustment, and implementation of a new round of digital economy policy in Heilongjiang Province in the future.

The remaining parts of this study are structured as follows: The second part is a literature review of public policy evaluation research and digital economy policy research; The third part is the research design, which introduces the data sources and research methods; The fourth part is analysis and results; The fifth part is the discussion; The sixth part is the conclusion.

Literature Review

Research on Public Policy Evaluation

Public policy is a comprehensive discipline to solve social problems and find policy solutions by analyzing public policy phenomena. Policy evaluation is essential to understand the objectives of the policy and its impact on reality (Christie & Lemire, 2019). Researchers and policymakers also recognize the need to evaluate policies (Miljand, 2020). Through policy evaluation, policymakers can find the shortcomings and promote the optimization of policymaking in the future (Dai et al., 2021). Public policy research can be traced back to the middle of the twentieth century. As the founder of public policy, American scholar Lasswell (1951) put forward the policy process model. Since the 1960s, public policy evaluation has attracted the attention of academia and relevant government departments. With the development of the new public management reform movement, public policy evaluation has gradually occupied an important position in the field of public policy research. Early public policy evaluation methods are represented by qualitative research methods such as Poland’s (1974) policy evaluation classification, Suchman’s (1968) five-category evaluation method, and Wollmann’s (2007) classic causal effect evaluation method. With the further development of public policy evaluation research, policy evaluation methods have gradually become rich, from a single qualitative method to a combination of quantitative and qualitative methods (Prior et al., 2012). For example, analytic hierarchy process, content analysis, double difference method, etc. However, these methods are mainly for the overall evaluation of macro policies, and there are some limitations (Xiong et al., 2023).

With the advent of the information age, text mining technology has promoted the development of policy evaluation methods (Kayser & Blind, 2017). For example, Ren et al. (2023) used text analysis to explore the dynamic changes and influencing factors of enterprise digital development level. Estrada (2011) constructed a policy modeling consistency (PMC) index model based on the Omnia Mobility (everything is moving) hypothesis. With the help of text mining technology, the model transforms unstructured text into structured data, which can not only evaluate the overall policy but also analyze the advantages and disadvantages of individual policies. Scholars have introduced the model into various fields for policy evaluation. For example, based on the 57 LTCI policies developed in 15 pilot areas in China, Peng et al. (2020) evaluate the LTCI policies by building a PMC index model with 10 primary variables and 44 two-level variables. F. Liu et al. (2022) used text mining technology and the PMC index model to quantitatively evaluate the waste separation management policy in the Yangtze River Delta. In addition, the model has also been introduced into the research of high-tech industrial policy (Y. Liu et al., 2022), development policy of traditional Chinese medicine (Yang et al., 2022), cultivated land protection policy (Kuang et al., 2020) and wind power industry policy (B. Wang et al., 2022).

Research on Digital Economy Policy

Since the term “ digital economy ” first appeared at the end of the 20th century, the connotation of digital economy has become more abundant with the development of scientific and technological innovation (Tapscott, 1996). Kling and Lamb (1999) study it and organizational change in digital economies: a socio-technical approach. Research has found how it requires complex organizational work takes to get information systems “up and running.”Dahlman et al. (2016) make a call for why the digital economy matters for developing countries and what they need to consider when developing a national digital strategy. The meaning and metrics of the digital economy are both limited and divergent. The purpose of Bukht and Heeks (2017) is to review what is currently known to develop a definition of the digital economy, and an estimate of its size. Shokhnekh et al. (2020) define the directions of the concept strategy for the development of innovative small businesses in the system of economic security, which is particularly relevant in the digital economy. K. Li et al. (2020) analyzed the “digital economy and society index” and the attributes of the country, society, and economy as the basis for framing ideas.

Research on digital economy policy. The research on digital economic policy mainly focuses on two aspects: the research on the current situation of digital economic policy and the evaluation of digital economic policy.

As for the research on the current situation of digital economic policies, scholars mostly analyze the current situation of digital economic policies in the world. For example, Romanova (2018) believes that one of the characteristics of the fourth industrial revolution is the emergence of the digital economy. The fundamental difference of this revolution lies in the coordination and integration of a large number of scientific disciplines, which puts forward new requirements for the formation of industrial policy priorities. Aturin et al. (2020) explores the problems and trends in the digital transformation of the world economy in the context of revolutionary changes in the technological sphere, the unstable state of several national economies, and international trade. Foster and Azmeh (2020) analyzed the current situation of digital economy policies in developing countries and found that the intervention of digital economy policies is crucial to the digital transformation of developing countries. L. Chen et al. (2019) took the G20 economies as the research object and found that imperfect digital economy policies affected the development of the digital economy. It can be seen that the improvement of digital economy policy has a positive effect on the development of the digital economy.

As for the evaluation of digital economy policy, scholars mainly focus on the impact of the digital economy on other fields. For example, Aniqoh (2020) provides support for the government to formulate digital economy policies by analyzing the impact of the digital economy on promoting sustainable economic development. Y. Chen (2020) found that the digital economy reduces market friction, but it also poses new challenges to the effective operation of the market. Guo et al. (2023) found that the digital economy significantly promotes high-quality economic development by improving human capital and promoting green technology innovation. Hojeghan and Esfangareh (2011) discussed the impact of the digital economy on tourism. J. Wang et al. (2022) explored the role of the digital economy in a low-carbon society. Cheng et al. (2023) found that the digital economy can affect the intensity of carbon emissions in a variety of ways, such as optimizing the industrial structure, promoting scientific and technological innovation, etc. the research findings can provide a reference for the development of regional digital economy and the formulation of carbon reduction policies. Many scholars have reached a consensus that the new driving force for sustainable economic development comes from the digital economy (Jiao & Sun, 2021).

In summary, scholars have begun to explore the research of digital economic policy. However, research on the evaluation of digital economy policy text itself is relatively rare. To fill this gap, this study introduces the PMC index model into the field of digital economic policy research, and on this basis, integrates the PMC index model with AE automatic coding technology, making the research results more objective. This study takes the text of the digital economy in Heilongjiang Province of China as the research object and uses the PMC-AE index model for quantitative analysis to provide a reference for the formulation, optimization, and implementation of digital economic policies in Heilongjiang Province in the future.

Experimental Design

Data Source

This study takes the policy text of the digital economy in Heilongjiang Province from 2015 to 2022 as the research object. The policy text comes from the official website of the Heilongjiang Provincial Government of China and the legal database of Peking University, with “Internet,”“digital economy,”“digitization,”“big data,” and so on as keywords. To ensure the comprehensiveness and authority of the policy text, remove documents such as replies, letters, and important speeches, the main forms of the policy text are selected as regulations, opinions, plans, and notices, and finally, 44 current effective policy texts are selected.

Method

This study comprehensively uses the text mining method, content analysis method, and PMC-AE index model to quantitatively evaluate the text of digital economic policy. The scholar Estrada (2009) proposed the Omnia Mobilis hypothesis in 2008, which holds that everything is constantly moving and interrelated, and no relevant variable should be ignored. The PMC (Policy Modeling Consistency) index model is a public policy text evaluation model based on this hypothesis. Therefore, the range of secondary index variables is not limited, and the weight of each index variable is the same (Estrada, 2011). This model can not only evaluate the consistency of policies but also intuitively reflect the advantages and disadvantages of individual policies. It is a popular quantitative evaluation model of policies in recent years. The PMC-AE index model is improved based on the PMC index model, and the AE autoencoder technology is introduced. The difference between the two models is the method used to calculate the index: the PMC-AE index model uses AE autoencoder technology to nonlinearly fuse the parameters, which is superior to the classical PMC index model.

Analysis and Results

Model Construction

The construction of the PMC-AE index model is mainly divided into four steps, as shown in Figure 1: variable classification and parameter identification; establish multiple input and output tables; AE parameter fusion process; Calculate PMC-AE index and draw PMC-AE surface.

Figure 1.

PMC-AE model policy quantitative evaluation process.

Variable Classification and Parameter Identification

This study uses ROSTCM text mining software to mine information from policy texts. Specific operation: The 44 digital economic policy texts are integrated and all imported into ROSTCM software for word segmentation and word frequency analysis, and the common words with high frequency such as “people’s government” and “department” but no practical significance for research are filtered. On this basis, high-frequency words are extracted to form a semantic network map of co-occurrence high-frequency words as shown in Figure 2, and the top 40 words are sorted out as shown in Table 1.

Figure 2.

Heilongjiang digital economy policy text co-occurrence high-frequency word semantic network map.

Table 1.

High-Frequency Words of Digital Economy Policy Text in Heilongjiang Province.

Serial number	Vocabulary	Frequency	Serial number	Vocabulary	Frequency
1	Enterprise	2,260	21	Mechanism	605
2	Service	2,073	22	Capacity	601
3	Technique	1,771	23	Intelligence	564
4	Innovation	1,639	24	Territory	564
5	Technology	1,354	25	Environment	563
6	Platform	1,138	26	Foundation	551
7	Agriculture	1,101	27	Government Affairs	538
8	Data	1,039	28	Equipment	516
9	Tourism	1,003	29	Base	516
10	Application	968	30	Standard	513
11	Management	891	31	Internet	499
12	Market	850	32	Figure	487
13	System	773	33	Government	483
14	Resource	757	34	Wisdom	455
15	Industry	683	35	Manufacture	451
16	Safety	679	36	Fusion	446
17	Regulation	676	37	Mechanism	441
18	System	644	38	Talent	439
19	Emphasis	637	39	Investigate	430
20	Item	619	40	Optimization	426

In the semantic network map, the high-frequency words in the policy text are connected in the way of the network, which more intuitively reflects the relationship between high-frequency words. High-frequency words are represented by nodes. The more nodes a node connects to other nodes, the stronger the centrality of the nodes, indicating that the more critical the nodes are. Figure 2 directly reflects the focus of the digital economy policy text. Among them, words such as enterprise, service, technology, innovation, and platform appear frequently and are closely related to other words.

Variable classification. Based on the basic research theory and existing research results, combined with the text information mining of digital economic policy, nine primary evaluation index variables of digital economic policy were selected. Because the PMC-AE index model considers all index variables as well as possible, there is no limit to the number of level two index variables. Based on level one index variables, 46 level two index variables are selected, as shown in Table 2.

Table 2.

Indicator Variables of the PMC-AE Index Model.

Primary variables	Secondary variables
X₁ Policy type	X₁₁	Plan for the class	X₁₂	Plan detailed rules for the class	X₁₃	Class methods regulations
X₁ Policy type	X₁₄	Opinion decided to class
X₂ Policy timeliness	X₂₁	1–3 years	X₂₂	3–5 years	X₂₃	More than 5 years
X₃ Policy nature	X₃₁	Encourage	X₃₂	Regulatory	X₃₃	Guide
X₃ Policy nature	X₃₄	Advice
X₄ Issuing agency	X₄₁	Multi-sectoral joint mandate	X₄₂	The provincial people issued a document	X₄₃	The provincial government issued a communication
X₄ Issuing agency	X₄₄	The provincial Office issued a document	X₄₅	Papers by other departments
X₅ Policy content	X₅₁	Digital transformation	X₅₂	Social governance	X₅₃	infrastructure development
	X₅₄	Industrial convergence	X₅₅	Safety of network	X₅₆	Scientific and technological innovation
	X₅₇	Big data	X₅₈	Market supervision	X₅₉	Digital technique
	X₆₀	Ecological platform
X₆ Participants	X₆₁	Government	X₆₂	Enterprise	X₆₃	College and universities
X₆ Participants	X₆₄	Scientific research institutions	X₆₅	Social organization	X₆₆	Miscellaneous
X₇ Policy areas	X₇₁	Social service	X₇₂	Technical level	X₇₃	Environment atmosphere
X₇ Policy areas	X₇₄	Rules and regulations
X₈ Incentive measures	X₈₁	Government subsidies	X₈₂	Technical support	X₈₃	Tax preference
X₈ Incentive measures	X₈₄	Pilot demonstration projects	X₈₅	Publicity and education	X₈₆	Talent training and introduction
X₉ Policy evaluation	X₉₁	Sufficient basis	X₉₂	Clear aim	X₉₃	Scheme science
X₉ Policy evaluation	X₉₄	Detailed planning

Parameter identification. Binary rules are used to set parameters, and the same weight is set for all parameters. To ensure the objectivity of the assignment, the text mining method is used to evaluate the two-level index variables, and the ROSTCM software is used to separate the segmentation of each policy text. According to the keywords of each policy text, the two-level index variables are assigned. The value 1 indicates that the variable requirements are met, and the value 0 indicates that the variable requirements are not met, as shown in the Equations 1 and 2. The two-level index variables are subject to the distribution of [0,1]. However, this assignment rule cannot be applied to mutually exclusive secondary index variables, and there will be equal assignment of index variables. In this study, the parameter settings of the second-level indicator variables of the first-level indicator variable policy type (X₁), policy timeliness (X₂), and issuing agency (X₄) are assigned item by item in the range of [0,1], as shown in Table 3.

X ~ N [0, 1]

(1)

X = {XR : [0, 1]}

(2)

Table 3.

Indicator Variable Decrement Assignment.

Primary variables	Secondary variables	Assignment
X₁ Policy type	X₁₁ Plan for the class	1
	X₁₂ Plan detailed rules for the class	0.8
	X₁₃ Class methods	0.6
	X₁₄ Opinion decided to class	0.4
X₂ Policy timeliness	X₂₁ 1–3 years	1
X₂ Policy timeliness	X₂₂ 3–5 years	0.8
	X₂₃ More than 5 years	0.6
X₄ Post agency	X₄₁ Multi-sectoral joint mandate	1
	X₄₂ The provincial people issued a document	0.8
	X₄₃ The provincial government issued a communication	0.6
	X₄₄ Provincial Office issued a document	0.4
	X₄₅ Papers by other departments	0.2

Construct a Multi-Input-Output Table

The multi-input-output table is an analytical framework that can store a large amount of data and objectively analyze a single indicator variable from multiple dimensions. Combined with the characteristics of the digital economy policy in Heilongjiang Province, a multi-input-output table is constructed, as shown in Table 4.

Table 4.

Multi-Input-Output Table.

Primary variables	Secondary variables
X₁	X₁₁;X₁₂;X₁₃;X₁₄
X₂	X₂₁;X₂₂; X₂₃
X₃	X₃₁;X₃₂; X₃₃; X₃₄
X₄	X₄₁;X₄₂; X₄₃; X₄₄; X₄₅
X₅	X₅₁;X₅₂;X₅₃;X₅₄;X₅₅;X₅₆X₅₇;X₅₈;X₅₉;X₆₀
X₆	X₆₁;X₆₂;X₆₃;X₆₄;X₆₅;X₆₆
X₇	X₇₁;X₇₂;X₇₃;X₇₄
X₈	X₈₁;X₈₂;X₈₃;X₈₄;X₈₅;X₈₆
X₉	X₉₁;X₉₂;X₉₃;X₉₄

AE Parameter Fusion Process

The AE parameter fusion process is divided into three steps as shown in Figure 3: The first step is to encode the data to obtain the hidden layer data; in the second step, the hidden layer data is decoded to obtain the output layer data; in the third step, conduct multiple experiments with the minimum difference between the input data and the output data as the standard to obtain the best constant term.

Figure 3.

AE parameter fusion process.

Calculate the PMC-AE Index

In this study, the Sigmoid function is used as the activation function of the encoding and decoding process, and the data in the multi-input-output table is nonlinearly fused, as shown in Equation 3.

$g$ represents the Sigmoid activation function, $W$ and $W'$ prime represent different weight values respectively, $X_{n}$ and $Y_{n}$ represent the input value and output value of secondary index variables respectively, $n$ represents the number of secondary index variables, $Y_{n}$ and $X_{n}$ are approximately equal, the hidden layer $H_{n}$ is obtained and scored as the first-level index variable. The value range is set to [ 0.97, 1). After many experiments with Python, it is concluded that the convergence effect is the best when $q_{1}$ and $q_{2}$ are 1.11 and 1.22 respectively.

{\begin{matrix} H_{n} = g (W X_{n} + q_{1}) \\ Y_{n} = g (W^{'} H_{n} + q_{2}) \end{matrix}

(3)

PMC-AE Surface Drawing

The PMC-AE surface map is a three-dimensional surface map that can visually display the policy evaluation score and comprehensively evaluate the advantages and disadvantages of each dimension of the policy. The PMC-AE matrix is the premise of constructing the PMC-AE surface. Based on 9 first-level indicator variables, the PMC-AE matrix of 3 × 3 is constructed, where $P_{i}$ $(P_{i} is P_{1}, P_{2}, P_{3} \cdot \cdot \cdot P_{23})$ is the policy serial number, as shown in Equation 4.

P_{i} = [\begin{matrix} X_{1} & X_{2} & X_{3} \\ X_{4} & X_{5} & X_{6} \\ X_{7} & X_{8} & X_{9} \end{matrix}]

(4)

Sample

Based on the advantages of the PMC-AE index model, this study does not need to follow specific rules when selecting policy evaluation samples. Taking the digital economy policy issued by Heilongjiang Province from January 1, 2021, to May 31, 2022, as the evaluation object, a total of 23 policy texts are shown in Table 5.

Table 5.

23 Digital Economic Policies in Heilongjiang Province.

Numbering	Date of publication	Heading
P₁	2022.05.13	Heilongjiang Province to promote the development of big data application regulations
P₂	2022.03.22	Notice of the People’s Government of Heilongjiang Province on Printing and Issuing the “14th Five-Year” Digital Economy Development Plan of Heilongjiang Province
P₃	2022.03.22	Circular of the General Office of the People’s Government of Heilongjiang Province on Printing and Issuing Several Policy Measures to Support the Development of Creative Design Industry in Heilongjiang Province
P₄	2022.03.22	Circular of the General Office of the People’s Government of Heilongjiang Province on Printing and Issuing Several Policy Measures to Support the High-quality Development of Biological Economy in Heilongjiang Province
P₅	2022.03.22	Notice of the General Office of the People’s Government of Heilongjiang Province on Printing and Issuing Several Policy Measures to Support the Accelerated Development of Digital Economy in Heilongjiang Province
P₆	2022.03.16	Circular of the General Office of the People’s Government of Heilongjiang Province on Printing and Issuing Several Policy Measures for Promoting Industrial Revitalization in Heilongjiang Province
P₇	2022.02.28	Guiding Opinions of the General Office of the People’s Government of Heilongjiang Province on Building a High-level Export Consumer Goods Processing Zone
P₈	2022.02.24	Notice of the General Office of the People’s Government of Heilongjiang Province on Printing and Issuing Several Measures for Further Promoting the Development of Scientific and Technological Innovation in Heilongjiang Province
P₉	2022.01.28	Implementation Rules for “Promoting the Construction of Digital Longjiang” and Accelerating the High-quality Development of Digital Economy (Trial)
P₁₀	2022.01.24	Circular of the General Office of the People’s Government of Heilongjiang Province on Printing and Issuing the Implementation Plan for the Action Plan for the Scientific Literacy of the Whole People in Heilongjiang Province (2021–2025 )
P₁₁	2021.12.31	Notice of the People’s Government of Heilongjiang Province on Printing and Issuing Several Supporting Policies and Measures for Investment Promotion and Capital Introduction of Cultural Tourism Industry in Heilongjiang Province
P₁₂	2021.12.31	Heilongjiang Provincial People’s Government on the issuance of Heilongjiang Province’s “14th Five-Year” digital government construction plan notice
P₁₃	2021.12.30	Notice of the General Office of the People’s Government of Heilongjiang Province on Printing and Issuing Several Measures for the Construction of Rural Delivery Logistics System
P₁₄	2021.12.20	Notice on the issuance of “Heilongjiang” 14th Five-Year Plan for Agricultural Science and Technology Development
P₁₅	2021.12.16	Notice on issuing the “14th Five-Year Development Plan for Agricultural Mechanization in Heilongjiang Province”
P₁₆	2021.10.26	Heilongjiang Provincial Market Supervision and Administration Bureau on the issuance of “Heilongjiang Province,” 14th Five-Year “market supervision planning” notice
P₁₇	2021.10.20	Circular of the People’s Government of Heilongjiang Province on Printing and Issuing Several Policy Measures to Promote the Construction of “Digital Longjiang” and Accelerate the High-quality Development of Digital Economy
P₁₈	2021.09.30	Circular of the People’s Government of Heilongjiang Province on Printing and Issuing the Medium and Long-term Scientific and Technological Development Plan of Heilongjiang Province (2021–2035)
P₁₉	2021.09.30	Notice of the People’s Government of Heilongjiang Province on Printing and Issuing the 14th Five-Year Plan for Scientific and Technological Innovation in Heilongjiang Province
P₂₀	2021.09.25	Notice of the General Office of the People’s Government of Heilongjiang Province on Several Measures for Further Expanding Consumption
P₂₁	2021.07.22	Opinions of the General Office of the People’s Government of Heilongjiang Province on Accelerating the Promotion of Agricultural Science and Technology Innovation
P₂₂	2021.07.15	Several Opinions of the People’s Government of Heilongjiang Province on Supporting the Construction of Heilongjiang Jian san jiang National Agricultural High-tech Industry Demonstration Zone
P₂₃	2021.12.31	Notice of the People’s Government of Heilongjiang Province on Printing and Issuing the 14th Five-Year Plan for Optimizing the Business Environment in Heilongjiang Province

Data Analysis

According to the index calculation steps of the above PMC-AE index model, the ROSTCM software is used for information mining to establish a multi-input-output table of digital economic policies in Heilongjiang Province and calculate the PMC-AE index, as shown in Table 6.

Table 6.

Heilongjiang Province Digital Economic Policy PMC-AE Index.

Numbering	X₁	X₂	X₃	X₄	X₅	X₆	X₇	X₈	X₉	PMC-AE index	Billing
P₁	0.523	0.897	0.941	0.669	0.994	0.995	0.976	0.923	0.993	7.911	7
P₂	0.871	0.717	0.976	0.502	0.999	0.998	0.993	0.995	0.993	8.044	6
P₃	0.523	0.717	0.941	0.334	0.949	0.980	0.941	0.997	0.993	7.375	19
P₄	0.523	0.897	0.941	0.334	0.981	0.995	0.941	0.995	0.993	7.600	13
P₅	0.523	0.897	0.871	0.334	0.994	0.995	0.941	0.923	0.993	7.471	16
P₆	0.523	0.717	0.976	0.334	0.971	0.997	0.871	0.782	0.993	7.164	22
P₇	0.349	0.717	0.941	0.334	0.971	0.923	0.871	0.923	0.993	7.022	23
P₈	0.523	0.897	0.976	0.334	0.949	0.998	0.941	0.923	0.993	7.534	15
P₉	0.697	0.897	0.976	0.836	0.994	0.923	0.941	0.980	0.993	8.237	2
P₁₀	0.697	0.717	0.976	0.334	0.971	0.998	0.941	0.995	0.993	7.622	12
P₁₁	0.523	0.717	0.941	0.502	0.949	0.995	0.976	0.995	0.993	7.591	14
P₁₂	0.871	0.717	0.993	0.502	0.998	0.998	0.993	0.997	0.993	8.062	4
P₁₃	0.523	0.897	0.941	0.334	0.949	0.995	0.941	0.782	0.993	7.355	21
P₁₄	0.871	0.717	0.993	0.167	0.993	0.998	0.993	0.997	0.993	7.722	10
P₁₅	0.871	0.717	0.993	0.167	0.997	0.995	0.993	0.998	0.993	7.724	9
P₁₆	0.871	0.717	0.993	0.167	0.994	0.980	0.976	0.998	0.993	7.689	11
P₁₇	0.523	0.897	0.976	0.502	0.994	0.998	0.993	0.997	0.993	7.873	8
P₁₈	0.871	0.897	0.993	0.502	0.997	0.998	0.993	0.997	0.993	8.241	1
P₁₉	0.871	0.897	0.976	0.502	0.997	0.995	0.993	0.998	0.993	8.222	3
P₂₀	0.523	0.897	0.976	0.334	0.949	0.923	0.871	0.923	0.993	7.389	17
P₂₁	0.349	0.897	0.941	0.334	0.949	0.995	0.941	0.980	0.993	7.379	18
P₂₂	0.349	0.717	0.871	0.502	0.981	0.998	0.976	0.980	0.993	7.367	20
P₂₃	0.871	0.717	0.993	0.502	0.997	0.997	0.993	0.997	0.993	8.060	5
Verage	0.636	0.803	0.961	0.407	0.979	0.986	0.956	0.960	0.993	7.681	-

The policy evaluation level of this study is divided into three categories: I, II, and III, as shown in Table 7.

Table 7.

Policy Evaluation Level.

Interval	Grade	Policy
[8,9]	I	P₁₈; P₉; P₁₉; P₁₂; P₂₃; P₂
[7.5,8)	II	P₁; P₁₇; P₁₅; P₁₄; P₁₆; P₁₀; P₄; P₁₁; P₈
[6,7.5)	III	P₅; P₂₀; P₂₁; P₃; P₂₂; P₁₃; P₆; P₇

Results

Overall Result Analysis

The average PMC-AE index of digital economic policy in Heilongjiang Province is 7.681, which indicates that the overall quality of digital economic policy in Heilongjiang Province is good, and the policy formulation is scientific. According to the evaluation grade, six policies belong to class I, nine policies belong to class II, and eight policies belong to class III. Based on the overall perspective, the planning and implementation are in line with the actual situation, but there are differences in the specific aspects of evaluation.

According to Figure 4, the PMC-AE index scores of policy nature (X₃), policy content (X₅), participants (X₆), policy areas (X₇), incentive measures (X₈), and policy evaluation (X₉) have significant advantages. Policy type (X₁), policy timeliness (X₂), and issuing agency (X₄) are the basic attributes of the policy. The secondary indicator variable is a single choice, so the index score is poor. Based on the sample analysis of policy evaluation, Heilongjiang’s digital economy policy tends to be a macro planning outline policy, with medium and long-term policy timeliness and a single department as the main issuing agency. Therefore, it provides a reference direction for policy improvement in the future.

Figure 4.

Radar chart of the mean value.

Single Policy Analysis

Due to the nonlinear fusion of scores, the final score is not limited to the result range of the PMC index model. According to the characteristics of the activation function, the higher the PMC-AE index value, the higher the policy quality. The PMC-AE index of 23 digital economic policies ranked from high to low: P₁₈>P₉>P₁₉>P₁₂>P₂₃>P₂>P₁>P₁₇>P₁₅>P₁₄>P₁₆>P₁₀>P₄>P₁₁>P₈>P₅>P₂₀>P₂₁>P₃>P₂₂>P₁₃>P₆>P₇.

Considering the limitation of space, according to Equation 4, the top three policies from class I, class II and the last three policies from class III are selected to construct the PMC-AE matrix of digital economic policies, as shown in Table 8. According to the PMC-AE matrix of Table 8, the PMC-AE surface diagram of nine digital economic policies is constructed. The results are shown in Figures 5 to 12. The numbers 1, 1.5, 2, 2.5, and 3 in PMC-AE surface chart represent the coordinate values of the x-axis and Y-axis of the matrix. Different colors represent different values of index scores. The convex part of the surface graph indicates that the policy scores higher on the corresponding evaluation index, and the concave part indicates that the policy scores lower on the corresponding evaluation index. We can use the PMC-AE surface chart and the specific scores of each representative policy to analyze the strengths and weaknesses of each policy.

Table 8.

Heilongjiang Digital Economy Policy PMC-AE Matrix.

Grade	PMC-AE matrix
I	$P_{18} = [\begin{matrix} 0.871 & 0.897 & 0.993 \\ 0.502 & 0.997 & 0.998 \\ 0.993 & 0.997 & 0.993 \end{matrix}]$	$P_{9} = [\begin{matrix} 0.697 & 0.897 & 0.976 \\ 0.836 & 0.994 & 0.923 \\ 0.941 & 0.980 & 0.993 \end{matrix}]$	$P_{19} = [\begin{matrix} 0.871 & 0.897 & 0.976 \\ 0.502 & 0.997 & 0.995 \\ 0.993 & 0.998 & 0.993 \end{matrix}]$
II	$P_{1} = [\begin{matrix} 0.523 & 0.897 & 0.941 \\ 0.669 & 0.994 & 0.995 \\ 0.976 & 0.923 & 0.993 \end{matrix}]$	$P_{17} = [\begin{matrix} 0.523 & 0.897 & 0.976 \\ 0.502 & 0.994 & 0.998 \\ 0.993 & 0.997 & 0.993 \end{matrix}]$	$P_{15} = [\begin{matrix} 0.871 & 0.717 & 0.993 \\ 0.167 & 0.997 & 0.995 \\ 0.993 & 0.998 & 0.993 \end{matrix}]$
III	$P_{13} = [\begin{matrix} 0.523 & 0.897 & 0.941 \\ 0.334 & 0.949 & 0.995 \\ 0.941 & 0.782 & 0.993 \end{matrix}]$	$P_{6} = [\begin{matrix} 0.523 & 0.717 & 0.976 \\ 0.334 & 0.971 & 0.997 \\ 0.871 & 0.782 & 0.993 \end{matrix}]$	$P_{7} = [\begin{matrix} 0.349 & 0.717 & 0.941 \\ 0.334 & 0.971 & 0.923 \\ 0.871 & 0.923 & 0.993 \end{matrix}]$

Figure 5.

The PMC-AE surface of P_18.

Figure 6.

The PMC-AE surface of P_9.

Figure 7.

The PMC-AE surface of P19.

Figure 8.

The PMC-AE surface of P_1.

Figure 9.

The PMC-AE surface of P_17.

Figure 10.

The PMC-AE surface of P_15.

Figure 11.

The PMC-AE surface of P_13.

Figure 12.

The PMC-AE surface of P_6.

Based on the micro perspective, combined with the policy PMC-AE surface diagram, the above nine policies are analyzed in depth:

The scores of P₁₈, P₉, and P₁₉ were 8.241, 8.237, and 8.222 respectively, and the evaluation grade was class I. The score of P₁₈ ranked first, as shown in Figure 5. The scores of all index variables were greater than the mean. The policy, issued by the provincial government, covers a wide range of fields and is a long-term guiding policy committed to the development of science and technology. The score of P₉ ranked second, as shown in Figure 6. The scores of the two indicator variables of the participant (X₆) and the policy area (X₇) are lower than the average. The policy is a policy document jointly issued by multiple departments, and the participants are mainly governments and enterprises, which are not rich enough; The policy field mainly involves the technical level and system construction, and other aspects are less involved. The score of P₁₉ ranked third, as shown in Figure 7. The scores of all index variables were greater than the mean. The policy, issued by the provincial government, is rich in content and diverse in incentives. It is a planning policy committed to scientific and technological innovation.

The scores of P₁, P₁₇, and P₁₅ were 7.911, 7.873, and 7.724 respectively, and the evaluation grade was class II. The score of P1 ranks 7th, as shown in Figure 8. The scores of policy type (X₁), policy nature (X₃), and incentive measures (X₈) are lower than the average, while the scores of other index variables are higher than the average. This policy belongs to the policy of methods and regulations, and the incentive measures are not rich enough, mainly in the form of publicity and education, and talent training. The score of P₁₇ ranked 8th, as shown in Figure 9. Only the score of policy type (X₁) is lower than the average, and the scores of other index variables are higher than the average. This policy belongs to the method regulation policy, and the policy strength is weak.

The score of P₁₅ ranked 9th, as shown in Figure 10. The scores of policy timeliness (X₂) and issuing agency (X₄) are lower than the average, while the scores of other index variables are higher than the average. The policy was issued by the Heilongjiang Provincial Department of Agriculture and rural areas, which is inclined to technological innovation in the field of agriculture, and is a guiding planning policy.

The scores of P₁₃, P₆, and P₇ were 7.355, 7.164, and 7.022 respectively, and the evaluation grade was class III.

The score of P₁₃ ranked 21st, as shown in Figure 11. The scores of the three index variables of policy timeliness (X₂), participants (X₆), and policy evaluation (X₉) were higher than the average. The policy is inclined to the development of rural logistics, combining digital technology, building a digital platform, and commitment to social services, but there is still room for improvement in the development environment and system construction.

The score of P₆ ranked 22nd, as shown in Figure 12. The scores of policy nature (X₃), participants (X₆), and policy evaluation (X₉) are higher than the average. The policy focuses on promoting the transformation and upgrading of industrial digitalization, and puts forward relevant measures in infrastructure construction and platform construction, while there is still room for improvement in system construction and social services.

The score of P₇ ranked 23rd, as shown in Figure 13. Only the score of policy evaluation (X₉) is higher than the average. The policy focuses on the construction of high-level export consumer goods processing zones with digital empowerment, but there is still room for improvement in the participants and system construction.

Figure 13.

The PMC-AE surface of P_7.

Discussion

The digital economy is an important part of the modernization framework of the economic system, and its high-quality development cannot be separated from the support of an effective supporting policy system. This study takes the text of digital economy policy in Heilongjiang Province as the research object, constructs the evaluation model of digital economy policy, and uses the text mining method and content analysis method to quantitatively evaluate and analyze the digital economy policy in Heilongjiang Province from both macro and micro perspectives. The results show that the digital economy policy in Heilongjiang Province is at an excellent level as a whole, but it needs to be optimized in terms of policy type, policy timeliness, and policy nature. This study provides a reference for the formulation, revision, and implementation of digital economy policy in Heilongjiang Province in the future.

The purpose of this study is to provide policy recommendations for policymakers to formulate a new round of digital economy policies. Based on the experimental results, this study puts forward the following seven policy suggestions:

In terms of policy types, there are signs that the development speed of the digital economy in Heilongjiang Province has improved since 2021. The digital economy policies that have been issued are mainly macroplanning policies. To further implement the policy implementation, policymakers should strengthen policy support in terms of specific implementation measures in the future.

In terms of policy timeliness, the policies are mainly in the medium and long term, and there are fewer short-term policies. At the same time, there are fewer digital economy-related policies involving the short, medium, and long term. The short-term policy and the medium-term policy are conducive to the implementation of the policy, while the long-term policy focuses on the overall planning. The coordination among the three makes the implementation of the policy more scientific.

In terms of policy nature, the policy focuses on encouragement and guidance. In future policy-making, it is suggested to combine guiding policies with regulatory policies. This measure not only guides the development of the digital economy but also establishes the corresponding regulatory mechanism to promote the implementation process of the policy.

In terms of document issuing agencies, documents are mainly issued by a single department. It is suggested that policymakers should strengthen inter-departmental cooperation in the future policy-making process, establish a reasonable departmental coordination mechanism through effective inter-departmental communication, and then reduce policy conflicts.

In terms of participants, the participants are mainly governments and enterprises, and the participation of universities, scientific research institutions, and social organizations is insufficient. It is suggested that policymakers should pay more attention to multi-agent participation in the process of policy formulation and improvement in the future, and promote in-depth cooperation between government, industry, University, and Research Institutes, to create a good development atmosphere and speed up the development of the digital economy.

In the policy field, focuses on improving digital technology, but social services are relatively less concerned. To promote the social responsibility of relevant enterprises, the development of the digital economy needs to give appropriate policy guarantees.

In terms of incentives, government subsidies, technical support, publicity, and education are the main measures. In the process of policy formulation and improvement in the future, policymakers need to pay more attention to fiscal and tax support, deal with the relationship between talent cultivation and talent introduction, and uphold the concept of talent cultivation as the main and talent introduction as the auxiliary.

In addition, this study puts forward policy suggestions for specific government departments from four aspects of digital industrialization, industrial digitization, digital governance, and data value:

In the aspect of digital industrialization, it is necessary to promote the sharing of data resources among scientific research institutions, colleges and universities, and enterprises, and improve the innovation ability of digital core technology (Wei et al., 2022). The education department should recognize the importance of talent training in digital economy-related disciplines, and develop talent training programs that can meet the new requirements of society for the development of the digital economy.

In the aspect of industrial digitization, we should pay attention to the integration of the digital economy and real economy, and promote the digital transformation of traditional industries (Dana, Salamzadeh, Mortazavi, Hadizadeh, & Zolfaghari, 2022). The Industry and Information Department is the leading department of industrial digitalization. It should promote the development of a platform economy, strengthen the data resource sharing ability between enterprises, and also pay attention to the digital upgrading of key industries, such as manufacturing, transportation, medical care, and other industries. At the same time, it is necessary to strengthen the coordination between departments. For example, the financial department can further launch industrial digital special funds and tax benefits in the future to provide financial support for digital transformation. It is recommended that the authorities ensure the standardization of taxation through strategies based on trust and power (L. Batrancea et al., 2019; L. M. Batrancea et al., 2022).

In terms of digital governance, it is necessary to improve the digital governance capacity and promote the construction of multi-agent cooperative governance. Market regulators should make full use of digital technology, build a digital governance platform (Hanisch et al., 2023), promote the participation of governments, public welfare institutions, universities, enterprises, and other multi-agent, and further improve the digital regulatory system.

In terms of data value, build a multi-level data market system, further improve the market order of data elements, and promote the orderly sharing of transaction data. The marketization of data elements is to take data as a kind of element resource, and trade, circulate and configure it through the market mechanism. The data trading platform has gradually become the core facility of digital economic resources trading (C. Li et al., 2022), but at present, the development of the national data trading market is unbalanced and insufficient. The government needs to accelerate the construction of a national unified data element market, break the barriers between regions, departments, and industries, build a multi-level data market system, and better release the value of data elements with the help of the data trading platform.

Conclusion

This conclusion will cover theoretical implications, practical implications, limitations, and future research avenues.

Theoretical Implications

The results of this study have multiple theoretical significance.

It broadens the research vision of the digital economy. At present, the research on the digital economy policy mainly focuses on the impact of the policy, while the research on the evaluation of the digital economy policy text itself is relatively insufficient. Therefore, from the perspective of policy evaluation, this study quantitatively evaluates the digital economy-related policies issued by Heilongjiang Province of China, analyzes the shortcomings of the policies, and puts forward policy optimization suggestions.

It enriches the research object of policy quantitative evaluation. It is more urgent for regions with slow economic development to promote high-quality economic development and common prosperity by improving the policy environment. This study takes the digital economy policy text of Heilongjiang Province in China, which is relatively slow in economic development, as the research object for quantitative evaluation, which enriches the research object of quantitative evaluation of policy.

It enriches the evaluation method of the digital economy policy text. The PMC-AE index model used in this study is relatively rare in digital economic policy research. Therefore, it can provide some reference for the construction of a digital economic policy evaluation system, and to a certain extent, it can provide new research ideas for future digital economic policy research.

Practical Implications

The practical significance of this study is mainly divided into two aspects.

It is of great significance to optimize the policy and institutional environment of the digital economy in Heilongjiang Province. Firstly, this study uses the ROSTCM text mining software to extract word segmentation and word frequency statistics, which is conducive to objectively and find the current focus in the field of digital economy policy in Heilongjiang Province; Then the PMC-AE index model is used to quantitatively evaluate the existing digital economy policies, to sort out the current policy and institutional environment, analyze its advantages and disadvantages, and put forward policy suggestions for the government functional departments, providing new ideas for optimizing the digital economy policy and institutional environment.

It provides a reference for the development and construction of a digital economy in other economically disadvantaged areas. The difficulties and challenges Heilongjiang province needs to eliminate in developing the digital economy may also appear in other economically disadvantaged areas. Therefore, our research on the development of the digital economy in Heilongjiang Province and the analysis of its policy and institutional environment has important reference significance for the construction of other areas with slow economic development.

Limitations and Future Research Avenues

Although the design of this empirical study is more rigorous, there are also some shortcomings: first, in terms of research methods, to reduce the subjectivity of the PMC-AE index model in the selection of evaluation indicators, we can use a variety of methods to mine the characteristics of policy text, such as LDA theme model, and then optimize the index system and enhance the universality of PMC-AE index model. Second, in terms of data sources, the data of this study selected the digital economy policy text of Heilongjiang Province, China. There are many differences between China’s digital economy development and other countries. Therefore, the amount of experimental data needs to be further expanded to improve the objectivity and scientificity of the research results.

In future research, we can conduct quantitative evaluation and comparative research on the digital economy policy texts of different countries, analyze the common points and differences of digital economy policies of various countries, and promote the complementary advantages of digital economy development among countries. In addition, the PMC-AE index model can be combined with the qualitative comparative analysis method (QCA). The qualitative comparative analysis method (QCA) can further study which factors affect the operation effect of digital economy policy design and then explore the configuration path of good development of the digital economy. The combination of the two methods can be used to investigate the digital economy policy from the perspective of policy text design and policy operation effect. This is of great significance to improving the digital economy policy system.

Footnotes

Acknowledgements

We thank the editors for their excellent editorial guidance and helpful comments from anonymous reviewers.

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This study was funded by the Heilongjiang Provincial Philosophy and Social Science Research Planning Project (20GLE385).

ORCID iD

Yi Yang

Data Availability Statement

Data sharing not applicable to this article as no datasets were generated or analyzed during the current study.

References

Aniqoh

N. A. F. A.

(2020). The role of digital economy to enhancing sustainable economic development. International Journal of Social Science and Business, 4(4), 519–527.

Aturin

Moga

Smagulova

(2020). Digital transformation management: scientific approaches and economic policy. Upravlenets, 11(2), 67–76.

Batrancea

Nichita

Olsen

Kogler

Kirchler

Hoelzl

Weiss

Torgler

Fooken

Fuller

Schaffner

Banuri

Hassanein

Alarcón-García

Aldemir

Apostol

Bank Weinberg

Batrancea

Belianin

… Zukauskas

(2019). Trust and power as determinants of tax compliance across 44 nations. Journal of Economic Psychology, 74, 102191.

Batrancea

L. M.

Nichita

De Agostini

Batista Narcizo

Forte

de Paiva Neves Mamede

Roux-Cesar

A. M.

Nedev

Vitek

Pántya

Salamzadeh

Nduka

E. K.

Kudła

Kopyt

Pacheco

Maldonado

Isaga

Benk

Budak

(2022). A self-employed taxpayer experimental study on trust, power, and tax compliance in eleven countries. Financial Innovation, 8(1), 96.

Bukht

Heeks

(2017). Defining, conceptualising and measuring the digital economy. Development Informatics working paper, (68).

Cheng

Zhang

Wang

Jiang

(2023). The impact of the urban digital economy on China’s carbon intensity: spatial spillover and mediating effect. Resources Conservation and Recycling, 189, 106762.

Chen

Cheng

Ciuriak

Kimura

Nakagawa

Pomfret

Schwarzer

(2019). The digital economy for economic development: Free flow of data and supporting policies. RIETI highlight, (75), 33–44.

China Academy of Information and Communications Technology. (2022). China digital economy development report. Beijing:China Academy of Information and Communications Technology. China Economic Review, http://www.caict.ac.cn/kxyj/qwfb/bps/202207/t20220708_405627.htm (in Chinese)

Chen

(2020). Improving market performance in the digital economy. China Economic Review, 62, 101482.

10.

Christie

C. A.

Lemire

S. T.

(2019). Why evaluation theory should be used to inform evaluation policy. American Journal of Evaluation, 40(4), 490–508.

11.

Dahlman

Mealy

Wermelinger

(2016). Harnessing the digital economy for developing countries. OECD Publishing.

12.

Dai

Zhang

Zong

Wang

(2021). How effective is the green development policy of China’s Yangtze River Economic Belt? A quantitative evaluation based on the PMC-index model. International Journal of Environmental Research and Public Health, 18(14), 7676.

13.

Dana

L. P.

Salamzadeh

Hadizadeh

Heydari

Shamsoddin

(2022). Urban entrepreneurship and sustainable businesses in smart cities: Exploring the role of digital technologies. Sustainable Technology and Entrepreneurship, 1(2), 100016.

14.

Dana

L. P.

Salamzadeh

Mortazavi

Hadizadeh

(2022). Investigating the impact of international markets and new digital technologies on business innovation in emerging markets. Sustainability, 14(2), 983.

15.

Dana

L. P.

Salamzadeh

Mortazavi

Hadizadeh

Zolfaghari

(2022). Strategic futures studies and entrepreneurial resiliency: A focus on digital technology trends and emerging markets. TEC Empresarial, 16(1), 87–100.

16.

Estrada

M. A. R.

(2009). The ceteris Paribus assumption vs. Omnia Mobilis assumption: Economic note. Ssrn Electronic Journal.

17.

Estrada

M. A. R.

(2011). Policy modeling: Definition, classification and evaluation. Journal of Policy Modeling, 33(4), 523–536.

18.

Foster

Azmeh

(2020). Latecomer economies and national digital policy: An industrial policy perspective. Journal of Development Studies, 56(7), 1247–1262.

19.

Guo

Wang

Zhang

Liang

Feng

(2023). Impact of the digital economy on high-quality urban economic development: Evidence from Chinese cities. Economic Modelling, 120, 106194.

20.

Hanisch

Goldsby

C. M.

Fabian

N. E.

Oehmichen

(2023). Digital governance: A conceptual framework and research agenda. Journal of Business Research, 162, 113777.

21.

Hojeghan

S. B.

Esfangareh

A. N.

(2011). Digital economy and tourism impacts, influences and challenges. Procedia - Social and Behavioral Sciences, 19, 308–316.

22.

Huang

Wang

Yang

Bao

(2020). Exploring the key driving forces of the sustainable intergenerational evolution of the industrial alliance innovation ecosystem: Evidence from a case study of China’s TDIA. Sustainability, 12(4), 1320.

23.

Jiao

Sun

(2021). Digital economic development and its impact on economic growth in China: Research based on the perspective of sustainability. Sustainability, 13(18), 10245.

24.

Kayser

Blind

(2017). Extending the knowledge base of foresight: The contribution of text mining. Technological Forecasting and Social Change, 116, 208–215.

25.

Kling

Lamb

(1999). IT and organizational change in digital economies: A socio-technical approach. ACM SIGCAS Computers and Society, 29(3), 17–25.

26.

Kuang

Han

Zhang

Fan

(2020). Quantitative evaluation of China’s cultivated land protection policies based on the PMC-index model. Land Use Policy, 99,105062.

27.

Lasswell

H. D.

(1951). The immediate future of research policy and method in political science. American Political Science Review, 45(1), 133–142.

28.

Liang

Zhang

Wang

Q. E.

Luo

(2022). Blockchain-based data trading in edge-cloud computing environment. Information Processing & Management, 59(1),102786.

29.

Kim

D. J.

Lang

K. R.

Kauffman

R. J.

Naldi

(2020). How should we understand the digital economy in Asia? Critical assessment and research agenda. Electronic Commerce Research and Applications, 44, 101004.

30.

Chen

Xiao

Wang

(2023). Digital economy, financing constraints, and corporate innovation. Pacific-Basin Finance Journal, 80, 102081.

31.

Liu

(2022). Quantitative evaluation of waste separation management policies in the Yangtze River delta based on the PMC index model. International Journal of Environmental Research and Public Health, 19(7), 3815.

32.

Liu

(2022). Quantitative evaluation of high-tech industry policies based on the PMC-Index model: A case study of China’s Beijing-Tianjin-Hebei region. Sustainability, 14(15), 9338.

33.

Miljand

(2020). Using systematic review methods to evaluate environmental public policy: Methodological challenges and potential usefulness. Environmental Science & Policy, 105, 47–55.

34.

Okpalaoka

C. I.

(2023). Research on the digital economy: Developing trends and future directions. Technological Forecasting and Social Change, 193, 122635.

35.

Peng

Chen

Q. K.

Chen

K. X.

(2020). Evaluating the consistency of long-term care insurance policy using PMC index model. DEStech Transactions on Computer Science and Engineering, 45, 34219.

36.

Poland

O. F.

(1974). Program evaluation and administrative theory. Public Administration Review, 34(4), 333–338.

37.

Prior

Hughes

Peckham

(2012). The discursive turn in policy analysis and the validation of policy stories. Journal of Social Policy, 41(2), 271–289.

38.

Ren

Lee

S.-J.

(2023). Digitalization improves enterprise performance: New evidence by text analysis. Sage Open, 13(2), https://doi.org/10.1177/21582440231175871

39.

Romanova

O. A.

(2018). Industrial policy priorities of Russia in the context of challenges of the fourth industrial revolution. part 1. Ekonomika Regiona= Economy of Regions, 2, 420.

40.

Shokhnekh

A. V.

Telyatnikova

V. S.

Pozdeev

V. L.

Nosov

A. L.

Mushketova

N. S.

(2020). Concept strategy of innovative small business in the system of economic security of the digital economy. Growth poles of the global economy: Emergence, changes and future perspectives, 73, 1397–1406.

41.

Suchman

(1968). Evaluative research: Principles and practice in public service and social action programs. Russell Sage Foundation (pp. 314–321).

42.

Tapscott

(1996). Promise and peril in the age of networked intelligence. McGraw-Hill Education.

43.

Wang

Xing

(2022). Evaluation of the wind power industry policy in China (2010–2021): A quantitative analysis based on the PMC Index Model. Energies, 15(21), 8176.

44.

Wang

Dong

Taghizadeh-Hesary

(2022). Assessing the digital economy and its carbon-mitigation effects: The case of China. Energy Economics, 113, 106198.

45.

Wei

Liu

(2022). The effects of information technology capability and knowledge base on digital innovation: The moderating role of institutional environments. European Journal of Innovation Management, 25(3), 720–740.

46.

Wollmann

(2007). The development of a sustainable development model framework. Energy and Policy Research, 31, 69–75.

47.

Xiong

Zhang

(2023). How effective is the fire safety education policy in China? A quantitative evaluation based on the PMC-index model. Safety Science, 161, 106070.

48.

Yang

Yin

Cui

Mao

Sun

Jia

Zhao

Wang

Liu

Ren

(2022). Quantitative evaluation of traditional Chinese medicine development policy: A PMC index model approach. Frontiers in Public Health, 10, 1041528.