Abstract
In October 2023 the 21st International Conference of Labour Statisticians adopted a new set of statistical standards on the informal economy, incorporating new definitions on different concepts related to informality and identifying a wide indicator framework. The process to develop those standards placed a central focus on improving the range of gender relevant data on the topic, drawing strength from recently adopted standards from the 19th and 20th ICLS on different forms of work, labour underutilization and work relationships. Beginning in 2021 the ILO launched a project with a range of workstreams but with a key focus on ensuring that the new standards would be sufficiently engendered, and that measurement guidance would be generated to support their timely implementation. A major part of this work was the organisation of comprehensive pilot studies in Uganda and Peru, involving a mix of qualitative and quantitative testing methods and a dedicated test of the impact of proxy response. This paper describes the evolution of statistical standards since 2013, highlighting their incremental and interrelated nature, providing a new and transformed platform for labour statistics. The paper further describes the pilot studies completed in Uganda and Peru and key findings emerging on the measurement of informality and heavily related topics through the labour force survey.
Keywords
Introduction
In 2018, at the 20th International Conference of Labour Statisticians (ICLS) delegates recognised the need to develop a clearer and more consistent framework for statistics on the informal economy, replacing existing standards adopted in 1993 and 2003. At the same time delegates noted the need to “ensure consistency with the 19th and 20th ICLS resolutions covering work, employment and work relationships”. 1 The mandate from the ICLS also recognised the necessity “to advance work on the development of measurement guidance and tools in parallel with work on new conceptual frameworks in order to ensure that timely guidance can be provided”. This process commenced with the establishment of a technical working group in 2019.
In continuation of longer-term trends in statistical standards on labour (as described further below), an important concern in the development of the new standards was to ensure they were sufficiently engendered, promoting the provision of data to more adequately understand and monitor changes in gender gaps in the world of work. With this objective in mind, and with support from the Bill & Melinda Gates Foundation, the ILO launched a project entitled ‘Engendering Informality Statistics’ in 2021.
A key output from this work is the recently updated labour force survey model questionnaire, that acts as the base for ILO technical assistance and capacity building for labour force surveys i. , and which will be incrementally expanded over time. The remainder of this paper describes the work undertaken under this project including the process to engender the statistical standards and the experimental pilot study work undertaken in Uganda and Peru to provide evidence for the published model questionnaire content. The main conclusions from this work and future steps are also described.
Engendering statistical standards on labour – an incremental process
The process of improving the gender relevance of labour statistics through statistical standards has evolved over an extended period of time and the inter-connectedness of different standards make it necessary to understand that evolution in order to establish how it has impacted statistics on informality.
One general notable trend in statistical standards on labour has been to slowly evolve from a primarily economic focus in statistical standards, to a more a more balanced focus attempting to meet both economic and social statistical requirements. For example, the standards adopted at the 13th ICLS in 1982 in the Resolution concerning statistics of the economically active population, employment, unemployment and underemployment established definitions of the economically active population, employment and unemployment with one of the objectives being to provide a measure of employment which corresponded to labour input within the system of national accounts (SNA) production boundary. This concept of employment (if faithfully applied) would incorporate various unpaid activities such as work done by persons “engaged in the production of economic goods and services for own and household consumption…. If such production comprises an important contribution to the total consumption of the household”. 2
The 1982 standards became increasingly widely applied in measurement over time and the statistics generated from surveys applying them did come to be used for both economic (e.g., productivity analysis) and social (e.g., monitoring unemployment and the impact of employment promotion policies) purposes. The 1982 standards also became the base for a range of subsequent statistical standards and classifications such as the International Classification of Status in Employment adopted in 1993 (ICSE-93) – still broadly applying a predominantly economic focus by separating all employment between workers in employment for pay and workers in employment for profit - a distinction aligned with the requirements of the SNA.
At the same time as adopting ICSE-93 the 15th ICLS also adopted the first statistical standard on the topic of informality through the resolution on statistics of employment in the informal sector, which, in the same manner as the 13th ICLS resolution was explicitly framed around the SNA with the definition of household enterprises stating “For statistical purposes, the informal sector is regarded as a group of production units which, according to the definitions and classifications provided in the United Nations System of National Accounts (Rev.4), form part of the household sector as household enterprises or, equivalently, unincorporated enterprises owned by households.”. 3
While these resolutions generally made reference to the dual social and economic value of indicators based on their concepts a more explicit social reference point became reflected over time. In 2003 the 17th ICLS adopted a checklist of good practices for mainstreaming gender in labour statistics, which, while not changing existing statistical concepts or definitions, promoted the production of more comprehensive statistics that would cover “all relevant topics for describing gender concerns” and “adequately describe all workers and work situations in sufficient detail to allow relevant gender comparisons to be made”. 4
A significant shift in the base of statistical standards on labour was signalled at the 18th ICLS in 2008 – whereby calls were made to undertaken work to provide statistics to represent the “changing structure of the labour force, including a revision of the International Classification of Status in Employment (ICSE-93) and a possible revision of the current international standards on statistics of the economically active population, employment, unemployment and underemployment adopted by the 13th ICLS (1982);”. 5
Among the motivations expressed by delegates for a fundamental review of the main reference standards was the need for statistics that would recognise unpaid work more explicitly, and the need for a framework to address perceived limitations in the unemployment rate as a sole headline indicator on labour underutilization. An underlying theme of those motivations was the need for a statistical framework that more adequately captured differences in the working activities and labour market engagement of women and men. The discussion at the 18th ICLS provided the mandate under which a working group was established to develop standards to replace those adopted at the 13th ICLS, thereby resetting the base of labour statistics standards and shifting the balance further towards meeting social statistical needs, while continuing to place a high priority on the need for economic statistics.
19th ICLS Resolution I – statistics on work, employment and labour underutilization
The process to engender and “socialise” labour statistics standards was projected forward with the adoption of resolution I of the 19th ICLS.
6
There were multiple inter-related developments that, in combination, are transformative, including:
The adoption of an overarching definition of work, comprising all activities done to produce goods or services for own-consumption or consumption by others. This definition was wider than the 1982 definition of employment, now including activities to produce services outside the SNA production boundary but within the general production boundary such as unpaid domestic and care work (called own-use provision of services in ICLS terms) and direct volunteering to provide services to people outside the household or family. The identification of multiple forms of work based on the dual criteria of beneficiary of the good or service produced and the intention underlying the work. Within the forms of work employment assumed a narrower definition than that adopted in 1982 – now focussed on work done in return for pay or profit, thereby separating some activities that were previously defined as employment as other forms of work such as own-use production of goods which includes subsistence farming among other activities. The definition of multiple indicators of labour underutilization, including but also complementing the unemployment rate.
The significance of these developments for labour statistics, and in particular their gender relevance cannot be overstated. The forms of work framework identifies and promotes for measurement a range of working activities that have often been invisible and predominantly performed by women, with own-use provision of services in particular incorporating child-care and various other unpaid domestic services.
Alongside the narrowing of the definition of employment this simultaneously promises to shed light on differences in the work burdens of women and men, and provide a more meaningful representation of gaps in participation in different working activities, such as a truer reflection of participation gaps in work for pay or profit – generally showing wider gaps due to the relatively greater participation of women in unpaid work that had previously been included within employment.7,8
The 19th ICLS resolution became the base for a new round of standard setting, and, as already proposed at the 18th ICLS 5 years earlier, the 19th ICLS called on the ILO to lead a process of development of a replacement for ICSE-93, observing the “need to ensure coherence with other conceptual frameworks for related statistics including those on decent work and informal employment, the system of national accounts and the proposed 19th ICLS resolution on work statistics”. 9 This ultimately led to the adoption of Resolution I of the 20th ICLS and an updated ICSE-18 5 years later.
At the 20th ICLS, while calling on the ILO to lead a process to develop statistical standards on the informal economy, participants made a similar request to “ensure consistency with the 19th and 20th ICLS resolutions covering work, employment and work relationships”. 1
In this manner, given the very high gender relevance and focus of the 19th ICLS standards, its use as a reference for subsequent standards has served to engender the 20th and 21st ICLS standards to an extent. For example, the lens of work relationships (20th ICLS standards) and informality (21st ICLS standards) could be applied to all forms of work. However, on its own this would be insufficient to ensure that the scope and detail of the standards would promote a framework of statistics to properly understand gender gaps within the domains covered. Rather, this would require a dedicated focus to ensure both that the definitions developed and the range of indicators proposed were developed in a gender intentional manner. It would also require work “on the development of measurement guidance and tools in parallel with work on new conceptual frameworks in order to ensure that timely guidance can be provided”. 1 It was in this context that the Engendering Informality Statistics Project was launched in January 2021.
The engendering informality statistics project and pilot studies in Uganda and Peru – design and objectives
As briefly described in the introduction the Engendering Informality Statistics project was developed through discussions between the ILO and the Bill & Melinda Gates Foundation in recognition of the value of data on informality for the assessment of women's economic empowerment. The ongoing statistical standard setting process would be a unique opportunity to ensure that gender considerations were mainstreamed in the new standards, and, while this would in any case be a theme of discussions in the working group, a project with a dedicated focus on this promised to add significant value.
The initial project proposal identified multiple overlapping workstreams including:
Work to understand the extent of gaps in informality statistics from a gender perspective; Work to support the ongoing development of statistical standards for discussion at the 21st ICLS, in particular to ensure those standards would be of high gender relevance, both in the definitions provided and in the range of indicators proposed; and Work to develop and test measurement approaches that could be integrated in labour force surveys, covering a range of gender relevant indicators related to the topic of informality.
Under the first stream of work, a review was done of the existing range and use of data on informality. This highlighted a range of opportunities to improve the relevance and availability of data on informality, and for better linkages to policy development and monitoring. While noting that there had been a positive trend in the availability and use of data on informality, major data gaps and poor data to policy linkages remained. Among other things the review noted that “analysis with an intersectional lens has found that women face structural and social barriers in access to credit, technology, business services, training, and the market compared to men” and highlighted “the need for indicators that compare between informality and formality but also to show how conditions within informal work compare between women and men.”.
10
The review fed into the other two workstreams, for example informing discussions on the range of indicators highlighted in the resolution adopted at the 21st ICLS ii. . With respect to the the workstream to develop and test measurement approaches, the review informed the range of topics built into the test process such as access to finance, ICT usage and others required to characterise informality and allow meaningful intersectional analysis, in addition to the inevitable focus on methods to accurately identify informality.
Design of the pilot studies
Undertaking pilot studies in parallel to (or based on) the revision of statistical standards brings major benefits. Such tests create the opportunity to develop and try different measurement approaches, reflecting proposals emerging from the standard setting discussions, while also allowing reconsideration of those proposals in the light of evidence on the measurement challenges they would create.
ILO accumulated experience in pilot testing to apply latest statistical standards through several rounds of studies, the first of which was initiated in 2015 to assess methods to apply the standards adopted at the 19th ICLS, initially with a central focus on those elements that would have the greatest impact on key labour market indicators, but also to measure some of the forms of work newly identified in the forms of work framework iii. .
This first round of pilot studies was followed up through several additional phases of tests building incrementally on guidance developed from the first phase which included model LFS questionnaires and supporting guidance that were launched in 2018 iv. . These new rounds of studies expanded the focus to additional forms of work including volunteer work v. and own-use provision of services vi. , as well as the implementation of ICSE-18 and a study focussed on the implementation of the 19th ICLS standards in different types of household surveys vii. .
The studies generally used a similar range of methodologies to apply a mix of qualitative and quantitative testing approaches. Each phase included cognitive testing as a primary method of qualitative assessment of questionnaire content, in some cases supplemented by additional qualitative assessments through focus groups or rapid ethnographic assessments.
Qualitative tests were used to refine questionnaires that were subsequently used for quantitative tests of different scale, but generally with a split-sample methodology whereby different sets of content were tested on different samples of households selected with the same sample design.
As proposed by Presser et al., 11 split-sample tests can be used to compare the outcomes of different survey questionnaires if all aspects of the sampling, methodology and implementation, other than the questionnaires, are the same. The analytical approach follows those proposed by Fowler 12 whereby statistics are compared for the concepts covered by both questionnaires. Where differences in key selected outputs are observed, further analysis is used to attempt to identify the source. This comparative approach has been found to be valuable in several experiments to identify potential improvements to questionnaire content viii. .
Under the Engendering Informality Statistics project it was initially envisaged to undertake multiple rounds of qualitative testing in two countries, followed by quantitative testing in one country. The country selected for both qualitative and quantitative testing was Uganda, given a desire to undertake testing in a country with a high prevalence of informal employment and a diverse range of informal activities in both urban and rural areas. The additional country selected for qualitative testing was Peru, a country with a lesser albeit still high prevalence of informality. As the project progressed sufficient funding became available to also enable a round of quantitative testing in Peru meaning that the scope of studies was ultimately expanded with the significant added value of more detailed assessment possibilities across two settings ix. .
Before the qualitative testing could take place the ILO developed questionnaires for testing covering the latest standards and available model questionnaires, with the addition of a range of topics selected based on reviews of available literature, consultations and existing surveys on informality such as mixed surveys
x.
. The range of topics selected on this basis were:
Identification of informality in line with emerging proposals from the working group Identification of dependent contractors (based on ICSE-18 as established in Resolution I of the 20th ICLS) Identification of contributing family workers (ICSE-18) plus measurement of their motivation for working in a family business Motivation of independent workers (as defined by ICSE-18) for operating a business Decision making in family businesses – this is related to ICSE-18 but also relevant to the understanding of agency Earnings of “independent workers” as defined by ICSE-18 Earnings of “dependent workers” as defined by ICSE-18 Asset ownership in business (types and valuation) ICT use in businesses (including digital platforms)
Additionally – later in the project it was decided to add the topic of access to finance among business operators but this was only included for the last round of quantitative testing in Peru.
The qualitative testing work took place between August and December 2021 based on the timeline presented in Table 1 below:
Key milestones and dates for the qualitative testing.
Source: Statistical Methodology Series 9 - Engendering informality statistics - Overview and methodology https://www.ilo.org/media/481621/download
Cognitive interviews were based on purposive samples of 40 participants per round of interviews for a total of 80 interviews in each country (160 overall total). The sample design targeted similar numbers of women and men in both urban and rural areas, covering different types of jobs, but with a primary focus on informal work. Age and education levels were used as secondary criteria. Unlike in the first round of ILO pilot studies only employed individuals were targeted as the testing focussed on questions that would be asked to employed respondents only (e.g., identification of informal employment).
Following the cognitive interviews various updates were made to generate a complete questionnaire for quantitative testing. Three rounds of quantitative testing took place in the second half of 2022, two waves in Uganda and one in Peru. In the case of Uganda the same households were revisited for the second wave in order to enable longitudinal analysis.
Purposive sampling was used for the quantitative phases with districts/regions for testing selected to ensure a mix of urban and rural areas, but with oversampling in urban areas with the intention to cover a wide variety of informal jobs and businesses, while still covering the main agricultural crops in each country. In Uganda, over 1600 households were interviewed in wave 1 and roughly the same number in wave 2. The test in Peru was a smaller scale with just over 800 households interviewed (see Table 2).
Designed and achieved sample of quantitative tests plus fieldwork timing.
Source: Statistical Methodology Series 9 - Engendering informality statistics - Overview and methodology https://www.ilo.org/media/481621/download
Two slightly different questionnaires were tested in each round referred to as approach A and approach B. The starting point of the questionnaires were ILO's published model labour force surveys (job-type start xi. ) with some adjustments plus the addition of topics considered to be of high relevance to the study of informality and gender. The large majority of questions were identical between approach A and B in order to focus analysis on the key selected topics - consequently content differed for about 10% of the questionnaires or approximately 30 questions.
While the samples of households for the quantitative tests were selected on the same basis it was nonetheless possible that differences in the achieved samples of individuals could impact the comparability of estimates between approach A and approach B in each round of the tests. To minimise the impact of this it was decided to apply a calibration approach whereby the benchmark population was an average between the two samples disaggregated by age group, location (urban/rural in both countries and additionally district in Uganda) and sex. In practice weights applied were close to 1 as the achieved samples were reasonably balanced but nonetheless the weighting reduced potential sources of incomparability and is considered to have been worthwhile.
A further feature of the studies was a dedicated test of the impact of proxy interviewing. This test was implemented in the second round of quantitative collection in Uganda and the round that was implemented in Peru. The objective was to achieve both a direct and proxy interview for at least one household member in each household (for households with at least 2 members). Comparisons of the information collected was expected to provide insight on the impacts proxy reporting had on various estimates. The proxy test was completed with the benefit of updates to the CSPro applications, whereby the software identified relevant household members for which the proxy could be completed. Clear protocols and training for the interviewers also supported implementation with the result that 591 successful test interviews were completed in Uganda and 530 in Peru, covering a range of respondent types.
The detailed findings on the different topics listed in the previous section are being incrementally published by the ILO and are extensive. Some key issues of priority have been selected for presentation in this paper.
Links to previous studies and interrelation of topics
Before assessing the results from the pilot studies in Uganda and Peru, a first point to highlight is that, while the different studies undertaken over time have had a different focus, the topics covered are inevitably interrelated. As one example, the first round of ILO pilot studies that commenced in 2015 and the study in Sri Lanka demonstrated that care is required to ensure that employment, as defined in the 19th ICLS resolution is captured comprehensively and accurately.
The first round of studies demonstrated that, in the countries covered in the studies, multiple questions are needed to identify employment activity in the reference week and that women's employment in particular was at risk of being undercounted if too few questions were used. 21 This is a finding that has also been reflected in other studies.15,22–25
The Sri Lanka study further highlighted this point whereby two different questionnaires yielded a more than twenty percent difference in measured employment among women – despite both questionnaires being designed to apply the 19th ICLS standards. Detailed analysis showed the issue to relate to differences in the identification of casual and part-time work, or work helping in family businesses and farms – with almost identical estimates for full-time employees. 21 In a subsequent round of testing, updates made to the questionnaire previously identifying less employment succeeded in narrowing the gap to approximately six percent. These updates took the form of dedicated recovery questions targeting help in family businesses or farms, or more casual work, already incorporated in the model LFS questionnaire.
Furthermore, the identification of informality, and the questions used to identify informal employment and the informal sector are directly related to the ICSE-18 category, as specific and separate operational definitions of informality are provided for different ICSE-18 categories. As such the work done in Sri Lanka to refine approaches to the application of ICSE-18 is inexorably linked to the measurement of informality, and thus informed the design of the studies in Uganda and Peru.
This emphasises that the different standards adopted in ICLSs since 2013 are linked, as is their proper application in household surveys. For example, if there is a significant misreporting of the level of employment, as shown for one of the questionnaires used in the Sri Lanka study, this can bias substantially estimates based on ICSE-18 or informality where the under-reporting is typically concentrated among those with casual jobs, contributing family workers or those in informal employment – in particular misrepresenting differences between women and men. Furthermore, the application of the 19th ICLS definition of employment requires the assessment of the main intended destination of production for those engaged in the production of goods on their own-account. Where the main intended destination of production is own-use then the activity is considered own-use production work, not employment – and in terms of informality under the new standards is considered to be part of the household, own-use and community (HOC) sector – not the informal or formal sectors.
Considering this, a holistic approach is needed to questionnaire development and testing, ensuring to apply all sets of standards and good measurement practices across the survey. With this said the next section of this paper focusses on the main findings from the studies in Uganda and Peru.
Identification of informality
Given that the topic of the 21st ICLS Resolution is statistics on the informal economy a key initial focus of the studies in Uganda and Peru was the set of questions used to identify informal employment and the informal sector. A review of practices undertaken by the ILO in preparation for the 20th ICLS had shown a degree of convergence across countries (although nowhere near complete) of practices in criteria that were operationalised in the identification of informality 26 xii. .
On the basis of this review initial discussions in the technical working group for the development of the new statistical standards sought to narrow down the key criteria that would be considered for recommendation. Through these discussions registration of the enterprise, institutional sector and the types of accounts maintained emerged as key proposed criteria for the identification of the formal/informal sector, as well as informal/formal employment among independent workers based on ICSE-18. Similarly, the key proposed criteria to identify informal/formal employment among dependent workers would be payment of job-related social contributions, access to paid annual leave and access to paid sick leave. For dependent contractors something of a hybrid approach was ultimately adopted in the standards, looking at a combination of registration and social contributions as a way to identify informality/formality.
The studies in Uganda and Peru therefore looked at testing approaches to the operationalisation of these criteria – initially through cognitive testing and subsequently through the quantitative tests.
Cognitive testing findings:
By and large the questions related to social contributions, registration, incorporation, book-keeping, paid annual leave and paid sick leave were generally found to be well understood by respondents. However, some findings did emerge that generated changes in the questionnaire content for quantitative testing, including:
It was found that in general relatively greater difficulties were reported by respondents with the understanding of incorporation than registration.
27
In both cases naming specific systems at the national level helped but beyond this some respondents indicated a lack of awareness of incorporation. In existing published ILO model questionnaires the question on incorporation had preceded the question on registration. It was decided to reverse the order of these questions for the quantitative tests, only asking the question on incorporation for those who said that the enterprise was registered – hopefully lowering the instance of don’t know responses with it being relatively more likely that if a respondent knew the enterprise was registered they may also know if it was incorporated. This change was ultimately found to operate well in quantitative tests and has been retained in newly published model questionnaires. Through cognitive testing a potential difficulty was identified with the understanding of ‘paid sick leave’ with the possibility that respondents may say yes if medical expenses were paid by their employer. On this basis it was decided to test two wordings of this question during quantitative tests, one a traditional wording asking about ability to take paid sick leave (Would you get paid sick leave in case of illness or injury?), the other applying a reverse logic asking whether pay would be reduced if they were on sick leave (Would you lose pay if you could not work due to illness or injury?).
27
During the quantitative testing assessment in most cases focussed on the perceived plausibility of results as well as feedback and observation of operation in the field. Almost universally questions were found to work well but with some notable reflections influencing the implementation of the questionnaire and data use. To give an example of an important finding it was clear that ‘don’t know’ responses were more common for certain questions related to informality and for certain groups. This is demonstrated in Figure 1 below which presents the responses received to questions on business registration by both independent and dependent workers.
Distribution of responses to questions relating to business registration by country, type of workers in employment and by type of response (direct or proxy). Source 27 : Statistical Methodology Series 11 - Identification of informality through labour force surveys – Frosch et al., 2023: https://www.ilo.org/publications/statistical-methodology-series-11-identification-informality-through-labour.
As shown in Figure 1 don’t know responses were minimal for independent workers who directly responded to the survey (0.6% in Uganda and 0.7% in Peru). However, rates of don’t know responses among independent workers became non-negligible (7.5% in Uganda and 9.2% in Peru) when proxy response was involved. Don’t know responses were more common among dependent workers – for example 12.5% even for direct interviews in Peru – illustrating the unsurprising lesser knowledge of employees of the status of the enterprise they are working in relative to the business operator. Don’t know frequency became particularly notable for dependent workers when proxy response was involved (28.5% in Peru).
This pattern was repeated to different degrees across informality related questions, being even greater for bookkeeping related questions, while somewhat lesser for questions to identify informality among employees (payment of social contributions, paid annual leave, paid sick leave). Further insight on this was drawn from the dedicated test of proxy effects illustrating that proxy response could lead to misreporting in different directions also confirming the higher prevalence of don’t knows in proxy versus direct interviews.
Multiple related conclusions can be drawn from this including but not limited to:
While proxy response remains an important mechanism to manage the cost and resource requirements for household surveys efforts should be made to reduce its prevalence to a reasonable degree, for example a limited number of revisits to increase the number of direct interviews. The impact of proxy response differs across topics, and in the case of informality likely to be of greatest impact in the identification of the formal and informal sector among dependent workers. Impacts on the sector appear to be less among independent workers, which is of note given that for independent workers the sector also determines whether the job is formal or informal. Related to this it is important to have a strategy in place to allow classification of cases where don’t know responses are received or more likely to be received. As a concrete example the new standards adopted at the 21st ICLS state that an enterprise will be formal if it has at least one formal employee – allowing an efficiency in questionnaire design whereby employees can be asked about social contributions, paid annual and paid sick leave first – and if identified as a formal employee it becomes unnecessary to ask about registration and bookkeeping as the enterprise will be automatically classified as formal. This reduces the reliance on questions where don’t know responses have been demonstrated to be more common while still allowing classifications to be made in line with the standards. Impacts on classification of formality appear to be potentially greater in countries where formality is relatively more common. For example, higher risks of misclassification due to proxy reporting were found in Peru than Uganda. The alternative wording for the question on paid sick leave did not ultimately seem to work well during quantitative testing, with reports from interviewers in the first round of tests in Uganda that the double negative wording of the question “Would you The strategy of combining different criteria and questions to assign a classification needs to be carefully considered. For example, the results showed it to be not uncommon for employers to pay some sick leave, whether or not paid annual leave was available. This led to a conclusion that paid sick leave, on its own may be too weak to indicate formality – thereby leading to a recommendation that both would need to be present, and that these criteria should only be relied on where the response to the questions on social insurance contributions was ‘don’t know’ – i.e., relatively reducing their importance but still allowing for them to recover cases of formal employment among a limited number of respondents. It has also been found to be important to ensure that, for questions where national systems exist - such as registration, taxation, social contributions – those systems should be named within the questions.
Considering all the results generated it is concluded that all the criteria explicitly identified in the new standards can be operationalised in labour force (or other) surveys, with the expectation being that the formulations now presented in the published model questionnaires are a good reference point for national implementation and should ensure accurate identification of informality as a platform for, among other things, a proper assessment of differences in prevalence of and experience in informality between women and men.
As discussed earlier the, ICSE-18 classification provides a much more powerful lens through which to measure and understand working relationships, than the previous ICSE-93 classification. When combined with the definitions from the 19th and 21st ICLS standards it forms a basis for a wide range of intersectional analysis touching on issues of recognition, social protection and control within employment. Reflecting the linkages between the different sets of standards the Uganda and Peru pilots not only presented an opportunity to test the application of questions to implement ICSE-18, but also their integration with questions to apply the standards on informality, which apply different criteria to identify informality depending on the ICSE-18 category. Indeed some questions, such as the question on incorporation, are directly relevant to the application of both sets of standards.
An extensive range of questions were included in the study questionnaires towards the application of the ICSE-18 classification including questions on type of remuneration, decision making within family businesses, contracts (type and duration), dependency and control, hiring of employees, incorporation and others. Several of these were already in relatively common use in labour force surveys (for example hiring of employees, contract details) while others were topics where new development was required such as those on dependency and control as a way to identify dependent contractors. While extensive results have been generated an overarching conclusion was that “multiple questions are required to properly assess working relationships and assign an appropriate category within ICSE-18 and a significant proportion of respondents can be reclassified from their self-reported status based on those additional questions”. 28
This conclusion was a reference to the relatively common existing practice to apply the ICSE-93 classification through a single self-identification question in labour force surveys. The more detailed nature of ICSE-18 and the wider range of criteria required to apply the definitions make this approach unrealistic – with, for example, it being impossible for respondents to accurately self-identify as ‘dependent contractors’ on a consistent basis.
Table 3 shows the redistribution of respondents from initial self-declaration to ICSE-18 category for the first wave of data collection in Uganda, whereby, depending on the initial self-declared status, a large proportion of respondent were ultimately classified differently by ICSE-18, for example 35.5% of self-reported employees were reclassified as dependent contractors.
Distribution of self-declared status in employment and ICSE18, Uganda round 1 (weighted data).
Distribution of self-declared status in employment and ICSE18, Uganda round 1 (weighted data).
Source 28 : Statistical Methodology Series 10 - Identification of status in employment ICSE-18 through labour force surveys– Frosch et al., 2023: https://www.ilo.org/publications/statistical-methodology-series-10-identification-icse-18-through-labour
These findings have a high gender relevance as some of the reclassifications (and the ultimate ICSE-18 classification) are heavily gender differentiated. Taking the case of contributing family workers, women were more likely to initially self-report as contributing family workers (for example for 15.7% of women versus 6.5% of men in the first wave of the Uganda study), but also significantly more likely to be reclassified because they reported having a decision making role in the business with the majority being reclassified as independent workers without employees (40% of female self-declared contributing family workers being reassigned this way in Uganda wave 1 compared with 20% of males). While this remained a more prevalent category for women than men in all rounds of the studies the gaps between women and men were narrowed by the additional questions, and based on the definitions from ICSE-18, a more appropriate reflection of the work relationship is presented. The pattern of reclassification itself is of interest, reflecting powerful gender norms, whereby women are more likely to perceive themselves as helpers in a family business, despite having an important decision making role.
The overarching conclusion drawn is that ICSE-18 can be applied through labour force surveys through a well designed sequence of questions and that, subject to good design, the additional questions required to any given respondent can be minimised while still rigorously applying the definitions.
Substantial updates to the published model questionnaires have been made reflecting the lessons learned.
The initial analytical focus following the pilot studies was on the identification of informality and ICSE-18 – given their status as recently adopted statistical standards. This was prioritised with a view to promoting and enabling the early uptake of the standards. However, as outlined earlier in the report additional topics were covered by the pilot studies – selected based on their relevance to the understanding of informality. These ranged across motivations for engaging in the work, earnings, asset ownership in businesses, access to finance and ICT access and use within businesses.
The analysis of these topics is continuing at the time of writing of this paper with findings to be published on an incremental basis. In addition, optional modules will be published for attachment to the main LFS questionnaire for countries interested in measurement.
A range of interesting conclusions are emerging on these topics, with a general perspective that covering them is both valuable and plausible within a labour force survey, albeit that some topics are relatively more sensitive and complicated to measure well than others. This is notably true for asset ownership (particularly valuation) and earnings (particularly of independent workers) and while initial guidance will be published further work can be envisaged.
In the case of earnings of independent workers, an example of the emerging conclusions would be that it is important to set a context for questions on earnings, rather than immediately asking for amounts of profits earned, and that it is crucial to attempt to match the reporting period to something that can be related to the respondent – for example asking about a season for a seasonal worker, while asking for a daily, weekly or monthly period for others – at a frequency linked to their frequency of payment or sales. However, sensitivity and complexity to generate meaningful estimates remain clear, for example with a relatively large group of respondents reporting that they made neither a profit nor a loss in the previous month – with feedback from interviewers indicating that this was a reflection of a reluctance to report true values (even if known).
Dedicated test of proxy effect
As described in the methodology section of this paper, at the request of the technical advisory group for the Engendering Informality Statistics project, a dedicated test of proxy effects on reporting was embedded in the last round of tests in Uganda and Peru, yielding over 1000 double interviews (both proxy and direct interview for the same person). This test will be the subject of a forthcoming report but results indicate that inconsistency between direct and proxy interviews is indeed potentially significant – depending on the topic being covered. Inconsistencies can work in different directions and for some topics/variables aggregate level differences were minor, while for others they were more notable. For informality it was noted that relatively greater inconsistency was found in reporting on bookkeeping with a relatively large increase in don’t know reporting and a potentially significant decrease in ‘yes’ responses – particularly in Peru (see Figure 2).
Distribution of responses to questions relating to business registration by country, type of workers in employment and by type of response (direct or proxy). Source 27 : Statistical Methodology Series 11 - Identification of informality through labour force surveys – Frosch et al., 2023: https://www.ilo.org/publications/statistical-methodology-series-11-identification-informality-through-labour.
The impact of proxy reporting even extended to labour force status with a general tendency that proxy respondents were likely to report less activity (i.e., less likely to report employment), and that where activity was reported it was less likely to be formal.
These findings support a recommendation to implement a strategy to manage and reduce proxy response to the extent practical, through a limited number of scheduled revisits for example. Low levels of difference in proxy rates is unlikely to generate significant differences at aggregate level in key indicators but higher levels of proxy response could impact results up to and including the assignment of labour force status, with results from the studies indicating that this would be more likely to reduce than increase estimates of employment.
The statistical standards on labour have gone through a transformational phase, commencing in 2013, with the most recent addition being the standards on the informal economy adopted in October 2023. The various standards adopted in this period are inter-related and draw strength from each other, particularly with regard to the gender relevance of the data generated promoting intersectional analysis of the forms of paid and unpaid work people do, the working relationships and conditions involved and the recognition and social protection of those workers. When implemented the range of analytical possibilities are significantly enhanced from those offered by older standards.
Work done by the ILO in recent years has led to the development of an extensive range of guidance to countries as a support to implementation of the standards. Most recent studies undertaken in Uganda and Peru focussed on measurement based on the standards on the informal economy within a labour force survey. This project aimed both to generate good measurement guidance and provide input to the ongoing standard setting on statistics on the informal economy – with a particular aim to ensure the standards and measurement guidance would be sufficiently engendered so as to provide meaningful and comprehensive data to assess differences in the working experiences of women and men.
A wide range of evidence was generated upon which existing published model questionnaires, tools and guidance were updated. The tools provide a strong platform for national measurement, but whether using those tools or not countries should, if they have not already done so, plan for the implementation of the latest standards through all sources of labour data, but in particular in a labour force survey. The strong interrelation between these topics extends to measurement and there are major synergies to be found in implementing all three sets of standards at once.
The cycle of standard setting continues to evolve and the latest ICLS asked the ILO to organise working groups to develop statistical standards on digital platform work and the care economy, two topics with high current interest and gender relevance. Work to advance the development of standards on these topics will take place in parallel with work on measurement, again building on the base provided by existing standards and guidance.
Footnotes
Funding
The author received no financial support for the research, authorship, and/or publication of this article.
Declaration of conflicting interests
The author declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.