Review of National Surveys And Censuses
Go Back 3.1: BACKGROUND AND MOTIVATION
Disability is a complex and evolving concept that can be defined in various ways. Definitions shape the questions that are used in household surveys and censuses to identify persons with disabilities (The 2021 Disability Data Report Appendix 3 Method briefs #1 and #2 ). This section provides an analysis of the disability questions in national censuses and household surveys globally between 2009 and 2021.
Although there are tools and recommendations to collect self-reported disability data (Altman 2016; United Nations 2017), the lack of disability questions in surveys in data sets collected before the COVID 19 pandemic (Mitra et al. 2022 2022a, Mitra and Yap 2021) may have continued during the pandemic. This could explain the lack of attention to the impact of the pandemic on persons with disabilities.
With the onset of the COVID-19 pandemic, face-to-face survey data collection efforts came to a halt. As a response, many countries launched longitudinal COVID-19 High-Frequency Phone Surveys (HFPS) programs to track their populations’ situation and evaluate the impacts of restricted mobility (or lockdowns) and other COVID-19 related policies. Some were conducted with the support of The World Bank, which now has a COVID-19 Household Monitoring Dashboard (World Bank 2022). In addition, UN Women conducted rapid gender assessments in at least 52 countries between March 2020 and March 2021 (UN Women 2022).
Even as face-to-face surveys resume in 2022, phone surveys may continue to be more commonly implemented. Therefore, it is important to assess the extent to which HFPS programs have been disability-inclusive during the pandemic.
This section builds upon the work of the 2021 Disability Data Report (the 2021 Report thereafter) (Mitra and Yap 2021) and reviews 532 additional dataset-waves: these datasets include recent datasets for 2019 to 2021 and any additional datasets that were found for the 2009-2018 period. We zoom in on HFPS datasets, given their potential for providing insights into the effects of the pandemic and related policies. We also consider recently collected datasets under MICS6, given the extent of the survey. MICS is a major source of internationally comparable data on children and women ages 15 to 49 worldwide. For some countries, data on men ages 15 to 49 is also available.
3.2: METHODS
Survey and census questionnaires from 2009 to 2021 were retrieved via online repositories from various sources: the International Household Survey Network Microdata, the World Bank Microdata Library catalog, the International Labor Organization, the repository of census questionnaires maintained by the United Nations Statistics Division, the Food and Agriculture Organization (FAO) Microdata catalog, and websites of individual National Statistical Offices.
Among the datasets under review, a pool of 71 HFPS data sets and 223 dataset-waves from 55 countries were screened for disability questions. We also reviewed 42 MICS 6 surveys from 41countries with available and unrestricted data as of December 2021[1]. Other reviewed datasets included national censuses, labor force surveys, European Union Surveys of Income and Living Conditions (SILC), and miscellaneous surveys.
First, to identify if a survey asked any question at all on disability, each questionnaire is searched looking for several disability-related words: disability, difficulty in seeing, hearing, walking, concentrating, self-care, communicating, impairment (blind, deaf, dumb, mental, physical), limited in usual activit(y/ies), limited in the amount of work or type of work or activities related to work, Activities of Daily Living (ADL)s (walking, dressing, bathing, eating, walking, toileting, urination, and defecation) and questions related to limitations in Instrumental ADLs (IADLs) (housework, shopping, cooking, managing money, and taking medicine, phone).
If disability questions are found, they are categorized as follows:
(i) questions of the Washington Group Short Set (WG-SS) covering six domains (seeing, hearing, walking, cognition, self-care, communication);
(ii) other functional difficulty questions (only four or five of the domains[2] in (i), or not the exact wording as in the WG-SS questions and/or answers); and
(iii) other disability questions that include the following: ADLs, IADLs, broad activity limitation question (e.g. “are you limited in the kind of, or amount of, work you do due to a health condition or impairment?), general disability question (e.g. “do you have a disability?”), other types of questions (e.g. disability benefits, impairments).
Only questions per (i) WG-SS and (ii) other functional difficulty questions are considered internationally comparable questions on disability as recommended by the United Nations Principles and Recommendations for Population and Housing Censuses (United Nations 2017, p. 207). Together, (i) and (ii) are referred to as functional difficulty questions.
Altogether, including earlier results of the 2021 Report, the pool of censuses and surveys under review includes 1,032 datasets from 184 countries and territories (countries thereafter).
Of course, this review of datasets is not without limitations. First, the list of search terms is not exhaustive: terms that seem outdated but might continue to be used in surveys, such as handicap or crippled, were not included in the list.
Second, while comprehensive, the list of datasets reviewed is not exhaustive. Some surveys were not covered as their questionnaires were not available or were not in a language that the contributors could read (the review includes English, French, Portuguese, or Spanish). Some surveys may have been missed and were not covered, such as surveys focusing mainly on children. National data sets were prioritized, thus missing subnational data collection efforts.
Finally, a note of caution is needed in that we focused only on reviewing the questionnaires and had no information on the sampling frames and on how surveys were implemented in the field. It is possible that while certain questions may have been in a questionnaire, they may not have been asked during the field survey or fieldwork staff may not have been able to communicate with households headed by persons with disabilities. We also did not consider survey staff training on asking disability-related questions to the respondents. Despite these limitations, this study has important results.
3.3: RESULTS
The entire set of results is available in the Dataset Review Results Tables ( 
). Table 3.1 shows the share of countries and datasets with functional difficulty questions based on our review: 123 countries and 216 datasets under review have functional difficulty questions in their surveys or censuses. Separating countries and surveys with the WG-SS and other functional difficulty questions, 68 countries[3] and 100 datasets have the WG-SS while 77 countries and 116 datasets have other functional difficulty questions.
Table 3.1: Overall results of the dataset review
| Countries or datasets | Number of countries | Share of countries | Number of datasets | Share of datasets |
|---|---|---|---|---|
| Under review in the study | 184 | 100.0% | 1032 | 100.0% |
| With functional difficulty questions | 123 | 66.8% | 216 | 20.9% |
| – With the Washington Group Short Set (WG-SS) | 68 | 37% | 100 | 9.7% |
| – With other functional difficulty questions | 77 | 41.8% | 116 | 11.2% |
Source: Authors’ calculations based on dataset review
Notes: Functional difficulty questions could be the WG-SS or other functional difficulty questions. The number of
countries with functional difficulty questions is not the sum of the numbers of countries with the WG-SS and with
other functional difficulty questions as some countries have both.
Figure 3.1 below maps countries with data on functional difficulties, i.e. with the WG-SS or other functional difficulty questions from 2009 to 2021. As shown in Figure 3.1, functional difficulty questions tend to be available in Asia, North and Latin America, and many countries in Sub-Saharan Africa. Most countries in Europe and Central Asia do not have national datasets with functional difficulty questions.
Figure 3.1 below maps countries with data on functional difficulties, i.e. with the WG-SS or other functional difficulty questions from 2009 to 2021. As shown in Figure 3.1, functional difficulty questions tend to be available in Asia, North and Latin America, and many countries in Sub-Saharan Africa. Most countries in Europe and Central Asia do not have national datasets with functional difficulty questions.
Figure 3.1: Countries with and without functional difficulty questions in national censuses or surveys (2009-2021)
Table 3.2 shows results for HFPS. Only 19 of the 55 the countries with HFPS under review and 21 of the 71 HFPS datasets under review had at least one disability question. More than half of the HFPS data sets with disability questions have questions that are not about functional difficulties; in fact, they often had a general disability question or a question about diagnosed health conditions (not shown in the table). These question types are unable to capture the population with disabilities because of either cultural stigma associated with general disability questions or limited health care access that can hinder clinical diagnosis (Mont 2007).
As HFPS are often longitudinal surveys with multiple waves, Table 3.2 also includes the number and share of dataset-waves with relevant questions. Only 17% of dataset-waves have at least one disability question and 7% of dataset-waves have functional difficulty questions. Only a handful of countries are found to have multiple waves of functional difficulty questions.
There are some HFPS datasets that can be merged with datasets collected before the pandemic that contain the WG-SS. To our knowledge, this is the case for three Living Standards Measurement Survey (LSMS) datasets: Ethiopia, Malawi and Nigeria. However, such data does not reflect onsets or changes in functional difficulty during the pandemic. In addition, the HFPS may inadvertently miss households with functional difficulties as they are less likely to have a phone. It may thus focus on a subsample of households with functional difficulties that are less disadvantaged that the broader samples interviewed prior to the pandemic and may thus not be suitable to identify households with functional difficulties. We explored if this was the case and results are in Box 1.
Table 3.2: Results of the dataset review for COVID-19 High Frequency Phone Surveys (HFPS)
| Countries or datasets | Number of countries | Share of countries | Number of datasets | Share of datasets | Number of dataset-waves | Share of datasets-waves |
|---|---|---|---|---|---|---|
| COVID 19 High Frequency Phone Surveys under review | 55 | 100.0% | 71 | 100.0% | 223 | 100.0% |
| With at least one disability question of any kind | 19 | 34.6% | 21 | 29.6% | 37 | 16.6% |
| With functional difficulty questions | 9 | 16.4% | 9 | 12.7% | 15 | 6.7% |
| – With the Washington Group Short Set (WG-SS) | 2 | 3.6% | 2 | 2.8% | 5 | 2.2% |
| – With other functional difficulty questions | 7 | 12.7% | 7 | 9.9% | 10 | 4.5% |
Source: Own calculations based on dataset review described in the text.
Notes: Functional difficulty questions could be the WG-SS or other functional difficulty questions. Countries with HFPS with the WG-SS are Ghana and the United States. Countries with HFPS with other functional difficulty questions are Côte d’Ivoire, Eswatini, Ethiopia, Guinea, Malawi, Mali and Senegal and all conducted rapid gender assessments.
Table 3.3 shows results for 42 MICS6 datasets. We find that most of them (35) have the WG-SS and the remaining six did not have any disability questions for adults.
We do not breakdown results for other types of surveys separately. It should be noted that one large survey program, the Survey on Income and Living Conditions (SILC), was found to consistently have disability questions but not functional difficulty questions. The questions under use are a broad activity limitation known as the Global Activity Limitation Indicator[4] and a chronic health condition question, which are problematic in cross-country comparisons and in contexts with limited health care access.
Table 3.3: Results of the dataset review for MICS6
| Countries or datasets | Number of countries | Share of countries | Number of datasets | Share of datasets |
|---|---|---|---|---|
| MICS6 under review | 40 | 100.0% | 41 | 100.0% |
| With at least one disability question of any kind | 35 | 87.5% | 35 | 85.4% |
| With functional difficulty questions | 35 | 87.5% | 35 | 85.4% |
| – With the Washington Group Short Set (WG-SS) | 35 | 87.5% | 35 | 85.4% |
| – With other functional difficulty questions | 0 | 0.0% | 0 | 0.0% |
Source: Own calculations based on dataset review described in the text.
| Box 1: Can HFPS data matched with pre-pandemic data on functional difficulty be used to track households with disabilities during the pandemic?
Ethiopia’s High Frequency Phone Survey of Households (HFPS-HH) did not collect any information on the functional difficulty status of household members. HFPS-HH is based on a subsample of the respondents of the 2018/2019 round of the Ethiopia Socioeconomic Survey (ESS) (LSMS), who had phones and left phone numbers as part of the ESS and could be reached on the phone in 2020. Can the information collected in the ESS on functional difficulties be used to identify the disability status of households during the pandemic with HFPS-HH? The answer to such a question has implications for Ethiopia and maybe for other countries with similar HFPS and LSMS data with functional difficulties (Nigeria and Malawi). The 2018/19 ESS had collected functional difficulty information from individual respondents about a year prior to the pandemic and can be matched to the HFPS-HH using household identifiers. We ended up with A sample of 2,390 households with a known functional difficulty status as of 2018/19 and that could be matched with HFPS and tracked during the first six rounds of HFPS. Of course, the matched sample has disability identification errors in that households who had functional difficulties in 2018/19 but no longer did in 2020 would still be considered as having a difficulty and those with new onsets of functional difficulties since 2018/19 would not be identified as having a disability. In addition, a main concern for HFPS-HH is the low phone penetration rate, especially in rural areas. The HFPS-HH sample is therefore only representative of households who have access to phones in urban and rural Ethiopia (Ambel et al 2020). We compared households with functional difficulties in the matched sample of HFPS-HH and ESS 2018/19 (Appendix 2.1). Functional difficulty prevalence is five percentage points lower in HFPS-HH (23.5%) compared to ESS (28.7%). In addition, while households with functional difficulties in HFPS-HH are demographically overall similar to those in ESS 2018/19 (e.g., household size), they are socioeconomically less disadvantaged for a number of indicators, in particular food insecurity, clean fuel, electricity, adequate housing, and cell phone ownership. The concern is thus that the HFPS-HH respondents may be socioeconomically better off compared to the nationally representative ESS and may thus not be a valid sample to work with in order to track the situation of households based on functional difficulty status during the pandemic. |
3.4: DISCUSSION
Compared to the results of the 2021 Report, more than 50 additional countries were found to have functional difficulty questions in national datasets. In particular, the availability of the WG-SS has improved in the countries under review, especially thanks to MICS6 datasets collected between 2017 and 2021. Yet, the MICS6 data collected for disability could be improved. Indeed, they are not adequate to measure prevalence among all adults as the WG-SS is only asked among women in each household age 18 to 49 who can answer for themselves. There is a men’s questionnaire with the WG-SS only for some countries. When available, only one man aged 18 to 49 per household is randomly selected to answer the WG-SS questions.
A key finding is the quasi-absence of disability questions in COVID-19 High Frequency Phone Surveys. This makes it impossible to track the situation of households with disabilities during the pandemic or capture the onsets of disabilities that may result from the pandemic (Spinney 2022). One suggestion for countries that continue to conduct HFPS is to adopt internationally-comparable disability questions such as the WG-SS in their ongoing HFPS to monitor the impact of the pandemic and to inform policy responses on disability prevalence and on persons with disabilities. This result highlights the importance during crises of other forms of data collection such as with mixed methods and by stakeholders other than national statistics offices: for persons with disabilities, organizations of persons with disabilities (OPDs) played an important role during the pandemic in documenting the situation of their members (IDA 2021).
Overall, results suggest there has been progress in recent years with the growing availability of functional difficulty questions in national datasets and the use of WG-SS in particular. Of course, a lot of work remains to be done to implement Article 31 of the CRPD for States Parties “to collect appropriate information, including statistical and research data, to enable them to formulate and implement policies to give effect to the CRPD.” During the 2022 Global Disability Summit, national governments, multilateral organizations, organizations of persons with disabilities made 1,413 commitments related to disability inclusion (GDS 2022). The 96 disability inclusion commitments about data are steps in this direction.
[1] Two MICS surveys covered two regions of one country.
[2] Questions need to cover at least the four essential domains of functional difficulties (seeing, hearing, walking, cognition) and may also cover the self-care and/or the communication domains.
[3] This is less than the 111 countries that have reported to the Washington Group that they have adopted the WG-SS in their surveys or censuses (U. N. Statistical Commission 2022). To our knowledge, some countries may have modified the WG-SS questions and still reported using the WG-SS. This Report would then consider their questions as other functional difficulty questions and not as the WG-SS per se.
[4] “For at least the past 6 months, to what extent have you been limited because of a health problem in activities people usually do?”