2. Disability Questions and Measures

Go Back

This method brief is adapted from materials in Mitra, Chen et al (2021).

There are various options when it comes to methods for data collection and measurement on disability. One option is to collect information on environmental barriers to identify changes that are required in terms of physical and social barriers (Goodley 2016). Another data collection option is to have a qualitative and participatory exercise involving multiple stakeholders including persons with the lived experience of a disability. It may go a long way in understanding the situation of persons with disabilities and informing policy and advocacy efforts at one place and one point in time. However, it would not provide nationally representative or internationally-comparable data that are needed to ensure that persons with disabilities worldwide are not left behind post-2015. This method brief focuses on ways to collect data on disability through questions in household surveys and censuses.

2.1 Disability Questions in Household Surveys and Censuses

With surveys, an option is to develop a national disability survey with a battery of questions to measure disability and identify service needs and barriers, as well as to understand the factors leading to disability related inequalities. This is suitable provided significant resources and capacity for data collection are available and have the potential to provide very rich information to inform national or even subnational policy and advocacy efforts. The World Bank and the World Health Organization have developed a Model Disability Survey to this end (Cieza et al 2018; Groce 2019; WHO 2020). Few countries may, however, have the capacity or resources needed to develop and implement the stand-alone survey or include the brief version in an existing survey, let alone repeat this over time for monitoring the situation of persons with disabilities post-2015 and in the context of the CRPD.

Another option consists of using a few questions to measure disability in a population census as well as in general or mainstream household surveys, such as the Living Standards Measurement Study. A population census or a mainstream survey makes it possible to measure prevalence as well as inequalities across disability status for general indicators such as employment rates and educational attainment. Unlike the participatory and disability surveys above, it will not allow for an in-depth analysis of disability-specific barriers or challenges, nor facilitate understanding of the drivers of inequalities. Nevertheless, it may give very useful information when it comes to tracking the situation of persons with disabilities in different aspects of their lives. Of course, measuring disability with a few questions is challenging, especially if it has to be done in an internationally-comparable manner for global SDG and CRPD related monitoring efforts.

Measuring disability through household surveys and population censuses is a very complicated task and there is not a gold standard approach. How disability is measured depends on the objective of the measurement exercise. The goal might be to document disability prevalence and incidence, assess inequalities associated with disabilities, or evaluate service needs as well as policies and laws.

Besides varying underlying measurement objectives and conceptual definitions, there are different ways to collect disability data. These are “self-perceived and observed” data (Murray and Chen 1992). Self-perceived measures give an individual’s own perception of limitations, while observed measures rely on an external party’s assessment. Both types of measures provide complementary and valuable information (Murray and Chen 1992). There is typically no observed data in surveys and censuses in low- and middle-income countries. Thus, self-reported measures are used in this study and are therefore the focus of this report.

As can be anticipated, there are different ways to measure disability through self-reports (Mitra 2018; Palmer and Harley 2012). We describe below the different types of questions that have been commonly used in surveys and censuses: functional difficulties, activities of daily living, broad activity limitations, general disability/impairment questions and other questions.

Functional Difficulties

Functional difficulties refer to difficulties experienced with particular bodily functions such as seeing and hearing. The term is used to also include basic activities such as walking and daily activities such as feeding oneself. The United Nations Statistical Commission (United Nations 2015, 2017) adopted revised guidelines for the collection of disability data in national censuses. “It is suggested that only those domains that have satisfied a set of selection criteria be eligible for inclusion in a short set of questions recommended for use in censuses. Criteria for inclusion include cross-population or cross-cultural comparability, suitability for self-reporting and space on the census form. Other suggested criteria include the importance of the domain in terms of public health problems” (United Nations 2017). The Commission recommends that the following four functional domains be considered essential in determining disability status in a way that can be reasonably measured using a census and that would be appropriate for international comparison: (a) Walking; (b) Seeing; (c) Hearing; (d) Cognition. It also notes that two other domains, self-care and communication, have been identified for inclusion, and if possible, upper-body functioning is another domain that should be considered for inclusion. There have been efforts to generate internationally comparable and tested questions, notably by the Washington Group.

Washington Group Short Set

Specifically, the Washington Group has developed and tested a set of six questions, also known as the Washington Group Short Set (WGSS). The WGSS’ strength is in its brevity and is well-suited for use in censuses and national surveys. In conjunction with other data collected on outcome indicators (access to education or employment), it additionally helps to inform policy on equalization of opportunities. It starts with an optional introduction as follows: “The next questions ask about difficulties you may have doing certain activities because of a health problem.” It then has questions covering six different domains as follows: 1. Do you have difficulty seeing, even if wearing glasses? 2. Do you have difficulty hearing, even if using a hearing aid? 3. Do you have difficulty walking or climbing steps? 4. Do you have difficulty remembering or concentrating? 5. Do you have difficulty with self-care (such as washing all over or dressing)? 6. Do you have difficulty communicating? The WGSS uses a four-level scale (no difficulty, some difficulty, a lot of difficulty, or cannot do entirely) to capture individuals’ degree of functional difficulty in each of the six domains.

The WGSS has undergone extensive cognitive and field testing in multiple languages and locations (Madans, Loeb and Altman, 2011, Miller 2016). It is an internationally tested and widely accepted tool (Groce and Mont 2017). The UN Statistical Commission and the UN’s Economic Commission for Europe’s Council of European Statistics recommended the WGSS for collection of disability information for the 2020 round of censuses.

These questions on functional difficulties are not without limitations. Although tested in 14 countries (Miller 2016), an understanding of functional difficulties in the WGSS may be limited in a context with limited access to health care and may lead to underreporting (Schneider 2016). The WGSS covers only a few domains and may well under-identify people with psychological difficulties, a concern that may be alleviated to some extent in the WGSS-Enhanced, which is the short set and four additional questions on anxiety and depression, and another two on upper body mobility.

Other Functional Difficulty Questions

Functional difficulty questions may be found in questions that are different from those in the WGSS and in other international questionnaires. First, although the WG recommends that the WGSS be adopted as is, some data sets have included selected questions only, changed the answer scale or the wording of questions, and/or altered the introduction to the questions. There are thus data sets that have questions somewhat similar to the WGSS.

In addition, there are functional difficulty questions that are independent of the WGSS. Some internationally tested survey tools such as the Model Disability Survey (Cieza et al. 2018) or WHO-DAS 2.0 (WHO 2020) and other recent tools (e.g. Trani et al. 2015, Eide et al. 2006) do include functional difficulty questions. For instance, WHO-DAS 2.0 has questions on cognition, mobility, and self-care.

Activities of Daily Living

Disability has been measured using difficulties in Activities of Daily Living (ADL). There are different types of ADLs, including basic and intermediate. Basic ADLs are fundamental for body functioning (e.g. walking a specific distance) and include self-care activities such as feeding oneself, going to the bathroom without help, and dressing without help, bathing, eating, walking, toileting, urination and defecation. Intermediate ADLs are more complex tasks such as shopping, housekeeping, food preparation, and managing money.

ADL questions were initially developed to capture the physical effects of aging, and, as such, in some surveys the questions are only administered to respondents above a certain age (e.g. 45 and above). Intermediate ADLs include role activities, such as caring for others, that can be broad (Stewart and Ware 1992). In an international context, a limitation of such questions is in how daily activities, especially beyond basic ones, may vary considerably across contexts, and thus, one drawback of ADLs in international studies is their potential lack of comparability.

Broad Activity Limitation

A broad activity limitation question asks the respondent if he/she is limited in their usual activities due to a chronic health condition or an impairment, such as work or housework for adults or attending school for children. It relates to role functioning, the extent to which an individual performs or has the capacity to perform activities typical for a specific age and social responsibility (Stewart and Ware 1992). Because it is only one question, it is easy to insert in a survey. Similar to intermediate ADLs, usual activities vary in an international context, and answers may thus not be comparable across countries. A broad activity limitation question has additional drawbacks (Mitra 2018). Conceptually, such a question gets at a health condition or impairment as well as a potential resulting deprivation. Thus, it tries to get the causal link from the health condition/impairment to the deprivation, as perceived by the respondent. Respondents may not be aware of the ways that their health condition or impairment affects their broad activities. Responses may also be subject to different types of biases. For instance, the rationalization bias may encourage a person who does not work to report a health condition as the primary reason for non-employment, even if it is not. Another example is when people have adapted to their impairment in such a way that they no longer perceive how it affects their employment.

A broad activity limitation question makes it difficult to monitor disability over time. As an example, using a broad activity limitation question related to schooling among children: ‘is your child limited in the amount or the type of schooling you can have due to a physical, mental or emotional condition?’ Such a question does identify persons with perceived limited schooling opportunities due to a health condition. This question does not identify children with health conditions who have been able to access schooling, which is problematic. It is problematic that the outcome (employment or education) is part of the question. That makes it impossible to disaggregate across disability status. Using such a question, say in an environment where education becomes more inclusive through the provision of accommodations in schools, one would get a decline in the prevalence of disability over time but the negative correlation between schooling attendance and disability may persist or worsen as disability may include people with the more severe health conditions.

General Disability

Censuses and surveys sometimes have a general disability question such as “Do you have a disability?” This general question has sometimes been used with answers about disability types or impairments (e.g. blindness, deafness, paralysis) or with a yes/no answer as a screen for a follow-up question asking about disability type or impairment. Such a general question was very common in the censuses of the 1970s, 1980s, 1990s and 2000s (e.g. United Nations 1990).

The general disability question is problematic in terms of the validity of what it measures. First, it is unclear what such a general question measures as disability may mean different things to different interviewees. Additionally, impairments may be unknown to respondents, especially in the context of populations with limited access to health care or information. Furthermore, disability and impairments may be stigmatized and thus interviewees may not feel comfortable self-reporting . Finally, the general disability question tends to lead to underestimates of disability prevalence (Mont, 2007) as they capture severe impairments and tend to miss disabilities associated with old age as older people may not think of themselves as disabled but simply as older persons. For these reasons, it is not considered good practice to use the general disability question, let alone for the purpose of producing internationally-comparable statistics on disability.

Yet, likely due to its conciseness and a lack of awareness on its drawbacks, this question can still be found in some surveys and censuses and this paper examines in the next section to what extent this has continued to be the case in the past decade.

Other Disability-Related Questions

There are many other types of disability questions beside functional difficulties, ADL, broad activity limitations, and general disability questions. Some surveys such as the Model Disability Survey have questions on barriers people may face in their environment, whether physical, social, or attitudinal. These are particularly useful in understanding the drivers of difficulties or deprivations people may experience. In some countries, surveys or censuses ask respondents if they are part of an official registry of people with disabilities which would give them access to services or benefits. More often, surveys have questions on impairments and on specific health conditions.

Finally, while the WGSS was initially developed for use in censuses among those 5 years of age and older, it may not be adequate to capture disability among children (Adans et al 2018) . We therefore calculate disability indicators only for adults who are 15 years old and older, and their households.

2.2 Disability Measures

 

In order to determine prevalence or identify a specific ‘functional difficulty status’ group, a threshold needs to be set on the answer scale of functional difficulties. The WG recommends “a lot of difficulty” as the threshold: persons who report “a lot of difficulty” or “unable to do” for at least one domain are considered to have a disability, and persons with ‘no difficulty’ or ‘some difficulty’ to all six questions are deemed as not having a disability.

Different thresholds for the WGSS produce vastly different prevalence estimates (e.g. Bourke et al 2021; Mitra 2018). In addition, if persons with some difficulty are potentially more disadvantaged than persons with no difficulty, this categorization will underestimate the extent of inequalities between persons with and without disability. There is mounting evidence that having ‘some difficulty’ is significantly associated with economic and social deprivations with a gradient from no difficulty to some difficulty to at least a lot of difficulty (Banks et al 2014; Clausen and Barrantes 2020; Mitra 2018). Besides, several censuses under study use yes/no answers to the functional difficulty questions, making the breakdown recommended by the WG impossible. Hence, this study categorizes individuals in three ways.

A. First, for all data sets, it groups individuals into two categories:

• No functional difficulty in all domains;
• Any functional difficulty in at least one domain (answer Yes for data sets with yes/no answers, or reports at least ‘some’ difficulty for graded scales).

B. For data sets with graded scales, we partition individuals into three categories:

• No difficulty for all domains;
• Some difficulty in at least one domain but no “a lot of difficulty” or “unable to do”.
• A lot of difficulty or unable to do in at least one domain.

C. Following the recommendation of the WG for data sets that have a graded answer scale, we group individuals as follows:

• A lot of difficulty or unable to do in at least one domain;
• No difficulty or ‘some’ difficulty for all domains

Categorization B is more granular than A and C and may be able to identify a possible gradient in socioeconomic disadvantage with the severity of functional difficulties. It moves away from a binary understanding and measure of disability.

The analysis conducted at the household level categorizes households depending on the functional difficulty status of its members age 15 and older along the three ways of partitioning the population described above.

References

Adans J., Cappa C., Crialesi R., De Palma E., Loeb M., and Mont D. (2018). The Development and Testing of a Module on Child Functioning for Identifying Children with Disabilities on Surveys, I: Background. Disability and Health Journal, 11: 495-501.

Altman, B. M. (ed.) (2016). International measurement of disability: Purpose, method and application, the work of the Washington Group. Social indicators research series 61. Switzerland: Springer International Publishing.

Banks, M. L., and Polack S. (2014). The economic costs of exclusion and gains of inclusion of people with disabilities: Evidence from low- and middle-income countries. International Center for Evidence in Disability.

Bourke, J.A., Nichols-Dunsmuir, A. Begg, A., Dong, H. and Schluter, P.J. (2021). Measuring disability: An agreement study between two disability measures. Disability and Health. In press.

Cieza A., Sabariego C., Bickenbach J. and Chatterji S. (2018) Rethinking Disability. BMC Med. Vol. 16(1):14.

Clausen, J., and Barrantes, N. 2020. Implementing a Group-Specific Multidimensional Poverty Measure: The Case of Persons with Disabilities in Peru, Journal of Human Development and Capabilities, DOI: 10.1080/19452829.2020.1828316

Eide, A. H., and Loeb, M. (2006). Living conditions among people with activity limitations in Zambia. A representative national study. SINTEF Report STF78 A262. Oslo: SINTEF.

Goodley, D. (2016). Disability Studies: an Interdisciplinary Introduction. London: Sage, 2^nd edition.

Groce N. (2019). Comparing the Washington Group Questions and the Model Disability Survey: A Review of Methodological Approaches to Disability Data Collection. Disability and International Development, 30:2, 16-22.

Groce, N. and Mont. D. (2017). Counting Disability: Emerging Consensus on the Washington Group Questionnaires. Lancet Global Health, Vol 5(7), pe649-650

Madans, J. H., Loeb, M. E., and Altman, B. M. (2011). Measuring disability and monitoring the un Convention on the Rights of Persons with Disabilities: The work of the Washington Group on Disability Statistics. BMC Public Health, 11(Suppl. 4), 20782. 

Miller, K. (2016). Summary of Washington Group question evaluation studies. pp. 69–84. In Altman, B. M. (Ed.). International measurement of disability: Purpose, method and application, the work of the Washington group. Social indicators research series 61. Springer: Cham.

Mitra, S., (2018). Disability, Health and Human Development. Palgrave McMillan: New York.

Mitra, S., Chen, W., Hervé, J., Pirozzi, S. and Yap, J. (2021). Invisible or Mainstream? Disability in Surveys and Censuses in Low-and Middle-Income Countries. World Bank Policy Working Paper.

Mont, D. (2007). Measuring health and disability. The Lancet, 369(9573),1658–1663.

Murray, C. J. L., and Chen, L. C. (1992). Understanding morbidity change. Population and Development Review, 18(3), 481–503.

Palmer, M., Harley, D. (2012). Models and measurement in disability: an international review. Health Policy and Planning, 27 (5), pp. 357-364.

Schneider, M. (2016). Cross-national issues in disability data collection. In Altman, B. M. (Ed.). International measurement of disability: Purpose, method and application, the work of the Washington group, pp 15-28. Social indicators research series 61. Springer: Cham.

Stewart, A., and Ware, J. (Eds.). (1992). Measuring functioning and well-being. The Rand Corporation.

Trani, J.-F., Babulal, G. M., and Bakhshi, P. (2015). Development and validation of the 34-item disability screening questionnaire (DSQ-34) for use in low and middle income countries epidemiological and development surveys. PLoS One, 10,12

United Nations (1990). Disability Statistics Compendium. Department of International Economic and Social Affairs, Statistical Office. Statistics on special population groups., Series Y ;, no. 4.United Nations (2015). Report on the forty-sixth session United Nations, Statistical Commission, (New York, 3-6 March 2015). Available from: .

United Nations (2017). Principles and Recommendations for Population and Housing Censuses. United Nations Department of Social and Economic Affairs. ST/ESA/STAT/SER.M/67/Rev.3. Accessed on Dec. 22^nd 2020:

WHO (2020). Model Disability Survey. Accessed on Dec. 2^nd 2020 at: