One of the most frequently-used international measures of social capital is the Position Generator. This asks respondents if they know people with specific occupations. As occupational status is a proxy for social resources, this enables
One of the most frequently-used international measures of social capital is the Position Generator. This asks respondents if they know people with specific occupations. As occupational status is a proxy for social resources, this enables researchers to measure the extent and reach of an individual’s access to social capital.
Studies using this methodology have found that access to social capital is associated with wealth, power and status. It helps people to find work if unemployed and helps people to gain promotion at work. There is also emerging evidence that it is important for mental health – low access to social capital is associated with higher prevalence of depression, for example.
Obviously, it is important that the tools which researchers use actually measure what they set out to measure. Before they are used research questionnaires are tested for their validity and reliability. However, the Position Generator has not been subjected to much rigorous pre-testing.
When I developed a version for use in the UK a few years ago, I tested its reliability and validity. This included discussing the occupations which were included in the measure in focus groups as well as using quantitative testing. I also used a method of cognitive appraisal, whereby respondents spoke their thoughts as they considered how to respond to questions. This helped me to discover if respondents understood each question and answered it appropriately.
This methodology has been replicated in The Netherlands to test the reliability of the Dutch Position Generator. 35 semi-structured cognitive interviews were conducted and found some quite surprising results. Only 6 respondents correctly identified all the people they knew with occupations in the Position Generator as currently being in paid employment. Some said they knew people in certain occupations who were actually unemployed, retired or even deceased. Others were unfamiliar with occupations, did not know the occupations of people in their social network, could not recall people they knew, or mis-interpreted occupations.
These findings have a number of implications. Firstly, if cognitive appraisal is not conducted as part of the process of research tool development, it may lead to completion errors by respondents and make the questions less reliable. Second, studies conducted using tools which have not been tested in this way may have unreliable findings. Third, it could be unethical to use tools which are unreliable, as it diminishes the value of research findings and makes their translation into policy or practice problematic.
We have published these findings in the Italian Journal Sociologia e Politiche Sociali. Although we conclude that they may result in an underestimation of social capital, the opposite could also be true. The key message is, though, to thoroughly test research instruments before using them.
Further information
If you are interested in reading the full-text of this paper in Italian, please contact me and I’ll send you a PDF. Alternatively, you can view a pre-publication version of the paper in English here. We are unable to publish the paper in English because of copyright reasons, but are seeking publication of an alternative paper on the same interviews in an English language peer-reviewed journal. You may also be interested in Martin van der Gaag’s excellent website, which has extensive additional reading on the Position Generator and other measures of social capital.