Methods for Producing Economic Data: Are Surveys Becoming Less Reliable? Evidence, Risks, and a Quality-Assurance Framework for Nigeria and Beyond

Economic data shape fiscal and monetary policy, social protection, investment decisions, credit ratings, development finance, and the evaluation of public programmes. Yet the reliability of the data used for these purposes cannot be taken for granted. This article critically examines whether survey-based economic data are becoming less reliable and, more importantly, identifies the conditions under which survey estimates remain fit for purpose. The paper is an integrative methodological review that reorganises and extends the original manuscript through the total survey error perspective, official-statistics quality frameworks, and recent evidence from household-survey practice, with particular attention to Nigeria. The analysis distinguishes sampling error from coverage error, nonresponse, measurement error, interviewer and mode effects, processing and imputation errors, and analytical or reporting distortions. It argues that declining response and changing communication technologies create genuine risks, but that low response rates do not mechanically imply high bias, and surveys should not be contrasted with “statistical data” because surveys are themselves a major source of official statistics. Evidence from Nigeria illustrates both the continuing value of well-designed probability surveys and the importance of mode-specific validation: recent experimental work finds meaningful differences between phone and face-to-face responses across common economic and welfare indicators. The article proposes a data-quality strategy based on probability sampling where feasible, mixed-mode and responsive designs, computer-assisted collection, transparent metadata, validation against administrative or transaction records, selective integration of geospatial and digital data, explicit treatment of uncertainty, and reproducible analysis. The central conclusion is that surveys are not obsolete; rather, single-source, weakly documented and poorly validated data-production systems are increasingly difficult to defend. Reliability is best achieved through transparent, multi-source statistical systems in which surveys remain a core but continuously audited component.

economic data; survey reliability; total survey error; nonresponse; measurement error; data quality; official statistics; Nigeria

1. Bethlehem, J. (2010). Selection bias in web surveys. International Statistical Review, 78(2), 161–188. https://doi.org/10.1111/j.1751-5823.2010.00112.x [Google Scholar] [Crossref]

2. Biemer, P. P. (2010). Total survey error: Design, implementation, and evaluation. Public Opinion Quarterly, 74(5), 817–848. https://doi.org/10.1093/poq/nfq058 [Google Scholar] [Crossref]

3. Biemer, P. P., & Lyberg, L. E. (2003). Introduction to survey quality. Wiley. [Google Scholar] [Crossref]

4. Campbell, D. T., & Stanley, J. C. (1963). Experimental and quasi-experimental designs for research. Rand McNally. [Google Scholar] [Crossref]

5. Cornesse, C., Blom, A. G., Dutwin, D., Krosnick, J. A., de Leeuw, E. D., Legleye, S., Pasek, J., Pennay, D., Phillips, B., Sakshaug, J. W., Struminskaya, B., & Wenz, A. (2020). A review of conceptual approaches and empirical evidence on probability and nonprobability sample survey research. Journal of Survey Statistics and Methodology, 8(1), 4–36. https://doi.org/10.1093/jssam/smz041 [Google Scholar] [Crossref]

6. de Leeuw, E. D. (2005). To mix or not to mix data collection modes in surveys. Journal of Official Statistics, 21(2), 233–255. [Google Scholar] [Crossref]

7. Dillman, D. A., Smyth, J. D., & Christian, L. M. (2014). Internet, phone, mail, and mixed-mode surveys: The tailored design method (4th ed.). Wiley. [Google Scholar] [Crossref]

8. Groves, R. M., & Lyberg, L. (2010). Total survey error: Past, present, and future. Public Opinion Quarterly, 74(5), 849–879. https://doi.org/10.1093/poq/nfq065 [Google Scholar] [Crossref]

9. Groves, R. M., & Peytcheva, E. (2008). The impact of nonresponse rates on nonresponse bias: A meta-analysis. Public Opinion Quarterly, 72(2), 167–189. https://doi.org/10.1093/poq/nfn011 [Google Scholar] [Crossref]

10. Groves, R. M., Fowler, F. J., Jr., Couper, M. P., Lepkowski, J. M., Singer, E., & Tourangeau, R. (2009). Survey methodology (2nd ed.). Wiley. [Google Scholar] [Crossref]

11. International Monetary Fund. (2012). Data Quality Assessment Framework (DQAF): Generic framework. IMF Data Quality Reference Site. [Google Scholar] [Crossref]

12. Little, R. J. A., & Rubin, D. B. (2019). Statistical analysis with missing data (3rd ed.). Wiley. [Google Scholar] [Crossref]

13. Markhof, Y., Wollburg, P., Palacios-Lopez, A., Castaing, P., Sagesaka, A., & Contreras, I. (2026). The effect of survey mode on data quality: Experimental evidence from Nigeria (Policy Research Working Paper No. 11302). World Bank. https://doi.org/10.1596/1813-9450-11302 [Google Scholar] [Crossref]

14. Meyer, B. D., Mok, W. K. C., & Sullivan, J. X. (2015). Household surveys in crisis. Journal of Economic Perspectives, 29(4), 199–226. https://doi.org/10.1257/jep.29.4.199 [Google Scholar] [Crossref]

15. National Bureau of Statistics. (2024). Nigeria General Household Survey-Panel (GHS-Panel) Wave 5 (2023/2024): Tracking Nigerian households to understand their resilience over time. NBS. [Google Scholar] [Crossref]

16. National Research Council. (2013). Nonresponse in social science surveys: A research agenda (R. Tourangeau & T. J. Plewes, Eds.). The National Academies Press. https://doi.org/10.17226/18293 [Google Scholar] [Crossref]

17. Schouten, B., Cobben, F., & Bethlehem, J. (2009). Indicators for the representativeness of survey response. Survey Methodology, 35(1), 101–113. [Google Scholar] [Crossref]

18. Tourangeau, R., Rips, L. J., & Rasinski, K. (2000). The psychology of survey response. Cambridge University Press. [Google Scholar] [Crossref]

19. United Nations. (2014). Fundamental principles of official statistics (General Assembly Resolution A/RES/68/261). United Nations. [Google Scholar] [Crossref]

20. World Bank. (n.d.). Living Standards Measurement Study: Survey methods. World Bank. [Google Scholar] [Crossref]

21. World Bank. (2020). Nigeria Living Standards Survey 2018–2019: Study documentation and data processing. World Bank Microdata Library. [Google Scholar] [Crossref]

• Defined Benefit Versus Defined Contribution Pension Systems in India: Comparative Evidence From EPS 1995, NPS, and International Pension Reform Experiences