Future of Big Data and Federal Statistical Agencies


If the APDU conference presentations are an indication, federal statistical agencies have ambitious plans for the improvement of their programs. Julia Lane of American Institutes of Research discussed new paradigms, analysis, and infrastructure involved in big data in the Keynote speech, titled Innovations in Public Data. In addition, Conference presenters from the Census Bureau and other statistical agencies detailed their plans

On the first day of the conference, APDU board member, Julia Lane, gave a comprehensive and thoughtful speech that analyzed both the future of data analysis and the future of APDU itself. The massive amounts of data available at our disposal is both a blessing and a burden: it forces users to think about what they’re measuring and why, while demanding that they make sense of an intimidating amount of information. In addition, she prescribed a set of actions for the APDU community, including using APDU’s comparative advantages for training and analysis, trusting in quality data for analysis, and making data available for new uses.

The closing session of the conference, Scenarios: The Role of Federal Statistical Agencies in 2020, featured a panel of experts presenting an overview of their future vision, challenges, and vision for their respective programs:

  • Michael Horrigan of BLS discussed alternative data’s potential use in BLS programs and BLS’s efforts to improve their “linking” of administrative data. He sees electronic data collection, algorithms, and data conversion as part of their vision for the future.
  • The Census Bureau’s Ron Jarmin spoke about their need to modernize – evaluating what to measure and how to measure it. They are working on collaborations, staff training and recruitment, and other data projects to help in these efforts. In the future, they expect to gain a better understanding of the costs and benefits of modernization, identify opportunities, improve collaboration, and maintain or enhance current programs.
  • The last speaker, Kimberly Vitelli of ETA, saw the future in terms of delivering actionable information for real people. For example, in order to help combat unemployment, the ETA could tailor information pushes to job seekers. They are also working on improving their credentialing data.

Brian Harris-Kojetin of OMB moderated the session, while Constance Citro of the National Academy of Sciences participated as a discussant in the panel. She spoke of technical changes to the Census for 2020, and how they will benefit other Census programs. As Big Data becomes even bigger, federal agencies are up to the task of helping make sense of it.