Information Today, Inc. Corporate Site KMWorld CRM Media Streaming Media Faulkner Speech Technology Unisphere/DBTA
Other ITI Websites
American Library Directory Boardwalk Empire Database Trends and Applications DestinationCRM Faulkner Information Services Fulltext Sources Online InfoToday Europe KMWorld Literary Market Place Plexus Publishing Smart Customer Service Speech Technology Streaming Media Streaming Media Europe Streaming Media Producer Unisphere Research

News & Events > NewsBreaks
Back Index Forward
Threads bluesky LinkedIn FaceBook RSS Feed

A Sea Change for Public Data
Posted On September 26, 2013
PAGE: 1 2

The U.S. federal statistical program first began with the 1790 Census. Its establishment was an innovation:

Enshrining this invention in our Constitution marked a turning point in world history. Previously censuses had been used mainly to tax or confiscate property or to conscript youth into military service. The genius of the Founders was taking a tool of government and making it a tool of political empowerment for the governed over their government.

A few decades passed and another innovation, gerrymandering, opened up new uses for census data and any other public or private data that might further the goal of favorable redistricting.

Today, the Association of Public Data Users (APDU) guides professional statisticians and social scientists through the constant change and innovation that is happening inside and outside of federal statistical programs. The association also coordinates professional user community input into government policy developments affecting the quality of federal statistical surveys. Members are “public data producers, disseminators, and users”—including businesses, universities, policy organizations, and government organizations at the local, state, and federal levels.

APDU convened on Sept. 16–17 in Washington, D.C. to update members on current challenges to the quality of federal statistical data and opportunities to innovate around those challenges. The Census Bureau in particular is challenged by the rising costs of conducting surveys, a reduced budget, and Americans’ growing reluctance to spend time providing answers to government surveys. Census and other federal agencies face continuing cuts to budgets and staffing. Global enthusiasm for raw government data, Big Data, and social media data has obscured the effort that goes into existing statistical programs. (Along this line, APDU is sponsoring a free webinar on Government Data and Confidentiality: Compatible Companions With the Help of Statistical Disclosure Control. For this and other APDU webinars, see the website.) The meeting agenda and links to available slide decks for the APDU 2013 conference, titled A Sea Change for Public Data, are available online.

Social Media Intelligence

Rising costs and shrinking budgets have pushed statisticians to explore sources of data to complement survey research results. Acknowledging alternative sources, APDU included a panel on Social Media Data as a Public Resource to showcase research on using social media data for analysis. Michael J. Paul of the Johns Hopkins University computer science department presented the results of his work evaluating user-generated content on Twitter as a predictor of local flu outbreaks. He found correlations of up to 99% when his Twitter analysis was compared with the authoritative data on seasonal influenza from Centers for Disease Control and Prevention. Paul and others on the panel acknowledged that there were limits for using Twitter as a predictor and the work needed to clean up sample sets to account for spammers, copycats, and other users who were of little value.

Paul presented other study results, including assessing the value of intelligence gathered from over-sharing drug abusers on an online forum. Mining the forum proved to be useful for identifying new drugs of abuse and new health symptoms associated with their use. The data was not as useful for demographic analysis given contributors’ incentives to lie about their personal location, age, and other data.

Another panelist, Paul Hitlin of the Pew Center for Excellence in Journalism, brought it all back to Earth with the observation that “[I]f Twitter users were reflective of the nation as a whole, Ron Paul would be President now.” Speaker Thomas Levine of the Zipfian Academy followed up with the observations that “a text search is not statistics,” statistics are applied to the search results, and we “still have to do good statistics.”

PAGE: 1 2

Peggy Garvin of Garvin Information Consulting writes, trains, and consults on the topic of U.S. government information resources and policy.

Email Peggy Garvin

Related Articles

9/25/2018What It's Like to Be a Journalist in 2018
2/6/2018Twitter as a Tool for Studying Emotions
10/10/2017The Supreme Court Review of Gerrymandering
9/20/2012APDU 2012 Conference Explores the Future of Federal Survey Data

Comments Add A Comment

              Back to top