Summary

In 2002, the E-Government Act was signed into law with noble goals, including "to promote access to high quality Government information and services across multiple channels" and "to make the Federal Government more transparent and accountable." In many respects, the law has been successful, including encouraging agencies to work together to build Web sites that allow users to find information by its content and not only where it is housed in the bureaucracy. However, as more individuals use commercial search engines to find government information, making information accessible to search by various sources has become an important goal. Unfortunately, many important information sources within the federal government are essentially hidden from the very search engines that the public is most likely to use.

In this report, we examine search queries that we believe Americans would expect to result in authoritative and trustworthy government information showing up prominently in their search results. In an examination of Google, Yahoo, Microsoft Live and Ask and the search function provided by USA.gov, we confirmed that many of these searches miss critical information simply because of the manner in which the government agency has published the information. For example:

• A search for "New York radiation" does not find basic FEMA and DHS information about current conditions and monitoring.
• A search to help grandparents with a question about visitation of their grandchildren in any search engine does not turn up an article of the same title located on the Web site of the Administration for Children & Families.
• A search for "small farm loans" turns up the commercial offers for loans, and statistics about government loans, but not most of the major federal government programs designed to help fund small farms.

We have several recommendations for the federal government. Each of these would encourage greater accessibility of government information by making it more searchable.

• Congress should pass the E-Government reauthorization act, which would require the Office of Management and Budget (OMB) to create best practices to encourage searchability of federal Web sites.
• OMB should officially recognize the importance of commercial search engines to Internet users and work with the CIO Council to adopt policies to help users find information.
• Agencies should adopt an information policy that makes public accessibility of online content and resources a priority.
• Agencies should create Sitemaps of content on their sites, with special attention given to materials stored in databases and accessible only through drop-down menus. For example, many agencies have FAQ databases that are not accessible to search crawlers but contain very succinct and useful answers to common questions.
• Agencies should review their use of robots.txt files in order to ensure they are used in the least restrictive way possible. Every effort should be made to include, rather than exclude, materials from the website, whether materials were excluded purposefully or accidentally in the past.

This report serves only to spotlight a critical gap in the accessibility of government information; we don't seek to punish or embarrass government agencies here. We also do not know whether some agencies purposefully choose to exclude their information from search engines, or whether the agencies don't know how to make this information more available. We hope that this report will call attention to this issue and encourage federal agencies to review their information policies.

Next section: Table of Contents