December 7, 2011 - Each year the federal government spends billions of dollars on information technology (IT) investments. Given the importance of program oversight, the Office of Management and Budget (OMB) established a public website, referred to as the IT Dashboard, that provides detailed information on about 800 federal IT investments, including assessments of actual performance against cost and schedule targets (referred to as ratings). According to OMB, these data are intended to provide both a near-real-time and historical perspective of performance. In the third of a series of Dashboard reviews, GAO was asked to examine the accuracy of the Dashboard's cost and schedule performance ratings. To do so, GAO compared the performance of eight major investments undergoing development from four agencies with large IT budgets (the Departments of Commerce, the Interior, and State, as well as the General Services Administration) against the corresponding ratings on the Dashboard, and interviewed OMB and agency officials.
Since GAO's first report in July 2010, the accuracy of investment ratings has improved because of OMB's refinement of the Dashboard's cost and schedule calculations. Most of the Dashboard's cost and schedule ratings for the eight selected investments were accurate; however, they did not sufficiently emphasize recent performance for informed oversight and decision making. (1) Cost ratings were accurate for four of the investments that GAO reviewed, and schedule ratings were accurate for seven. In general, the number of discrepancies found in GAO's reviews has decreased. In each case where GAO found rating discrepancies, the Dashboard's ratings showed poorer performance than GAO's assessment. Reasons for inaccurate Dashboard ratings included missing or incomplete agency data submissions, erroneous data submissions, and inconsistent investment baseline information. In all cases, the selected agencies found and corrected these inaccuracies in subsequent Dashboard data submissions. Such continued diligence by agencies to report complete and timely data will help ensure that the Dashboard's performance ratings are accurate. In the case of the General Services Administration, officials did not disclose that performance data on the Dashboard were unreliable for one investment because of an ongoing baseline change. Without proper disclosure of pending baseline changes, OMB and other external oversight bodies may not have the appropriate information needed to make informed decisions.
(2) While the Dashboard's cost and schedule ratings provide a cumulative view of performance, they did not emphasize current performance--which is needed to meet OMB's goal of reporting near-real-time performance. GAO's past work has shown cost and schedule performance information from the most recent 6 months to be a reliable benchmark for providing a near-real-time perspective on investment status. By combining recent and historical performance, the Dashboard's ratings may mask the current status of the investment, especially for lengthy acquisitions. GAO found that this discrepancy between cumulative and current performance ratings was reflected in two of the selected investments. For example, a Department of the Interior investment's Dashboard cost rating indicated normal performance from December 2010 through March 2011, whereas GAO's analysis of current performance showed that cost performance needed attention for those months. If fully implemented, OMB's recent and ongoing changes to the Dashboard, including new cost and schedule rating calculations and updated investment baseline reporting, should address this issue. These Dashboard changes could be important steps toward improving insight into current performance and the utility of the Dashboard for effective executive oversight. GAO plans to evaluate the new version of the Dashboard once it is publicly available in 2012. GAO is recommending that the General Services Administration disclose on the Dashboard when one of its investments is in the process of a rebaseline. Since GAO previously recommended that OMB improve how it rates investments relative to current performance, it is not making further recommendations. The General Services Administration agreed with the recommendation. OMB provided technical comments, which GAO incorporated as appropriate.
