Agency CIOs are apt to underestimate the risk of failure in their IT investments, a congressional watchdog discovered in a new audit.
The Government Accountability Office reviewed 95 federal IT investments across 17 agencies and found that in just 22 of those projects did the risk rating match those listed by CIOs on the federal IT Dashboard. In 60 of the cases, or about 65 percent of the time, GAO’s rating was a degree of higher risk than what the CIOs had listed, and, in 13 cases, GAO’s review found lower risk than the CIO reported.
Agency CIOs are required by the Office of Management and Budget to update the risk ratings of IT investments every month, to provide transparency for the notoriously risky projects. The Federal IT Acquisition Reform Act codified these requirements in December 2014.
This variation often occurs, the GAO authors wrote, because despite OMB’s standard set of investment evaluation factors — risk management, requirements management, contractor oversight, historical performance, human capital and an option for other concerns — to consider, CIOs act with subjectivity based on the specific operational circumstances of their agencies, missions and corresponding investments.
(GAO)
“According to OMB’s guidance, CIO ratings ‘should reflect the CIO’s assessment of the risk and the investment’s ability to accomplish its goals,'” the report says. “Such assessments of risk inherently involve a great deal of human judgment.”
For instance, the Defense Department CIO’s office told auditors DOD’s major investments are “inherently high risk,” and therefore the ratings are “assessments of relative risk implemented within this risk baseline.” So while DOD might not see an investment as particularly risky compared to the rest of its portfolio, another agency probably wouldn’t report it the same.
The Department of Homeland Security, on the other hand, overestimated the risk with its Continuous Diagnostics and Mitigation program, GAO believes.
GAO acknowledged that “in many cases, agency CIOs could have more information than we examined in our assessments.”
Other factors likely played into the discrepancies, GAO reported. For example, many of the agencies’ failed to update the IT Dashboard monthly — particularly in the first month of the review, April 2015 — or complete the ratings process in less than a month’s time.
Likewise, GAO found that many agencies did not focus their reviews on active risks. This triggers “additional questions about the degree to which information reported on the Dashboard provides full and accurate information about an investment’s risk,” GAO reported.
“While agencies’ consideration of active risk is not explicitly called for by OMB’s guidance, this represents a gap in the agencies’ processes that is understating the amount of risk reflected in the Dashboard’s CIO ratings.”
This isn’t the first time GAO has reported inaccuracies in the IT Dashboard since its creation in 2009.
In a December 2013 report, GAO found that of 80 reviewed investments, “53 of the CIO ratings were consistent with the investment risk, 20 were partially consistent, and seven were inconsistent.”
Another report from 2012 observed that “six agencies rated a majority of investments listed on the IT Dashboard as low or moderately low risk” from June 2009 through March 2012, and two agencies — DOD and the National Science Foundation — rated no investments as high or moderately high risk.
Read the full report here.
Contact the reporter on this story via email at Billy.Mitchell@FedScoop.com or follow him on Twitter @BillyMitchell89. Subscribe to the Daily Scoop to get all the federal IT news you need in your inbox every morning at fdscp.com/sign-me-on.
