VA reforming EHR deployment following 12-week review

The Department of Veterans Affairs will restructure how it implements its electronic health records modernization program following a 12-week strategic review of the $16 billion program, Secretary Denis McDonough told lawmakers Wednesday.

User training is a major theme of many of the changes that need to be made to improve the way clinicians and patients use the new system, McDonough said during a Senate Veterans’ Affairs Committee hearing.

“I think that there is just no doubt the training was wanting,” McDonough told the

Included in those changes will be the introduction of new virtual testing environments — “sandbox” deployments of purely technical changes to improve underlying IT infrastructure that shouldn’t impact user experience. While the updates will bring fixes to some technical aspects, McDonough said they are mostly programmatic improvements.

Training and testing issues led to a delay in the initial go-live of the system in Spokane, Washington, in February and were the subject of multiple government watchdog reports, including one that sounded the alarm on potential system failure without adequate testing.

McDonough also reiterated the VA’s commitment to sticking with its prime contractor, Cerner, which has been working on developing cloud-based overhauls for both the Department of Defense’s and VA’s health IT systems.

The secretary said the continued rollout of the EHR across the nation would be dependent on testing and the ability for the facilities to make progress on deploying the system, rather than based on their location as was the initial plan.

“You will see us pursue a surge of activity in the coming weeks and months, intently focused on veteran experience, patient safety and employee engagement. Specifically, VA will pursue technical-only (“sandbox”) deployment of Cerner technology at previously planned sites in Veterans Integrated Service Networks (VISNs) 10 and 20 – ensuring technical readiness without affecting veterans or frontline clinical employees,” McDonough said in his prepared opening statement.

There will also be new data teams that will work to integrate databases associated with the new system. The VA’s EHR system is eventually supposed to seamlessly connect with the DOD’s MHS Genesis modernization program.

“However, gaps remain in our ability to govern and manage data between the two EHRs and with DOD to ensure seamless veteran and employee-centric information sharing and provision of managed, trusted data,” he said, adding that a clinician told him that most of the data integration happens in workers’ heads, not on the computer systems they use.

By the end of the year, the VA will also publish a data strategy specific to making EHR data more interoperable, McDonough revealed.

New ‘chaos engineering’ tool shared between DOD software factories

The Air Force’s Kessel Run software factory is transitioning to the Navy a tool that it has been developing for the past two years that is designed to emulate persistent enemy attacks on a system.

The Navy’s Black Pearl software factory will be the first group outside of Kessel Run to get the tech stack and list of best practices on implementing it. But eventually, the goal is for as many coders to get their hands on it as possible, lead engineer Omar Marrero told FedScoop.

The tech stack and Air Force team behind it are jointly known as Bowcaster, named after the weapon Star Wars character Chewbacca used in the film series. And the discipline behind their work is something referred to as chaos engineering.

“You have to constantly break the system to find where our weaknesses are,” said Marrero, whose official title is chaos and performance engineering lead. “That’s essentially what chaos engineering is.”

The idea behind chaos engineering is to unleash unpredictable, persistent attacks that can still be controlled in what exactly they target within a system to emulate an enemy. Kessel Run launched its first internal attack using the system in the summer of 2020 after launching the program in 2019.

Marrero said the idea to put resources into chaos engineering came organically from the need to more thoroughly test systems. He said he attended several tech conferences to learn from others that had deployed similar systems, even though he already has a background in this type of cybersecurity testing.

“As part of my career in the Air Force I have always done some flavor of chaos,” Marrero said in an interview.

The lessons the Air Force learned from others and in its own practice developing the tech stack is part of what it will be transitioning to Black Pearl as part of a chaos engineering “playbook.” It will also be porting code into Platform One’s software repository Iron Bank for others to start experimenting with.

One of the biggest lessons Marrero and the team learned was to “control the blast radius,” meaning don’t let the code start unplugging too many things.

Sharing tech stacks and tools like Bowcaster is a practice Kessel Run plans to continue. The Air Force and Navy are working on a new memorandum to share even more code between the two services.

Mike Brown backs out of nomination for top Pentagon contracting job

Mike Brown on Wednesday requested that his nomination to be the Pentagon’s top contracting officer be withdrawn amid an ongoing Department of Defense Inspector General investigation.

Brown, currently director of the Defense Innovation Unit, sent a letter to Department of Defense Secretary Lloyd Austin requesting that President Biden withdraw his nomination, saying that the investigation has roadblocked his confirmation to be DOD’s head of acquisition and sustainment and “our men and women in uniform deserve Senate-confirmed leadership as soon as possible.”

Brown came under scrutiny after one of his former employees alleged to the IG that he handed contracts and jobs to close allies and friends in a way that pushed the ethical boundaries in that role. The allegation came soon after Brown’s nomination was announced.

“Unfortunately, it appears that an ongoing investigation by the Department of Defense Office of Inspector General into personnel practices at the Defense Innovation Unit will delay consideration of my nomination by up to a year,” he wrote in the letter, which was obtained by FedScoop.

In his withdrawal request letter, Brown signals that he will continue leading DIU.

Brown would have brought a unique background to the role traditionally held by contract lawyers and former defense industry executives. He developed his career in Silicon Valley namely as the CEO of cybersecurity company Symantec before becoming a Presidential Innovation Fellow prior to taking over DIU. During his time leading DIU, he has championed the need for DOD to better scale the innovative technologies through rapid contracting.

Scaling cloud training programs a major challenge for CIOs

Scaling up internal training programs for staff within federal agencies is one of the biggest challenges facing agency CIOs, technology recruitment consultants have told FedScoop.

Departments currently use a range of tools, including 90-day and 180-day interagency job rotations, as well as sessions sponsored by cloud service providers. But experts say a more programmatic approach could help retain existing staff and attract new employees with specialist skills.

Britaini Carroll, principal director of Accenture Federal Services’ human capital division, told FedScoop it’s been an even bigger issue during the remote work of the pandemic. “A lot of CIOs I’ve talked with about this past year, whether it’s cyber or cloud, have had trouble getting their training online and leveraging the broader tech skilling that is out there.”

“100-150 [trainees] at a time isn’t going to scale, so there needs to be more intentional programming that enables both hiring new folks, bringing them up so speed, and broad-based learning on prioritized cloud roles,” Carroll said.

Meghan Sullivan, a principal within Deloitte’s government and public sector consulting practice, told FedScoop that she has seen a similar focus from CIOs on concerns about the best way to implement best-in-class cloud training for all staff. “[They are asking] how do I do this at scale, so that I’m not just investing in five people, but 50 people, so that if five or 10 leave I still have enough there,” she said. “How do I do it in a holistic manner, looking at training programs?”

Advocates say that programmatic training allows agencies to keep staff engaged, and also can help to address skills gaps left by the departure of staff amid a still-tightening technology labor market. Federal agency technology leaders in recent months have told FedScoop of an increase in the pace of departures from the government for the private sector, while a report last year by research firm Global Knowledge found that cloud and IT security skills continue to be most in-demand and that IT decision-makers were “struggling” to hire in these areas.

Advocates say also that a broad agency-wide training program helps to ensure staff remain engaged and that they have the necessary knowledge of different cloud platforms to be able to work on multi-system procurement contracts.

This story is part of FedScoop’s Special Report — The Continued Push to the Cloud.

How the DOD plans to approach cloud differently outside of the U.S.

The Department of Defense wants cloud computing to support everything from back-office tasks to battlefield operations. But how it gets cloud in regions outside of the continental U.S. comes with significant extra barriers.

The DOD’s process for addressing those barriers was outlined in a new strategy published in May. The department shed new light to FedScoop on how exactly it will execute the technical and resource-intensive hurdles involved in getting cloud at the so-called “tactical edge.”

“Cloud computing can help solve today’s national defense challenges, but its true potential is to solve tomorrow’s challenges,” the strategy states. “Collaboration across these domains, increasingly enabled by high-tech, software-driven solutions, must occur at the global point of need, at the tactical edge, and at the fight.”

The department wants more cloud and data storage capabilities in areas it operates to enable multi-domain operations: the ability to transmit data between airplanes, ground vehicles and any other platforms in battle, which will rely on the ability to expand networks with cloud storage capabilities to turn that data into actionable information. By having cloud computing capabilities in the field, military operations have more of a technical backbone to support that kind of rapid data transfer and the computing power to analyze that data.

Physical challenges

The harsh environments the military often operates in present a range of challenges for the computers that often need carefully controlled setups. One solution the DOD says it is pursuing smaller, less-power hungry machines.

“OCONUS Cloud Strategy acknowledges that space and power is limited because the locations will be hosted in U.S.-controlled military locations that are treated as U.S. soil. This approach is necessary to avoid all data sovereignty issues with a host nation,” a DOD spokesperson told FedScoop.

The hardware that forms the computing backbone also needs to be mobile as DOD is constantly shifting its operations.

“The rapid pace of advances in mobile cloud compute capabilities creates the belief that a mobile cloud could be managed like any other set of forward-deployed resources, such as planes, ships, or infantry battalions,” the spokesperson said.

Not only does the physical hardware need to be mobile, but the elasticity of data processing is critical. With unsuspecting surges in data possible in an environment where the military would need to respond rapidly, the ability for the systems the expand to meet the demand is one of the benefits of cloud that the military wants to take advantage of.

“Similarly, the data capacity needs will be very elastic and linked to mission objectives. One of the core values of the commercial cloud is that it has freed commercial enterprises from having to plan for and purchase against the largest but, isolated compute surges. Building on the above, as mission demands expand, and more resources are deployed in-theater, a planned expansion of data and compute capacity can accompany those units,” the DOD spokesperson said.

Cybersecurity challenges

Cybersecurity is also complicated when cloud hardware is outside the borders of the U.S. DOD regulations require sensitive data to be kept on U.S. soil as foreign internet connections and easier access to hardware presents a ripe target for attackers. But the solution, DOD says, will be similar to how the department wants to protect its networks and cloud capabilities inside the U.S.

“The principles and pillars of the Department’s zero-trust strategy and reference architecture will guide the implementation of all Cloud-based services deployed by the Department regardless of physical location.  This applies to both CONUS and OCONUS deployed assets,” the spokesperson said.

Zero trust is a network architecture based on the principle that every user is granted “zero” trust and not given free roam to move about a network just because they have credentials. That framework is designed to segment networks to stop attackers that make it past the first line of defense, whether it’s inside or outside of the U.S.

All of the challenges OCONUS cloud operations face are also being met with human resources. The DOD plans to deploy teams of cloud engineers and experts into the field to set up and run the unique technology.

“As an illustration, instead of transporting hard drives back to the CONUS for processing because of bandwidth limitations, the strategy calls for deploying research teams to explore novel ways to process this data closer to the tactical edge. These focused research and development efforts are intended to lead to capability advancements that very well may necessitate a faster tech-refresh cycle,” the spokesperson said.

This story is part of FedScoop’s Special Report — The Continued Push to the Cloud.

Austin commits to $1.5B for DOD’s Joint AI Center over next 5 years

Secretary of Defense Lloyd Austin on Tuesday said he will commit $1.5 billion for the Department of Defense’s Joint Artificial Intelligence Center over the next five years.

While Congress ultimately decides what funding the JAIC will get, in recent years, it has shown willingness to appropriate funding to develop it as DOD’s AI “enabling force.” The most recent budget requests have yielded around $200 million annually for the JAIC, a number that would increase to about $300 million per year if Austin’s promised $1.5 billion is authorized by Congress.

“Done responsibly, leadership in AI can boost our future military tech advantage — from data-driven decisions to human-machine teaming. And that could make the Pentagon of the near future dramatically more effective, more agile, and more ready,” Austin said at the National Security Commission on AI conference Tuesday.

Austin said that the department has more than 600 AI projects running, many more than the year prior. The JAIC has been at the center of the DOD’s AI push, at first working on individual projects but now focusing on assisting the DOD’s myriad of AI offices. One of the programs the center continues to focus on is the Joint Common Foundations, an AI development platform that it eventually hopes will be the central tool developers across the department will use to write code, work with data and advance AI projects of their own.

The vision for AI in the DOD revolves around “integrated deterrence,” Austin said. In essence, the idea is to weave AI tools and the concept of operation into everything the DOD does, from logistics to waging war in all domains. It includes the new framework of Joint All-Domain Command and Control, where all sensors from across the domains of battle are integrated and use AI to make sense of data from the battlefield.

“AI and related technologies will give us both an information and an operational edge,” Austin said.

He also acknowledged that in order to achieve this type of integration, new acquisition methods will need to be used to field the rapidly changing technology. “But we know that truly successful adoption of AI isn’t just like, say, procuring a better tank,” he said.

One of the new tools in the DOD’s acquisition toolbox is the Rapid Defense Experimentation Reserve (RDER), which helps get promising tech across the so-called “valley of death” — the struggle for the Pentagon to transition and scale research-and-development efforts, particularly innovative technologies, to use on the battlefield

“In today’s world, in today’s department, innovation cannot be an afterthought. It is the ballgame,” Austin said.

GSA stresses agencies align IPv6 and zero-trust plans ahead of next week’s deadline

The General Services Administration continues to stress that agencies align their IPv6 and zero-trust architecture implementation plans, ahead of the deadline for the latter next week.

Zero-trust architectures should be protocol agnostic, and plans should ensure there aren’t any gaps in the cybersecurity products used, said Tom Santucci, director of IT modernization within the Office of Governmentwide Policy, at an ATARC event Tuesday.

GSA‘s main concern is that agencies also crafting their IPv6 implementation plans, due before the end of fiscal 2021, will end up undertaking duplicative work if the two plans aren’t coordinated.

“Those two should coincide with each other,” Santucci said.

President Biden‘s cybersecurity executive order issued May 12 gave agencies 60 days to develop their zero-trust plans, with an emphasis on accelerating the purchase of secure cloud services.

Agency cloud adoption strategies require planning at the C-suite level, and the Data Center Optimization Initiative that Santucci oversees developed a Cloud Smart guide to help. The guide advises agencies to evaluate their people, processes and tools, followed by the business value of cloud migration.

Examining cloud procurement and acquisition strategies is also important.

“One of the things we find is that people aren’t buying it right or aren’t using it right,” Santucci said.

Agencies will need to evolve cloud automation and monitoring tools over time and share information among their offices and teams because it’s often “amazing” how much one may know about how a modernization effort is going that the others don’t, he added.

DOD needs to better collaborate with DHS on cyberdefense, IG says

The Department of Defense’s work to help defend the cybersecurity of critical infrastructure needs a stronger implementation plan in its collaboration with the Department of Homeland Security, the DOD inspector general reported Tuesday.

The IG examined the implementation of a 2018 memorandum that outlined the partnership between the two departments on how they can coordinate the protection of critical infrastructure without violating their jurisdictions. The watchdog found DOD’s work lacks milestones and implementation plans for joint operations and general collaboration with DHS, which could put the nation’s cyberdefense of its critical infrastructure at risk.

“Without an implementation plan that clearly defines roles and responsibilities and identifies milestones and completion dates, the DoD may not be able to sustain collaboration with the DHS in protecting the Nation’s critical infrastructure,” the report states.

DHS’s IG also did an investigation, but it does not appear to be published yet.

DHS has the authority to protect the U.S. homeland, with its Cybersecurity and Infrastructure Security Agency taking the lead on cyberdefense, whereas DOD operates largely outside of the U.S. in both offensive and defensive operations. With the omnipresence of cyberthreats that can emanate both in and outside the U.S., the two departments have worked together to defend critical infrastructure. The first memorandum between the departments was signed in 2010.

The report complimented some of what the two agencies have done to date, like processes the departments established for how each can request help from the other. But recent hacks like the SolarWinds breach show the criticality of implementing more collaborative processes, the report said.

“[T]he compromise continues to show the importance and criticality of the DoD’s and DHS’s ability to respond to any and all cyber threats, which would be significantly improved by implementing a plan to accomplish shared goals in the 2018 joint memorandum,” it stated.

The Pentagon agreed with most of the specific recommendations, stating it would work to create implementation plans and more collaboration. But the vice director of the Joint Staff said the military would seek “interdepartmental consensus” on how to best move forward.

CDC looks to modernize its immunization information system

The Centers for Disease Control and Prevention intends to make sole-source contract awards to seven vendors to modernize its immunization information system, according to a presolicitation.

Vendors will make changes to the system allowing connectivity to the Immunization (IZ) Gateway, a message router that operates as a single connection point for participants like hospitals, clinics and pharmacies.

The contracts are part of a CDC initiative to enhance reporting of routine vaccination data and automate reporting of COVID-19 vaccination data, a weakness during the pandemic.

Vendors will also ensure connectivity to new IZ Gateway features like multi-jurisdictional queries and data submission to CDC.

The vendors CDC selected are Blue Cross Blue Shield of North Dakota, Deloitte Consulting, Gainwell Technologies, HLN Consulting, Optimoz, Software Partners, and Myriddian.

CDC selected the seven vendors citing Federal Acquisition Regulation 6.302-1, which allows sole-source awards when only one contractor can satisfy agency requirements. In this case, the vendors currently provide system operations and maintenance support to their specific jurisdictions, and the Office of Acquisition Services argues it would be “disruptive and inefficient” to have two contractors or system integrators providing the same services and duplicate costs while violating best practices.

Other vendors may submit capability statements, proposals or quotes for consideration by CDC by July 15. If the agency determines other vendors can meet its requirements, a competitive procurement will be conducted.

NASA seeks cybersecurity and privacy enterprise support

NASA wants contract personnel to provide cybersecurity and privacy enterprise solutions and services (CyPrESS) in support of all its centers and facilities, according to a request for proposals (RFP).

The contract will consist of a single award for an indefinite-delivery, indefinite-quantity contract of solutions and services over a nine-year period.

NASA‘s IT Procurement Office issued the RFP on behalf of its Office of the CIO in June.

The cost-plus-award fee (CPAF) contract covers CPAF and firm-fixed-price task orders, with the latter being phased in over a 60-day period at the outset.

A top-secret facilities clearance is required.

NASA anticipates making its award Nov. 8 and beginning core work on Feb. 1, 2022.