Silicon Valley VA hospital gets taste of ‘mixed reality’ through Project Convergence
The Department of Veterans Affairs is building out an emerging technology program at a medical facility in Palo Alto, California, that has already put the VA on the map with its first “5G hospital.”
Project Convergence is a public-private partnership that includes 5G wireless connectivity from Verizon, augmented reality visualization software from Medivis and Microsoft’s HoloLens 2 headset. The program will bring “science fiction to life” with augmented reality imagery that gives clinicians, students and patients a more in-depth understanding of a medical procedure or condition, Thomas Osborne, director of the National Center for Collaborative Healthcare Innovation at the VA, told FedScoop.
The hospital currently has three Hololens 2 headsets but will scale to 10 by the end of the month, the VA says. The program will also use Microsoft’s Azure cloud services.
“It opens doors to things we could only imagine before,” Osborne said.
For instance, for a veteran with kidney disease, the new HoloLens headset will allow her to see the problem with her own eyes instead of just listening to a doctor’s description of it. Data from a CT scan or other imaging procedures can be converted to a three-dimensional picture using the Medivis software, then transmitted quickly via 5G’s enhanced bandwidth and displayed on the HoloLens, Osborne said in an interview.
“We know patients do better when they understand what is going on better,” he said. “It is a form of medical literacy.”
The headset, which looks like a pair of sunglasses with a large wrap-around housing on top, generates more than just augmented visualizations. It also provides “mixed reality” by tracking hand motions that can interact with images displayed by the headset. The tech allows surgeons to better prepare for going under a patient’s skin by seeing, and in a sense feeling, what is inside before making an incision.
For medical students, the headsets will bring their studies to life “in a way that transcends jargon and very domain-specific language,” Osborne said.
The partnership is housed in a nesting doll of innovation units in the VA. The National Center for Collaborative Health Care Innovation, which directly oversees the partnership, is under the umbrella of the VA’s “Innovation Ecosystem.”
Despite the layers of bureaucracy, Osborne is pleased with the pace of the program. A contract for the partnership was signed at the end of January. Verizon brought its 5G signal live on Feb. 4, and the VA had in-hospital demonstrations of the headsets running within a week.
The VA also said it would host demonstrations of the technology at a medical conference in Florida this week. Currently, tests on the tech are only using non-personal health information to ensure compliance with health information regulations.
“Our hospital in Palo Alto, California is currently one of only a handful of 5G enabled health facilities in the world,” VA Secretary Robert Wilkie said in a release in February. “We’re excited to use this hands-on opportunity to work with our partners to develop foundational practices and deploy advanced medical solutions to serve our nation’s Veterans.”
The list of possibilities under the program is long: Osborne rattled off a number of applications, from telehealth clinicians receiving access to vast amounts of patient data to specialty surgeons watching a procedure in real-time in virtual reality from across the country.
Osborne said initial tests have also “elevated morale” at the hospital in Silicon Valley by giving clinicians fancy new tools to use. The only negative feedback Osborne heard from the staff was potential discomfort in wearing the headsets during extended surgeries.
