How federal agencies are adapting to AI hardware acceleration

A hyper-accelerated variant of Moore’s Law is unfolding across AI-dependent hardware, driving compute performance far beyond traditional doubling cycles. For federal IT leaders attempting to deploy AI beyond centralized data centers and into bandwidth-constrained, highly secure tactical edge environments, this quickening pace has shifted from a technological curiosity into an urgent planning problem.

“Hardware velocity is just one piece of a larger puzzle,” explained Lewis Shepherd, Federal Chief Technology Officer at Broadcom, in a new video interview produced by Scoop News Group for Broadcom. “It’s a blend for the AI era of the traditional infrastructure challenges agencies have in delivering world-class IT, along with the data-specific challenges of AI.”

To successfully deploy AI at scale, Shepherd said federal agencies must navigate three intertwined hurdles:

• Cross-Environment Security: Coordinating unified security across infrastructure spanning from remote tactical edge devices to core data centers.
• Hardware Velocity: Keeping pace with rapid GPU and hardware acceleration without triggering continuous, expensive architectural re-engineering.
• Hybrid Orchestration: Synchronizing edge AI workflows alongside enterprise data systems while protecting taxpayer dollars.

Securing private AI with zero trust

These compounding pressures are driving a massive federal pivot toward private AI architectures. Shepherd emphasized that long-standing zero-trust principles are fundamental for AI.

“Zero-trust principles are proving essential to secure AI adoption,” said Shepherd. “That means enforcing trusted identity, strict authorization and full lifecycle management across both AI development and user access.”

Data access governance is equally critical. Shepherd highlighted the Model Context Protocol (MCP) server as a vital control point governing how AI systems interface with underlying datasets and third-party tools. Implementing enterprise-grade gateways at this layer gives CIOs and CISOs direct oversight over autonomous activity.

This control is vital as agencies experiment with agentic AI and are seeing security risks emerge in testing environments: “We’ve seen frontier labs where agents are going ‘quote unquote’ rogue and escaping sandboxes,” said Shepherd. “By deploying dedicated visibility and governance tools, ‘we’re redressing the balance back for cyber defense.’”

Taming unpredictable token economics

Beyond cybersecurity, financial management remains an operational hurdle for federal leadership. “Token consumption remains one of the most unpredictable costs facing federal IT budgets,” said Shepherd, particularly for agencies reliant on public cloud access to frontier models where usage and data egress fees compound continuously.

To restore fiscal predictability, Shepherd advocates shifting core AI workloads onto agency-owned infrastructure. “Bring validated models, including small language models suited for edge deployments, onto agencies’ own infrastructure, trading unpredictable per-token bills for predictable, on-premises economics.”

Standardizing for scalable, long-term growth

Rather than restructuring systems around every new hardware or software release, Shepherd advises agency leadership to follow proven IT modernization strategies: standardize early on an open enterprise abstraction layer.

“It’s best to have a really solid, flexible, open-standards ecosystem,” said Shepherd. Building on an open framework allows agencies to manage cost, control complexity and seamlessly incorporate emerging GPUs, models and edge use cases as technology continues to evolve.

This executive video interview and summary article were produced by Scoop News Group for FedScoop and underwritten by Broadcom.

Learn how Broadcom helps federal agencies modernize IT infrastructure, leverage a private cloud operating model, protect their network, and support mission-critical operations.