Google open-sources HEIR compiler for encrypted AI inference

Homomorphic encryption has been the 'someday' answer to cloud AI privacy for years: run inference on encrypted user data without the server ever seeing it in the clear. The engineering has been the catch. Someone with deep cryptography knowledge has to hand-convert a model to run over ciphertext, which is why the technique lives mostly in research papers. In a post on the Google security blog, Staff Software Engineer Jeremy Kun says Google's HEIR project, short for Homomorphic Encryption Intermediate Representation, is meant to automate that conversion for pre-trained models, with the stated ambition of becoming 'a one-click solution to enable non-experts to incorporate encrypted inference into production applications.'

Kun frames the case for HEIR around regulated industries, writing that 'critical sectors like healthcare and finance are even more averse to these risks, and strict regulations limit data sharing.' That is where encrypted-in-flight inference stops being a novelty and starts being a compliance argument. To show what is now possible, the team walks through four workloads: a deep learning recommendation model with Belfort Labs, LG and NYU; credit-card fraud detection with Niobium and hardshell.ai; a network intrusion detector called Kitsune, also with Niobium; and a voice hotword model with Belfort Labs.

Google is also trying to seed the accelerator layer this technology needs to be usable at all. The company says it has partnered with hardware developers Belfort, Niobium, Cornami and Optalysys, and lists academic collaborators including Georgia Tech, Carnegie Mellon, UC Santa Barbara, Illinois Institute of Technology, Purdue, the University of Edinburgh and Tsinghua University. Four peer-reviewed papers have been built on the toolchain so far.

What the post does not do is quantify the cost. It refers to latency numbers 'presented for a single-threaded CPU' without publishing the actual figures, does not say which FHE scheme parameters or security levels are default, and does not commit HEIR to any Google Cloud product surface. Those gaps matter because the historical objection to homomorphic encryption isn't whether it works; it's whether it works fast enough for anything past a demo. It lands in a week our cybersecurity tracker has already spent on encrypted-reasoning attacks against major labs, so the appetite for private-by-construction inference is not hypothetical.

The more interesting downstream story is who else picks it up. HEIR is open source on GitHub, which means rival clouds and privacy-focused startups can build on the same compiler layer Google is using, and the four accelerator startups get a common target that isn't tied to one buyer.