The U.S. government’s cyber defense agency is recommending for the first time that companies embrace automated continuous testing to protect against longstanding online threats.

The guidance, from a cluster of U.S. and international agencies published on Wednesday, urges businesses to shore up their defenses by continually validating their security program against known threat behaviors rather than a more piecemeal approach.

“The authoring agencies recommend continually testing your security program, at scale,” according to an alert from the Cybersecurity and Infrastructure Security Agency and several other U.S. and international agencies. The alert warned malicious cyber actors allegedly affiliated with the Iranian government’s Islamic Revolutionary Guard Corps are exploiting known vulnerabilities for ransom operations.

An official at CISA told Bloomberg ahead of the announcement that emulating adversaries and testing against them is key to defending against cyber attacks.

Central to the effort is a freely available list of cyber attackers’ most common tactics and procedures that was first made public in 2015 by MITRE, a federally funded research and development center, and is now regularly updated. While many organizations and their security contractors already consult that list, too few check if their systems can actually detect and overcome them, the CISA official said.

Automated threat testing is still not very widespread, according to the official, who added that organizations sometimes don’t really follow through after deploying expensive tools on their network and instead just assume they’re doing the job.

Automating security controls will make it easier to stop attackers from relying on established tactics. The top threat actors are still going back and leveraging vulnerabilities that are up to 10 years and older, warned the CISA official.

CISA is making the recommendation in collaboration with the Center for Threat-Informed Defense, a 29-member nonprofit formed in 2019 that draws on MITRE’s framework.

Iman Ghanizada, global head of autonomic security operations at Google Cloud, a research sponsor of the Center, said automated testing is important for creating continuous feedback loops that can steadily improve protection.

“Whether you are a large company or a startup, you have to have visibility, analytics, response and continuous feedback,” he said. It makes a big difference to test cybersecurity protections in the real world, rather than just in lab conditions, Ghanizada said.

A growing number of cybersecurity companies, including AttackIQ, Cymulate, Mandiant, Picus Security and SafeBreach, offer so-called breach and attack simulations and other security validation services. The CISA official said the agency is agnostic about which vendor companies use.

Martin Petersen, chief information security officer at facilities management giant ISS A/S, said he persuaded his company to start automated testing following a 2020 ransomware attack. That breach had left hundreds of thousands of employees without access to email and other systems.

The company’s three-year contract with AttackIQ, a founder member of the Center for Threat-Informed Defense, costs $300,000 a year. ISS calculated that the price was cheaper than employing so-called penetration testers, who do similar work but less regularly and effectively, he said.

Petersen said the company had improved tamper protections around its 60,000 endpoints, making it harder to deactivate malware protection as a result of continuous testing. It also fixed “funny” Windows configurations and local firewall settings that could be vulnerabilities.

He added the company had also “significantly raised” its cybersecurity budget, which he said now stands at 7.5 percent of its information technology budget. He declined to say what the number was before the attack but said it would continue climbing into next year.

JetBlue Airways Corp. also relies on AttackIQ, a California-based company founded in 2013. The airline turned to automated continuous testing in part because a government alert about threats is “usually fairly slow and of little value by the time it gets to us,” said Tim Rohrbaugh, its chief information security officer since 2019.

Current protections often aren’t up to the task, according to a new study from AttackIQ due out on Wednesday. Cloud-based customers’ common cybersecurity controls—known as endpoint detection and response systems, which are intended to automatically detect and block compromises in real time—stopped what the company assessed are the seven biggest attack techniques 39 percent of the time in 2021, it found. And none of the more than 100 cloud-based companies’ controls in the study prevented all seven of the “deadly” techniques, according to the report.

Jonathan Reiber, AttackIQ’s vice president for cybersecurity strategy and policy and one of the report authors, argues that continuous automated testing can help catch changes in personnel and equipment that undermine cybersecurity protections. He likens the approach to actively seeking out potential threats rather than scouring for fingerprints in the wake of an incident—a retroactive approach known as looking for “indicators of compromise.”

“People just don’t have enough data,” he said. “Often the only feedback mechanism people have is the attacker.”