case study

AES engages 5x more employees with Hoxhunt compared to legacy SAT tools

Client logo
About

Industry: Energy, critical infrastructure

Number of employees and contractors: 9,000 +

Challenge

Legacy security awareness training fell short of engaging the AES workforce to reduce human risk. They needed a solution that fixed this, while supporting effective scaling of training in multiple languages, positive security culture and enthusiasm for cybersecurity, and automated analysis of reported threats.

Solution

Upending the traditional awareness model, AES turned to Hoxhunt to measurably reduce risk across their global workforce, and raise resilience by orders of magnitude above the biggest legacy SAT tools on the market.

Key takeaways:
Featured image

Hoxhunt performance vs. AES’s previous security awareness software tools:

  • Reporting rate increased by 526%, from the 3-tool aggregate of 11.5% to 60.5% (this only reflects the proportion of AES employees whose work is computer-based)
  • Failure rate decrease by 79%, from the 3-tool aggregate of 7.6% to 1.6%  
  • Miss rate decreased by 58%, from the 3-tool aggregate of 80.9% to 34%
  • Resilience ratio increased by 2533%, from 1.5 to 38

A resilience ratio of 38 is astonishing given the industry, size, and scope of AES...similar companies will strive for scores of 10-15, and typically top out at 20. The resilience ratio is calculated by dividing the engagement rate by the failure rate, yielding a more accurate metric for risk than either engagement or failure alone.

Innovation and sustainability are in the corporate DNA of AES. But the security team realized after five years of lackluster results with the biggest SAT tools on the market, that they needed a different approach to reducing risk.

The challenge: “Clearly, it was time for a change.”

AES understood the importance of protecting their people and company from social engineering and breaches. Over the course of 5 years, they tried all of the biggest SAT tools on the market to reduce human risk, but there was one problem.

Engagement remained stagnant, no matter the tool.

“Over five years, we tried multiple solutions and different models with mixed results, navigating from specialized technologies to integrated solutions. We saw modest reductions in click rate, but the engagement was stagnant. Our challenge was not with the solution, but with the security awareness training model itself. Clearly, it was time for a change.”

DAVID BADANES
Director of Cybersecurity Strategic Initiatives, AES

This led AES to question the very model of SAT itself. The legacy training models seemed ineffective, given the current threat landscape:

  • The one-way awareness training model meant minimal engagement
  • Manual translation for phishing messages was resource-intensive
  • Reliability of phishing metrics were questionable due to the volume of false positives
  • Year-end heightened season for phishing attacks (benefits enrollment, holidays, etc.)

They needed a solution that would help them drive interest in cybersecurity to new levels and make good security behavior a habit. And they hoped to go a step further, integrating human intelligence into threat detection and response.

Hoxhunt vs. legacy security awareness training tools

The AES team turned to Hoxhunt, a human risk management platform that delivers gamified, individualized security training at scale. Compliance, awareness, and behavior change training fed directly into and augmented threat detection and response. It was effectively a self-reinforcing closed loop of protect-detect-respond capabilities.

They assessed Hoxhunt with a broad group of users in a defined pilot and created a robust change management process, starting with obtaining leadership and stakeholder buy-in.

Hoxhunt delivered immediate results upon its adoption:

  • 🚀 Engagement skyrocketed
  • 🚀 Risk posture measurably improved
  • 🚀 SOC response to email threats accelerated, without added resources
  • 📈 User skills increased
  • 📈 Failure rate plummeted
  • 📈 Culture lifted  
Tested across multiple test groups, results with Hoxhunt were superior to the aggregate of 3 legacy SAT tools. Measurable reduction of risk was achieved after just a few months with Hoxhunt.
A resilience ratio of 38 is astonishing given the industry, size, and scope of AES...similar companies will strive for scores of 10-15, and typically top out at 20.
"We needed a new model to gain better visibility into our human risk and manage that risk, and Hoxhunt enabled that. Now we have a reporting rate of 70% and a click rate of like 2%. We’ve measurably reduced risk and improved security culture in a way that aligns with our cultural values, and people seem to really like it.”

DAVID BADANES
Director of Cybersecurity Strategic Initiatives, AES

Why these results matter

Good security posture is a competitive advantage today. Threats can significantly impact business operations, particularly given the rise in ransomware and software supply chain attacks. The AES team noted that phishing and training metrics are increasingly requested by external parties, such as cyber insurance, customers, and investors.

The high engagement levels give the AES security team unprecedented visibility into human risk across the organization, enabling a metrics-based approach to risk reduction as a core business value.

These measurable outcomes helped cement leadership buy-in and further embedded a culture throughout the organization that security is a shared responsibility.

“The end goal is behavior change, and while we originally started with awareness as the focus, I can say we’ve seen a shift in user behavior for the good because people are engaged with Hoxhunt. I know if I go a week without seeing a Hoxhunt phishing simulation, I go through my inbox and look for them.”

RODRIGO GARCIA
Manager, Cyber Threat Management, AES

Change management: New cyber awareness model

Communication is critical for a behavior change program to thrive. The Hoxhunt platform accelerated the deployment of AES’s new cyber awareness model, to include:

  • Review of a monthly phish in each safety meeting
  • Communication of the leaderboard dashboard and recognition of high performers (star collectors) via:
  • Monthly safety meetings
  • ELT Recognition
  • Safety Day / Values Day
  • Highlight in the Newsletter as “Always on for Digital Safety”
  • Yammer and other internal social media

“The sense of community, the sense of awareness, and the instant feedback on reported real-world phishing attempts are all things people like about Hoxhunt. The gamification, the collecting of stars, and the camaraderie that comes with seeing yourself on a leaderboard have had a positive effect on our culture, which has helped us measurably transform our human risk posture.
With phishing simulation engagement rates reaching above 60 percent and failure rates dropping below 2 percent, Hoxhunt has helped us push our resilience into new territory, with our resilience ratio jumping by over 2000 percent in just a few months.
Hoxhunt has helped us surpass anything our legacy SAT tools could deliver.”

RYAN BOULAIS
VP & Chief Information Security Officer

Ryan Boulais, CISO, and David Badanes, Director of Cybersecurity Strategic Initiatives, talked with us at the CSO50 2023 awards in Arizona about how Hoxhunt fueled their cybersecurity behavior change journey. Learn how they went from stagnant awareness results with old-school SAT tools to CSO50 Award recognition with Hoxhunt's leading edge Human Risk Management platform.

Want to match these results?
Hoxhunt adaptive phishing training dramatically increases training engagement and security resilience.
Request a demo