How Avanade reduced human cyber-risk and saved 5 SOC analyst FTEs with the Hoxhunt Human Risk Management platform

Overview: Breaking down the Protect-Detect-Respond siloes
‍

The NIST framework contains five cybersecurity categories including Protect, Detect, and Respond. Traditionally, awareness is stuck in the Protect silo, and Detect and Respond operate within their own. But 82% of breaches begin with people, and employees are the first alarms for the sophisticated phishing attacks that bypass technical filters.

Avanade looked beyond the traditional SAT model and adopted security behavior change and human risk management capabilities with Hoxhunt.

“With Hoxhunt, we’ve gotten more visibility into how resilient our people are and more capability at being able to increase that resilience with education and training.”

– Ray Reyes, IS Cybersecurity Operations Manager, Avanade

‍

Avanade dissolved the security siloes and augmented threat detection and response capabilities with human threat intelligence. Employee threat detection reached new levels in both volume and accuracy. The transformation led to an enviable problem: too much threat intelligence.

“People began reporting far more threats, which was good. But it was a huge volume to analyze and that created a new problem that we had to solve by seeking out new capabilities. The Hoxhunt Response platform automates threat analysis incredibly accurately and efficiently, and lets us manage that volume, which is just massive, and very hard to keep up with manually.”

– Ray Reyes, IS Cybersecurity Operations Manager, Avanade

‍

‍

Hoxhunt helped augment protect-detect-respond capabilities while reducing the security team’s workload. By one calculation, the Hoxhunt Respond platform did the work of over 5 full-time threat analysts per month. Legitimate emails, malicious emails, spam, and incidents were automatically categorized and prioritized for accelerated incident response.

“The Hoxhunt Response platform lets us focus on real threats.”

– Greg Petersen, Senior Director of IT Security, Avanade

‍
‍

PROTECT: Beyond awareness to Security Behavior Change

‍

The Hoxhunt challenge benchmark study

To validate the effectiveness of Hoxhunt, Avanade performed a benchmark study in a proof-of-concept trial. It compared the phishing training results of 1,000 Hoxhunt users against 1,000 users of their existing awareness tool, which required a great deal of manual effort to create phishing campaigns.

More importantly, they weren’t confident their old tool was delivering meaningful results. The findings of the benchmark study proved them right.

‍

‍

*Resilience ratio divides the phishing simulation engagement rate by the failure rate

• Success rate equals reported phishing simulations
• Miss rate equals neglected phishing simulations
• Simulations sent more frequently and automatically adapt to user background and skill level
  ‍

Hoxhunt outperformed the SAT tool across the board. The AI-enabled automation coupled with a dedicated customer success team worked to significantly lighten the operational load for the security team. The automated threat data orchestration of the response platform did the heavy lifting for the SOC team. All that and seamless Microsoft integration, too.

Avanade felt confident that the Hoxhunt results provided a more accurate and actionable picture of their risk and resilience. Higher engagement meant a larger sample size. And the adaptive learning model enriched the failure rate with more context and meaning; Hoxhunt phishing simulations are designed to get harder as user skill level rises.

“We’ve been able to augment our humans with this behavior change training as well as augment our SOC with the Hoxhunt AI.  It’s not just making the human layer stronger and smarter, it’s making them smarter in specific ways that can augment our technology.”
‍
– Greg Petersen, Senior Director of IT Security, Avanade

‍

Powered by AI and managed by a committed team of customer success professionals, Hoxhunt could automate a behavioral science-based training program containing:

• Automatic translations for a global workforce
• Personalized learning journeys automatically tailored to individuals’ backgrounds
• Adaptive learning model that automatically trains users at the edge of their skill levels
• Up to 36 phishing simulations per year derived from actual phishing attacks
• Positive micro-training experiences
• Continuous, ongoing practice with dozens of realistic phishing simulations
• CXO reports to help communicate the progress, value and ROI of the security behavior change program

‍

Phish your friends

‍

Avanade’s “Annual Phishing Tournament” was introduced to educate employees on the perils of phishing and increase their threat reporting behavior. The first phase of the tournament was called “Phish your friends,” in which employees created phishing emails in order to prompt hands-on learning of a phishing email’s anatomy. The second phase of the tournament then asked all employees to “Go Phish” one another, with the most effective social engineers winning prizes and recognition.

The results were great, and further validated the Hoxhunt resilience metrics:

• 2.1% failure rate
• 95,000 emails sent
• 63.5% phishing simulations reported
• 84,200 views on phishing tournament comms
• 17,900 engagement moments
• 4,700 employees reported all phishing tournament simulations

“Phishing training can get repetitive for people, and Hoxhunt totally turned that around. Phish Your Friend helped boost engagement and really raised the bar. There was not insignificant work to drive that campaign, but partnering with Hoxhunt helped move it along and the generated reports were great.”

– Ray Reyes, IS Cybersecurity Operations Manager, Avanade

‍

Avanade had done Phish Your Friend campaigns with their previous tool, but Hoxhunt lightened the load with automated reporting, and improved the quality of data for better results.

‍

DETECT: Human intelligence is a terrible thing to waste
‍

From awareness, to security behavior change, to human threat intelligence

Avanade employees have demonstrated progress across every key area of security training: overall user engagement, simulated threat reporting rate, and simulated phishing test failure rate.

But the most impressive metric? Real threat detection. With Hoxhunt, both the volume and accuracy of real threat reporting has soared. A threat report is the ideal outcome of a phishing attack because it alerts the security team to the danger and removes the threat from the system. And accuracy is important because reports can include legitimate emails and spam. SOC analysis for each report takes time, and false reports can disrupt workflows and operations.

In just three months, as behavior change participation soared, the total number of real threat reports doubled. Meanwhile, the number of false positives dropped by 98% with the adoption of the Hoxhunt Response platform.

“What Hoxhunt is doing is bringing the power of human intelligence into the security stack, and not just as a passive last line of defence. Now you have human threat detection as an integral part of the whole stack.”

– Greg Petersen, Senior Director of IT Security, Avanade

‍

Connecting phishing prevention to enhanced threat detection

As excellent as Microsoft’s security suite is, Hoxhunt makes it stronger with human threat intelligence. Reported threats alert Avanade to what has infiltrated the system and enables them to respond. These threats are removed from Avanade’s ecosystem and simultaneously reported to Microsoft, ultimately training the filters to stop similar attacks. This is the global human threat detection network in action.

People are doing more than just avoiding phishing attacks. They are actively detecting them. The benefits of a human threat intelligence network that plugs into the center of the security stack are significant.

‍

RESPOND: Closing the loop

‍

Building a Protect Detect Response engine with Microsoft integration

The Hoxhunt platform is powered by AI and fueled with human intelligence. First, with Protect, people learn to recognize and report phishing simulations as a habit. Then, with Detect, real detected threats are transformed into a learning experience by the Hoxhunt AI. Finally, with Response, the detected threats are automatically orchestrated for response, and new attacks are shuttled back into the training at the Protect stage to keep it cutting edge.

“It’s more of a closed loop from that protect-detect-respond capability. We’re able to close that loop and feed data from Respond into how we better protect ourselves. With Hoxhunt, you’re augmenting capabilities in multiple places: at that last line of defense, where the human is being attacked, and you’re also augmenting the SOC team.”  

– Greg Petersen, Senior Director of IT Security, Avanade

‍

Avanade used the Hoxhunt Response platform to do the heavy lifting on threat analysis at enterprise scale. The AI-powered response platform automatically categorized and prioritized employee-reported threats for response and mitigation. As training goes on, the accuracy of employee threat detection improves continuously. Real threat detection accuracy more than tripled after 3 months. Employees effectively reported spam as spam, threats as threats, and did not report legitimate email as phish.

“The augmented intelligence—the human plus AI—is reducing the burden on the SOC team. You’re not just catching and stopping attacks, you’re enabling the people being targeted and you’re enabling the SOC with this augmented intelligence to solve those problems and respond to those attacks at scale.”

– Greg Petersen, Senior Director of IT Security, Avanade

‍

The Hoxhunt Response engine does the job of between 3-5 full-time equivalents of threat analysts on a monthly basis, according to an analysis that assigned a time value to each incident report that was handled by the Response engine, and how that compares with a threat analyst. This efficiency has allowed Avanade to enter new territory in human risk management.