Measuring the success of your SOAR: 5 criteria for enterprise security teams

As more and more organizations use automation and orchestration to streamline their security operations, defining clear success criteria becomes critical to ensure the effectiveness and scalability of their program.

Recently, an enterprise prospect approached us seeking help on establishing success criteria for their upcoming journey with Tines workflow automation. While there are many factors that contribute to the success of your automation program, we identified five critical criteria that serve as pillars for achieving impactful outcomes.

These criteria not only set the stage for a successful Security Orchestration, Automation and Response (SOAR) journey, but also lay the groundwork for driving efficiency, scalability, and resilience within security operations teams.

This is not an exhaustive list - of course, there are other criteria to consider - but we wanted to share our list of the most critical SOAR performance indicators.

We've also included a list of questions to ask when auditing your automation journey, which are particularly important to ask during SOAR evaluation.

Performing a SOAR evaluation: key success criteria and questions to ask 

1. Reduction in response time 

Question: How much time does it take your organization to detect, analyze and respond to security incidents? 

Successful SOAR implementation should significantly reduce response times, allowing security teams to respond to threats quickly and minimize the potential impact of incidents.

2. Increased efficiency in incident handling

Question: How efficiently is your incident handling processes today? How many incidents are being handled per week, month, quarter, and year? 

A successful automation program should streamline incident response workflows. You should be looking to increase in the number of incidents managed without compromising performance or accuracy. Reduced manual effort is a great indicator of efficiency gained through automation.

3. Enhanced threat visibility and intelligence integration

Question: What sort of threat intelligence feeds are being assessed? Do you have a comprehensive view of the security landscape? 

A successful SOAR platform should seamlessly connect your threat intelligence sources, providing teams with relevant, real-time information. Improved threat visibility, which enables proactive threat detection and response, is a great indicator that your SOAR platform is performing well.

4. Improved collaboration and communication

Question: What processes are you using to facilitate cross-functional communication and collaboration between security teams, key stakeholders, and other teams in the organization?

SOAR platforms should enhance communication channels and collaboration across security and non-security teams. Success is demonstrated by improved information sharing, coordination, and decision-making, all of which lead to a more unified and responsive approach.

5. Quantifiable reduction in mean time to remediate (MTTR)

Question: How long does it take your teams to remediate security incidents end to end? 

After automation remediation tasks, you should see the positive outcomes of streamlining the process, including a significant reduction in MTTR.

Building your automation program - how Tines can Help  

Incident handling 

These stories from our library of pre-built workflows demonstrate how Tines can be used to automate tasks associated with incident handling, including detection, communication and remediation.

1. Create issues in Jira from alerts in Cybersixgill and notify with Slack and email: This workflow connects Cybersixgill, Jira, and Slack to handle incidents by creating issues in Jira from alerts and sending the relevant notifications.

2. Identify and remediate high AWS EC2 Disk Usage with Elastic Observability and document with Tines cases: This workflow connects AWS, Elastic, and Tines to monitor and handle high disk usage incidents on EC2 instances.

3. Detect and remediate high AWS EC2 CPU usage with Elastic Observability and document with Tines cases: Similar to the previous workflow, this focuses on high CPU usage and connects the same tools for incident handling.

4. Remove a user account in Microsoft Entra ID, BambooHR, AWS, and Slack: This workflow involves several platforms, including AWS, BambooHR, Jira, Microsoft Azure, and Slack, and uses them to handle removing a user account.

5. Create & manage incident communications with Slack and Jira: This workflow allows teams to manage incident communication directly through Slack and sync conversation backups to the relevant Jira ticket.

Pre-built workflows for incident handling

Create issues in Jira from alerts in Cybersixgill and notify with Slack and email

Tools: Cybersixgill, Jira Software, Slack

Identify and remediate high AWS EC2 Disk Usage with Elastic Observability and document with Tines cases

Tools: AWS, Elastic, Tines

Detect and remediate high AWS EC2 CPU usage with Elastic Observability and document with Tines cases

Tools: AWS, Elastic, Tines

Remove a user account in Microsoft Entra ID, BambooHR, AWS, and Slack

Tools: AWS, BambooHR, Jira Software, Microsoft Azure, Slack

Create & manage incident communications with Slack and Jira

Tools: Jira Software, Slack

Quantifiable reduction in Mean Time to Remediate (MTTR) 

Here are just a few examples of how Tines customers use automation to have a positive impact on their MTTR, either by reducing MTTR or speeding up remediation time.

1. How Turo aligns teams across platforms: Turo automated the exchange of information between Jira tickets, saving between 15 and 45 minutes per investigation.

2. How Snowflake reimagines case management with Tines: Snowflake reported saving approximately 10 hours per day with alert correlation and enrichment.

3. How Applied Systems regains 4–5 days of work time monthly: Applied Systems' team reduced time to respond to just 30 seconds per alert.

4. How Sophos frees up one analyst per week: A Tines workflow allows analysts at Sophos to receive all the information they need the first time they look at a case.

Improved communication and collaboration 

1. Communicate across channels in Microsoft Teams and Slack: This workflow helps teams keep on top of messages across several platforms. It uses the Microsoft Graph and Slack APIs to send text messages in real-time across Slack and Microsoft Teams channels.

2. Create shift handover notes in Confluence send reminders in Slack: This workflow helps create shift handover notes in Confluence and send prompts in Slack, ensuring smooth transitions between shifts.

3. How PathAI created a better working relationship with their people team: PathAI's security team built a workflow that allows them to streamline onboarding through automation, using a Tines Pages form to gather information from their people team.

Pre-built workflows that enhance communication and collaboration

Communicate across channels in Microsoft Teams and Slack

Tools: Microsoft, Microsoft Teams, Slack

Create shift handover notes in Confluence send reminders in Slack

Tools: Confluence, Jira Software, Slack

These success criteria serve as a framework for leveraging SOAR to strengthen an organization’s security posture. By incorporating them into a security automation programstrategy, organizations can proactively address security threats, streamline incident response, and enhance operational efficiency.

As the security landscape evolves, it’s essential for organizations to evaluate how they're measuring against these criteria and how they're continuing to improve, ensuring that their SOAR platform remains aligned with their challenges and business objectives.