Using Records to improve story performance in Tines

In a guest blog post, Aaron Jewitt, Principal Detection Engineer at Elastic, walks us through a Tines workflow he built to improve the efficiency of his team's automated alert triage processes. He recently shared how his InfoSec team uses Tines to automate alert triage over on Elastic's blog, which makes a great companion piece to this post.

The problem: automated triage result tracking

This use case for Records came from a need to be able to track the automated triage results from parallel asynchronous branches of a story, and to improve performance by providing a way to cache values for a longer period of time. This allows you to check if a value exists in a record before running through the entire workflow.

In this example workflow, also known in Tines as a story, we're taking an alert from our Elastic SIEM and checking a lot of different Elasticsearch index patterns for the source.ip field from the alert. Which index patterns we check is based on tags in the rules, as explained in my blog post on elastic.co.

Read Aaron's blog post on how Elastic's InfoSec team uses Tines to automate alert triage.

The solution: using Records to improve story performance in Tines

For this workflow, we use a record containing two fields, a source.ip text field, and a boolean field called is_managed that we set to true if a source.ip belongs to a system that we manage. We set the record to "editable" so we can update the fields based on the results of our story. We also set the retention period to one day, this will delete all older records, which saves space on our tenant, and forces a re-check of a source.ip once per day.

Let's take a closer look at the steps. Or you can skip to the end of the blog post to explore the workflow for yourself.

Before the alerts are triaged, we check to see if a record already exist for that source.ip and if it's been marked as managed. If it has, the source.ip is a known good and we can close the alert and skip all of the following actions. This dramatically reduces the load on our tenant and our Elasticsearch clusters from unnecessary queries.

As we start our triage path, we create a new record for the source.ip and mark is_managed as false.

After creating the Record, we split the processing into multiple different paths based on alert tagging, where each path will run different checks against the source.ip. The branch on the far right will sleep for 5 minutes, then check the record to see if the IP has been marked as managed. If not, it will send an alert to Slack.

If the source.ip is found, it's sent to a Send to Story action to close the alert. The Send to Story action will deduplicate by record ID, so we don’t need to update the same record twice. It will then update the record to mark it as managed, and tag and close the alert in the Elastic SIEM. We pass the following values to our Send to Story action:

The record id field, in this case it is create_record_for_source_ip.id.
The _id field from the Elastic Alert so we can close and tag the alert
A tag field containing the string that we will add to the alert in our SIEM to make it easier to track alert metrics.