In this episode of The Future of Security Operations podcast, I'm joined by Andrew Santell. Andrew is an experienced security leader who worked for the U.S. Navy for over a decade before moving into the private sector. In 2021, he founded the Security Operations program at Netflix, and recently, he joined edge cloud platform Fastly, where he is the Director of Security Operations and Cyber Defense.
Andrew and I discuss:
Navigating the unique challenges of the Navy, from log management to prioritization
Making the leap from the Navy to tech
Building a security operations team and program from scratch at Netflix
Red teaming phishing response playbooks at Netflix to test their effectiveness
Recognizing the value of good processes
Why teams should design processes first, automate later
Creating a feedback loop between teams at Fastly
How “shifting left” has helped Andrew’s team reduce vulnerabilities
Using automation for risk assessment at Fastly
Andrew’s approach to incidents like the Log4J vulnerabilities
Why growth in the vendor market is a good thing for practitioners
Why automation should be a requirement, not just a best practice
What advancements in AI mean for threat detection
The importance of risk-based decision-making
The potential of self-remediation
Why good security leadership starts with taking care of your people
The Future of Security Operations is brought to you by Tines, the smart, secure workflow builder that powers some of the world’s most important workflows.
Where to find Andrew Santell:
Where to find Thomas Kinsella:
Resources mentioned:
In this episode:
[02:05] Andrew’s career journey so far
[05:35] The unique requirements of working in the Navy
[09:12] Risk-driven decision making
[11:11] Self-assessing phishing response controls and mitigations at Netflix
[14:28] Andrew’s decision to leave the Navy and his transition to the private sector
[16:12] Comparing approaches to security at the Navy and in tech
[19:26] Breaking free of bad processes
[23:20] Broadening roles to include pen testing, application security, and vulnerability management
[27:27] How Andrew approaches automation at Fastly
[31:56] Protecting Fastly’s infrastructure
[33:57] How SecOps has changed and where it’s going next
[40:18] Embracing automation for vulnerability management
[42:45] Taking care of your people as a security leader
[44:56] Making engineering and automation part of prioritization
[47:19] Connect with Andrew
TL:DL? Read Andrew's take on…
Security operations at the Navy:
“Being able to have visibility, and the right telemetry was really, really important. There were more than a handful of investigations where it was difficult to gather. There are some unique requirements that really only exist at the Navy. If you think of all these little floating cities called ships, it's really difficult to pull all the data back at any given time. So we had to get creative about how do we collect and store this. Because it's not like we have massive data centers on these ships, either. Being able to collect the right data and being really specific about what that is, was really important."
Self-assessing phishing response at Netflix:
“Basically, we red-teamed ourselves. We started looking what would happen if someone clicked on the link and so on. And what we found through this assessment was that we were actually pretty well protected through the controls we had in place. We had a really good zero trust environment, so that, if someone got in, they're not quickly able to move around laterally, they were pretty much contained to a specific environment."
One of the first realizations is an obvious one, but we calculate risks a lot differently. In the Navy, if you’re looking at the sensitivity of data or incidents, you’re often measuring loss of life, unfortunately. In the private sector, it's about loss of revenue.
The similarities between the Navy and the private sector:
“I remember the first one was I heard someone talking about NIST Cyrus Security framework, CSF, and it really caught me off guard that people in the private sector would be talking about NIST CSF. I thought, 'Wait, I know that, I can help with this!' Finding those commonalities helped with any like imposter syndrome."
Leveraging feedback across teams:
“The feedback loop is so tremendously important. If you see an incident, your job doesn't stop as soon as you've mitigated and stopped that threat. You need to gather what were your lessons learned from this and then ask, ‘hey, can we improve our development process, our CICD pipeline?’ ‘Do we have vulnerabilities out there that haven't been patched, and this is the reason why we got exploited?’ We've created that feedback loop between these teams, and it's been really helpful so far. By knocking down the noise or just preventing these attacks from happening in the first place, that makes incident response a heck of a lot easier."
Vulnerability management:
“We’re trying to take a new approach to vulnerability management and not just send thousands of tickets over to engineering and say, ‘Here, you go, fix this!’ We can prioritize this truly based on risk and make sure we're going after the ones that really matter most to us, not just take the CVSS score and say, ‘Okay, we have 100 highs based on CVSS’. If you actually do that analysis, you might only have three that are really applicable to your environment, that you really care about.”
There's a very bad word in private industry - process. It was almost like we don't talk about process, we want to give developers and engineers freedom. But as companies mature and grow, you recognize the value of process. To be honest, at the Navy, there was mountains of process, and a lot of them did slow us down to an extent. It did teach me the value of a good process, and what a bad process looks like.
Automating risk assessment:
“We've, for the most part, automated the assignment of a risk score with a given priority. So we do take CVSS, and we do take a first EPSS, there's CISA’s KEV... Vulnerability on a public-facing asset that processes customer data, we care a lot more about that. So taking into all this information, we can quickly assign. If it's getting a lot of the same attention, we know it's being publicly exploited. All these things will quickly escalate on that. And if we actually see any evidence of us being potentially attacked - Log4j is one example - as soon as these proof of concepts are out there to exploit them, you just see massive scans across the Internet. So if we start seeing us being probed or potentially being attacked here, then we're gonna quickly go into incident mode.”
The importance of leadership in incident response:
“In severe incidents, everywhere I've worked at, everyone just comes together, and it's just a beautiful thing to see. No one says, ‘I have another priority’, or ‘Why did you page me?’ You don't hear those things when the sky actually is falling. And the reason you get that is, not only are people passionate about their jobs, but because you don't abuse it. If you're using people for Sev 3s and 4s and low vulnerability, then they're not going to be there for you when you really need them."
Staying agile in incident response:
“As you learn more through your triage phase, or you know, Log4j, I'm sure we all remember, did actually change. There was a patch that came out, we think we're all good, and then was a few days later, ‘Oh, we found another vulnerability in here and we have to start the process all over again.’ So it’s about making sure that you're flexible and adaptable, and making sure that you're shifting your process as the severity of the incident is also changing.”
Shifting priorities in SecOps:
“We've moved more towards cloud infrastructure. We’ve moved towards a zero trust model. So then, on the security operations side, you see less reliance on network-based IPS or PCAP. There’s an old saying - it didn't happen if there's no PCAP. But today I don't really hear people saying that. We care more about the access logs. What's the host-level telemetry like? So I think this shift in priorities from security operations is directly tied to how the whole industry and how different companies are architecting their security posture.”
[AI] is not here to take our jobs. There’s not a world where that's not part of security operations. Maybe it doesn't take over everything, but I would love to see continued advancement there, being able to automate more, be able to identify threats that we just wouldn't have been able to otherwise.
The potential of self-remediation:
“We've seen some examples of self-remediation. I think that's the dream. And I think, that was DEF CON’s Cyber Grand Challenge, like several years ago. The idea was, you're getting attacked - self heal. I think it'd be really great. We can get to the point of known vulnerability - 'Hey, here's a patch. Can we patch this and then push us out to prod?' The biggest concern with that is making sure you're getting the right testing in there. Especially for us at Fastly. We do have a lot of hardware infrastructure. And we have to time our patches accordingly. We definitely don't want to impact customers or anyone, and that's probably true for most companies. But being able to self-remediate and self-patch. Maybe it's not your really high ones, maybe you start with the lower severity ones. But if we can get to that point to be able to identify that patch, that's that's the dream.”
What makes a good security leader:
“Being able to take care of your team, building a strong team, having a good diverse hiring pipeline, and bringing on the right people, then doing everything you can to retain them. I mean, they're what makes you successful, and just being a good human, taking care of them, making sure you're building trust and supporting their career growth. That's absolutely the best thing that you can do.”
There are so many things in security that we need to do, and at the same time, you're constantly fighting incidents... If it's just in a backlog, your team sees that backlog and they they stress about it. They try to get something done so they can pull something off that backlog. If you pull that backlog, and put it in a 'parking lot', something that you don't actually look at, I think that'll help you really focus on your top priority.
Security team burnout:
“You do have to be proactive in supporting mental health. And it's not just about hours of work, it's not just making sure people don't get paged in the middle of the night. It's about taking care of your folks and making sure that they're challenged, they're able to work on things that they want to work on. If they're constantly dealing with a process or manual task and not actually prioritizing and putting in your strategy, then they're gonna start to burn out or look elsewhere.”
His SecOps wish list:
“Everyone embraces, yes, we need to automate more security operations. But I think there's still a big struggle to actually make it happen. What I'd hope is that it becomes not just a best practice but a standard or a requirement... Rather than just it being a best practice, you don't start your operations unless you built in this automation from the beginning.”
Listen to more episodes of the Future of Security Operations podcast.