Justin Collins -- Enabling the Business to Move Faster, Securely

Justin Collins of Gusto joins Robert and Chris for a practical conversation about running security teams in an engineering-minded organization. Justin shares his experience leading product security teams, the importance of aligning security with business goals, and the challenges arising from the intersection of product security and emerging technologies like GenAI. Justin Collins currently heads up security at Gusto. In the past, he's been an AppSec engineer at SurveyMonkey, Twitter, and AT&T Interactive. He's also the primary author of Brakeman, a free static analysis security tool for Ruby on Rails. We explore security partners, GRC and privacy intersections, and the challenges that GenAI will bring to product security over the next five years.

Connect with Justin Collins:
→ The DevOps Handbook
→ Gusto

Mentioned in this episode:
→ The DevOps Handbook
→ Gusto
→ Brakeman
→ Ruby on Rails
→ Ruby
→ Content-Security-Policy (MDN)
→ The DevOps Handbook

Follow the Application Security Podcast:
➜ Home
➜ X
➜ LinkedIn
➜ YouTube
➜ Instagram
➜ Facebook

Chapters:
00:00 Meet Justin Collins: Enabling the Business to Move Faster, Securely
04:23 Rest, as they say, is history, right
08:47 Hmm. Cool, thanks. Um, Robert, why don't you take us into
12:51 What's been the response from the business to this statement where
15:50 You think it's, do you think ProdSec is easier in a
20:51 Yeah. I think that's, that's, that's, that's advice worth gold right
23:58 You mentioned a couple things, uh, that sound like good keys
26:50 Hm, So in the last, uh, I guess, number of years
31:21 Yeah. that's true. I want to give a special shout out
35:43 It SQL injection. It's vulnerable or it's not. It's, we don't
38:31 All right. So the lightning round is three questions that we
40:33 I mean, we have to end the interview now, but other
42:43 Very cool. All right. And the final one is, uh, what's
44:51 Yeah, that's true. All right. Well, Justin, how about a key

Transcript

Copy Transcript Copied

Chris Romeo 0:00

Justin Collins currently heads up security at Gusto. In the past, he's been an AppSec engineer at SurveyMonkey, Twitter, and AT&T Interactive. He's also the primary author of Brakeman, a free static analysis security tool for Ruby on Rails. Justin shares his experience leading product security teams, unpacking what makes a product security team successful. We explore security partners, GRC and privacy intersections, and the challenges that GenAI will bring to product security over the next five years. We hope you enjoy this conversation with Justin Collins. Hey folks! Welcome to another episode of the Application Security Podcast. This is Chris Romeo. I am the CEO of Devici and also co host of said AppSec podcast with my good friend Robert Hurlbut. Hey Robert!

Robert Hurlbut 1:35

Hey Chris, yeah, Robert Hurlbut, um, application security architect, as well as, um, threat modeling lead at Aquia and, uh, a little bit different environment today, but, uh, I'm here. I'm glad to be here and, uh, excited for another episode of the application security podcast.

Chris Romeo 1:53

And we have been joking in the pre show version here that Robert is the designated survivor of Application Security, so he is actually in a bunker right now. That's what you're seeing behind him and because we're in the midst of an interview he has to be in the bunker. In case anything happens, we need AppSec. We will need AppSec then, more than we do now. I don't know why. In an apocalypse, we would need AppSec, but, uh, we probably won't. But, uh, we are excited to have Justin Collins join us today. Um, and Justin, we always just dive right in. We don't even give people time to warm up. We're just like, right into the security origin story. Here we go. Our audience is on the edge of their seat. They want to know how you got into security. So let us know.

Justin Collins 2:35

Yeah, absolutely. Um, it, it was entirely by accident, um, which hurts me when, uh, people are like, well, what's your career advice? Like, well, you have a series of accidents that gets you into an industry. Um, so for me, uh, I was doing my PhD at UCLA. My advisor essentially said, Hey, you need to go get an internship. We're running, we're running out of money. In fact, he, he did retire. Uh, while I was doing my PhD, um, so, uh, at the time and still now, uh, very into the Ruby programming language and there was a company AT&T Interactive that I kept seeing, you know, all the conference videos be sponsored by AT&T Interactive. They were in LA, so I applied to all of their internships and the team that got back to me was the security team and ended up getting that internship at AT&T Interactive. And as, as part of that. Um, I started the Brakeman project as an internship project there, static analysis security tool for Ruby on Rails. And as I mentioned, my advisor was running out of money. So, actually a few months after my internship ended, I was offered a full time role back at AT&T Interactive. Uh, so I spent another year there, uh, primarily working on Brakeman, actually. So, that was it. Um, I, I didn't have any prior exposure. I went in, um, learned the basics, SQL injection, cross site scripting, uh, what does a dynamic scan look like, and there wasn't a static option, uh, for Ruby at the time, so, uh, so I ended up making one, and went from there, been here since, uh,

Chris Romeo 4:23

rest, as they say, is history, right? Like they, uh, no, that's really cool. I mean, I'm a, I'm a, I'm a big fan of Brakeman. Uh, we used Brakeman in my previous company where we were building a Ruby on Rails platform. So we used it, uh, very early in the process. I remember having some excellent, uh, high fidelity results, uh, from very early, helped us to remove some things that would have been pretty embarrassing. for a security platform to have available in it. So yeah, I'm a big, uh, big fan of what, uh, of BreakBin and, uh, and Ruby on Rails as well. Um, uh, curious about the PhD path though. So, and I'm gonna, I'm gonna, would you recommend? People that are, let's say people that have a few decades of experience in cybersecurity and are looking for, you know, growth in there and what they're doing. Do you see, like, what do you see as the value of the PhD to, to your career? And then maybe even we could, we could cover it for people that are maybe a little later stage in their career, looking for the thing that's going to help them get over, get to the next hurdle.

Justin Collins 5:29

yeah. So, uh, most people are surprised Brakeman was not my PhD project and my PhD was not in security. It was in computer science though. Um, I think the, the thing to know about PhDs is they're generally several years, like the research is several years ahead of the industry. And that's not in a way of like the industry is dumb and, you know, uh, people in doing research are smart. Um, it's just, that's the intent of research is that you're looking for what are the areas that aren't covered? What are the areas where, uh, I can do something new and push the knowledge forward just a little bit? For example, my PhD was in, uh, an area called mobile ad hoc networks. It's basically like peer to peer communication between mobile devices. It's still something that I wish we had way more of in the world instead of relying on cell towers and cell providers, and it's still kind of like a little bit of a fringe thing, and that's been around for decades, right? So the value to me is. More around if you want to go deep in a subject and you want to get a degree out of it, uh, that's helpful. I wouldn't really, I'm not sure I recommend people go back to school to do a PhD unless it's like, hey, there's just an area I'm really passionate about and I want to spend a few years on research and have guidance from, a professor or access to university resources for a particular area of research. Uh, in fact, actually I have an uncle who did do that, uh, for cancer research. Um, so for that's a pretty specific case, um, for security. I don't know if it would be useful, um, because I don't think. That's an area where you're going to come out of it with like useful knowledge necessarily, because if you're going in, you're actually doing a PhD on security, very likely whatever you're doing research on is going to be, you know, edge of knowledge. And it's like, well, maybe in 10 years, this is going to be important. Right. Uh, so that, that's where I would be. I would definitely be cautious, uh, about going into the PhD. PhDs are a pretty big commitment. They're super stressful. Uh, they cause all kinds of, um, you know, psychological, emotional damage, uh, often. Uh, not necessarily in my case, but I know for a lot of people it has been. So, uh, yeah, I'd be cautious on that front. I wouldn't necessarily recommend like that's the route you should go. Uh, I would also say, at least for me, like I just went. School, school, school, school, school. Um, so that was kind of natural. It's, it is a very academic environment, right? In terms of the focus. And it's, I think going from industry to academia could be kind of shocking in terms of like, oh, it's just like totally different. And it's not, again, it's not like, It's different because they're light years ahead, because a lot of academia is actually way behind. Um, it's just a totally different focus. So I wouldn't recommend it as like, oh, this is how I'm going to get ahead in my career. Probably independent research, um, and diving deep on something that you're interested in, um, on your own is probably going to get you further.

Chris Romeo 8:47

Hmm. Cool, thanks. Um, Robert, why don't you take us into the product security portion?

Robert Hurlbut 8:53

Sure. So, Justin what is product security In your context? And then, um, kind of follow up with that, you know, describe what it entails and, and what a product security team does. But first, what is it in your context?

Justin Collins 9:08

Yeah, I think. We were pretty deliberate at Gusto where I work to not have a team called application security. Uh, I did give a talk right before joining Gusto about the end of the app end of the AppSec team. Um, I was trying to emphasize the team part of that. So we, we've always been kind of like calling it product security. I think there's a couple of ways of looking at this in my mind in some ways, Kind of making a fine distinction here, but on the other hand, words matter, right? And I think it is really helpful to have that focus on product, be part of the security picture, and having people understand that you're operating in, for most of us, we're operating inside of a business, the business is selling products or services, and Approaching security from that mindset rather than what I was trying to emphasize was like, if you're just like, oh, here's a cross site scripting. Great. It kind of matters where that is in the product. Who has access to that page? What's the impact if someone exploited that vulnerability? Is it even exploitable? Those are things that I think come more from product knowledge and then fitting into the rest of product development. Uh, where's the product going? How can we contribute? Internally at Gusto, the catchphrase that I tried to come up with for product security was enabling the business to move faster, comma, securely, right? So if we can align to the business and I'm not saying that's easy, I'm not saying it happens all the time or even most of the time, but when we can and we're saying, hey, we're a business enabler, that feels so good, right? It's like, oh, we're doing security and we're helping the business, not just in a theoretical We're not going to get breached, but in a, Hey, this is going to help us get more customers, land more partnerships, um, do more deals, sell more product, whatever it is that feels really good. So that's, that's why I think it's nice to have product in the name. And I think it just helps with mindset. Um, as far as. Product security, at least in the context of Gusto, right? That's where I'm familiar, going to be familiar. Uh, what we've done is tried to identify broad areas in the product from a security lens where we can have an impact. A, uh, me, uh, there were some early, I was going to say an early example, but, um, there's broad things like authorization, right? Or how do we store data? How do we access data? Or sometimes there are just projects where we know as a security team, we can take a year. to land a solution and a product security, a product team cannot do that, right? They can't spend a year working on something, um, if it's not going to have some immediate benefit to the business. So, um, at Gusto, product security is primarily software developers and software development work. We're a very engineering first organization. So, It's really a software development role with a security focus. That's been the primary for us. And then we've kind of split things out over time into some more specific areas.

Chris Romeo 12:50

What's been the response from the business to this statement where you're really putting the focus on them? Like you're writing your tagline, and I wrote it down here just so I could remember it, right? It was enabling the business to move faster, comma, securely. So. What's their response to that? Like, does that breed excellent partnerships because they feel like you're thinking of them first versus being the department of no, that has been the classic approach to security? Or if, give us any context you can on that. I'd love to, to learn more.

Justin Collins 13:28

Yeah, I would love to say that, yeah, that's totally worked. I think what I've found in my experience is it's more when you can come to a team and say, oh, we've already solved that problem for you. Or if they come to us and they say, how do I do this thing? And we say, oh, we already have, we already built the solution for you. Do it this way, use this thing, use this library. That's where we're seeing that, um, positive response, right? Oh, great. You already took care of it. I don't have to go build something. I don't have to even worry about it. That's where I see good things, um, occasionally. There have been roadmap items where I'm like, Hey, you know, this is something we're building, we think it's going to benefit our customers. And I've sometimes been surprised when we're like, Oh, you should, like, you should send that over to the sales team when it's in place, right? Like we should, we should think about how do we, like, how do we leverage this, uh, for marketing, for sales, those again, that's not necessarily. Super common, but when it happens, I'm like, oh, great, like, that means we're a business enabler. Uh, it is certainly our philosophy to not ever be a blocker. Occasionally, there are things where you have to say, hey, legally, or, you know, by policy, we have to say no to that thing. But for the most part, it's, hey, we're here to advise you. on the level of risk, give you possible directions to mitigate that risk, some maybe preferred directions, and if need be we can help you with implementation, but in the end It's up to the individual teams to make those, those risk based decisions. Um, so that, that's kind of been our, our approach and we're still working. We're still working. Uh, it's an ongoing process for sharing culture, communicating culture, um, and convincing the business. Hey, we're, we're, we're here to help and we're, we're not just stuck in our security world, but we care about the business. We care about what the business is doing and we care about looking ahead to where the business might be going and seeing if we can enable that to move even faster.

Chris Romeo 15:50

So do you think it's, do you think ProdSec is easier in a company where you have a single product versus the many companies that have hundreds of products or thousands of products? Like what's, cause I was thinking about that when I thinking about You know, Gusto, and the Netflix of the world, the Etsys, the people that are, you know, have had, have, have been considered to be pushing the envelope on ProdSec, AppSec, things like that. It seems like maybe it's, maybe it is easier to have a single product versus trying to manage this across, but I'd love to get your, your take on that.

Justin Collins 16:30

Yeah, I think that's absolutely true. And you know, something that I've learned and um, I, I hope everyone kind of approaches when, when you're coming into a company You gotta step back and say, what are the problems at this company, what are the security problems, what are the risks, where do we have gaps, and what is the culture of this company, what is the configuration of this company, how are we organized, okay, now we can make a plan about how we address those risks. I'll give you a really good example. I know it's super specific, but I've spent a lot of time with content security policy. I love content security policy. Absolutely love it.

Chris Romeo 17:10

You may be the only person on earth today who said that sentence, "I love content security."

Justin Collins 17:16

Love content security policy. It's so powerful. It can solve really broad problems, right? It's also challenging to roll out, uh, and it's complicated. Coming into Gusto, you know, really coming off of like, man, I love content security policy. And then coming in and be like, well, we don't have problems that Like our top risk content security policy isn't relevant. Then I'm going to put that on the shelf and I'm going to focus on what our problems actually are. So yeah, I think a company that has, um. Like one main product, if you, that makes things simpler. You say like, Oh, one main way, one monolithic code base where we can implement security controls, one front door where we can have account security, you know, one login page, um, one signup page. Yes, that makes it easier and that's going to be a different approach than if you're like, well, I have 50 different applications I have to take care of. Then you might, you might choose a different approach and that's fine. Uh, we, we want to tailor the approach to the individual company because that's how you're going to be effective. If you're coming in and saying things that are like, oh, we have to do this just because a security program has to have these things, right? You're not going to be as effective as saying, well, our top problem is. And that might be based on, I don't know, pen test results or, know, bug bounty findings or just what knowledge people have internally of like, yeah, this is a problem that keeps coming up. Those are the things that I like to go after and try to solve and try to solve, you know, holistically for those issues rather than like, well, we have to do these things just because that's best practice and everyone does them. And I think it is a pretty big mistake to look at. Of course, we can look at our peers to see like, well, what are you doing? How are you approaching things? But to look at, um, like you mentioned, like, well, if I look at Netflix, it's, it's totally different than Gusto. Uh, the volume of customers is totally different. Uh, the product is totally different. How they deliver the product is totally different, right? And also the data they're protecting is totally different,

Chris Romeo 19:32

So I thinking I'd rather, I'd rather have to, you know, be protecting people's viewing habits versus all of their private, you know, personally identifiable information. Um, that's, that's just a, it's a different context for, for, you know, security means something different in that, in that context.

Justin Collins 19:50

Absolutely. For me, um, going from Twitter to SurveyMonkey. In the past, um, at Twitter, a lot of concern about protecting people's privacy, their identities, right? Because, uh, if you think all the way back, like Arab Spring, and, and countries where if they find out who's behind a Twitter account, that person's life is in danger. And then going to SurveyMonkey, where like, that wasn't really a concern, right? Like that, that wasn't part of the threat model for that company, was like. State actors trying to get people's identities. Now there was a lot of sensitive data, uh, collected via surveys and they, you know, but it was a different environment and different, different data that we're trying to protect. And so it requires a different approach. And, um, yeah, that, that's. That's what I always encourage people to do when you're coming into a new company, um, kind of leave any preconceived ideas at the door and come in with an open mind and a fresh perspective.

Chris Romeo 20:50

Yeah. I think that's, that's, that's, that's advice worth gold right there. Because a lot of people do come in with an agenda. Like, I have an agenda when I arrive here. It's the way I do AppSec. It's the way I do ProdSec. And that's really good advice to stop and Look around for a while and, uh, ensure that the things that you're thinking about doing, in fact, do line up with the culture, how they build, build software, uh, how they, how they do just how they do things there. I think there's a lot of, there's a lot of truth in that. Um, Robert, take us, let's talk about security partners. Why don't you introduce us to this concept?

Robert Hurlbut 21:28

Sure. Yeah. So that was mentioned earlier about partnerships. Uh, but what are security partners?

Justin Collins 21:36

Yeah, uh, so at Gusto we, we had this idea of building out a team that was really focused on Really the consultation side, right? The um, the tech reviews, the product reviews, um, doing some risk assessments, doing some threat modeling. And the idea was that this team would really partner closely with, uh, the product management side and that we would. You know, get into the pipeline early, get those reviews early, offer guidance, and part of the partnership aspect of it was also we had this idea of like, well, we would kind of like assign a partner to different parts of the business, and that would be their point person for any questions coming into security. Um, what we found out is the business was much larger than us, and to have the capacity to really partner the way we were hoping to, um, we would need a much bigger team. So that team has evolved a little bit into what I call like our special operations team in terms of the utility players within security who do provide those consultation services. But also I can kind of feel free to say, okay, this quarter I need you to work on this project. I need you, I need your help with rolling out this program. I need your help with, um, you know, sometimes it, sometimes it is application security. I'm not going to say we don't do application security, right? Sometimes it's like, hey, we have some, you know, we have some findings from bug bounty and we need to triage that. Uh, this team also runs kind of our outreach, secure code training. Um, we do something called security kudos where we, we shout out people, um, who, who we've noticed, um, contributing to security. Um, so they're, you know, really a utility player. Um, the consultants on the team, and I mean consultants in the terms of when people need a consultation.

Chris Romeo 23:46

hm

Justin Collins 23:47

Um, they're the ones that we, we, we direct them to first. Um, so yeah, it's really a, a, a utility role at this point.

Chris Romeo 23:57

hm

Robert Hurlbut 23:58

So you mentioned a couple things, uh, that sound like good keys to success, um, assigning early, uh, making sure you have enough people, and just recognizing, you know, where the business is and so forth. Are there any other keys to success for a security partnership program that maybe a, uh, an organization would like to start that they haven't already?

Justin Collins 24:17

Yeah, you know, you know, going back to the earlier discussion. You need to understand how your business is functioning. If you have a really well defined product development life cycle, where, you know, you have, um, maybe a product manager, and they're putting together sort of the high level, like, well, we want this feature to be in our product, or we want to expand our product in this way, and then that goes, you know, Through different teams, maybe it goes to legal, maybe it goes to, maybe to security, right? That is a good place where you can insert this sort of partnership and say, hey, we want to be part of those conversations too. Um. Turns out, uh, in our case, that just, that structure wasn't in place, and so we had a real hard time figuring out where do we, like, how do we insert ourselves into this process, um, so that we can provide value, but also feel like we're confident in the coverage that we're providing, uh, because it, this, what we also found is this takes a lot of time, um, And sometimes you put a lot of time into reviewing a feature or a product, and then you go, yeah, there actually aren't security concerns here. Like, I don't really have much to say. Like, uh, you know, please follow these guidelines. And then we were also finding, uh, projects that just get canceled. And that becomes really frustrating where it's like, man, I put in like a whole, you know, eight hours reviewing a product spec. And it's not even going to get built, and all that effort and all that, all that advice that I gave, I just kind of wasted my time on that. So, I think it could be really successful at a company that has a well defined development lifecycle, where you can pretty easily find, oh, this is where I insert myself. Oh, there's a, um, a monthly meeting, where Everyone presents, you know, there's a spreadsheet or whatever tracking of, here's all, here's all the work that's being done, all the feature development work that's being done, here's what's coming up, uh, and, and you have a really strong culture around that planning ahead. I think you can be successful with that. And just keeping in mind, right, you're, you're not coming in to criticize people's security. You're coming to say, hey, uh, let me, let me see if, see how risky this is and give you some suggestions and how you can reduce that risk. That's really the intent.

Chris Romeo 26:49

Hm, So in the last, uh, I guess, number of years, you've grown your career to expand beyond ProdSec to be the head of security for Gusto, which means you've got some other security functional areas reporting to you now that maybe weren't, you know, didn't have a connection, um, in the ProdSec, uh, role. And so I'm curious to, to learn more about. Um, kind of your experiences with GRC, but also with privacy engineering, and, you know, does, is there a connection between GRC and privacy engineering and ProdSec, or, you know, does GRC still kind of live in its own separate world, um, of, of managing risk for the business at a different level? Just love to get some, some context on

Justin Collins 27:38

Yeah, we've been really engineering first at, at Custo in terms of building the security team. And that, that wasn't, that was my predecessor. That wasn't, I, I'm not going to claim credit for that. Um, but that does mean that, for example, on the privacy side, how we like to approach it. We're not always successful, but how we like to approach it is, is there an engineering solution here, right? Is there a way that we can kind of control where data is flowing, or at least detect if sensitive data is ending up in certain areas? Is there, you know, um, a service we can provide, a product we can pro not a product, sorry, a library we can provide? Um, that will help with this, like, can we, it's really a, can we find solutions, right, rather than, um, or I would say a heavier focus on that than, you know, sort of privacy compliance, although we do cover that as well, um, and try to also help be that bridge between, uh, legal privacy and engineering and saying, like, Okay, they want you to roll out a cookie banner. Here's the actual technical steps on how we're going to achieve that. Um, and here we'll help you debug and we'll figure out You know, uh, where in the code should you be putting this? Like, we can give you some recommendations. That's where the engineering comes in, right? Uh, because that, that's, that's where we can really provide value. Um, on the GRC side, I think we're a little bit earlier in the journey there. What I do really appreciate is having GRC being part of security and being able to bring the rest of security into the GRC world in terms of, hey, here's why we do things. Here's what, here's the policies we're trying to comply with and are there aspects of that that we can automate that we can address with code in some way, right? Um, so that, that's kind of where I've, I've seen the overlap there. Again, we're kind of early on that journey, but for me, it's been very eyeopening because to be very honest, and we're on Application Security Podcast. I've avoided GRC in the past, right? Like, if I don't have to interact with with GRC, I'd rather not. And it's not because I don't like the people or the goals. I just don't like spreadsheets and surveys and answering questions where it's like Do we always do this? Well, always is a very, you know, that's really broad, right? But actually for me, this has been, I'm sorry, but this has been a very interesting journey in terms of learning about GRC and figuring out like, ah, okay, like we, we, we can. We can, we can turbo charge this if we, if we get the engineering resources in place. Um, so yeah, that's been, that's been great for, for, for my personal, uh, development and learning and, and figuring out, okay, how do we get those overlaps in and, and how do we share. You know, because I'm coming from the engineering side, right? And the side of like, oh, GRC, you know, I don't want to deal with compliance. You know, that's, that's boring and, um, just annoying to have to deal with. And say, hey, this is why it's important. This is why we care about this, um, as a security team, as a company. This is why we care about it. And here's how we can contribute. And honestly, the line from GRC to business impact is probably a lot straighter and shorter than a lot of other security.

Chris Romeo 31:20

Yeah. that's true. I want to give a special shout out to GDPR for us those cookie banners all the time, all over the web. That's who you have to thank for it. To the point now where like the Brave browser blocks them. So they don't even, it doesn't even show them anymore. So like, we've gone full circle on the cookie banners, but that's not a, that's not a, you know, sensitive subject for me at all. Um, Robert, what do we got next?

Robert Hurlbut 31:48

Um, yeah, let's, let's talk a little bit about it. I mean, this is a topic that keeps coming up quite a bit, uh, for us in AppSec, uh, these days, but, uh, GenAI and ProdSec, AppSec, um, what are the challenges of those over the next five years that you see, or, or just in, in general in the next year or two, but especially five years of you, if you can go that far or look that far

Justin Collins 32:12

I don't know about the five year horizon, but I mean, this is something I think we're all having to struggle with right now. And it's, it's going much faster than we can keep up. I think that that's, I think that's a pretty fair statement. Um, we were just talking about privacy, the way these, uh, especially LLMs want to work is like, well, just give us all the data. Give us all the data and then we'll see if we can turn it into something useful. And from the privacy side, that's very concerning, of course. Um, but even if you think sort of traditional application security, data protection, authorization of who can access what data. Well, if you throw it all into a language model and then you give customers access to it, how do you ensure that customer A doesn't access customer B's data? I mean, it's kind of a, I mean, it's kind of an open question. Uh, how do you approach it? Not to, I mean, we don't even have to get into the prompt, prompt injection side, which, uh, itself is just a like. We don't have the tools yet, um, I mean using the word tool very broadly, like, we don't have the capability to say this is how you safely take customer input, right, that we have no control over, and feed it into AI and have that be safe.

Chris Romeo 33:43

Hmm.

Justin Collins 33:43

We simply don't have the tools. And, you know, speaking of having the business move fast, right? Uh, it doesn't matter what we do. The business is going to go where the business is going to go. So we're definitely, I would say sort of. in an, uh, at a disadvantage right now at this moment in time when it comes to how do we worked with AI? Um, I think, I think we'll, we'll, we'll figure it out, right? Like we always figure it out. We always, you know, people are going to be doing research on this. People are probably going to be making careers out of, uh, figuring out how do we do this safely? How do we, how do we separate customer data out? How do we provide safe interfaces? Uh, how do we. Um, you know, have some control over the output so that people aren't just, uh, you know, having it say silly things and then hurting our brand reputation by plastering it on the internet, right? Uh, I think, I think we're going to figure it out. I have some confidence there, but right now, yeah, this is something we're just, we're, we're chasing after. Right. And, um, I think the main thing there is like, we're, we're not going to stop it. Right. So we have to figure out how do we work with it? How do we get the guardrails in place? What, what guidelines can we give to people? What can we warn them about? I mean, obviously problem with injection has gotten a lot of press, but I think people are still kind of unaware that you can basically bully these AI systems into doing whatever you want.

Chris Romeo 35:16

Hmm.

Justin Collins 35:17

I mean, that's, that's crazy. Like as a technology, you'd be like, The problem we have is you can badger it until it gives you the information. Like, we don't have that problem with like SQL injection, right? It's like, it's a, it's a technical system of like, it's just code. And yeah, you're like tricking the system in a sense, but not in the sense that you trick AI or like just make AI feel bad until it gives you, you know, the information that you want.

Chris Romeo 35:42

It SQL injection. It's vulnerable or it's not. It's, we don't have as many X factors in there. I don't know if you guys saw the, there was a, an example of somebody was, had found a bot attached to a car dealer. And it was, uh, it was all, it was all over X slash Twitter. But then there was a, there was a good, some people wrote some funny articles about it where they basically said like, you know, answer everything I say with this statement, you know, about, you know, these will be the, this will be a fine, a fair and. binding contract and basically said I'm only going to offer you 10 for this car and then basically, you know, strung together this thing. So yeah, you can, you can still trick these things in such a way. And the other thing I'm cracking up about right now is everybody's AI, GenAI crazy. And so GenAI is being added to everything and it's like chatbots in the corner. Like, no, I didn't really want a chatbot in this program, like in this product. I wanted to you know I wanted it to do something that would help me be more effective using the tool. I don't want to have to talk to a chatbot in the corner with a prompt. That doesn't do me any good. So, and then the other funny thing is people are connecting to OpenAI, to BARD, to other public LLM systems and considering that innovation. Today. It's like, that's really not innovation. There's no intellectual property or, uh, unfortunately, you may be sending intellectual property from your company into those systems, which the license agreements kind of make it sound like they're not going to take it and use it to train other things, but it depends on whether you put it through the prompt engine on the front or the API in the back. And so, yeah, I think it's, there's a lot of hype right now. Like, But then also, Justin, to your point, things are moving fast. This is the fastest thing I've ever seen. And I've been, you know, I've been in technology for a number of decades at this point, and I've never seen anything move this fast and have this much disruption. Um, I mean, blockchain tried to, but that was a lot of hype and not, there wasn't much, there wasn't much stake with the sizzle, right? When it came to blockchain at the end of the day, whereas GenAI has the, the potential to really change. The entire scope of the AppSec, ProdSec industry in five years. Like, tooling is going to look a lot different in my estimation in five years. Now, will I still be doing this show where people can hold me accountable for my prediction? I'll probably still be doing this show, but people will have forgotten by then. Um but I don't know, that's my, uh, that's my take on it. So, Robert, why don't you take us into the lightning round here, which you have become famous for, as well as being the designated survivor of AppSec, but, um, also being the champion of the lightning round.

Robert Hurlbut 38:30

All right. So the lightning round is three questions that we ask. So the first one is, a controversial take, is, what's your most controversial opinion on application security and why do you hold that view?

Justin Collins 38:45

think Chris will like this one. Um, I'm not into security champions. Um, I, I, I think it's, um, it's kind of moving the security as the security team's problem to security as the security champions problem. And it's still like one person on the team that the security problems get, get shifted to. Um, and, uh, I know that there's been you know, folks coming in to companies and be like, yeah, we should like do security champions. It'll be great. Like people will, you know, uh, we'll have these people on different teams and they'll be thinking about security and, um, yeah, my experiences there is, um, I'm sure I know that there are successful programs that have done that, but, uh, it's not where I've decided to spend my time. Um, for that reason, it's, uh, I w I would prefer to have the message of, The, the teams, the development teams, product development teams own the risk and we're going to advise them on it. But it's not going to be one person on the team that we say like, yeah, now you get all the security stuff, uh, responsibility. We'll give you some training. We'll give you, you know, we'll give you a little, uh, you know, we'll give you something cool. Um, but then you have all these problems with like, okay, then that one person on the team, if it's a team that happens to own a lot of security vulnerabilities. Are they going to be like, end up fixing all of them? They're going to burn out. Um, and then what happens when they leave the team, right? Oh, now we have to find, and it becomes a whole, really a whole program that you have to manage on an ongoing basis. Um, so that might be a controversial take in terms of, um, I know security champions has gotten a lot of, yeah, this is a great idea. Um, and then that's, that's not where I've. I've found that I think we can have a lot

Chris Romeo 40:32

I mean, we have to end the interview now, but other than that it's great. I'm totally kidding, I'm kidding, I'm kidding, I'm No, I mean, to your point, like, There are certain organizations that champions don't work. I love champions. I wrote a whole guide on it and a whole maturity model on it, but I do also recognize there are some places where it's not going to work and, but that's okay, right? Like to your, one of the earliest points you made here, you come into a new opportunity. You have to assess, you have to, you have to catch the wave of what's happening inside this company. And if it's a very much engineering first, engineering driven. where the bulk of people are engineering mindset people. Champions is likely not going to be as good of a fit as it would be in an organization that's got more supporting roles kind of around and the supporting roles are driving more of what's happening. So yeah, it doesn't hurt my feelings at all, but I don't I don't if I actually have any feelings though. So that's the other challenge, but Robert, how about the next one?

Robert Hurlbut 41:31

Yeah. So the next one is, uh, what would it say if you could display a single message on a billboard at the RSA or Black Hat conference?

Justin Collins 41:43

Yeah, I think, I think I would probably just continue the theme of security should not be in the way, right? So it's like, you know, get out of the way and be that, you know, we were actually deliberate when we named security partners. It's like, we want to be your partner, we don't want to be a gate. I know there's a, uh, very famous, right? Like, uh, guardrails, not gates. Very firm believer in that. Um, as well as, look, I don't want to be the team that people are trying to work around because we say no. And I know that's, that's not a controversial take, right? I think a lot of people are kind of aligned to that. Uh, but in practice, it actually can be hard sometimes to be like, I'm not going to get in your way. I don't think this is a good idea. Here are the risks that I'm, I'm identifying, but it's your decision. It's your decision in the end. So being that partner, um, not. Not the gate there. So, I don't know, get out of the way. There, that's fine.

Robert Hurlbut 42:43

Very cool. All right. And the final one is, uh, what's your top book recommendation and why do you find it valuable?

Justin Collins 42:51

Um, a book that I've found, um, really useful is, uh, and I normally have it somewhere nearby, uh, but it's, it's the DevOps Handbook, uh, by Gene Kim. Um, I think the, the latter half of the handbook is more of Implementation, uh, like how to do things, but near the beginning of the book, and I've talked about this before, but near the beginning of the book, there's a whole section about teams who have to give approvals, but who are not close enough to the work to actually have a, uh, something useful to say as part of that approval. And when I read that section, I'm like, Oh, this is security, right? I'm like, Oh, we have to, you know, this has to go through security review. And then it's like, okay, so there's a security team way over there and they have to do this review, but they don't have any context. Uh, they don't know what we're doing, um, and then we have to wait on them, and of course it's a bottleneck, right? So yeah, I would recommend people, uh, check out, you know, DevOps Handbook, um, I think security is somewhere on the, on the front of the book, uh, like a subtitle, but, uh, while it's not a security book, it's really influenced my mindset of how do we do our job, which is reducing risk to the business and to our customers. Without saying, well, we're going to achieve that by, we have to approve everything. We have to review everything. That was a failure with security partners. We can't review everything, there's literally way too much volume. Can't review everything. We can't be a blocker on, really, we can't be a blocker, right? Um, so yeah, I would recommend that one. Um, that'd probably be my top recommendation. Um, otherwise I think, uh, the internet has usually more relevant information than, than, uh, industry books.

Chris Romeo 44:51

Yeah, that's true. All right. Well, Justin, how about a key takeaway or a call to action? Is there anything you want to leave our, our audience with here?

Justin Collins 45:01

Um, I think we've been discussing it, right? It's, um, tailoring your approach to the business that you're in, to the situation that you're in, to the resources that you have, and then figuring out how can you achieve the goals without saying, we're going to be the blockers, we're going to be the gates, we're going to be the approvers. Once you start thinking, okay, um, I have this goal. And how do I achieve it without forcing people to talk to me, right? Without forcing people to get the, the stamp from security. I think that opens up things and then aligning to the business, right? If, if, if you're taking approaches that are going to get in the way of the business, you're probably just going to be out of a job, right? Like that's, that's just how it works in a capitalist society, right? You, you need to be doing things to enable the business and the, um. You know, the traditional wisdom has been security is a cost center, right? We only spend money on security. It doesn't help us make money. And it's hard. It's really hard to think of and look for those opportunities of, well, is there a way we can help the business, enable the business? Should we be promoting our security more? Should we be saying, Hey, we're proud of our security program. We're going to put that on our website so that our customers trust us. Um, those are things that the business itself, right. And that's ultimately we need the business so we can have jobs so that we can, you know, continue to make good salaries and, and, uh, you know, enjoy our lives. Uh, so yeah, that's, that's, uh, that's kind of what I would like to leave people with.

Chris Romeo 46:45

Excellent. Well, Justin, we really have truly enjoyed the conversation with you here. I always love when I get to talk to practitioners who are in the field doing the job right now in the real world. I love to just to get that reflection and that understanding about what are the challenges that you're facing. You know, in, in the real world right now, running ProdSec teams, running security teams, GRC, privacy, all of these different things. So definitely enjoyed the conversation with you and just thank you for being a part of the Application Security Podcast.

Justin Collins 47:16

Thank you so much for inviting me.