Jay Bobo & Darylynn Ross -- App Sec Is Dead. Product Security Is the Future.

Jay Bobo and Darylynn Ross from CoverMyMeds join Chris to explain their assertion that 'AppSec is Dead.' They discuss the differences between product and application security, emphasizing the importance of proper security practices and effective communication with senior leaders, engineers, and other stakeholders. Jay proposes that product security requires a holistic approach and cautions against the current state of penetration testing in web applications. Darylynn encourages AppSec engineers to broaden their scope beyond individual applications to product security. With enlightening insights and practical advice, this episode thoughtfully challenges AppSec professionals with new ideas about application and product security.

Links:
Jay recommends:
How to Measure Anything in Cybersecurity Risk, 2nd Edition
by Douglas W. Hubbard, Richard Seiersen
https://www.wiley.com/en-us/How+to+Measure+Anything+in+Cybersecurity+Risk%2C+2nd+Edition-p-9781119892311

Darylynn recommends:
Kristin Hannah: https://kristinhannah.com/

FOLLOW OUR SOCIAL MEDIA:

➜Twitter: @AppSecPodcast
➜LinkedIn: The Application Security Podcast
➜YouTube: https://www.youtube.com/@ApplicationSecurityPodcast

Thanks for Listening!

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Transcript

Chris Romeo: 0:00

We have a special double guest episode featuring two prominent figures in AppSec. First, we have Jay Bobo, a cybersecurity leader, entrepreneur, and the driving force behind product security engineering at CoverMyMeds. He's also the founder of BreachSiren, a cutting edge data breach intelligence startup. Joining Jay is Darylynn Ross, a senior application security engineer at CoverMyMeds. With a career spanning roles at EDS, Verizon Wireless, and Chipotle's security operations team, Darylynn brings a wealth of knowledge. She's a staunch advocate for fostering a security first culture through relationship building and mentoring. We'll dive deep into the provocative topic, AppSec is dead. We'll explore the nuances between product and application security. And we'll discuss the importance of building solid relationships within the engineering organization. and delve into the intricacies of communicating vulnerabilities to senior leaders and engineers. So buckle up for an engaging and insightful conversation with Jay and Darylynn as we navigate the ever evolving application security landscape. Hey folks, welcome to another episode of the application security podcast. This is Chris Romeo. I am a co host of the podcast, but I am flying solo on this mission today. Uh, I'm the CEO of Devici. I'm also a general partner at Kerr Ventures, and I've been in the world of security for what feels like hundreds of years. I'm super excited to be joined today by Darylynn and Jay. And so we're going to do security origin stories, folks. Like we have to start there. Our audience always wants to know from where you're coming. Where, how'd you get into this industry? How did you find your way here? And so Darylynn once you go ahead and, uh, and kick us off from that perspective and share your superhero origin story.

Darylynn Ross: 2:29

Sure. Well, I never intended to be in security. So it's kind of interesting. I was a developer for a good 25 years. Um, finally wanted to maybe change and do something else. So I moved over to more of us infrastructure support team. Two weeks after I started, they all got laid off. And the director called me up and said, I want you to do security for my org cause I, we're getting killed on this. I'm like, I don't know anything about that. You know, he's like, well, you better learn it cause this is your new job. And I've been in security ever since. Uh, he was right. It was a good match and, I really enjoy it.

Chris Romeo: 3:11

okay, very cool. All right, Jay, how about you? How, what's your security origin story?

Jay Bobo: 3:16

Yeah, it's kind of crazy. I actually wanted to be in security or something security related. I was 10 years old when Sneakers, the movie, came out. Robert Redford, Sidney Poitier, you know, and we, I grew up in the inner city. We were one of the few families that actually had, like, a computer. My dad didn't understand. He was like, hey, Here it is, go do something with it. So that's kind of what started me there. But I was always into, you know, the hacking movies and Dick Tracy. And from there I got into, you know, uh, to my parents chagrin, you know, wardialers and RATs and all types of fun stuff. Um, the days of, of bulletin board systems and stuff like that. Um, but I ended up getting into sports and entertainment and the ad agency world, and it ended up coming back to it much later in my career. And starting off, um, in application security after being a software developer for about 15 years. Um, so that's kind of how I got pulled back into it. It's always something that I wanted to do, but You know, I like mu music and having a good time in my 20s sort of pulled me in a different direction Then I came back home. So that's my word.

Chris Romeo: 4:23

Yeah, and it's, it's, it's cool that you're both coming to application security from the world of development, having spent time in the trenches as developers, because I find that those are the people that are the most in tune with the impact of what we do. Like when we add a new tool, those that have been developers before are processing it through the eyes of how, how is our group gonna, how, how are other developers gonna.Be impacted by this? How are they, what's going to be their response and their feelings about it? So,.... All right, so we should, we should dump, jump into this particular topic, which is, it's, it's very non controversial. Um, but you know, we're, we're, we're fine in the AppSec podcast of, uh, of pushing the envelope a little bit. So the primary topic that you, uh, that you shared with us is quite aggressive. And that is the concept that AppSec is dead. And so Darylynn I want to, I want to let you kick this off for us. Give us some context and some perspective on why you think AppSec is dead.

Darylynn Ross: 5:26

Well, in our world today, really, we have to look at the whole product. And looking at the whole product is a lot more than AppSec. So AppSec is a little part of product security, right? And it's not the most important part of product security. Now, I'm an AppSec engineer, so I'm kind of kicking myself here. But, um, when we look at what goes on in our world today, where we're being attacked, how we're being attacked, it is our product that is being attacked. Application security is part of that product security, but there has to be a whole culture around our security programs, and you can't do that with just AppSec. Right? To get that whole culture, you need to promote product security. Buy-in from your executives, buy-in from your product people. Buy-in from your engineering directors and managers. Buy-in from your developers. And then buy-in from all those other people that surround you. Right? So that's why we use product security. Um, I don't know, Jay, if you want to fill in some other things. I could talk on this for a long time, but I want to give you a chance too.

Jay Bobo: 6:49

Yeah, I think it's really important and those are really good good points I think it's important for us to shout out Justin Collins, the author of Brakeman and Head of Security I think he's still over at Gusto. He had a talk probably a couple years ago at LocoMocoSec And his talk was AppSec is dead. He really sort of focused on it not scaling well, right? This idea of having these small sort of application security teams, and they're outnumbered and they're outgunned by developers and other sorts of engineers, right? And they're having these sort of conversations just about sort of application security. It's like, how am I supposed to go through and I'm involved in pin testing? Am I supposed to sit down and do code review for everybody? You know, am I supposed to go through and do all these various things, you know, in addition to implementation of tools? Right, it's, it's, so that's, that's one piece of why application security doesn't work. And then the other piece, I think, is getting back to what Darylynn says. We need to utilize a systems thinking approach, right, as opposed to local optimizations. And application Or applications and APIs and so forth. They're just components of a product. And when we kind of just focus on those things, we don't have an opportunity to kind of pull all that stuff together and give leadership an opportunity to say, Hey, here are the technical risks to your revenue generating products, right? And maybe even products that don't generate any revenue. Um, and so just kind of really thinking more holistically about, Hey, what is it that impacts the company? What is it that product owners want to know about? Um, a good example is you hear developers say all the time, Hey, I love to work on this thing, but the business wants me to do something else, right? My product owner wants me to do something else. You can sit down and have a conversation with the product owner about applications and watch their eyes sort of gloss over. You can have a conversation with them about products. They understand products. The business understands products. So when we talk about sort of moving security left and having, you know, really focusing on people, we need to speak in the language of the business and speak in the language of people and everybody collectively understands what a product is and the importance of products. And that also allows us to really think about sort of DevOps sec, you know, DevSecOps as well, right? What are the other pieces of this besides You know, just, you know, SAST and Pintest DAST and SCA and so forth. So, um, it's, it's really sort of thinking more about the system as a whole, right? And sort of how the company operates as opposed to just sort of local optimization and just focus on a product. I mean, I went on just on an application, which is I think one of the big faults often of the traditional application security. Um, you know, Paradox.

Chris Romeo: 9:20

So the longer I've been around AppSec and, and worked inside of companies that had developers and, and sat as a member of the security team, the more I see a future where, whether we call it AppSec or ProdSec. It feels like this function needs to be integrated inside of development. Like it feels like a, a, a future state that we should be aiming for is we don't have an AppSec team, we don't have a ProdSec team. We have a development team, and the development team has people that are part of that team that are dedicated security people, perhaps, but they don't sit in some other organization through some other channel. They share a management structure. They share a, uh, organizational hierarchy that leads to the CTO and not to the CISO. It just seems like that would get us better results. What, what's your reaction? Both of you, well like either, anybody can jump in on this one, like what's your reaction though to that kind of idea? Is that, does that make sense from what you've seen?

Jay Bobo: 10:29

Yeah, I think, I think so. I mean, my personal opinion is that there, there are those opportunities that exist, but it, it, that comes down to sort of, sort of resources as well. I think a good way to think about it is what is the role of your security champions? In a lot of organizations, security champions are just sort of people who have an interest in security, but they don't really necessarily have any real work to do. And so I think that when we think about them as more sort of like, Um, you know, people who have a actual responsibility to get into some of those things, right? They're almost, they're part of the security team, but they're embedded. They still have development work. And I think there's that opportunity to sort of grow in that particular direction as well. Is it, is it a possibility that your company says, Hey, we want to make sure that all of our teams, product teams are fully, um, you know, cross disciplinary and they have all the various roles that they need to sort of operate in these sort of like independent silos. Possibly, but that thing that really comes down to is your company wanting to make that sort of investment. So I think it's going to need to be a little bit of a back and forth, right? You're going to need to have a independent, uh, you know group that is there supporting that, uh, those product teams and helping them and consulting. You know, uh, they're in, in that capacity while also, I think, having the same thing, which you're saying is like, we need to educate people on, on, on those teams to, uh, we need to be able to do both, right? And having a hybrid approach, I don't think it's like a, you know, it's not, you know, it's, it's, it's not a binary sort of situation. I think when we need to look at, look at more sort of what's the right balance and how do we have sort of a hybrid system that allows us to move at the speed that the organization wants to move at. But that's my opinion. Darylynn, do you have any thoughts on that?

Darylynn Ross: 12:11

Um, well, I would go right back to our security champions. We have, I would put my group of security champions against anybody. They're, they're great. They, um, they don't have the education that I do in. AppSec, but they are very passionate about security. So where we get the win is because I have this really great group of security champions. The problem I see with embedding in development teams, a security person is there aren't enough of us. I mean, I know in the whole industry, there aren't enough security people, truly AppSec engineers. There really aren't enough of us. Um, so I think that's one of the shortfalls in trying to embed us into development teams like that. Um, and then, once we're there, there are a lot of things I do that I think are really important for our product security that I probably would be limited on if I were sitting in a dev team. So, I think you just have to work out the model. I think that you're right, Chris. We're gonna have to be closer and closer and closer to engineering teams. But we also, um, have to have the influence across the entire organization. So I think you're always going to have some kind of security group to have that level of influence. Um, we do that with a security community of practice, we do it with our security working group, we do it with just, um, one on one meetings and things that, you know, we do very intentionally to get that culture. So, I feel like if you lose The security group, you lose a little bit of that and that's going to hurt you.

Chris Romeo: 14:02

Yeah, there certainly would still be a problem with governance. Because the development team with security contained within is not going to govern themselves with the same rigor that an outside group. So that's one of the arguments for keeping security separately, is that we need a governance structure to drive it. And I'm a giant champions proponent. I mean, I wrote a whole framework on building security champions programs. But I really see this kind of evolution. As I think it will play out as a top of the top tier kind of answer and security champions is that that'll be the state where people will start to get to is they will start to add those security resources and I don't think it's even embedding I think if we think of it as embedding, we're still thinking about a central security team that's deploying. people into various development teams, keeping the, the, the holding the rope kind of back to a group in, in, in a central security team. And I think that's a, I think this is five to 10 years. This is how, how organizations are going to mature. Um, and I, I can't wait to see how, how it, how it plays out, uh, in, in kind of the way that, uh, the way that organizations mature and, and grow in this. And there'll always be some organizations that are way behind. Even if the top, even if Netflix, for example, who always, and, and Etsy, they always seem to lead, everybody's always like, oh, they're got the bleeding edge of whatever's happening in anything. Like even if they went to no AppSec team and having security people deployed across, it would still be 20 years before everybody caught up to that model. I'll be long retired from this industry by the time everybody catches up and, and figures out how to, how to address this, this kind of style or approach.

Jay Bobo: 15:58

Yeah, I think it's also really interesting, Chris, too, is like, what is the right ratio as well, right? So if you move to that model, then how many security people need to be on a team? Right. I think this is one of the points that Justin was attempting to make. Okay. Well, what is the right, it's kind of like, you send your kid to school, it's like, how many students, right, are in the classroom, how many teachers, and you're going to need that. And it also requires, you know, someone to, to Darylynn's point, to have a lot more experience and also have a much more varied skill set. So, you know, and, and also possibly to have less coaching. What we've seen is that that, you know, And when that, when you have the wrong sort of ratio, even with security champions, and they have a lot of work to do, for us, because they have real work to do, and they actually have real power within our organization to dictate sort of when certain security initiatives happen. It leads to burnout, and I've seen it happen again in the annual security champions. I've concluded burnout. I'm doing development work, and I'm also participating in some vulnerability management talks, and I'm participating in code review, you know, in addition with the, you know, helping out with the application security team, you know, at times, all these various sort of things are like, Hey, I can't, I can't do this, you know, anymore. Um, and I, I've seen that happen a couple of times. So I think it really comes down to, again, focusing on people and having, you And communicating about sort of how things are happening. What's the right, right thing for everyone? I'm not sure necessarily whether our particular structure works for every organization, you know, but we have to optimize for people and that's what we really try to do, try to optimize for people and for,

Chris Romeo: 17:31

a good statement. I'm going to borrow that one. Optimize for people. I'll give you attribution the first two times I use it, by the third time it's mine, but like optimize for the people. That's, that's a good, because like one of the, one of the, the dark sides, I guess, of champions. And nobody ever does a talk about this is what you were talking about. Burnout, um, environments where there's tension between management, manager and champion and security team. It's like a, a tension triangle almost because the champion is like, I kind of like security. I want to spend more time on it. The manager's saying, no, you need to go build features because that's what I'm. My, my charter is, and you as the security team, you're trying to support the champion and, and help the manager. That's, that's a, that's a dark side of champions that we never talk about is that that ends up in burnout, ends up in people having stressful job worlds because there's, they're, they're, they're fighting with everybody pretty much. Cause they, they, they, they want to work with the security team, but the manager wants, has. As you could expect. I mean, we already talked about business goals and stuff. The manager has things that they need to achieve or the manager gets fired. Like it's a, it's yeah, it's just a slippery slope, I guess.

Jay Bobo: 18:43

yeah, I think it's different for us because one. We are in a regulated environment, right, you know, so we're healthcare security, which is completely different situation than let's say if you're some company has some sort of like Twitter for dogs, you know, sort of app or something like that. Right? So we take things very seriously around sort of PHI and, you know, protecting patients and so on and so forth. So for us, it's interesting that our, you know, engineering leaders really at the director level are the ones who go through a night with their engineering managers, who their security champions are. So there's less conflict on our side because they're a part of the conversation early on. They recognize that they're responsible for keeping an eye on that risk and communicating that risk to their product leaders. We also have some of those conversations as well. So we, we experienced less conflict there. It's more of a situation which maybe a team is unbalanced. It doesn't have enough security champions, or maybe for some reason they have, let's say a good amount of risk on a particular team. More applications, as an example, more developers, and that person is saying, okay, I'm, I'm doing a lot here, especially if, in that case, maybe the security champion themselves isn't a developer, right? So our security champions are also database administrators, they're SREs, because our focus is on a product as a whole. So all of these various sort of groups and roles. You know, who are building out these assets and components that make up that product have to be a part of the conversation. So, you know, that's kind of maybe how we're different and I've noticed that a lot of other security programs or security campaign programs are That's kind of maybe where they're different than than us is kind of really a focus on developers And that's not that's not you know, that's that's not everybody

Chris Romeo: 20:24

Yeah, that's, yeah, that's, that's, uh, a good perspective and we need to have a follow on startup conversation, by the way. Twitter for dogs? Are you kidding me? Like, what happens? The dog barks and that makes a tweet? Or a woof? Or, I mean, I guess that's already been, woof's already been used before, but, like, that's a, yeah, that's a, that's a, that's probably a 25 idea. Like, we could probably make 25 if we started Twitter for dogs. Alright, so Darylynn, you gotta help me understand. The difference between ProdSec or product security and application security in the environment that we've just been describing here, because we filled in a lot of details and context because I kind of jumped around a little bit, but it's good because now we, we have a better perspective on your organization and kind of the way you're. How your program is fitting together, and I'm already getting an idea about what the difference is, but help, let's address this one head on, so ProdSec versus AppSec, what's the difference? So,

Darylynn Ross: 21:22

Um, I think where it really comes like rubber meets the road in our organization is that we have to have buy-in like Jay was talking about. We're highly regulated, right? We have a ton of PHI. We have to have the buy-in of those product people. They really don't know anything about my SAST scans. Right? That's AppSec. They really don't know anything about, um, some of the other work we do, pen testing on our apps, those kind of things, for our truly pure AppSec work. But what they do know is we make them come to our HIPAA technical risk assessments and go through a threat model with us. And go through a gap assessment with us and say, listen, we have really valuable, important data we're protecting here. Are there any gaps? Do we have this control in place? Do we have this control in place? And they actually have to, like, um, understand enough of that to be part of the conversation. So, I think that's one of the big things about, um, where we are. And, uh, the way that we handle product security versus AppSec. AppSec is a little piece of the overall picture. Even for me, as the senior AppSec engineer, I'm working in product sec a lot more than I'm doing some of those very specific AppSec roles. So, um, And that's just, that's one example, but, you know, we're working with engineering directors, trying to get them to understand these are the vulnerabilities you have. This is your mean time to remediation. I'm not trying to shame you with this number. I'm trying to tell you, you might have a problem here because there's a gap. And for an engineering director, that's not That's his product, right? That's what he's trying to build. It's not AppSec. That is one of the things that we're doing as an overall culture and it's bigger than AppSec.

Chris Romeo: 23:45

your HIPAA kind of risk assessment process is at the product level, and so, do you consider risk analysis, risk mitigation, threat modeling to be a product security capability versus an AppSec capability? See, I always, I always lump threat modeling just into the AppSec world because it just makes sense. But a lot of times I would say in my mind, I think of AppSec and product security slightly as a synonym of synonyms of each other. It's not exactly a perfect match, but I kind of put them, I just lumped them together and because I spent 11 years at Cisco and at Cisco, we did product security. Right? Because we were building metal boxes. We're writing software and firmware for metal boxes, but the metal box was kind of the beginning of the conversation, which you can't do AppSec against that metal box. And so we were doing product security, but we were integrating a lot of AppSec things, and we were just calling it product security. Um, it wasn't until I left Cisco that I started referring to myself as an AppSec person.

Darylynn Ross: 24:51

And it's, it's interesting because my thought process when you were asking that question was, Well, threat modeling is an AppSec tool. That doesn't mean you can't use it in product security. It's just a tool that we're using. Can, uh, can people who are not developers contribute to a threat model? Of course, right? They know things, they're thinking about things differently. So that's awesome because Threat modeling is just brainstorming. So now you have another perspective in the room. You have another opinion. You have another person looking at it a different way. So you're using an AppSec tool more broadly. Um, and you're right. We, we toss product and AppSec around a lot. They are interchangeable in a lot of circumstances, but I think where you get the difference. is the culture piece. The overall, um, I've read before AppSec is baking a cake, ProdSec is like making sure the whole bakery works. You know, it's the, it's a little piece of the overall picture. And I can't tell you where I read that or attribute it to somebody, but that is somebody else's analogy. But, you know, it's

Chris Romeo: 26:06

too. I can't remember. I can't remember the attribution either.

Darylynn Ross: 26:09

Yeah, I don't know who said it, but they're right. So we take some of these AppSec tools that we've developed and we do, like, spread them across into other disciplines and get those people's input. Same way, we take some AppSec concepts and we teach them to people who aren't developers or security engineers, right? We have a security community practice where we talk about security things and part of our goal there is to teach the whole organization about security concepts. So, it's a culture thing. Again, that's product security. That's not app security. That's product security.

Chris Romeo: 26:49

So do you insulate your development teams from SAST tools? Results from SAST tools, or do you somehow feed them results from SAST tools without them running the SAST tools themselves? Or how do you, is like, cause like, that's what I think of is like, AppSec is the SAST tools themselves, it's the results, but are you somehow synthesizing those and, and providing them at a product security layer different from what I would think of as SAST tools for someone who's doing AppSec.

Darylynn Ross: 27:23

We are. Is it okay to mention, like, products we use to explain this?

Chris Romeo: 27:29

Yeah.

Darylynn Ross: 27:29

Okay. So, we use GitHub Advanced Security. We use CodeQL scanning. It's part of our GitHub actions. Um, it just, it happens. On a PR, it scans. Right? So, um, but what we do with that is our developers obviously have access to that. It's right there for them. They have a security tab they can see. We also use Dependabot and GitHub Advanced Security for SCA functionality. But then we have a tool that we, we have those results right there for the developers. We have a tool that takes those results and pretty much integrates them into other scan results like endpoint scanning, container scanning, and provides a view for everyone of these are your vulnerabilities. You can dig down into the vulnerabilities from there, but it gives you a great overview of, hey, engineering director, Here are your five apps, and look, these are your criticals that are past due, you know, these are vulnerabilities that you guys probably should take a look at, you know, those kind of things, so it's there for them to look at. And we've had a great response from that, people really love that. First of all, they love seeing it all together, but second of all, they love just being able to see it because so often it's hidden from them. You know, with all of, we have all these tools on the security side, showing that all to them, making them look at four different tools, is not realistic. They don't have enough time to do that. So showing it on one tool is really where we've landed. And Jay is like a major product owner and architect of that. So I feel like he should probably be talking about that more than me, but it came from his brain. It's good stuff.

Jay Bobo: 29:25

I can't, I can't take credit for it. I think it comes down to, I was at, um, ISC Squared, uh, Security Congress some years ago, and Kurt Lieber, who was, uh, I think CISO for Aetna at the time, had got up and named and shamed a bunch of vendors, and this was like after WannaCry or NotPetya, and he's just like, You know, I'm sticking tired of this vendor sort of like lock in, you know, you guys are charging me more for APIs. And he was really kind of pushing the idea of going through and connecting your security tools to create tools that didn't exist in the marketplace for the needs of the business in order to do sort of more automated risk management. Because if we're sitting around and have a tabletop, you know, like we do for a tabletop, and you call me and I call somebody else, he's really talking about ransomware. He's like, I've already lost all my assets, right, after someone's made a third phone call. And so for us, what I noticed looking at sort of traditional security is that we had all this sort of, you know, security reports, pen test reports that aren't really being shared often with developers. Developers don't know when sometimes when pen testing happens, you know, they don't know, they don't have access to the tools themselves. I wouldn't expect them to have access to these various security tools, but it's kind of really hard to talk about optimizing for people and democratizing security if we're not giving people information. And what developers are used to is having a tight feedback loop, right? I go look at my logs. I go look at my tests as I'm developing something, if I'm doing test driven development or REPL driven development, whatever it is, you're used to that constant feedback. But in security, here we have, until recently, in the last couple years, right, I had less of that. I gotta go through, I gotta log into Veracode, and I gotta understand that UI, and blah, blah, blah, blah, they gotta come back somewhere else. How do we get you the information that you need, without you having to look at a thousand different screens? You just want to know, what is the problem, right, where is it within my source code, right, and maybe even some suggestions on how to fix it, and so on and so forth. So, for us, okay, great, that helps with developers. But you have the exact same thing for your SREs, exact same thing for your DBAs, exact same thing for all these various roles in addition to your product owners who want to know about Hey, am I complying with policy? Am I, you know, how many criticals do I have? How much maintenance time should I be sort of baking into my sprints? You know, should I just have a maintenance only sprint? Well, you don't know that if you don't really have an understanding of sort of what your vulnerabilities are, right? What your risks are. And for us, I think it was very difficult to find something in the marketplace that made it easy for us to map our stuff back to our products. Right. Um, the things that we report to the street as a publicly traded company. Um, even mapping back, let's say revenue, right. It was just really important. You know, if I go sit down with leadership and say, Hey, I think we have a problem here. It helps for me to be able to talk in the language of the business. Hey, here's a product that makes us money. And there's also some really critical vulnerabilities in it with a high likelihood of being exploited? Do we see that this is a potential problem depending on what, sort of, you know, how much sensitive data's there and, again, how much revenue is at play. So, you have to have all meaningful conversation, otherwise we're setting up developers for failure, you know, we're setting up our engineers for failure, we're also setting up our security team for failure. You know, I was just at our head of speak engagement, yesterday and so, one of the things that we don't talk about enough is how much does doing security wrong cost our organizations. We vary, we talk in CBSS scores, we talk in CBEs and CWE and blah, blah, blah, blah, blah. What we don't talk about is, okay, great, so and so got popped. How much did it cost them?

Chris Romeo: 33:00

Hmm.

Jay Bobo: 33:00

You know, lost funds, class action settlements. So that's one of the places that we're trying to go next with our program is mapping this back to sort of the data that cyber insurers will have to say, okay, great, yes, we knew it was a critical, but let's say you have, I don't know, 10 criticals, 20 highs, I don't know, it depends on the size of your organization. Which one do you work on first? You know, especially if it looks like they're kind of similar, you know, we need some other way to do vulnerability management or vulnerability remediation better. And it starts there and having that product security, you know, systems thinking point of view. And far too often, I think we, because application security usually gets started with like a single developer is really passionate. We kind of like nerd out on the technical details of particular issues as opposed to thinking about like, what is the actual risk to our customers and to the company as a whole? In order to do so, you kind of have to pull yourself out of just thinking about a single application. If you only got one application in your company, I get it. But if you got a bunch of them, you gotta, you know, and the, or you have multiple applications that make up a product probably should think about, you know, figuring out a way to communicate that stuff in a way that's going to have a product owner say, Oh, okay, I see what you're saying there. Okay, what help do you need for me? You know, and, and, you know, say, uh, I would say also be ready to have the same sort of conversation for other people within your organization. So, you know, again, not some idea that we came up with. I would. Justin Collins has talked about this stuff. Kurt Lieber's talked about this stuff. There's a lot of people who've talked about these things and we were just lucky to have a cross disciplinary team to kind of work on it and start building out, you know, tying our tools together to really sort of focus on these particular issues.

Chris Romeo: 34:46

Yeah. And it seems like a good, uh, it's, it's, it's the right strategy. It's the right way forward is, you know, you've mentioned business results a couple of different times and that needs to be part of the conversation. Right? Like we don't just do security because we love security, which we all do. That's why we do this. But that doesn't, that doesn't make money for the business. Like it's a business function that we need. We need to think like business people, like the security people that rise to the highest levels are the ones that think like business people. They're not the smartest technical people. They're the ones that understand how businesses work and how can we, how can we align with the business? So we're not a friction point, but we're a, we're a, We generate value for the business. That's the key.

Jay Bobo: 35:29

We have to 100 percent because what happens can tell you how many friends with an application security say, I'm the only application security person. They're extremely great technically, right? They know the ins and outs of it. They can go back and recite OWASP ASVS, you know, line by line as an example, or a testing guide and so on and so forth. They're great. You know, you're stuff. But what happens is that person is like, oh, I'm going through all this stuff. I got 50 developers, the company's grown. I got a hundred developers. I'm the only application security person. I can't get any help. It's like, well, what's happening here? The problem is, is that it's a failure to communicate in the language of the business and everybody else. We're so focused on the technical details. And yeah, you're doing great work, but in order for you not to, you know, end up with some security nihilism, and kind of saying all the same sort of things that we complain about in the security world, you're going to need to grow a team. You're going to need some additional resources. You're going to need that tool. And if you can't have that sort of conversation, you're going to find yourself in a situation like, you know, I started out with a team of one. Me, you know, um, and we grew from there. And so I had to kind of, sort of evolve my thinking in order to, you know, bring on awesome people like, you know, Darylynn and the rest of our team.

Chris Romeo: 36:43

And we kind of, we've kind of already talked around the edges of this, the next question, the next direction I want to go. I want to talk about communicating vulnerabilities throughout the organization to senior leaders and engineers. Because I think that's an important piece of what we need to do as security professionals. Coming back around to understanding the business, right? Proper communication. Soft skills, those types of things that allow us to communicate what the real challenges are in a way that the business person or the product owner understands is how we truly unlock value. So Darylynn, excuse me, what's been your experience here in the communication side?

Darylynn Ross: 37:23

Um, I've had not so much where we're at right now at CoverMyMeds, but I've had some experience at other companies where I have had to, um, go into a VP's, um, Weekly staff meeting and explain to him this is what it is. You know? And I think the most important part there is we can do that. We're his security resource, like I'm his SME. He needs me to be able to clearly explain to him, not in technical terms, but in business terms, what's going on. Excuse me. So. Um, I think that's one of the skills we develop in product security. It's one of the things like, uh, Jay and I were just recently talking about when you're a developer, you're working with peers and your direct manager. When you're working in security, you're working with management. and even upper management, even possibly C level, depending on what your role is. And you need to learn to talk to them so that they understand you don't want to waste their time. They've got a lot to do. Um, they have things that they need to worry about that are not you, but, uh, you need people to get them the information they need. Um, and is it a list of vulnerabilities? Probably not. That is probably not useful for them. Is it Um, we do something called a state of security report. Is it a state of security report where your engineering teams are reporting that your security is in a good state, and this is what that means. These three things are what that means. Or, your, your Developers are telling me this is why, you know, those are the kind of things they want to hear. They don't want to hear, oh, we have this, you know, critical thing over here and this, that, that is very technical and not useful for them. They're just going to say go talk to the engineering manager anyway. So, um, I feel like I kind of wandered away from your question a little bit there. But I think that some of it comes back to security is about people and security is about people communicating and that's a piece that we have to be able to do as security professionals. We have to be able to communicate to people who don't do things like us. So I think that comes in vulnerability management. It comes in. When we're talking about resources that comes in, when we have, um, questions, you know, your director has a question about something, you know, he doesn't want the technical details. He wants to know what it is so that he understands it and can tell his boss.

Chris Romeo: 40:18

Yeah, I mean, what I've seen is, especially in the C level, C level executives are risk people at the end of the day. They just care about, so what's the risk to me, to my, my functional area of the business? I could, they could care less about. The technical details of the latest vulnerability, like don't even go there because they're, if they don't rudely tell you to stop talking, they'll just glaze over and pick up their phone and start doing something else. Cause they're like, I just don't, this is not information I need in my head. And so it just comes down to, to knowing that audience and, and being able to put it in a, make it a risk based conversation. Like if, so here's the problem. If we don't do this, then here is the risk to the business. And, and, and oftentimes that's enough for them, and they may, then they'll start locking into, okay, let's explore some more about what that, that risk is that you're talking about here. How much is that going to, what's the potential monetary damages, reputational damages? What are the, what are the things that could go wrong here? And that's when they can make a decision, because now they're, now you're speaking their language, and that's the key. It's just speaking their language. They don't, just don't talk technical, talk about the business need, the business function, the business result that we're all trying to get to. All right, so let's do the lightning round here. It is, it is, we've reached that time. Normally, this is Robert's segment, but I'm filling in today, so I apologize in advance. I'm not going to be as good at the lightning round as he is, but, um, but you're, we're going to, we're going to do this. So, um, let's see, Darylynn we're going to come to you first, and then we'll ping pong back and forth. So, we'd love to hear people's most controversial opinion on application security. And so, what is your most controversial opinion on AppSec, and why do You hold this view?

Darylynn Ross: 42:20

You know, it's interesting because thinking about that earlier, I'm like, I'm not a controversial person, you know, but I do have a lot of strongly held opinions. So, um, I'm going to tell you the most important person in my work world is not Jay, who's my boss. or even RVP. It's my developers. For my role and what I do, it is my security champions. So if my director's pinging me about something and one of my security champions is pinging about something, I'm going to the security champion first. The director can wait. I know sometimes people don't like to operate that way, but literally, though, I can't live without those people. They are my priority.

Chris Romeo: 43:06

Hmm. Okay. Jay, how about you? Most controversial opinion on AppSec, and why do you hold this view?

Jay Bobo: 43:13

I, I got a lot about security, so I'm gonna just, I mean, hey, I can, I can, I can talk about this for a while, I'm gonna say, so here's one that we need to talk about more. A lot of penetration web app. Web app penetration testing is crap. Let's just be absolutely honest. A lot of web app penetration testing is crap. So let's say a second time. A lot of web app penetration testing is crap for a third time. So, I mean, here's the thing, right? So some of the stuff you're gonna do from compliance perspective, right? I gotta go through and demonstrate per client contract or whatever it is, you know, that I'm doing this. But if we want to get something that is valuable, For, uh, us, then we need to move away from sort of this black box, gray box testing, and we need to get into something where just sort of opening stuff up and starting to get into some of the business logic issues. I think that's really important because those are the things that are, in most cases, are going to be exploited. If you're a web app penetration testing team, or this internal, or it's an agency, there's a way of just clicking a button and seeing what comes back, which is what happens probably in the majority of cases out there. Um, we're not getting a whole lot of bang for your buck besides being able to check off the box of compliance. And so I would say is, um, I get it in those instances, but I think there's a, an opportunity for us to talk a little bit more about how to help, you know, uh, you know, those vendors and, um, you know, that, that domain of security sort of, you know, uh, be better for all of us. Um, cause I, I think it can be very beneficial, but um, but 2023, a lot of it ain't great.

Chris Romeo: 44:53

Yeah, we could have a whole episode discussion just on that one because I agree with you wholeheartedly. So it would be us kind of trashing the state of the industry, but I don't, but I don't, I don't mind doing that either. But, um, so we should have a follow up conversation on that because that's a, that's a fun topic. And, uh, people tend to, people to this day, they still lead with the pen test. Oh, you got to do a pen test, but you've never run a SAS tool against your code base. So, or, so what are you going to find with a pen test? It's just going to find a bunch of things you could have ran a scanner to find. The first time around, like that's the, that's the challenge, but we'll keep that for another, we'll do another episode where we'll, we'll unpack that one. We'll spend a whole hour just on that one topic and we'll, we'll make a lot of people happy on the internet. So Jay, I'm going to go to you first with the second one, just because that way Darylynn doesn't have to go first on all of these, uh, billboard message. Say we, we, we were going to, we gave you a billboard at RSA or Black Hat Conference. And so you've got a limited amount of space, but what would be the one, a message you would want to get to our entire industry?

Jay Bobo: 46:01

Yeah, I mean, I would say, you know, optimize, uh, first one, optimize for people, you know, share your security results, reports, findings with your engineers. They love feedback. They love feedback. Tighten the feedback loop. So maybe the first one, optimize for people, tighten the feedback loop, sentences, boom, boom, boom, share your results with your engineers so they can actually fix stuff.

Chris Romeo: 46:24

I like it. All right, Darylynn how about you?

Darylynn Ross: 46:28

Um, I'm going to go with just collaboration. I get really tired of, um, people thinking security people are mean and they don't want to work with you. I want to work with you. I want to work. I don't care who you are. I want to work with you. So, collaboration is key in creating a security culture. I, I never want somebody to think that I don't have time for them or that they're not important. So I'm going to take the time and I'm going to listen to their problem and then, uh, collaborate with them to try to find an answer.

Chris Romeo: 47:06

Excellent. All right, Darylynn how about you be our first answer on the last question in the lightning round here? What's your top book recommendation and why do you find it valuable? And just as a caveat, this doesn't have to be a security book. Like most people have answered this outside the realm of security because there's so many good things, but it can be a security book too.

Darylynn Ross: 47:28

So, I'm not a big technical book reader, okay? Um, I read a lot of smaller sources. I'm not a big technical book reader. I'm a fiction book reader, so I'm going to tell you anything Kristin Hanna writes. You might want to buy your wife that for christmas, like a Kristin Hanna book. There you go.

Chris Romeo: 47:52

That's excellent. This will be our first ever Christmas gift recommendation on the application security podcast after 250 episodes So I have to have to I'll have

Darylynn Ross: 48:01

You can't go wrong.

Chris Romeo: 48:02

I'll track that down. That's my wife is a big reader so I'll have to And she doesn't listen to the podcast either. It's just kind of funny. But Jay, how about you? What's uh, what's a Top book recommendation that you would make for the audience

Jay Bobo: 48:17

Yeah, I'm looking at my, my bookshelves and there's so many, I guess the one that I've quoted a lot, um, lately, it is a security book, um, is, uh, how to measure anything in cybersecurity risk by Doug Hubbard and Rich Cyruson, which I think more people need to read. I'm a little, uh, ashamed that I didn't actually read this book until this year. And I was like, Oh. Here is someone who kind of agrees with my opinion and then has done a lot of work on that risk quantification side. So I think it's awesome because I think when you start off in security, it's very easily easy just to talk about security without recognizing that, as you said earlier, security is just a component of risk, right? And as you need to have conversations with people and help them sort of understand security, it's kind of sometimes easier to do that and just having a conversation about risk because then we can take that and tell really great stories about. Stuff like, hey, you know, working on your home or something like that. Hey, I got a hole in my roof or I live in a rough neighborhood. I want to secure it. People get the risk conversation. Sometimes they don't necessarily get, you know, security and application security sort of things until it's, it's, it's within that risk framework. And I think that book helps with it a ton.

Chris Romeo: 49:27

Excellent. All right, Darylynn, how about key takeaway and call to, or a call to action? Is there something you want our audience to do as a result of our conversation, or do you want to just leave them with a key takeaway? The floor is yours.

Darylynn Ross: 49:41

I just want our AppSec engineers to think broader, to think more than AppSec. Still do all your AppSec things, but start thinking about your influence in your organisation and broaden it so that you can have a more secure product. instead of just some secure applications.

Chris Romeo: 50:05

Excellent. So Jay, how about you? Final thought.

Jay Bobo: 50:09

Yeah. On a similar note, that was a really great point, Darylynn. I would say stop throwing stuff over the wall. Don't, don't, don't, don't do that. What I mean by that is kind of getting back to the influence part. If you're a, you know, security manager, security leader, you know, just even an IC, right? Senior Application Security Engineer. At some point in time, we have to kind of listen to the why behind why things may not necessarily be happening the way that you want them to happen. I get sick and tired of all these stupid engineers, all these stupid developers. If they just did this, When we should be maybe walking into HR and saying, Hey, these people just need more developers on their team. They got a ton of applications. They need some help. Right? Um, we need to be a part of that conversation and helping them address the underlying issues. Maybe they just don't have the information. Maybe you guys aren't getting them real time vulnerability information and they don't get it till too late. You know, maybe you need to work on integrating some of these security tools into CICD. Maybe that's the real issue. And as they get it, then I'll work on it. Maybe you don't fully understand the risk. So stop throwing stuff over the wall and kind of just pointing fingers, right? It's, it's, you're in a team environment. Let's focus on the team. And, uh, you know, help them, help be advocates for your engineers. Especially with legal and finance and leadership. So that they can go through and accomplish the goals that The organization has, so

Chris Romeo: 51:37

I think that's a good place to

Jay Bobo: 51:39

be a good team.

Chris Romeo: 51:40

is be advocates for your engineers. Um, it's, it's just really good advice. And so this episode has been, been, we've, we've had good advice from, from both of you sprinkled throughout this entire episode. So, um, this is, this has been great to get your perspective as practitioners who are in the. In the moment, in the, you're in the mix. I always love to talk to people who are not theorizing, but are actually, we're doing this, we're, we did this, we did that. Like that's, that's just such a great perspective. So thanks for coming on the show, for sharing your perspectives. And we look forward to a, uh, a future episode where we'll unpack something else in the realm of AppSec or ProdSec. So thanks for being here.

Darylynn Ross: 52:24

Thanks, Chris.

Jay Bobo: 52:24

Thanks.