Chris Hughes -- Software Transparency

Chris Hughes, co-founder of Aquia, joins Chris and Robert on the Application Security Podcast to discuss points from his recent book Software Transparency: Supply Chain Security in an Era of a Software-Driven Society, co-authored with Tony Turner. The conversation touches on the U. government in the software supply chain, the definition and benefits of software transparency, the concept of a software bill of materials (SBOM), and the growth of open-source software. Chris Hughes is a proven cloud cybersecurity leader. with nearly 20 years of experience in the federal and commercial industries. Chris has a dynamic skill set with a blend of IT, cyber cloud security, and DevSecOps experience.

Connect with Chris Hughes:
→ Software Transparency: Supply Chain Security in an Era of a Software-Driven Society
→ Application Security Program Handbook

Mentioned in this episode:
→ Software Transparency: Supply Chain Security in an Era of a Software-Driven Society
→ Application Security Program Handbook
→ Agile Application Security
→ CNCF Catalog of Supply Chain Compromises
→ Aquia
→ Software Transparency: Supply Chain Security in an Era of a Software-Driven Society
→ OpenSSF
→ OpenSSL
→ Let's Encrypt
→ Sigstore
→ Minimum Viable Secure Product
→ Executive Order 14028
→ Agile Application Security: Enabling Security in a Continuous Delivery Pipeline
→ The Application Security Program Handbook by Derek Fisher

Follow the Application Security Podcast:
➜ Home
➜ X
➜ LinkedIn
➜ YouTube
➜ Instagram
➜ Facebook

Chapters:
00:00 Meet Chris Hughes: Software Transparency
03:42 Yeah, and you're, uh, you're co author of the latest book
04:31 Yeah, one thing that I kind of went down the thought
08:13 Yeah, yeah, and I think it's going to become a bigger
09:37 I think we've got, uh, we've got a couple different angles
11:52 Yeah, that's, that's always, you know, security and privacy, not privacy
15:32 I mean, that's why, when you think about one of the
17:01 Yeah, we're talking about the funding of nation states. Right, they
20:10 Yeah. So I've, uh, you know, I've kind of been waiting
22:48 Yeah. This is a weird situation in the sense that, you
25:14 Yeah. I mean, it's definitely a real risk. Uh, I feel
28:54 Yeah, that's, that's, but that's the reality of the, the world
31:51 All right. Well, let's, uh, let's talk about the, the kind
35:23 A billboard, what would it say for a message if you
37:14 Then, uh, yeah, Derek, excuse me, Derek was a guest on

Transcript

Copy Transcript Copied

Meet Chris Hughes: Software Transparency

Chris Romeo 0:00

Chris Hughes is a proven cloud cybersecurity leader. with nearly 20 years of experience in the federal and commercial industries. Chris has a dynamic skill set with a blend of IT, cyber cloud security, and DevSecOps experience. He enjoys working across interdisciplinary teams to solve complex organizational and industry wide problems to achieve technological transformation securely. Chris writes and speaks extensively about cyber security under the heading, Resilient Cyber. Chris joins us to discuss software transparency in a supply chain context. We cover the role of threat modeling in the supply chain, the significance of SBOM, and AI's impact on the supply chain landscape. We hope you enjoy this conversation with Chris Hughes. Hey folks, welcome to another episode of the Application Security Podcast. This is Chris Romeo, CEO of Devici, also general partner at Kerr Ventures, and AppSec aficionado, I don't know, I'm into adding names to things that people are doing now, but joined by Robert Hurlbut, my longtime co host on the podcast here. Hey Robert.

Robert Hurlbut 1:53

Hey Chris, yeah, Robert Hurlbut and I'm a Principal Application Security Architect and Threat Modeling Lead at Aquia, and really excited about interviewing our guest, very familiar to me, and, uh, been, been trying to get him on the podcast for a while, so really glad to see. Uh, Chris here, or another Chris, Chris Hughes.

Chris Romeo 2:11

yeah, without further ado, so Chris Hughes is our, our guest today. And Chris, as you know, we like to dive right in to security origin story. Take us through how you got to AppSec, how you got to security. I know you do a lot of things outside of AppSec, too, but just kind of take us from the early days and bring us forward.

Chris Hughes 2:28

Yeah, definitely. I got my start, you know, obviously, like many in their career field, like, you know, tinkering with computers as a teenager and stuff like that, but ultimately joined the military and served in the U. S. Air Force, where I did everything from starting on the help desk to advancing to network and system administration over to cyber security, because I was fascinated with the security side of things, and then got out from there and worked as a federal civilian a couple of times with the Navy doing cloud and DevSecOps over there. And then also, um, Working with an agency known as the General Services Administration, which runs FedRAMP, if anyone's ever heard of FedRAMP. So, uh, reviewing a lot of the cloud services come to the federal marketplace. And then ultimately ended up co-founding a company named Aquia, where I serve now as the co-founder and president. And yeah, just pretty involved and engaged in the community in terms of writing and speaking, you know, whether blogs, articles, you know, books. Uh, podcasts, webinars, I host a podcast of my own called Resilient Cyber, and I just have been really fascinated in everything around application security, software supply chain security, especially as we've seen these topics evolve over the last several years with the push for DevSecOps and adopting agile methodologies, and now the, you know, software supply chain attack accelerations, whether it's proprietary software, open source software, uh, cloud service providers, managed service providers, and everything in between. Uh, so I'm excited to be here and chat with you guys.

Chris Romeo 3:41

Yeah, and

Yeah, and you're, uh, you're co author of the latest book

Chris Romeo 3:42

you're, uh, you're co author of the latest book you did, Software Transparency. Tony Turner, who was a guest on the podcast a couple of months ago. So, uh, we got a chance to meet Tony and I think the book had not yet come out at that point. And so, um, we were, we were waiting to talk about it. I was specifically waiting to talk about kind of some of the details in the book. And so that was, that's the, I guess, the genesis of some of the questions we have today are things that I was scanning the book and said, Oh, I want to, I want to learn more about this and about Chris's thoughts and these different areas. So, um, yeah, Robert, where do we start here? What do you think?

Robert Hurlbut 4:15

Yeah, so we got a few questions to ask today. Let's start with, uh, first of all, how would you define software transparency in the supply chain context? And what are the tangible benefits of achieving full software transparency for an organization?

Yeah, one thing that I kind of went down the thought

Chris Hughes 4:31

Yeah, one thing that I kind of went down the thought process with this when I was writing the book is we've seen like things like the CIS critical security controls that have things like software asset inventory, you know, listed for what decades now, right? Uh, but when we look at the software supply chain, most organizations don't understand what software they're using, where it's coming from, what's inside of the software that they're using, you know, whether you're talking about ingesting and using open source software components for internal development purposes or consuming, you know, proprietary software that overwhelmingly includes a lot of open source software as well. Uh, we're obviously consuming services from cloud service providers. We have managed service providers using software. Uh, so software transparency to me is kind of looking at that broader enterprise environment for an organization, for example, and understanding who are all of our suppliers, you know, what's in the software that we're consuming, whether we're developing it organically in house, uh, first, you know, first party code, third party code. And consuming it from third party, you know, COTS providers, for example, as well, commercial off the shelf software suppliers, because we've seen like this increasingly, it kind of reminds me of parallels to globalization in the sense that we're consuming all these goods and services from all over the place, but we really have no insight into where it's coming from, who touched it, what they did to it. You know, how it was created, uh, what, you know, maybe nefarious activities were done along the way before it got to us and anything in between there. So, I think software transparency is just like an organization taking a step back and looking at that, you know, looking at all the software they consume and use and understanding, you know, where it's coming from, what's been done to it, what's inside of it, etc.

Chris Romeo 5:54

So given that the book is Software Transparency and it's going to have a software bent, from your perspective, what's, what's the role of hardware though? Like do we even have to care about hardware anymore? Like in the, in the context of the things that we build? Like, because I remember the old days where when you studied supply chain you would say Hey, there's threats to the hardware in production and meaning production being built, hardware being built, components being snuck in. But now in the age of cloud, things are so logically allocated and whatnot. It's like, I don't even know if there is a server. There's got to be a server underneath this thing, but I don't know what it is. But so what are, what are your thoughts on, on hardware in this, in this world as well?

Chris Hughes 6:36

Yeah, I think it's, it's most definitely part of the supply chain, you know, I don't want to call it software supply chain, but obviously there's a lot of things like firmware and other aspects of hardware that come into play, you know, from a software perspective. We just saw a publication. I think it was from NSA and CISA about, you know, I think it was Chinese malicious actors that were kind of embedding themselves in all these networking devices around the world and monitor communications, you know, and then I've seen other, uh, folks like Tom Pace of NetRise, for example, say he's sitting on millions of, Uh, vulnerabilities that don't have CVE identifiers and hardware and devices. Uh, so it definitely, uh, you know, still has a play. Uh, it's just, you know, I've come up in a world where, you know, uh, I remember the days of running cable and stacking servers and doing things like that in the military, but, you know, thankfully I've been part of the evolution that's moved towards, you know, declarative infrastructure as code and cloud native environments. And it is a little bit easier to discuss, you know, the, the supply chain and the software context where hardware, you know. It's a whole other beast and that's not my area of expertise, but I think it's definitely relevant and it's still some significant serious threats when it comes to hardware as well.

Chris Romeo 7:36

Yeah. And that's, it's, I think it's going to become a bit of a forgotten. I love making predictions because no one can no one can hold you to them. They can't they have to come back five years later and say, oh, he was wrong, or he was right. But when you think

Chris Hughes 7:52

I mean, I was just going to add real quick on that. You know, you think about the fact that, you know, like I talked about globalization, uh, same thing in the sense of manufacturing. We don't natively produce a lot of things here in the United States, whether you're talking chips and hardware and physical assets and things like that. You need to be concerned about where it came from, who touched it along the way, and what may have been done to it before you got it. Uh, so it's definitely a major concern, at least for me, I think it is.

Yeah, yeah, and I think it's going to become a bigger

Chris Romeo 8:13

yeah, yeah, and I think it's going to become a bigger problem in the future here as we go forward. I think as I think it will become potentially a forgotten. angle of a forgotten threat vector, right? Cause we just think of the cloud as somebody else's computer, but we never really worry about the fact that it, what they're doing with that computer, right? So there's, there's some opportunities and that's a nice transition into threat modeling of, you know, thinking about those, those challenges that we might be struggling with. And so from your perspective, um, What do you see as the connection between threat modeling and supply chain?

Chris Hughes 8:51

I, I mentioned this before it went on air, but you know, a little shout out to OpenSSF, who published recently a notional kind of threat model of a software supply chain. Um, and to me, you know, I always kind of go back to the fundamental four questions, like, you know, what are we working on? What can go wrong? You know, what are we going to do about it? Did we do a good job? Et cetera. It's looking at, like, from an organizational perspective, your software supply chain, understanding, you know, where are we getting things from? How do we ingest things? What kind of security processes and procedures do we have in place? Uh, to implement rigor and governance around how we consume software, how we produce software, how we distribute software for a supplier. Uh, so doing that kind of threat modeling exercise is a critical piece of this activity in my mind of just understanding how we even, how we even, you know, ingest, consume, distribute, et cetera, software is an activity. I think every organization should be doing to really get a handle on where they stand.

I think we've got, uh, we've got a couple different angles

Chris Romeo 9:37

And I think we've got, uh, we've got a couple different angles to threat modeling of the supply chain. One of the things I've been recommending to folks recently is do a threat model of your build pipeline. We know that there's been a number of different breaches in the past couple of years where attackers have used the build pipeline as a vehicle for chaos. So, why, why are we not threat modeling the supply chain or the, uh, the build pipeline as, as a component of the supply chain? And then the other piece that, that keeps. Hitting me, like making me think about how do we, how could we do this is it seems like if we could introduce threat modeling to the open source world, like those that are building these packages and whatnot, like we could unlock a whole bunch of capability because we know open source people are scrappy, they're smart, they build the things that drive, let's be honest. Everything we do is pretty much driven by open source at this point. But imagine if we could get threat modeling to them, how that would impact the supply chain further down the field. If we had them thinking through. As they're building libraries and stuff, just, I don't know. It's a dream of mine to

Chris Hughes 10:43

Yeah, I was gonna, I was gonna jump, jump in there real quick. Uh, you know, a couple of good points you make is like the build process and pipeline. You know, we had, we've seen, uh, OWASP, essentially, I know you're closely, uh, involved with that organization, adopt the CICD top 10. That's a great resource that we cover in the book for folks to take a look at. And malicious actors have realized that if they can compromise that build process and pipeline and tools, now everything coming off of it could potentially be a downstream cascading impact across all the software consumers, whether in terms of the organization or externally as customers and consumers of the software. Um, and I like the idea of threat modeling, uh, to the open source software, you know, uh, contributor maintainer community. But I think, um, it's an interesting kind of, uh, you know, uh, to say ethical, but like philosophical discussion in the sense that like, you know, are they incentivized to take the time to do those activities or are they just doing this on their own free time? And, you know, you know, it's kind of like that, you know, age old debate of like, how much, uh, time would they put into security? Some of them. I think it was Harvard, uh, and Lynx Foundation found that, you know, most, uh, open source software contributors describe security as kind of a dreadful, soul withering activity, I think is the term they used. Um, so, you know, it's like, will they take the time to do those things? I sure hope so. But, you know, I know many of them, you know, they may not be inclined to do those kinds of things, obviously.

Yeah, that's, that's always, you know, security and privacy, not privacy

Chris Romeo 11:52

yeah, that's, that's always, you know, security and privacy, not privacy, not so much in the open source world. Cause there's not, they don't, they don't have data. Maybe they're processing data, but they don't have the data, but it's been a challenge for years of trying to infuse security into the open source world. And some of it has been the result of resources. Like I remember the days when open SSL was still running with like two people. There were two people that were creating this library, and the entire industry was building their technology products on top of it, and there were literally two people creating the library. And so, I mean, it's gotten better with them being, I think they were one of the initial cases of funding an open source project through foundations and contributions from those that are benefiting from it, um, which is kind of the model, which I think it's the only model that's going to work for these, these types of very, you know, crucial kind of pieces of software. I mean, that, I mean, can you call OpenSSL critical infrastructure? Kind of feels like it is because it is everywhere, right?

Chris Hughes 12:54

there's, there's actually discussion, you know, going on in organizations like CISA and others and some of the federal dialogue around critical infrastructure and, uh, uh, Definitely a plug for an individual, a researcher named Chinmayi Sharma, whose research I cite, and she talks about, she makes a case for open source software being a critical infrastructure, you know, because it's so pervasive across everything from industrial control systems, you know, national security systems, our leisurely consumer goods. Like you said, it powers our modern digital society. Uh, but it also suffers, you know, uh, from economic issues in the sense of the, they call it like a tragedy of the commons where people continue to exhaust this resource. And if someone else is going to do it, why would they bother investing the time, money, you know, money or resources to, to handle it if someone else is going to do it for them. And the problem is everyone's kind of taking that approach. And now it's, you know, we have what we have.

Chris Romeo 13:41

Yeah. Yeah. And it's, I think of the cases of companies that have built, even in our security space, companies that have built on top of an open source project and, it's not the classic open source company model where you provide a great open source project and then you provide support to it. That's the business that goes with it. Kind of the Red Hat model. It's let's take this open source project and make our product kind of sit on top of it, which means we don't do much by way of innovation. We rely upon the open source project underneath. And so that's one of those challenges I think we got to solve. And I know OWASP is wrestling with it and others in the industry are wrestling with how do we, how do we, how do we fund those projects that other people are building companies on top of? And I think at the end of the day, they need to fund it. Like if you're building a company on top of it, you have an obligation to, to support the thing underneath. Like, it seems like that's just a classic business risk move. Like my risk is that this open source project goes away and then my company's gone. Well, I should probably fund it and make sure it doesn't happen.

Chris Hughes 14:46

Yeah, most definitely. And that comes back to like, uh, you know, things I was diving into with the book around licensing, for example, they may not feel inclined to have to contribute or, or be responsible for it if they think they can use the licensing to their advantage. Uh, but as you mentioned, if you build your entire business around this, this project, you know, or capability or tool or whatever, and the open source software community dries up, uh, well now you have a real business risk on your hands. And, and one thing I found fascinating when I was digging into this is like some of the metrics, I think it was, Uh, 25 percent of all open source software projects have only one contributing, uh, maintainer and like 94 percent have less than 10. Uh, so it's this massive ecosystem that we all rely on for our products and services, but you know, it's maintained by this very small group of individuals largely doing it on their own free will.

Chris Romeo 15:31

I mean,

I mean, that's why, when you think about one of the

Chris Romeo 15:32

that's why, when you think about one of the scariest software supply chain threats, it's the infiltrator. It's the, the person who spends six months, 12 months becoming part of that core team and then gaining their trust all for a moment in the future where they can pull the trigger on putting some type of package or payload into the library and get it published and, um, that's, I mean, there's, there's the, the hidden, there's the dark seedy underbelly of the software supply chain right in front of us. And it's, I don't see a lot of people talking about that though. I mean, Chris, do you see people, is that a threat that, that people are considering right now?

Chris Hughes 16:11

So that's, uh, that's captured and you know, there's some good resources out there. One plug I want to give is for CNCF's catalog of software supply chain attack types is you can go in there and it's one of the ones that are cataloged there, but it's not one that I think people pay attention to. They pay attention to things like credential compromise of a maintainer or contributor, for example, but if I'm a resource strapped, you know, uh, open source software project, I need, I need additional help and someone comes along, you know, and it seems capable and competent and willing to help and they start helping. And the next thing you know, you kind of have an insider threat. And obviously these open source software projects don't have insider threat programs like large enterprise would. Um, so, you know, I think it's definitely something to think about and these, uh, malicious actors are nefarious and they're patient. You know, especially APT type groups, they'll get in there and they'll, you know, stay active for a long time and, and seem, you know, legit. And then next thing you know, they make a malicious, uh, contribution and it could have a devastating impact downstream.

Chris Romeo 17:00

Yeah, we're talking

Yeah, we're talking about the funding of nation states. Right, they

Chris Romeo 17:01

about the funding of nation states. Right, they have time to put a resource, they have time to send a resource to, to college or university, get a CS degree and then become a programmer full time almost supporting a, uh, an open source, a prominent open source project, like that's, they have the time to be able to pull that type of thing off and, and You know, it's, it's, it's, there's a lot of challenges in the supply chain. So let me bounce another, this other crazy idea I've had, and I've been bouncing it off supply chain people that, that people that think about software supply chain a lot, um, just to see if, if there's any kind of, um, there's anything here, and then I'm hoping to, to, to encourage somebody else to go build it. Cause I'm not the one to build it, but you think about the impact Let's Encrypt has had. on the SSL slash TLS certificate generation process. Like who pays for a certificate anymore? I don't pay for any of them. I have multiple web properties and I don't pay for a certificate and I haven't in five, 10 years or whatever. Since Let's Encrypt first came out, I was like, that's brilliant. It'll refresh every night, 60 days or 45 days or whatever. Why don't we have a Let's Encrypt? Version or a, a similar approach for open source packages where it could be as easy as let's encrypt for every package maintainer producer out there to sign their packages according to this kind of common infrastructure that we've had. So, like, is somebody doing this already? Like, am I inventing something that's already come up that someone's already built for us? Or like, what are your thoughts? Like, or is that a viable, is that a viable approach?

Chris Hughes 18:39

Yeah, I think that, you know, a couple, a couple of things on that note is, um, you know, one is we're seeing the federal government and we talked a little bit about this off air and I know we're going to dive into this some more, but the federal government talk about what role do they play? You know, if we're talking about things being critical infrastructure or public good, for example, and I know organizations like the Atlantic Council have made some cases for the federal government to get more involved in terms of resources and investments. So maybe this is something they help fund this kind of thing. And then there are some projects out there. Uh, like SigStore, uh, which includes Cosign, they're trying to make it easier for software developers to kind of sign, you know, their activities and things like that to give some level of assurance and trust, uh, but I think you're spot on, like, you know, we need to see something like this that can be widely accessible, easy to use, and easy for folks to adopt, uh, that way they don't, you know, feel it's too cumbersome, uh, because it could be a massive contribution to the community.

Chris Romeo 19:26

I just need somebody else to build it now. It's always the key, right? It's like, it's like, I think we should really do this, but I'm not the one to do it. I'm just going to keep talking about it until somebody else is like, I have this good idea and I'm gonna be like, yeah, that's a really good idea. You should do that. A hundred percent. I'm behind you.

Robert Hurlbut 19:43

that's how it starts right though, get good, good ideas and somebody runs with it. Uh, so Chris, uh, you know, this is a topic we've, we've sort of covered a different times here on the podcast. I know, uh, Chris Romeo here, it's, is one of his, uh, hot topics. Uh, but I'm going to ask you, uh, can you elaborate on the significance of the, uh, software bill of materials or SBOMs and how do they enhance security integrity of software supply chains for business?

Chris Hughes 20:09

Yeah.

Yeah. So I've, uh, you know, I've kind of been waiting

Chris Hughes 20:10

So I've, uh, you know, I've kind of been waiting for this one. Cause I listened to your guys show and I know Chris is a big proponent of SBOMs. No, I say that jokingly. I've heard him kind of, you know, dive in on the topic with different guests. And, you know, for me, I don't think of SBOM as improving the integrity. I think of it addressing, you know, longstanding information asymmetries that have existed between software suppliers and software consumers, uh, to show them, you know, what's in the software that you're consuming, at least give that visibility or level of transparency, like we see, I kind of find it ironic that we're seeing this widespread push for zero trust. Right. No implicit trust, but we're supposed to implicitly trust software suppliers and consume the software they produce without really understanding what's inside of it. But that said, and I know Chris has made these points before, there's some major challenges to the practicality of it in terms of formatting, depth of the SBOM, tools to ingest, enrich, analyze, and report on the SBOMs. Uh, so as he talked about with previous guests, if you don't really have that in place, uh, you know, you're kind of just filing this in a cabinet somewhere and doing it as a compliance exercise, not really using it as a tool to actually drive down risk. As an organization. Um, but that's, I think it's being pushed for as a means to address, you know, that lack of transparency, that lack of information that consumers have from software suppliers. Um, and the last thing I'll say, you know, is like, you know, I think it's something where we don't want to let perfect be the enemy of good. Uh, is the industry in the position right now to take advantage of SBOMs and really use them to the full potential of what some folks see them to be? I don't think so. Uh, but is the alternative of just kind of burying our head in their sand and keep consuming things without understanding what's inside of them, uh, viable alternative? I also don't know that that's, you know, the right way to go either.

Chris Romeo 21:41

Yeah. And I'm a, I'm a proponent of transparency. Like I don't have any, I don't have, honestly, I don't have any problem with the whole idea of SBOMs. It's really what's been just been bugging me for the last year or two is people are promoting this as the answer to all of our problems. And it's like, it's not the answer. It doesn't do anything. It's a, it's an information collector of, and that transparency is great. If you can act upon it and generate business value from it. Like I think about companies like Cisco, where I used to work with it, have tens of thousands of SKUs. And so for them to build an SBOM factory. It'd have to be a factory because there's so many moving pieces and parts and things like they would have to it's just it's a gigantic undertaking. It would cost them probably my guess is tens of millions of dollars to implement it. And so I just want to see us get to the point where there is an easier way to derive value from it.

Chris Hughes 22:40

Yeah.

Chris Romeo 22:40

behind it and I'll say, let's do it. Let's, let's, let's call it. Let's force everybody to do it. But I just, that's where I think we're missing. I think we're

Yeah. This is a weird situation in the sense that, you

Chris Hughes 22:48

Yeah. This is a weird situation in the sense that, you know, we're actually seeing kind of regular typically I've been around government and like government tends to lag behind technology. Right? And this is a weird case where regulation and requirements are ahead of where the industry is in terms of capabilities, in some sense. Uh, people are being kind of forced in this direction to start producing and providing these artifacts, even though the people requesting have no real capability to ingest, you know, reason about them, make sense about them, and use them to drive down risk to the organization. So I think you're spot on. Uh, and it could become an incredibly expensive endeavor, not really generating any real risk mitigation or risk reduction. Uh, but I do think, you know, in the long run that having that transparency is fundamental to, you know, just trusting the broader software, the supply chain, you know, software we consume and use.

Chris Romeo 23:32

Yeah, and I'm fine with the government even mandating it, right? Like I, I helped stand up Cisco's FIPS 140 program. For those people, I'm realizing everybody on the line here, everybody knows what I'm talking about. But for those people that don't know, um, FIPS 140, it's a, it's a dash three is the standard now. It was dash two in those days, but NIST put together this document of requirements that describes how you create and generate products and services and things that do crypto correctly. And they came up with this standard of, um, and they do these really cool things like they generate algorithm testing. So when you're being certified, they, they send you a bunch of like encrypted data and they give you the key. And then, and then you have to generate, you know, kind of the other side of the cryptographic equation to, to prove that your AIS implementation works. Um, so I don't have a problem... so, so like I see, I've watched the value of that. And at the end of the day, a lot of people didn't like FIPS 140. And they said, this is just a compliance thing. It made sure you had to prove that crypto worked. And so the SBOM is in the same way. Like, even if we mandated it, it proves what stuff is in there. I just want to see action. I want to see people taking action before we force people like big companies to spend millions and millions of dollars to really get nothing other than another checkbox. Cause like, we know checkbox compliance, it just doesn't. If you're going to put the effort in, you might as well get some value out of it. That was always been my philosophy, even though I was around FIPS 140 common criteria, these types of certifications. I'm like, let's get some value out of it. Yes, we could just check the box, but let's make our product better at the end of the day.

Chris Hughes 25:13

Yeah.

Yeah. I mean, it's definitely a real risk. Uh, I feel

Chris Hughes 25:14

I mean, it's definitely a real risk. Uh, I feel like we deal with this dichotomy in cybersecurity often when we look at things like, you know, SOC2, FedRAMP, ISO, et cetera, like all these compliance certifications where organizations are still getting breached for lacking fundamental security controls and capabilities, as well as, you know, how many times have we been through the security questionnaire exercise of, hey, send me this list of 372 questions. We're not even going to review the response. We're just going to tuck it away in a folder and say, we asked it. You know, from you, uh, is it really driving down risk or is it just security theater?

Chris Romeo 25:43

I mean, it's, yeah, as the recipient of many of those in my, uh, first company's days, like, and it was so funny because like. I would go through and have to answer these things, and then I get on a call, and it's somebody who, and I'm not, I'm not hating on anybody, don't get me wrong, but like, they've got, they, they graduated from college two years ago, they got two years experience out of the belt, and they're like, kind of arguing with me about kind of architecture and things, and I'm like, I mean, I've been doing this a long time. Like you were in elementary school when I started doing this. Like, and so I don't, I don't want to argue with you about the best practice of security, but I feel like I've been around a little bit. Like, I feel like I I've, I've learned from people who, you know, kind of expanded my mind. So, um, yeah, that's, that's been a sore subject for me. And, and, uh, but I know a lot of people are, a lot of people, a lot of companies aren't started by security people. So they, they maybe aren't, you know, everything I did from what we were building, I'm like, I was, Oh, security first. Of course we're doing the right thing for security. Cause that's the only thing I can think of how to do it. Like, I can't think of another way to do it.

Chris Hughes 26:48

Yeah. Yeah. One thing I want to throw in there, I listened to a recent conversation, you know, and you were talking about doing security, you know, we're talking about kind of the distinction between real security versus compliance exercises here. Uh, but the reality is for a lot of these, you know, software startups and, and new companies like that, they're just focused on speed to market, market share, driving in a return on investment for investors. And the SOC 2, for example, could mean the difference between a sale or losing a potential lead where, you know, doing some of the more rigorous security activities that we hope to see, you know, it's great from our perspective, but it's not really going to necessarily drive revenue for them. Uh, and that's why I think that we see people focus on compliance more than security in some cases.

Chris Romeo 27:24

Yeah. I mean, having gone through SOC 2, like it's, I mean, it is, it's a minimum bar, right, but it's not a bad minimum bar. I'm not, I don't dislike SOC 2 either. Like. I see the value in raising the bar and one of the things that we have an upcoming episode that it's going to release soon was the minimum secure viable product. I always get that wrong. Um, the, the guy from Google, Chris John Riley, who, uh, another, another Chris in security, but he, um, they've got this standard that, and, and I went through and read it. Before we did the interview with him and I'm like, this is brilliant. Like it's not, they're not asking people to build a fortress of solitude to protect the data. You know, it's, it's a, but it's a solid set of requirements that you can build on that are more layered. You don't have to do everything and that there's a minimum standard, but there are things you can do to exceed. Um, and that's, you know, that's the key in the startup game because you don't have all the funds in the world to to build the most secure possible thing that you could do. You want to do the best you can.

Chris Hughes 28:21

Yeah, one thing I want to throw back at you guys, I know you guys are the podcast hosts here, but you know, I like to ask questions too, is like last week we saw NSA and CISA come out with their top 10 cybersecurity misconfiguration list, which I think you guys probably saw, and the whole security community just kind of collectively yawned at that, like, oh, everyone should be doing that. That's, we all know that, that's basic. But those are still the most pervasive issues that are getting organizations in trouble and exploited. Um, so it's weird that we have this situation where we all know, like, what the basics are and what should be done, but they're almost never done, especially at scale in large, complex environments.

Yeah, that's, that's, but that's the reality of the, the world

Chris Romeo 28:54

Yeah, that's, that's, but that's the reality of the, the world that we live in, unfortunately. Like, I mean, I've assessed different companies over the years and people, you wish people would do kind of the basic things. But they don't, and that's why a thing like SOC 2 is so valuable. And it, it will counter a number of those misconfigurations that are included, and SOC 2's become, it's not as difficult to do anymore as it used to be, given you have a couple of vendors who will help you to put together your package of data, um, and really drive it through. So yeah, I mean, I think whenever I see one of those new lists and things, like one, we've got way too many top 10 lists in our world, um, but, It is, it just continues to reinforce the fact that we're just not, we're not even taking care of the simple things across the board. And, you know, it's, it's, we, we just need to, we just need to do better from that perspective. Um, and somebody needs to push, you know, the, the, or the market needs to push. I think over time, the market will start to push more where people will not be able to ignore security and privacy, or that will result in their company just going away. We're just not there yet. So, all right. One more SBOM thing I want to, I want to throw at you, cause I'm curious about your take on this. So I keep seeing new BOM standards come out. Um, there is a SASBOM, there is a CryptoBOM, we've had HBOMs forever. That's what we used to do at Cisco. We had an HBOM for a product that was going down the line, like that told you all the components. But, um, and I know I'm, I'm, I'm forgetting literally a hundred of them, cause there's all these other things like. Is this generation of 12, 000 different BOMs, is this helpful or which would we be better served if we just focused on SBOM as the core and, and trying to get that squared away first?

Chris Hughes 30:49

Yeah, I gotta answer this one carefully, because some of the folks that are involved in that, you know, uh, are folks like Steve Springett, who is, uh, one of the, he was the technical editor for the book, and I highly respect his opinion on

Chris Romeo 30:59

yeah, I mean, we love Steve here. He's been on the podcast a number of times before, brilliant man. I, but, but yeah, I mean, but, but I'm just asking the question, I'm just trying to understand, you know, and Steve don't, uh, don't, don't get mad at Chris

Chris Hughes 31:11

Yeah, I think at the end of the day, you know, these efforts are essentially trying to get at the same thing, which is transparency of X, whether it's an MLBOM, or a CryptoBOM, or HBOM, SBOM, you know, is really just trying to get to the transparency level of what, what is inside of the thing. And I think it's helpful to have that, but I think throwing the additional acronyms and things like that, obviously makes it more confusing. I still meet people who don't know what an SBOM is, never heard of SPDX or Cyclone DX, or they don't understand the NTIA minimum elements for an SBOM. So I think as that kind of expands and the acronym soup grows, it, it definitely makes our, you know, our, our case of educating the community on why we need these things and what they do a little bit more challenging for sure.

All right. Well, let's, uh, let's talk about the, the kind

Chris Romeo 31:51

All right. Well, let's, uh, let's talk about the, the kind of the government side. Cause, and then after that we'll go to the, uh, the lightning round. But I know since you're, you're a lot more plugged into the U. S. government world than a lot of people that we talk to. And so what's kind of, how do you see the government's role evolving in software supply chain? Are there things that you kind of see that they're working on now that are going to pop in a couple of years? Like what's your, what's your general take there?

Chris Hughes 32:20

Yeah, I think, uh, this is a pretty, I don't want to say unique situation, but you talked about, you know, things will evolve, things will change, things will be, you know, security will become a competitive advantage you talked about kind of in the sense that people will start to use it when they purchase products. Uh, but for a long time, people have considered security, cybersecurity to be a market failure in the sense that it's not going to reconcile itself on its own. Uh, and I think we've seen the federal government, especially around software supply chain security, uh, get more involved, be more active participant with things like the Cyber Security Executive Order. Uh, there's various OMB memos that have come out now. If you're a software supplier signed to the federal government, you're going to start to have to self attest to following this secure software development framework or potentially, you know, provide an SBOM or, you know, just, you know, things of that nature, showing that you're following secure development practices. And I think what the government is trying to do is use that massive purchasing power that they have, you know, they consume so many goods and services and spend tens of billions of dollars a year on software and technology, trying to use that to push that, you know, systemic tide across the ecosystem and drive that, you know, change that we hope to see, that insecurity we hope to see, and they've also been using a lot of language, especially SISA and others, You know, uh, around, you know, pushing the onus on the individuals or entities in a position to do something about the software suppliers, the manufacturers, rather than, you know, my grandma who just is using a router that she got from the ISP that services her home, you know, um, you know, and then we've seen efforts around like, uh, uh, labeling as well, cybersecurity labeling. You know, if you talked about trying to provide, you know, transparency to consumers, if you give an SBOM to the average person, they won't know what the hell to do with it. But if you give them a, Hey, this, this product is a C, well, maybe I'll go for the one that's a B or an A, you know, like trying to make informed purchasing decisions for consumers, for example. So, I think the government's trying to step in there and. Uh, change the tide of, you know, moving towards more of a secure by default ecosystem, more transparency around things that we consume and software now that software drives, you know, nearly every aspect of our society. Uh, but it's a difficult challenge because, you know, you got to be careful what you wish for. The heavy handed government can also stifle innovation and make things cumbersome and overly expensive. And, uh, so it's, it's, it's a double edged sword for sure.

Chris Romeo 34:22

Yeah, that's helpful to get, uh, to get your take, though, as an insider there. So, all right, Robert, it's time. We gotta, you gotta lead Chris into the lightning round here, so take it away.

Robert Hurlbut 34:32

All right, so we've got a few questions that we ask each of our guests on the podcast. So the first one is a controversial take, what's your most controversial opinion on application security and why do you hold that view?

Chris Hughes 34:46

I guess I'd say, and some may share this, I think many may share this, is we do a lot of complaining about developers and engineers and people should do X, Y, and Z, but I think a lot of the problems we face in cybersecurity are kind of self inflicted. We've heard a lot of DevSecOps and breaking down silos, but often we're making people's lives hell by putting, you know, massive vulnerabilities lists on them with no context, not really building rapport, you know, overemphasizing the technology rather than the relationships, the trust, the dialogue, communication, things like that. Uh, so I think that in security, a lot of the problems we face are kind of self inflicted. Maybe that, maybe that'd be controversial for some in the security community.

A billboard, what would it say for a message if you

Robert Hurlbut 35:23

So a billboard, what would it say for a message if you can display a single message on a billboard at RSA or Black Hat Conference?

Chris Hughes 35:32

Yeah, I was joking about this one because I, I talk a lot and write a lot. So, uh, to try to put it succinctly, I would, uh, put a billboard up that says, "People. Process. Technology." in that order.

Chris Romeo 35:44

Hmm. What about governance?

Chris Hughes 35:47

Governance, we'll tack on governance.

Chris Romeo 35:49

Alyssa, Alyssa Miller was a guest on the podcast years ago, and she She taught us that kind of concept because we, somehow I mentioned people process tools and she's like, Oh, wait a minute, people process tools, governance. And I'm like, Ooh, so it's just stuck with me now that you mentioned that, um, props to Alyssa for, for sharing that with me and Robert years and years ago. But, um, it's just, it's a nice collector. To bring those other things kind of together. And so,

Robert Hurlbut 36:20

And then, uh, final question is, uh, what's your top book recommendation and why do you find it valuable? And I'm curious about this one because I know you are a big reader, uh, like all of us, so very curious.

Chris Hughes 36:29

Yeah, definitely a couple. I'm going to make two. Uh, you know, we talked a lot about application security, so I'm a big fan of a book called Agile Application Security, and it's, it's produced by three to four authors. I can't cite their names right now, but it's incredibly, uh, an awesome book about how to integrate security into Agile and DevSecOps and do it effectively. Uh, and then another little plug for Derek Fisher's Application Security Program Handbook. I thought that was a great book as well, and talks about some of the things that we discovered around, you know, building relationships, you know, vulnerability management, a lot of topics that we just discussed.

Chris Romeo 36:58

yeah, those are two, two excellent ones for the library. I, uh, Agile AppSec, it's, it's been out for a number of years though, cause I think it was Laura Bell and, um. Forgetting who the others, but I always remember Laura as kind of one of the

Robert Hurlbut 37:12

right, Laura. Yeah,

Then, uh, yeah, Derek, excuse me, Derek was a guest on

Chris Romeo 37:14

And then, uh, yeah, Derek, excuse me, Derek was a guest on the podcast here right after his book came out too. And so Chris didn't mention it, but I'm going to mention it here because I can. I'm one of the hosts. Uh, Chris and Tony's book is called Software Transparency, Supply Chain Security in an Era of a Software Driven Society. It's available from Amazon, published by Wiley. So, uh, definitely, uh, is a solid one to add to your uh, bookshelf and, and to read and, and really understand, uh, very, very in depth approach to the, the problem. And we just scratched the surface here in this interview of, of the depth that's there. So Chris, what about, what about a key takeaway or a call to action for our audience?

Chris Hughes 37:56

This one, I'd say, you know, one piece of advice I often give security peers is be empathetic, you know, go out and try to understand the incentives and things that people are doing and why they're doing them, what kind of things are driving them to do what they're doing, because often it's, it's simple for us to say, oh, you should be doing X or you should be, we have to understand what their role is. How are they graded? How are they, you know, their performance, you know, graded, for example, what's driving their behaviors and building that empathy has gone a long way for me in my career. So I think just putting yourself in someone else's shoes and can go a long way.

Chris Romeo 38:25

Very cool. Well, Chris, thanks for being a part of the podcast here and sharing your opinions, thoughts, and trying to convince, or trying to change my mind on SBOM. That was a good take. Everybody has to spend some time trying to do that. But yeah, I really enjoyed the conversation and I always appreciate being able to listen to you and hear you speak at conferences. You just always got great things to say and you got things that challenge my thinking, which those are the only types of talks or things I listen to anymore is people that are pushing the envelope with me. So we appreciate you and look forward to having you on again as a guest at some point in the future.

Chris Hughes 39:03

sounds good. Thanks so much for having me on. I'm a big fan of the show and look forward to continue to tune in.

Chris Romeo 39:08

Cool. Thanks.

Robert Hurlbut 39:09

Thank you.