Chris John Riley -- MVSP: Minimum Viable Secure Product

Chris John Riley joins Chris and Robert to discuss the Minimum Viable Secure Product. MVSP is a minimalistic security checklist for B2B software and business process outsourcing suppliers. It was designed by a team that included experts from Google, Salesforce, Okta, and Slack. The MVSP objectives are targeted at startups and other companies creating new applications, helping such organizations meet security standards expected by larger enterprises like Google. The MVSP is designed to be accessible for users, as a way to streamline the process of vendor assessment and procurement from the start to the contractual control stages.

Using MVSP, developers and application security enthusiasts can establish a baseline for building secure applications. MVSP includes controls about business operations, application design, implementation, and operational controls. For instance, it encourages third-party penetration testing on applications, as it believes that every product has an issue somewhere and needs regular testing to maintain a good security posture. The controls are designed to be reasonable and achievable, but also evolutionary to keep up with changes in the cybersecurity landscape.

Moving forward, MVSP intends to continue updating its guidelines to reflect the realities of the software development landscape but to keep the number of controls manageable to maintain wide acceptance. Chris encourages firms to consider MVSP as a baseline during the Request for Proposal (RFP) process to ensure prospective vendors meet the required security guidelines.

Links:

• Minimum Viable Secure Product:  https://mvsp.dev/

Recommended Books:

• Cybersecurity Myths and Misconceptions... by Eugene H. Spafford, Leigh Metcalf, and Josiah Dykstra: 
  • https://www.pearson.com/en-us/subject-catalog/p/cybersecurity-myths-and-misconceptions-avoiding-the-hazards-and-pitfalls-that-derail/P200000007269/9780137929238
• Phantoms in the Brain: Probing the Mysteries of the Human Mind  by V.S. Ramachandran
  • https://www.harpercollins.com/products/phantoms-in-the-brain-v-s-ramachandran?variant=32130994110498

FOLLOW OUR SOCIAL MEDIA:

➜Twitter: @AppSecPodcast
➜LinkedIn: The Application Security Podcast
➜YouTube: https://www.youtube.com/@ApplicationSecurityPodcast

Thanks for Listening!

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Transcript

Chris Romeo: 0:00

We're thrilled to have Chris John Reilly, the driving force behind Google's Minimum Viable Secure Product Initiative, MVSP for short. As a key player in Google's security team, Chris aims to help the world bolster its software security. Before his impactful journey at Google, where he also led vendor security assessments and worked on third party security integrations, Chris made a mark as an IT security consultant across the UK, Germany, and Austria, specializing in security testing and research for financial services. We'll dive deep into the genesis of MVSP. Explore its development process and understand its alignment with other industry standards like NIST, ISO, and OWASP ASVS. From its real world applications to the intriguing story behind specific MVSP elements, prepare for a comprehensive tour of MVSP with Chris. So without further ado, let's embark on this insightful journey with Chris John Riley. Hey folks, welcome to another episode of the Application Security Podcast. This is Chris Romeo. I am the general partner at Kerr Ventures and also the CEO of a stealth application security startup. Joined, as always, by my good friend Robert Hurlbutt who's done traveling the world and has returned to whatever he calls home.

Robert Hurlbut: 2:01

Yeah, hey Chris, uh, Robert Hurlbut, and I'm a principal application security architect and threat modeling lead, and yes, it's good to be back home, and uh, it's sort of in my normal, uh, mic and, and lighting and all the other good stuff that, uh, when we put our podcast together.

Chris Romeo: 2:18

Yeah, but you don't have your production assistant with you anymore that was moving lights around and everything.

Robert Hurlbut: 2:23

Right,

Chris Romeo: 2:24

at one of your recording locations. I was so jealous. I'm like, where's my staff? How come I don't have a staff of people, you know, doing my hair and makeup here like Robert does? But, you know, you reach a certain level and, uh, you have a whole entourage that goes with you. Do you have a guy that carries your cell phone in your entourage?

Robert Hurlbut: 2:40

I don't yet, but maybe I should,

Chris Romeo: 2:43

I mean, that would be a great feature. I mean, some of the big hip hop moguls have people that just carry their cell phones. I've always wanted somebody with that job. All right. Enough fun along the way here. Uh, we're, we're super excited to be joined by Chris John Riley, um, who is the creator of, or the principal behind MVSP. We're going to unpack what that means. Minimum Viable Security Product. I think I got that right. But...

Chris John Riley: 3:10

Almost.

Chris Romeo: 3:11

Oh, you're gonna, you're gonna fix it for me. Cause I'm, I'm, I'm notorious for getting things wrong, but let's, let's, let's hear, uh, Chris, how you got into, to security, where did your journey begin? And just take us, uh, take us through that story if you would.

Chris John Riley: 3:25

Sure. Um, I mean, thanks so much for the invite. I appreciate the opportunity to chat to you. But I, I guess my, my journey started 25 years ago, dating myself a little bit now, uh, managing a Novell. Novell network environment in London, um, and I decided that this Novell network system, system admin thing was just not for me. It was just too tedious day to day, you know, creating accounts, deleting accounts. It's just, it wasn't what I wanted to do for the rest of my life. So I ended up moving to Germany. Um, to, to kind of broaden the horizons, work more in large Microsoft environments. I was working for a trading company at the time, um, and that was a interesting and eye opening experience, kind of moving from, from Novell to like wide scale, uh, NT environments. Obviously going through that learning curve of, you know, can you break a domain controller? The answer is yes, you can break a domain controller. Um, and, and I did several times. Um, And then after that, I kind of got the bug for security. It was something that cropped up a number of times. I ended up gravitating towards, you know, managing the SMS server for Windows, you know, managing the WSUS server and things like that. And it was like, Oh, patching and keeping things up to date. And why do we have VNC on these servers? You know, this is easily crackable. And, you know, and the most fun in that job was proving that, like, VNC is crackable. Okay, prove it. Okay, cool. Go off, crack VNC, come back. Look, like, we're wide open, right? Patch it and then move on to the next one. And I realized that that's just what I wanted to do. For a job, right? You know, getting paid to prove to people things are not as secure as they, as they seem. Um, and that prompted a move to Austria. I like moving countries. Um, so, uh, and I, I joined, uh, uh, a bank, um, as one of their pen tests leads, uh, doing penetration tests, PCI pen tests in their environment, but also like large scale tests for a variety of government agencies and things like that in Austria, lots of good experience there kind of diving into different environments and things like that, including on site pen tests Which are one of my favorite things that I never get to do anymore is like drop in Sit on someone's network and just go why can I listen to all your customer service calls? This is not right. And things like that. It's it was a lot of fun, but also You hit the barrier there where like, Oh, I've seen, seen this, I've done this. It's another web app test. It's a, it's another this. Um, and I decided that Austria was not for me anymore. I wanted to move somewhere else. And that's where I, uh, reached out to some friends in the industry, you know, through presentations at like DEF CON and local European conferences. I had some good connections in the industry and I reached out to a few and they said, sure, you should totally come and work for Google. So, uh, I went, turned up for the interview an hour late and somehow still managed to get the job. So, um, I think luck more than, more than anything else. And I've been doing, uh, you know, penetration testing, red teaming, vendor security work for, you know, close to nine years now with Google, uh, including running their vendor security assessment process or being the tech lead for that process for about seven years. So yeah, a good range of stuff from sysadmin all the way through. Just, just don't ask me to code anything. If anyone's seen my Python code. I'm sorry. It's not, it's not good.

Chris Romeo: 6:53

uh, that's a, a, a good nostalgic reminder of Novell Netware. So, Thank you. Which version though? Three or four?

Chris John Riley: 7:02

Uh, ironically, I did all of my, all of my learning on four, and then the only work I ever did was 3.12.

Chris Romeo: 7:09

Okay.

Chris John Riley: 7:10

um, and almost the immediate project I got when I joined the, the, the company, uh, in London was, hey, can you migrate this to NT4? Uh, which when I say migrate, it was, hey, you have 35 accounts on here. There's not a lot of migration, but I can get this done in a day. Just, I'm going to set you up a new domain. So, yeah,

Chris Romeo: 7:29

Very cool. Very cool.

Robert Hurlbut: 7:33

So we want to dive in, Chris, on, on MVSP, uh, minimum, Minimum Viable Secure Product, if I remember correctly. Uh, but what's the difference between MVP and MVSP, uh, the difference and also what inspired the creation of MVSP?

Chris John Riley: 7:54

yeah, I mean, one of the things I wanted to make sure we call out here is this is an effort by a large group of people, right? So I tend to talk about it a lot. I work a lot on MVSP as part of my job at Google. But also that there's a lot of other people who've publicly put effort into this, um, Marat at Salesforce, um, you know, he, he's put a lot of effort into this and, and we started working on this in 2019 timeframe, right? So it's taken us a reasonable amount of time to even get to the point where there was an initial release. So, you know, agreeing with a large set of people. What a minimum is for AppSec is not an easy task. Let's put it that way. Uh, there, there is dissenting views and we should go in this direction. We should include these things. Um, but one of the core goals we had for the project since day one was it has to be minimum. It has to be realistic. We want to push the industry, but if you push the industry too far, too fast as a minimum. I mean, you're going to lose it, right? So how do you keep it relevant? How do you keep it in that sweet spot between this is too, too tough for a minimum, or this is ineffective as a minimum. And I don't know if we're succeeding in that. There's a lot of people saying this is too high. A lot of people saying this is too low. If we even them out, we seem somewhere in between those two. So I feel like we're, we're hitting that sweet spot somewhere in the middle. Um, and then the, the reason why It's called MVSP is really that we wanted to play on that MVP is that, you know, an MVP is often very much, you know, quick to market, you know, make this product, prove that it works, put it out there, get some customers start to sell this. And we wanted to be able to say, look, if you want to move into the enterprise business to business, large scale market, this is the S that you should have in your MVP, right? If you go out for MVP and you're selling it to end users, there's a certain amout of leeway that you can rely on for you know, people who are just coming into the industry, it's a new product. They're not expecting a hundred person team on your security team, right? They know what they're getting. But once you move into that enterprise arena, there's a certain amount of things that you should have in place if you want enterprises to start adopting this technology. And that's really where we wanted to say, this is the minimum set that we expect to be in place. And, and that was built from opinions on Google side, opinions on, um, uh, Salesforce's side, Okta, um, Slack, which were the, the original, um, team who released the product. And then there's a bunch of other people who've come in over the last couple of years to bring in additional viewpoints, um, and a lot of people from additional companies who, You will have heard of, but they can't put a logo on a website because someone says logos on websites are hard. So, you know, we appreciate everyone who provides input in any way. So, you know, all of that has been been really, really helpful over the years.

Chris Romeo: 10:49

Does MVSP apply if I'm in Enterprise then? Or is it really, cause I think, I mean I know MVPs exist in Enterprise, like Enterprise is using that. That terminology now, when you send a small group of people off to spin something up quickly and, and try to make it viable under the umbrella of the large enterprise. So from your perspective, is this, if I am a large enterprise, can I use MVSP for myself, or is it only for me to assess startups that I'm working with?

Chris John Riley: 11:20

I mean, when we, when we came up with MVSP, um, the different players in the game had very different opinions on where they wanted to use this. Um, you know, personally, from my experience working in the vendor security assessment process, um, I wanted to use this as a, Uh, RFP, prequal, questionnaire, to make sure that we were getting good quality things into our pipeline and weren't realizing that a product doesn't meet our minimum security guidelines four months down the road of an assessment, right, which, which does happen. Um, other people wanted to make sure that this was in their contractual language because they were getting a lot of pushback on contractual language from vendors who they deal with. And there were other people who were saying, look, I work a lot with startups personally. I work a lot. I have worked a lot with investments in new and upcoming companies who were looking at moving into enterprise realms and they often say, what do you expect from a security point of view to be in a product before you'll buy it, before you'll use it. And this is kind of the answer to that is, hey, if you want to sell a product to a large scale enterprise, this is what should be there. So it's very much a case of you can use it for whatever you want to use it for and whatever makes sense. We've seen people using it for acquisition work, for example, where they're looking at an acquisition or they're looking to buy and saying, Where are they on this scale? Do they meet all the minimums? Or are we going to have to put in half a million in the first six months? Right? Where are you on this journey? Um, and we've seen it at large scale companies. We're using it as an underpinning for our new vendor assessment processes to be able to improve those, speed them up, and make sure that things coming into our process are a lot better quality.

Chris Romeo: 13:00

So can you take us through the kind of the origin story of MVSP itself? I'd love to hear. Like, what was the, what was the moment where you decided let's do something here and then what did the next period of time look like as you were working up towards the first release?

Chris John Riley: 13:19

Sure. So, I mean, this came about from a conversation I had with Uh, Marat from Salesforce, um, in Tokyo. He's based in Tokyo, and we were out there for a conference. Um, I, I was working in the vendor security area. He was working in a similar area, so we'd had some contact previously and, and we, we sat down and had lunch one day and we're talking about some of the issues that we were facing. Um, he had this idea of defining what we expect as a minimum set of controls. We had something like that, because obviously we're working in that area, but it was very much, uh, we're working with a company to develop something, hey, here's what we expect from you, right? And we had extensive guidelines on those kind of things, but we hadn't necessarily put out a set of controls, or a list, or an opinionated view to the world to say, hey, this is what we expect from the industry as a whole, um, which is a very different thing. And then over the course of, Probably 12 months, I think, um, we started to talk to other companies who are interested, we started to draft out what we thought the initial list was going to look like, and that went through many, many iterations, um, and a lot of those controls, um, from our side, we wanted to try and identify the building blocks of what we see as a secure application, right? You can't necessarily measure everything at the RFP stage because you don't want someone filling out 180 page questionnaire on, you know, and then not winning the business. That doesn't seem fair. So you, you want to ask pointed questions that give you enough information about the security of a product, how it's going to fit into your environment, how they're going to react if there's any kind of vulnerabilities. But also it needs to be grounded in, in facts, right? If you're asking for controls that make no sense, why are you asking this? And a lot of the controls that are in MVSP now come down to areas where we saw companies consistently failing when they would come through some of our processes. You know, they'd get to four months in and they'd say, sorry, we don't support single sign on. And you're like, okay, this is like, fundamental for an enterprise to be able to have control over these kind of things. Or we'd say, great, can you make sure you make logs available in the event that there's an issue? And they're like, well, we don't log these things. It's too late now. You know, if there's an issue, you need to have identified the logs you require up front, because if you're trying to change your logging when an incident happens, you've already lost. So a lot of this stuff was very much the building block. And there's going to be more stuff on top of this, right? I mean, there's always a maturity model for, for application security that builds up and up and up, right? And all the way to, you know, are you implementing trusted types? Are you implementing these new technologies that are likely to break a lot of stuff, but are very, very useful in certain areas, but you can't expect that from a minimum. So, we started to figure out what those building blocks look like, figure out what the best coverage was, but also still maintain that minimum vibe, right? If you're asking for too much, you're losing people. If you're not asking for enough, it's not useful. So making sure we balance. And there's differing opinions, right? We solicited opinions from, I think we probably got opinions from like 30 different companies. Some were saying this definitely needs to be here. Others are like that same thing definitely shouldn't be here. So as an industry, we are an opinionated set of people and, and based on the experience that people have, it's like, this is a critical for me, but maybe it's not critical for you. So we tried to take controls in and out. And then we finally came up with, at the time, 24 controls. Um, since then we've kind of extended, extended out to 25, but we really want to keep it. Short and sweet. Yes, no, not applicable. We, you know, it should be easy to go through. And to answer the question of, you know, if you're an enterprise, can you use this for your products? Sure, but it should be a five minute exercise. It should be, sure, we do this and so much more, tick, tick, tick, tick, tick. If, uh, if there's a, if there's a big area where you're not doing those things, it's probably worth looking at the AppSec program and figuring out where there's a gap.

Chris Romeo: 17:15

hmm. Mm hmm. You mentioned that, uh, a number of the requirements came from what you were already doing at Google as kind of when you entered into this project. What percentage of those requirements and controls would you say carried forward into the, what, what an MVSP exists as today? Like, is it still primarily a foundational pieces that came from Google's best practices, or I know you said there's 30 different companies that have played in this, how is it, how has it morphed and blended over time?

Chris John Riley: 17:54

I mean, I'd say I think at the beginning we tried to bring in the top 10 findings that we had from from our processes, where you'd say, okay, this is a key gap here. Some of them had to be discarded because they're not application security, right? They're enterprise security issues, which is not where MVSP wants to be, right? If there's a question about, do you have EDR on your systems? That's enterprise security and not application security. So there's a lot of misconceptions there that MVSP misses all of these enterprise controls. And the answer is yes, it does. It's not designed to be an enterprise control checklist. Um, the minimum enterprise security product, maybe for the future. But, um, I think that originally we probably had less than, less than we have now. And we tried to build it up, but then we tried to figure out kind of where there's duplicates, where we needed to remove things. And more critically, where a control relied on another control to exist, in which case, we should ask for the lowest control there and say, does this exist? Because if it does, and you, as a company, you want that higher control, it's easier to build on a strong foundation than it is to... Try and underpin your, your, your building, right? If you build on the sand, your, your building is going to crumble, right? So our goal with MVSP is can you build that solid foundation and then if you want more from, from a product, it's easier to build that on top of a strong foundation and a good understanding of, of how things fit together. It, it also helps us to understand, I think, one of the controls that. We got a lot of pushback. We've, we've, we've, uh, discussed it with many different people, is this idea about having an annual third party penetration test. And a lot of people say, sure, this is normal, right? If you're PCI, DSS, you have a penetration test that's tightly scoped to. Two systems, depending on how you architect it, um, but you have a pen test, right? And in certain other areas, you know, in SOC compliance, there's, there's testing there. So it's not out of, outside the realms of possibility that you're already doing this, but for companies who don't necessarily need those kinds of controls, having an annual penetration test tells me so much, right? It tells me that you understand the value of manual testing and the patching and releasing products is not the end all and be all. It tells me that you know how to scope a test, so you know where your systems sit, you know where the data sits, you know where the, um, the issues in your system are. You value feedback on where you're doing things wrong, and you can handle remediation, patching, and correction of issues. All of that comes from one control. If you don't ask about penetration testing, then... Maybe you're patching, maybe you're not, maybe you're, maybe you know where your data is, maybe you don't, but some of these controls give you a lot of signals when you dig in and understand how they really fit together.

Chris Romeo: 20:42

Let me read back then the use case for MVSP and, and maybe you can elaborate a little bit more and provide a little bit more perspective here, so. As a company that's looking to purchase SAST solutions, or maybe even other software based solutions, MVSP then gives me a basic set of standards and controls that I can evaluate quickly, I can evaluate that provider of technology against this basic set of controls. and that can then be a, I guess a goal line of one of the marker lines in the procurement process. It may not be the end all be all. There may be more things, but at least it gives me a good understanding that there are basic security controls that exist in this product. So like your example, I'm not going to get four months in and go, but wait, you don't do single sign on. Hold on. This is going to be a problem for the enterprise because we have 27, 000 users and I'm not going to enter them all into a web form and hit submit 27, 000 times. So is that, what other context would you give on kind of the benefits behind it?

Chris John Riley: 22:00

I mean, that's one of the key benefits there is really being able to, the way we've described is top and tail. So, at the very beginning of the process, at the RFP stage, you want to know if a vendor is going to fail your follow up processes. Because the earlier you realize they're going to fail, the easier you can disregard them, or at least advise the business this is going to take you longer. Right? Because if this is the only player in the game, and they miss a bunch of controls, You know, maybe the business still needs to go with it, right? But it's an information source. And then at the other end of the process, when you're doing contractual controls, um, you don't want to give this to a lawyer and have them say, well, we can't do this, we can't do this, we can't, we can't provide logs, we can't contact you if there's a breach. And all of this sits in MVSP. So if you meet the requirements at the beginning, you're not gonna, you know, cross these out on the contractual controls at the end. So it streamlines both of, both of those processes. I mean, as a guideline, the reason why It was particularly interesting for us as it was seeing vendors come into a process where they would answer questions about the quality of their product and we would rate them, you know, based on like varying different controls and the risk. And if if a vendor came in and they were initially related as quite poor, um, it took. Like hundreds of days for them to get from poor to a very good status. And a lot of vendors were unable to get from poor to good, right? Which is like one jump up. In which case you're spending hundreds of days, thousands of engineering hours, delaying projects, and. Um, maybe possibly getting risk exceptions in certain cases for a vendor who is never going to be able to offer the product that you expect, you know, everyone wants to use products that rate as perfect across the board. It's not realistic, right? There's always going to be some missing thing here or there, but taking a product that doesn't meet even the minimum, um, and putting it into your process just eats up the resources. And this is the reason why you start to see, um, your vendor assessment programs, either they move towards compliance, they outsource everything, they, you know, up their risk bar, right? All of these things to try and deal with the growth of things that need to get reviewed. And one of those controls that we hope to put in place and have been putting into place is earlier detection of vendors who are not going to meet the The requirements. And at that point, if you can filter those out, you can spend 80 percent of your time on the 20 percent highest risk instead of 80 percent time on the 20% worst vendors or lowest rated vendors to try and bring them up to acceptable levels, um, which seems like it's a, it's a hamster wheel of pain. Effectively. You're constantly trying to remediate minor things that should never really exist. So I'd rather spend that time on the, the most risky things.

Chris Romeo: 24:47

Yeah, that makes sense. And, and, you know, you mentioned contractual requirements. I hadn't thought about that. This, these controls going from the assessment phase and then becoming part of the contract. And I mean, it's, it doesn't surprise me that you have some startups that can't get from poor to good. Because they're just creating things or trying to move fast. I guess that's one of my unfair advantages is when I start companies, they're, I'm a security engineer at heart. And so like, what do you mean we're not doing all these things? Of course we're doing these things. This is just how we build products from the ground up, but I realize not everybody has that perspective when they're building products in this day and age.

Chris John Riley: 25:28

I mean, one of the, one of the things that I, MVPs that prove the point are perfect, right? You're going to market, you're proving that it's viable, you're proving that there's a market for this. Great. Then if you keep that MVP and continue to try and tweak it and move it, was it architected from day one to be scalable, to be secure, to meet all those contractual requirements? And do you keep logs? You know, things like this, going back and attaching all of these things on later. Sometimes it works, sometimes it doesn't. Sometimes what you end up with at the end of the day is a, you know, some kind of mutated thing with 15 different modules, each trying to do one thing, but failing at it. So even if you look at MVSP and say, For our MVP right now, that's not where we are. We're not going in at the enterprise level. We're going in a, you know, small Soho mom and pop shops trying to prove that there's an industry. But there's three things here that we should consider for the architecture and design of our product so that we can move there in the future. You don't have to be there on day one, but if you design your product, so it's never going to be there. That's, that's a problem. So you can use this as a control framework to say we don't need to do this now, but we need to know how we can do this in the future when we have the resources, the time and the money.

Chris Romeo: 26:43

Yeah, that's, that's a good, a good use case right there. And, and for folks that are building startups from scratch right now, they should at least be aware of the framework and the controls available. To your point, even if they don't implement them all from the very beginning, there are things you can be doing things that you should not be doing that will save you much pain in the end. So, um, I think we've been, we've been talking about this for a long time. We should probably. I'm going to and, and do a little demonstration of it for our listeners, or I guess for our watchers, those watching on YouTube, the listeners, we won't, uh, be able to, perhaps they'll have to follow along, um, and the site is mvsp.dev, just in case you're listening and you want to stop and follow along. But, um, Chris, why don't you just take us through, um, Take us, just give us a high level tour of this. Like I'm, I'm curious to get kind of your take on some of these things and just give me a heads up if you want me to scroll down further or whatever.

Chris John Riley: 27:42

Yeah. So we try and break out the controls into four different sections and feel free to scroll down. So starting with business controls, um, we're talking very much about, you know, The controls that your business has around your product, right? So MVSP is very product centric. So it's, you know, how do you report bugs in these kind of products? How does customers come in and test your products if they, if they want to take a look at how your product interacts and functions? Um, we obviously reference self assessment because we want people to. Do the MVSP checklist on an annual basis. We understand things change during the year, right? You know, a year is a long time in technology. Um, and then we cover some of the high level stuff on, you know, what we were talking about before, which is external testing, penetration testing of your environment on an annual basis. Make sure you're doing the training, the compliance, and the incident handling that You expect and your customer expects to be in place. Um, so one of the things I want to call out as we call out compliance here and some of the feedback that we've got quite recently is that, well, if you're doing these compliance, why do you need this checklist? The answer is, if you have these compliances, you probably don't need this checklist, right? If you comply with SOC 2, you probably have all of this where there are some differences here is that. ISO 27001, um, you know, SSAE 18 and a few other standards. There's a few examples here, but they don't specifically look at products, right? So ISO 27001 is a company wide certification that you're following these guidelines. Does that mean that every single product in your portfolio meets the exact same set of controls and has no deviations? Probably not, right? A product that you've just created six months ago is likely not as mature as a product you've been running for 10 years. So, MVSP tries to deal with things on a product level. You know, if you have 10 products, you should have 10 MVSP reviews of those products to see where there are deviations in the products. Um, so, a lot of this stuff is. Different, taking a different view of, of compliance and saying, well, this is more about how you secure your product, less about how you secure your company and how you get compliance for your company. Well, we can't ignore the fact that if you are handling credit card details, you need some kind of PCI reviews and PCI compliance. And this you outsource it, obviously. Looking at the second section, we look at application design controls. That's very much where a lot of the meat is on the technical controls. Things like making sure you support single sign on, HTTPS only. These things, you know, should be simple and out of the box. Um, you know, make sure you have some security headers in place. Particularly, we're talking about content security policy, which is another one of these areas where... If you design it in from the, from the day one, this is easy, right? If you, if you do CSP from day one, it's easy. If you try and do CSP on a five year old application, where you've suddenly realized you want to sell it to the enterprise, it's going to be a horror show because CSP is something you need to architect for, as opposed to just slap on there, right? And then we start to talk, we try and align to the password policies in NIST 800. 53 because it doesn't make sense for us to reinvent the wheel here. So if you do have a requirement to have, um, single sign on and local accounts, which does happen, right? Um, then, you know, making sure that you're doing that in a secure way. Things like the things that spring to mind are. Insurance companies, health companies, things like that. Stuff where if, um, if I have a health insurance policy through my company, if I leave my company, I still need access to that portal and I can't do single sign on anymore. So there are occasions where you're going to need to support both of these things. It does happen. And then we talk about security libraries, making sure that. Don't reinvent the wheel, right? Why, you know, having individual libraries for every single application in separate ones doesn't make sense. Like solidify on a single library, make sure you trust that library, your engineers know how to implement that library, and you're less likely to have bugs in your, in your products. Um, and then obviously we talk about patching, logging, and encryption, right? Encryption, um, both in, you know, in transit and at rest, right? Which all seems... Relatively straightforward, right? There's nothing here that jumps out and is like, Oh my God, this is like five years work for an application. This is, this is all pretty, pretty basic. Uh, for application implementation controls, um, we talk a little bit about, um, listing data and data flow, um, which sounds very complicated, but realistically for an application, just understand where your data is. Right. You know, if your data goes through a load balancer that offloads SSL, and then it goes to a reverse proxy, and then it goes to a database server in the back end, sure, you could diagram that out on a, on a napkin and understand where your data flows, where your data sits. But for more complex environments, this is critical to understand where your data is sitting. Um, so just having a diagram where you understand kind of where the data flows and sits and what kind of data is in your. in your application, uh, is, is really, really useful. Um, we talk about training, uh, developers for vulnerability prevention, and we, we have a number of categories here, but obviously dependent on what your application does, how it functions, there's going to be different training depending on what your developers need and don't need, right? If your developers are specifically working with SOAP or things like that, then spending time, you know, teaching them how to do that in a secure way is much more effective than teaching them this bog standard list of things that they may not find useful in their day to day. And then we talk about time to fix and 3. 5 control, which is build processes, is the one that we brought in last year during review, where we say, um, we're aligning to SLSA. I don't know if you're familiar with the slsa.dev project, which is something that people at Google are working on, which is very much around automating building software with provenance, so you can trust that that software hasn't been altered, um, or, um, the supply chain hasn't been affected. So we start to try and align to level one, which is very, very low requirements as an introduction to, Hey, you should look at SLSA for, for various areas.'cause I think this is useful beyond MVSP, but obviously we don't wanna push to towards higher levels. And then for operational controls, we talk about some, some of the slightly broader areas that start to touch on enterprise security. And we are, uh, very aware of. How far in that direction you want to go, but for application security, you want to talk about physical access, who has physical access to to the key systems that are providing these services. You want to talk about logical access, who has access to this customer data? You know, can they just VPN in with a username and password? Um, and you download all of the data from your systems. Um, and then we start to talk about subprocesses, backup and disaster recovery, right? So understanding where your data sits, understanding who has access to your data, um, and making sure that you can recover from a disaster and there's been some recent examples, I believe, of people who back up to a different region and then wonder why a hacker has deleted both their production environment and their backup in a different region. So, you know, making sure you're backing up to different locations and actually have a good quality disaster recovery program that's tested is very important for an application. And that's very much it. It's 25 controls. It doesn't, uh, doesn't go beyond that. It doesn't need to go beyond that. And if it, if it did, then I don't think it would meet the purposes that it's designed to meet. So,

Chris Romeo: 35:10

Yeah, I don't see anything that's, that I'm looking at this and going, that's just unreasonable. Like everything here is, is, and that's, that's something that I've been investigating and thinking about a lot is this concept of reasonable application security. Because there are a lot of things we require of people in, in, for those of us that have been in insecurity for a long time, we require a lot of unreasonable things and expectations. But I'm looking at this and I'm going, this, this seems reasonable. Like, I don't see anything where I'm like, if I was on the startup or small company side and you told me, you sent me this list of requirements, there's really nothing here. You know, because you're not telling me that I have to comply with particular standards, for example. It's the relevant standards for your business needs. And like we know, you know, from PCI DSS, if I use an outsourced credit card processor, I don't have to do anything with PCI DSS. I outsource that responsibility, right? And so, um, It's, I've got that coverage there. So you're not, you're not saying I have to be SOC 2, for example. You're saying the things that are relevant to our business. Incident handling, things like this, um, you know, we want to be able to deal with an incident and it shouldn't be chaos, right? In

Chris John Riley: 36:23

well, I mean, it will always be chaos, but, uh, it was controlled chaos, I think is the, is the best plan for, for most people.

Chris Romeo: 36:29

Yeah, definitely. And I worked incident response early in my career. And, uh, yes. Controlled chaos was a, a very good way to think about what was happening in the process and who was yelling at me at any given moment.

Chris John Riley: 36:44

I think one of the things that's important to call out is kind of how MVSP differs from some of the other areas is, is that. We're not specifically saying you need to check your application sets, this individual control in the Java configuration in order to be secure, right? MVSP is designed to be broadly applicable to a large range of systems, environments, and architectures. Where there are things that people call out and say you're missing this, this needs to be in place. It's often very specific architectures and and designs that people have in mind where it's like, but that's oddly specific. It's 1 percent less than 1 percent of what we're looking at here. And we try and be as broadly applicable to the, to the industry as we can. Otherwise people start to look at it and say, well, we don't use Java. So I don't care about these five things. And, you know, we, we don't, we don't have databases. So we don't care about these three things, right? At that point, you start to lose people because a lot of the controls don't really line up to what they are looking for, right? So we try and be broadly applicable.

Chris Romeo: 37:47

Okay. So you might have to, you might have to do some noodling on this one, but I'm going to ask it anyway. Let's imagine a, let's just a startup. Okay. We've got a startup. They're coming in cold to this list. And so they've done nothing. I know that's probably not reasonable. There's probably some, most people probably done something on the list, but let's just imagine for, for a thought experiment, they've done nothing. They're coming in completely cold. And based on all what you know about this set of controls and requirements and whatnot, how much effort, you can give me an order of magnitude if you need to, how much effort are they going to have to put in to get to that? Let's just say that first notch you said it was a good was the first kind of notch.

Chris John Riley: 38:35

Poor.

Chris Romeo: 38:36

Poor. Okay, so poor. To get past poor. Let's say to get from above poor, so they're not in that category. What is a rough order of magnitude based on what you know about the controls for what someone would have to put into it?

Chris John Riley: 38:48

I mean, it's very difficult to answer. I mean, you're right. I have to noodle on this one because it's very application specific. It's very environment specific. It's very industry specific. I mean, for example, if you come in and you don't have any of these controls and you process credit cards, you're 12 months away. from having a product that is even compliant with the minimum set of things, right? So, uh, if you can engineer away some of those issues by saying we don't need to be PCI compliant because we'll, we'll use a third party provider, we'll, we'll, then you start to look at things like how do you iframe them in and things like that, but moving beyond that, I think a majority of these things are second nature for someone who's rolling out an application. I very much doubt anyone could walk in and say we don't meet any of these things. Right. I think, I think off the bat, off the bat, people could, could roll in and say, we probably meet 50 percent of these things purely by having an understanding of how our application works and an understanding about whether or not we need to meet GDPR compliance or we do credit card processing.

Chris Romeo: 39:48

I mean, and physical security. If you're using a cloud provider, you're already ticking all those boxes. Like,

Chris John Riley: 39:54

Right. And, and it's often, often a misconsideration where people look at it and they go, well, we don't have our servers, so we're failing that. Like, no, if you don't have servers, then your cloud provider is providing this for you, right? This is, this is reasonable. I'm A lot of. A lot of startups now, when they come on board and they're producing their MVP, the answer is they don't have an environment. They don't have a physical environment. They have five people with laptops located at various ends of the country. And do you have secure authentication on access to production data? Yes, because it's all in the cloud and it's all single sign on and it's all 2FA. And, you know, if you don't have 2FA turned on, that's not going to be a lot of effort for five people. Right. You log in, you turn it on. Great done. If you've architected your application specifically to be, um, the old school of, of web, where it's like username and password, you can't log on with anything else, um, there's a database in the background and, you know, it uses HTTP as the initial login point before it goes to HTTP, you know, all of those bad, It's 1999 type design aspects, then it's going to take you a little while to get to these things. But the way I look at this is, and this is another way that MVSP is being used in certain areas is, um, mostly with acquisitions, is that this is like the hit list of things you need to address, right? You can't address all of these issues at once. You need to systemically go through and say, okay, we don't meet five of these controls. This is our roadmap. in six months we'll meet all of those controls, right? And this is also a good way if you're in development, you're in security, you're at a small startup and someone says, how do we get to enterprise grade? It's like, well, here's a, here's a top 20 list of things we need to do, right? Do we have all of these? Cause if we don't, these three things should be on our objectives for this quarter. Right. Can we get these things done? Right. And I think that spending time on these things, as opposed to spending time on trusted types and other much more complex controls that might break a lot of the applications is likely a lot more worthwhile. Um, obviously securing your enterprise is also worthwhile, but we're talking about AppSec, so let's skip over that one for now.

Chris Romeo: 42:05

So where do you see MVSP going in the future? Like is, I mean, you mentioned that the SLSA control was added in for Build Providence. Makes sense. You want to be able to trace software in the supply chain and have providence for that. What do you think's gonna happen in the future? You gonna add a bunch of additional controls? Are things ever gonna come off the list? What do you, what do you see?

Chris John Riley: 42:26

So MVSP is designed to be Evolutionary, not revolutionary, right? I think is the term we throw around in the working group is that if someone comes in with a revolutionary idea about an amazing control, it's likely not suitable for MVSP, right? Every year we do a refresh of controls, we talk to a variety of people in the industry, we talk to a variety of people in the working group, and we agree on what changes we need to make, and we should make changes on a regular basis. You know, things become less important. Sometimes things become more important. Um, I mean, specifically if, if modern browsers start to no longer be vulnerable to some of these issues, then why do we insist that this is a control for minimums, right? You know, suddenly this becomes less of an issue. Obviously, people use old browsers all the time, but once you hit a certain point, you can say, well, this doesn't need to be. Be a control anymore. And I think the idea is not to grow the controls. The idea is to maintain the controls, make sure it's clear about what we're expecting to meet these controls. Where we can push the industry, we do so in small ways. Like, you know, we, we started with third party penetration testing to say, hey, this is important, right? Don't just trust your internal services and, you know, the developers have someone else take a look at it with fresh eyes, not because anyone's wrong, but because sometimes fresh eyes and more experienced eyes find things. And that's, that's really the way it is. And we started to push a little bit on build environments, but I think. This year is very much about clarity and making sure that we're clear about what's expected here. And that's, it's been useful. We've had a number of feedback from a variety of people about where there's not as much clarity as we'd want. This is written by engineers for engineers, right? You know, no, no one said we need the ISO editor to look at these controls and come up with a 350 page report and put it behind a paywall, right? This is designed to be free. It's designed to be accessible and it's designed by engineers and as such engineers will... yeah, we'll spell things wrong occasionally, right? But we'll do our best to make sure that it makes sense to people. Um, it's, it's going to slowly move upwards, but there again, every minimum should slowly move upwards. Otherwise it's, it becomes dated after a period of time.

Chris Romeo: 44:45

All right. Well, Robert, take us into our lightning round here, if you would with

Robert Hurlbut: 44:50

Sure. So Chris, uh, we have three questions that we usually ask. Uh, first question is what's your most controversial opinion on application security and why do you hold that view?

Chris John Riley: 45:06

I don't know if my views are controversial. Um, I mean, I guess I would go back to the penetration testing, right? I think that no matter how big you are, no matter how secure you think you are, and not having regular testing of your environment, Not having a, uh, a vulnerability rewards program or a bug bounty program, um, is a mistake, right? Every single company has an issue somewhere, right? If you don't have one now, you may have one in a week, right? Things, things change very fast. Um, but I don't think that's particularly controversial. Um, I guess the closest I get to controversial on that is I wouldn't buy a piece of software if you'd never done a pen test on it. Simple as that. If you've never tested it, as far as I'm concerned, it's, it's just unproven software. And it shouldn't go anywhere near sensitive data in any way.

Robert Hurlbut: 45:58

Makes sense. Uh, second question is, uh, what would it say if you could display a single message on a billboard at the RSA or Black Hat conference?

Chris John Riley: 46:10

Um, translation failed. I don't know. Um, so I'm thinking about the Chinese, the Beijing Olympics. Um, but, uh, don't know. I guess it would probably say, uh, none of these products are going to solve your problems. Uh, maybe that's the controversial opinion.

Chris Romeo: 46:29

Way to bring both of them together there. Controversial opinion.

Chris John Riley: 46:32

let's see if I can bring the third one in and it's the same thing. But I, I honestly think that if, if you have a budget and you're spending it on something with blinky lights in the hope that it's going to solve all your problems, you'd have two problems.

Robert Hurlbut: 46:48

Excellent. And then a third question is, uh, what's your top book recommendation and why do you find it valuable?

Chris John Riley: 46:56

So I had a couple, actually, I was thinking about this a little bit before the call. Um, the first one I, I, I have is, um, Eugene Spafford. Um, did a, did a great book, uh, Cybersecurity Myths and Misconceptions, um, which talks a lot about, you know, he did a presentation at the first conference, uh, earlier in the year, and we had a chance to chat on, on the first podcast, uh, plug for the first podcast there, um, but, uh, we, uh, we went in depth on how the security industry tries to describe things by using analogies. But people take different things from those analogies, and I think that that's a very interesting take when we talk to people about certain things and describe them in a way that certain cultures and areas of the world are just not going to understand them in the same way that we do. So if you're interested in that area of things, then take a look. And my other recommendation was, um, Phantoms in the Brain, which is a book about brain injuries, but it's very interesting about what the body does and weird brain stuff. I'm kind of... I'm not a fan of those kind of weird books, but, uh, highly recommend.

Chris Romeo: 48:03

Very cool. Well, Chris, give us a key takeaway or maybe a call to actions or something you want the audience to do as a result of our conversation today.

Chris John Riley: 48:14

I guess if there was one thing I'd ask the people to do, um, you know, is, you know, use your RFP processes to filter out vendors who don't meet your requirements. And don't be afraid to turn around to those vendors and say we didn't proceed with you because you don't meet security requirements. I think as an industry, we have a tendency to focus a lot on business requirements. and ignore what security needs to do to make it work. And I think that, uh, being more opinionated about what we expect to be in place is the only way that we can bring up that minimum level of the industry. You know, rising tide lifts all boats, as they say, but effectively if we don't If we don't try and raise that tide, we're always going to be at a deficit. So, you know, that and spay and neuter your pets. Isn't that the thing?

Chris Romeo: 49:08

Uh, yes. It is. All right. Well, Chris, thank you for joining us to share all things about the MVSP. And thank you to all the team members out there who participated in this that are behind the scenes. We know that a project like this does take a lot of effort from a lot of different people, and it's a very valuable set of controls. And I'd encourage anybody that's building a company, building new products, you should be looking at this set of controls as your baseline. And you can, you can grow into bigger requirements in the future, but this is an excellent baseline to get started and be able to say, when someone asks, what is your approach to security with your products? Well, we use MVSP and that's our baseline set of controls. You're going to be in good shape. You're going to be at the top 10 percent in the new startup world, if that's your answer right off the bat. So, uh, Chris, thanks for being here.

Chris John Riley: 50:04

A problem. And, uh, if you like MVSP, let us know. And if you hate it, let us know, right? Any feedback's great. And thanks very much.