Dr. Nicole Forsgren does research and strategy at Google Cloud following the acquisition of her startup DevOps Research and Assessment (DORA) by Google. She is co-author of the book Accelerate: The Science of Lean Software and DevOps, and is best known for her work measuring the technology process and as the lead investigator on the largest DevOps studies to date. She has been an entrepreneur, professor, sysadmin, and performance engineer. Nicole’s work has been published in several peer-reviewed journals. Nicole earned her PhD in Management Information Systems from the University of Arizona, and is a Research Affiliate at Clemson University and Florida International University.

Mike Julian: This is The Real World DevOps Podcast, and I'm your host Mike Julian. I'm setting out to meet the most interesting people doing awesome work in the world of DevOps. From the creators of your favorite tools to the organizers of amazing conferences, and the authors of great books to fantastic public speakers, I want to introduce you to the most interesting people I can find.

Mike Julian: Ah, crash reporting. The oft-forgotten about piece of a solid monitoring strategy. Do you struggle to replicate bugs, or elusive performance issues you're hearing about from your users? You should check out Raygun. Whether you're responsible for web or mobile applications, Raygun makes it pretty easy to find and diagnose problems in minutes instead of what you usually do, which if you're anything like me, is ask the nearest person, "Hey, is the app slow for you?" And getting a blank stare back because hey, this is Starbucks, and who's the weird guy asking questions about mobile app performance? Anyways, Raygun, my personal thanks to them for helping to make this podcast possible. You can check out their free trial today by going to raygun.com.

Mike Julian: Hi folks. I'm Mike Julian, your host for the Real World DevOps Podcast. My guest this week is Dr. Nicole Forsgren. You may know her as the author of the book Accelerate: The Science of Lean Software and DevOps or perhaps as a researcher behind the annual State of DevOps report. Of course that's not all. She's also the founder of the DevOps Research and Assessment, recently acquired by Google, was a Professor of Management Information Systems and Accounting, and has also been a performance engineer and sysadmin. To say I'm excited to talk to you is probably an understatement here. So, welcome to the show.

Nicole Forsgren: Thank you. It's a pleasure to be here. I'm so glad we finally connected. How long have we been trying to do this?

Mike Julian: Months. I think I reached out to you, it's March now. I reached out in November, and you're like, "Well, you know, I have all this other stuff going on, and by the way, my company was acquired."

Nicole Forsgren: Well, back then, I had to be sly, right? I had to be like, "I've got this real big project. I'm sorry. Can we meet later?" And, God bless, you were very gracious and kind, and you said, "Sure-"

Mike Julian: Well thank you.

Nicole Forsgren: ... "we can chat later." And then I think you actually sent me a message after saying, "Oh, congrats on your 'big project'." I said, "Thank you."

Mike Julian: That sounds about right.

Nicole Forsgren: I appreciate it. Yeah. And then, you reached out again, and I said, "Oh, I'm actually working on another big project. But, this time ..."

Mike Julian: It's not an acquisition.

Nicole Forsgren: Yeah, it's not an acquisition. This time, it's a normal big project, and it's this year's State of DevOps report. And we just launched the survey, so I'm super excited we're collecting data again.

Mike Julian: So we can get that right out of the way, where can you find the State of DevOps report?

Nicole Forsgren: All of the State of DevOps reports are hosted at DORA's site. We still have the site up. And all of the reports that we've been involved in from, I want say we started in 2014, I'm so old I already forgot. All the reports that we've done are hosted. We'll post them in the show notes. If you can grab yourself a Diet Coke or coffee or a tea or a water, or if you want a bourbon. Get comfortable. Sit back, takes about 25 minutes. I know, right, everyone's like, "Girl, 25 minutes?"

Mike Julian: That's a big survey.

Nicole Forsgren: I know. It is. But it's because the State of DevOps report is scientific, right? We study prediction, and not just correlation. But sit back, get comfy and let me know what it's like to do your work. Because we're digging into some additional things this year; productivity, tool chains, additional things around burnout and happiness, and how we can get into flow, and really what that looks like. And some really great things are a bunch of people have already chimed in after taking the survey in really thoughtful ways. Also, by the way, I love you all for taking it if you have. Share it with your colleagues, share it with your peers.

But they've said that just by taking the survey, they've already come away, even before the report has come out, they've already walked away with really interesting ideas and tips and insights about how they can make their work better.

Mike Julian: Yeah, that's wild to think about, that the act of taking a survey actually improves my work. Because most surveys I take, I'm finished, I'm like, "Well, that was kind of a waste of time." It feels like I just gave away a bunch of stuff without getting anything.

Nicole Forsgren: Yeah, and I think the reason it works that way is because we're so careful about the way we write questions that sometimes just the act of taking the survey helps you think about the way you do your work. So just the act of kind of taking some of these questions helps people think about what they're doing. And then, of course, like I joked already, it's my circle of life, the survey will be open until May 3rd and then I will go into data analysis and report writing. And we expect the report itself to come out about mid-August.

Mike Julian: Well, why don't we take a few steps back and say ... Everyone loves a good origin story. I believe you and I met at a LISA many, many years ago. You were giving a joint workshop with Carolyn Rowland on-

Nicole Forsgren: Oh, I love Carolyn.

Mike Julian: Yes, she's also wonderful. I should have her on here.

Nicole Forsgren: My twin. Yes. Absolutely.

Mike Julian: So you were a professor then when I first met you. I'm like, you know that's kind of interesting that a professor's hanging out at a LISA and giving all this great advice on how to understand business value, which I thought was absolutely fascinating. Professor, hanging out in the DevOps world, how'd that happen?

Nicole Forsgren: Oh my gosh. Okay so, the interesting thing is, I actually started in industry. My very first job was on a main frame, writing medical systems, and then writing finance systems. So I was a mainframe programmer. And then supported my main frame systems, right? Which is how so many of us in Ops got our start in Ops was someone was like, "Well somebody's gotta run this nonsense." Right? I was still in school, and then I ended up as a DEV, right? I was a Software Engineer at IBM for several years, and then pivoted into academia. Went and got a PhD, where I started asking questions about how to analyze systems, so I was actually doing NLP, natural language processing.

Mike Julian: Interesting.

Nicole Forsgren: Yeah, I was doing…

Mike Julian: Yeah, that's a weird entry point into that. Definitely not what I would have expected.

Nicole Forsgren: Yeah, so the crazy thing, my first year was actually deception detection.

Mike Julian: I bet that's awesome.

Nicole Forsgren: It was really interesting, it was super fun. But I leveraged so much of my background from systems work, right? Because what do we do? We analyze log systems.

Mike Julian: Right.

Nicole Forsgren: Right? We're so used to analyzing a ton of data in a messy format, many times text based, super noisy, can't always trust it, right? Right now people are like, "I can't trust surveys. People lie." Kids, so do our systems.

Mike Julian: All the time.

Nicole Forsgren: Right? And so, they loved me for a bunch of this work. All of a sudden, I randomly did a usability study with sysadmins. We wrote up the results, gave them back to IBM, and IBM was like, "Well what do you mean? We followed UCD guidelines, user center design guidelines. This should be applicable." And I was like, "Wait, whoa whoa whoa whoa, what?"

At the time, they had one set of UCD guidelines, for all users. Super super advanced, high level advanced, sysadmins, who were doing back-up, disaster recovery, everything. And people who had bought a laptop and were using email for the first time in their lives.

Mike Julian: I'm sure that went over super well.

Nicole Forsgren: What? I'm like, "That's it. Changing my dissertation." Which of course, panicked my advisors. They were like, "You're gonna what?" So I start doing what, at the time, was kind of the groundwork for DevOps. Which is, how do you understand and predict information systems? And by information system, technology, automation, usage and prediction and then outcomes and impacts of the team, individual team at an organizational level.

Which now, I say all that, that's big words, that's academic words, for basically what's DevOps. How I do I understand when people use automation and process and tooling and culture, and how do I know that it rolls up to make a difference and add value? Which now we're like, "Oh that's DevOps."

This is late 2007.

Mike Julian: Oh wow. So you were early days with us.

Nicole Forsgren: Yeah. It was a really interesting parallel track, because now we look back and we're like, oh this is about 10 years ago. That was kind of the nascent origins about the same time as DevOps, right? So, so many of us kind of stumbled into it about the same time. I had no idea this was happening in industry. I kept plugging away, I kept doing it, stumbled into LISA, trying to connect data, of course, like every good academic does. Desperately trying to find data.

Stumbled into, bumped into a group collecting similar things but using different rough methods. A team from a cute little configuration management startup called Puppet, right? Started working with them, invited myself onto the project. God bless them, I have so much love and respect for them because they basically let this random, random academic tear apart their study and redo it and lovingly tell these two dudes I had never met before, on the phone, named Jean and Jez, that they were doing everything wrong and that this word they were using wasn't the right word. Redid, in late 2013, the State of DevOps report, made it academically rigorous, and then that, kept going for several years, right? And then suddenly, we redid a bunch of stuff after a couple years.

I left academia, walked away from what was about to be tenure, to go to another cute little configuration management startup called Chef, that was fun, right? So I'm working on the report with Puppet, and working for Chef, and continuing to do research and work with organizations and companies. And I left academia in part, because I was seeing this crazy DevOps thing make a difference. But in academia, they weren't quite getting it yet. And I wanted to make sure I could make a bigger difference, because I'd started working at tech in college in 98, 99, 2000; we lift this crazy dot com bust.

And it wasn't a bust because everything crashed and the world ended like people thought but companies failed, it had huge implications and impacts for what happens to people. They lose their jobs, it breaks apart families, they get depressed, it impacts their lives, some people were committing suicide. And I was so worried about what happens when we hit this wave again and we're starting to see that hit again. So what happens if companies and organizations don't understand smart ways to make technology, because you can't just keep throwing people at the problem, or throwing the same people at the problem. And when I say throwing the same people I mean, seven day forced marches.

I was at IBM when they made us do that, right? They got pulled into a class action lawsuit, you can't do that. That's not a way to live.

Mike Julian: Yeah, I've been on many of those, they're brutal. And they don't result in anything useful.

Nicole Forsgren: It's just broken hearts and broken lives, right? And so, some people like say, you really care about this. I'm just this nerd academic who just cares too much about what I do. And so if we really can, fundamentally change the way that people make software, because if it will in fact, actually, fundamentally make their lives better ... let's do it.

And then, thank God, what we found is that it really does. Sure, it's nice that it delivers value to the business but that matters because then, what it does, is it helps them make smarter investments because then in turn, it reduces burnout. It makes people happier, it makes their lives better, and I think that's the part that's important.

Mike Julian: So what you've been finding is that by a company implementing all these better practices of continuous deployment, and faster time to delivery, faster time to value ... it makes the lives of the people doing the work better?

Nicole Forsgren: Yeah and John Shook has found this as well. Right? He did this great work in Lean, in that in order to change ... some people have said like, "How do you change culture?" Let's find ways to change culture. Sometimes the best ways to change culture is to change the way you do your work and I'm sure we've seen that ourselves, right? In other aspects of our lives. To change the way we feel, to change the way our family works, to change the way our relationships work. You actually physically change your lived experience, or some aspect of your lived experience.

And so if we change the way that we make our software, we will change the way that our teams function, which is changing the way that the culture is. And so, said another way, if we can tell our organizations which smart investments to make in technology and process, then we can also improve the culture. We can also change the lives of the people, right? And the Microsoft Bing team found this, right? They wanted to make smart investments in continuous delivery.

And in one year, they saw work life scores go from, I'm pulling this off the top of my head, but I want to say it went from 38% to 75%. That's huge.

Mike Julian: That's an incredible jump.

Nicole Forsgren: Right. And it's because people are able to leave work at work and then go home. You can go see your families, you can go to a movie, you can go eat, you can have hobbies, or you can go binge watch Grey's Anatomy. You can do what you want.

Mike Julian: That's one of the most incredible things to me is that there's this idea of in order for a company to be successful they have to push their employees, kind of put them through the ringer. Intuitively, that's never felt right. And you actually have data that shows that's not right. Doing these things, actually makes everyone better. The business improves dramatically, the people's lives improve dramatically, and everything's awesome.

Nicole Forsgren: Right and if we want to push people, that's not sustainable. And if anything, we want to push people to do things that they're good at and we want to leverage automation for things that automation is good at. So what does that mean?

We want to have people doing creative, innovative, novel things. Let's have people solve problems, let's have automation do things that we need consistency for, reliability for, repeatability for, autotability for. Let's not have people bang a hammer and do manual testing constantly. Let's have people figure out how to solve a problem, do it once or twice to make sure that's the right thing, automate it, delegate that to the automation and the machines and the tooling, hand it off, be done, and then pull people back into the loop into the cycle, to figure out something new.

I think it was Jesse Purcell that said, "I want to automate myself out of a job constantly." Right? Automate yourself out of your current job, and then find a new job to automate yourself out of again. We will never be out of work.

Mike Julian: Yeah, I used to worry about that when I first started getting into DevOps and actually, when I first started working on automation it wasn't DevOps at the time, it was automating Windows desktop deployments at a University. And this is in the early 2000s. And one of my big worries was, well because I spend half my week doing this, if I were to automate it I'd spend an hour doing this, what am I gonna do the rest of the time? They're just gonna fire me 'cause they don't need me anymore.

As it turns out, no, that's not what happened at all. Higher value of work became work because I wasn't focused so much on the toil.

Nicole Forsgren: Right, and those types of things, machines and computers can't do. And the other thing, I used to tell all my friends, don't think about that in terms of job security, right? Don't try to paint yourself into a thing that no one else can ever do because then you can't be replaced, because that also means that you can never get promoted.

If we always make sure that there are aspects of our job that can be automated so that there are opportunities for us to pick up new work, that only creates more opportunities for amazing things. There are always going to be problems, there are always problems for us to solve. I don't want to be stuck doing boring work.

Mike Julian: Yeah, God knows that's the truth.

Nicole Forsgren: Oh my gosh I know. I don't want to be stuck doing boring, repetitive work. That's just a headache. If we can find, especially really challenging, complex things, and if we can find ways to automate that, trust me, we will never dig ourselves to the bottom of that hole. That is always there.

Mike Julian: So I want to talk about the State of DevOps report and I want to start off by asking a question about something you mentioned earlier. You mentioned this phrase, academic rigor. What is that, what does that mean?

Nicole Forsgren: Academic rigor includes a few things, okay? So one part of academic rigor is research design. So it's not just yoloing a bunch of questions ... sorry, yolo is my shorthand for like, "Your methodology is questionable."

Mike Julian: I've been seeing a lot of those surveys come out recently.

Nicole Forsgren: Yeah. So one is research design. And some people say, "Nicole, what do you mean by research design?" So research design is, are the types of questions you're asking appropriately matched to the method that you're using to collect the data? Right? Are these things matched? And for some things, a survey is appropriate. A one time, so one time is cross-sectional, one slice in time survey across a whole industry. Some things this is appropriate for. Some things this is not appropriate for.

One good example, a whole bunch of people really want me to do open spaces, questions, in State of DevOps report.

Mike Julian: What does that mean? Like open ended questions?

Nicole Forsgren: No, open spaces. So a lot of people have a lot of feels about open office spaces. Should I work in an open office space? Is open office space influence productivity? Or pair-programming ... does pair-programming affect productivity? Does pair-programming affect quality? People have a lot of feels about these things. The type of research design, employed in the State of DevOps report, is a survey that is deployed completely anonymously, across the entire industry at a single point in time, is not the appropriate research design to answer either of those questions.

Mike Julian: Why is that?

Nicole Forsgren: Because what you would need to do is have a much more controlled research design. So I would need to know, for example, who you were working with. I would need to know, so let's go with the peer review one, I would need to know the types of problems you're working on, the types of code problems, I would need to now the complexity of the problems, I would need to know how long it's taking you, right? If you're wanting to now productivity, right? 'Cause I would need to know a measure of productivity. I would need to know what the outcome is. So if my outcome's productivity, I would need to measure productivity, because I'm gonna need to control for perplexity, right? Because things that are more complex, we expect to take longer. Things that are less complex, I expect to take not as long, right?

And then I would need to match and control. Right? So even things like open office spaces, right? Because if you're doing peer programming in an open office space versus not an open office space, if you're doing it at an office, I would need to know seniority of the person, or some proxy of seniority. I would need to now how you're paired, are you paired with someone at your approximate experience level, if not seniority experience level. I would need to know how the pair-programming works, I would need to know the technology involved, I would need to know if you're remote, or if you're actually sitting next to each other. I would need to know if you're both able to input text at the same time or if one person is inserting and the other person is not.

So that when I do comparisons, I know what the comparisons are like.

Mike Julian: That's an incredible amount of information. I never expected that you would have to know so much in order to get a good answer out of that.

Nicole Forsgren: And that's off the top of my head. Right, I'm spit balling because you asked me a good question. And that's just on research design and then you move on to analysis, right? When you move on to analysis, then we need to get into the types of questions that you have asked. Are these types of questions, are we looking at correlation? Are we looking at prediction? Are we looking at causation? What types of data do we have available and which types of analysis and questions are they appropriate for?

Again, they need to match up the right way. Some types of data, are not appropriate for certain types of analysis or questions. So you really need to make sure that each one is appropriate for the right types of things. Right? Certain types of analysis, like mechanistic, survey questions will never be appropriate for mechanistic analysis, right? Although, quite honestly, no one's every gonna be doing mechanistic analysis. Never and by the way, if anyone comes to me and says they're doing mechanistic analysis, I'm gonna sit back and listen to you very intently, very interested because I don't think anyone's doing mechanistic ... it's not a thing.

Mike Julian: So when you're analyzing the results of the survey, what we're seeing is one question followed by another question, followed by another question, and you know hundreds of questions. When you're analyzing this stuff, are you looking at a question at a time, or are you looking at multiple questions and then interpreting the answers based on what you're seeing across several different questions?

Nicole Forsgren: So when I'm writing up the results, when I'm writing up the report, I am writing up the results of my analysis, and my analysis is taking into account a very, very careful research design. Now what that means is, my research design has been very carefully constructed to minimize misunderstandings. It tries to minimize drift in answers. So, one way that we do that, and this is outlined in part two of accelerate if there's any stats nerds that want to read up on this, we do things called latent constructs.

So, you asked about having only a few questions or several questions. One way we do this, I mentioned, is called latent constructs. If I want to ask you about culture, right, I could ask 10 people about culture and I would get 15 answers. 'Cause culture could mean so many different things, right? In general, when we talk about culture in a DevOps context, we tend to get something that ... people will say very common things like, breaking down silos, having good trust, having novelty, right?

So what we do is we start with a definition, and then we will come up with several items, questions, that capture each of those dimensions. So you might want to think about a dope Venn diagram, where each of the questions is overlayed and then all of the things where they have the biggest, or the perfect overlay, that very center, that little nut, that is what the construct is. That is what culture is, that is what's represented by culture.

And then each of the individual circles is each question. That's what we do in research design. One part of research design. When I get to stats analysis mode, I take all of the questions, all of the items, across, not just culture, but every single thing that I'm thinking about. So in years past I've done monitoring observability, I've done CI, I've done automated testing, I've done version control, I've done all of these things, and I throw all of them into the hopper, right?

Mike Julian: Which is probably your massive Excel spreadsheet I'm sure.

Nicole Forsgren: No, it's SPSS. I use SPSS but you can use several different stats tools. And we do principal components analysis. And what we do is we say, how do they load? Basically, how do they group together and do we have convergent validity? Do they converge? Do they only measure what they're supposed to measure? And do we have discriminant validity? Do they not measure what they're not supposed to measure? And do we have reliability? Does everyone who's reading these questions, read them in a very very similar way?

Once we have all of those things, and there's several statistical tests for all of those, then I say, "Okay, these several items, usually three to five items, all of these items together are culture," or "all of these items together are CI," or "all of these, right these grouping of items, represent this." Okay, now, now, I can start looking at things like correlations, or predictions, or something else and then I get to the report, and now I will just talk about it like, culture.

So I talk about it as one thing, but it's actually several things and then when I talk about culture, I can say, "This is what culture is," and I can talk about it in this nuanced, multidimensional way, and I know what those dimensions are because it's made up of three to five, to six to seven questions, and by the way, if one of those questions didn't fit, because I know from the stats analysis, I can toss it, and I know why. And I always have several items. That's the risk, if you only have one question or if you only have two questions. If one of them doesn't work, which one is the wrong one? You don't know. Right? Because, is it A or is it B? I don't know.

At least if I start with three and one falls out, then it's probably the two that are good.

Mike Julian: Yeah. Many listeners on here have taken a lot of the surveys run by marketing organizations, except the surveys are also designed by people in marketing …

Nicole Forsgren: They're designed by people who want a specific answer.

Mike Julian: Exactly.

Nicole Forsgren: And that's the challenge.

Mike Julian: Right, whereas, to make this very clear, the State of DevOps report is not that at all. There's a lot, as you said, rigor that goes into this.

Nicole Forsgren: So the nice thing is that we have always been vendor and tool agnostic.

Mike Julian: You're not looking for a very particular answer to come out, you want to know what is actually out there.

Nicole Forsgren: And we're not looking for an answer to a product. So, in the example of CI, what is CI? I don't care about a tool. I'm saying, if you're doing CI and if you're doing CI, continuous integration, in a way that's predictive of smart outcomes, you will have these four things. The power in that, is that anyone can go back and look at this as a evaluative tool. If you are a manager, or a leader, or a developer, you can say, "Any tool that I use, any tool in the world, I should look for these four things," or "Any tool I build myself, or if I'm doing CI, I should have these four things."

If you're a vendor, you should say, "If I think I'm building or selling CI, I better have these four things. Right? So that's the great thing and I've gotta say, God bless my new team. They're letting me run this the same way. It's still the same way. It's still vendor and tool agnostic, it's still capabilities focused. Every single thing you look for, whether it's automation or process or culture or outcomes, it's vendor and tool agnostic, it's capabilities focused, and again, the power is that you can use it as a evaluative tool.

Is my team doing this? Is my tooling doing this? Is my technology doing this? Am I able to do this? If I'm not, what is my weakness? What is my constraint? Because if I take us back to the beginning, what is it that drives me and the DORA team, what is it that we want to get out of this? We want to make things better. And how do we do that? We can give people an easy evaluation criteria. And I'm not saying it's easy, because all of this is easy, it takes work. But if there's clear evaluation criteria, we've got somewhere to go.

Mike Julian: Since I know that you love talking about what you found in your several years of doing this. What are some of the most interesting results you've come up with?

Nicole Forsgren: Oh, there's so many good ones.

Mike Julian: Let's pick your top three.

Nicole Forsgren: Okay, I think one of my favorites is, and I'm gonna do this in cheesy marketing speak …

Mike Julian: Please have at it. We have prepared ourselves.

Nicole Forsgren: Someone who had a little startup and had to fake it as a marketer for a minute, we'll see how I do at this.

Architecture matters, technology doesn't. Number one. Okay. So what does that mean? What that means is, we have found that if you architect it the right way, your architectural outcomes have a greater impact than your technology stack. So architectural outcomes, some key questions are: Can I test? Can I deploy? Can I build without fine grained communication and coordination?

Mike Julian: What does the fine grained mean?

Nicole Forsgren: Do I have to meet and work with and requisition something among, do I have to spin up some crazy new test environment or do I have to get approvals across 17 different teams? Notice, I just mentioned teams. Communication and coordination can be a technology limitation or it can be a people limitation. This harkens very much back to Conway's law.

Mike Julian: One of my favorite laws.

Nicole Forsgren: Right? This is very much a DevOp thing. But, it's very true. Whatever our communication patterns look like, we usually end up building into our tech. Now, I will say this is very often easier to implement in Cloud and Cloud native environments, but it can absolutely be achieved in Legacy and Mainframe environments as well. We did not see statistically significantly differences among Brownfield and Greenfield respondents in previous years.

Mike Julian: That's good to know.

Nicole Forsgren: Yeah, so I love that one. That one's super fun.

Okay, number two. Cloud matters, but only if you're doing it right.

Mike Julian: Oh, what does right mean?

Nicole Forsgren: Dun dun duh. So, this was one of my favorite stats. We found that you are 23 times more likely to be an elite performer if you're doing all five essential Cloud characteristics. I guess you could say if you're doing all five essential characteristics of Cloud computing according to NIST, the National Institutes of Standards in Technology. So I didn't make this up, this comes from NIST, okay?

So it was interesting because we asked a whole bunch of people if they were in the Cloud. They're like, of course we're in the Cloud, we're totally in the Cloud, right? But only 22% of people are doing all five things. So what are these five? So these five are on demand self service. You can provision resources without human interaction, right? If you have to fill out a ticket and wait for a person to do a ticket, this doesn't count. No points.

Another one is broad network access. So you can access your Cloud stuff through any type of platform; mobile phones, tablets, laptops, workstations. Most people are pretty good at this. Another one is resource pooling, so resources are dynamically assigned and reassigned on demand. Another one is rapidly elasticity, right, bursting magic. We usually know this one.

Now the last one is measured service. So we only pay for what we use. So the ones that are most often looked is usually broad network access and on demand self service.

Mike Julian: Yeah, what's interesting about that, to me, there's nothing in there that prevents, like say, an internal open stack cluster from qualifying.

Nicole Forsgren: Exactly, right. So this could be private Cloud. I love that you pointed that out. The reason that this is so important to call out is, it just comes down to execution. It can be done and the other challenge is so often organizations, executives, or the board says you have to go to the Cloud and so someone says, "Oh yes, we're going to the Cloud." But then someone has redefined what it means to be in the Cloud. Right? And so, you get there, someone checks off their little box, puts a gold star on someone's chart, they walk away, and they're like, "Well we're not seeing any benefits." Well yeah, 'cause you're not doing it.

Mike Julian: Right. Yep.

Nicole Forsgren: It's like, "I bought a gym membership, I'm done." No. And again, I'm not saying it's easy, right? There's some work involved. Now the other thing that I love is that, let's say you're not in the Cloud, for some reason you have to stay in a Legacy environment, you can look at these five things and you can implement as many possible, you can still realize benefits.

Mike Julian: Right. It's not an all or nothing approach. You can do some of these and still get a lot of benefit from it.

Nicole Forsgren: It's almost like a cheater back to number one, which was architecture matters, technology doesn't. How can I do a cheat sheet to see some really good tips on how to get there?

Mike Julian: So what's your number three here?

Nicole Forsgren: My number three would probably be, outsourcing doesn't work.

Mike Julian: Yeah.

Nicole Forsgren: Which some people hate me for and they shoot laser beams out of their eyes. So let's say outsourcing doesn't work*.

Mike Julian: Okay, what's the asterisk?

Nicole Forsgren: Asterisk, the asterisk is going to be that functional outsourcing doesn't work.

Mike Julian: Okay, so say outsourcing my on call duties, probably isn't going to work so well.

Nicole Forsgren: Taking all of DEV, shipping it away. Taking all of TEST, shipping it away. Taking all of OPS, shipping it away. Now, why is that? Because then, all of you've done is taken another set of hand offs, you've created another silo. You've also batched up a huge set of work, and you're making everyone wait for that to happen. The goal is to create value and not make people wait. If now everyone has to wait for everything to come back, if you're making high value work wait on low value work, because it all has to come back together, which is usually the way it works, you're boned.

Now, functional outsourcing. If you have an outsourcing partner that collaborates with you and coordinates with you and delivers at the same cadence, that's not functional outsourcing. That's the asterisk.

Mike Julian: Okay, gotcha.

Nicole Forsgren: Also, if they're part of your team and they're part of your company but they basically disappear for three months at a time. Sorry kids, that's functional outsourcing. I worked no points, may God have mercy on your soul. It's not helpful.

Mike Julian: Right. It seems to me, how you could tell if you're in this predicament, is if there is a noticeable hand off between your team and whoever you have given these items to, you have functional outsourcing. Would that be about right?

Nicole Forsgren: Yes, and especially if there's a noticeable hand off and then a black box of mystery.

Mike Julian: Of like, how is the work getting done?

Nicole Forsgren: Step one, something, step two, question mark, step three: profit.

Mike Julian: Maybe. So the first two, it's all good because we can kind of see where to go from there, but this third one actually seems a bit harder because if I'm a sysadmin, I have absolutely no control over this functional outsourcing. I may hate it just as much, I may hate it myself, but I don't have any control over it. What can I do as a sysadmin, or someone in ops, someone in dev, how can I improve that situation?

Nicole Forsgren: So some ideas might include things like, seeing if there's any way to improve communication or cadences in the interim. Right? You might still have that outsourcing partner, because that's just the way it's gonna be. But, let's say that you've batched up work in three month increments, is there any way to increase handoffs to once a month? Is there any way that we can take capabilities that we know we import, working in small batches, and just increase that handoff? Is there any way that we can integrate them into our cadence, into our teams?

Now I realize there is some challenge here because from a legal standpoint, we can't treat them like our team because then, at least from the United States standpoint, once we treat them like an employee, then we're liable for employment taxes and all of that other legal stuff. But if we can integrate them into our work cadence, or more closely into our work cadence, then our outcomes improve.

Mike Julian: Okay, cool. That makes a lot more sense. That doesn't sound nearly as hard as I was fearing.

Nicole Forsgren: So it can be starting to decrease the delay on the cadence, asking for slightly more visibility into what's happening, if it's a complete black box, looking for that.

Mike Julian: Nicole, this has been absolutely fantastic. Thank you so much for joining me. I have two last questions. Where can people find this State of DevOps report to take the survey? Where is the survey at?

Nicole Forsgren: Oh, we've got the survey posted. Can I include it in show notes?

Mike Julian: Absolutely. Alright, folks, check the show notes for the link. And my last question for you is where can people find out more about you and your work, aside from this survey?

Nicole Forsgren: I'm a couple places. So my own website is at nicolefv.com and I'm always on Twitter, usually talking about ice cream and Diet Coke, that's @nicolefv.

Mike Julian: I do love you Twitter feed. It's one of my favorites.

Nicole Forsgren: Yeah, everybody come say hi. My DMs are open.

Mike Julian: What I love most about your Twitter feed is roughly around the time that you're writing the report and saying, "Oh my God, why did I do this?"

Nicole Forsgren: Yeah, I try to keep it locked down, but every once in a while something will slip, like "Oh my gosh everybody, something good is happening," or "oh I forgot this one thing," or "So much good is happening."

Mike Julian: Yeah, I remember last year like, "Oh my God this is so cool but I can't tell you about it."

Alright, well thank you so much for coming on and thanks to everyone else listening to the Real World DevOps podcast. If you want to stay up to date on the latest episodes, you can find us at realworlddevops.com and on Itunes, Google Play, or wherever it is you get your podcasts. I'll see you on the next episode.