Government Innovation with Katherine Tobin of National Intelligence

untoldcontent.com/trainings/data-storytelling-training

TRANSCRIPT – Government Innovation

This episode, Government Innovation is powered by data storytelling training from Untold Content and Data+Science. Transform your data into powerful visual stories by learning best practices in data visualization and technical storytelling. Whether you’re a PowerBI or a Tableau person—or just want to better communicate your data—this workshop will inspire you to see the stories that lie in the data. Learn more at https://untoldcontent.com/datastorytellingtraining/.

Katie: [00:00:04] Welcome to Untold Stories of Innovation, where we amplify untold stories of insight, impact and innovation. Powered by Untold Content. I’m your host, Katie Trauth Taylor. Our guest today is Katherine Tobin. She is director of the Lateral Innovation Division at the Office of the Director of National Intelligence. She’s an innovation leader inside the U.S. intelligence community where she leads the cultural and process changes that build a robust innovation environment. Katherine, I’m so honored to have you on the podcast today.

Katherine: [00:00:39] Thank you. I’m thrilled to be here.

Katie: [00:00:41] What led you to the innovation space? As you know, what sort of guided you there in your career path?

Katherine: [00:00:48] Well I’ve always been a creative person, but as you know, there’s a big difference between creativity and innovation and creativity is something new and unusual, but for it to be innovation, it actually has to be useful. So for me, the big shift in mindset was when I took a class in design thinking a couple years ago and, you know, I’d always been trying to mix things up in every job that I did and sometimes that worked and sometimes it didn’t. And it’s so frustrating, you know, why did it not work? Why do people not like these ideas as much as I did? And when I took this class on design thinking the first exercise was: the instructor said, “hey, good morning, everyone. I’d like you to design a wallet for me. Are there any questions?”

Katherine: [00:01:30] And of course, everyone was so “I got this. This is amazing. This is going to be the best wallet ever.” So we got to work, starting drawing out all these new features and, you know, thoughts or pictures or keys or whatever it was. And I was so excited because I wanted to buy this wallet and why had no one thought of it already? And so then our time was up. We eagerly went to the instructor waiting for praise for our awesome ideas. And he said, “You all fail.” And we were like, “wait, what? Why?” And he said, “you were supposed to design a wallet for me. And no one asked me a question. No one asked what I need a wallet to do. No one asked, what’s wrong with my current wallet? No one really cared about me. And I was the hopeless customer.” And it just really hit home that that’s what I’ve been doing wrong all along. You know, it’s not. It wasn’t really innovation. It was just creativity that got lucky. So the real key to innovation is to know who you are designing for and their struggles and what they really need and then empathizing with them and knowing their story almost as well as they do and trying to build for them. And that was just the key to all of it. And I’ve been fusing that into my career ever since. And I’ve been unfortunate to find a group of people in the intelligence community who are also working that way.

Katie: [00:02:52] I love that story because I think all of us can relate to that feeling of having a great idea or thinking that something is going to be effective and sort of getting caught in our own heads about it, if you will, and forgetting about user experience and collaboration and getting feedback and building those loops along the process. I think storytelling can give us such an interesting opening for empathizing with one another, for understanding our stakeholders or our consumers problems or challenges and innovating against that in particular. Could you tell me some of your thoughts, you know, around storytelling, the ways in which your storytelling has changed over the years as an innovator?

Katherine: [00:03:35] Well storytelling, in the professional sense, is still pretty new. I mean, storytelling as humans is really old. We’ve been doing it since the beginning of history. As children, we like stories. After work, we read books. We watch movies. We are immersed in stories, except for some reason, the nine to five. That seems to be the desert of anti-stories, for the most part.

Katie: [00:03:59] Yeah.

Katherine: [00:04:00] So it surprises people when we approach them, not with facts and figures and flowcharts, but rather with a story. But it’s so incredibly effective. The one example is: my team runs a program called the Intelligence, Science and Technology Partnership or ISTA. And its goal is to help connect the really new innovative capabilities from startups and smaller nontraditional businesses with the mission needs of the intelligence community. And, you know, my team runs the program. We think it’s pretty awesome. But we wanted to make sure that it was actually serving our customers. And so we thought, well, can they even find it? Are we communicating it well? And so we asked our interns who were unfamiliar with the program to pretend that they were startup founders and wanted to do business with the intelligence team. So we had them document and basically write the story of their journey to try to do business with ISTA. And, you know, we thought it would be a love story where they did a little research and they found out about us and aren’t this great, happily after. It turned out it was more of a horror story like, you know, when you’re watching that movie and you want to yell at the character, “don’t open the door!” It was like that. We were reading their stories and what they were searching for was wrong and the resources they were reading were not the right ones. And they didn’t understand how things worked. And it was just emotional for us to read because we knew how it was supposed to be. And it was emotional for them because they were so frustrated and so eager and it was just heartbreaking. And so we thought, you know, we know this story because we asked them [unclear wording]. Think of all the people who just run instead. And so it was so impactful and it completely changed how we thought about advertising the program to people. And we shared this. We packaged it as it is, sent it to other government agencies who are running similar programs and said, “hey, guys, this is what’s actually happening [unclear wording].” And everyone sat up and paid attention. And that just goes to show that, like, if we had sent over “best practices for communicating with startups.” Eh, you know, those are helpful, but it wouldn’t really grip people like the story did.

Katie: [00:06:26] Absolutely. What did you decide to title that report? Do you remember?

Katherine: [00:06:32] This was before we got into catchy titles. We are now on and always trying to have captivating titles. I think it was something about, you know, the customer journey to finding the intelligence community or something.

Katie: [00:06:46] Sure, sure. Sure. It’s incredible. I love that. And those interns must have felt like their voice mattered so much in that process.

Katherine: [00:06:55] Absolutely.

Katie: [00:06:57] So can you tell us about some of your favorite innovation stories coming out of the work that you and your teams are doing within the CIA?

Katherine: [00:07:05] Well, I’m not at the CIA anymore. I was there for a couple of years and I’m at ODNI now. I mean, it’s an intelligence community, we all work together very closely.

Katie: [00:07:14] If you could actually, could you tell us about sort of for people who aren’t as familiar with the structure of the intelligence community… Can you tell us about the relationship between your office, CIA and some of the other main players that you’re often collaborating with?

Katherine: [00:07:30] Sure. The intelligence community includes 17 elements and some that you have heard about and know about from TV, such as CIA or NSA or FBI. And some are smaller and often they’re found as part of other cabinet level agencies. For example, the State Department’s Office of Intelligence and Research, part of the intelligence community. But it’s also part of the State Department. So if you include the 16 elements plus ODNI, you get 17. And ODNI, it plays a coordinating role. So coordination, oversight, convening, basically, while the agency, the independent IC [intelligence community] elements themselves, such as CIA or NSA, are the ones who are building the satellites and analyzing the imagery and writing intelligence products. The ODNI instead is working on behalf of the whole intelligence community and really making sure that we’re working together and giving a unified voice to support policymakers. So I think a good example that helps illustrate that is people are familiar with the president’s daily brief. And what people might not realize is that that is a product that is by ODNI. It includes pieces from all over the intelligence community based on the topic at hand. And people go on rotation from whatever their home agency is to ODNI for a few years and are part of the president’s daily brief team to compile the book and brief it downtown. So it’s kind of that all-star team of people going on rotation from different IC elements to do something on behalf of the community. And then how that works for me and my team is lateral innovation is here to amplify and accelerate innovation and innovators across the intelligence community. So our role is to work with innovators across the community, either advocate. So we often produce training materials such as playbooks on the best way to run a hackathon or how to do a pitch event, working with startups, or how to do storytelling, things like that. These [unclear wording] materials and guidebooks, we also are working on incentives. And so how might we reward people for being innovative? Whether that is an award for great innovations or [unclear wording]. Or also how might we shape the incentive structure to get people to think in terms of balancing innovation and moving quickly with also our security [unclear wording] and incentivizing the lawyers to focus on: “Yes, if” as opposed to “no, because.” That’s kind of the incentive side. And then the implementation side is we’ve noticed that innovation is happening all across the IC. Has been for years and it’s great, but it’s not often carrying across to the different agencies and elements as quickly as it could. So, for example, if the NSA has figured out how to do something, they want to use that product. Right? They want to use it for missions, which is wonderful. But who at NSA would want to share it with the CIA or FBI? So we’ll work with NSA, package up what we can, document the lessons learned, and then make it more of a franchise opportunity for other agencies to pick up and adjust and implement it as meets their mission needs rather than starting from scratch.

Katie: [00:11:24] Wonderful. Thank you so much. That adds so much context to all of the forms of collaboration and innovation, the ways that it spans across the entire community. Thank you. That’s perfect. So something that our team was really excited about. You actually worked with some colleagues to create a panel a year or so ago called “Seek Truth, Speak Truth: A Spy’s Guide to Storytelling.” Can you tell us a little bit more about that?

Katherine: [00:11:51] It was wonderful. It was such a great experience. This was a panel at the South by Southwest Interactive Conference in March of 2019. And if you haven’t had a chance to attend, I highly recommend it. People from all over the world come together for a week focused on innovation, design, technology. So many new ideas are shared and you hear from people from industry and government and academia. And there are a lot of really interesting mashups and collaboration and free food as well, which is great. And so I was on a panel with several other IC officers from the Office of the Director of National Intelligence and CIA. We all had a different take on the topic of storytelling and our roles. There was one person who had been a speechwriter for several senior leaders and someone who is the head of equal opportunity and diversity in the IC, and someone who is our deputy transparency officer at ODNI, and then myself. And so we each talked about how storytelling is useful for our role. So the speechwriter talked about how storytelling can help paint a really compelling picture and the importance and the balance of not using emotion to manipulate. So what… how do you use enough emotion that the audience understands what you’re trying to do without going overboard and really not playing fair? So he talked about how he does that. The head of equal opportunity and diversity talked about, from her perspective, storytelling helps us bring our whole selves to work. And that is so key to diversity and inclusion. And how the IC really values, diversity and inclusion. That’s one of our core values. And so how we are really highlighting the stories of our vast diversity of officers and understanding not just their work selves, but full selves. And that way we can appreciate each other. And of course, for me, picking right up on that diversity is so important [unclear wording] because all of those perspectives and solutions from very different places and metaphors and analogous worlds. So we need people who bring their full selves and their full array of experiences to work. And so the transparency officer was talking about how storytelling helps bring transparency and it really opens the black box. The intelligence community is secretive for a reason, but that doesn’t mean we don’t have a duty to explain what we do and to show what we can with the American public. And so he talked about how rather than just sending out declassified information in structure format and letting people do what they want. How do you tell the stories of what the intelligence community is doing so that it’s actually a compelling and understandable package for the public? And so he shared, for example, some videos from the wonderful website, intelligence.gov, I highly recommend people check that out, which features stories of current and historical intelligence officers. And then from my perspective, I was talking about storytelling and innovation and how we use storytelling, whether that to get buy in from leadership when we’re seeking resources to pursue a project or to get audiences and customers or would be customers to understand their need to change their behavior and also how we’re teaching storytelling as a part of the core curriculum for innovators across the community.

Katie: [00:15:49] It’s wonderful. And I think you’ve actually hosted an event as well inside the CIA where you essentially trained people on what we call at Untold, “innovation storytelling,” right?

Katherine: [00:16:00] We did. We did. This was a couple years ago. And it was a great example of an intelligence agency thinking really broadly. So, it was a wide variety of people, not just a designated innovation office, but volunteers all over the agency who put this event on, and it included actual storytelling, as well as teaching people how to hone the craft. And so people talked about different story archetypes, so you have the hero’s journey. You have the love story. You have the revenge story. And when to use each and how to plot your story, how to practice your story… We gave out these workbooks that people can use to start with something as simple as post the story of your name. It’s a great icebreaker and get people in the habit of storytelling, as it can be scary. And also some great infographics about why stories are important and how they work. And we were able to release those to the public at South by Southwest and share those with you for the show notes as well.

Katie: [00:17:03] Yes, I will link the – it’s really incredible. It’s essentially a workbook called “Innovation Story Craft.” It reminds me of some of the elements that we cover in our innovation storytelling training. So I think it’s a really wonderful resource. And I’m so grateful you’re able to get that cleared so that we can share it with listeners. But I’ll definitely link that workbook. And then there’s also a really awesome, essentially, what are those called? Like an interactive word bubble image?

Katherine: [00:17:33] Yes, it’s like a graphic notetaking mural in paper format.

Katie: [00:17:35] Yes. Yes. Yes. And I’m assuming that’s from the session as well.

Katherine: [00:17:41] Yeah. One of my very talented colleagues put that together, and we use that as a kind of an ever-great handout when teaching workshops like this. There’s so much interest in this topic that workshops on the topic will come and go throughout the year.

Katie: [00:17:58] I love this so much. These are both fantastic resources and definitely check them out. I think. Let’s dive in a little more deeply because it is so fascinating to think about a culture. You know, the innovation community is really set up as a culture of, like you said, secrecy, privacy, trying to be as stealth as possible for, you know, to protect the American people, of course, and to protect themselves as well. Tell us the reaction of the intelligence community going through this innovation story training.

Katherine: [00:18:31] It’s definitely a wake up call for people because it’s so different from what we’ve been trained to do. You know, the culture really values truth and fact. And there’s a quote on the wall at the original headquarters building at the CIA. And it says, “And you shall see the truth, and the truth shall set you free.” It’s all about understanding the truth. And when analysts join, regardless of what agency they are at, they go through multiple months of training in how to do proper research, how to cite your sources, how to spot misinformation, how to do structured analytic technique, how to, you know, basically remove the emotion from the analysis. And now here we come with stories. And so it’s certainly surprising to people and sometimes we get a bit of skepticism in that, there’re people who are innovators tend to be generalists who are very creative and who have worked in a wide variety of [unclear wording]. So we’re not an analyst who has spent 20 years on a certain topic. And so when we come in and are trying to work with people who are specialists, we sometimes get questions like, how do you know better? You haven’t been doing this job for 20 years. And I have. And so that’s a part of the reason why stories are so important. [Unclear wording], doing our interviews with people and really developing empathy and understanding what people’s struggles are, we’re able to compellingly convey the challenges that are afoot and see that system and that whole ecosystem of what’s going on. So we can explain to someone that, you know, here’s the name of an analyst and say here is their day to day job. And here’s why it is expensive and difficult and risky and might not yield the best answer and really get people to think, oh, I have not experienced that. I didn’t see it that way. But you’re right. That’s a real problem. [Unclear wording.] So we use stories as kind of a call to action and to build credibility, because while the innovators might not have 20 years of experience on a certain topic, we do know how to do interviews, and understand the whole system around it. So we do have data to back up whatever it is we’re doing.

Katie: [00:20:54] Yes. One of the quotes that I love inside of your mural is from Robert McKee. And you say, “what happens is fact. What we think about it is truth.” And it is such a challenge when you’re asked to tell a story, so many of us sort of go back to our days as a child when we were, you know, accused of lying if we were telling a story, you know, “well you’re just telling stories.” So I think when we’re in a position of pursuing truth and trying to always represent the data and the information as correctly and accurately as we can and as ethically as we can, but also spark action or create change in light of that information or help decision makers determine how best to act. There’s such a fine line between, you know, presenting the facts as facts and those facts and, you know, forming a story. So it makes sense that it is such a complex initiative inside of the intelligence community. I think it’s challenging for all innovators to do this, but especially in that world, where to be called a storyteller could be a very negative accusation.

Katherine: [00:22:10] That’s a very good point. I hadn’t thought of it that way. So luckily, people have been pretty receptive. You know, we like to explain… We have a lot of stories that we use time and time again. So if people are skeptical about doing innovation, and you know, the process of innovation at all and they say, well, you know, we are already financially very, very strapped. We don’t have a lot of time. We don’t have a lot of people. Why do you want to take additional time and additional resources to try to do it better, knowing that it might fail? We can point to examples of stories of, well, here’s how it took just a couple of weeks and a couple of interviews and some quick idea exploration and a couple of very quick failures to be able to come up with a solution that was significantly cheaper and faster and safer. And now we do it this way. And it’s only because of taking the risk with the innovative process, that we’re able to do things better. And that usually wins people over.

Katie: [00:23:19] Oh, absolutely.

Katherine: [00:23:19] So I think, yeah, it’s you know, there’s the values that we have are for truth and for facts and for expertise. But then also there’s always that question of: is this worth our time? As time equals taxpayer dollars. Right? And so it’s not a matter of profit margins. It’s not a matter of meeting quarterly earnings reports. It’s: is this a good use of our taxpayer dollars? And so by telling the stories, I think we can help people understand that we’re solving human problems. This is not just a matter of shiny technology. I think people have two big misconceptions about innovation, that it’s something fun and quirky and creative and an excuse to not do your day job. Or that it’s very technical and the realm of science and technology and therefore opaque and so not meaningful to someone. And so…

Katie: [00:24:16] I was going to ask you… I was going to ask you if people who are outside of your professional circle hear what you do and ask, “so you’re sort of like “M” from the James Bond movies?

Katherine: [00:24:27] Sometimes,sometimes. Or they’ll say, are you like “Q?” Which is science and technology.

Katie: [00:24:31] That’s right.

Katherine: [00:24:32] I like to have it. Yeah.

Katie: [00:24:34] The one who introduces the really sharp, sophisticated, you know.

Katherine: [00:24:38] Yeah.

Katie: [00:24:39] Technologies.

Katherine: [00:24:41] Yes, exactly. So that’s the Director to Science and Technology. The Director of Science and Technology that I see is actually the office next to mine, which is kind of funny. But, so, we’re different. There is no James Bond equivalent in my office yet, which is a shame, really. They should work on that. But, you know, we like to show how we are not just building technology cause it’s fun. And in fact, a lot of the times the innovations are not technology. They are process innovation. There are policy innovation. They are a physical space innovation. And they’re doing it for a purpose. And that’s to help people. A lot of our innovations are focused on our internal workforce and helping us do our jobs better as opposed to, unlike with the [unclear wording] example I gave earlier where our customers were external to the public. Mostly what we’re doing is trying to improve the business of intelligence. 

Katie: [00:25:38] Absolutely. It’s incredible. You’ve identified several what we call innovation story patterns, too, and you’re using those to train your workforce to start thinking about how to get buy-in for ideas that could help improve the community. And so I see that you’ve kind of found the innovator’s journey, you know, saying this is why I’m doing this, why I’m the right person to do this, or why my team is the right team and building credibility in that way or personal relatability. I like the other one that you’ve identified, the user journey. So thinking really clearly about who this is going to impact and sort of painting a picture, walking your stakeholders through how this is going to change the workflow or change the lives of the employee or the person who’s going to be touched by this innovation. You also say the brand journey story, the making of it. I think that one is interesting. We’ve done a lot of analyses at Untold around different innovation story types and patterns. And I’d probably fit a lot of the ones that we’ve identified into this category of the brand journey story or maybe the innovation journey story, because it seems like the ones that get the most buy-in are the ones that say, “here’s what we tried or here’s what we’ve tried so far. Here are the challenges, the technical challenges or the implementation challenges or the personnel challenges that we know or we think we know we’re going to face. And here’s our plan for getting through those challenges” and the clearer you can build the stakes around those challenges and the impact of overcoming them and the more that you can create credibility for the team that’s going to get through those challenges. It just seems to create the best kind of buy-in. So I was wondering, could you tell us a little bit more about your views on other story patterns or the ones that you share inside of your storytelling training?

Katherine: [00:27:30] Well, I love the ones that you named. Absolutely. As you were going through the different types of stories, I was like, yup, called that one and yup told that one. Yup. And it’s like a highlight reel of the past few years. We also – it really depends on what part of the journey we’re in when we [unclear wording]. 

Katie: [00:27:48] Yes.

Katherine: [00:27:48] Because at the beginning. It’s a matter of, hey, we think there’s a problem. We don’t know exactly what the solution is. But we need cover with resources, maybe some money. Can we get that? And so then it’s the story of the struggle of the customer. And, you know, we’re looking for a hero in this story. And you can be that hero by giving some money, to work for it, [unclear wording], and then they were midway through. And the heroic struggle of the person. We have an OK solution. We have an OK idea of what the solution is. It’s not fully baked yet, but we definitely know what we want, some wonderful future life. That’s kind of a mystery story. Like, how do you get through the sticky part? And so that’s usually around the point where we’re trying to get the organizational buy-in, figure out that bureaucratic viability for the solution. We need that trifecta. You need the user desirability. The technical feasibility. You need the bureaucratic, the organizational viability. So when we’re working on the viability, then it’s a matter of presenting it as the story of the problem and get them really invested. “Oh, yeah. We need to solve that.” And what the shared vision of the future is. “Yeah, I get it. I want to go there” and then we can talk about the solution a bit. But it’s locked up and it’s sticky. There’s a problem to be solved that is caught in this contract trap. Can you, the contract office, help us get it unstuck? And how do we sort through this? And really, the goal for that type of story is to get them in Problem-Solving mode rather than approval mode. Because if you present it as: here are the specs of the solution, if a personal comes like an infomercial and those are difficult to do, and also if they find one thing wrong with it, they’ll say no. It’s like watching an infomercial. You’re like “oh, machine washable, no thanks.” So they – it changes the role a bit. And so instead, if you presented as: this is a mystery and you need to help us unlock the mystery, then they’ll work with you. Because I think a big misconception is that people view the lawyers or the contract specialists or security relationship as adversarial. “Oh, they’re going to say no. They won’t let us have nice things,” etc. But really, we’re all on the same team. We are all being paid by the same people. We have the same mission. It’s just we need to figure out better ways to work together. And so that’s why I like to present it, as: they are part of the story with us. And it’s kind of like a “choose your own adventure” and very active. And so they can help us get to the other side. And those have been really helpful. And in fact, something that we’re working on right now, a team, is we’re interviewing those key stakeholders in the organizational viability side, so the lawyers, the security people, the contract specialists, and asking them, “how can we better tell stories? What questions would you want answered in a story?

Katie: [00:31:09] I love it. Yeah.

Katherine: [00:31:09] And and so that way we can train people on if you’re working with the lawyers. They want your story to include the following. And you know, what questions the contract people have, et cetera. And obviously we’ll be testing and iterating with those guidebooks as innovators do.

Katie: [00:31:29] I think that what you just broke down for us on the podcast is so critical that I want to restate it essentially and just say what you said one more time, cause I think it’s so important: when you are trying to get Internal Buy-In from other departments, other units, other areas where you’re all still working towards the same mission. But your day to day or your roadmap, you know, priorities might be very different. The key is to position the opportunity in a way that’s conversational and pull this – make the story become a collaborative story as best as you can. Invite them to write the story with you. Don’t just come at them with a, you know, like you said, a requirements list or in the non-government space, this would often be, you know, a pitch deck. Right? that you’re presenting to a particular group.

Katherine: [00:32:23] Right.

Katie: [00:32:23] Instead, ask them to write the technical brief with you. Ask them to write the RFP with you. And that kind of transition of roles can create all kinds of innovation in and of itself. So I just wanted to pause on that because I thought it was such an important point that you said about getting internal buy-in, it’s really about playing well together.

Katherine: [00:32:44] Absolutely.

Katie: [00:32:46] You know, I just love everything that you’re doing. I want to go back to this workbook that you created too, because I like – you said earlier in the podcast that this can be an intimidating practice for people who aren’t trained to do it. And I think all of us, you know, we can remember days in early elementary school where we learned about the story arc and character development and the climax and the resolution and all of that. And in a way that in and of itself can make the story feel like it’s something that is so basic or that doesn’t need to get reviewed. But it can get quite complex to tell powerful, clear stories when we’re talking about technical information or systems change, you know, involving multiple players, and looking at different budgets and risks. And it’s so much different in that context. But I think sometimes story can get blown off as being less important because we sort of associate it with a child-like lesson in our lives. But I also think sometimes the basics are they never go away in terms of their importance. 

Katherine: [00:33:50] Absolutely.

Katie: [00:33:52] So I’m curious, though, when you’re working inside of innovation teams the way that you are. When do you find that evidence is too much evidence or data is too much data? Or alternatively, have you found that some storytelling techniques kind of trigger the audience to not demean the storyteller, but sort of associate that opportunity with something that’s not evidence based? If it’s all about the story, like how do you find that balance between the story, in the sort of traditional way that we define stories, and data and the traditional way we define like a bullet point or a statistic?

Katherine: [00:34:33] That’s a really good question. And it’s always a balance, right? How much research is too much research? How much research is enough research? And I think there’s actually a book called Just Enough Research. Right. So it’s clearly a topic on all of our minds. And so we like to start with a minimum of eight interviews. And I’m careful to say interviews because as I mentioned, we’re always thinking about taxpayer resources and how we can be most efficient with our time. And so we actually don’t go the survey route or the focus group route. Even though you might say, well, those are more efficient, you can cover more people and get more data from that. And the reason why we usually do interviews is because we can ask follow-on questions. You can see people’s facial expressions, really understand the emotion better, know when it’s appropriate to press for more information. When someone might be showing or signaling discomfort and we want to rephrase the question, etc.. So we try to do the one on one interviews, even though they don’t produce as many data points as, say, a survey would. And so we’ll do an initial ballpark goal of eight and see how the data shakes out from there. So if we’re starting to get a lot of really similar answers, a pattern. OK. So probably we’ve covered it. If after about eight interviews. We find that the answers are contradictory. They’re all over the place. Then we regroup and think, well, maybe we’re misunderstanding the users. Maybe there’s a wider variety of use cases or personas than we’re originally anticipating. Maybe we solved the problem wrong. And so it’s good to know that after eight interviews and not after two months of interviews, right? So we try to work really quickly and iterate. And the way we’ll capture that data is we’ll work in pairs to do the interviews. One person will be doing talking and one person will be doing the note-taking. Unlike with a lot of people in the real world, as we like to say, our techniques for doing interviews and recording data has to be a bit different. We work in secure facilities where we’re not allowed to bring our personal electronic devices. Whereas right now we’re recording this just through our phones. And we can take pictures of if you were to do a sticky note exercise and conference rooms, take pictures, record it. We can’t do that. So we have to take our notes longhand and type them up. And then when we go over the notes, we anonymized the name and in with our current projects. We’re using names from the game of “Clue” to encode the data because A) it shows to people that it’s clearly fake names. So don’t try to read too much into it. And it also gives us a fun sense of whimsy as we do it. So we have the “Clue” names and then we record all of it, go through the data and we’ll cluster things. So we like to do the Rosethorn/bud techniques, which is where you look at all of the positive things that people have to say about a certain topic. For example, we did one a couple last year about how we might improve our recruitment of people with subject matter expertise in science, STEM recruitment in the intelligence community. So we had some interviews with people who were recent hires or graduate students or potential hires and coded the information. So what’s going well, what’s not going well, those were the “thorns.” And then the “buds” are what are the areas [unclear wording]. So kind of cluster them thematically and look for patterns. And at this point, you would normally take pictures of the other sticky notes and record them all. And I just have boxes of sticky notes in my office from past projects because you don’t want to get rid of it. You always want to go back to the data, right? [Unclear wording.] And so we do the data. We might create personas based on the interviewees we’ve done. So Persona is basically a fictional amalgamation of the types of people that you interview. And so it’s a really good way to explain to people who are not present for the interviews, such as your project sponsor, who you are solving for and their needs, their fears, the capabilities they have, and you can also do a double check for any solution against the personas. Would this really help Sam? Would this really help Professor Plum? And so we have our data and we’ll do the questions. And then from there, we’ll really figure out what is our problem? What is our root problem? Right? Ask the question, “why?” Five times to really make sure we’re solving the right problem. And at that point, it is often different from the problem we were asked to solve. And if the person, if the product sponsor is not in the room at that point, we need to find them ASAP because there is nothing worse than surprising your project sponsor at the end and saying, “guess what, you were wrong. That wasn’t the problem at all, but don’t worry, we figured out the right problem, which is these other things.” They don’t like that. So it’s really important to go back and say and keep them posted on the data and what you’re learning, because if our data is showing something different from what they asked us to solve and they still want to solve the original problem, then we need to understand their needs. And what they’re trying to do is to talk to different people, that maybe we missed the first time. So that is a really, really important status check. So once we’ve deconstructed the problem, we will then do any number of techniques to generate a lot of solutions. Always go big before you go small. And so we’ll bring in creative people, we’ll transpose the problem lots of different ways to try to come up with as many solutions as possible. [Unclear wording] then find a couple of favorites and then figure out how to really quickly test and iterate on them. And for that, we are also trying to be good about documentation. And if you’ve ever seen – there’s a template, a free template, called “Strategyzer,” strategyzer cards. Or if you’ve ever graduated from elementary school and have had elementary school science, you know the scientific method. So Strategyzer cards are basically the same as the worksheet you got in fifth grade when you were learning the scientific method. It’s so accessible. It’s great. And so it’s asking you, you know, what are you trying to learn? What do you think? You know what? And how will you test it? What do you think you’ll see when this happens? For example, if we wanted to test the hypothesis that. You know, if we. And this is a complete, complete hypothetical. If we allowed people to apply to multiple IC elements at once, a common application, then applications to some of the lesser known elements would go up. And so then we might test that with people and see if that happens. And if it doesn’t happen, well, now we learned, right? So the reason why those cards are so important is it forces us in advance to think about: what are we testing, what are we looking for? What does a, you know, quote unquote success look like? And not. Because that way we’re documenting when it doesn’t work. And we can easily share that with others. Thought a common application would increase applications to this agency. It did not. Please, guys, don’t go down that road again. And if we’re right, then it also shows that we didn’t just cherry pick evidence. No P hacking. As is sometimes a problem in the science world. That really builds up credibility with people when we’re trying to explain it to them. And, you know, show your work, show the process. But how does that come out with storytelling? Right. Like, I just give you a very detailed description of the steps we take. Our audience doesn’t necessarily need to know that, but we need to have done the work so that we can choose the right elements. It’s all about picking, understanding the story that the audience needs to hear as opposed to the story that you want to tell. 

Katie: [00:42:54] Yes! Yes. 

Katherine: [00:42:54] And that means we might need to combine some parts to get some parts that always stay really true to the message and not cherry pick. And, you know, change it or whatever you are, you still have done the work, because what was problematic is if you get up there and you tell the story of your user and their struggle and our process for how we created this great new thing for them and don’t you love it? And they say, well, you know, the analysts that you’re building for what does their boss say about it? And you’re like, oh, I don’t know. I didn’t talk to the boss. Right? Then you lose your credibility. So you need to have done all of the research and be able to point to. You’ve done interviews with these people. This is what they’re saying. Or if they ask you on the fly. “Well, you know, you told me the story of the analysts, how they’ll use their tools to write their report. Well what happens to the report after that? How does it fit into the ecosystem?” And so if we are doing our jobs well, we can say, “well, we have also talked to the policymakers who will use the product. And here’s their story of how they’ll use it.” And so, you know, it’s part of the storytelling curriculum. We also do a lot of improv because it is a story. That presentation never goes off exactly as you think it will. And so you need to be ready for questions from the audience. For what about or push back or whatever and be able to respond with an equally polished answer and try to keep that storytelling still alive and not just revert back to, “well, 54 percent…” Or whatever it is. 

Katie: [00:44:36] Incredible. Yes. Yes. So you’ve given so much advice in that example, I think it’s so helpful to hear your process. And as a qualitative researcher, of course, my heart is soaring over here. That’s my background for sure, is you know, collecting human-based research and trying to analyze and find the trends within it. So, you know, the fact that you’re doing that as a way of pulling out the right innovation story… I think is so important. And, you know, again, I think one of the major challenges that we see, whether we’re working with industry or government innovation teams, is that we just, we like… The innovator collects all this information and all of this data, as you said, and we get kind of close to it, you know, it’s our babies, if you will. And it’s such a labor of love that it’s hard for us to extract ourselves and determine what is the most critical information to share and which slides should we have there. But hide and have at the ready for when someone asks that, you know, that question that’s going to sort of call into question the conclusions that you’ve come to. So I love that. And I’m so grateful that you’ve walked us through the process. I think a lot of – I’m hopeful that listeners are taking that process into account and thinking through, you know, are you paying attention to the story as you go about doing the innovation work? I think we need more visibility for the role that story plays in the innovation process and especially for internal purposes. The way that you’re describing, I think the majority of innovation storytelling, especially after conducting 100 interviews with innovation leaders like yourself, at this point, we’re just seeing it is kind of surprising, I guess, from the outside, if you haven’t thought of it this way. But innovation, storytelling is far more critical internally to an organization, typically, than it is externally. And so, yes, it’s important that consumers or your communities see the ways in which you’re innovating and that they understand what it is that you’re doing to make your organization stronger and better. But if those projects can’t get internal buy-in and get momentum through the power of story internally, we never even get to get to that point of being external-facing. So that’s kind of a surprise finding, I think, at Untold when we first started this research question of “where does story live inside the innovation world?” More powerfully than not, I would say most people really talk about its critical importance to internal buy-in. And so I’m really grateful to hear you talk through your process and your ways of, you know, thinking through how to pull different stakeholders in, get their reactions to stories, get their feedback, and then refine the way that you tell it and deciding which data you present based on their preferences, priorities, concerns. So I think there’s so much to that. And if I were a listener, I would just go back five minutes and listen to that whole section again. I’m so grateful for the time we’ve had together, and I’d love for you to leave us with some key pieces of advice that you’d like to give to innovators as they continue to share their great ideas. 

Katherine: [00:47:54] Absolutely. A piece of advice that I give to everyone is to fall in love with the problem. Not with any particular solution. So I like to give an example of dating. Right? First, you need to figure out who you are and what you need and what makes you happy. And then when you go on your first date. Don’t just book at the wedding venue right away. Just maybe get excited. But maybe try it out a couple times. Try meeting different people. And if it doesn’t work out, that’s OK, right? Think of it as: you’re the quirky heroine in the rom-com. And this is all part of the process. This is the point. We try multiple solutions and eventually you will have your happily ever after. 

Katie: [00:48:40] Yes. 

Katherine: [00:48:40] The key is really just to know yourself and appreciate yourself and know that you’re worth the right. 

Katie: [00:48:46] Katherine, I love that advice. That’s wonderful. Thank you for leaving us with that metaphor. It’s so great. I think we can all relate to, you know, whether storytelling or writing or content creation, whether that comes naturally to us as individuals or not, we’re all humans. Therefore, we are storytellers. And so finding, you know, what makes sense for you and being true to who you are as you represent the work that you’re doing is so important. And I think your metaphor speaks really beautifully to that. 

Katherine: [00:49:16] Thanks! 

Katie: [00:49:16] Katherine, where can people find out more about you and your team? Where can they find you online? 

Katherine: [00:49:24] To learn more about this topic or to learn about my work in the intelligence community and innovation, I recommend you check out intelligence.gov. There’s a profile and an interview with me from a year or so ago and on the dni.gov website. You can find the email address for additional press inquiries. 

Katie: [00:49:44] Wonderful. Katherine, I will link to your article on intelligence.gov in our show notes. And once again, thank you so much for making time to be here today and talk with us about innovation storytelling. 

Katherine: [00:49:56] Thank you. This was really fun. 

Katie: [00:49:58] Thanks for listening to this week’s episode. Be sure to follow us on social media and add your voice to the conversation. You can find us @untoldcontent. 

You can listen to more episodes of Untold Stories of Innovation Podcast.

*Interviews are not endorsements of individuals or businesses.