See the System: Path for Improvement (Part 4)

Before you jump into tools and solutions, you need to take a step back. In this episode, John Dues and Andrew Stotz discuss the four questions you need to ask in order to make the system you want to work on visible, and decide on a direction.

TRANSCRIPT

0:00:02.5 Andrew Stotz: My name is Andrew Stotz and I'll be your host as we dive deeper into the teachings of Dr. W. Edwards Deming. Today I am continuing my discussion with John Dues, who is part of the new generation of educators striving to apply Dr. Deming's principles to unleash student joy in learning. And the topic for today is, what is the topic for today, John?

 

0:00:25.4 John Dues: See the System.

 

0:00:27.3 AS: All right. Well, why don't you take it away?

 

0:00:30.3 JD: All right. Great. Yeah, it's good to be back. Yeah, just as a quick refresher for people following along in the series, so two episodes ago we briefly talked about the importance of framing the challenge as an improvement goal. And then last episode, if you remember, we went deeper into those three sort of faces of performance measurement. There was research, there was accountability, and then there was improvement. And if you remember, you took the avocado quiz and you got 100% on that quiz. But the whole purpose of sort of taking that time to dive into those three different types of measurement or three different types of goals was this idea that problems arise when measures aren't used for their intended purpose. And what we talked about was, if you're gonna do a research goal, then, you know, the basic goal for research is to contribute to the knowledge base.

 

0:01:32.4 JD: And for accountability, the purpose of the goal is to determine the application of rewards and sanctions. And then for improvement, the goal is to learn our way to a system that produces a higher level of performance. And basically, at the end of that sort of episode, we established that the challenge or direction should be framed as an improvement goal, in my opinion, and not as an accountability goal, not as a research goal. And then we have this model that we've been working through which is up on the screen for those that are also watching the podcast, and it's this four-step improvement model. So we've outlined these steps. So step one is to set the direction or challenge. That's what's gonna be your improvement goal, not your accountability goal.

 

0:02:28.7 JD: Step two is to grasp the current condition or conditions on the ground where you are in your organization. Step three is to establish the next target condition. Step four is to experiment, overcome obstacles in the way of meeting that next target condition on the way to that ultimate goal or that direction or challenge that you've set for your organization. And then we've said multiple times that all of the steps are done with this team that's including those working in the system, those with the authority to work on the system, and then someone that has this System of Profound Knowledge lens. That's sort of a quick recap. And so what I thought we could do today is to sort of go back to, actually to step one.

 

0:03:16.8 JD: So we've gotten an overview of it, and talk about some other things to think about when you're on this particular step. And if you remember in step one of the model, what we're asking is, where do we want to be in the long run? Or what do we as a team or as an organization, what do we wanna accomplish? And we've talked about that this is this longer range goal, and it's a big goal. It may seem even impossible or nearly impossible at the outset, but if we achieve it, what we're saying is this thing, whatever it is, is really gonna differentiate us from other organizations, from other schools in our case, or you know, it could be differentiating you from other businesses or other hospitals or whatever the thing is that you're working in.

 

0:04:07.9 JD: We said this challenge or direction typically has this sort of long-term timeframe, six months to three years as sort of a rough guide. And then I talked about this example, one of our goals or challenges that we sort of set in our organization is we wanna significantly reduce student chronic absenteeism. If you remember, when we looked at this a couple episodes ago, that number rose significantly coming out of the pandemic. And so it's right around in the 50% range. So 50% of our students are chronically absent, which is a very high number. And we wanna take that all the way down to 5%. So right now as things currently stand, that seems like a very tough, nearly impossible challenge that we've set for the organization.

 

0:05:07.4 AS: I'm curious about how stretch of a challenge should someone be thinking about in this case? I mean, it's one of the hardest things when you start setting goals, is like, are we thinking about 10 years from now? Are we thinking about one year from now? Are we thinking about, come on guys, let's dream big, let's envision what we could be, or are we talking about realistically we think we can get to this point? How do you think about that?

 

0:05:32.7 JD: Yeah. I like this six month to three-year timeframe for step one, for the challenge or direction, that sort of ultimate goal that you're reaching for. It's not so long that it seems like it's, you know, there's no urgency. It's not five years, it's not 10 years. But it's not so short that it's gonna be impossible to achieve this tough thing. Six months to three years, when you're talking about a significant organizational effort, you gotta be realistic. There's, you gotta have a team working on this. You have to train people, you have to educate people on why this goal is an important goal. And so I think that all of those things take time. Some infrastructure within your organization has to be set up, resources have to be deployed.

 

0:06:21.5 JD: And so to be realistic but not so far off into the future, I think that six month to the three-year timeframe is a pretty good one. And then step three is to establish your next target condition. So along the way you're setting these intermediate sort of goals that you're working towards along the way ultimately to reach that challenge or direction. So I like the six month to three-year timeframe. And previously when we talked about set the challenge, almost all of our focus was on that quantitative goal. Like, how are we gonna quantify that challenge? And I talked about this chronic absenteeism example. But during this first step, I think it's also really helpful to take a couple sort of sub-steps. And one of them that we talk about in our organization is See the System.

 

0:07:16.8 JD: Like literally see the system in which we work. And I've found that it's really helpful to have some questions to ask the team that's working on this, questions like on the outset of, so we've set this challenge, we wanna reduce chronic absenteeism from 52% to 5%. Well, what is the largest system to improve when we're working towards that goal? What's the aim of that particular system? What are the opportunities for improvement within that target system? How will the opportunities for improvement be prioritized? Those are sort of four of the questions we ask at the outset of one of these improvement projects, when we're working towards one of these long-term goals. And when we've started asking those questions, at this point in the process, we formed an improvement team.

 

0:08:15.9 JD: For something like attendance, it's cross-disciplinary, meaning we have a number of people in a number of different roles. We have four school buildings in our school system, so there's representation from all four school buildings. So it's really a cross-functional team that's working through those particular questions. So I think that's really, really important. And then throughout the process, even if folks aren't a member of the team that's meeting on a regular basis, in this case it's a weekly team meeting, there are other people that are going to be important to gather information from; students and families, for example. That's an obvious one. Other staff members that aren't necessarily members of the team that are meeting every week, but that there are ways to gather their input as well.

 

0:09:04.5 AS: And in this...

 

0:09:05.5 JD: Oh, sorry, go ahead.

 

0:09:06.7 AS: In this graphic you show on the left, some people work on the system, some people work in the system, and then you've got your System of Profound Knowledge coach. Is that what you mean by improvement team or are you just highlighting some roles?

 

0:09:19.3 JD: Yeah. Well, so the improvement team members should be composed of each of those three groups. So in our particular team there's actually two people that have sort of like enough of the knowledge of the System of Profound Knowledge that they could, I'm one of them and then there's another person that also has that knowledge that's working on this project. And then there's a number of people that are working on the system, working in the system in different ways when it comes to this chronic absenteeism, this attendance problem.

 

0:10:00.3 AS: Okay.

 

0:10:01.3 JD: And really what we're thinking about during this See the System step in the process is it's really a funneling process. So, even within just the chronic absenteeism, student attendance realm, there's almost unlimited opportunities for improvement and many different areas where we could focus our efforts. And so we're trying to funnel down, focus down to the most vital areas that will have the biggest impact on this chronic absenteeism problem.

 

0:10:38.5 JD: And we're also really working hard to make sure that people on the team and will be impacted and responsible for implementing whatever this new theory is, this new improvement theory or this new improvement system, they're a part of the team from the get go. One reason is because they're close to the work and they're gonna inform a lot of the ideas that we try. And ultimately for things that are put in place, they're also the people that are gonna have to implement. So if they were a part of developing those solutions, they're much more likely to sort of be bought in from the get go versus just being told you're gonna do this new thing.

 

0:11:17.3 AS: Yeah.

 

0:11:17.9 JD: And I think something that I've found to be very, very critical, when you work with teams like this and you're answering those questions like I talked about, what system are we gonna focus on? What's the aim of that system? Is we have a series of tools that are gonna be known to a lot of people that listen to this podcast, maybe some of them are new to people, but we have this series of tools that we use that help us visualize the group's thinking. I'm a big believer when you're working in groups that you write stuff down and you have people react to it, right?

 

0:11:53.5 JD: You actually do that. You write it down, can be as simple as chart paper with post-its, and you paste that, you post that to a whiteboard or whatever, the wall, and everybody can see it as it's being made. And you can ask clarifying questions, you can ask for what people are thinking when they put up that idea or that brainstorm, and I just think it's really, really important. And so I think a key part of our improvement process here is that we've matched the tools to these guiding questions.

 

0:12:29.3 JD: So for this question, we use this tool or this tool because we know we use that particular tool, it's a good way to represent that particular question. So we've provided some of the structure because a lot of times people see the System of Profound Knowledge, and even when they start to grasp that as a management theory or a management philosophy, they need some type of structure beyond that to then work with the team in their organization to answer those questions to, like I said, aggregate the group's thinking and put in into something that's coherently understood by the whole organization.

 

0:13:12.6 AS: Right.

 

0:13:14.3 JD: So I thought we could just start with the first two questions. What is the largest system to improve and what is the aim of this system? So when we started working with this team on this chronic absenteeism problem this year, those were the first two questions that we asked. And we use a tool called a System Map to represent our system as we understand it. And then we also worked with the group to write an aim for this attendance work. So this may be a little small on the screen, but the target system is up here as attendance. And then we've written an aim as a group to figure out, oh, I can put this in Slide Mode. That'll make it a little bigger. We've written an aim as a group so that we all know like, what is it that we are aiming for as the attendance team? That's the first thing that we did.

 

0:14:19.7 JD: So for this particular system, this target system that says the aim of the United Attendance System is to first, or one, define strong attendance for students and staff. Two, ensure that students, families, and staff have a shared understanding of what it means to have strong attendance. And three, create and improve systems that identify and remove barriers to strong attendance for students and staff. So it's not necessarily, this is sort of a qualitative statement, there's not necessarily right and wrong answers, but as we were sitting with the group, it's about a 10-person improvement team. And we just took a Google form and said, what do you think the aim of this work should be? There was like one or two people that included staff, like, we should actually think about staff attendance in addition to student attendance, and the whole group was like, oh yeah, that's a really good idea.

 

0:15:15.9 JD: And so right from the get go, that became a part of the aim. And you can see how the inclusion of staff in that would maybe change the nature of the improvement effort. Again, it's not right or wrong, but it just, when you explicitly set that aim for the group, there's a much clearer guide for what you're gonna be working on. There's also things in here that may not have been obvious to people. So that first part of the aim is define strong attendance. So, many people may think, well, everybody knows what good attendance is in the school, but that's not true. That's really not true.

 

0:15:54.3 JD: And the problem is, the main problem is, is because when you think of what's good attendance in your brain, your mental model, what I've found with both students and adults is that there's an association with grade scales. And so when you think about a grade scale, if you get a 90% on a test, for most students would say that's a pretty good grade. In most systems, that's an A minus, right? And so when you say, well, what if a student has 90% attendance, is that good attendance? People say, oh yeah, 90% is good, but 90% means you're missing 10% of the year. So in a traditional 180 school day year, 180 times 10% is 18 days. So when you ask people, is it okay for students to miss 18 days? Almost 100% of them say, no.

 

0:16:45.0 AS: Yeah. One day every two weeks.

 

0:16:47.9 JD: Yeah. Basically one day every two weeks. But there's a disconnect between 90% and 18 days for people. And so one of the things that we're building as a framework that defines how many days is sort of reasonable and if you miss so many days per quarter or semester, what's that gonna add up to at the end of the year? And then how's that translate into a percentage? So just little stuff like that becomes more apparent when you start to write this stuff down, when you get input from the group. And then, so we have the aim. And then we also sort of represented the attendance system in the System Map. So on the left hand side of our System Map, we have the contributions and conditions of the attendance system. So there's things like Ohio Attendance Laws that we have to attend to.

 

0:17:37.9 JD: There are things like family dynamics and stability and beliefs, that's gonna contribute to attendance at our schools. There's sort of health perceptions, especially coming out of COVID. We trained people to sort of be more cautious and keep kids at home during the pandemic. Well, some of those habits took and there's sort of a different, for a lot of people there's a different interpretation of when you should keep a kid home versus when they should be in school. And so all of these things sort of contribute to what we're seeing in our system. And then there's all these core activities that we do that impact attendance. Now, one thing that's really interesting with the core activities is that when the group originally sat down, a lot of the core activities had to do very explicitly with our attendance systems.

 

0:18:28.8 JD: Like at what point is a student considered truant, truancy systems, the attendance tracking systems. And this is one of the areas where I pushed the group to think, well, what are all of the core activities that we do as a school that contribute to a kid coming to attendance, I think... Or sorry, coming to school, and effect their attendance. Now, could some of those attendance systems have an impact? Yeah. And you can see that attendance intervention systems makes up one of these sort of core activity boxes here in the middle. But there's many other things like for example, student and family onboarding. Like, how was a student brought and their family brought into the school community? Do they feel welcome? Do they make a connection to a staff member? Their school culture and trust, is it safe? Do I have friends in my classroom?

 

0:19:20.2 JD: There's academic systems. Are the classes engaging? Do I want to go to science class because we do experiments or whatever? And so you can see there's many other core activities just besides the core attendance systems that influence attendance. Some of the other ones we have on here are health and wellness and transportation, 'cause I've spoke before that that's a big problem in Columbus right now. And then, so we have these sort of inputs, the contributions and conditions, and then things happen while the kids are at school. And then there's these positive and negative outputs, right, that our system is producing. A positive output would be student academic growth, a negative on the flip side would be student academic stagnation.

 

0:20:04.8 JD: And then as you sort of go up from the outputs, you have to think about sort of how are we collecting feedback from our constituents, our families, our students. And then how do we use that information to design and redesign our system? And then it sort of loops back, sort of forms a feedback loop to the contributions and conditions. So this may seem like, is it worth the time or not? But it really is very helpful at the outset to sort of represent your system visually. So you have, this gives you an idea of all the different things going on in your system, in this case, our attendance system. And it also starts to give us ideas where we might focus our efforts because we have a, like a comprehensive picture of what these core activities are in our organization.

 

0:20:57.3 AS: When I look at this, I just think, no wonder it's so hard to improve. When you look at the system and all the different aspects and when you bring someone into this group and they say, wait a minute, we didn't think about staff or as an example, or wait a minute, we didn't think about transportation, oh, yeah. And then you realize like how big the system is, how interdependent it is and how difficult it is to really make lasting change. Whereas it's kind of just natural state that all these things fall into place and bring, I wouldn't say stagnation, but they bring the current state.

 

0:21:36.2 JD: Yeah. I mean, it certainly helps you understand the complexity level, right? Yeah.

 

0:21:43.7 AS: It also makes you think that once you've gone through this, you've got everything there, then you have to really think as a team, where can we have the most influence or what is the area that we think has the biggest, or is the leading item that if we could fix this, it can impact other parts, 'cause when you look at what you're putting up on the screen, it's just overwhelming. Like, we can't do all this.

 

0:22:11.4 JD: Nope. And we're not gonna focus on all of these things. So as a part of this improvement model and this improvement process that's sort of nested within the model, that's the whole purpose, is to sort of represent the whole system and then start to narrow the focus of the team. And that's a perfect segue because the next two questions that we're asking when we're trying to see the system is, what are the opportunities for improvement within the target system and how will the opportunities for improvement be prioritized? So probably many people on here will be familiar with something that looks like this. And again, there are certainly places for technological resources, but I will say the vast majority of the work I do with improvement teams is done with chart paper like this, post-it notes and a marker.

 

0:23:16.7 JD: You can be pretty powerful when you're armed with those tools when you have the System of Profound Knowledge philosophy. But basically what we did was just post that question right on top of the chart paper. What are the opportunities for improvement within the United Schools system of attendance? And then all of these blue and pink post-its, which you probably can't read, but those are just the people on the team answering those questions. Basically one idea per post-it note. And then we basically took those ideas and kind of grouped them categorically. And then once it sort of emerged what the categories were, they were given these labels, these category headings.

 

0:24:04.6 JD: So we have data, we have about four post-it notes about data, academic systems, communication. That's a big one, you can see has a lot of post-it notes, not surprising. Transportation, it's a pretty big category and that's on the top of everybody's mind in Columbus right now. We have sort of improvement and intervention ideas and then we have culture and engagement. So once we brainstorm, we categorize, put them in these affinity groups, basically. And then from there, and see, it's a tool. You can see this, this isn't just people sit around and talk. They have to write their ideas and then there's actually a tool that we use to organize those ideas. And then we store this in our project and meeting tracker so people can refer back to this. And then we use this information to help us complete subsequent steps.

 

0:24:56.2 AS: So was this a source of the chart that you showed us prior?

 

0:25:03.0 JD: When you say source of the chart...

 

0:25:06.3 AS: The layout of the system that you showed previously that had all these different things in it, did that originate from this discussion with post-it notes and putting stuff up on the wall and then later you put that into that, or is that different?

 

0:25:18.4 JD: System Map happened first. So I'm actually going in like sequential order. So the System Map happened first and then the affinity diagram.

 

0:25:26.3 AS: Now it's opportunities. Okay. Yep.

 

0:25:28.5 JD: Yep. Now it's opportunities. Once we understand what our system looks like, what are those opportunities for improvement? Then what we did is we said, how will the opportunities for improvement be prioritized? And we created this tool, it's called an Interrelationship Dye Graph. So we basically took the categories from the affinity diagram and those are these boxes here. So you have transportation, improvement and intervention, culture and engagement, academic systems, communication and data. And basically we systematically go around the circle and the first question we say is, is there a relationship between transportation and improvement and intervention? And if the group sort of consensus is, yep, those two things, there's a relationship, then we say, which one is impacting the other thing the most? And we draw an arrow away from that sort of dominant cause.

 

0:26:35.8 JD: So you can see transportation, that group thought transportation was having an impact on improvement and intervention. And then you just basically keep doing that for each of the categories. Then we compare transportation and culture and engagement. We ask those same two questions about what, is there a relationship and if so, what's the directionality of the relationship? We go around the circle until we've done that with every one of these categories. And then we basically tally up the number of "in" arrows versus the number of "out" arrows. And the thing with the most out arrows is sort of starred. And that's thought of as the dominant cause. Now, you have to know context, what this team is talking about and thinking about, 'cause there's not a lot of information just right on this page.

 

0:27:29.2 JD: But you might ask like, why is transportation having such an impact on improvement and intervention? Well, or why is transportation having an impact on culture and engagement? It's because the transportation system right now is so flawed in Columbus, buses aren't showing up at all sometimes, kids don't get picked up, so their parents have to bring them in. They're often an hour, two hours late to school. And so what people are saying is there's so many of those problems being caused by transportation right now that it's impacting the ability to do interventions with the kids, for example, or it's impacting the ability to build culture and engagement 'cause kids are constantly having to be making up school and making up work and those types of things. So...

 

0:28:16.8 AS: And in Thailand, when we have problems like that, the free market solves it because we can get, people can apply to transport through vans and other activities. But, well, is it a socialist state of Ohio these days or what's going on?

 

0:28:33.4 JD: Well, I would say there are various transportation possibilities. I would say there are no great systems for, you've been assigned and you've opted to ride the school bus and it doesn't come on a particular day, there's no sort of like on-demand on that particular day type of solution for the kid to take besides the parent taking them or that type of thing, so, unfortunately.

 

0:29:01.4 AS: Just a funny, a fun story. When I was a kid, we grew up, the piece of land that our development was built on was a piece of farmer Barlow's land that he sold off in parcels. And we worked in his farm on the summertime. And then he drove the bus to school during the wintertime. So see Farmer Barlow right there and you get on the bus and he showed up every single day. I don't think Farmer Barlow...

 

0:29:22.5 JD: I was gonna say...

 

0:29:22.8 AS: Ever missed a day.

 

0:29:24.3 JD: I know a lot of farmers and I'm assuming that bus driver was pretty reliable.

 

0:29:27.4 AS: Yeah.

 

0:29:28.3 JD: So yeah, unfortunately that's just not the case right now. So, but context's also important. So we actually have a few different improvement teams going on in our school system right now. And one of them is transportation. So as a group, we sort of decided like, while transportation is having a large impact this year, it's not the thing for this group to work on because there is a transportation improvement project also going on, but there's also an acknowledgement that there's gonna be overlap between those two things. And as the group talked about knowing the kids and knowing the students, it's certainly not the only reason kids are absent. And I think that someone in the group actually said even if transportation was perfect, we would still have this chronic absenteeism problem. So there are other things that we need to focus on and work on, so.

 

0:30:21.1 AS: Great.

 

0:30:21.3 JD: Those are just some examples of tools and you know, visualizing the thinking of the group and you have to know your system. You can't just sort of go straight off the tools, there's other sort of analysis that has to happen along the way. And so that's kind of where I wanted to go today. And I think sort of wrapping up See the System is where we're at now. So I can kind of wrap that up and we can go from there.

 

0:30:48.4 AS: Fantastic. If you had it in one simple statement that could wrap up what we talked about today, what do you think is the most valuable thing for people to take away from this discussion?

 

0:31:03.1 JD: From this discussion? I think there is a desire, an impulse almost to jump in and create solutions. But I think it's... What I've seen, it's very important to step back and make sure you really see the system and do a number of things around seeing the system. Next time we'll talk about defining the problem, to really make sure you do those things before you start jumping to solutions. And in fact, about 50% of the work or more in our improvement process occurs before we ever even talk about possible solutions. And I think it goes to that old Einstein quote where it's like, if I had an hour to save the world, I'd spend 55 minutes, make sure I understand and define the problem and five minutes. You know, putting that solution into action.

 

0:32:00.3 JD: I'm sure I'm messing up the quote, but you get the gist of it. So I think what we've seen is we set this challenge at the organizational level, that we have to understand the difference of those three faces of performance improvement or performance measurement. We wanna focus the challenge or direction, we wanna frame it as an improvement goal. And it's important to do that and it's important to make that ambitious, to quantify that. But then there's these other sort of qualitative things that we need to step back and do that allow us to see our system. And there's, you know, we went through some guiding questions and some related tools that can help put this into action with an actual team that you're working with in your system. And these tools are not education specific. You can use these in any type of organization.

 

0:32:45.0 AS: I'm gonna sum it up in five words. Slow down to speed up.

 

0:32:48.7 JD: I like that. Slow is smooth and smooth is fast.

 

0:32:52.7 AS: Smooth is fast.

 

0:32:54.3 JD: I like it. Yep.

 

0:32:55.4 AS: So John, on behalf of everyone at The Deming Institute, I want to thank you again for this discussion. And for listeners, remember to go to deming.org to continue your journey. You can find John's book, Win-Win: W. Edwards Deming, the System of Profound Knowledge and the Science of Improving Schools on amazon.com. This is your host, Andrew Stotz, and I'm gonna leave you with one of my favorite quotes from Dr. Deming, people are entitled to joy in work.