Field notes · 5 min read
How do you know it's working?
The steering group slide is all green. Training sessions delivered: on track. Communications sent: on track. Go-live date: met. Then someone walks the floor, or listens in on a team call, and finds people doing the work much the way they did it before.
Both things can be true. The programme did what it said. The change hasn't happened yet. The gap is usually in what was being measured.
Three different questions
It helps to separate three questions that often get blurred together:
Did we do what we planned? Sessions run, messages sent, systems switched on. This is activity.
Are people using the new way of working, and using it well? This is adoption.
Is it making the difference we wanted? This is the result.
In our experience, programme reporting is usually strongest on the first question. The second and third are harder to measure and slower to show up, so they get less attention.
Researchers have been clear about the middle question for a long time. One influential paper defined successful implementation as "the consistency and quality" of people's use of something new. It described a range of responses: resistance, avoidance, going through the motions, and real commitment (Klein and Sorra, Academy of Management Review, 1996). Someone can attend every session and still be at the avoidance end.
Health services researchers went further and listed eight separate implementation outcomes, kept apart from the end results: whether people find the change acceptable, take it up, think it fits, find it workable, use it as intended, what it costs, how far it has spread, and whether it lasts (Proctor and colleagues, Administration and Policy in Mental Health, 2011). It comes from healthcare, but the lens works well elsewhere. It gives you words for "it's live, but is it real?"
Early signals you can measure
Results take time. Readiness and commitment can be measured much earlier, and they give leaders something to act on while there is still time.
One well-tested questionnaire, developed with more than 900 people in public and private organisations, found that readiness for change rested on four beliefs: that I can do this, that this is the right change for the organisation, that leaders are really behind it, and that it will be good for me (Holt and colleagues, Journal of Applied Behavioral Science, 2007). Each of those is a question you can ask, and each points to a different fix if the answer is weak.
At the level of the whole organisation, readiness has been described as two shared things: a shared resolve to make the change happen, and a shared belief that together, people can pull it off. When both are high, people are expected to start sooner, try harder and keep going when it gets difficult (Weiner, Implementation Science, 2009). That is a theory rather than a tested finding, but it matches what the questionnaires pick up.
The kind of commitment matters too. In studies with nurses, commitment to a specific change predicted support for it better than general loyalty to the organisation. And commitment that came from believing in the change, or feeling a duty to support it, went with more support than commitment from feeling there was no other option (Herscovitch and Meyer, Journal of Applied Psychology, 2002). "They have no choice" is not the same as "they're on board."
Feelings belong in the picture as well. A review of readiness research argued that it has an emotional side, not only beliefs, and that it works at the level of individuals, teams and the whole organisation (Rafferty, Jimmieson and Armenakis, Journal of Management, 2013). A team can believe a change is right and still feel worn out by it.
Plain measures
For every measure on a change dashboard, we suggest writing three short lines:
What we count. For example: the share of team leads using the new planning process for their weekly review.
How we measure it. For example: a check of the planning tool each month, plus a two-question pulse to the teams.
What success means, and by when. For example: four in five team leads using it by the end of the second quarter, with most of their teams saying it saves them time.
If a measure can't be written that plainly, it is hard to act on. In our own work we use team check-ins and culture insight tools such as CultureLens for the people side of this, but any honest, regular way of asking will do.
What we suggest instead
Report activity, adoption and results separately.
Don't let a green activity line stand in for the other two.
Measure readiness early.
Ask the four readiness questions before and soon after launch. Where scores are low, the type of belief tells you what to fix.
Ask about commitment, not just awareness.
"Do you know about the change?" tells you little. "Do you believe in it?" and "Do you feel you have a choice?" tell you more.
Look for use in the work itself.
Where you can, check what people actually do: the tool, the meeting, the decision. Self-reports are useful, but they are not the whole picture.
Keep measuring after go-live.
Lasting change is one of the outcomes. Plan to look again at six and twelve months.
Write every measure in plain words.
What we count, how we measure it, what success means. If the team can't understand it, it won't guide them.
The short version
Green status reports often measure whether a programme did what it said, not whether anything changed. The research gives leaders practical things to watch: how people use the change, how ready they are and what kind of commitment they feel. They can be measured early, plainly and often.
So, at your next steering group: which line on the slide tells you people are working differently?
Sources and further reading
- Klein, K. J. and Sorra, J. S. (1996). The challenge of innovation implementation. Academy of Management Review.
- Proctor, E. and colleagues (2011). Outcomes for implementation research. Administration and Policy in Mental Health and Mental Health Services Research.
- Holt, D. T., Armenakis, A. A., Feild, H. S. and Harris, S. G. (2007). Readiness for organizational change. Journal of Applied Behavioral Science.
- Weiner, B. J. (2009). A theory of organizational readiness for change. Implementation Science.
- Herscovitch, L. and Meyer, J. P. (2002). Commitment to organizational change. Journal of Applied Psychology.
- Rafferty, A. E., Jimmieson, N. L. and Armenakis, A. A. (2013). Change readiness: A multilevel review. Journal of Management.
Sources are cited from published research and industry reporting. Copies are held on file and available on request.