You can game most agile assessments in an afternoon.

I know because I used one of the leading ones for years. I ran it inside real organizations, with real teams, and I watched what it actually measured. It measured how well a company performs an audit. It told me almost nothing about how that company actually works. That gap, between "we documented it" and "we actually live it", is where transformations die. I have watched it happen enough times that I stopped complaining about the tool and built a replacement. This article explains why binary scoring fails, what gaming looks like in practice, why Leadership breaks first almost every time, and what a real assessment should surface.

The Problem With Binary Criteria

The assessment I used was built on a simple mechanism. For each practice, you checked whether it was observed, documented, or verbally confirmed. Hit two of three, and the box gets a check mark. Think about what that actually requires from an organization. Someone writes a document. Someone else tells the assessor "yes, we do that." Two of three. Check. The organization scores as agile on that criterion without a single behaviour changing. Binary scoring fails for a structural reason. Organizational health lives on a spectrum, and a checkbox flattens it into yes-or-no. A team that surfaces problems early and a team that buries them until they explode can both produce the same documentation. The checkbox treats them as identical. The data from a binary assessment is clean, easy to chart, and comfortable to present to executives. That comfort is the problem. Clean data about the wrong thing sends a transformation confidently in the wrong direction.

What Gaming Looks Like in Practice

Gaming rarely looks like fraud. It looks like preparation. Here is the pattern I saw repeatedly. An assessment gets scheduled. Middle managers get nervous, because scores flow upward and careers ride on them. So the two weeks before the assessment turn into a staging exercise.

• Wiki pages get written or refreshed the week before, purely so the "documented" box passes.
• Teams get briefed on what to say when the assessor asks about their practices. The "verbally confirmed" box passes.
• Ceremonies get performed with unusual discipline during the observation window. The "observed" box passes.

Nobody involved thinks of this as cheating. They think of it as tidying up before company arrives. And in real life, the moment the assessor leaves, the standups revert to status meetings, the retrospectives are skipped, and the documented process sits untouched on the wiki. The score says the organization matured. The organization stayed exactly the same. Leadership then makes investment decisions based on a number that describes a performance, and the transformation stalls for reasons nobody can see in the data. The maturity framing makes this worse. The tool I used claimed to measure maturity in agile environments, and the maturity model itself made no sense for how agile organizations actually evolve. I gave a talk at Agile 2023 on evolution versus maturity in agile spaces based on exactly what I found running that assessment. Maturity implies a fixed endpoint. Real organizations evolve continuously, and an assessment needs to show a path; a destination alone is useless.

Why Leadership Breaks First

After running assessments across organizations of every size, one pattern holds with remarkable consistency. Before I even start, I can predict where the deepest problem sits. Leadership. Almost every time. The reason is specific, and it has little to do with bad people. We have all been taught that the leader is the boss and that the title grants power. Plenty of people swear they will never operate that way. Then they get the role, and the system pulls them toward the power grabs and the prestige of the title, and they become the thing they promised to avoid. It works like gravity. The system rewards boss behaviour, so the system produces bosses. I am no boy scout on this. I fell into that trap myself a few times. What pulled me out was the moment right after, when I took a breath, felt like garbage, and recognized I had gone against my own values. I apologized to my teams directly. Those apologies hurt my credibility at first and built it stronger over time, because the team saw that a leader makes mistakes, owns them, and learns.

Your title means nothing. Trust, positive conflict, agreement, and accountability together get the results. The title means more responsibility, because now you are helping more people become better at what they do and who they are.

Here is why this matters for assessments. A checkbox instrument cannot see any of this. Decision-making behaviour, how power actually flows, whether accountability is real or performed- none of it shows up in a document or a scripted interview answer. So the assessment scores Leadership as healthy while Leadership quietly poisons every other dimension of the organization. Every downstream failure I diagnosed- blocked teams, competing priorities, buried problems- traced back to how leaders made and reinforced decisions. Measure Leadership wrong and every other number you collect becomes decoration.

What a Real Assessment Should Surface

The assessment took shape in my head starting in 2020, while I was still using a different one and watching its holes widen. I built it around eight pillars. Each one targets the living system underneath the org chart.

The Eight Pillars

• Leadership: How decisions get made, communicated, and reinforced to create clarity and accountability.
• Strategic Alignment: Whether priorities are clear, competing work is controlled, and the organization moves in the same direction.
• Team Enablement: Whether teams can execute without constant blockers or unnecessary interruptions.
• Value Creation: Whether effort translates into outcomes customers notice and appreciate.
• Trust and Transparency: Whether issues surface early for resolution or fester and explode later.
• Team Health and Growth: Whether the pace of work is sustainable and continuous learning is embedded in the culture.
• Strategic Budgeting: Whether funding supports priorities or quietly creates conflict against them.
• Delivery and Flow: Whether work moves through the system with minimal delays, bottlenecks, and rework, so teams finish what they start.

Notice what these pillars have in common. Almost none of them can be answered with a document. You have to look at behaviour over time. You have to compare what leaders say with what teams experience. You have to trace where money actually flows against where priorities supposedly sit, because Strategic Budgeting exposes contradictions faster than any interview. Trust and Transparency is a useful example. A checkbox asks whether a retrospective will take place. My assessment asks whether problems surface early. Those are different questions with different answers, and the second one predicts whether the organization survives its next crisis. The system is intensive by design, and it scales. It works the same whether the company has 10 employees or 5,999, because the pillars describe how humans organize work, and that holds at every size. Most importantly, it creates a path toward evolution rather than a maturity badge to hang on the wall.

Before You Commission Your Next Assessment

If you are a leader considering an agile assessment, ask the vendor three questions before signing anything. First: can a motivated middle manager improve our score without changing any behaviour? If the honest answer is yes, you are buying a performance review of your organization's acting skills. Second: does the instrument distinguish between documented and lived? Ask specifically how it detects the gap. If the answer involves counting artifacts, walk away. Third: does it examine how leaders actually make decisions, or does it stop at team-level ceremonies? Leadership breaks first. An assessment that skips it will hand you a diagnosis that misses the disease. The organizations that transform are the ones willing to see themselves clearly, including the uncomfortable parts. A checkbox protects you from that discomfort, but it costs you the transformation. Pick the instrument that tells you the truth. Then do the harder thing and act on it.