COSO Assessment: Evaluating Control Effectiveness

The CFO looked at me across the conference table with exhaustion written all over his face. "We've implemented every control our auditors recommended," he said. "We've spent $2.3 million over eighteen months. We have policies for everything. But I still can't sleep at night because I honestly don't know if any of it actually works."

That conversation happened in 2017, and it perfectly captures the challenge most organizations face with internal controls. Having controls isn't the same as having effective controls. And in my fifteen years of conducting COSO assessments, I've learned that the difference between the two can mean the difference between a company that thrives and one that becomes a cautionary tale in the Wall Street Journal.

What Nobody Tells You About Control Effectiveness

Here's something that might surprise you: most organizations have plenty of controls. What they lack is evidence that those controls actually work.

I once assessed a financial services company that had 347 documented controls. Impressive, right? Except when we tested them, we found:

• 94 controls that nobody was actually performing

• 67 controls that were performed but not documented

• 52 controls with such vague procedures that different people did them completely differently

• 31 controls that were technically being performed but wouldn't actually prevent or detect the risks they were designed to address

Only 103 controls—less than 30%—were both effective and properly documented.

The company thought they had robust internal controls. What they actually had was an expensive illusion of control.

"Control effectiveness isn't about what you've documented. It's about what you can prove actually works when it matters most."

Understanding COSO: The Framework That Changed Everything

Before we dive into assessment, let's get grounded in what COSO actually is and why it matters.

The Committee of Sponsoring Organizations (COSO) framework was developed in the 1990s as a response to a wave of corporate fraud scandals. It's since become the gold standard for internal control evaluation worldwide. If you're a public company, your auditors are using COSO to assess your controls. If you're implementing SOX compliance, you're living and breathing COSO whether you know it or not.

The framework consists of five interconnected components:

I learned early in my career that most control failures aren't because organizations don't understand these components. They fail because organizations treat COSO as a compliance checkbox rather than a living, breathing system.

The Real Purpose of Control Assessment

Let me share a story that fundamentally changed how I think about control assessment.

In 2019, I was brought in to assess controls at a rapidly growing tech company. They'd just raised $50 million in Series B funding and were preparing for an eventual IPO. Their board wanted assurance that internal controls were "ready for the big leagues."

The CEO pulled me aside before we started. "Look," he said, "I know we need to do this for the investors. Just tell us what boxes to check so we can get back to building the business."

Six weeks into the assessment, we discovered their revenue recognition process had a fundamental flaw. Due to the way they'd structured their subscription contracts and the controls around contract modifications, they'd been overstating revenue by approximately 12% for the past eighteen months.

The CEO's face went white when I showed him the analysis. "This could kill the company," he whispered.

But here's the thing: finding that issue before they filed for IPO potentially saved the company. If it had been discovered during SEC review or after going public, the consequences would have been catastrophic—class action lawsuits, regulatory penalties, complete loss of investor confidence.

That's when the CEO understood: control assessment isn't about checking boxes. It's about finding problems before they become disasters.

"A control assessment is like a medical checkup. The goal isn't to prove you're healthy. The goal is to find anything that might kill you while there's still time to fix it."

The Four-Phase Assessment Methodology That Actually Works

After conducting over 60 COSO assessments across industries ranging from healthcare to manufacturing to financial services, I've developed a methodology that actually reveals whether controls are effective. Here's how it works:

Phase 1: Understanding the Business (Weeks 1-2)

Most assessors skip this phase or give it lip service. Big mistake.

You cannot assess control effectiveness without understanding what the business actually does, how it makes money, and what could go wrong. I spend the first two weeks of every assessment doing nothing but learning:

Key Activities in This Phase:

I once assessed a healthcare provider where the documented patient intake process bore almost no resemblance to what actually happened at the front desk. The documented process took 27 minutes. The actual process took 6 minutes because staff had developed workarounds to handle patient volume.

Those workarounds? They completely bypassed insurance verification controls, resulting in over $4 million in unbilled services annually. Nobody had caught it because auditors only looked at documented procedures, not actual practice.

Phase 2: Control Identification and Documentation Review (Weeks 3-4)

This is where we inventory what controls actually exist and how they're supposed to work.

Here's my systematic approach:

Control Inventory Framework:

For Each Business Process:
├── Identify inherent risks
├── Document management controls
├── Map process-level controls
├── Review supporting technology controls
├── Assess control design effectiveness
└── Identify control gaps

Critical Questions I Ask:

1. What could go wrong in this process? (Risk identification)

2. What's supposed to prevent or detect it? (Control identification)

3. Who performs the control? (Control ownership)

4. How often is it performed? (Control frequency)

5. What evidence is created? (Control documentation)

6. Who reviews the evidence? (Control oversight)

I worked with a manufacturing company that had beautiful control documentation—flowcharts, narratives, the works. But when I asked, "Who actually performs this control?" the answer was often, "I think Sarah does... or maybe it's John's team?"

Ineffective controls almost always have unclear ownership. If nobody knows whose job it is to perform a control, I can guarantee you it's not being performed consistently.

Phase 3: Control Testing (Weeks 5-8)

This is where the rubber meets the road. We're going to prove whether controls actually work.

I use a risk-based sampling approach that focuses testing efforts where they matter most:

Control Testing Matrix:

Real-World Testing Example:

I assessed purchase order approval controls at a distribution company. The control stated: "All purchase orders over $10,000 require CFO approval."

Initial Test (Sample of 30 POs over $10,000):

• 28 had CFO signature

• 2 had no approval

Looks pretty good, right? 93% effectiveness?

But then I dug deeper:

• Of the 28 with signatures, I checked dates: 7 were signed AFTER the goods were received

• I checked against email records: 4 showed the CFO was on vacation when they were supposedly signed

• I interviewed the CFO: He admitted he "often signed stacks of POs without really reviewing them"

Actual effectiveness? Maybe 60%.

This is why testing methodology matters. Surface-level testing gives you surface-level confidence.

"Testing controls isn't about proving they work. It's about proving they work when you're not looking."

Phase 4: Evaluation and Reporting (Weeks 9-10)

The final phase is where we make sense of everything we've found and communicate it in a way that drives action.

Control Effectiveness Rating Framework:

The Five Control Effectiveness Tests That Reveal Everything

In my years of assessment work, I've found five specific tests that reveal more about control effectiveness than dozens of standard procedures:

Test 1: The "Surprise Me" Test

Pick a random Tuesday. Show up unannounced. Ask to see evidence that a control was performed yesterday.

If people can produce the evidence immediately, the control is probably working. If they need to "pull some things together," you've got a problem.

I did this at a financial institution with daily cash reconciliation controls. When I showed up at 10 AM and asked for yesterday's reconciliation, the controller said, "Oh, we usually do that on Friday for the whole week."

The control that was supposed to be performed daily was actually performed weekly. That's not a minor deviation—it's a fundamental control failure that could mask problems for days.

Test 2: The "What If" Scenario

Present a hypothetical failure scenario and ask: "How would our controls catch this?"

Example: "What if an employee started submitting fake expense reports for $800 each—just under the $1,000 approval threshold?"

The answers reveal whether controls are designed to catch realistic threats or just check compliance boxes.

At one company, when I posed this scenario, there was a long silence. Finally, someone said, "I guess we'd catch it in the annual audit?"

Translation: They had a 12-month window to steal money before anyone would notice.

Test 3: The "New Person" Test

Could a new employee, using only documented procedures, perform the control correctly?

I've found that controls requiring "tribal knowledge" or relying on "That's just how we've always done it" are controls living on borrowed time. When that key employee leaves (and they always do), the control collapses.

Test 4: The "High-Pressure" Test

How do controls perform during month-end close, quarter-end crunch, or system outages?

I assessed a SaaS company with beautiful change management controls—proper approvals, testing requirements, rollback procedures. But during a critical product launch, every single control was bypassed to "move faster."

If your controls disappear the moment things get difficult, they're not controls—they're suggestions.

Test 5: The "Evidence Trail" Test

Can you reconstruct exactly what happened, when it happened, who did it, and who approved it—without asking anyone?

This is the ultimate test. If you can't prove a control happened by following the evidence trail, external auditors won't believe it happened either.

Common Control Weaknesses I See Everywhere

After assessing controls at over 60 organizations, certain patterns emerge. Here are the most common control weaknesses I encounter:

Top 10 Control Effectiveness Issues:

Real-World Assessment: A Case Study

Let me walk you through an actual assessment I conducted for a mid-sized healthcare provider (details modified for confidentiality).

Background:

• $250M annual revenue

• 1,200 employees

• Processing 500,000+ patient visits annually

• Preparing for potential acquisition

Initial Scope: Assess internal controls over financial reporting and HIPAA compliance.

Week 1-2 Findings:

During process walkthroughs, I noticed something interesting. The revenue cycle—from patient visit to cash collection—involved 17 different systems and 23 handoffs between departments.

More concerning: when I asked different departments to describe the revenue cycle, I got 23 different answers.

Week 3-4 Documentation Review:

The organization had documented 127 controls across the revenue cycle. On paper, it looked comprehensive. But diving deeper revealed issues:

Control Documentation Analysis:

Notice anything? The areas with the worst documentation (insurance verification, denial management) were also the areas where revenue leakage was most likely.

Week 5-8 Testing Results:

I tested 340 control instances across the revenue cycle. Here's what we found:

Insurance Verification Controls:

The policy stated: "Insurance eligibility must be verified within 24 hours of scheduling."

Test Results (Sample of 60 scheduled appointments):

• 23 verified within 24 hours (38%)

• 19 verified after patient arrival (32%)

• 18 never verified at all (30%)

Impact: Approximately $6.2 million in denied claims annually due to insurance verification failures.

Charge Capture Controls:

The control required: "Clinical documentation must be reviewed for charge capture completeness within 48 hours of visit."

Test Results (Sample of 50 patient visits):

• 31 reviewed within timeframe (62%)

• 12 reviewed late (24%)

• 7 never reviewed (14%)

Impact: Estimated $3.8 million in unbilled services annually.

Week 9-10 Reporting and Recommendations:

I presented findings using this framework:

Control Effectiveness Summary:

Total Identified Revenue Impact: $14.1 million annually

The CFO was stunned. "We knew we had some issues, but we had no idea the magnitude."

Six months later, after implementing our recommendations:

• Revenue cycle controls redesigned with clear ownership

• Insurance verification moved to point of scheduling with automated system checks

• Charge capture automated with daily exception reports

• Denial management centralized with dedicated team

Result: They recovered $8.3 million in the first year and positioned themselves for a successful acquisition at a $40 million higher valuation than initially projected.

"Control assessment isn't an expense—it's an investment that pays for itself many times over when you actually fix what you find."

Technology's Role in Control Effectiveness

Let me be blunt: manual controls are dying, and they should be.

Every manual control is a control waiting to fail. People get busy. People get distracted. People leave the company. People develop creative workarounds.

Here's a comparison I show every client:

Manual vs. Automated Control Effectiveness:

I assessed a company that manually reviewed expense reports for policy compliance. Each review took 12 minutes. With 2,000 expense reports monthly, that's 400 hours of manual work.

We implemented automated policy checks that flagged 95% of issues instantly. The review time dropped to 2 minutes per exception. Total monthly time: 40 hours—a 90% reduction.

More importantly: the automated control caught 100% of policy violations. The manual control? Testing showed it caught about 67%.

But here's the critical caveat: Automated controls can fail spectacularly if they're poorly designed.

I once found an automated purchase order approval control that had been configured incorrectly. For 14 months, it had been auto-approving instead of routing for approval. Nobody noticed because the control was "automated" so everyone assumed it was working.

Building a Culture of Control Effectiveness

The best controls I've ever seen aren't the most sophisticated—they're the ones where everyone understands why they matter.

I worked with a manufacturing company where the production floor had a near-religious devotion to quality controls. Not because management demanded it, but because five years earlier, a control failure had led to a product recall that almost bankrupted the company.

Everyone who worked there during that recall remembered it. New employees heard the stories. The quality controls weren't seen as bureaucratic requirements—they were seen as the thing that stood between the company and disaster.

That's the culture you need to build.

Elements of a Control-Conscious Culture:

The Assessment Report That Actually Drives Change

Most control assessment reports are terrible. They're 80 pages of dense findings that nobody reads and nothing changes.

Here's how I structure reports to drive action:

Executive Summary (1-2 pages):

• Overall control environment assessment

• Summary of critical findings (rated by financial impact)

• Three most important actions needed

Control Effectiveness Dashboard (1 page):

Visual representation showing:

• Overall effectiveness by business process

• Trend analysis (if repeat assessment)

• Heat map of risk areas

• Progress on prior findings

Detailed Findings (organized by business process):

For each finding:

1. Control Description: What's supposed to happen

2. What We Found: What's actually happening

3. Impact: Why it matters (quantified when possible)

4. Root Cause: Why it's failing

5. Recommendation: How to fix it

6. Management Response: Their plan and timeline

Implementation Roadmap (1 page):

Prioritized action plan with:

• Quick wins (implement within 30 days)

• Critical fixes (implement within 90 days)

• Strategic improvements (implement within 6-12 months)

Common Assessment Pitfalls (And How to Avoid Them)

After watching dozens of control assessments go sideways, here are the mistakes I see repeatedly:

Assessment Failure Modes:

Your Assessment Action Plan

If you're ready to assess your control effectiveness, here's your roadmap:

Month 1: Planning and Scoping

• Define assessment objectives and scope

• Identify key business processes and risks

• Assemble assessment team

• Develop assessment methodology

• Create communication plan

Month 2: Documentation and Design Review

• Inventory existing controls

• Review control documentation

• Assess control design effectiveness

• Identify obvious gaps

• Prepare testing plan

Month 3: Testing and Evaluation

• Execute control tests using risk-based sampling

• Document exceptions and failures

• Interview control owners

• Analyze root causes

• Develop preliminary findings

Month 4: Reporting and Remediation Planning

• Finalize findings and recommendations

• Present to management and board

• Develop remediation roadmap

• Assign ownership and deadlines

• Establish monitoring and follow-up process

A Final Truth About Control Effectiveness

I want to end with something important: perfect control effectiveness is a myth.

In fifteen years, I've never seen an organization with 100% effective controls. And you know what? That's okay.

The goal isn't perfection. The goal is:

1. Know where your controls are working (so you can maintain them)

2. Know where they're not (so you can fix them)

3. Understand your residual risk (so you can make informed decisions)

I've seen organizations drive themselves crazy chasing perfect control effectiveness. They create controls for controls. They test endlessly. They document obsessively.

Meanwhile, their business suffers because nobody has time to actually work.

The best control environments I've seen are ones where:

• Critical controls are highly effective (99%+)

• Important controls are generally effective (95%+)

• Lower-risk controls are reasonably effective (85%+)

• Everyone knows where the gaps are and what they mean

Remember: The CFO who couldn't sleep at night? After our assessment, he told me, "I still don't have perfect controls. But now I know exactly what keeps me up at night and what doesn't. That's worth everything."

"Control effectiveness isn't about eliminating all risk. It's about knowing exactly which risks you're taking and making sure they're the right ones."

Control assessment done right transforms organizational confidence. It turns "I think we're okay" into "I know where we stand." It converts vague anxiety into specific action plans.

That's the power of truly understanding whether your controls work.

Now go find out if yours do.