How to Verify a Marketing Agency's Case Studies Before You Sign

How to Verify a Marketing Agency's Case Studies Before You Sign

Verify an agency case study by working backwards from the headline number. Ask what the baseline was, what date range it covers, and whether a named person at that client will confirm it on a fifteen-minute call. Most case studies stop at the percentage. A percentage with no starting number is not evidence.

By Avi Vatsa, CEO, Exchange Four Agency. Avi leads the team that installs and runs AI-leveraged revenue systems for owner-led companies. His background spans law, technology, and marketing; he also co-founded Dialora, an AI voice-agent platform for automated lead capture and booking. (Background sourced from Marketer of the Day #1411 and the Jeremy Ryan Slate Show.)

Why don't agency case studies survive scrutiny?

Because most of them were written by a marketer, for a sales deck, from a dashboard screenshot — not reconciled against the client's books.

The structural problem is that the agency usually doesn't have access to the one system that would prove the claim. They can see sessions, form fills, cost per lead, maybe opportunities created if the CRM was shared. They often cannot see closed-won revenue, refunds, churn, or margin. So the case study reports what the agency could measure rather than what the owner cared about, and the gap gets papered over with a percentage.

This is the same gap that shows up in the research. An industry survey distributed via Businesswire found 71% of brands report frustration demonstrating marketing ROI effectiveness. That number is not describing bad agencies specifically. It's describing an industry in which attribution is genuinely hard and almost nobody has built the plumbing to do it properly. A case study is that same unsolved problem, printed on a nicer background.

Your job as the buyer is not to catch a liar. It's to find out whether the number was ever connected to a dollar in the first place — which is a different, much more answerable question. If you want the mechanics of doing that on your own account, read how to calculate marketing ROI and why most agency dashboards never tie to a dollar.

What should you check in every case study?

Work through these in order. The early ones kill most claims before you've spent any political capital asking for a reference.

1. Find the baseline. "312% increase in leads" from a starting point of 8 leads a month is 25 leads a month. That may still be fine. But you need the absolute numbers at both ends, and if the agency won't give them, the percentage was chosen precisely because the absolutes are unimpressive.
2. Find the date range. A result over 24 months is a different product than the same result over 90 days. Ask when the engagement started, when the number was captured, and whether the client is still a client. An undated case study is the oldest trick in the file.
3. Separate the metric from the money. Traffic, impressions, and MQLs are inputs. Ask what happened to closed revenue in the same window, and if the agency says they don't have that figure, that's an honest and useful answer — note it and move on. What you've learned is the limit of what they measured.
4. Check what else changed. Did the client also raise prices, hire two salespeople, launch a product, or benefit from a competitor going under? Good agencies volunteer this. It's the single fastest test of intellectual honesty you have.
5. Ask who did the work. Named senior people, or a pod that has since been reorganized? If the team that produced the result no longer exists in that configuration, you are buying a logo, not a capability. The related pattern is covered in what account manager turnover actually signals.
6. Request the reference, and specify the person. Not "a client" — the operator named in the study, or the person who held the budget. The substitution you get offered instead tells you a great deal.

What does each common claim actually require as proof?

That last row deserves attention, because it's the most common case study in the industry and the one owners most often mistake for proof. A redesigned homepage is a deliverable. It is not a result you can point to.

How do you run the reference call so it's worth doing?

Most reference calls are theatre. The agency hands you a friendly client, you ask "were you happy?", they say yes, and nobody learns anything.

Ask different questions. "What was the first 90 days like?" surfaces onboarding chaos better than any direct question about quality. "How many people have you worked with there, by name?" surfaces turnover. "What did you have to do yourselves that you expected them to do?" surfaces the scope gap — and this is where real reviews live. One G2 reviewer, writing about an agency that is not ours, put it plainly: "the inability to help solve issues... made it so we ended up doing much of the transition ourselves." Another described "constantly changing project managers and the issues with communication." Neither of those failures would ever appear in a case study. Both are the thing that actually went wrong.

Then ask the money question in the client's own frame: "Did you renew, and why?" A client in year three with an expanded scope is worth more than any chart.

For the full question set to run before you sign, see the questions to ask a marketing agency whose answers you can check and the broader owner's diligence checklist.

What does a case study look like when it's real?

It names the client. It gives absolute numbers at both ends with dates attached. It says what else was happening in the business at the time. It usually admits something didn't work in month two. And it quotes a real person with a real title who you could find on LinkedIn in ten seconds.

Here's the standard we hold ourselves to, which is also the standard I'd apply to anyone pitching you: we publish the client's own words, attributed, and we don't publish an outcome number we haven't proven. Riggs Eckleberry, Chairman of OriginClear, describes the work as: "They know exactly how to connect marketing execution to real business outcomes." Nishant Bijani, Co-Founder and CTO of Dialora.ai, calls it "a turning point in our revenue trajectory." Those are real voices, named and checkable. They are deliberately not dressed up as a percentage, because a number we can't independently prove is worth less than a client who'll pick up the phone.

Hold every agency to that. Including us.

What if the agency has very few case studies at all?

This objection comes up constantly, and the honest answer is that a thin portfolio is not automatically a problem. A boutique team of senior people might have nine clients, six of whom are under NDA, two in regulated industries that will never approve a public write-up, and one who'd happily take your call. That's a better situation than an agency with forty glossy studies and a 14-month average client lifespan.

What should worry you is a specific combination: lots of case studies, none with absolute numbers, none with named references, and a sales process that moves you toward signature before you've spoken to a single past client. That pattern is the templated-agency signature described in how to spot a cookie-cutter agency before you sign.

The deeper point is about what you're buying. Diligence on case studies only tells you about the past. The contract tells you about the future, which is why the structure matters more than the portfolio: does the agency own the outcome, or own a scope? We cover the difference in what changes when an agency owns the outcome instead of the scope, and the practical test in how to test a marketing agency before you sign a 12-month retainer.

Is hiring an agency even the right move here?

Worth asking before you spend three weeks verifying anyone's portfolio. Running several specialist vendors instead looks cheaper per line item and usually isn't: vendor-management research puts coordination overhead at 8–15% of annual vendor spend in hidden internal labor that never appears on an invoice, with businesses reporting roughly 30% higher total cost versus one integrated partner. That's the bill nobody quotes you, broken down in what using multiple marketing vendors actually costs.

The honest comparison of every route — in-house, fractional, multi-vendor, installed system — is in alternatives to hiring a marketing agency, and what each one costs. Read that first if you're still deciding the shape of the solution rather than the supplier.

The one check that settles it

If you do nothing else: pick the single most impressive number on the agency's site, and ask them to walk you through how it was calculated, from the source system to the figure on the page.

A team that built the measurement will answer in about ninety seconds, including the caveats. A team that didn't will reach for context, defend the methodology in general terms, or offer to "get you the full deck." You'll know which one you're dealing with before they finish the first sentence, and that's the entire exercise.

Once they're hired, the same discipline applies monthly. The metrics an owner should demand every month is the version of this check you run forever, not once.