How to Evaluate an Outsourcing Vendor's Code Quality Before You Sign

Most vendor evaluations for outsourced engineering work stop at the case study and the reference call. Both are useful, but both are curated — the vendor picked the case study, and the reference is someone who agreed to say nice things. If you want a real signal on whether a team can write and maintain production code, you have to ask for artifacts that are harder to stage: recent pull requests, architecture decision records, and actual test coverage reports, not the summary slide about them.

Ask for sample pull requests, not portfolio pieces

A portfolio piece is retrospective and polished. A pull request from the last 90 days, ideally from an active client engagement with identifying details redacted, tells you how the team actually works day to day. When we're on the other side of this conversation — being evaluated as the vendor — the request we respect most is: "show me three PRs, one that got a lot of review comments, one that got approved quickly, and one that got reverted or hotfixed."

What to look for:

• PR size. Consistently 500+ line diffs suggest infrequent integration and higher review-skip risk. Most healthy teams keep the median PR under 200-300 lines.
• Review comment quality. Comments like "LGTM" on every PR are a red flag. Comments that ask "what happens if this call times out" or "this duplicates logic in X, can we extract it" indicate reviewers who actually read the diff.
• The reverted PR. How was the revert handled — quietly, or with a written note on what went wrong and what test would have caught it? The second is the sign of a team that learns from incidents instead of just moving past them.

Architecture decision records reveal how a team thinks, not just what it built

Code shows you the output. ADRs show you the reasoning — and reasoning quality is a much better predictor of how a team will handle the decisions you haven't thought of yet. A useful ADR names the decision, the context that forced it, at least one alternative that was seriously considered, and the trade-off accepted. If a vendor can't produce any ADRs, that's not necessarily disqualifying for a smaller engagement, but it does mean architectural decisions are living only in people's heads or in Slack, which doesn't survive staff turnover.

When reviewing a vendor's ADRs, three questions separate strong from weak:

1. Does the record show a genuine alternative that was rejected, with a stated reason — or does it just justify what was already decided?
2. Is there a revisit trigger — a condition under which the team said "we should reconsider this if X happens"? Teams that write these tend to actually catch the moment X happens.
3. Are ADRs referenced later, in code comments or subsequent ADRs? If they're written once and never touched again, they're theater, not practice.

Test coverage reports need context, not just a percentage

A vendor quoting "92% coverage" without qualification is telling you almost nothing — line coverage on getters and setters counts the same as coverage on the payment reconciliation logic. Ask instead for:

• Coverage broken down by module, so you can see if the critical path (auth, payments, data integrity) is actually well-tested versus the coverage number being propped up by trivial code.
• Whether coverage includes integration tests or only unit tests. A codebase with 95% unit coverage and no integration tests can still ship a broken API contract.
• Mutation testing results, if they exist. Few vendors run mutation testing, but a vendor that does is signaling a level of rigor worth noting — it measures whether tests actually catch bugs, not just whether lines execute.

A red flag worth naming directly

Be wary of coverage numbers that appeared suddenly right before the sales conversation. Ask when the reporting was set up and whether it's tracked continuously in CI, not generated on demand. A dashboard that's been running for a year and shows a coverage trend line is far more credible than a single point-in-time screenshot.

Running a live technical exercise changes the conversation

The most reliable signal we've seen clients use — and the one we welcome most when we're being evaluated — is a paid, time-boxed exercise: a small real (or realistic) ticket, reviewed the same way the vendor reviews production code, with the client sitting in on the review. It costs a few thousand dollars and a few days, and it tells you more than three rounds of reference calls. You see actual review comments exchanged in real time, actual test-writing habits, and how the team responds to pushback on their own design choices.

None of these checks are exotic. They're the same rigor a strong engineering manager applies when interviewing a senior hire, applied to a vendor relationship instead of an individual. The vendors worth working with won't flinch at the request — they'll usually have the artifacts ready before you ask.