Quality is not something a team decides to care about. Every team cares. The teams that ship reliable software have built a system that makes carelessness hard, and that system is what separates one software development firm from another.

I run quality engineering, and that distinction is most of my job. When I assess a software development firm, I am not looking for enthusiasm about quality. I am looking for the mechanics.

Here is what those mechanics look like.

Bugs do not appear because engineers are careless. They appear because someone had two days for work that needed four.

So the first thing I look at is not the test suite. It is whether the team has ever moved a date to protect quality, and what happened to the person who suggested it.

A team that has never delayed anything has not been protecting quality. It has been hoping. The evidence shows up later, in the defect numbers.

Mostly small tests

Our suite is built from many fast tests and a small number of slow ones. Unit tests run in seconds. They tell you exactly what broke.

Teams that skip this layer end up with slow suites nobody runs before pushing. Then the feedback arrives an hour later, when the engineer has moved on.

Very few end to end tests

End to end tests are valuable and expensive. We keep about twenty. They cover the paths that would cost us money if they broke.

Every one we add slows the pipeline for everyone. So each new one needs a reason, and we remove them when the reason expires.

Flaky tests get one week

A test that fails at random is worse than no test. It teaches the team that red does not mean broken.

Our rule is simple. A flaky test gets fixed within a week or deleted. No exceptions, no long conversations. That single rule did more for our build trust than any tool we bought.

A gate that people routinely override is not a gate. It is a suggestion with extra steps.

Ours are deliberately few. The suite must pass. Coverage on new code must not fall. Someone other than the author must review. That is it.

Three real gates beat ten decorative ones. Ask any software development firm how many times a gate was overridden last month, and by whom. Any partner offering automation testing services should be able to tell you which gates they enforce and how often they are bypassed. The second number is the honest one.

You learn more in four weeks than in any procurement process.

Watch what happens the first time something breaks. Does the team find it, or do you? Is there a fix and a test, or just a fix? Does anyone ask why it was not caught earlier?

Watch the first estimate too. A partner who gives you a date without asking about your test environment has not thought about how the work will actually land.

None of this needs a formal assessment. It needs you to pay attention early, while changing course is still cheap.

Defect escape rate is the one I would keep if I could only have one. It measures the outcome rather than the effort.

AI writes a good share of our test code now. It is genuinely useful for the repetitive parts. It drafts cases from a specification, covers the obvious edges, and updates selectors after a design change.

What it does not do is decide what matters. It will happily generate two hundred tests for a screen that needed six.

The numbers suggest caution about the bigger claims. Gartner surveyed 782 infrastructure and operations leaders in late 2025 and found only 28% of AI use cases fully met ROI expectations, while 20% failed outright. Of those reporting a failure, 57% said they had expected too much too fast.

We now apply a plain AI ROI framework to every tool. What does it replace? How many hours is that today? What does it cost, maintenance included? We check at ninety days. Two of our tools passed that check and stayed. One did not and went.

The pattern I look for is a team that treats quality as part of building, not a stage after it.

Warning signs are easy to spot once you know them. A separate QA phase at the end. Testers who join in the final week. A definition of done that stops at "code complete."

Good signs are equally clear. Tests written alongside the feature. Engineers who can explain their own coverage. A partner who tells you a story shipped late because the tests were not ready.

That last one is the real test of any software development firm. It is uncomfortable to say and it means the standard is holding.

What coverage number should we require?

Be careful with coverage targets. They are easy to hit with weak tests. I care more about defect escape rate and whether the team trusts the build.

Who should own the test suite?

Whoever writes the code. Handing tests to a separate group creates a queue and removes the feedback from the person best placed to act on it.

Should we outsource testing completely?

You can outsource the work, not the standard. Many teams bring in automation testing services to build the framework and raise the level, then keep ownership internally afterwards.

A healthy quality system is quiet. Builds are green. Releases are dull. Nobody is heroically fixing something at midnight.

There is no demo for that, which makes it hard to fund and easy to erode. It is also the clearest signal you will get about how a software development firm really works.

Ask any partner what their last three releases looked like. Not the successes. The releases. If the honest answer is "nothing much happened," you have found a team that has built the system rather than one that talks about caring.