How to evaluate the success of your software engineering relationship

Externalizing your application development to a partner outside of your organization is a challenging and often time-consuming process, but fortunately for you, that part is now over.

You found a good engineering company to help you meet your business objectives by developing smart software solutions. Now you may want to measure the success, to evaluate your partner’s main activity, which is application development. However, if your plans are long-term, you should not only look at bits and bytes but also overlook the partnership angle of this new business relationship.

Intro to code quality audit

There are several methods to audit code quality. And most of them are measurable.

If you represent a software company yourself, you will know how to evaluate this, as the KPIs (Key Performance Indicators) are usually the software industry recommendations. Therefore, you will likely look at both Qualitative and Quantitative Aspects of the code.

You may be interested in static code analysis; you’ll want to review the Reliability of the code, the Maintainability Index, the Analyzability, Testability, Stability, Portability, and Security aspects of the code, or the level (and quality) of the documentation provided.

If metrics are what you’re after, you’ll find automated tools to help you measure the cyclomatic complexity, the depth of inheritance, or the class coupling, just to give some examples.

But we will cover code quality in a different post.

In this article, we aim to cover the partnership aspect of the Buyer’s relationship with their software engineering Provider.

Questions to ask yourself before evaluating your partner

You may have found yourself asking these questions:

• How does it “feel” working with my software partner?
• Do they care?
• Are they proactive?
• Are they actively looking for ways to innovate? Or at least to improve?
• Are they interested in finding ways of saving us money? (when possible)
• Do they think long-term? Or are they more interested in quick wins?
• Are they good at planning? (proposing the right steps, engagement plan, assessing risks, providing realistic estimates, etc.)
• Are they honest?
• How good is the delivery team (e.g., professionally competent)
• How good is their management? (do they have a strong leadership team?)
• Do they depend too heavily on our business? Or another client’s business?
• Do they care for their staff?
• Is their staff content with their employer? Do they take pride in their work?
• What emoji would best describe the way we feel about our engineering partner, after a few (or more) sprints of work?
• CAN THEY DELIVER?

Measurable and non-measurable factors in evaluating the success of your engineering partnership

Moving further on the evaluation thread, we will look at some metrics you may want to focus on when evaluating your engineering partner.

Without measuring the success of the relationship against your initial business goals, how else can you tell if you are on the right path to achieving your objectives?

You should be aware from the beginning that some ‘soft’ metrics focus more on the quality of the collaboration. Such factors may be harder to measure, but ultimately, these qualitative metrics can determine the ultimate success of your engagement with your provider(s).

4 measurable evaluation factors for your software engineering provider

4 non-measurable evaluation factors for your software engineering provider

Evaluation Factor	What to assess
Relationship with the vendor	●       Communication and collaboration: type of communication (open, neutral, timely), problem-solving skills, conflict resolution approach
Proactive development	●       The preoccupation for value-add activities (e.g., hackathons, workshops, training, aiming at adding value for the customer and improving service) ●       Supplier’s ability to improve ●       Approach to training/onboarding new staff
Flexibility	●       Ability to respond to unanticipated demand or changes in products or services
Corporate Culture Compatibility	●       Company strategy alignment, shared vision and objectives, shared values, and operating philosophies ●       Market knowledge ●       Behavior with 3rd party teams (other suppliers, other development teams, external stakeholders, business units, etc.)

Don’t overlook the Customer’s evaluation

With any relationship, there is also the other side of the coin.

So you may also want to introspect your own role as a Buyer of professional services:

1. Can you assess your own role as part of the Service Quality effort?
2. How you do Supplier Management
3. How you handle Finance and Invoicing
4. Are you open to value propositions from your supplier?
5. Are you giving your supplier enough ‘room’ and business context to innovate with/for you?
6. How good is your Product Owner in his role as part of the delivery team? 

3 evaluation factors for the Customer/Buyer

Final thoughts

In conclusion, when working with a software engineering partner – regardless of location or time zone –  you’ll want to benefit from experienced staff, formal software solution design, a modern tech stack, structured programming, regular code walkthroughs, and a comprehensive quality management process (or at least a good test plan).

You will also look for:

• • team management – the leadership team that gets involved, understands the real status of projects, and communicates transparently;
  • frequent communication – use all possible channels – to get the right engagement (written, audio, video);
  • clarity;
  • honesty – give and ask for feedback.

This list could go on.

Selecting the right partner can be a challenging process. So is choosing the right KPIs that capture the value that you are both trying to bring to the partnership. None of these steps are trivial. But then again, not everyone gets to build awesome software, right?

Whether your engineers are outsourced or not, nearby or in a different time zone, by investing some energy in building the right team culture, by periodically evaluating the effectiveness of your collective efforts and openly discussing with them about ways to improve the relationship, you are closer to success than you have probably imagined.

If you found this article insightful, follow the Softelligence blog for more articles, how-to guides, and software delivery best practices.