Are system evaluations important? (Yes, definitely!)
I worked at a company once where we used a particular software tool to manage our projects. It wasn’t a bad tool…. But it didn’t quite fit our needs. My team spent a LOT of time figuring out how to make it work, which not only distracted us from work it made us more than a bit frustrated. We didn’t have a choice though, so we suffered through using it, and finally, after about a year of pounding a square peg into a round hole we asked why we used that particular piece of software.
The answer… was a bit shocking.
Basically an executive was asked what we should use, and they pointed at the one we had and said “that one”. That was it. No further discussion. No questions asked. No requirements gathered. Nothing.
I had a feeling back then, and have since learned, that that is NOT a good way to pick a system!
Positives and Negatives
The positive aspects of randomly picking an option is it’s quick. You’ll get to the end of your “selection” process faster than you could otherwise. This can be an attractive prospect as system evaluations can take a lengthy amount of time and resources to complete.
In some extreme situations there can be an argument for speed, but I have never encountered one where every other aspect of the evaluation was thrown out the window. Needing to move quickly isn’t an excuse to not conduct due-diligence.
That positive aside, there’s a TON of negatives… things like:
Price - other options may do the same things (or do them “well enough”) more cheaply, or can be implemented more quickly. If you don’t evaluate those systems you won’t even realize you’re paying more for something.
Features - The one you pick may not do what you need, or may do it in a way that makes it challenging or onerous for your teams to adopt. Critical features can fall into this bucket, making a quick selection incredibly painful for teams when they realize their new system doesn’t do something they need to be successful.
Compatibility - The one you pick may not integrate or be compatible with other systems or processes you have. This can result in your teams having to do additional manual work to transfer information or get what they need. At best you may need to build custom integrations between systems, which introduces greater cost and risk, and in some cases may not even be possible if you don’t have the resources to integrate (either money or people).
Duplicative systems - You may already have a system that meets your team’s needs. Failing to recognize this could result in you ending up with two, VERY similar systems. This increases the administrative and budget overhead and adds unnecessary complexity to your tech stack.
Any one of these on its own should be enough to not randomly pick an option… all of them together make it a painful, and costly experience.
Doing it right
Instead of pointing randomly, groups should go through a more formal evaluation of their requirements and their options. This doesn’t have to be a months long, in depth endeavor, but it should at least involve some thought about what is needed, and how the options meet those needs.
Typically this process includes things like:
Identifying stakeholders - who will be using the system, or be impacted by it? Depending on their level of involvement stakeholders should be involved in the selection process to help ensure their needs are met. This list should include direct users, but also teams who would have to support or build the system (e.g. IT, engineering, etc).
Determine requirements - what challenge or issue needs to be solved and how should the system do that? The stakeholders identified in #1 can help pull this list together, but this is a critical step (one that the company I worked at definitely skipped!). By the end you’ll have a list of things you need to be successful, and while you may not get all of them, understanding what is needed will help you make a better choice.
Get options - After you know what’s needed, go and find potential options. This could be as simple as some quick googling, or an in-depth examination of what’s out there. Personally I find this to be an interesting step as I get to learn about what’s available.
Eliminate some - Compare your options against the requirements. Do any clearly NOT meet your needs? Sometimes you can figure this out just by looking at their website, other times you may need to send an email and ask for more info. Doing this helps weed out options that are clearly not a good fit, and saves time later.
Get more information - Once you’ve got a shorter list, dig in and really compare the rest to your requirements. Get a formal demo. Talk to other customers. Figure out how it really works. While it’s uncommon to find a system that really does everything you need, this will help you find the best possible fit. Have a core group of your stakeholders (especially from the group who will use the platform) get involved in this step as they’re the ones who will be stuck with the final outcome!
User Acceptance - If you decide to go forward with a new system make sure your stakeholders have a formal acceptance. Let them use the system and determine if it meets your needs… if not, work with the vendor to update or change things, but don’t just blindly accept that it works as expected!
All of this does take time and effort… however, it’s always cheaper to go through this process than to end up with a system that doesn’t meet your needs. Not only will you have to replace it, you’ll have to retrain everyone, find a new system, and rebuild credibility.
A great side effect of this process is it also gives you a great opportunity to build buy-in for the new system, to update processes and policies, and further enhance how your team operates.
Does it take time and effort? Yes.
Is it worth it? Definitely.
