• Managing Your Innovation Program

Best Practices for Idea Evaluation

A competitive advantage comes with focusing on projects that your business can execute on and deliver the most impact with. In this guide, we show you how to develop and apply idea evaluation best practices to move the best ideas forward and keep people in the loop.

You’ve launched your Kindling program and are now getting amazing Ideas from previously untapped sources throughout the organization—maybe you’ve even begun to use Kindling as a better way to engage strategic customers and partners. Congratulations! Now you’re faced with a new challenge and are asking yourself, How can I manage all of these Ideas? You might even be thinking:

  • The volume of Ideas is overwhelming;
  • What evaluation criteria should I use?
  • How can I evaluate duplicate or under-developed Ideas? and
  • How will I say “no” to some Ideas?

You promoted change, you’ve expressed your desire to listen to your teams to find new areas of improvement, but now you actually have to do something with all these Ideas. Your community is waiting for feedback, waiting for you to make decisions, and, most importantly, waiting for recognition. But where do you begin? How do you make some sense from the insights you’ve received, sort through all the feedback, and begin evaluating?

The good news is you’re not alone. Many organizations struggle with this same challenge, which is why we’ve developed this guide compiling approaches our customers have taken to help their teams tackle the challenge of managing all the Ideas in their innovation programs. Your goal should be to thoughtfully evaluate all Ideas while spending as little time and resources necessary in identifying the Ideas with the best potential. It is then those Ideas, those filled with the most potential to solve the problem at hand, that justify increased attention.

Determining Evaluation Criteria

Consider the evaluation criteria before you begin evaluating Ideas to determine which Ideas show the most promise and which Ideas can be easily put to the side. By investing upfront in identifying these criteria, the evaluation process will be much simpler and more efficient. Think of the Ideas in your Category or Campaign as your innovation pipeline and evaluation criteria as the funnel you put them through to filter.

You may also use different evaluation criteria at different points in an Idea’s life cycle. For instance, your first round of Idea evaluation may rely solely on popularity while your second or third may require you to take a closer look at the timeline or budget for implementation.

Typical questions evaluation criteria might address include:

  • Does the innovation community support this Idea?
  • Did it receive a lot of interest through active conversation, voting, or volunteering?
  • Is this Idea in line with our strategic goals for this year?
  • Is it in line with the goals of this Category or Campaign?
  • Do we have the budget to implement it?
  • Do we have the necessary tools or team to implement it?
  • Can it be implemented within the desired timeframe?
  • Is the executive committee on board with moving forward with it?
Your evaluation criteria will help you determine which Ideas are worth your investment of time and resources, and which are not a good fit and can thus be declined.

There are two aspects to the strategy for evaluating Ideas: your evaluation workflow and your evaluation timeline. Evaluation workflow encompasses the stages your Idea will pass through on the way to a decision being made. This evaluation workflow maps to the evaluation criteria you’ll consider at each stage in the evaluation process. Your evaluation timeline is the pace at which you’ll move Ideas from one stage to the next. Ideally, you’ve determined both your workflow and timeline prior to your Campaign launch, but if you haven’t, the following two exercises will help you determine a process most appropriate for your organization.

Exercise: Define Your Evaluation Workflow

To determine the appropriate workflow stages or states, start by pinpointing what needs to happen to make the Idea a good fit—ask yourself, What are my must-haves? What characteristics must this Idea have in order to move forward to the next stage? And what are my deal-breakers? Which attributes make this Idea a no-go? Go through the exercise of writing these down! These characteristics are your evaluation criteria and will help you define your Idea workflow for Kindling.

For example, if all Ideas must pass through the legal review team, Legal Compliance is an evaluation criterion and should be a state in your workflow. If all Ideas must cost less than $10,000, cost should also part of your evaluation criteria with the potential to become a workflow state such as Cost Analysis.

Examples of workflow states used by Kindling customers:

  • Cost Analysis / Budget Analysis: An analysis of implementation cost is necessary to determine whether you can more forward with this Idea.
  • Legal Compliance: An analysis by the legal team is necessary to determine whether you can more forward with this Idea.
  • Executive Review / Committee Review: Ideas that you would like to move forward with that require executive approval or approval by a review committee.
  • Innovation Team Review / Under Evaluation / Expert Assessment: This Idea shows potential and is currently being evaluated or assessed to determine whether it can move forward.
  • More Information Required / Needs Elaboration: Ideas that require more information before they can be evaluated.
  • Duplicate: Another participant has already suggested a similar Idea.
  • Mid- to Long-term / Some Day / Backlog / Parked: Ideas that should be reviewed at a later date but are currently paused.
  • Declined / Not Progressed: Ideas you want to decline that don’t make sense for your organization at this time.
  • Previously Attempted: Declined because it was previously attempted by your organization but was not successful.
  • Approved: Ideas that you plan to move forward with.
  • Prototyping: Approved Ideas that you are attempting to implement as test cases.
  • Completed: Ideas that have made it through the implementation phase.

Exercise: Map out Your Evaluation Timeline

Your evaluation timeline depends on a number of factors including the type of challenge you’re running (Category or Campaign), whether a limited number of Ideas can be approved or “win” the challenge, and whether your Moderators are empowered decision-makers. First identify whether you'll be using a Category or Campaign to collect Ideas. For Categories, Idea submission can be ongoing with no set deadline, while in Campaigns, the timeline is fixed with Campaign start and end dates. Idea evaluation can happen at specific points, such as a week after a Campaign ends or at a quarterly committee review meeting.

Next, consider whether there are a limited amount of Ideas that can be approved. If there can be only one or two winning Ideas, then you’ll need to wait until the end of the Campaign or until a designated time before closing a Category, when all Ideas have been submitted, to make your final decision. Finally, think about who will ultimately make the decision about Ideas submitted. If Ideas must be approved by a review committee (budget, legal compliance, executive), then your evaluation timing will need to take into account the frequency and timing of committee meetings. However, if your Moderators have full decision-making power and budget approval, they may have more flexibility with timing than a committee that meets at specific intervals.

Define your criteria and strategy for evaluation prior to Campaign launch and be sure to share them with all Moderators.
Share your evaluation criteria with innovators, too, in a Kindling Post about your Campaign. This will provide added context to your Category or Campaign and help to make sure that you get the most relevant Ideas in response.

Managing the Volume of Ideas

It’s important to remember that the purpose of evaluating Ideas is not to simply reduce the volume, but to try and identify, as soon as possible, and with minimal resources or distractions, which Ideas warrant further consideration.

An effective process that we have seen from many of our customers is to evaluate your Ideas in rounds or stages. Begin with an easy to apply filter that will help you determine which Ideas to respond to first. For example, you might use built-in metrics like votes, views, comments, or time of last activity to determine which Ideas are in most need of response, which ones are potential winners, and which can easily be taken out of the mix. Ideas that have broad support from your participants through votes or volunteering show promise for implementation and are good candidates to move to more comprehensive stages of evaluation, while Ideas that linger and draw minimal engagement may be prime for declining or deferring for future consideration. Using Kindling’s Moderate features, you can select and make decisions on multiple ideas at once. Then, after you’ve reduced the volume of Ideas, you can spend more time on an in-depth evaluation for those remaining Ideas.

Another approach to handling the volume of Ideas is by scheduled Moderator check-ins. These meetings can be a great way to spread the responsibility of managing a high volume of Ideas. Moderators can collaborate with each other, evaluate Ideas as a team, and discuss any challenges or questions they have about evaluation criteria. Designating a specific time for your Moderators to evaluate Ideas increases the likelihood that it will be accomplished and, by working together, your Moderators can move through Ideas more quickly and ensure a consensus about which Ideas make it to the next stage.

A third approach is to engage other Kindling participants in your decision making, drawing on the wisdom of the crowds to help reduce the volume of Ideas. With Kindling Evaluations, you can crowdsource the validation process allowing you to rapidly identify the best Ideas. Imagine you're a Moderator in the Private Wealth division of your Finance organization, and you have 8 popular candidate Ideas to consider. So you quickly build an Evaluation and send it to your best 10 engineers to see which Ideas could be built the fastest. Then you go to lunch, and when you come back, 6 of them have replied and now you know the 3 easiest Ideas to build. So you put the other 5 Ideas aside and ask your top 4 business managers which of the 3 remaining Ideas will have the biggest impact for your clients. And before you go into a late afternoon meeting, you have your answer. Relying on other Kindling participants for Idea evaluation means moving more quickly through the evaluation process and ending up with better, more informed decisions.

Avoiding Under-developed Ideas

The best way to pre-empt incomplete or under-developed Ideas is to be as specific as possible about what you’re looking for when starting a Category or launching a Campaign. Establish a strong context and use the Category or Campaign description along with a Kindling Post to state what your goals are and what factors you’ll need to evaluate Ideas. By being transparent about your evaluation criteria, you can help ensure that participants propose Ideas relevant to the challenge and include all of the information you’re looking for.

If you’ve already launched your Campaign and you’re finding that you keep coming up against under-developed Ideas, you can share a Post reminding participants of the evaluation criteria and expectations for Idea submission and highlight a well-developed Idea that better illustrates what information you’re looking for.

View Ideas with vague details as an opportunity to reach out to the submitter. Suggest a revision to the Idea where you outline the missing information, or use a Kindling Assessment for a structured set of questions you’d like the author to address. To engage the community, add a comment to the Idea eliciting contributions from other participants. With additional feedback from your participants, each with their unique experience and perspective, Ideas can evolve into more fully formed concepts.

Always write a Post describing a new Campaign. Use this as an opportunity to catch people’s attention enough to get them to participate, and provide enough context so that people’s submissions best meet the goal of the Campaign.

Managing Duplicate Ideas

Kindling helps cut down on duplicate Ideas by scanning for similar content and informing the user during the submission process. Even so, duplicate Ideas will occasionally make it into your program. An easy way for Moderators to address redundant discussions is to create a workflow state named Duplicate and move the most recently entered or least fully formed Idea into that state. Then, add a comment to the Idea marked as a duplicate that you will be continuing the conversation in the Idea you are keeping open, and include a link to that Idea. You might also contact the owner of the Idea that will persist asking them to work with the duplicate Idea owner to include them as a contributor and to capture their unique aspects of the concept.

Saying “No”

People are realistic and understand that not all Ideas can be approved. We hear regularly that people want their Ideas considered—they want to be listened to—but understand the constraints of the organization.

Saying “no” is a necessary and important part of the Idea evaluation process. Declining an idea is as much a decision as approving it. Close the loop on the conversation and steer clear of that void where submitters never learn if their Ideas were even considered.

An Idea can be declined for any number of reasons: it’s not feasible, for example, or it doesn’t align well with established goals, or it has already been tried without success before. Since not all Ideas will be implemented, it’s important to design a strategy for declining Ideas that cannot move forward. An Idea that is deemed to be impractical but stays in its open state indefinitely appears no different to a participant than an Idea that has never been looked at. Participants need feedback, especially when their Ideas won’t be implemented.

To respond “no,” you may choose to use the built-in Declined state, or to create custom states that provide additional detail about why an Idea is being declined. Some examples are:

  • Exceeded Budget Limit;
  • Turned Down by the Executive Committee;
  • Rejected by Client; and
  • Failed End-user Testing.

Each decline action should always be accompanied by an explanatory message, even if it just shows that the Idea was considered. Some examples are:

  • “This does not meet the budgetary limits for this Campaign but we’re open to reconsidering if you have ideas for cost reduction.”
  • “This Idea isn’t projected to meet the minimum cost-saving requirements.”
  • “We can’t purchase the software you’ve recommended at this time.”
  • “This technique was attempted several years ago, but with little result.”

Being transparent about your reasoning helps mitigate disappointment about an Idea being declined. More than having every Idea approved, most users are concerned that their Ideas are heard and given due consideration. So, don’t be afraid to say “no,” just do it transparently and thoughtfully.

Work with your Moderators to design a strategy for saying “no.” This will empower them to take action more quickly and consistently while providing ways to acknowledge the effort put in by the Idea author.

Conclusion

Every business is faced with constraints and most organizations don't have the luxury of allocating valuable resources to projects that don't stand a strong likelihood for success. A competitive advantage comes with focusing on projects that your business can execute on and deliver the most impact with. Through a thoughtful evaluation process, teams can eliminate wasted expenditure on Ideas that aren’t a great fit and focus on those Ideas that best meet the issues at hand. Everyone is busy, including your Moderators, so create a process that maximizes people’s time while still providing feedback to everyone who took the time to contribute an Idea to your program.