INDUSTRY ANALYSIS

Analyzing the Analysts

Research firms benchmark other companies, but they don’t benchmark themselves. This is one of the biggest ironies of the research industry, and CRM analysts are right in the middle of it.

Financial analysts are now benchmarked as they give their stock recommendations on CNBC. When you watch CNBC and see a positive endorsement of a stock from an analyst with holdings in that stock, you discount the credibility of the recommendation.

It’s all about trust. In an ideal world there would be that level of disclosure for industry analysts as well.

Full Disclosure Fantasy

Wouldn’t it be great to get on a web seminar see a sidebar graphic showing the times the industry analyst you’re watching has mentioned your company relative to the largest ERP vendors or their favorite best-of-breed vendor on deals? Or the times they have given negative rankings of different vendors? That would be immensely helpful to see if you and the analyst are on the same page with regard to your firm’s strengths and weaknesses.

If the analysts are telling you nothing but great things about your company — or, worse, ignoring you — chances are your company is not seen by the analyst as a key player in the market. Smaller vendors rarely get mentioned on deals because of viability concerns or questions about their ability to scale for the large, multi-million dollar deals.

If you’re a small vendor, it’s time to have a very honest discussion with your analysts about whether they think you are viable. Don’t go into the discussions beating your chest and being defensive, but get to what analysts really think. Do they honestly see you as a long-term player in the market, or will ERP consolidation gobble up the space? Are M & A deals on the table that will quickly collapse the space? Of course, the analysts can’t tell you that even if they know, but you should find out what your scorecard really looks like. Think of the CNBC quick scorecard and go after that. Ask for some tough love. If you don’t get any, someone is not being honest.

The User Is King

If you’re a manufacturer and have contracted with a research firm for guidance on managing your IT infrastructure, research firms need you in order both to grow and to understand the market. You’re a case study and a customer at the same time. As a result, users rule.

Given how critical users are to all industry analyst firms, there’s a tendency to recommend only the largest, most financially viable companies. The exceptions exist in niche markets where ERP vendors are too light on functionality and as a result do not have enough good customer references.

The real value of analysts to user accounts is in mitigating risk and guiding clients to the best approach to accomplishing their line-of-business objectives using IT investment. In many instances, if an analyst is honest and really understands the manufacturers’ needs, no additional software or services investment may be needed.

That’s where an analyst’s experience really starts to pay off. They save you from making a massive financial mistake, saving your company from upheaval and interruption. The analyst gets a major jump in ROI for saving the user millions. But the analyst would have never been able to get to that conclusion without having some knowledge of the vendor’s scope of capabilities in the specific area of interest.

Chances are research firms will never have the CNBC-like approach to full disclosure of its analysts, but it would be fascinating to see. But if you are either a vendor or user, you can start benchmarking analyst firms today. Start by looking at these metrics and measuring what you can.

Getting Started With Benchmarking

If you’re a vendor get away from looking at analyst relations as a PR function, because it’s much more than that. Get away from the lead count issue — the ROI for analyst relations will be horrible. Instead, vendors should look at some of the following measures of performance.

  • The ratio of user references to mentions. Notice that this is not just how many mentions you get, but the ratio of user references you provide to how many mentions you get. This is critical since it shows both how responsive your company is being to analysts and whether analysts are reciprocating by using the research from your customer base.
  • Number of references analyst firms really followed up on. This is related to the first point, and its critical to check back with users and see if the analyst firms actually did follow up. Ironically some of the smaller best-of-breed vendors have better programs for generating responsive user references than the world’s largest software companies. That’s a major reason best-of-breed vendors win deals against ERP vendors — superior references delivered very quickly.
  • Number of strategy recommendations acted on. Be honest: Did your analysts deliver recommendations that you took action on? You know the truth. That’s part of the ROI you’ll need for renewal. And if the analysts delivered no recommendation you acted on, that’s a clear sign you may be with the wrong analyst firm.
  • Quarterly SWOT analysis. SWOT stands for strengths, weaknesses, opportunities and threats. This is a quarterly assessment your company, ideally by a team of analysts in the firms you work with. Your analyst firm should deliver this as part of their base level of service and not charge you extra for it.
  • Times long-listed with vendors. Analyst firms are starting to walk the talk when it comes to tracking how many times they long-list vendors. While analyst firms cannot tell you who they long-listed (that would violate their NDAs with user accounts), they can tell you how many times you were long-listed. If you get push-back on this from an analyst firm, it means they have no way of tracking user interaction and don’t use a CRM system. Their CRM system is really the notebooks and Outlook folders of their analysts.
  • Times your sales force encounters analyst recommendations in sales cycles. If an analyst firm really does have influence in your markets, your sales force will at least hear from prospects about them. Look at how many times your sales force encounters analysts in prospect accounts.
  • Bottom line: Every other investment you make today is held to accountability. It’s time your investment in analysts gets held to that same standard. Nearly every analyst firm offers benchmarking of business and IT processes. It’s only a matter of time until one offers benchmarking of itself.


    Louis Columbus, a CRM Buyer columnist, is a former senior analyst with AMR Research. He currently works in the software industry.


    Leave a Comment

    Please sign in to post or reply to a comment. New users create a free account.

    Related Stories

    CRM Buyer Channels