Published on January 19, 2024, 8:22 am

Generative AI has become increasingly popular in various industries as companies seek to cut costs and enhance user experiences. However, not all generative AI tools live up to their promises, leaving enterprise leaders overwhelmed by service providers and their ambitious claims.

As organizations turn to generative AI for improved user experiences, increased efficiency, and reduced administrative burdens, it’s crucial for leaders and procurement teams to be discerning during the solution selection process. Maxime Vermeir, senior director of AI strategy at ABBYY, warns against falling for superficial claims of AI integration. Instead, IT leaders should evaluate whether a solution genuinely addresses a relevant business challenge and if the vendor can provide evidence of its success.

To identify potential risk areas when considering generative AI tools, Vermeir recommends four key questions for IT leaders:

1. What is the level of transparency around the risks and limitations of the tool?
2. Are there any red flags or concerns during due diligence on vendors?
3. How well do agreement terms, model specifics, and expected benefits align with organizational requirements?
4. Is the technology essential enough to outweigh any potential problems or shortcomings?

While every generative AI solution has pros and cons, inaccurate outputs are a common concern. Executives can address this by upskilling employees in fact-checking and promoting skepticism when assessing generated outputs. However, some risks may require more thorough evaluation. If a red flag or unresolved issue arises that cannot be addressed satisfactorily, it’s essential to reevaluate whether engaging with the technology vendor is necessary.

To gain a comprehensive understanding of the risks associated with different vendors, CIOs can conduct a thorough market analysis. With competition increasing in the marketplace, solutions that are currently popular may change over time.

Bret Greenstein, data and analytics partner at PwC, advises buyers to consider whether a generative AI solution is worth the time and resources required to ensure its safety, certification, and differentiation.

Despite the challenges, staying up-to-date with the evolving generative AI marketplace can help teams better comprehend the impact and severity of risks through comparison. Maneesha Mithal, partner at Wilson Sonsini Goodrich & Rosati, suggests that companies establish their risk tolerances and determine areas where mitigations can be implemented for selective use.

When a vendor fails to meet expectations, CIOs should take a systematic approach to address the issue. Sharon Mandell, SVP and CIO at Juniper Networks, recommends documenting concerns, discussing them openly with the vendor, reviewing contractual agreements, and considering options for customization. Mandell also advocates for building legal protections into contracts that account for solution performance and offer flexibility in case expected outcomes are not delivered.

The discontent surrounding generative AI progress isn’t deterring enterprises from pursuing their aspirations. According to Boston Consulting Group (BCG) data, more than 80% of executives plan to increase spending on AI and generative AI in 2024.

In the rapidly evolving field of generative AI, careful evaluation of vendors and solutions is crucial to ensure that organizations invest in technologies that deliver tangible benefits while effectively managing associated risks.


Comments are closed.