PolyAI raises $50 million series C Read more

Top safety concerns for generative AI platforms

December 15, 2023


Security blog

Generative AI platforms have opened up new possibilities for businesses, helping them streamline operations, diversify services, and improve customer experience.

Despite its vast potential, the rapid pace of change can be understandably daunting for enterprises and individuals.

PolyAI has been rapidly building solutions to enhance customer experience with AI. To get a sense of attitudes and concerns, we invited customers and recent PolyAI VOX 2023 attendees to share their views on generative AI platforms.

Let’s explore the results and uncover key insights.

The positivity wave for generative AI platforms

Having already embraced AI solutions, our customers showcased an overwhelming 80% positivity towards generative AI platforms. As individuals already invested in AI, these early adopters have seen the positive impact that implementation can have on operations.

For those outside of our customer base yet to take the AI plunge, we predict a more neutral stance that might approach generative AI with a blend of curiosity and caution. However, the optimism among our customers suggests that positive attitudes are likely fueled by the promise of what generative AI has to offer.

Graph 1 – On the whole, what is your attitude towards Generative AI?

Caution in customer conversations

Positive attitudes declined from 80% to a more apprehensive 50% when asked about using generative AI platforms to power customer conversations. This decline suggests a desire among customers to see the success of more use cases and the experience of other organizations before fully embracing generative AI in their contact centers.

Graph 2 – How would you feel about using generative AI to power customer conversations through your voice assistant?

Top customer concerns about generative AI platforms

Table 1 – Which of the following concerns do you share about Generative AI?

Table 2 – Which of these risks associated with generative AI are most concerning to you?

As with any nascent technology, there are plenty of concerns to consider. Topping the list for our clients were worries about the accuracy of responses, data privacy, and the potential for causing offense.

Accuracy of responses from generative AI platforms

In the contact center, accurate responses build trust and credibility. Inaccurate information can lead to frustration, dissatisfaction, and potential loss of customers. Therefore, an understandable standout concern for our customers was the accuracy of generative AI platform responses, highlighting a hesitation to assign responsibility in case of misinformation. When accuracy is essential, our customers understandably want assurances that AI’s responses align with their brand identity and customer expectations.

Data privacy with generative AI platforms

Despite existing regulations such as GDPR and local data privacy laws in the United States, data privacy remained a top concern, highlighting a commitment to keeping customer information secure.

However, the findings suggest worries extend beyond compliance with concerns of potential breaches, prompt injections, and unauthorized access. Even with regulations, the fear of external threats that exploit vulnerabilities is clear.

Will generative AI platforms cause offense?

The third key concern centers on generative AI platforms inadvertently causing offense. Beyond the potential of generative AI hallucinations, will it say something unacceptable from a brand perspective or an ethical perspective? If it does, can it be controlled and stopped?

Exploring concerns and priorities: PolyAI VOX 2023 poll results

The findings from our annual AI conference, PolyAI VOX 2023, reinforce these concerns. A total of 114 respondents took part in our poll, emphasizing worries about the unpredictability of responses and the potential risks associated with data breaches and compromises of internal data.

Table 3 – Which of these risks associated with generative AI are most concerning to you?

Conference attendees turned their attention toward voice AI technology and how it can be improved. The top priorities included a strong emphasis on improving contextual understanding, followed closely by enhancing accuracy in natural language understanding and bolstering privacy and data security measures.

Graph 3 – Which area do you think needs the most improvement in voice AI technology?

Despite concerns about voice AI technology, respondents shared a positive outlook on the level of service they receive when using AI technology for customer service and support.

Graph 4 – As a consumer, how satisfied are you when using AI technology for customer service or support?

The theme of control over generative AI platforms

Throughout our survey, one theme resonated loud and clear – control. Customers emphasized the need for control over generative AI platforms in mitigating unpredictability, ensuring accuracy, or establishing adequate guardrails. The desire for control underlines a sensible and realistic approach to adopting generative AI platforms.

Open-ended insights about generative AI platforms

Open-ended responses from our customer survey also reflected additional insights with concerns about how to get other stakeholders on board with generative AI platforms within their business. IT largely understands the power, but many other teams have more fear than imagination and understanding of the potential.

The overarching concern is using robust guardrails to ensure safety and accuracy. Customers are curious about the level of control they’ll have over the information that models are trained on.


While the excitement for generative AI platforms is clear, the wide range of AI enthusiasts we surveyed approach it pragmatically. They balance enthusiasm with an awareness of the associated risks, emphasizing the importance of control, accountability, and a customer-first approach.

Ready to hear it for yourself?

Get a personalized demo to learn how PolyAI can help you
 drive measurable business value.

Request a demo

Request a demo