A lot has changed in the world of automation since Apple first launched its virtual assistant, Siri, on Oct. 4, 2011. It’s a date that musician, actress and public speaker Susan Bennett remembers well because she’s the original female, American voice behind Siri.

“It is still weird,” she said on this episode of The Insuring Cyber Podcast. “It’s like, ‘How many millions of people know my voice?’…I don’t really think about that aspect of it.”

After getting her start in entertainment as a musician working on commercial jingles and singing backup vocals on tour with Burt Bacharach and Roy Orbison, Bennett began voiceover work at Doppler Studios in Atlanta. She did radio and TV spots for Coca Cola, IBM and Papa John’s Pizza, among others, and worked on camera for clients like Ford, IBM and Kimberly Clark. Eventually, she found herself doing voice recordings for Siri, albeit sort of unexpectedly.

“We were sort of told that we were just doing generic phone recordings, and so we had no idea what ultimately was going to be done with our voices,” she said. “I did the recordings in 2005, and then six years later, Siri showed up on Oct. 4, 2011, and I’m going, ‘What? That’s me!'”

After wrestling with concerns about being typecast, she decided to reveal herself as the original voice of Siri in 2013, which she said landed her many other voice acting and public speaking opportunities.

“I guess you would say I was facing a fear by jumping out into the public as calling myself Siri, but it turned out to be the right thing to do,” she said. “It ended up being a fun thing to do, ultimately.”

Since Siri launched more than 10 years ago, virtual assistants and customer service robots have popped up in many industries, including insurance. This has become so prevalent that Washington State University researcher Soobin Seo made customer service robots a focus of a recent study detailed in an online International Journal of Hospitality Management paper. The study found that of 170 people surveyed in hypothetical customer service robot scenarios, participants preferred interacting with female robots. Focused exclusively on the hospitality industry, Seo examined how certain traits, such as gender, play a role in how people interact with AI robots.

But what does this mean for insurance as automation is increasingly used to enhance the consumer experience?

“I think the spotlight will be on this issue,” said Liz Grennan, expert associate partner at McKinsey Stamford, earlier in the episode. “As humans, we have these preferences that are underscored by gender bias. Some of them may be harmful, and some of them may be neutral, but I think that the spotlight on the issue will only be turned up. I think every organization from insurers to consumers to banking, everything, will have to think about this. In 10 years from now, folks will look back and say, ‘What did you perpetuate? What did you contribute to?'”

Previously, Grennan served as the global managing counsel for McKinsey Digital. She is a leader of McKinsey’s initiatives related to technology and risk and a member of several firm leadership committees, including the McKinsey’s digital-risk committee. She said that as a woman, she can see the clear line between the use of automation for customer service and potential gender discrimination.

“I’m speaking as a woman who also has an 11-year-old daughter who’s just marveling at some of the things I tell her weren’t possible when my mom, for instance, was younger,” she said. “For instance, my mom could not have applied to my law school, because it was not co-ed at the time, which blows both of our minds. Women could only open bank accounts in the 1960s, so I think we forget how recent gender discrimination, jarring gender discrimination, has been.”

Bennett said that female voices being used for virtual assistants like Apple’s Siri and Amazon’s Alexa could point back to the days of telephone operators.

“I was asked a question like this a lot when I first revealed myself as the voice of Siri, and I think what happened with the AI voices and a lot of the original voices being female was kind of a progression from the operators that they used to have with telephones,” she said. “It was like a precedent had been set that females were doing that type of work.”

Grennan added that thoughtfulness about how to proceed and eliminate bias will likely become an important part of organizations’ risk and compliance strategies in the future as they embrace automation.

“You want to strengthen the relationship between the user and the device, but you want to do that in a way that withstands advanced future scrutiny,” she said. “Some thoughtful reasoning about what you are doing and why you are doing it kind of plays into your risk and compliance framework.”

Doug McElhaney, partner in McKinsey’s Washington D.C. office, agreed earlier in the episode, adding that certain questions will become important for insurers as they integrate more of this technology into their business.

“This is another good example of where careful scrutiny of the development of some AI-related capability is really important, because all of the AI is trained on massive volumes of historical data, so you can very quickly get caught in this trap where you run these devices, you train them up, and then you start to see a trend,” he said. “You have to ask yourself, ‘Is that what I should be saying? Is that something that’s an artifact of the data that was used to train this?’ Then you have to ask yourself the question, ‘What should we do about that?’ Because I don’t think that’s an obvious, easy answer. It’s obvious to say maybe that should change, but how? How do we go in and correct for that, and to what degree? I think those are the thorny issues that we’re going to have to navigate over the next couple years.”

McElhaney leads a global team of data scientists, AI experts, and data engineers at McKinsey, and he has recently focused on helping insurers expand their use of external data in their AI models and solutions. Before joining McKinsey, he spent 10 years at General Electric.

While risks and questions exist for insurers in the world of automation and robotics, McElhaney is ultimately optimistic.

“I think the future, while uncertain, has the potential to be very exciting as well,” he said.

Grennan agreed.

“I’d say that the winners in the future are those who keep pace with the innovation and all of the things that you can do with predictive analytics to make it more personalized, to make it faster, to make it more efficient,” she said, “but [users] also want to know that the offers of the insurance carriers are being incredibly transparent and clear. I think those are the winners. Keep up with the innovation and also keep up with the trust.”

Returning to the question of which personality traits shape the way customers engage with AI robots, Bennett said one of the most endearing things about the original Siri wasn’t necessarily a nurturing or warm quality at all. It was her attitude.

“She definitely had an attitude, and I think that’s why people really loved talking to Siri, especially at the beginning,” she said. “It was fun to hear what she would say and how she would respond to some of your questions.”

With that attitude in mind, Bennett said that Siri would likely have a message to share if she came to life.

“I think she’d want to tell people to pay more attention in school,” she laughed. “[She would say,] ‘Don’t ask me so many stupid questions.'”

To find out what else Liz, Doug and Susan had to say, check out the rest of this podcast episode and be sure to check back for new episodes publishing every other Wednesday along with the Insuring Cyber newsletter. Thanks for listening.