Facing thousands of Deutsche Bank AG investors, many of them mutinous, Anshu Jain knew a lot was riding on his speech at the annual meeting of Germany’s largest bank.

“On this day, every word matters,” Jain said in German at the May gathering. For that reason, he said he’d continue in his mother tongue. The Indian-born British national delivered the rest of the 2,000-word address in English.

Less than three weeks later, Jain resigned as co-chief executive officer after losing the confidence of investors. Brady Dougan, an American who struggled with German during eight years running Credit Suisse Group AG, left the Zurich-based bank at the end of June.

While their failure to master German didn’t cost Jain and Dougan their jobs, it drew criticism in their host countries and deprived the CEOs of a valuable tool for connecting with local shareholders, customers and colleagues. At a time when border controls have disappeared in much of Europe, language can still be a barrier.

“You have to be understood by your clients, and your clients in Germany speak German, said Kerstin Altendorf, a spokeswoman at the Berlin-based Association of German Banks.

Many multinationals based in Europe—including Airbus Group SE, Daimler AG and SAP SE — have adopted English as their corporate language. They reason this will help integration, ease negotiations on acquisitions and speed globalization of tasks and resources.

But English isn’t enough. As the firm’s public face, CEOs are expected to reflect the local culture, including the language, which is important at a time when the growing dominance of English can breed resentment at home. Georges Plassat, the head of Carrefour SA, the global retailer based outside Paris, makes a point of speaking French in public appearances in the home country.

Leadership Skills

Europe is a polyglot’s paradise. The European Union alone comprises 28 countries speaking 24 official languages. Stakeholders such as lenders and labor representatives may not speak English. Likewise, local politicians, regulators and media figures aren’t necessarily multiligual.

All this helps explain why CEOs without local language skills are so rare, even at Europe’s multinational firms.

“In Germany, it’s very important that a CEO who isn’t German can communicate with the business,” said Bjorn Johansson, head of a Zurich-based executive search firm that bears his name. He added, however, that leadership skills, knowledge of the industry and experience with management change should trump multi-lingualism in the search for a global CEO.

Stronger Connection

In Switzerland, a non-EU country with German, French and Italian as official languages, executives are expected to speak at least two in addition to English. Paul Bulcke, the Belgian CEO of Nestle SA, speaks fluent Dutch, German, French, Spanish and Portuguese, according to the company website.

“Being multilingual creates a stronger connection with peers, employees and consumers, which is critical for a business like ours,” Bulcke said in an e-mail.

Dougan said recently that he regrets not learning more German.

“I speak some French and Japanese and consider myself talented at languages, but I was just too busy,” he told Swiss business journal Finanz und Wirtschaft.

Jain’s successor, John Cryan, is a British national who, according to colleagues, speaks German well. Dougan’s successor, Tidjane Thiam, who was born in Ivory Coast and is also a French citizen, studied German in high school in his homeland.

“As you might have noticed, my German is somewhat rusty,” Thiam said at a news conference the day he was named CEO. “I will use the next few months to improve.”

–With assistance from Jan Schwalbe and Corinne Gretler in Zurich, Geraldine Amiel in Paris and Macarena Munoz in Madrid.