生成式 AI 客服真實案例：AI 怎麼回覆客戶、何時該轉真人Generative AI customer service: real cases, and when to hand off to a human

Generative AI has finally given customer service bots the ability to understand how people actually talk, and many business owners are wondering whether to let AI take the first pass at customer messages. Done well, AI support removes a mountain of repetitive work. Done badly, one wrong answer can turn into a refund, a complaint or even a legal claim. This article walks through four publicly documented cases, both successes and cautionary tales, and turns them into practical rules for small businesses: which questions to give AI, when a human must take over, and which numbers to watch after launch.

Klarna: big numbers, then a renewed focus on people

The cases below are public, documented examples, not Rejoicing Fountain clients; details are summarised from the references at the end.

In February 2024 the Swedish payments company Klarna announced that its AI assistant, powered by OpenAI, had handled 2.3 million conversations in its first month, about two-thirds of its customer service chats and the equivalent work of 700 full-time agents. Klarna said customer satisfaction was on par with human agents, repeat inquiries fell by 25%, and the time to resolve an errand dropped from around 11 minutes to under 2, across more than 35 languages. Notably, the release also said customers could still choose to talk to a live agent.

What happened next is just as instructive. TechCrunch reported in June 2025 that Klarna's CEO told SXSW London that AI can take away boring, routine work, but the company still wants to promise customers a human connection, and positioned human service as a premium tier. A February 2026 CX Dive report says the CEO acknowledged the company had leaned too heavily on cost, and that Klarna began hiring human agents again in May 2025.

Our reading: AI can absorb a large volume of simple questions, but "how many staff can we save" should never be the only yardstick. Being able to reach a person at any time is the floor for good service.

Air Canada: the bot got it wrong, the company paid

In November 2022, on the day their grandmother died, a Canadian traveller asked the chatbot on Air Canada's website about bereavement fares. The bot said they could buy a regular-fare ticket and apply for the reduced rate within 90 days of the ticket being issued. They did, only to be told that bereavement fares do not apply to completed travel; the correct rule was on a different page of the same website.

British Columbia's Civil Resolution Tribunal ruled in February 2024 (Moffatt v. Air Canada, 2024 BCCRT 149). Air Canada had suggested the chatbot was responsible for its own information; the tribunal member called this "a remarkable submission", holding that the chatbot is simply part of the airline's website and the company is responsible for everything on it, whether it comes from a static page or a bot. Customers, the decision noted, have no reason to know which part of a website is accurate. Air Canada was found liable for negligent misrepresentation and ordered to pay CA$812.02 in damages, interest and fees.

The amount was small, the lesson is not: what your AI tells customers may well be treated as what your company told them. Questions about refunds, prices and policies cannot be left to improvisation.

Intercom Fin: answers from your own knowledge base

Fin, the AI support agent from customer service software company Intercom, runs on Anthropic's Claude models. According to the customer story on Anthropic's site (checked October 2026), Fin answers from each company's own knowledge base, follows its tone and policies, and the vendor says its goal is for Fin to handle actions such as refunds. The page cites figures including resolving up to 86% of support volume and a 51% average out-of-the-box resolution rate across Intercom's customers.

These are vendor-reported figures whose definitions may not match your situation, so treat them as indicative. The gap between them is the interesting part: the same product produces very different resolution rates at different companies, and the difference usually comes down to how well the knowledge base is organised and how clearly the rules are written.

Taiwan Mobile: retrieve the answer, then respond

For a Taiwanese example, iThome reported in October 2023 that Taiwan Mobile's support chatbot handled around 420,000 questions a month with a satisfaction rate of about 95%, while roughly 1,500 human agents continued to provide round-the-clock service.

The most useful part of the report is how the team changed course. Their first attempt fed company data to a language model and let it answer directly, which produced wrong answers, because company data is a tiny fraction of what the model learned from. They switched to converting their knowledge base into vectors: the model interprets the customer's question, then the system returns the existing answer with the highest confidence rather than generating one freely. Taiwan Mobile said this roughly tripled answer accuracy on difficult questions.

What to let AI answer

Across these cases, AI does best with questions whose answer is already written down in your own material:

• Business basics: opening hours, address, directions, parking, contact details.
• Published policies: prices, payment options, delivery times, and explanations of the returns process.
• Products and services: specs, who it suits, common usage questions.
• Lookups and routing: pointing customers to the right form, page or next step.

The rule of thumb: let AI answer only from the FAQ and policies you provide, and never guess beyond them. When that material changes, update what the AI reads at the same time, so you never end up with two pages saying different things, as happened to Air Canada.

When a human must take over

• Low confidence: if there is no matching material or the question is ambiguous, the bot should say "Let me get a colleague to reply."
• Complaints and frustration: emotional messages or the same question asked repeatedly mean automation has stopped helping.
• Money: refunds, credits, compensation and price exceptions. AI can explain the process, but approvals and promises belong to people.
• Personal data: viewing or changing personal details or account settings needs identity checks and an authorised staff member.
• Legal, health and safety: contract disputes, liability, medical or safety questions should not be answered by AI.
• The customer asks: if someone wants a person, make that easy; never trap them in menus.

When handing off, pass the conversation history along so the customer does not have to start over.

Disclosure, logging and review

Say it is AI

Open the conversation with something like "You are chatting with our AI assistant; you can ask for a person at any time." When customers know they are talking to AI, their expectations are more realistic and they are more willing to ask for a hand-off when needed.

Keep logs and spot-check them

Store AI conversation logs and review a sample every week, especially conversations that were handed off, rated poorly or mention refunds. When you find a wrong answer, fix the knowledge base or rules first, then check whether things improve. Logs can contain personal data, so restrict access, set a retention period and follow Taiwan's Personal Data Protection Act.

Metrics to watch after launch

• Resolution rate: the share solved without a human, read alongside satisfaction so that customers giving up are not counted as "resolved".
• Hand-off rate and reasons: the questions most often passed on show where the knowledge base has gaps.
• Error rate: the share of wrong answers found in spot-checks; this is your most important risk metric.
• Customer satisfaction: a quick rating after each conversation, compared with your human agents.
• Repeat contacts: the same customer asking the same thing again soon after means the first answer did not really solve it.

FAQ

Does a small business need to train its own model?

Usually not. The practical route is an off-the-shelf model combined with a well-organised FAQ, price list and policies, with the AI restricted to answering from that material. Cleaning up your content tends to matter more than switching models.

Is the company responsible if its AI gives a wrong answer?

In the Air Canada decision, the tribunal treated the chatbot as part of the company's website and held the company responsible for its information. Laws differ by country, so ask a lawyer about your situation, but design on the assumption that whatever the AI says, your company said.

Can this run on a LINE Official Account?

Yes. Connecting AI through the LINE Messaging API still lets you define hand-off rules, with colleagues taking over from the Official Account console. The exact setup depends on your plan and existing systems.

Wrapping up

Taken together, these four cases point the same way: let AI handle repetitive questions that have a standard answer, leave judgement, commitments and emotions to people, and keep checking what the AI actually says. Start with a small set of your most frequent questions and expand only once the error rate is low enough. To see how we plan AI support and hand-off flows, read about our AI integration service, or start with 5 steps for small businesses to adopt AI and automation. Book a free 30-minute consultation and we will help you work out where your customer service should start.

References

• Klarna AI assistant handles two-thirds of customer service chats in its first month, Klarna, 2024-02-27
• Klarna CEO says company will use humans to offer VIP customer service, TechCrunch, 2025-06-04
• Klarna pursues 'Uber' style customer service model, CX Dive, 2026-02-20
• Moffatt v. Air Canada, 2024 BCCRT 149, Civil Resolution Tribunal (British Columbia), 2024-02-14
• Air Canada found liable for chatbot's bad advice on plane tickets, CBC News, 2024-02-15
• Intercom case study, Anthropic (vendor customer story), 2026-10 (date checked)
• Taiwan Mobile uses AI 2.0 to improve its support chatbot (in Chinese), iThome, 2023-10-19