Who Is Liable When an AI Tool Gives a Customer Wrong Advice?

TL;DR — When an AI tool gives a customer wrong advice, responsibility is usually shared: the AI model provider may be responsible for the model service, the software agency for how it was built and integrated, and your business for where and how you chose to use it. The single most useful fix is to define clear ownership in the contract and in the product itself: limit what the AI is allowed to do, require human review for high-risk answers, and keep logs showing what the system saw and said. Reading time: ~7 min

What it is and where it sits

This question is really about accountability (who answers for harm), not just software bugs. In a typical AI feature, there are several layers:

• Your customer-facing app — the chat widget, help center, agent assist panel, or email reply tool.
• The agency-built application logic — the code that decides when to call AI, what instructions to send, what company data to include, and whether to show warnings or require approval.
• The AI provider — the external service or self-hosted model that generates the answer.
• Your business process — the policy for what topics AI may answer, what must go to a human, and what records are kept.

That matters because "the AI said it" is usually not the whole story. In practice, bad advice often comes from one of four places:

1. The model generated a false answer.
2. The app sent bad or incomplete context.
3. The system let AI answer a question it should never answer alone.
4. No one reviewed, monitored, or constrained the output.

In a normal request flow, AI sits in the middle of your support or sales stack. It often replaces a simpler rules-based system (hard-coded FAQ search, canned responses, keyword routing) with a probabilistic system (one that predicts likely text rather than following fixed rules).

[Customer] ---- 1. asks question ----> [Web/App Chat]
[Web/App Chat] ---- 2. send prompt ----> [Agency App]
[Agency App] ---- 3. fetch docs/data ----> [Knowledge Base]
[Agency App] ---- 4. model request ----> [AI Provider]
[AI Provider] ---- 5. draft answer ----> [Agency App]
[Agency App] ---- 6. rules/checks ----> [Policy Layer]
[Policy Layer] ---- 7. show or escalate ----> [Customer]

If the answer is wrong, responsibility usually follows the same path:

• AI provider: responsible for uptime, model behavior within its published limits, security of its service, and sometimes content filtering.
• Software agency: responsible for integration choices, prompt design, retrieval setup (how documents are fetched), guardrails, testing, and warning labels.
• Your business: responsible for deciding acceptable use, approving risky workflows, training staff, and not deploying AI in places where wrong advice can cause serious harm without review.

A useful mental model: if the agency built a calculator app and the math library had a bug, the library vendor may share blame. But if the agency used that calculator to approve mortgages without checks, that deployment decision is not the library vendor's fault.

How it actually works

Let’s walk one realistic example end to end.

Example: AI support bot gives a customer the wrong refund policy

A customer asks: "Can I get a full refund after 45 days?"

Your company’s real policy says refunds are allowed only within 30 days unless a manager approves an exception.

Step by step:

1. Customer submits the question in chat. The website sends the message to your app.

2. Your app builds the AI request. The agency’s code may add:

   • a system instruction like "Answer using company policy only"
   • recent chat history
   • policy documents from your knowledge base
   • customer metadata such as product tier or country

3. The retrieval layer fetches documents. Retrieval means "find likely relevant documents first." If the wrong refund document is indexed, or an old policy is still present, the AI may be working from stale information.

4. The model generates an answer. The model might say: "Yes, full refunds are available within 60 days." That could happen because:

   • it guessed from general patterns
   • it misread the provided policy
   • the prompt was vague
   • the wrong document was supplied

5. Your app decides whether to show the answer directly. This is a major responsibility point. The agency could have built rules such as:

   • if the topic is refunds, billing, medical, legal, or safety, require human review
   • if no supporting document is found, do not answer
   • show citations under the answer
   • add "This is general information, not a final decision"

6. The customer acts on the answer. If the customer relies on it and suffers loss, the question becomes: who failed in the chain?

Where responsibility usually lands in this example

• If the model hallucinated (confidently invented a policy) despite being given the right document, the AI provider may bear some responsibility for model quality, but rarely all of it.
• If the app passed outdated documents, the agency and/or your internal content owner are more likely responsible.
• If the bot was allowed to answer binding policy questions without review, that is usually a deployment and governance decision by the business, often implemented by the agency.
• If the contract promised "accurate automated policy answers" without limits, legal exposure increases for whoever made that promise.

The practical lesson: liability is usually about system design and operational choices, not just the model output.

When to use it (and when not to)

You can safely use AI for many customer interactions, but not every interaction should be fully automated.

You probably don’t need direct AI-to-customer answers if...

• a wrong answer could create legal liability or safety risk
• your policies change often and your content is not well maintained
• you cannot keep logs of prompts, retrieved documents, and outputs
• no one in your business owns review and escalation rules
• you mainly need search, routing, or canned replies rather than free-form advice

In those cases, a safer design is: AI drafts, human approves.

Trade-offs

Every benefit comes with a cost.

• Benefit: faster customer responses
  Cost: higher risk of confidently wrong answers if you skip review.

• Benefit: lower support workload
  Cost: more setup work for content quality, testing, and escalation paths.

• Benefit: 24/7 availability
  Cost: incidents can scale 24/7 too; one bad policy answer can be repeated to many customers.

• Benefit: more natural conversations than rule-based bots
  Cost: less predictability; outputs are not fully deterministic (exactly repeatable every time).

• Benefit: quick launch using a third-party model API
  Cost: vendor dependence, changing model behavior, and contract review for data handling.

• Benefit: broad knowledge synthesis
  Cost: harder root-cause analysis when something goes wrong; you need logs and evaluation tests.

From a security and risk angle, the biggest hidden cost is operational burden: someone must own prompt changes, source-document updates, incident review, and customer remediation when the AI is wrong.

In practice

Below are two practical patterns a software agency can implement today to reduce disputes about responsibility.

1) Add a policy gate before showing risky answers

This example shows application logic that blocks direct answers on sensitive topics and routes them to a human queue.

{
  "blocked_topics": ["refunds", "billing disputes", "legal claims", "medical advice"],
  "action_if_blocked": "escalate_to_human",
  "require_citation": true,
  "fallback_message": "I can help collect details, but a human agent needs to confirm this answer."
}

What it does: this policy file defines topics the AI may not answer on its own. The gotcha: if your topic detection is too loose, risky questions may slip through; if it is too strict, too many chats go to humans and you lose the efficiency benefit.

A matching application check could look like this:

function shouldEscalate(topic, citations) {
  const blocked = ["refunds", "billing disputes", "legal claims", "medical advice"];
  if (blocked.includes(topic)) return true;
  if (!citations || citations.length === 0) return true;
  return false;
}

What it does: this code escalates if the topic is sensitive or if the answer has no source citations. The gotcha: this only works if your app reliably classifies the topic and actually records citations from the retrieval step.

2) Log the full decision trail for disputes and incident review

If a customer later says, "Your AI promised me X," you need a record of what happened.

⚠️ Logs can contain personal data and sensitive business information. Before turning on detailed logging, open your provider dashboard or app admin panel and set a retention period, then update your privacy notice and access permissions. If your vendor offers a dashboard setting for data retention or training usage, review that first in your provider's dashboard (for example, look for Settings → Privacy, Data controls, Logs, or Retention; names vary by provider).

{
  "timestamp": "2026-10-30T14:22:11Z",
  "customer_id": "cust_1842",
  "channel": "web_chat",
  "user_message": "Can I get a full refund after 45 days?",
  "retrieved_documents": [
    {"id": "policy-refunds-v3", "title": "Refund Policy", "version": "3.0"}
  ],
  "model_output": "Yes, full refunds are available within 60 days.",
  "citations": ["policy-refunds-v3"],
  "topic": "refunds",
  "decision": "escalated_to_human",
  "shown_to_customer": false
}

What it does: this event log captures the exact message, source documents, output, and whether the answer was shown. The gotcha: if you log everything but never review incidents, the logs help after damage but do not prevent it.

If your team stores these events in a database, a simple table could be:

create table ai_interaction_log (
  id bigserial primary key,
  created_at timestamptz not null default now(),
  customer_id text,
  channel text not null,
  user_message text not null,
  retrieved_documents jsonb,
  model_output text,
  citations jsonb,
  topic text,
  decision text not null,
  shown_to_customer boolean not null default false
);

What it does: this creates a structured audit table for AI interactions. The gotcha: set a deletion schedule and role-based access (who is allowed to view it), or you may create a privacy and security problem while trying to solve an accountability problem.

Contract and product checklist

For a customer buying AI work from an agency, ask for these items in writing:

• a list of allowed and blocked use cases
• who owns source-content accuracy
• whether answers are shown directly or reviewed by staff
• what logs are kept and for how long
• incident response steps when AI gives harmful advice
• wording for customer-facing disclaimers
• service boundaries between agency, AI vendor, and your business

If these are not written down, responsibility will be argued later under pressure.

Further reading

• NIST AI Risk Management Framework
• OWASP Top 10 for Large Language Model Applications
• ISO/IEC 23894 Information technology — Artificial intelligence — Risk management
• The "Security and Privacy" sections of your AI provider's official documentation
• The "Logging" and "Access Control" chapters of the OWASP Cheat Sheet Series

This article was written by an AI system and published pending human review. Verify anything you intend to act on.