Reported operating use · Customer-service software
AI customer service actions: Intercom example
Intercom uses OpenAI GPT-4.1 in Fin Tasks and the Realtime API in Fin Voice for refunds, account changes, subscription updates and troubleshooting. The source reports offline tests and live A/B trials on real support transcripts, with GPT-4.1 rolled out after evaluation.
Refunds, account changes, subscription updates and troubleshooting
Intercom
Production Fin Tasks model rollout; customer-facing platform use is reported, without named end-customer action breakdown.
A published third-party example, not a Nuvepro customer case or evidence of a partnership.
The workflow at a glance
Intercom reported work
Illustrative summary of the published account
Reported work components. Their arrangement does not imply a fixed execution order or simultaneous work.
- Support product
Fin Tasks
Refund, account, subscription and troubleshooting work.
- Evaluation
Offline and live comparisons
Tests assess instructions, tool calls and coherence using support transcripts.
- Voice testing
Interruptions and noise
Voice evaluations include disrupted conversations.
What the source reports
Intercom evaluates instruction following, tool-call accuracy and coherence offline and through live A/B trials. Tests use real transcripts; voice tests also cover interruptions/noise. GPT-4.1 was rolled out after evaluation.
Reported technology: OpenAI GPT-4.1 in Fin Tasks; Realtime API in Fin Voice. Scope and outcomes are source-reported.
What the enterprise environment would need
The following is a proposed project design for an enterprise doing similar work. Published details are marked; the remaining systems, data and scenarios would be agreed for your project.
Proposed practice environment
Verify a support resolution across connected records
Illustrative project design to scope with your enterprise.
- 1 · Set up the environment
Support-state fixtures
Fictional tickets, accounts, subscriptions, refunds and interrupted-call histories.
- 2 · Preparation and agent contributions
Complete permitted test task
Prepare a support resolution and recover from a function-call failure.
- 3 · Independent checks
Verify actual state changes
Independent checks reconcile records, policy and complete person handoff.
- 4 · People decide acceptance
Support owner accepts resolution
Review repeated-run outcomes separately from conversation containment.
A reviewable evidence package
A verified test resolution record and context-preserving handoff.
If a check fails → withhold acceptance.
A tool claims success while the requested record stays unchanged. Withhold acceptance, repair, independently recheck and return to owner review.
Failure or owner rejection returns to review
- Hold
- → 2 · Revise work
- → 3 · Rerun independent checks
- → 4 · Owner review
Operating boundary. Test task tools only; no live account edits, refunds or external customer messages.
- Systems and tools
- Inferred: integrate with existing helpdesk and task tools; reproducing them externally may reduce fidelity.
- Data and state
- Inferred: ticket, subscription, account and refund state plus conversation history.
- People and handoffs
- Inferred: customer/support-agent/escalation handoffs need task-specific observed evidence.
- Rules and authority
- Inferred: refund eligibility, account-change permissions and escalation limits.
- Normal and exception cases
- Inferred: duplicate refunds, identity ambiguity, interrupted calls and failed backend actions.
- Reset and replay
- Not disclosed as reset system. Published transcripts and repeat tests provide reusable evaluation assets.
A proposed simulation
An illustrative practice design derived from this work, not an executed assessment.
Complete a refund or subscription update, recover from a function-call failure and preserve context when a person takes over.
Review gate: Check actual record changes, policy compliance and repeated-run success; measure successful resolution separately from containment.
Challenge the result: Make a tool report success without applying the requested record change. The outcome verifier must fail the case despite a convincing customer response.
What needs checking
- Published: Structured offline instruction/function-call tests, five-run completeness criterion, live resolution/CSAT A/B tests.
- Proposed: Only add external cases where they improve independent policy or cross-system outcome coverage.
- Published: Benchmarks use actual support transcripts; live tests compare customer outcomes.
- Proposed: Compare simulated tool state and voice conditions with deployed behavior.
Review environment fidelity, human capability and agent performance separately. The enterprise’s accountable owners decide acceptance of the shared workflow and any operating use.
Questions about this example
- What does Intercom use AI for in customer service?
- Intercom uses OpenAI GPT-4.1 in Fin Tasks and the Realtime API in Fin Voice. The workflow covers refunds, account changes, subscription updates and troubleshooting. The source reports that GPT-4.1 was rolled out after Intercom evaluated it.
- Who checks the AI's work at Intercom?
- The source describes Intercom's own evaluation of instruction following, tool-call accuracy and coherence, offline and in live A/B trials. It does not say which actions need human approval. A proposed check is to measure successful resolution separately from containment.
- How strong is the evidence for Intercom's use?
- It is reported use: a production model rollout for Fin Tasks, described by the company. The source gives no named end-customer action breakdown, and customer-specific action results and independent replication of the reported evaluation outcomes are not supplied.
- What would an enterprise need to build something similar?
- This is Nuvepro's proposed design, not Intercom's. It would need ticket, subscription, account and refund state, conversation history, and refund and permission rules. Tests would include duplicate refunds, identity ambiguity, interrupted calls and failed backend actions, with a verifier checking actual record changes.
Shape this around your enterprise
Bring a similar workflow and the people responsible for it. Together, define the records and systems to use, the result your team will build or configure, and how its owners will check it. Nuvepro scopes the simulation, environment and preparation. Agree any existing-agent evaluation, implementation assistance and production responsibilities separately.
Sources and limitations
Curation review: . This is the date this source example was reviewed, separate from source publication dates.
- Customer-specific action results and independent replication of the reported evaluation outcomes are not supplied.
- Intercom's three lessons for creating a sustainable AI advantageSource publication: 2025-07-30