AI model porting and optimization: Deepgram example

What the source reports

Deepgram and AWS FDE engineers report porting the Nova-3 speech model to Trainium2. They used NAD profiling skills to investigate bottlenecks and kernel-writing skills to help produce replacement kernels. Engineers checked layer outputs against a CPU reference and end-to-end word error rate against Deepgram’s targets. The Rust serving integration reproduced reference token sequences at every tested batch size.

Reported technology: AWS Trainium2, Neuron and Neuron Agentic Development (NAD). Scope and outcomes are source-reported.

What the enterprise environment would need

The following is a proposed project design for an enterprise doing similar work. Published details are marked; the remaining systems, data and scenarios would be agreed for your project.

Model and serving stack

Proposed: use an approved model, pinned compiler and runtime versions, the enterprise’s serving interfaces and a permitted accelerator configuration. Agree hardware access explicitly; a substitute cannot establish performance on the target hardware.

Data and reference results

Proposed: use permitted audio and transcripts, fixed evaluation splits, expected outputs and representative batch sizes. Keep a separate test set that the coding agent cannot change.

Engineering responsibilities

Proposed: the agent investigates profiles and suggests kernel changes. Engineers review the code, numerical differences and integration failures. The model and service owners set acceptance criteria and decide whether to roll out a change.

A proposed simulation

An illustrative practice design derived from this work, not an executed assessment.

Port a permitted speech model to an agreed target runtime, use an agent to investigate one performance bottleneck, and submit the change with reproducible correctness and performance results.

Review gate: A reviewer other than the coding agent reruns the checks from the pinned inputs. The accountable owner reviews failed cases before accepting the change.

Challenge the result: Introduce a faster kernel that changes recognition results, then a batch size that exceeds a resource limit. The checks should expose both failures.

What needs checking

• Compare outputs and word error rate with the agreed reference. Reject faster kernels that breach correctness tolerances.
• Confirm operations run on the intended accelerator. Test compilation, memory limits and the serving interface across agreed batch sizes.
• Measure throughput and resource use on the same workload and hardware baseline. Retain failures, versioned results and the reviewer’s decision.

Review environment fidelity, human capability and agent performance separately. The enterprise’s accountable owners decide acceptance of the shared workflow and any operating use.

How an enterprise could build similar work

A proposed build order for an enterprise doing similar work, written by Nuvepro from the published account and our interpretation of the environment and checks above. It is not how Deepgram and AWS Forward Deployed Engineering built or operates this work.

1. Agree model, hardware and scopeThis is a proposed design for a similar project, not Deepgram's own process. Use an approved model, pinned compiler and runtime versions and a permitted accelerator. Agree hardware access explicitly, because a substitute cannot establish performance on the target hardware.
2. Prepare data and referencesUse permitted audio and transcripts, fixed evaluation splits, expected outputs and representative batch sizes. Keep a separate test set that the coding agent cannot change, so passing it carries real meaning.
3. Assign agent and engineer rolesThe agent investigates profiles and suggests kernel changes. Engineers review the code, numerical differences and integration failures. The model and service owners set acceptance criteria and decide whether to roll out a change.
4. Check correctness firstCompare outputs and word error rate with the agreed reference. Reject faster kernels that breach correctness tolerances, because speed gained by changing recognition results is not an improvement that anyone should accept.
5. Verify hardware and serving fitConfirm operations run on the intended accelerator. Test compilation, memory limits and the serving interface across agreed batch sizes, and measure throughput and resource use on the same workload and hardware baseline.
6. Rerun independently and reviewA reviewer other than the coding agent reruns the checks from pinned inputs. Introduce a faster kernel that changes results and a batch size beyond a resource limit; both failures should appear. The accountable owner reviews failed cases.

Your systems, data, rules and accountable owners decide the real project. Agree acceptance criteria, escalation owners and operating responsibility with the people accountable for the work before anything runs on live records. Treat the first run as a rehearsal on test records: compare the draft output with an independently prepared answer, and record what the people reviewing it change.

Questions about this example

What does Deepgram use AI for in model porting?

Engineers used Neuron Agentic Development profiling skills to investigate bottlenecks and kernel-writing skills to help produce replacement kernels while porting Nova-3 to Trainium2. The work covers offline batch speech recognition, and a Rust serving integration was evaluated.

Who checks the agent's kernel changes at Deepgram?

Engineers checked layer outputs against a CPU reference and end-to-end word error rate against Deepgram's targets. The Rust serving integration reproduced reference token sequences at every tested batch size. The source does not describe autonomous operation.

How strong is the evidence for this Deepgram example?

The evidence class is reported use: a joint operator and vendor account of a completed engineering port, not an independent audit. Results concern the offline batch configuration. The source does not establish fleet-wide production rollout, fine-tuning or general production readiness.

What would an enterprise need to build something similar?

This is Nuvepro's proposed design, not Deepgram's method. It would use an approved model, pinned compiler and runtime versions, agreed hardware, fixed evaluation splits and a test set the coding agent cannot change, with correctness and throughput checked by a separate reviewer.

Shape this around your enterprise

Bring a similar workflow and the people responsible for it. Together, define the records and systems to use, the result your team will build or configure, and how its owners will check it. Nuvepro scopes the simulation, environment and preparation. Agree any existing-agent evaluation, implementation assistance and production responsibilities separately.

Sources and limitations

Curation review: 20 October 2026. This is the date this source example was reviewed, separate from source publication dates.

• This is a joint operator and vendor account, not an independent audit. Results concern the reported offline batch configuration.
• Agent assistance in engineering does not establish autonomous operation, clinical suitability, model fine-tuning or general production readiness.
• The environment and checks proposed here have not been built or executed by Nuvepro. Hardware, model rights, data access and implementation scope would need agreement.

1. AWS: How Deepgram optimized voice AI at scale with AWS TrainiumSource publication: 2026-10-19

More software engineering examples

• AI engineering agent coordination: Deel exampleDeel
• AI synthetic evaluation: Microsoft exampleMicrosoft
• AI test failure triage: Rescale exampleRescale
• AI chip verification workflows: OpenAI exampleUnnamed semiconductor company

All 9 software engineering examples