Skip to main content

​
What is selected

The production model is the final structured-value 20k checkpoint based on microsoft/deberta-v3-large. It is the only retained checkpoint. The model saw 418,000 training examples in three stages: The frozen VoiceAgent ITN dataset was not used for training or model selection.

​
Training sources

The base stage used the Google Text Normalization training partition. The conversational stage combined Google positive replay with conversational examples derived from:
  • SLURP real textual annotations
  • Schema-Guided Dialogue
  • SpokenWOZ
  • Taskmaster-1 spoken data
The final structured-value stage contained:

​
Training kind distribution

The table counts every example seen by the selected checkpoint.

​
Training configuration

The Google stage used AdamW, learning rate 2e-5, weight decay 0.01, batch size 8, local length bucketing, and a deterministic suffix shuffle. Both adaptation stages used AdamW, learning rate 5e-6, weight decay 0.01, batch size 8, one epoch, and a fresh optimizer state. The structured stage used microbatch size 2. Validation and test rows were not used for training.

​
Why this checkpoint was selected

The final checkpoint was selected because it gave the best product-relevant balance for structured values and conversational normalization while retaining strong general ITN accuracy. It improved electronic values, identifiers, numeric IDs, phone values, conversational validation, and the targeted validation set. The final recorded development results were: The Numb3rs result was below its predeclared regression floor of 76.39%. The checkpoint was accepted as a documented promotion exception because its gains on the product-relevant structured and conversational tasks were larger. This tradeoff must remain visible. The development results must not be presented as the final blind VoiceAgent benchmark result.

​
Benchmark separation and current limits

Before deleting the training rows, all 418,000 exposed training inputs were compared with all 1,500 frozen VoiceAgent inputs. There were no exact matches and no matches after lowercasing and removing punctuation. Token n-gram and embedding-similarity contamination checks were not completed. The VoiceAgent dataset passed a full independent mechanical and semantic audit, but blind human adjudication remains pending. First Evaluation has run on the frozen dataset. Its retained outputs are preserved under eval/voice_agent_itn/results/first-evaluation/. This is the release measurement and candidate equivalence audit. The bulk training corpora and historical raw evaluation outputs were deleted. The repository can load and verify the selected checkpoint, but it cannot reproduce the full training run without reacquiring and rebuilding the source datasets. Dataset licenses and provider terms must be checked before any source corpus is reacquired or redistributed.