Skip to main content
Keep one loaded PremoveITN instance resident and reuse it across requests.

​
Separate the lifecycle costs

The benchmark excludes download and initialization. A one-shot CLI measurement is not comparable to retained warm latency.

​
Select a device

device="auto" selects CUDA when available, then Apple MPS, then CPU. It prefers CUDA when available, but CUDA is not a validated v0.3.0 platform. Use CPU on supported Linux or MPS on supported Apple Silicon when you require a validated runtime. Pass cpu, mps, or cuda to select a device explicitly. An unavailable explicit device fails with a runtime error. Release wheels are validated on macOS 14+ arm64 and manylinux_2_28 x86_64 for Python 3.11–3.13. Frozen-model inference is validated on Apple Silicon MPS and Linux x86-64 CPU. CUDA, Windows, macOS Intel, Linux ARM64, and other accelerators are not validated v0.3.0 support claims. See the platform support matrix for the exact release boundary.

​
Pin production deployments

Pin the package release:
The package pins and verifies the model artifact. For an offline deployment, download the artifact in advance and pass its local directory to PremoveITN.from_pretrained(). For an existing installation that should follow the current release, upgrade the package explicitly:
The upgraded package selects its pinned v0.2.0 model revision automatically when from_pretrained() runs. It downloads only that revision when it is not already in the Hugging Face cache; it does not mutate an older cached release.

​
Protect service capacity

The scorer has 435.6 million parameters. Account for model memory and startup time when you choose process counts and readiness checks. Do not load a new model instance for every request.