On August 28, 2026, NVIDIA released TensorRT Model Connect in public preview, an open-source toolkit that takes a supported Hugging Face model ID or local checkpoint to a versioned .bundle artifact and then runs inference through native C++ task APIs — no PyTorch, no Python interpreter, and no intermediate ONNX export required at runtime.

The workflow splits into two phases. A Python CLI builds the bundle: trtmc build Qwen/Qwen3-0.6B -o qwen3-0.6B.bundle. The resulting artifact contains TensorRT engines and model-specific assets. A C++ application then loads the bundle and calls task-level APIs such as generate(), transcribe(), generate_image(), embed(), and solve().

What's new

  • Two-command deployment: trtmc build resolves the checkpoint, constructs TensorRT engines, and packages them; trtmc run or the C++ trtmc::load() executes inference.
  • Versioned .bundle artifact: The handoff between Python build and C++ runtime is auditable via trtmc inspect, which exposes bundle kind, model family, precision, runtime identity, and engines.
  • Two C++ API levels: A semantic API for task-level inputs/outputs and a module-level API for direct tensor and component control. Both use the same underlying implementations.
  • Custom kernel integration: TVM FFI lets developers replace targeted portions of a model with custom GPU kernels while TensorRT executes the rest of the pipeline.
  • Platform support (preview): Linux aarch64 wheels for Python 3.10/3.12, glibc 2.39+, TensorRT 11.1.0.106. x86_64 users must build from source inside Docker.
  • Coverage snapshot (July 29, 2026 GB300): 105 profiles across 76 model families; 102 profiles beat their declared reference by more than 5%.
  • License: Apache-2.0.

Why it matters

The conventional path — PyTorch → ONNX or TorchScript → TensorRT → per-model C++ glue — introduces export gaps, repeated integration work, and validation spread across multiple conversion artifacts. TensorRT Model Connect removes those steps for supported families, giving robotics, automotive, medical-device, and edge teams a PyTorch-free runtime path on ARM-based NVIDIA hardware (Jetson, Grace, GB-series). The aarch64-only wheel constraint means x86_64 teams face a source-build workflow today, which will filter early adoption toward teams that already own their inference stack on NVIDIA ARM platforms.

Our take

The agentic development claim — the entire project built with OpenAI Codex agents under human review — is the most striking signal. If a first-party infrastructure vendor can ship a shippable, multi-family inference toolkit this way, the bottleneck for new model support shifts from engineering capacity to validation rigor. The real test is whether nightly releases and automated validation can sustain quality as the model zoo expands beyond the 76 families covered in the GB300 snapshot.

Sources