2.6.8

Issue

Conv.make_node() reads input_tensor_shape before the workaround that transposes
an NCHW input to NHWC, and never refreshes it.

The depthwise test compares group against input_tensor_shape[-1]. For a transposed
input this compares the channel count against a spatial dimension, the test fails, and
the node falls through to the grouped-conv path — producing a filter of shape
[C, K, K, 1] instead of a DepthwiseConv2D.

Symptom: in a MobileNet/ConvNeXt-style network only the first depthwise conv of each
stage converts correctly (it consumes a conv output directly), while the following ones
are silently degraded (they consume a residual Add whose input required the transpose).

The ONNX model is valid — depthwise is Conv with group=C and weight [C, 1, K, K].

Fix

Refresh input_tensor_shape from input_tensor right before the depthwise test.

What's Changed

• Fix depthwise Conv detection after NCHW->NHWC input transpose (Legacy converter) by @AL3708 in #959

Full Changelog: 2.6.7...2.6.8

2.6.7

Summary

This pull request continues the characterize-first refactoring of the TensorFlow-free flatbuffer_direct backend. It restructures late shape, layout, topology, fallback, and terminal-cleanup orchestration without changing public conversion policy or using newly recorded evidence to alter graph-rewrite control flow.

The main outcome is a smaller and more reviewable central lowerer whose phase boundaries are explicit, whose repeated orchestration sequences have focused owners, and whose observation-only results are retained in a bounded conversion-session store under stable phase IDs.

The branch also includes the full sequential Tier 0–4 TFLite/native-PyTorch regression evidence for the final checkpoint and updates the package version metadata to 2.6.7.

Why this change is needed

The direct FlatBuffer path has accumulated many interacting shape, layout, topology, compatibility, and model-recovery rules. Even when each rule is locally correct, several structural properties made maintenance difficult:

• adjacent pass calls were coordinated directly inside the central lowerer;
• repeated sort/layout-refresh and shape/reconciliation sequences had multiple owners;
• useful mutation and validation results were sometimes discarded or kept in unconsumed local dictionaries;
• phase identity was implicit in source position rather than represented by a stable ID;
• guard-skipped work was difficult to distinguish from work that ran and produced no mutation;
• terminal invariants and fallback recovery boundaries were spread across a very large function;
• changing ownership could easily invalidate structural assumptions in distant tests.

This branch makes those contracts explicit before attempting more aggressive scan elimination or control-flow optimization.

Architecture changes

Bounded phase-result storage

ConversionSession now owns a bounded, conversion-local phase-result store.

• Results are recorded under 128 stable phase IDs.
• Each phase has one deterministic owner expression.
• Stored values are compact counters or bounded summaries rather than graph-sized snapshots.
• The store copies mappings so later caller mutation cannot silently change recorded evidence.
• “not invoked” remains distinct from “invoked with zero mutations”.
• No recorded counter is consumed to skip or reorder production work in this PR.

This removes observation-only lowerer locals while giving future optimization work a stable basis for differential testing.

Shared topology and shape owners

Repeated multi-step sequences now have focused pass-module owners, including:

• topology sort followed by logical-layout refresh;
• complete static-shape reconciliation followed by topology restoration;
• terminal topology sort and logical-layout validation;
• bounded primary and fallback topology checkpoints;
• safe transpose reduction, dynamic reshape resolution, and terminal expand/squeeze reconciliation evidence.

The owners preserve the original arguments, guards, ordering, cycle behavior, metadata updates, and temporary-state lifetime.

Explicit late and terminal orchestration

Large groups of previously adjacent lowerer calls were extracted into narrowly scoped orchestration owners. The extracted families include:

• QLinear and quantized layout recovery;
• attention/QKV, BatchMatMul, window, and reshape-layout recovery;
• binary adapter and broadcast convergence;
• pre-Concat, Slice/Concat, channel-slice, and terminal bridge cleanup;
• late Conv1D/decoder and Pad/InstanceNorm recovery;
• fallback norm, precision, unbound-input, and recurrent-alias repair;
• SiNet CSP attention, residual, HardSwish/SE, and terminal recovery;
• final-input dynamic-shape, final boundary, and terminal stabilization cleanup;
• prune-aware summaries for PRELU, Conv-input, affine, hard-activation, SPP, and normalization clusters.

These owners return the same child mappings or strict summaries in the same source order. Existing lowerer compatibility wrappers are retained where instrumentation or callback injection depends on them.

Central lowerer simplification

The central lower_from_onnx2tf.py no longer manually coordinates each extracted cluster. It delegates orchestration while retaining the existing phase order and records only the bounded result required by the conversion session.

The refactoring deliberately preserves:

• graph mutations and ModelIR serialization inputs;
• logical/physical layout decisions;
• producer/consumer and graph-index behavior;
• diagnostics and failure propagation;
• public compatibility wrappers;
• option guards and per-pass evaluation counts.

Compatibility guarantees

• Public CLI and Python APIs are unchanged.
• flatbuffer_direct remains the default backend.
• Artifact names, report formats, return behavior, and exception behavior are unchanged.
• Normal direct TFLite conversion and -cotof do not import TensorFlow.
• Optional TensorFlow exporters remain behind the existing optional boundary.
• No dependency was added.
• No multiprocessing or parallel inference path was introduced.
• TFLite and native PyTorch exporters continue to consume the same validated conversion state.

Characterize-first workflow

Each extraction was split into two checkpoints:

1. characterization tests fixed the owner boundary, call order, arguments, no-op behavior, result schema, graph effects, and neighboring phase position;
2. production code was then moved behind the characterized owner, followed by focused and affected sequential test gates.

When an ownership move intentionally invalidated a structural source assertion, the initial failure was captured first. Tests were then updated to assert the new owner boundary while preserving runtime behavior. This avoids masking behavioral regressions as mechanical test maintenance.

The branch contains detailed implementation and checkpoint records in:

• docs/fb_refactor8_improvements.md
• docs/flatbuffer_direct_handoff_fb_refactor8.md
• docs/fb_refactor8_pull_request_description.md

Full Tier 0–4 regression audit

The final branch checkpoint was validated against every eligible root-level ONNX model in Tiers 0–4.

Fixed scope

• 456 root ONNX files were inventoried.
• 420 loadable models contained 1–1,999 ONNX nodes.
• 29 managed timeout models and 12 explicit user exclusions were omitted.
• 379 models were executed in the fixed profile order.
• Each model had a 600-second timeout.
• The entire descendant process tree was monitored for VmSwap.
• Inference concurrency remained exactly one.
• Each invocation generated and checked TFLite plus native PyTorch output only.

Result

• 379/379 eligible models completed.
• 0 timeouts.
• 0 models reached ten minutes.
• 0 SWAP detections.
• 0 combined-classification differences from fb-refactor7.
• 0 TFLite component-state differences.
• 0 native-PyTorch component-state differences.
• 0 normalized failure-signature differences.
• 0 confirmed regressions attributable to this branch.

Raw component counts remained exactly equal to the previous branch:

The strict runner intentionally retains inherited missing reports and accuracy failures. Those entries were compared model-by-model rather than counted as new regressions.

The only numeric difference was randnlike4.onnx, whose native-PyTorch output is intentionally nondeterministic. Its classification was unchanged. The two DEIM models retain the user-approved near-tied TopK-index disposition. The two DPT-LeViT conversion errors are inherited and unchanged.

Timing

Total per-model conversion time decreased from 7,689.33 seconds to 7,376.09 seconds (-4.07%). Every Tier 0–4 aggregate total and median was below the fb-refactor7 observation. The slowest model completed in 208.95 seconds, well below the 600-second boundary.

Pass-efficiency metrics were exactly equal between the two branches:

• 377 models with pass metrics;
• 196,325 pass events;
• 24,535,648 preflight operator visits;
• 22,321 state builds;
• 5,419 snapshots;
• 0 fingerprints.

Full evidence is recorded in:

• docs/flatbuffer_direct_tier0_4_full_regression_2026-07-20.md
• docs/baselines/flatbuffer_direct_tier0_4_tflite_native_pytorch_fb8_b35c36a0.json

Test coverage

All inference and conversion tests were run sequentially under uv.

The branch includes focused contracts for:

• pass match/guard/rewrite/no-op behavior;
• owner call order and argument identity;
• graph and layout-state identity preservation;
• topology cycles and stale validation metadata;
• dynamic/static shape reconciliation;
• quantized and QLinear layout recovery;
• terminal, fallback, attention, SiNet, and Conv1D composite owners;
• deterministic phase ordering and bounded result storage;
• public compatibility wrappers and instrumentation callbacks;
• TensorFlow/tf-keras import blocking for direct conversion;
• lowerer architecture boundaries.

The final bulk-runner and corpus-manifest gate reports 44 passed tests. Numerous focused and affected suites were run at each characterize/extract checkpoint; their commands and exact counts are preserved in the branch documentation.

Temporary-storage handling

The full corpus run retained logs, JSON reports, pass metrics, and text diagnostics while deleting only reproducible artifacts after their model entry was atomically persisted. It reclaimed 51,216,397,607 bytes (47.70 GiB), then removed the uniquely named temporary run directory after condensed evidence validation.

Package metadata

The package version, lockfile entry, and documented container tags are updated consistently from 2.6.6 to 2.6.7. uv lock --check and an import-time metadata consistency assertion both pass.

Intentionally deferred work

This PR does not use the new phase evidence to eliminate graph scans or skip passes. Such optimization should be a separate change with mutation-positive and stable-path differential evidence for operator order, layout state, cycle handling, ModelIR digest, serialized artifacts, and numerical output.

The broader flatbuffer_direct refactor is therefore not declared complete by this PR. This checkpoint provides explicit ownership, bounded evidence, strong regression characterization, and a safer foundation for subsequent optimization without changing current public behavior.

What's Changed

• Refactor flatbuffer_direct phase orchestration with bounded evidence by @PINTO0309 in #958

Full Changelog: 2.6.6...2.6.7

Summary

This branch continues the staged refactoring of the TensorFlow-free
flatbuffer_direct backend. It makes pass ownership, mutation evidence, graph
index reuse, reconciliation decisions, and terminal validation substantially
more explicit while preserving the existing conversion policy and public
interfaces.

The central goal of this checkpoint is not to introduce a new converter or to
change the order of the existing compatibility rules. It is to make the
current behavior safer to evolve:

• pass results are retained instead of being silently discarded;
• graph-stability decisions use complete, typed mutation evidence;
• stable paths avoid redundant full-graph reconciliation scans;
• repeated adjacent cleanup passes share one live ModelIRGraphIndex;
• terminal layout validation observes the graph after the applicable repairs;
• focused owners and contracts isolate changes from unrelated model families;
• every production change follows a characterize-first regression workflow.

The release metadata is updated consistently from 2.6.5 to 2.6.6 in the
package, project metadata, lockfile, and documented container tags.

Motivation

The direct FlatBuffer path has accumulated many interacting layout, shape,
quantization, recovery, and compatibility rules. Historically, a number of
those rules were invoked through raw calls in the central lowerer, returned
useful information that callers discarded, rebuilt producer/consumer state at
nearby boundaries, or triggered reconciliation even when the preceding owner
made no graph mutation.

Those patterns make apparently local changes difficult to reason about. A
counter can be incomplete because cleanup happened outside the counter, a
zero result can be mistaken for graph stability, or two adjacent passes can
independently rescan the same graph. This branch establishes explicit contracts
for these boundaries before using them for conservative scan elimination or
index sharing.

Detailed changes

Efficient diagnostic bookkeeping

run_model_ir_pass_group() no longer scans the complete and growing
diagnostic history for every emitted event. One scan initializes the global
event count, maximum group sequence, and per-pass invocation counts. Those
counters are then updated as events are appended.

ConversionSession retains this state across pass groups. Ordinary
caller-owned lists remain supported through a safe one-scan fallback. The
external diagnostic fields, numbering semantics, list identity, skip/cycle
behavior, and invariant-failure reporting are unchanged.

Explicit pass results and mutation evidence

Late, very-late, terminal, recovery, attention, quantized, binary-layout,
Conv1D, and safety-fallback boundaries now return or stage fixed-schema result
dictionaries instead of dropping child results. Multi-pass orchestration
preserves child order and shared pass-state scopes.

Observation-only results remain intentionally unconsumed when their counters
are not yet safe control-flow inputs. Where a stability decision is made, the
owner contract accounts for every relevant mutation, including cleanup-only
tensor pruning. Non-mutating iteration counters are excluded from mutation
summaries.

This gives later work auditable evidence without adding ModelIR copies,
fingerprints, or extra graph walks.

Complete static-shape reconciliation accounting

Static-shape reconciliation can now opt in to a complete mutation count. In
addition to ordinary tensor-shape updates, it records operator-option changes,
constant shape-parameter writes, and direct tensor metadata updates performed
during the existing fixed-point walk.

The default result schema remains compatible. Only characterized call sites
request the additional evidence, and their original guards, pruning behavior,
layout synchronization, and pass order are preserved.

This complete evidence is retained across primary, fallback, very-late, and
post-split reconciliation boundaries, including Conv-input, mixed-Concat,
Concat-axis, high-rank BatchMatMul, Pad, placeholder-MatMul, singleton Reshape,
SE/FC/Gather, PReLU, and SiNet paths.

Conservative stable-path fast paths

Broad reconciliation is skipped only when all declared mutation evidence from
the immediately preceding owner is zero. This applies to selected singleton
Reshape, PReLU, SiNet, SE/Gather, late binary, shared late, final-shape,
post-fusion, placeholder-MatMul, and bounded convergence paths.

Mutation-positive behavior is unchanged. Cleanup-only owners use clamped net
tensor-count deltas where required, so a rewrite counter of zero is never
silently treated as stability when pruning may still have changed the graph.

Binary-layout convergence can also stop after a stable round instead of
running the remaining bounded rounds. The maximum round count and all
mutation-positive behavior remain unchanged.

Shared graph indexes for adjacent cleanup work

run_indexed_prune_reconcile_cleanup() shares one ModelIRGraphIndex across
dead-operator pruning and static-shape reconciliation at three repeated phase
boundaries. It performs exactly those two existing operations and does not add
dynamic-reshape resolution, retries, or a new convergence branch.

run_indexed_binary_layout_adapter_cleanup() similarly shares one index
between the exact rank-four binary adapter and the singleton-broadcast adapter
at four repeated boundaries. The exact adapter still runs first. Both owners
now use indexed candidate lookup, operator insertion, and input replacement,
while preserving names, tensor metadata, quantization cloning, pruning, result
schemas, and existing guard inputs.

Fallback unbound-input repair also avoids one redundant reconciliation because
its indexed compatibility wrapper already performs the required positive-path
reconciliation with the live graph index.

Terminal validation after repair

Primary and fallback terminal layout validation now observe the graph after
the applicable terminal mutations and final topological ordering. When the
terminal graph is valid, validation clears only stale validation errors
inherited from recursive lowering. The diagnostic schema and actual error
conditions are unchanged.

The branch retains the complete terminal mutation dictionaries used to make
this boundary reviewable, including bounded binary convergence, high-rank
binary coalescing, boundary-signature realignment, high-rank BatchMatMul, Pad,
Conv-input, mixed-Concat, Concat-axis, stale-binary, and SiNet repairs.

Smaller, testable internal owners

Several responsibilities were moved behind focused internal interfaces:

• typed ONNX Constant lowering lives in its op-family module;
• demand-driven shape readiness lives in a dedicated core helper;
• late binary recovery has a dedicated runner and stable result schema;
• hard-activation, SPP, QKV, channel-slice/pad-Mul, affine, quantized, and
  attention orchestration exposes ordered results directly;
• NodeView value-list annotations and ONNX graph-input typing are clearer to
  static analysis without changing runtime behavior.

Private compatibility wrappers remain where existing structural callers rely
on them. The central lowerer keeps the same production call positions and
ordering unless a characterized shared runner replaces an exactly adjacent
pair.

Characterize-first safety workflow

Each behavior-changing implementation unit was preceded by a focused contract
that froze the relevant result schema, no-op path, mutation path, order,
arguments, state-scope ownership, and surrounding phase boundary. The strict
expected failure was removed only after the corresponding implementation and
the broader sequential gate passed.

This branch adds extensive focused coverage for result propagation,
cleanup-only mutations, deterministic ordering, idempotence, shared-index
construction, layout recovery, attention and quantized paths, fallback
relowering, terminal validation, and architectural ownership.

Compatibility and dependency boundaries

• The public CLI and Python API are unchanged.
• flatbuffer_direct remains the default backend.
• Existing artifact names, report formats, return behavior, and pass order are
  preserved.
• Normal direct TFLite conversion and -cotof do not import or execute
  TensorFlow.
• Optional SavedModel/H5/Keras/TFv1 behavior remains behind the existing
  TensorFlow-optional boundary.
• No dependency was added.
• No new exporter, quantization path, split path, inference worker, or
  multiprocessing behavior was introduced.
• Corpus inference and conversion validation remained strictly sequential.

Validation

Focused and structural regression gates

At the final implementation checkpoint before the corpus run:

• dedicated indexed binary-adapter contract: 3 passed;
• focused affected-owner suite: 424 passed;
• all 116 branch-changed test files: 1707 passed;
• targeted Ruff, Python bytecode compilation, and whitespace validation:
  passed.

The broad gate includes the changed indexed owners, late and terminal
orchestration, static-shape reconciliation, shared ModelIR pass context, core
contracts, pass-efficiency checks, architecture constraints, and the optional
TensorFlow import boundary.

The corpus runner and manifest tests were also repeated after the full run:
44 passed.

Tier 0-4 TFLite and native PyTorch corpus

All 379 active managed Tier 0-4 models were converted strictly one at a time at
converter checkpoint 0bf1bab4. The scope excludes only previously recorded
timeouts and explicit user exclusions.

• completed models: 379 / 379;
• per-model timeout ceiling: 600 seconds;
• models at or above the ceiling: 0;
• converter process trees with nonzero SWAP: 0;
• TFLite/native-PyTorch component-state differences from fb-refactor6:
  0;
• combined-classification differences: 0;
• exit-state and strict-pass differences: 0;
• normalized failure-signature differences: 0;
• confirmed branch-specific regressions: 0.

The combined classifications exactly match the authoritative fb-refactor6
comparison:

The run requested TFLite and the native PyTorch package only. It did not
request TorchScript, Dynamo ONNX, ExportedProgram, or TensorFlow artifacts.

Timing and pass-efficiency observations

This full-corpus run is a single regression observation, not the broader
three-run warm-median performance gate.

• total wall time: 7,689.329 s (+1.945%);
• median model wall time: 8.755 s (+0.109%);
• maximum model wall time: 201.599 s (+5.332%);
• every tier total and median remained within +10% of the comparison run.

Both runs recorded exactly 196,325 pass events. state_build_count,
snapshot_count, and fingerprint_count were unchanged, while
preflight_operators_visited decreased by 16. This provides no evidence of an
added scan or state-build regression.

Known inherited outcomes

This PR does not claim that every corpus model has numerical parity. It claims
that the branch introduces no new result or failure-signature regression
relative to the authoritative previous checkpoint.

• Two DPT-LeViT models retain their existing Softmax/rank conversion errors.
• Existing TFLite numerical failures, missing reports, native PyTorch failures,
  and native PyTorch missing reports keep the same classifications and
  normalized signatures.
• The two DEIM models remain accepted TFLite successes under the recorded
  near-tied TopK-index policy.
• randnlike4.onnx retains the same native-PyTorch known-failure class. Its
  nondeterministic max-absolute value decreased from 1.828334451 to
  1.811047643; this is not treated as a converter regression.

Evidence and review guide

• Cumulative design and validation notes:
  docs/fb_refactor7_improvements.md
• Architecture and pass-boundary contracts:
  docs/flatbuffer_direct_architecture.md
• Full Tier 0-4 regression report:
  docs/flatbuffer_direct_tier0_4_full_regression_2026-07-18.md
• Condensed 379-model evidence:
  docs/baselines/flatbuffer_direct_tier0_4_tflite_native_pytorch_fb7_0bf1bab4.json
• Detailed continuation history:
  docs/flatbuffer_direct_handoff_fb_refactor7.md

The generated model artifacts were intentionally not committed. After the
evidence was persisted and validated, 48.109 GiB of reproducible temporary
artifacts and the remaining run-owned /tmp files were removed.

Remaining scope

This is a coherent refactoring checkpoint, not completion of the entire
long-term flatbuffer_direct redesign. Later work may continue the fixed
ConversionRequest/ConversionSession contract, phase/pass manager,
centralized layout planning, unified lowering registry, exporter separation,
quantization/split refresh, and PyTorch-family restructuring. Those items are
not silently mixed into this PR.

What's Changed

• Make flatbuffer_direct pass execution explicit, indexed, and regression-safe by @PINTO0309 in #956

Full Changelog: 2.6.5...2.6.6

2.6.5

Summary

This pull request is the sixth staged checkpoint of the flatbuffer_direct
refactor. It moves the remaining large groups of already-supported ModelIR
compatibility logic out of the central ONNX lowerer and into focused,
characterized pass and orchestration modules.

The central implementation in lower_from_onnx2tf.py is reduced from 25,647
lines on main to 5,585 lines. The branch adds 79 focused pass/orchestration
modules and updates or adds 92 test files. The large net addition is primarily
the extracted owners, synthetic characterization fixtures, architecture
contracts, and the recorded Tier 0-4 regression evidence; it is not a second
conversion pipeline.

The main goals of this checkpoint are:

• make pass ownership explicit and local;
• preserve the existing CLI, Python API, artifact formats, pass order, and
  conversion policy;
• prevent partially applied rewrites when a late guard rejects a match;
• reuse graph indexes, layout state, diagnostics, and pass state instead of
  reconstructing equivalent context in nested helpers;
• keep direct TFLite conversion and -cotof outside the TensorFlow import
  boundary;
• provide a reproducible, sequential Tier 0-4 regression result for both
  TFLite and native PyTorch output.

No dependency was added. Project and container metadata are synchronized at
version 2.6.5.

Why this change is needed

The direct FlatBuffer backend accumulated shape repair, layout recovery,
quantization compatibility, fusion, and late cleanup rules in one central
module. Many rules were individually valid, but their implicit ordering and
shared mutation made them difficult to understand or modify safely. A guard
that failed after an early tensor/layout mutation could leave the graph in a
partially rewritten state, and nested recovery helpers repeatedly assembled
nearly identical ModelIR/LayoutState/diagnostic contexts.

This branch follows a deliberately conservative sequence for each family:

1. characterize the existing match, guard, ordering, statistics, metadata,
   and no-op behavior;
2. add rejection, fan-out, boundary, dynamic-shape, quantization, and
   idempotence fixtures;
3. correct mutation atomicity where a rejected candidate could leak changes;
4. extract the implementation into one focused owner while retaining the
   original private lowerer symbol as a thin compatibility wrapper;
5. verify owner/wrapper ModelIR fingerprints and architecture boundaries.

This separates ownership without attempting a broad semantic rewrite at the
same time.

Main improvements

1. Focused pass ownership

The branch extracts shape, layout, quantization, attention, binary, affine,
Concat, and terminal recovery behavior into op-family-oriented modules under
onnx2tf/tflite_builder/passes/.

Representative extracted families include:

• static Squeeze and shape-signature sanitization;
• dynamic boundary-signature realignment;
• ExpandDims/Squeeze-to-Reshape compatibility;
• exact and singleton rank-four binary layout adaptation;
• channelwise broadcast-constant and stale binary-adapter repair;
• Conv/Pool output passthrough and Conv/Mul/Add affine folding;
• mixed-layout Concat input repair and legacy pre-Concat recovery;
• Concat/Mul/Add, nested-Concat, Mean/Reshape, and
  StridedSlice/Pad/Concat bridge recovery;
• Softmax/Transpose canonicalization and Reshape/Transpose collapse;
• BatchMatMul affine input, SE, adjoint, and attention preprojection recovery;
• attention Gather cleanup and QKV reshape compatibility;
• elementwise round-trip, fan-out, residual affine, and terminal PReLU/FC
  recovery;
• QLinear SiLU, QLinear Concat/Conv, QDQ bridge, quantized Swish, and other
  quantized recovery orchestration;
• late and terminal recovery clusters for Slice/Concat, affine/Concat/Split,
  normalization/attention, channel shuffle, SPP, hard activations, and
  singleton reshape patterns.

The original call sites and their relative order remain visible in the
lowerer. Existing private names remain available through compatibility
wrappers, so downstream internal imports are not forced to change in this
checkpoint.

2. Transactional rejection and deterministic rewrites

Several rule families previously performed metadata or graph mutations before
all rejection conditions had been evaluated. The branch adds preflight
planning or snapshot-based rollback behavior before extracting those owners.

The corrected families include Conv input adapters, Conv/Pool passthrough,
mixed-layout Concat, stale binary adapters, QLinear output propagation,
QLinear SiLU prefix recovery, Mean/MaxPool/Concat recovery, Softmax transpose
canonicalization, Concat bridge families, Reshape/Transpose collapse, and
attention cleanup/rank-lift patterns.

The tests explicitly verify that rejected candidates leave operator lists,
tensor metadata, quantization records, LayoutState, graph outputs, and
diagnostics unchanged. Successful rewrites retain deterministic candidate
priority, fixed-point restart behavior, unique naming, pruning order, and the
historical statistics keys.

3. Indexed graph access and shared pass state

Extracted owners use ModelIRGraphIndex for producer/consumer lookup and
differential updates where the family contract permits it. Nested pass groups
share one pass-state scope so graph indexes can be reused across compatible
owners instead of being rebuilt for each helper.

Architecture and efficiency tests cover:

• one graph-index build across multi-owner orchestration fixtures;
• stable pass IDs and diagnostic ordering;
• deterministic owner selection for all policy combinations;
• no import cycle from pass modules back into the central lowerer;
• preservation of target-specific contexts and child-model boundaries.

4. Consolidated ModelIR pass contexts

Repeated frozen context dataclasses are consolidated around
ModelIRPassContext, owned by ConversionSession. Twenty-five historical
context names remain compatible internal aliases, while main-model consumers
reuse the Session-owned ModelIR, LayoutState, and diagnostics identities.

Callback-bearing recovery contexts now compose the shared context while
retaining their exact callback objects and argument contracts. Target-specific
and child-model conversions still receive fresh context where isolation is
required. No orchestration dataclass now duplicates the common core identity
fields.

5. Typed NodeView tensor references

NodeView.inputs and NodeView.outputs previously created a new dynamic class
for every tensor reference. Runtime attribute access worked, but static
analysis could not prove that .name, .onnx_name, .shape, and .dtype
existed. They now use lightweight typed instances with the same mutable
metadata and remapping behavior. This removes the Pylance attribute-access
error without suppressing diagnostics.

6. More reliable sequential corpus runner

The bulk runner now publishes its increasingly large state JSON through a
same-directory temporary file followed by os.replace(). Concurrent readers
therefore observe either the previous complete state or the new complete
state, never the truncated interval created by direct open(..., "w") output.

The change preserves filenames, JSON schema, indentation, file mode, resume
behavior, and classification. Serialization or replacement failure removes
the unpublished temporary file and leaves the previous state intact. No
per-update fsync was added, avoiding unnecessary I/O cost for state files
that reached approximately 17 MiB.

The pre-correction evidence was committed before changing the writer, so the
observed failure and the correction remain independently reviewable.

Compatibility guarantees

• Public CLI and Python APIs are unchanged.
• flatbuffer_direct remains the default backend.
• Artifact names, formats, return behavior, and report schemas are unchanged.
• Existing pass order, private compatibility entry points, statistics keys,
  and requested-artifact behavior are preserved.
• Direct TFLite conversion and -cotof do not import or execute TensorFlow.
• Optional TensorFlow artifacts remain behind their explicit optional-extra
  boundary.
• No package was added or removed.
• All validation commands used the repository uv environment.
• Model inference was strictly sequential: no ProcessPool and no concurrent
  model-inference workers were used.
• The 2,000 threshold is only the ONNX operation-count boundary between corpus
  tiers; this change does not impose a 2,000-line source-file requirement.

Tier 0-4 full regression validation

The final managed run covered every active Tier 0-4 model that was below the
ten-minute policy boundary, did not trigger SWAP, and had not been explicitly
excluded by prior validation policy.

The component observations were:

These are raw observations, not regression counts. The existing PyTorch
failures and missing reports remain visible and are not classified as
successful conversions.

Compared with the finalized fb-refactor5 baseline:

• the TFLite problem set is unchanged;
• all known baseline failures retain their prior classification/signature;
• yolox_nano.onnx and rtdetrv4_s.onnx remain improved passes;
• the user-approved DEIM near-tied TopK index cases remain accepted;
• no combined classification regressed;
• combined pass count improved from 136 to 137;
• PyTorch failure count improved from 32 to 31;
• no fb-refactor6-specific TFLite or native-PyTorch regression was confirmed.

The full model-level result, timing, normalized signatures, exclusions, and
baseline comparison are recorded in:

• docs/flatbuffer_direct_tier0_4_full_regression_2026-07-17.md;
• docs/baselines/flatbuffer_direct_tier0_4_tflite_native_pytorch_fb6_13fcf7ce.json.

Automated verification

The refactor was developed as many small characterization/correction/extraction
checkpoints. Focused owner/wrapper, architecture, efficiency, core lowering,
and TensorFlow-import-blocked gates were run at each boundary. The detailed
command history and checkpoint results are retained in
docs/fb_refactor6_pull_request_description.md and
docs/flatbuffer_direct_handoff_2026-07-14.md.

Latest relevant gates include:

• bulk runner, active corpus manifest, and architecture: 292 passed;
• typed NodeView plus architecture: 250 passed;
• representative core lowering, error reporting, QLinear, and rank inference:
  43 passed;
• atomic state writer focused suite: 42 passed;
• non-inference atomic writer stress test: 40 successive 8,000-entry states,
  1,742 successful reads, zero partial JSON reads, and no leaked temporary
  file;
• repeated TensorFlow-import-blocked direct-backend gate: 11 passed at the
  relevant extraction checkpoints;
• focused Ruff checks, Python compilation checks, uv lock --check, version
  consistency, and git diff --check: passed.

The authoritative 379-model conversion run is tied to converter checkpoint
13fcf7ce. The later atomic-writer and NodeView typing commits do not alter
lowering, export policy, scheduling, or model inference and were covered by
their focused gates rather than another multi-hour corpus run.

Known limitations and deliberate scope

• Tier 5 models (2,000 or more ONNX operations) are outside this final corpus
  gate.
• Models already classified as timeout, SWAP-risk, or explicitly excluded are
  outside the 379-model regression conclusion.
• Native PyTorch still has substantial inherited accuracy failures and missing
  reports. This PR preserves and records them; it does not claim to solve the
  PyTorch exporter backlog.
• Several inherited TFLite limitations remain, including the documented DPT
  Softmax/rank error, Conv-TasNet layout failures, MIRNet missing tensor data,
  and known numeric mismatches.
• The extraction intentionally preserves many legacy compatibility policies.
  Further semantic simplification should be proposed only after the new local
  owners have equivalent corpus evidence.
• The atomic state replacement guarantees complete visibility to concurrent
  readers, not durability across sudden power loss.

What's Changed

• Refactor flatbuffer-direct pass ownership and shared ModelIR contexts by @PINTO0309 in #953

Full Changelog: 2.6.4...2.6.5

2.6.4

Summary

This PR continues the flatbuffer_direct refactoring after the fb-refactor4 work merged in #949. It moves a large remaining set of layout, shape, quantization, fusion, and graph-repair rules out of the central lowerer and into focused, indexed ModelIR passes while preserving the existing CLI, Python API, default backend, artifact names, report contracts, and optional-exporter behavior.

The branch contains 168 commits affecting 230 files. Its largest structural change is the addition of 71 focused pass modules and 83 focused test modules. The large line count is primarily the explicit extraction of production rules and their characterization tests, rather than a new parallel conversion path.

Why this change is needed

The direct TFLite path had accumulated many order-sensitive graph rewrites in a very large lowering function. Those rewrites repeatedly scanned the whole ModelIR, inferred layout locally, and mutated shared state through implicit conventions. This made individual fixes expensive to reason about and increased the probability that a change for one model family would disturb another.

This PR makes rule ownership and state boundaries explicit. The intention is not to remove supported behavior: it is to retain that behavior behind smaller semantic passes, indexed graph access, typed request/artifact policy, and focused regression contracts.

Main improvements

1. A single normalized request and artifact boundary

• The direct conversion entry point now creates ConversionRequest once and stops propagating raw kwargs into lower layers.
• Requested outputs are represented by one immutable ArtifactPlan.
• Quantization, split, SavedModel, PyTorch, calibration, report, and progress options are resolved only when the corresponding artifact is requested.
• Unrequested exporters and their option parsing are not entered. For example, an unused PyTorch timeout cannot break a TFLite-only conversion.
• Direct TFLite evaluation selects files from the builder's returned artifact mapping instead of reconstructing paths from naming conventions.
• Direct-result completion validation and compatibility logging have one owner, and unreachable fallback routes through the legacy TensorFlow conversion path were removed.

These changes retain the legacy public keys, defaults, artifact order, filenames, failure messages, and return dictionary.

2. Shared indexed pass state

• Adjacent registered pass groups now share a lazy ModelIRPassStateScope when there is no intervening raw ModelIR mutation.
• ModelIRGraphIndex owns producer/consumer and operator-type lookups; LayoutState is kept synchronized through differential mutation helpers.
• Scope identity checks prevent graph or layout state from leaking across models or fallback ModelIR instances.
• Prepared pass data is session-local, consumed once, and cleared on rollback.
• Transactional guards, stable pass ordering, bounded iteration, and fingerprint-based convergence remain the safety boundary for risky rewrites.

This removes repeated index construction across many mean/attention, gate, concat, shuffle, boundary, slice, singleton-reshape, QDQ, SPP, and terminal-cleanup clusters without changing their historical execution order.

3. Focused semantic pass extraction

Seventy-one new production pass modules replace central-lowerer rule blocks. The extracted families include:

• Conv/Conv1D input, output, unary, BatchMatMul, InstanceNorm, affine, and deconvolution layout repair;
• concat/global-pool, concat/transpose-conv, split/concat, mixed-concat, input-adapter, and binary-bridge recovery;
• InstanceNormalization residual, dual-statistics, pre/post, bias, and affine-chain handling;
• quantized activation, HardSigmoid, Logistic, Softmax, MaxPool, PReLU, Swish, and TransposeConv cleanup;
• attention/QKV reshape, window partition/reverse, channel shuffle, singleton gate, passthrough, and terminal layout repair;
• dynamic Reshape resolution, static shape reconciliation, HARD_SWISH shape sanitization, and activation fusion;
• specialized historical SiNet chains decomposed into smaller pattern-owned passes with explicit guards.

The central dispatch remains responsible for orchestration, while match/guard/rewrite behavior and its invariants live beside focused tests. Existing rule order was preserved during extraction.

4. PyTorch exporter policy separation

Six fast-precanonicalization policies were moved out of the large exporter into pytorch_fast_precanonicalize_policy.py:

• downstream binary layout evidence;
• Resize/BatchNorm evidence;
• aligned BatchNorm constants;
• local-response-normalization layout propagation;
• rewritten static-shape recording;
• NHWC AveragePool bridge state.

The managed bulk runner also gained an opt-in native-only PyTorch mode. It emits -fdopt without implicitly enabling TorchScript, Dynamo ONNX, or ExportedProgram, while the runner's existing default behavior remains unchanged.

5. Sequential regression and process isolation hardening

• Tier 0-4 validation is fixed to one model at a time with a 600-second per-model ceiling.
• Every converter starts in a new POSIX session. Timeout, SWAP detection, and parent exit now terminate and reap the complete process group before the next model starts.
• Process-tree SWAP is monitored from /proc; any model that generates SWAP is excluded from future managed validation according to the documented policy.
• Resume metadata includes artifact mode and rejects incompatible resumes.
• The managed profile now records vit_b_encoder.onnx and superpoint_lightglue_end2end_fused_cpu.onnx as timeout_after_600s exclusions.
• All run state, normalized failure signatures, timing, accuracy, and exclusion rationale are documented in the checked-in baseline and regression reports. Large generated artifacts remain outside Git.

Regressions found and corrected

The full current-branch run initially exposed four genuine TFLite regressions. Each was recorded, reproduced against detached main, bisected to its first bad extraction commit, and corrected with a generic guard rather than a model-name exception:

1. Nighttime dehaze / InstanceNorm residual concat: accept only the exact valid case where declared TensorIR shape (1,) has a scalar, one-element NumPy backing, then normalize it to the declared shape.
2. RTMPose L and M / concat global-pool fan-out: permit an extra self-gating MUL only when the concat is one operand and the other operand is proven to depend on the matched global-pool Conv output. Arbitrary fan-out remains rejected.
3. GridSample / binary bridge recovery: validate consumer ordering against the binary operator that becomes the producer after rewrite, while retaining the stricter order guard for the preserved adapter.

Final sequential validation passed all four repaired models with zero SWAP and maximum absolute errors of 0.000294536, 4.67896e-06, 2.72691e-06, and 0.00802997, respectively.

Compatibility and dependency boundaries

• No new package dependency is introduced.
• All development and validation use the existing uv environment.
• Normal -tb flatbuffer_direct conversion and -cotof remain behind the TensorFlow import blocker.
• TensorFlow remains optional and is reachable only for explicitly requested SavedModel/H5/Keras/TFv1-style artifacts.
• Existing float32/float16 TFLite outputs, quantization variants, weights, split/crop, custom/pseudo op handling, reports, PyTorch-derived outputs, public defaults, return behavior, and filenames are preserved.
• The 2,000-node value is only the Tier 4/Tier 5 corpus boundary; it is not a source-file line limit.

Validation performed

Focused and architecture tests

• Changed focused suites: 253 passed.
• Architecture/contract suite: 218 passed.
• Indexed-pass regression suites after the four fixes: 154 passed.
• Bulk-runner suites, including timeout parent/grandchild cleanup and SWAP process-group termination: 40 passed.
• PyTorch extraction policy/emitter suites: 59 passed.
• Ruff checks passed for changed Python files.
• py_compile checks passed.
• git diff --check passed.
• Version consistency checks passed for onnx2tf.__version__, pyproject.toml, and README Docker examples at 2.6.4.

Tier 0-4 sequential corpus run

The authoritative sanitized-environment run processed all 381 then-active models sequentially with the exact command shape -tb flatbuffer_direct -cotof -fdopt:

• total model wall time: 9,347.523 seconds;
• median model time: 9.528 seconds;
• no completed model or descendant process used SWAP;
• TFLite observations before causal correction: 343 pass, 20 accuracy fail, 14 missing report, 2 conversion error, and 2 timeout;
• 341 of 355 managed baseline passes stayed passes;
• two known failures improved to passes (yolox_nano.onnx and rtdetrv4_s.onnx);
• the 14 baseline-pass observations were split into four branch regressions, six pre-existing/stale-baseline limitations, two user-approved DEIM results, and two timeout exclusions.

After correction, the four confirmed branch regressions passed exact sequential real-model revalidation. The next managed profile contains 379 active and 41 excluded Tier 0-4 records.

Native PyTorch assessment

The run generated and evaluated the native PyTorch package for every active model. Because no managed pre-fb-refactor5 native-PyTorch baseline exists, non-passes were not automatically labeled as regressions. Eight representative high-signal failures were rerun sequentially on detached main; all eight produced the same behavior on both implementations. No fb-refactor5-specific native PyTorch regression was confirmed by that comparison.

Known limitations and review notes

• The complete 379-model corpus was not rerun from zero after the four localized fixes; the affected real models and focused neighboring/sentinel suites were rerun instead.
• The monolithic legacy tests/test_pytorch_exporter.py run was intentionally stopped at 89% when one exported-program archive test spent several minutes recompiling. At interruption it had 942 passes and 81 failures; this is characterization, not an acceptance result. Representative fast failures reproduced identically on detached main.
• Native PyTorch still has pre-existing runtime/accuracy limitations recorded in the regression report; this PR does not claim to solve them.
• Tier 5 (2,000 or more ONNX nodes) was not part of this Tier 0-4 corpus run.
• Optional TensorFlow artifact combinations were not included in the authoritative direct-path corpus run.
• Remaining work includes further op-family lowering separation, broader quantization/split/artifact matrices, and warm-run time/RSS comparison. These are documented rather than hidden by the current checkpoint.

What's Changed

• Refactor flatbuffer_direct passes and harden Tier 0-4 regression safety by @PINTO0309 in #951

Full Changelog: 2.6.3...2.6.4

2.6.3

Summary

This pull request is the accumulated fb-refactor4 checkpoint for the flatbuffer_direct backend. It restructures the converter around explicit internal contracts, indexed graph state, ordered and validated passes, request-driven artifact generation, and a substantially decomposed PyTorch export stack.

The public CLI and Python API remain compatible. The default backend remains flatbuffer_direct, normal direct TFLite conversion and -cotof do not import TensorFlow or tf-keras, no new third-party dependency is introduced, and TensorFlow remains available only behind explicitly requested TensorFlow-family artifacts.

The package version is updated from 2.6.2 to 2.6.3 in the package metadata, lock file, and documented container examples.

Why this refactor was needed

The previous direct backend had accumulated a very large set of interacting graph rules and repeated whole-graph scans. This made small changes expensive to reason about, allowed ordering assumptions to remain implicit, duplicated producer/consumer and layout knowledge, and made optional artifact generation perform work that had not been requested.

The PyTorch exporter had the same problem at a different layer: native code generation, source rewriting, state-dict handling, runtime wrappers, fallback selection, TorchScript/Dynamo ONNX/ExportedProgram export, naming, shape policy, and layout policy were concentrated in a monolithic module with many implicit compatibility bindings.

This branch turns those implicit relationships into bounded internal interfaces while preserving the established conversion behavior.

Main architectural changes

1. Explicit conversion pipeline contracts

The direct path is organized around this stage order:

1. normalize public arguments into ConversionRequest and ArtifactPlan;
2. preprocess ONNX;
3. create a single ConversionSession, GraphIndex, and LayoutState;
4. execute registered passes in deterministic PassPhase order;
5. lower operators into ModelIR;
6. validate ModelIR invariants;
7. invoke only exporters selected by ArtifactPlan;
8. adapt ConversionResult back to the legacy return dictionary.

Raw option dictionaries no longer need to propagate through the new internals. Artifact controls, split thresholds, reporting options, and quantization calibration controls are resolved only when the associated artifact is requested.

2. Indexed and transactional graph mutation

GraphIndex and ModelIRGraphIndex provide shared producer, consumer, duplicate-producer, operator-position, and operator-type views. Rewriters update those indexes differentially instead of rebuilding maps or repeatedly scanning the entire graph.

Passes have stable IDs, phases, priorities, maximum iteration counts, explicit change results, and graph fingerprints for deterministic cycle termination. Risky rewrites can run transactionally and roll back when shape, dtype, layout, unresolved-tensor, duplicate-producer, or public-output invariants fail.

LayoutState is carried through the session and reconciled with graph changes so layout knowledge is no longer reconstructed independently by every rule.

3. Semantic pass extraction and indexed lowering support

Large rule families were moved out of the central lowerer into focused pass modules. This includes dynamic reshape, graph cleanup, high-rank binary and MatMul handling, rank-4/rank-5 Concat families, quantized layout families, Pad, split fallback, channel shuffle, multi-branch gates, NDHWC gates, cost-volume/ScatterND patterns, PyTorch compatibility, recurrent/control-flow preparation, and layout validation.

The extracted passes retain compatibility wrappers where existing imports require them. Production call sites use registered runners, indexed root enumeration, explicit guards, shared pruning utilities, and transactional layout reconciliation. Model-name checks are not introduced; repairs are expressed as semantic graph patterns.

4. Request-driven artifact and memory behavior

The artifact pipeline now avoids entering unrequested exporters, quantization paths, split planning, and report writers. Constant buffers and graph indexes are shared where possible, precision clones have bounded lifetimes, and redundant ModelIR copying and repeated validation-index construction are reduced.

The direct backend releases legacy graph objects before ModelIR lowering when they are no longer needed. Sequential ONNX/TFLite accuracy checking materializes large prepared ONNX graphs once in a managed temporary file rather than transferring serialized model bytes through a multiprocessing pipe.

5. PyTorch exporter decomposition

The PyTorch stack is split into dedicated, mostly Torch-free policy and implementation owners, including:

• artifact exporters and example-input/export metadata support;
• native codegen context, stages, emitters, indexing, values, and naming;
• capability and fallback-package selection;
• binary, concat, reshape, reduction, constant, expression, layout-bridge, fusion, NMS, recurrent, and shape policies;
• generated source parsing and graph/source rewrites;
• state-dict and package-source support;
• runtime-wrapper and StringNormalizer exporters;
• ONNX artifact metadata/layout handling;
• ExportedProgram child execution and archive cleanup.

Native codegen reuses ModelIRGraphIndex, and compatibility bindings required by the dynamically compiled legacy body are explicit and covered by architecture tests. Torch is imported only after a PyTorch-family artifact has actually been requested.

TorchScript, Dynamo ONNX, and ExportedProgram generation share the generated native package and common artifact support. TFLite-backed and SavedModel-backed fallbacks reuse the same package scaffolding without pulling TensorFlow into ordinary direct conversion.

6. Source canonicalization hardening

The latest checkpoints repair several inherited PyTorch exporter failures without broad layout-policy changes:

• structured subtraction parsing and rank-3 softmax-mask handling;
• lazy generated-model class resolution;
• restored native-codegen helper bindings after module extraction;
• TensorFlow-free TFLite-input PyTorch artifact paths;
• scalar materialization for CONCAT;
• signed and dynamic CONCAT target-shape parsing;
• rank-4 evidence requirements for NCHW CONCAT rewriting;
• preservation of scalar tensors when they are the direct input to torch.reshape.

The last group removes all five inherited NoneType.group() CONCAT canonicalization crashes. Three of those tests now pass completely; the two remaining Yolov7 cases reach older structural assertions rather than crashing.

7. Managed sequential regression workflow

The repository now contains a managed Tier 0-4 corpus workflow, immutable quick-run manifests/results, normalized failure classifications, timeout exclusions, and SWAP detection for the active converter process tree.

Validation is always sequential. The runner does not use a process pool or parallel inference workers. A model that causes SWAP is classified and added to the managed exclusion policy before a later run.

The 2,000 threshold refers to ONNX graph node count for Tier classification. It is not a source-file line limit.

Compatibility and dependency boundaries

• Existing CLI and Python entry points are retained.
• The default backend and artifact naming remain unchanged.
• Direct TFLite conversion and -cotof do not import TensorFlow or tf-keras.
• SavedModel, H5, Keras, and TFv1 remain optional TensorFlow-family outputs.
• PyTorch remains optional and is imported only for requested PyTorch-family artifacts.
• No dependency was added; all development and validation use uv.
• Generated FlatBuffer schema code is unchanged by the architectural policy.
• Inference validation is sequential, including isolated subprocess execution.

Validation performed

Tier 0-4 quick corpus

A fixed short-runtime selection of 54 Tier 0-4 models was run sequentially:

No regression specific to fb-refactor4 was confirmed. Focused comparison showed that silero_vad.onnx fails identically on fb-refactor3, while d3net_dnn_double_44.onnx reaches the same quick ceiling on both branches. nchw.onnx passes direct execution on both branches with byte-identical artifacts and metrics; only the instrumented bulk run consumes the timeout headroom.

Apart from the explicitly accepted DEIM result, the largest maximum absolute error among passing models was 0.0443115234375, below the required 1e-1 ceiling. DEIM remains accepted according to its recorded near-tied TopK policy.

PyTorch exporter regression investigation

The complete central PyTorch exporter suite contains 1,120 tests. At checkpoint 3127e39d, with seven previously classified long-running cases deselected, the exact result was:

All 94 non-timeout failures were reproduced against fb-refactor3 with the same exception type and normalized first message. Six belong to the explicitly excluded bread model family. Therefore, no non-timeout regression specific to fb-refactor4 was confirmed in that comparison.

The latest focused CONCAT/NMS work additionally passed 18 relevant tests, and the architecture suite passed 97 tests. The two remaining Yolov7 assertions are documented rather than hidden: one is a local cv64_in versus cv64_in_cf spelling expectation for the same channel-first tensor, and the other compares an unsanitized direct Dynamo ONNX graph with the helper path that intentionally applies the common ONNX sanitizer/optimizer.

PyTorch artifact smoke

A sequential rfdn_64x64.onnx integration gate generated and validated float32/float16 TFLite, a native PyTorch package and state dict, TorchScript, Dynamo ONNX, and ExportedProgram artifacts.

• TFLite maximum absolute error: 0.0000371933
• PyTorch maximum absolute error: 0.0000433922
• SWAP detected: no

Focused and architecture tests

The branch has also run the modular PyTorch policy/exporter tests, architecture/ownership checks, bulk-runner tests, TensorFlow import blockers, public-contract characterization, pass-level fixtures, quantization/split/report tests, and targeted compatibility-binding checks. The exact commands and checkpoint-specific results are retained in:

• docs/flatbuffer_direct_architecture.md
• docs/flatbuffer_direct_quick_regression_2026-07-14.md
• docs/flatbuffer_direct_pytorch_regression_2026-07-14.md
• docs/flatbuffer_direct_handoff_2026-07-14.md

The final version update was checked with uv lock --check and an import assertion confirming onnx2tf.__version__ == "2.6.3".

Known limitations and remaining work

This PR is a substantial checkpoint, not the end of the complete refactor plan.

• Tier 5 models (2,000 or more ONNX nodes) were intentionally not used as an early development gate.
• The full artifact matrix and complete Tier 0-4 corpus still require final end-to-end reruns after later Goal phases.
• Seven inherited PyTorch exporter tests remain classified as long-running.
• The remaining inherited source/layout assertions require semantic, model-independent fixes and should not be addressed with broad source-text rewrites.
• d3net_dnn_double_44.onnx and instrumentation-sensitive nchw.onnx are unsuitable for the 60-second quick profile.
• Optional TensorFlow-family output compatibility still needs its final dedicated matrix run.
• Warm-run conversion-time and peak-RSS measurements across the active tiers remain future work.

What's Changed

• Refactor the flatbuffer_direct pipeline and PyTorch export stack by @PINTO0309 in #949

Full Changelog: 2.6.2...2.6.3

2.6.2

Summary

This PR substantially refactors the TensorFlow-free flatbuffer_direct conversion path around deterministic, indexed, transactional ModelIR passes while preserving the existing CLI/Python API and compatibility entry points.

The main goal is to make converter behavior easier to reason about and safer to extend. Previously, many layout and cleanup rules lived in the central lowerer, repeatedly scanned the complete graph, and depended on implicit call order. This branch gives those rules explicit ownership, stable ordering, shared graph/layout state, transactional validation, focused characterization tests, and observable pass diagnostics.

The branch contains 202 commits and changes 104 files. Much of the apparent diff size comes from mechanically extracting existing rule implementations and their large inline tests from the central lowerer into dedicated pass and test modules.

What changed

Deterministic pass infrastructure

• Introduced and expanded the ordered pass manager with stable pass IDs, phases, priorities, preconditions, explicit change results, maximum iteration counts, and graph fingerprints.
• Added shared ModelIRPassState so repeated pass groups reuse one ModelIRGraphIndex and one LayoutState instead of rebuilding graph maps for every rule.
• Added differential ModelIR mutation APIs for input/output replacement and operator insertion/removal. Producer, consumer, duplicate-producer, and operator-position indices stay synchronized as rewrites are applied.
• Added cheap graph preflight predicates so irrelevant pass groups return before creating snapshots, fingerprints, or full state.
• Added deterministic pass diagnostics and bulk-runner metrics for invocation count, candidate visits, snapshots, fingerprints, changes, rollback, and timing.
• Transactional validation now compares invariant problems before and after a rewrite. A pass is rolled back only when it introduces a new problem; inherited temporary issues are not incorrectly attributed to a later pass. Final ModelIR validation remains strict.

Layout state and rewrite safety

• Centralized logical/physical layout tracking in LayoutState and synchronized it through indexed graph mutations.
• Protected graph inputs/outputs, public intermediate tensors, fan-out boundaries, shared constants, quantization metadata, and tensor lineage during rewrites.
• Added copy-on-write handling where layout conversion would otherwise mutate a constant used by an unrelated branch.
• Replaced model-name-oriented coverage with semantic graph-pattern guards. Production passes do not add model-name exceptions.
• Preserved compatibility wrappers in lower_from_onnx2tf.py while moving implementation ownership to focused pass-family modules.

Pass-family extraction and migration

The branch extracts and/or migrates the following families from the central lowerer into dedicated modules under onnx2tf/tflite_builder/passes/:

• boundary input, passthrough, Pad, high-rank MatMul, constant fold, Cast, and graph cleanup;
• quantized PReLU, quantized Reshape, Q/DQ and quantization cleanup;
• singleton Reshape/MaxPool/channel/spatial cleanup and consecutive/flatten-Concat-Reshape cleanup;
• Transpose cleanup, unary/fan-out bridges, Gather-axis/channel fan-out, and trailing-output adapters;
• NCHW/NHWC channel shuffle, Mean, LayerNorm statistics, and terminal Mean layout;
• attention/QKV, SE, elementwise gate, multi-branch gate, complementary gate, NDHWC/Conv3D gate, and post-add/post-convolution gate families;
• cost-volume/ScatterND, dual-Mul/Concat, Add/Concat constant suffix, axis-3 constant Concat bridge, Dequantize/Concat/Quantize, Concat/unary/Conv, generic SPP, and NDHWC pre-Concat families.

Each migrated family follows a staged process:

1. characterize current success and rejection boundaries with compact generated ModelIR/ONNX fixtures;
2. mechanically extract the existing implementation without changing behavior;
3. migrate traversal and mutation to the shared index/layout state;
4. register a stable transactional pass and replace raw production calls;
5. run focused, architecture, efficiency, and representative model regression tests.

Reporting and numeric evaluation

• Extracted coverage and tensor-correspondence reporting into tflite_builder/reporting.py, retaining thin compatibility wrappers in the lowerer.
• Kept reporting within the TensorFlow-free dependency boundary.
• Added sequential accuracy-evaluation fallback for fully static models:
  • evaluate with LiteRT default delegates first;
  • retry without default delegates only if the primary result fails either the existing evaluation_pass rule or the hard max_abs_error <= 1e-1 contract;
  • never overlap inference processes;
  • deterministically select the better completed report;
  • retain the primary report if fallback evaluation fails.
• The public accuracy JSON schema and command behavior are unchanged.

Compatibility and packaging

• Existing CLI/Python API entry points and legacy lowerer helper signatures remain available through adapters/wrappers.
• No new dependency package was introduced.
• Normal -tb flatbuffer_direct conversion and -cotof remain TensorFlow-free. Optional TensorFlow exporters remain behind their existing optional boundary.
• Updated the package and documented container version from 2.6.1 to 2.6.2; uv.lock remains consistent.

Validation

All local commands were run in the uv environment. Model conversion and inference were executed sequentially with at most one inference subprocess at a time.

Automated tests

• Latest complete sequential direct regression selection before the final recovery fixes:
  • 1170 passed, 5 deselected, 2 warnings in 162.39s
• Focused and related suites after the final pass-validation and evaluator fixes:
  • 92 tests passed
  • 65 tests passed
  • 157 distinct tests passed in total
• These selections cover transaction rollback, graph/index consistency, layout state, graph cleanup, pass ordering and efficiency, evaluator input layout/name mapping/seed behavior, subprocess isolation, architecture boundaries, and optional-TensorFlow import blocking.
• Python bytecode compilation of the final changed Python files passed.
• uv lock --check passed.
• Importing the package reports onnx2tf.__version__ == "2.6.2".
• git diff --check passed.

Model regression coverage

• A representative eight-model sentinel set passed.
• A 74-model blast-radius cohort covering the affected duplicate-Transpose, Reshape, and terminal-Mean passes passed conversion and the strict numeric contract.
• The four regressions attributed to the refactored pass-manager behavior were recovered:
  • birdnet.onnx: max_abs_error=6.4849853515625e-05
  • imageclassifier.onnx: 6.67572021484375e-06
  • model_resnet15x224_swish-072.onnx: 7.43865966796875e-05
  • resnet18-v1-7.onnx: 7.152557373046875e-06
• The three delegate-evaluation regressions were recovered:
  • keras_rnn.onnx: 6.258487701416016e-07
  • pose_estimation_mediapipe_2023mar.onnx: 0.05517578125
  • pose_estimation_mediapipe_2023mar_int8bq.onnx: 0.0443115234375
• All seven recovered models returned exit status zero, reported evaluation_pass=true, and satisfied max_abs_error <= 1e-1.

Performance-oriented changes

• Reuse graph indices and constant buffers instead of rebuilding equivalent maps for each pass.
• Skip irrelevant pass groups before allocating snapshots or computing fingerprints.
• Use differential updates instead of repeated whole-ModelIR scans after each mutation.
• Collect deterministic per-pass metrics so future changes can identify unexpected repeated scans, snapshots, or cycles.
• Execute only requested evaluation/export work; evaluator fallback is not run when the primary result already passes the strict contract.

Formal tier-wide warm-up/three-run median timing and peak-RSS gates are not claimed as complete in this PR.

Known remaining work

• The full fixed-order Tier 0 through Tier 4 corpus has not been rerun after the final two recovery commits. The final evidence currently consists of the seven recovered models, eight sentinels, and the 74-model blast-radius cohort.
• Expected-timeout models remain excluded from the active regression corpus, and DEIM is intentionally classified as successful.
• Fifteen previously investigated non-timeout candidates remain unresolved:
  • two DPT producer-rank cases (known first-bad commit 881b714);
  • double_gru.onnx (known first-bad commit 94af81b);
  • numeric mismatches in model1.onnx and fast_acvnet_generalization_opset16_192x320.onnx;
  • ten inherited runtime/report-generation failures documented in the handoff.
• Tier 5 models (2,000 or more ONNX operations) remain intentionally deferred until the Tier 0 through Tier 4 contract is fully revalidated.
• GitHub Actions had no run records for this branch at the final local checkpoint, so this PR does not claim external CI completion.

Detailed implementation and validation history is recorded in:

• docs/flatbuffer_direct_architecture.md
• docs/flatbuffer_direct_handoff_2026-07-12.md
• docs/flatbuffer_direct_handoff_2026-07-13.md

What's Changed

• Refactor flatbuffer_direct around indexed transactional passes by @PINTO0309 in #948

Full Changelog: 2.6.1...2.6.2

2.6.1

Summary

This pull request continues the flatbuffer_direct refactoring after #945. It focuses on making the TensorFlow-free ONNX-to-TFLite path more reliable, easier to reason about, and less expensive to maintain while preserving the existing CLI, Python API, artifact naming, report schemas, and conversion behavior.

The branch contains 51 commits and changes 76 files. The work is intentionally incremental: it improves the current conversion pipeline and records the remaining architectural work, but it does not claim that the full long-term refactoring plan is complete.

Main improvements

1. Quantized lowering is split into focused operation families

The former combined op_builders/quantized.py implementation has been removed. Quantized lowering is now organized into dedicated modules:

• dynamic_quantize.py
• quantize_linear.py
• qlinear_binary.py
• qlinear_activation.py
• qlinear_concat.py
• qlinear_conv.py
• qlinear_fc.py
• qlinear_pool.py
• conv_integer.py
• quantized_common.py for shared quantization, shape/signature, padding, and requantization primitives

quantized_common.py contains no build_* entry point. Public builder imports and registry dispatch remain compatible.

Each mechanical extraction is guarded by a normalized ModelIR fingerprint covering operators, tensors, constant buffers, options, versions, shapes, shape signatures, dtypes, and quantization metadata. The fingerprint serializer is shared across the family tests instead of being duplicated in each test module.

2. Correct ONNX quantization semantics

DynamicQuantizeLinear now uses nearest-even rounding and performs ROUND(x / scale) before adding the integer zero point. This avoids both the previous half-up behavior and FLOAT32 precision loss when a large zero point was added before rounding.

QLinearAdd, QLinearSoftmax, QLinearConv, and related Q/DQ paths retain explicit ONNX-compatible requantization where LiteRT fixed-point tie handling could otherwise differ by one quantum and become amplified downstream.

The direct MatMulInteger/QLinearMatMul implementation continues to follow portable INT32 ONNX semantics. Host-specific ONNX Runtime U8S8 saturation behavior is documented rather than emulated in the converter.

3. Layout, shape, and high-rank robustness

This branch adds or strengthens generic handling for:

• quantized pooling output-shape planning;
• two-input mixed-layout concatenation;
• channel-last ConvInteger input repair;
• dynamic Unsqueeze shape preservation;
• stale rank/shape hints without overriding valid producer semantics;
• static high-rank binary and BatchMatMul compression;
• high-rank ScatterND normalization;
• zero-safe dynamic ScatterElements;
• sequence LSTM reshape reconciliation;
• dynamic arange Loop scan lowering;
• fixed-shape detector input contracts;
• zero-batch-safe paths used by Mask R-CNN.

The changes use semantic graph properties rather than model-name-specific rules.

4. Dynamic GridSample without TensorFlow or custom operators

Dynamic rank-4 GridSample is lowered with builtin LiteRT operators. The implementation supports bilinear/linear and nearest interpolation, zeros and border padding, both align-corners modes, runtime spatial dimensions, and ONNX Runtime-compatible NaN coordinate handling.

This recovered the large encoder.onnx path while keeping all 24 GridSample nodes builtin and avoiding a TensorFlow dependency or unresolved custom op.

5. Large-model memory and evaluation improvements

The direct branch now releases the unused legacy GraphSurgeon graph before ModelIR lowering, avoids unnecessary lifetime extension of large serialization objects, and passes a managed ONNX model path to the isolated evaluator instead of pickling a complete protobuf through a multiprocessing pipe.

Evaluation remains sequential. At most one isolated inference subprocess is active at a time; no ProcessPool or parallel model worker was introduced.

Static-input evaluation may use the standard LiteRT delegate. If delegate preparation fails, the evaluator retries once with the builtin interpreter, still sequentially. Dynamic-input crash isolation remains delegate-free.

6. Model recoveries and explicit failure classification

The changes recover or promote multiple Tier 0-4 models, including:

• Mask R-CNN;
• dynamic GridSample encoder models;
• onnx_dense_optimized.onnx and its byte-identical counterpart;
• modnet_old.onnx;
• rf-detr-nano.onnx;
• LibreRFDETRn.onnx;
• fixed-shape YOLOv9 variants;
• a fixed-batch YOLO11 OBB model;
• additional quantized pooling, sequence, and detector paths.

Remaining active non-passes have normalized causes instead of unexplained skips. These include invalid source geometry/ranks, documented ONNX Runtime U8S8 kernel divergence, quantization-boundary amplification, and discontinuous TopK/GridSample sensitivity. The two DEIM variants are recorded as accepted successes per explicit project direction.

No global tolerance relaxation, model-name lowering rule, or forced index ordering was added.

7. TensorFlow-free direct boundary

The default flatbuffer_direct conversion path and direct accuracy evaluation remain TensorFlow-free. Architecture tests cover the new operation-family and shared quantization modules. TensorFlow remains optional and is required only for explicitly requested TensorFlow/SavedModel/Keras-style outputs or the tf_converter backend.

No new dependency was added. All implementation and verification work uses the existing uv environment.

8. Tier policy clarification

The number 2,000 is an ONNX graph node/operation threshold, not a source-file line limit:

• Tier 4 ends at 1,999 ONNX nodes.
• Tier 5 starts at 2,000 ONNX nodes.

There is no 2,000-line acceptance gate for production or test modules. Modules are split according to ownership, coupling, and reviewability.

Compatibility

• Existing CLI and Python entry points are retained.
• Existing builder imports and registry dispatch are retained.
• Artifact names and report formats are retained.
• The default backend remains unchanged.
• No new package dependency is introduced.
• Direct TFLite conversion does not import TensorFlow.
• Inference validation remains single-process and sequential.

Managed corpus state

The managed Tier 0-4 profile currently records:

• 368 passes;
• 6 missing_tflite_report entries;
• 20 tflite_fail entries;
• 26 excluded historical timeouts.

All 26 active non-passes have an explicit normalized cause or expected invalid/custom-runtime classification.

Tier 5 is retained as historical/final-phase coverage and is not used as an early refactoring gate.

Verification

The latest full core-environment regression was executed with one pytest process and no parallel workers:

985 passed, 5 deselected, 2 warnings in 121.63s

Additional focused verification includes:

• quantized family fingerprint and architecture suite: 26 passed;
• focused QLinearConv/ConvInteger suite: 12 passed, 760 deselected;
• QLinearConcat and QuantizeLinear/DequantizeLinear focused suite: 12 passed, 761 deselected;
• QLinear activation family suite and runtime checks;
• QLinear binary rounding and chain runtime checks;
• sequential corpus -cotof runs for recovered and diagnosed models;
• uv lock --check and an import-time version assertion for the 2.6.1 version update.

The five deselected tests require optional environments unavailable in the core uv setup:

• four TensorFlow converter tests requiring the TensorFlow extra;
• one Torch GroupNorm integration test affected by an incompatible system Python 3.10 Torch binary under Python 3.12.

The two warnings are expected FLOAT16 overflow warnings in existing edge-case tests.

Documentation and handoff

The architecture and handoff documents record:

• completed work;
• remaining work;
• managed corpus state;
• exact test commands/results;
• known failures and normalized reasons;
• the next safe refactoring boundary.

The next planned implementation step is to extract coverage/correspondence reporting from lower_from_onnx2tf.py, followed by staged ownership of its large layout-pass collection. That work is deliberately not partially included in this PR.

Version

The package, lockfile, and Docker examples are updated consistently from 2.6.0 to 2.6.1.

What's Changed

• Continue flatbuffer_direct reliability and quantized-family refactoring by @PINTO0309 in #947

Full Changelog: 2.6.0...2.6.1

2.6.0

Summary

This PR substantially refactors and hardens the flatbuffer_direct backend while preserving the existing CLI/Python API and artifact conventions. It turns the direct converter into a more explicit, testable pipeline, improves shape/layout/control-flow handling across a broad ONNX corpus, expands TensorFlow-free validation, and modularizes the PyTorch export path.

The branch contains 126 focused commits and changes 120 files. The implementation intentionally stays within the existing dependency set and keeps normal flatbuffer_direct conversion and -cotof evaluation independent of TensorFlow.

Motivation

The direct backend had accumulated a large number of interacting, order-sensitive rules. A local fix could easily invalidate unrelated models, producer/consumer relationships were repeatedly reconstructed, layout assumptions were spread across individual operator lowerers, and large characterization tests consumed significant review and maintenance effort.

This work establishes clearer internal contracts and adds regression evidence before and alongside behavioral changes. The goal is not merely to make individual sample models pass, but to make future changes easier to reason about and safer to validate.

Architecture and pipeline changes

Explicit conversion core

• Added conversion contracts and session-scoped state under onnx2tf/tflite_builder/core/.
• Added reusable graph indexing for producers, consumers, constants, and tensor usage.
• Added centralized layout state instead of relying exclusively on local NCHW/NHWC guesses.
• Added pass specifications/results, deterministic phase orchestration, diagnostics, and validation helpers.
• Added ModelIR utilities and invariant checks for tensor availability, shapes, dtypes, graph outputs, and recurrent state handling.
• Added structured lowering registry support and smaller operator-family modules for normalization, quantization, recurrent, reduction, selection, and shape operations.
• Added focused pass modules for constant folding, padding/layout behavior, precision handling, and PyTorch compatibility.

The resulting flow is substantially closer to:

request -> ONNX analysis/normalization -> conversion session and graph index -> ordered passes -> operator lowering -> ModelIR validation/cleanup -> requested exporters

Mechanical decomposition

• Split reusable PyTorch layout, ONNX bridge, optimizer, code-generation, and export-support logic into dedicated modules.
• Reduced central orchestration responsibilities and made unit-level testing possible without running complete conversions.
• Added architecture tests that protect dependency direction and the intended internal boundaries.

flatbuffer_direct correctness improvements

Shape, rank, and layout recovery

• Added demand-driven static shape reconciliation around attention, gather, layer normalization, matmul, and related dynamic tails.
• Preserved semantic reshape/flatten contracts instead of committing placeholder dimensions too early.
• Improved unknown-rank and high-rank handling for GatherElements, BatchMatMul, padding, slicing, grouped convolution, Conv3D, Resize, DepthToSpace, and transpose convolution paths.
• Added idempotent layout-recovery passes and guards against stale or duplicated boundary transposes.
• Repaired channel-axis handling for concat/split/conv pipelines, channel shuffle, residual paths, attention blocks, and rank-3 broadcasts.
• Added explicit handling for non-image recurrent inputs and regression profile support for -kat axis preservation.

Control flow

• Added deterministic branch-local tensor namespaces to prevent collisions between If branches.
• Recursively remapped lexical captures for nested control-flow graphs.
• Improved branch dtype and shape reconciliation, mixed numeric operands, scalar handling, and graph-output preservation.
• Made speculative/inactive flattened branches executable when an otherwise valid Squeeze sees a non-singleton extent from the inactive path.
• Corrected Where lowering so a rank-1 singleton condition uses SELECT_V2 when prefix-style SELECT broadcasting is invalid.
• Added ONNX graph repairs for torchvision NMS guard captures and several legacy control-flow/reference-evaluation patterns.

Recurrent operators

• Preserved recurrent input axes for GRU/RNN/LSTM-style non-image tensors.
• Safely initialized constant zero LSTM states.
• Added primitive forward-LSTM lowering for explicit runtime states where TFLite's mutable sequence-LSTM state contract cannot accept an ordinary runtime producer.
• Preserved recurrent states through reshape cleanup and validated ONNX/TFLite parity for branch-local and direct state I/O.

Quantized operators

• Improved QLinearConv, QLinearMatMul, pooling, concat, residual, softmax, split, and Q/DQ handling.
• Matched ONNX rounding, truncation, zero-point, padding, and requantization semantics more closely.
• Preserved quantization boundaries and constant bias contracts through layout cleanup.
• Added tests for uint8 and mixed-encoding pipelines, integer TopK, integer resize, and quantized layout recovery.

Additional builtin coverage and compatibility

• Added or improved builtin paths for grouped Conv3D, rank-5 Resize, integer TopK, nearest GridSample, standard linear GridSample mode, edge padding, dynamic resize scales, and half-pixel round/ceil resize behavior.
• Added guarded custom fallback for genuinely unresolved shapes or unsupported capability instead of emitting structurally invalid builtin graphs.
• Improved Microsoft-domain/operator compatibility and legacy opset preparation for ONNX Runtime reference evaluation.

Accuracy evaluation without TensorFlow

• Kept default direct TFLite conversion and -cotof evaluation TensorFlow-free.
• Added ONNX Runtime compatibility preparation for legacy opsets, GroupNorm, sequence If, integer MatMul, optional tensors, unknown-rank Conv, Microsoft contrib operators, and graph ordering.
• Added deterministic seeded control inputs, including static TopK values and audio sample-rate inputs.
• Added narrowly scoped ONNX ReferenceEvaluator compatibility for malformed nested-LSTM shape inference, including complete Y, Y_h, and Y_c outputs.
• Preserved sequential evaluation: inference validation uses one active process at a time.

Representative recovered regressions include:

• superpoint_lightglue_end2end_fused_cpu.onnx: maximum absolute error 1.946091651916504e-05.
• silero_vad.onnx with -kat input state sr: evaluation_pass=true, maximum absolute error 1.375097781419754e-06, RMSE 1.222495367934982e-07.
• Multiple recurrent OCR, RF-DETR, quantized pooling, MobileFaceNet, ResNeXt/FPN, MIRNet, GridSample/attention, D3Net, and other Tier 0-4 regressions promoted after focused validation.

Regression corpus and execution policy

• Added recorded root-model baselines for Tiers 0 through 5.
• Added an active managed profile for every root model with 1-1,999 ONNX nodes (Tiers 0-4).
• Tier 5 is retained only as historical baseline data and is excluded from the active profile.
• Models previously recorded as timeouts are retained for provenance but excluded from subsequent validation.
• The active profile currently contains 394 models: 343 expected passes and 51 known non-passes, with 26 recorded timeouts excluded.
• Added corpus manifest and bulk-runner validation for fixed model ordering, root-only discovery, tier limits, per-model shape hints, -ois, and -kat options.
• All corpus inference is designed to run sequentially with concurrency fixed to one.

PyTorch-family export refactoring

• Extracted shared layout utilities, ONNX utilities, bridge passes, layout passes, optimizer orchestration, code-generation helpers, and export error handling.
• Kept common canonicalization reusable across native PyTorch, TorchScript, Dynamo ONNX, and ExportedProgram paths.
• Added focused tests for each extracted layer to reduce reliance on a single monolithic exporter test surface.

Public compatibility and packaging

• Preserved the existing public CLI/Python entry points and the flatbuffer_direct backend selection behavior.
• Preserved optional TensorFlow boundaries for explicitly requested TensorFlow artifacts while avoiding TensorFlow imports in direct TFLite paths.
• Added no new package dependency.
• Updated the project, lockfile, Python package, and documented container image version to 2.6.0.

Tests

The final relevant local suite was executed with uv:

uv run pytest -q \
  tests/test_tflite_builder_direct.py \
  tests/test_accuracy_evaluator_seeded_input.py \
  tests/test_onnx_reference_compat.py \
  tests/test_flatbuffer_direct_bulk_runner.py \
  -k 'not test_tflite_backend_matrix_add and not test_tflite_backend_matrix_hardswish_rewrite_on_off and not test_tf_converter_resize_cubic_avoids_flex_resize_bicubic and not test_tf_converter_resize_cubic_honors_cubic_coeff_a and not test_flatbuffer_direct_group_norm_alias_builtin_conversion'

796 passed, 5 deselected, 2 warnings

Additional focused validation included:

• 61 control-flow/LSTM/Squeeze/Where tests.
• 30 ONNX reference-compatibility and deterministic-input evaluator tests.
• Direct isolated -cotof verification of the recovered Silero model.
• A package/version consistency check for 2.6.0.

The five deselected tests are environment-specific rather than code regressions:

• Four tf_converter tests require the optional TensorFlow/tf-keras environment, which is intentionally absent from the core environment.
• One GroupNorm alias test resolves an external Python 3.10 Torch binary under Python 3.12 and fails with an ABI symbol error (_PyCode_GetExtra).

The two warnings are existing NumPy float16 overflow warnings from tests that intentionally cast extreme values.

Known limitations and intentionally retained non-passes

• silero_vad (1).onnx references 14 lexical captures that do not exist in the serialized model, including all four LSTM weight/bias captures. It cannot produce a trustworthy ONNX reference result and remains documented as an input-model defect.
• inverse_11.onnx contains a 224x224 matrix inverse. The current exact direct lowering is intentionally limited to matrices up to 16x16; larger matrices fall back to ONNX_INVERSE, which stock LiteRT cannot execute. The observed reference matrices are nearly singular, so a low-order approximation would not satisfy the required absolute-error bound and has not been introduced.
• The active profile still records 51 known non-passes. This PR improves and characterizes the backend but does not claim that every Tier 0-4 model is fully recovered.
• Final whole-corpus performance/RSS measurements and the remaining non-pass improvements are follow-up work, not hidden in this PR.

Suggested review order

1. docs/flatbuffer_direct_architecture.md for scope and design.
2. onnx2tf/tflite_builder/core/ for contracts, session state, graph/layout indexing, pass management, and validation.
3. onnx2tf/tflite_builder/op_families/, op_registry.py, and representative op_builders/ changes.
4. onnx2tf/utils/onnxruntime_compat.py, onnx_graph_repair.py, and accuracy_evaluator.py for TensorFlow-free validation.
5. PyTorch utility/pass extraction modules.
6. Managed baseline/profile files and the focused test modules.

Change size

126 commits
120 files changed
34,855 insertions
11,788 deletions

What's Changed

• Refactor and harden the flatbuffer_direct conversion pipeline by @PINTO0309 in #945

Full Changelog: 2.5.2...2.6.0

2.5.2

1. Content and background

This PR improves flatbuffer_direct strict integer quantization so more mixed-dtype and detection-postprocess-adjacent graphs can be converted without producing invalid full-integer artifacts. It also keeps the int8 and full-int8 outputs available when optional int16-activation variants are not supported by LiteRT kernels.

2. Summary of corrections

• Added strict full-integer quantization support for additional ops including GATHER_ND, SCATTER_ND, TOPK_V2, ARG_MAX, EXP, CAST, LESS, LOGICAL_NOT, WHERE, and SHAPE.
• Extended same-qparams policy handling so mixed input/output ops only quantize activation/value tensors and leave index, axis, shape, condition, and integer outputs unquantized.
• Added fallback qparam derivation from existing qparams, calibration ranges, or float constants when calibration cannot observe an output tensor.
• Added explicit skip reporting for int16-activation variants when a model contains SCATTER_ND, because LiteRT does not support INT16 updates for that kernel. The int8 and full-int8 variants still validate and are emitted.
• Added unit/integration coverage for the newly supported mixed-dtype ops and int16-activation skip reporting.
• Bumped package/docker references from 2.5.1 to 2.5.2.

3. Before/After (If there is an operating log that can be used as a reference)

Before:

• Pre-NMS YOLOX conversion could write int8 artifacts, then fail the overall -oiqt command when *_integer_quant_with_int16_act.tflite validation reached SCATTER_ND: Updates of type 'INT16' are not supported by scatter_nd.

After:

• python -m onnx2tf -i /tmp/onnx2tf_yolox_quant/yolox_nano_ti_lite_26p1_41p8_pre_nms.onnx -o /tmp/onnx2tf_yolox_quant/pre_nms_quant_out_after_skip -oiqt -cind images calib_data_416x416_n200.npy "[[[[0.0,0.0,0.0]]]]" "[[[[1.0,1.0,1.0]]]]" completed successfully.
• Generated and validated:
  • yolox_nano_ti_lite_26p1_41p8_pre_nms_integer_quant.tflite
  • yolox_nano_ti_lite_26p1_41p8_pre_nms_full_integer_quant.tflite
• The int16-activation variants are now explicitly skipped with the reason: SCATTER_ND: LiteRT SCATTER_ND does not support INT16 updates.

Validation run locally:

• pytest -q tests/test_strict_integer_quantization.py tests/test_tensor_buffer_builder.py tests/test_tflite_builder_direct.py::test_flatbuffer_direct_integer_quantized_smoke
• ruff check onnx2tf/tflite_builder/quantization.py onnx2tf/tflite_builder/__init__.py tests/test_strict_integer_quantization.py tests/test_tflite_builder_direct.py
• npx --yes pyright onnx2tf/tflite_builder/quantization.py onnx2tf/tflite_builder/__init__.py tests/test_strict_integer_quantization.py tests/test_tflite_builder_direct.py
• python -m py_compile onnx2tf/onnx2tf.py onnx2tf/tflite_builder/__init__.py onnx2tf/tflite_builder/quantization.py tests/test_strict_integer_quantization.py

4. Issue number (only if there is a related issue)

Related issue: #929

onnx2tf \
-i yolox_nano_ti_lite_26p1_41p8_pre_nms.onnx \
-coion \
-oiqt \
-cind "images" "./calib_data_416x416_n200.npy" "[[[[0.485,0.456,0.406]]]]" "[[[[0.229,0.224,0.225]]]]"

onnx2tf \
-i yolox_nano_ti_lite_26p1_41p8_pre_nms.onnx \
-coion \
-oiqt \
-qt per-tensor \
-cind "images" "./calib_data_416x416_n200.npy" "[[[[0.485,0.456,0.406]]]]" "[[[[0.229,0.224,0.225]]]]"

• yolox_nano_ti_lite_26p1_41p8_pre_nms.onnx.zip
• yolox_nano_ti_lite_26p1_41p8_pre_nms_integer_quant.tflite.zip

What's Changed

• Improve flatbuffer_direct strict int8 quantization by @PINTO0309 in https://github.com/PINTO0309/onnx2tf/pull/944

Full Changelog: https://github.com/PINTO0309/onnx2tf/compare/2.5.1...2.5.2

1. Content and background

This PR fixes and hardens flatbuffer_direct integer quantization behavior.

Related context:

• Issue #941: numpy scalar tensor buffer serialization caused invalid handling for scalar constant buffers.
• Issue #942: flatbuffer_direct integer quantization previously used fixed / pseudo I/O qparams and could emit artifacts that were not strict full-integer quantized models.

The changes make flatbuffer_direct strict integer quantization calibration-based, validate generated quantized TFLite files with LiteRT, and expand strict quantized operator coverage.

2. Summary of corrections

• Fix numpy scalar tensor buffer serialization.
• Require calibration data for flatbuffer_direct strict integer quantization and collect tensor ranges using LiteRT Interpreter.
• Replace fixed integer I/O qparams with calibration-derived qparams, while preserving TFLite fixed qparams for SOFTMAX, LOGISTIC, and TANH.
• Hard fail unsupported or invalid strict integer quantization instead of emitting misleading full_integer_quant artifacts.
• Add calibration reporting for strict integer quantization.
• Expand strict full-integer operator coverage, including forwarding/layout ops, reduce ops, activation ops, PRELU, and TRANSPOSE_CONV.
• Avoid unnecessary tf_keras -> tensorflow.compat.v2 import chain in test_tflite_builder_direct.py.
• Resolve Ruff/Pylance/Pyright diagnostics in test_tflite_builder_direct.py.

3. Before/After (If there is an operating log that can be used as a reference)

Before:

• flatbuffer_direct -oiqt could use fixed qparams such as uint8 scale=1/255, zero_point=128.
• Some quantized outputs could be written even when strict full-integer validation was not guaranteed.
• test_tflite_builder_direct.py imported common_functions only for external-data detection, which could trigger tf_keras import and fail when tensorflow.compat was unavailable.

After:

• flatbuffer_direct -oiqt requires representative calibration data and writes strict quantized artifacts only after LiteRT allocation validation succeeds.
• Calibration-derived qparams are used for graph/internal tensors, with fixed TFLite-spec qparams only for relevant activations.
• Additional strict integer quantized ops are covered and tested.
• test_tflite_builder_direct.py no longer triggers the tf_keras import chain for check_model_has_external_data.

Validation performed:

pytest -q tests/test_strict_integer_quantization.py tests/test_tensor_buffer_builder.py tests/test_tflite_builder_direct.py::test_flatbuffer_direct_integer_quantized_smoke
ruff check tests/test_tflite_builder_direct.py tests/test_strict_integer_quantization.py onnx2tf/tflite_builder/quantization.py
npx --yes pyright tests/test_strict_integer_quantization.py tests/test_tflite_builder_direct.py

4. Issue number (only if there is a related issue)

#941 #942

What's Changed

• Fix flatbuffer_direct strict integer quantization by @PINTO0309 in https://github.com/PINTO0309/onnx2tf/pull/943

Full Changelog: https://github.com/PINTO0309/onnx2tf/compare/2.5.0...2.5.1

2.5.0

Summary

• Update Python dependency pins for TensorFlow 2.21, tf-keras 2.21, Keras 3.15, ONNX Runtime 1.26, OpenCV 4.13, NumPy 2.2.6, ml-dtypes 0.5.4, h5py 3.14, setuptools 81, and onnxsim 0.6.5.
• Regenerate uv.lock for the updated dependency set.
• Align Dockerfile package installation with pyproject.toml, including PyTorch 2.11 from the CPU wheel index.
• Bump package metadata/docs for the updated environment.

Validation

• Ran uv lock / lock consistency checks during the dependency update.
• Verified Dockerfile pinned package versions match pyproject.toml.
• Ran pip dry-run checks for the Dockerfile dependency sets, including TensorFlow-backed packages and the PyTorch CPU wheel.

What's Changed

• fix: support Python 3.13 by updating numpy pin to 2.2.6 by @AL3708 in https://github.com/PINTO0309/onnx2tf/pull/934
• Update dependency pins for TensorFlow 2.21 by @PINTO0309 in https://github.com/PINTO0309/onnx2tf/pull/939

New Contributors

• @AL3708 made their first contribution in https://github.com/PINTO0309/onnx2tf/pull/934

Full Changelog: https://github.com/PINTO0309/onnx2tf/compare/2.4.3...2.5.0

2.4.3

Summary

• stop re-transposing CONV_3D filters in the SavedModel exporter; flatbuffer-direct ModelIR already stores them as [d, h, w, in, out], which matches tf.nn.conv3d
• stop re-transposing CONV_3D_TRANSPOSE filters for the same reason; ModelIR stores [d, h, w, out, in], matching tf.nn.conv3d_transpose
• add SavedModel regression coverage for both 3D convolution kernels using TFLite/TF filter layouts
• Tile.py line 122. When the multiples input is a numpy array, the code does range(len(input_tensor_2.shape)) instead of range(len(input_tensor_2)). For a 1-D multiples array like [4, 1, 1], .shape is (3,) so len(.shape) is 1 instead of 3. This produces a broken initial permutation and the conversion produces wrong outputs.
• Slice.py, two separate lines. tf.clip_by_value(t=begin_, clip_value_min=0, clip_value_max=1) and the same for end_ crash because begin_ and end_ are int64 but the integer scalar bounds resolve to int32. Fix is to cast: tf.cast(begin_, tf.int32) and tf.cast(end_, tf.int32).
• LayerNormalization.py. RF-DETR's DINOv2 backbone uses windowed attention which produces a normalisation axis that is not statically known at graph build time. tf_keras.layers.LayerNormalization(axis=axis)(input_tensor) throws TypeError or ValueError when axis is dynamic. Fix is a try/except that falls back to tf.nn.moments for those cases.

Validation

• .venv/bin/python -m py_compile onnx2tf/tflite_builder/saved_model_exporter.py tests/test_tflite2sm_phase1.py
• direct exporter smoke for CONV_3D and CONV_3D_TRANSPOSE SavedModel export/load/inference, both matching TensorFlow native outputs with np.testing.assert_allclose(..., rtol=1e-5, atol=1e-5)

Issues

• #928
• #936

What's Changed

• Bug fixes for RF-DETR TFLite export by @EHxuban11 in https://github.com/PINTO0309/onnx2tf/pull/937
• Fix SavedModel Conv3D filter layouts by @aaronday-dev in https://github.com/PINTO0309/onnx2tf/pull/938

New Contributors

• @EHxuban11 made their first contribution in https://github.com/PINTO0309/onnx2tf/pull/937

Full Changelog: https://github.com/PINTO0309/onnx2tf/compare/2.4.2...2.4.3

Summary

Support for ScatterElements with reduction="add". Added support in both conversion paths.

• tf_converter
  • Uses tf.tensor_scatter_nd_add when reduction="add"
  • Keeps the existing tf.tensor_scatter_nd_update path for reduction="none"
• flatbuffer_direct
  • Lowers reduction="add" as output = data + SCATTER_ND(indices, updates, data_shape)
  • Keeps the existing mask/replace behavior for reduction="none"

What's Changed

• Support ScatterElements reduction="add" by @napronald in https://github.com/PINTO0309/onnx2tf/pull/933

New Contributors

• @napronald made their first contribution in https://github.com/PINTO0309/onnx2tf/pull/933

Full Changelog: https://github.com/PINTO0309/onnx2tf/compare/2.4.1...2.4.2

Summary

• prefer the identity permutation for NHWC-preserved InstanceNormalization inputs when candidate permutation errors are effectively tied
• require non-identity permutations to beat identity by more than the existing 1e-2 strict-mode acceptance threshold before selecting them
• keep the original minimum-error behavior for non-NHWC inputs and clear non-identity wins

Refs #930

Review notes

The issue attachment was downloadable and the reported command path was exercised locally. In this scratch environment, that full model still fails later in a downstream Conv shape mismatch, so this PR focuses on the InstanceNormalization permutation tie-break described in the issue rather than claiming full model conversion success.

Checks

• python3 -m py_compile onnx2tf/ops/InstanceNormalization.py
• git diff --check
• lightweight tie-break assertion script for near-tie, clear-win, and non-NHWC cases

Local smoke context

A synthetic InstanceNormalization conversion completed in the side-run scratch environment before review. A follow-up tf_converter smoke after review reached TensorFlow/TFLite converter logging but hung in this local environment, so it was terminated without changing the repo.

What's Changed

• Prefer layout-preserving tie-break for InstanceNormalization by @aaronday-dev in https://github.com/PINTO0309/onnx2tf/pull/931

New Contributors

• @aaronday-dev made their first contribution in https://github.com/PINTO0309/onnx2tf/pull/931

Full Changelog: https://github.com/PINTO0309/onnx2tf/compare/2.4.0...2.4.1

2.4.0

Breaking changes

This PR switches the default TFLite backend from tf_converter to flatbuffer_direct and aligns the surrounding API, CLI, tests, helper scripts, and docs with that behavior.

What changed

• changed the default tflite_backend in onnx2tf.convert() and the CLI to flatbuffer_direct
• kept tf_converter available as an explicit compatibility path
• updated helper tooling and README examples to avoid relying on the old implicit SavedModel behavior
• clarified documentation so SavedModel-related flows now require explicit direct-export flags or an explicit tf_converter selection
• added regression coverage for the default-backend path without TensorFlow
• updated the migration guide to describe flatbuffer_direct as the current default
• renamed the README supported-layers heading to make it clear that it refers to tf_converter coverage
• bumped the package version/lockfile for the 2.4.0 change

Why this improves the feature

This makes the faster direct path the out-of-the-box experience, reduces accidental dependency on TensorFlow-backed conversion, and makes the remaining legacy path explicit. It also removes ambiguity in docs and tests around which backend is responsible for SavedModel generation.

Validation

• pytest -q tests/test_optional_tensorflow.py
• pytest -q tests/test_tflite2sm_phase1.py -k "flatbuffer_direct_output_saved_model_validation or tflite_direct_input_validation or tflite_direct_input_new_conflict_validation or tflite_direct_input_rejects_mixed_onnx_and_tflite_input"
• python tests/test_model_convert.py --help

What's Changed

• Make flatbuffer_direct the default TFLite backend by @PINTO0309 in https://github.com/PINTO0309/onnx2tf/pull/925

Full Changelog: https://github.com/PINTO0309/onnx2tf/compare/2.3.19...2.4.0

2.3.19

[!IMPORTANT] Starting with onnx2tf v2.4.0, tf_converter will be deprecated and the default backend will be switched to flatbuffer_direct. With the v2.3.3 update, all backward compatible conversion options have been migrated to flatbuffer_direct, so I will only be doing minor bug fixes until April. If you provide us with ONNX sample models, I will consider incorporating them into flatbuffer_direct. I'll incorporate ai-edge-quantizer when I feel like it, but that will probably be about 10 years from now.

1. Content and background

This PR manually ports the optional dependency work from the experimental branch into feat-torch15 and updates the current codebase so that TensorFlow and PyTorch are no longer forced at base install time.

The main goal is to make the default onnx2tf installation lighter and safer for LiteRT / flatbuffer_direct users while keeping TensorFlow-backed and PyTorch-backed features available through explicit extras.

The branch also removes the remaining implicit TensorFlow dependency from -cotof when --tflite_backend flatbuffer_direct is used.

Important compatibility note: TensorFlow is now fully optional. Users who need SavedModel export, H5 export, Keras v3 export, TFv1 PB export, or tf_converter must install the TensorFlow extra explicitly.

Recommended commands:

uv pip install -U "onnx2tf[tensorflow]"

or

uv sync --extra tensorflow

or

uv sync --all-extras

Without that extra, TensorFlow-backed features now fail fast with an explicit install hint instead of importing TensorFlow eagerly at module import time.

Additional note for reviewers: on the current flatbuffer_direct path, a root SavedModel is still written by default unless --disable_model_save is specified. --flatbuffer_direct_output_saved_model is only needed when the intent is to request/guarantee the explicit flatbuffer_direct SavedModel export behavior, especially for split output handling. In all such cases, TensorFlow extra installation is still required because SavedModel generation itself is TensorFlow-backed.

2. Summary of corrections

This PR includes the following functional changes.

1. TensorFlow is no longer a mandatory base dependency.

• tensorflow and tf-keras were moved out of the base dependency set into onnx2tf[tensorflow].
• import onnx2tf and python -m onnx2tf --help no longer require TensorFlow.
• TensorFlow-backed features now go through lazy loading and explicit fail-fast checks.

1. PyTorch is no longer a mandatory base dependency.

• PyTorch-backed export and validation features were isolated behind onnx2tf[torch].
• Native package export, TorchScript export, Dynamo ONNX export, ExportedProgram export, and PyTorch validation now fail fast with explicit install guidance if PyTorch is not available.
• The torch extra is pinned to torch==2.11.0 and uses the CPU wheel index in the uv configuration.

1. TensorFlow-free runtime helpers were extracted for flatbuffer_direct.

• TensorFlow-independent runtime / schema / dummy-inference helpers were moved into a dedicated LiteRT runtime utility module.
• This allows flatbuffer_direct report generation and runtime checks to execute without pulling TensorFlow into the import path.

1. flatbuffer_direct -cotof no longer depends on TensorFlow.

• For --tflite_backend flatbuffer_direct, -cotof now means the TensorFlow-free report path only.
• The flag continues to generate ONNX↔TFLite comparison output and, when requested, ONNX↔PyTorch or TFLite↔PyTorch comparison output.
• The implicit SavedModel validation path was removed from the flatbuffer_direct -cotof flow, so -cotof itself no longer forces TensorFlow there.
• tf_converter keeps the existing TensorFlow-dependent behavior.

1. SavedModel behavior is now explicit.

• SavedModel export is still supported, but it is now clearly treated as a TensorFlow-backed optional feature.
• Users who need SavedModel-related workflows must install onnx2tf[tensorflow].
• Documentation was updated so this requirement is visible from the install section and from the relevant option descriptions.

1. Failure messages and developer ergonomics were improved.

• Optional dependency errors now suggest uv-based install commands.
• PyTorch runtime import failures caused by foreign PYTHONPATH / LD_LIBRARY_PATH contamination now return a clearer diagnostic instead of a raw traceback.

1. Documentation and tests were updated.

• README / CONTRIBUTING were updated for the new extras-based installation model.
• New regression tests were added for TensorFlow-absent and PyTorch-absent environments.
• Existing tests were updated to match the new flatbuffer_direct -cotof behavior.

3. Before/After (If there is an operating log that can be used as a reference)

Before:

• Base installation always pulled TensorFlow.
• Base installation also effectively assumed PyTorch for several flatbuffer_direct-related flows.
• import onnx2tf and some CLI entrypoints could fail early if TensorFlow / PyTorch were not available.
• flatbuffer_direct -cotof still implicitly required TensorFlow because it triggered SavedModel validation.
• SavedModel-capable flows were not clearly separated from the lightweight LiteRT-only installation path.

After:

• Base installation is lightweight and does not force TensorFlow or PyTorch.
• TensorFlow-backed features are explicitly installed with onnx2tf[tensorflow].
• PyTorch-backed features are explicitly installed with onnx2tf[torch].
• import onnx2tf and CLI help work without TensorFlow / PyTorch.
• flatbuffer_direct -cotof works without TensorFlow and stays on the TensorFlow-free evaluation/report path.
• SavedModel users now have a clearer contract: install the TensorFlow extra before using SavedModel export or other TensorFlow-backed outputs.

Validation executed on this branch:

pytest -q tests/test_optional_tensorflow.py tests/test_optional_pytorch.py
pytest -q tests/test_flatbuffer_direct_op_error_report.py tests/test_pytorch_bulk_runner.py
pytest -q tests/test_tflite_builder_direct.py -k 'flatbuffer_direct_accuracy_report_generation or split_accuracy_report_fail_on_threshold'
pytest -q tests/test_optional_tensorflow.py
pytest -q tests/test_tflite2sm_phase1.py -k 'cotof and (flatbuffer_direct_output_saved_model or tflite_direct_input_saved_model)'

4. Issue number (only if there is a related issue)

N/A

What's Changed

• Make TensorFlow/PyTorch optional and decouple flatbuffer_direct -cotof from TensorFlow by @PINTO0309 in https://github.com/PINTO0309/onnx2tf/pull/924

Full Changelog: https://github.com/PINTO0309/onnx2tf/compare/2.3.18...2.3.19

2.3.18

Summary

• remove conversion-test from the CodeQL workflow
• switch PyPI publishing to Trusted Publishing and harden GHCR login handling
• add a 7-day uv dependency cooldown with PyTorch index packages exempted
• bump the package version to 2.3.18

Verification

• validated workflow YAML parsing after workflow changes
• regenerated uv.lock with the cooldown policy in place
• confirmed the PyPI workflow no longer references PYPI_API_TOKEN

What's Changed

• Harden release workflows and bump to 2.3.18 by @PINTO0309 in https://github.com/PINTO0309/onnx2tf/pull/923

Full Changelog: https://github.com/PINTO0309/onnx2tf/compare/2.3.17...2.3.18

2.3.17

Summary

• remove all remaining download_test_image_data call sites
• stop relying on implicit network downloads for validation sample data
• require explicit calibration input mapping for -oiqt integer quantization
• update CLI/help/README text to match the new behavior
• bump version to 2.3.17

What changed

• removed the download_test_image_data import and deleted the obsolete helper from common_functions.py
• removed the validation fallback that auto-downloaded sample NHWC image data when --test_data_nhwc_path was not provided
• kept validation behavior intact by falling back to the existing dummy input generation path in dummy_onnx_inference / dummy_tf_inference
• changed -oiqt so it now exits with a clear error unless --quant_calib_input_op_name_np_data_path (-cind) is provided
• updated docstrings, CLI help, and README entries that previously described automatic sample downloads or automatic MS-COCO calibration

Behavior changes

• --test_data_nhwc_path
  • no longer triggers an automatic sample download when omitted
  • only uses user-provided NHWC test data when explicitly specified
• -oiqt
  • no longer auto-generates calibration input from downloaded images
  • now requires explicit calibration data via -cind

Validation

• python -m py_compile onnx2tf/onnx2tf.py onnx2tf/utils/common_functions.py
• rg checks confirming that download_test_image_data and the old auto-download wording no longer remain in the code/docs

https://github.com/PINTO0309/onnx2tf/issues/921

2.3.16

[!IMPORTANT] Starting with onnx2tf v2.4.0, tf_converter will be deprecated and the default backend will be switched to flatbuffer_direct. With the v2.3.3 update, all backward compatible conversion options have been migrated to flatbuffer_direct, so I will only be doing minor bug fixes until April. If you provide us with ONNX sample models, I will consider incorporating them into flatbuffer_direct. I'll incorporate ai-edge-quantizer when I feel like it, but that will probably be about 10 years from now.

What's Changed

• Fix TopK numpy 2.4 compatibility: use .item() before int() cast by @cdeil in https://github.com/PINTO0309/onnx2tf/pull/919
• Update add PR invisible Unicode guard by @PINTO0309 in https://github.com/PINTO0309/onnx2tf/pull/920

New Contributors

• @cdeil made their first contribution in https://github.com/PINTO0309/onnx2tf/pull/919

Full Changelog: https://github.com/PINTO0309/onnx2tf/compare/2.3.15...2.3.16

2.3.15

[!IMPORTANT] Starting with onnx2tf v2.4.0, tf_converter will be deprecated and the default backend will be switched to flatbuffer_direct. With the v2.3.3 update, all backward compatible conversion options have been migrated to flatbuffer_direct, so I will only be doing minor bug fixes until April. If you provide us with ONNX sample models, I will consider incorporating them into flatbuffer_direct. I'll incorporate ai-edge-quantizer when I feel like it, but that will probably be about 10 years from now.

Summary

This PR improves the flatbuffer_direct PyTorch export path with a focus on feature-level robustness rather than new surface area.

The branch tightens several failure-prone code generation and evaluation paths that were still blocking reliable use of the native PyTorch package flow in more mixed-layout models.

What changed

1. Fix mixed-layout PyTorch export bridging

The largest part of this PR hardens native PyTorch code generation for mixed channel-first / channel-last graphs.

In particular, it improves how the exporter rewrites:

• channel-last GAP -> conv bridge patterns
• dynamically shaped binary add alignment around GAP outputs
• post-canonicalization mixed-layout binary bridges
• native generated model postprocess repairs in the codegen pipeline

This is implemented mainly in onnx2tf/tflite_builder/pytorch_exporter.py and onnx2tf/tflite_builder/_pytorch_exporter_native_codegen_pipeline.py.

The practical effect is that generated PyTorch packages are less likely to produce invalid shape alignment code when the graph mixes NHWC-style public layout flow with internal channel-first execution.

2. Eliminate canonicalization stalls / timeout-prone rewrites

This PR also fixes a pathological case in the raw export canonicalization path.

The affected logic could spend too long in rewrite passes while processing shape-matched channel-first binary and batch-normalization style patterns. The branch adds explicit forward progress handling and more targeted parsing so the canonicalizer no longer gets stuck on these cases.

This includes:

• safer assignment parsing
• explicit progress through rewrite loops
• regression coverage that uses an alarm-based timeout guard in tests

3. Align TFLite evaluation pass criteria with PyTorch-style metric gating

The TFLite accuracy evaluator previously treated allclose mismatches as hard failures even when the numeric metric thresholds were still within acceptable bounds.

This branch changes the evaluation contract so that pass/fail gating matches the PyTorch evaluator more closely:

• overall metric thresholds remain authoritative
• per-output metric judgements are surfaced in the report
• threshold-safe numeric outputs no longer fail purely because allclose is stricter

This applies to both the standard evaluator and split-manifest evaluation.

Why this matters

These fixes improve the reliability of the flatbuffer_direct PyTorch workflow in real export scenarios:

• fewer broken native PyTorch packages from mixed-layout models
• more stable raw-export canonicalization
• more consistent pass/fail semantics between TFLite and PyTorch evaluation paths
• better diagnostics in generated accuracy reports through per-output metric threshold judgements

Tests

Added / updated coverage includes:

• seeded-input evaluator tests for the new pass-resolution behavior
• regression tests for canonicalization progress on channel-first shape-matched patterns
• parser regression tests for typed assignment handling without regex backtracking
• PyTorch exporter regression tests covering the new rewrite behavior

Release impact

• bumps the package version to 2.3.15
• updates README image / install references to 2.3.15

Notes

This PR is intentionally focused on PyTorch export quality and evaluation consistency. It does not introduce new user-facing flags or broaden the public API.

What's Changed

• Improve flatbuffer_direct PyTorch export stability and evaluation consistency by @PINTO0309 in https://github.com/PINTO0309/onnx2tf/pull/918

Full Changelog: https://github.com/PINTO0309/onnx2tf/compare/2.3.14...2.3.15

2.3.14

[!IMPORTANT] Starting with onnx2tf v2.4.0, tf_converter will be deprecated and the default backend will be switched to flatbuffer_direct. With the v2.3.3 update, all backward compatible conversion options have been migrated to flatbuffer_direct, so I will only be doing minor bug fixes until April. If you provide us with ONNX sample models, I will consider incorporating them into flatbuffer_direct. I'll incorporate ai-edge-quantizer when I feel like it, but that will probably be about 10 years from now.

Summary

This PR significantly improves the feat-torch10 native PyTorch export path and the flatbuffer_direct workflow, with an emphasis on replacing model-specific fixes with generalized structural logic.

The branch expands native package generation so that difficult ONNX graphs can stay on the native PyTorch backend more reliably, finish export faster, and preserve accuracy across a broad set of real-world models.

What changed

1. Generalized native PyTorch layout and structural repairs

A large portion of the work in this branch moves the exporter away from brittle model-name- or op-name-based routing and toward shape-, layout-, and dataflow-based repair rules.

Representative improvements include:

• generalized mixed-layout repair logic for Add, Concat, Resize, Reshape, Slice, pooling, channel-shuffle patterns, and public output bridges
• generalized structural repair rules for ALIKE, PIDNet, ShadowFormer, HumanSeg, NanoDet+, FastestDet, detection heads, and feature-last / channel-first bridge patterns
• generalized rank-3 and rank-4 canonicalization fixes in native package generation
• generalized depth-to-space and public-head bridge rewrites
• generalized skip / avoid-model-ir predicates so fast native canonicalization can be selected based on graph structure instead of exact generated code shapes

The practical effect is that many previously fragile native PyTorch conversions now succeed through common structural rules instead of accumulating one-off exceptions.

2. Faster native export routing and reduced pathological raw canonicalization cost

The branch adds broader fast-path routing and avoid-model-ir detection for native package export. This is especially important for models that are small but were previously taking an unexpectedly long time in write pytorch because they fell into expensive raw canonicalization.

Examples of the improvements in this area include:

• generalized fast-path detection for multiscale detection tails
• generalized avoid-model-ir predicates for structurally safe native exports
• native export timeout handling for child export steps
• native layout inference and native codegen shape inference speedups

A representative case is version-RFB-640.onnx, where the write pytorch phase now returns to a practical runtime while preserving native execution and accuracy.

3. Better evaluation robustness and tooling

This branch also improves supporting tooling around export validation:

• a new flatbuffer_direct_bulk_runner utility and its test coverage
• updates to native package runtime helpers
• improved accuracy evaluation behavior for low-energy outputs, preventing false negatives caused by cosine-only gating in cases where absolute error is already negligible

4. Expanded regression coverage

The test suite was significantly expanded, especially around the native PyTorch exporter and structural repair logic. This includes:

• targeted unit tests for newly generalized repair rules
• native package generation checks for representative model families
• tests for the new bulk runner and accuracy evaluator behavior

Why this matters

The main goal of this branch is not just to fix isolated regressions, but to improve the maintainability and reproducibility of the native PyTorch export path.

Instead of continuing to accumulate exact-name patches, the branch pushes the exporter toward a more defensible strategy:

• infer intent from graph structure
• preserve layout semantics explicitly
• canonicalize only when evidence is strong
• prefer reusable repair rules that cover families of models

This should reduce future regression risk and make new model support less dependent on ad hoc exceptions.

Validation

The branch was validated with native PyTorch output comparison on the following regression set, all passing:

• age_googlenet.onnx
• alike_t_opset11_192x320.onnx
• Atan_11.onnx
• baseline_simplified.onnx
• bread_180x320.onnx
• bread_nonfm_180x320.onnx
• deeplabv3_mobilenet_v3_large.onnx
• detpth_to_space_17.onnx
• detr_demo.onnx
• digits.onnx
• efficientformer_l1.onnx
• FastestDet.onnx
• human_segmentation_pphumanseg_2021oct.onnx
• iat_llie_180x320.onnx
• mobilenetv2-10.onnx
• nanodet-plus-m_416.onnx
• pidnet_S_cityscapes_192x320.onnx
• reducemax_softmax_workaround.onnx
• resnet18-v1-7.onnx
• rfdn_64x64.onnx
• shadowformer_istd_160x240.onnx
• sinet_320_op.onnx
• swinir-m_64x64_12.onnx
• ts_ad_model.onnx
• version-RFB-640.onnx
• yolox_s.onnx

In addition, targeted exporter tests covering the restored version-RFB-640 and yolox_s fixes were re-run and passed.

Notes for reviewers

Because this branch contains a long sequence of generalization work, the most useful review lens is by subsystem rather than by individual commit:

• onnx2tf/tflite_builder/pytorch_exporter.py
• onnx2tf/tflite_builder/_pytorch_exporter_native_codegen_pipeline.py
• onnx2tf/tflite_builder/accuracy_evaluator.py
• onnx2tf/utils/flatbuffer_direct_bulk_runner.py
• tests/test_pytorch_exporter.py
• tests/test_flatbuffer_direct_bulk_runner.py
• tests/test_accuracy_evaluator_seeded_input.py

Recommended focus areas:

• native export routing and canonicalization boundaries
• generalized structural repair predicates
• postprocess rewrites in native code generation
• regression coverage for mixed-layout and detection-tail cases

What's Changed

• Improve native PyTorch export robustness and flatbuffer_direct tooling by @PINTO0309 in https://github.com/PINTO0309/onnx2tf/pull/917

Full Changelog: https://github.com/PINTO0309/onnx2tf/compare/2.3.13...2.3.14

2.3.13

[!IMPORTANT] Starting with onnx2tf v2.4.0, tf_converter will be deprecated and the default backend will be switched to flatbuffer_direct. With the v2.3.3 update, all backward compatible conversion options have been migrated to flatbuffer_direct, so I will only be doing minor bug fixes until April. If you provide us with ONNX sample models, I will consider incorporating them into flatbuffer_direct. I'll incorporate ai-edge-quantizer when I feel like it, but that will probably be about 10 years from now.

Summary

This PR improves flatbuffer_direct conversion stability, native PyTorch package generation robustness, and constant-folding quality for several previously failing or regressing models.

The main goals of this branch were:

• reduce or eliminate write pytorch regressions in the fast path
• repair layout/shape corruption in generated native PyTorch packages
• improve constant folding for ALIKE-style exported graphs
• keep previously successful models from regressing while fixing new failures

What Changed

1. Improved native PyTorch export repair coverage

This branch expands the fast precanonicalization repair logic in pytorch_exporter.py to handle several classes of broken generated code more reliably.

Key improvements include:

• propagation of channel-first aliases so downstream softmax, concat, and related layout-sensitive ops are repaired consistently
• repair of malformed DepthToSpace-adjacent gather patterns in NHWC/CF bridge code
• restoration of broken or missing target shapes in resize, pool, scalar-binary, and alignment chains
• preservation and restoration of missing internal/output shape annotations for generated Dynamo ONNX artifacts
• repair of malformed pool/LRN/conv chains and stage-local layout bridges that previously caused accuracy loss or runtime failure
• targeted repair for nanodet-plus-m_416 stage-0 max-pool layout corruption, which also removes a severe write pytorch slowdown

These changes were validated against the regressions reported during branch development, including:

• alike_t_opset11_192x320
• efficientformer_l1
• nanodet-plus-m_416
• FastestDet
• age_googlenet

2. Better constant folding for ALIKE and similar exported graphs

This branch adds new constant-folding coverage in both the TFLite-side lowering flow and the generated Dynamo ONNX sanitization flow.

Highlights:

• constant ScatterND evaluation/folding
• constant Reshape folding
• constant binary op folding for Add/Sub/Mul/Div
• TFLite-side folding of constant ScatterND and follow-up binary chains on cloned export IR only, so the original IR remains safe for PyTorch package generation

These changes simplify generated artifacts and remove unnecessary constant computation chains in ALIKE-derived models.

3. Safer handling of integer-sensitive division lowering

lower_from_onnx2tf.py now avoids over-aggressive reciprocal-multiply lowering for precision-sensitive paths that later feed integer Cast consumers.

This preserves exact division semantics where needed, which was necessary to fix descriptor/indexing accuracy regressions in ALIKE without weakening the general constant-division optimization path.

4. AveragePool exclude-pad correction behavior refined

This branch also tightens AveragePool(count_include_pad=0) handling so that:

• TFLite export avoids incorrect extra correction in SAME cases that should already behave correctly
• native PyTorch package generation still retains the correction behavior required to preserve ONNX semantics for models such as efficientformer_l1

This separation was important to fix TFLite accuracy while avoiding PyTorch-package regressions.

5. Regression coverage expanded substantially

The test suite has been extended with focused regression tests for:

• ALIKE dynamic score sampling and constant-folding cases
• missing Dynamo ONNX output/internal shape restoration
• constant ScatterND / constant Reshape / constant binary folding
• EfficientFormer attention scalar-mul target repair
• dynamic and malformed pool target-shape repair
• NanoDet fast-path detection and stage-0 pool bridge repair
• TFLite-side integer-division preservation and constant scatter folding
• AveragePool count_include_pad=0 handling across SAME, explicit pad, and ceil-mode cases

Validation

In addition to the new unit tests, the branch was checked with real model conversions on the flatbuffer_direct path.

Confirmed as passing on this branch:

• alike_t_opset11_192x320: ONNX/TFLite pass=True, ONNX/PyTorch pass=True
• efficientformer_l1: ONNX/TFLite pass=True, ONNX/PyTorch pass=True
• FastestDet: ONNX/TFLite pass=True, ONNX/PyTorch pass=True
• age_googlenet: ONNX/TFLite pass=True, ONNX/PyTorch pass=True
• nanodet-plus-m_416: ONNX/PyTorch pass=True, and the severe write pytorch slowdown was removed

For nanodet-plus-m_416, the native PyTorch package regression was fixed and the generation time returned to a practical range, but ONNX/TFLite still reports a remaining numerical mismatch on this branch. That issue is not introduced by this PR; the work here focuses on removing the package-generation/runtime regression and preserving previously validated models.

Version

• bumped package version from 2.3.12 to 2.3.13
• updated README image/tag examples accordingly

What's Changed

• Improve flatbuffer_direct export robustness and repair coverage by @PINTO0309 in https://github.com/PINTO0309/onnx2tf/pull/915

Full Changelog: https://github.com/PINTO0309/onnx2tf/compare/2.3.12...2.3.13

2.3.12

[!IMPORTANT] Starting with onnx2tf v2.4.0, tf_converter will be deprecated and the default backend will be switched to flatbuffer_direct. With the v2.3.3 update, all backward compatible conversion options have been migrated to flatbuffer_direct, so I will only be doing minor bug fixes until April. If you provide us with ONNX sample models, I will consider incorporating them into flatbuffer_direct. I'll incorporate ai-edge-quantizer when I feel like it, but that will probably be about 10 years from now.

1. Content and background

This branch focuses on improving the practical stability of the flatbuffer_direct pipeline and the native PyTorch export path for transpose-heavy and layout-sensitive models.

The main motivation was a set of real regression cases that were already known to convert successfully in the past, but started to fail or degrade after recent changes. The affected patterns were not isolated to one model family: they spanned PIDNet, SiNet, DepthToSpace-based super-resolution models, detection heads such as FastestDet, and Dynamo ONNX artifacts generated from exported native PyTorch packages.

From a feature-improvement perspective, this PR is not just a collection of one-off fixes. It expands the optimizer and exporter so that more difficult graph topologies can stay on the fast path while preserving layout semantics, output shapes, and numerical parity.

2. Summary of corrections

• Extended flatbuffer_direct graph optimization coverage for SiNet.

  • Added multiple SiNet-specific rewrite passes to eliminate redundant transpose-heavy residual, skip, resize, affine, concat, and PReLU chains.
  • Improved the transpose-gather-transpose optimizer so it can preserve the pre-transpose when mixed consumers remain, while still removing redundant round-trips for NHWC-friendly subgraphs.
  • Added focused regression tests for the newly optimized SiNet patterns.

• Hardened the native PyTorch export canonicalization and fast-repair path.

  • Fixed the PIDNet pag4 binary alignment regression in generated native packages.
  • Added fast pre-canonicalization repair handling for SiNet-style generated packages and avoided unnecessarily expensive raw canonicalization when the fast repair path already resolves the problematic pattern.
  • Fixed DepthToSpace(mode=CRD) NHWC gather repair so the repaired gather slices the channel axis instead of corrupting the spatial height.
  • Fixed channel-first alias propagation through simple assignments so downstream axis rewrites also cover FastestDet-style heads.
  • Restored missing graph output shapes in exported Dynamo ONNX artifacts by recovering public output shape metadata from the generated package when ONNX shape inference leaves graph outputs empty.

• Improved exporter artifact consistency.

  • Native PyTorch package export, TorchScript export, Exported Program export, and Dynamo ONNX export now behave more consistently on the repaired model families.
  • The Dynamo ONNX sanitizer now preserves usable output shape annotations for downstream tooling instead of leaving terminal outputs shape-less.

• Expanded regression protection.

  • Added and updated targeted tests in tests/test_pytorch_exporter.py and tests/test_tflite_builder_direct.py for PIDNet, SiNet, DepthToSpace, FastestDet alias propagation, and Dynamo ONNX output-shape restoration.
  • Removed the obsolete mean/variance normalization parity test that was no longer needed and was surfacing third-party warnings unrelated to the project logic.

• Updated package versioning.

  • Bumped the project version and container tag references from 2.3.11 to 2.3.12.
  • Refreshed uv.lock accordingly.

3. Before/After (If there is an operating log that can be used as a reference)

Representative regression cases addressed by this branch:

• rfdn_64x64

  • Before: ONNX/PyTorch output check failed. reason=Evaluation output shape mismatch after layout alignment. onnx_shape=(1, 3, 256, 256) tflite_shape=(1, 3, 192, 256)
  • After: ONNX/PyTorch output check complete! ... max_abs=3.8147e-05 rmse=4.11336e-06 cosine=1 pass=True

• FastestDet

  • Before: max_abs=0.514514 rmse=0.0449929 cosine=0.984069 pass=False
  • After: max_abs=1.37091e-05 rmse=3.60957e-07 cosine=1 pass=True

• age_googlenet

  • Before: max_abs=0.787734 rmse=0.28074 cosine=0 pass=False
  • After: max_abs=2.11e-05 rmse=9.26852e-06 cosine=1 pass=True
  • Additional export artifact fix: the final Dynamo ONNX output shape is now restored correctly from [] to [1, 8].

Validation performed during the branch work included:

• Targeted pytest coverage for the new exporter and optimizer regressions.
• Re-running native export / accuracy-check flows on age_googlenet, FastestDet, and rfdn_64x64 to confirm that previously successful conversions did not regress.

4. Issue number (only if there is a related issue)

N/A

What's Changed

• Improve flatbuffer_direct and native PyTorch export stability by @PINTO0309 in https://github.com/PINTO0309/onnx2tf/pull/914

Full Changelog: https://github.com/PINTO0309/onnx2tf/compare/2.3.11...2.3.12

2.3.11

[!IMPORTANT] Starting with onnx2tf v2.4.0, tf_converter will be deprecated and the default backend will be switched to flatbuffer_direct. With the v2.3.3 update, all backward compatible conversion options have been migrated to flatbuffer_direct, so I will only be doing minor bug fixes until April. If you provide us with ONNX sample models, I will consider incorporating them into flatbuffer_direct. I'll incorporate ai-edge-quantizer when I feel like it, but that will probably be about 10 years from now.

Summary

This PR brings the feat-torch7 improvements into main as the 2.3.11 update for the PyTorch export path.

The main focus of this branch is exporter robustness around layout bridges, rank-3 reshape/transpose edge cases, and Dynamo ONNX post-processing. In practice, it makes the generated PyTorch package, TorchScript / ExportedProgram artifacts, and sanitized Dynamo ONNX outputs much more stable for models that mix channel-first runtime values with channel-last logical layouts.

What this PR improves

1. Native PyTorch export canonicalization is more robust

• Fixes rank-3 reshape materialization when the runtime tensor is effectively channel-first but the logical tensor metadata is channel-last.
• Preserves required NHWC materialization before rank-3 reshapes instead of collapsing it back to an unsafe alias during later rewrite passes.
• Restores correct public output resize target shapes in generated packages.
• Improves rank-3 transpose bridge handling in generated PyTorch code.

2. ExportedProgram rewrite passes are now safer

• Prevents _rewrite_generated_model_source_for_exported_program() from undoing already-correct materializations for tensors that must stay in channel-last form before reshape.
• Keeps the generated model.py semantically aligned with the original ONNX / lowered IR instead of reintroducing layout mismatches during artifact export.

3. Better handling for channel-last GAP / conv and gated bridge patterns

• Repairs channel-last global-average-pooling outputs before they are consumed by following convolution blocks.
• Adds canonicalization for channel-first hardsigmoid gate patterns so that bridge-heavy code is simplified without breaking downstream binary alignment.
• Improves bridge folding around GAP, conv, unary, and binary patterns that previously produced fragile generated code.

4. Dynamo ONNX sanitization is stronger and less shape-fragile

• Replaces rank-4-only output-shape repair with rank-agnostic static-shape recovery.
• Preserves output metadata for rank-3 transpose outputs and unary outputs that were previously at risk of losing valid shape information.
• Expands transpose/unary/binary/reduce-mean bridge folding so the exported ONNX graph is cleaner and carries more reliable boundary metadata.
• Adds folding for Mul + Add + Clip -> HardSigmoid style patterns.

5. Regression coverage is significantly expanded

This branch adds focused tests for:

• runtime channel-first aliases feeding rank-3 reshapes
• ExportedProgram rewrite safety for already-materialized NHWC aliases
• rank-3 transpose output-shape preservation in sanitized Dynamo ONNX
• unary / binary / reduce-mean layout-bridge folding
• GAP-to-conv channel-last repair
• hardsigmoid bridge folding
• model-backed regressions such as yolox_s output-shape preservation and existing bread helper behavior

Real regressions addressed by this branch

yolox_s.onnx

The generated *_dynamo.onnx could lose its final output shape metadata after sanitization. This branch preserves the expected output shape and adds regression coverage for it.

iat_llie_180x320.onnx

The generated PyTorch package could drift badly from ONNX because a required NHWC materialization was rewritten back into a raw channel-first alias before a rank-3 reshape.

After the fix, the conversion command below now passes the ONNX/PyTorch comparison:

python -m onnx2tf -i iat_llie_180x320.onnx -o /tmp/iat_llie_fix2_run -tb flatbuffer_direct -cotof -fdopt -fdots -fdodo -fdoep

Observed result after the fix:

• max_abs=4.76837e-07
• rmse=1.40854e-07
• cosine=1
• pass=True

Validation

I validated the branch with targeted exporter / sanitizer regressions, including:

pytest -q tests/test_pytorch_exporter.py -k 'runtime_cf_alias_before_rank3_reshape or exported_program_preserves_rank3_reshape_materialize_without_model_ir or preserves_transpose_rank3_output_shape or preserves_atan_rank4_output_shape'

pytest -q tests/test_pytorch_exporter.py -k 'exported_program_preserves_rank3_reshape_materialize_without_model_ir or runtime_cf_alias_before_rank3_reshape or preserves_transpose_rank3_output_shape or preserves_atan_rank4_output_shape or preserves_yolox_output_shape_when_model_is_available or helper_postprocess_is_noop_for_bread_nonfm_when_model_is_available'

Both validation sets passed locally, and the branch also includes the version bump to 2.3.11 in:

• onnx2tf/__init__.py
• pyproject.toml
• README.md

Notes for reviewers

Most of the code volume in this PR comes from two areas:

• broader layout-bridge folding / shape-repair support inside onnx2tf/tflite_builder/pytorch_exporter.py
• the matching regression suite added to tests/test_pytorch_exporter.py

The intention is not to change the overall export architecture, but to make the existing PyTorch export and Dynamo ONNX cleanup pipeline much more reliable for real-world bridge-heavy models.

What's Changed

• Improve PyTorch exporter layout bridge rewrites and Dynamo ONNX sanitization by @PINTO0309 in https://github.com/PINTO0309/onnx2tf/pull/913

Full Changelog: https://github.com/PINTO0309/onnx2tf/compare/2.3.10...2.3.11

2.3.10

[!IMPORTANT] Starting with onnx2tf v2.4.0, tf_converter will be deprecated and the default backend will be switched to flatbuffer_direct. With the v2.3.3 update, all backward compatible conversion options have been migrated to flatbuffer_direct, so I will only be doing minor bug fixes until April. If you provide us with ONNX sample models, I will consider incorporating them into flatbuffer_direct. I'll incorporate ai-edge-quantizer when I feel like it, but that will probably be about 10 years from now.

Summary

This branch hardens the native PyTorch export path so the raw-export canonicalization logic remains consistent across multiple previously fragile model families instead of fixing them one-by-one and reintroducing regressions elsewhere.

The main goal of this PR is to make the flatbuffer_direct + -cotof + -fdopt + -fdots + -fdodo + -fdoep workflow stable on a mixed regression set that repeatedly exposed layout-bridging, reshape, resize, transpose-conv, and output-boundary issues in the generated PyTorch package.

What this PR improves

• Strengthens raw-export canonicalization in pytorch_exporter.py for channel-first/channel-last alias handling, including:
  • rank-3 and rank-4 alias materialization before reshape consumers
  • singleton channel-first reshape propagation
  • resize target-shape normalization across NHWC/NCHW bridges
  • transpose-conv bridge recovery when an intermediate rank-4 view was accidentally collapsed into rank-3
  • softmax axis correction only when the tensor is provably channel-first
  • final output boundary cleanup so channel-first outputs are not needlessly re-permuted
• Preserves compatibility across previously conflicting fixes for:
  • YOLOX
  • PPHumanSeg
  • TS AD Model
  • PIDNet
  • Version-RFB
  • IAT LLIE
  • Bread
  • Bread Non-FM
• Reduces spurious native export regressions where helper-generated packages and raw-exported artifacts diverged for implementation-detail reasons rather than observable model behavior.
• Expands regression coverage in tests/test_pytorch_exporter.py and tests/test_tflite_builder_direct.py to lock in the previously fragile canonicalization paths.

Why this matters

The recent failures were not isolated bugs in a single model. They came from the same class of canonicalization decisions being correct for one topology and incorrect for another. This PR treats the problem as a cross-model consistency issue and tightens the rewrite rules so they are applied only when the exporter has enough evidence about the intended logical layout and shape.

That makes the native package generation path more predictable and reduces the cycle of model-specific hotfixes that accidentally break another regression target.

Validation

Targeted regression tests were updated/added and the generated source was revalidated with py_compile.

In addition, the following end-to-end conversions were re-run with native export enabled:

• python -m onnx2tf -i yolox_s.onnx -o tmp_yolox_s -tb flatbuffer_direct -cotof -fdopt -fdots -fdodo -fdoep
• python -m onnx2tf -i human_segmentation_pphumanseg_2021oct.onnx -o tmp_human_segmentation_pphumanseg_2021oct -tb flatbuffer_direct -cotof -fdopt -fdots -fdodo -fdoep
• python -m onnx2tf -i ts_ad_model.onnx -o tmp_ts_ad_model -tb flatbuffer_direct -cotof -fdopt -fdots -fdodo -fdoep
• python -m onnx2tf -i pidnet_S_cityscapes_192x320.onnx -o tmp_pidnet_S_cityscapes_192x320 -tb flatbuffer_direct -cotof -fdopt -fdots -fdodo -fdoep
• python -m onnx2tf -i version-RFB-640.onnx -o tmp_version-RFB-640 -tb flatbuffer_direct -cotof -fdopt -fdots -fdodo -fdoep
• python -m onnx2tf -i iat_llie_180x320.onnx -o tmp_iat_llie_180x320 -tb flatbuffer_direct -cotof -fdopt -fdots -fdodo -fdoep
• python -m onnx2tf -i bread_180x320.onnx -o tmp_bread_180x320 -tb flatbuffer_direct -cotof -fdopt -fdots -fdodo -fdoep
• python -m onnx2tf -i bread_nonfm_180x320.onnx -o tmp_bread_nonfm_180x320 -tb flatbuffer_direct -cotof -fdopt -fdots -fdodo -fdoep

For all eight models:

• no export warnings remained
• ONNX/TFLite checks passed
• ONNX/PyTorch checks passed

Scope

The largest changes are in the native PyTorch exporter and its regression coverage. The accompanying test updates are intentionally extensive because the core issue here is regression prevention across multiple model topologies, not just a single bug fix.

What's Changed

• Stabilize native PyTorch export canonicalization across regression models by @PINTO0309 in https://github.com/PINTO0309/onnx2tf/pull/912

Full Changelog: https://github.com/PINTO0309/onnx2tf/compare/2.3.9...2.3.10

2.3.9

[!IMPORTANT] Starting with onnx2tf v2.4.0, tf_converter will be deprecated and the default backend will be switched to flatbuffer_direct. With the v2.3.3 update, all backward compatible conversion options have been migrated to flatbuffer_direct, so I will only be doing minor bug fixes until April. If you provide us with ONNX sample models, I will consider incorporating them into flatbuffer_direct. I'll incorporate ai-edge-quantizer when I feel like it, but that will probably be about 10 years from now.

Summary

This PR improves the feat-torch5 line of work before merging back into main, with a focus on flatbuffer_direct lowering quality and generated PyTorch package reliability.

The branch combines structural cleanup in the PyTorch native codegen path with a broad set of behavior-preserving export fixes and graph optimizations that were validated against real-world models such as PIDNet, YOLOX, MobileBERT, SwinIR, ts_ad_model, human_segmentation_pphumanseg, iat_llie, and bread.

What Changed

1. Native PyTorch codegen refactor

• Extracted the giant native codegen implementation behind pytorch_exporter.py into private pipeline/common modules.
• Kept onnx2tf/tflite_builder/pytorch_exporter.py as the stable entrypoint while moving the heavy orchestration logic out of the main file.
• Added a dedicated native codegen pipeline and state containers to reduce editor/Pylance complexity in the public module.
• Preserved generated package behavior and added structural regression coverage.

2. flatbuffer_direct lowering improvements

• Resolved static RESHAPE -1 dimensions when the input shape is fully known.
• Added handling for 0 copy-dim semantics when allowZero=False, while keeping allowZero=True behavior unchanged.
• Folded redundant consecutive RESHAPE chains more aggressively after late shape reconciliation.
• Rewrote Div(variable, constant) patterns into Mul(variable, reciprocal_constant) when safe.
• Folded consecutive constant Mul chains into a single Mul.
• Reduced redundant Transpose/Reshape chains in multiple real models.

3. Generated PyTorch package/runtime fixes

• Fixed TorchScript export blockers in generated runtime helpers.
• Fixed Dynamo ONNX and ExportedProgram failures caused by layout-bridge handling and symbolic export incompatibilities.
• Corrected mixed NHWC/NCHW codegen around binary ops, concat/split, resize bridges, gather_nd, and broadcast constants.
• Improved channel-first shape inference for tensors whose names suggest channel-last layout but whose runtime layout is already channel-first.
• Added a safe ExportedProgram fallback path when post-export graph optimizations violate exported-program signature validation.
• Fixed generated load_state_dict typing so it matches the PyTorch base class contract and avoids Pylance override warnings.

4. Exported graph optimization work

• Removed redundant layout bridges from generated Dynamo ONNX and ExportedProgram artifacts using pattern-based rewrites instead of model-specific one-off handling.
• Simplified several NHWC<->NCHW permute chains around concat, add/mul trees, softmax, pooling, and segmentation head logic.
• Preserved output correctness while reducing graph noise in generated artifacts.

5. Test coverage

• Expanded regression tests for:
  • static reshape concretization
  • reshape chain folding
  • direct builder optimization behavior
  • TorchScript/Dynamo ONNX/ExportedProgram export regressions
  • layout-bridge folding and mixed-layout binary paths
  • native PyTorch package generation for representative models

Why This Matters

These changes make flatbuffer_direct outputs cleaner and more predictable, while also improving the reliability of the optional generated PyTorch artifacts (jit.pt, dynamo.onnx, ep.pt2).

In practice, this branch reduces redundant graph structure, resolves several export-time crashes, and improves maintainability of the PyTorch native codegen path without changing the public entrypoint.

Validation

Representative validation on this branch included:

• pytest -q tests/test_tflite_builder_direct.py -k flatbuffer_direct
• pytest -q tests/test_pytorch_exporter.py -k 'not when_model_is_available and not convert_flatbuffer_direct'
• targeted regression tests in tests/test_pytorch_exporter.py
• targeted regression tests in tests/test_tflite_builder_direct.py
• repeated end-to-end CLI verification for models including:
  • pidnet_S_cityscapes_192x320.onnx
  • yolov7_tiny_head_0.768_post_480x640.onnx
  • yolox_s.onnx
  • lite_model_mobilebert_1_metadata_1.onnx
  • human_segmentation_pphumanseg_2021oct.onnx
  • swinir-m_64x64_12.onnx
  • ts_ad_model.onnx
  • iat_llie_180x320.onnx
  • bread_180x320.onnx

Notes

• This branch includes both functional fixes and non-trivial internal refactoring, so review is easiest if split by area: native codegen pipeline, flatbuffer_direct lowering, and PyTorch artifact/export cleanup.
• The public pytorch_exporter.py entrypoint remains intact even though the native codegen internals were reorganized.

What's Changed

• Improve flatbuffer_direct lowering and PyTorch export reliability by @PINTO0309 in https://github.com/PINTO0309/onnx2tf/pull/911

Full Changelog: https://github.com/PINTO0309/onnx2tf/compare/2.3.8...2.3.9

2.3.8

[!IMPORTANT] Starting with onnx2tf v2.4.0, tf_converter will be deprecated and the default backend will be switched to flatbuffer_direct. With the v2.3.3 update, all backward compatible conversion options have been migrated to flatbuffer_direct, so I will only be doing minor bug fixes until April. If you provide us with ONNX sample models, I will consider incorporating them into flatbuffer_direct. I'll incorporate ai-edge-quantizer when I feel like it, but that will probably be about 10 years from now.

1. Content and background

This PR continues the feat-torch* line of work by tightening the native export path around flatbuffer_direct and the generated PyTorch artifacts.

The branch focuses on practical reliability improvements for the direct ModelIR-based path that is intended to become the main conversion flow:

• fixes native PyTorch export failures and numerical regressions seen in real models such as Faster R-CNN style graphs and finder.onnx
• improves flatbuffer_direct lowering and validation behavior for shape/index/control-flow related edge cases
• expands direct-path test coverage so regressions are caught earlier in both export and evaluation stages
• updates the package version to 2.3.8
• refreshes the publish workflow dependency baseline for Node 24

From a feature-improvement perspective, the main outcome is that flatbuffer_direct now behaves more like a first-class backend end-to-end: direct export, native PyTorch package generation, accuracy checking, and related validation/reporting are more consistent and substantially more robust on non-trivial graphs.

2. Summary of corrections

Key improvements included in this branch:

• Fixed native torch export regressions affecting flatbuffer_direct output packages.

  • corrected reshape/layout handling in generated PyTorch code so rank-2 flatten paths and singleton-axis drops no longer introduce invalid channel-last permutations
  • resolved a GridSample-adjacent accuracy regression where coordinate helper reshapes were reordered incorrectly in the generated package
  • improved native export behavior for Faster R-CNN related graph patterns

• Strengthened flatbuffer_direct lowering and runtime shape handling.

  • extended lowering support in shape/index/control-flow related builders and registry validation
  • improved ModelIR-to-PyTorch codegen around reshape, slice/strided-slice, gather/gather_nd, integer division, and related dynamic-shape cases
  • added runtime-side support needed by the updated generated code paths

• Expanded test coverage for direct export and native artifact generation.

  • added focused PyTorch exporter regression tests for reshape/layout preservation and singleton-axis handling
  • added/extended flatbuffer_direct tests covering direct lowering, validation, and numerical/runtime edge cases
  • kept README and package metadata aligned with the new behavior and release version

• Updated release and CI metadata.

  • bumped version metadata to 2.3.8
  • updated the publish workflow dependency baseline for Node 24

3. Before/After (If there is an operating log that can be used as a reference)

A representative real-model regression fixed in this branch was the finder.onnx direct export flow.

Command:

onnx2tf -i finder.onnx -o tmp_finder -tb flatbuffer_direct -cotof -fdopt -fdots -fdodo -fdoep

Before:

• ONNX/TFLite: pass
• ONNX/PyTorch: fail
• representative metrics:
  • max_abs=0.043901
  • rmse=0.01257
  • cosine=0.999059

After:

• ONNX/TFLite: pass
• ONNX/PyTorch: pass
• representative metrics:
  • max_abs=6.84522e-08
  • rmse=2.1783e-08
  • cosine=1

Validation performed for this branch:

• targeted PyTorch exporter regression tests for the new reshape/layout fixes: passed
• full flatbuffer_direct focused test run:
  • pytest -q tests -k flatbuffer_direct
  • result: 685 passed, 393 deselected, 3 warnings

4. Issue number (only if there is a related issue)

No related issue was linked for this change set.

2.3.7

[!IMPORTANT] Starting with onnx2tf v2.4.0, tf_converter will be deprecated and the default backend will be switched to flatbuffer_direct. With the v2.3.3 update, all backward compatible conversion options have been migrated to flatbuffer_direct, so I will only be doing minor bug fixes until April. If you provide us with ONNX sample models, I will consider incorporating them into flatbuffer_direct. I'll incorporate ai-edge-quantizer when I feel like it, but that will probably be about 10 years from now.

1. Content and background

This PR brings the feat-torch3 work to main with a focus on two practical improvements:

• significantly broader native PyTorch package generation coverage, especially for recurrent models and dynamic-shape cases
• substantially better flatbuffer_direct correctness, graph cleanup, and validation diagnostics

Before these changes, several recurrent models still fell back to non-native execution backends, which meant TorchScript / Dynamo ONNX / ExportedProgram artifacts were skipped even when the generated package was otherwise usable. In parallel, flatbuffer_direct still had a few correctness gaps, including recurrent alias repair issues, layout-sensitive flatten behavior, redundant transpose chains, and a major AveragePool border-semantics mismatch that could produce visible accuracy regressions on real models.

The goal of this branch was not just to patch isolated failures, but to make the native export path and the direct flatbuffer path more robust as product features.

2. Summary of corrections

• Added broader native recurrent support in the PyTorch exporter, including sequence LSTM / bidirectional LSTM handling, sequence RNN support, and native codegen for REVERSE_V2-based reverse-direction recurrent paths.
• Reduced avoidable fallback to saved_model / tflite execution backends by keeping supported recurrent graphs on the native backend instead of forcing unrolled primitive fallback in those cases.
• Improved generated native package reliability for dynamic-shape models by making the emitted shape helpers, layout handling, and recurrent wrappers compatible with TorchScript, Dynamo ONNX export, and ExportedProgram export.
• Fixed NHWC/NCHW public-boundary inference and flatten/layout handling so native PyTorch packages preserve the intended logical tensor layout in more cases.
• Repaired recurrent lowering issues in flatbuffer_direct, including orphan final-step tensor aliases that could leave unbound internal inputs in GRU-derived graphs.
• Optimized multiple redundant transpose chains in flatbuffer_direct, especially the split/concat patterns exposed by res2net50_48w_2s_Opset16.onnx, to reduce unnecessary NCHW/NHWC round-trips.
• Fixed AveragePool(count_include_pad=1) border semantics in flatbuffer_direct by materializing zero padding explicitly before VALID pooling when needed, instead of relying on TFLite SAME behavior that does not match ONNX exactly at the padded border.
• Improved additional direct-builder correctness paths, including lowered flatten metadata propagation, shape-chain dtype consistency, selected INT64 unary/scatternd handling, and op-error reporting coverage.
• Expanded regression coverage substantially across native exporter tests and flatbuffer_direct runtime/lowering tests.
• Updated GitHub Actions workflows to Node 24.

3. Before/After (If there is an operating log that can be used as a reference)

Native export behavior before this branch:

• supported recurrent models could still degrade to execution_backend=saved_model or execution_backend=tflite
• requested TorchScript / Dynamo ONNX / ExportedProgram artifacts could be skipped for dynamic recurrent packages

Native export behavior after this branch:

• covered recurrent models remain on execution_backend=native
• the generated native packages emit the expected artifact set for the validated dynamic LSTM cases

flatbuffer_direct accuracy before the AveragePool semantic fix:

• command:
  • onnx2tf -i res2net50_48w_2s_Opset16.onnx -cotof -tb flatbuffer_direct
• result:
  • max_abs=0.22682 rmse=0.0500467 cosine=0.999387 pass=False

flatbuffer_direct accuracy after the fix:

• command:
  • onnx2tf -i res2net50_48w_2s_Opset16.onnx -cotof -tb flatbuffer_direct
• result:
  • max_abs=4.76837e-06 rmse=8.87476e-07 cosine=1 pass=True

Validation executed on this branch:

• pytest -q tests/test_tflite_builder_direct.py -k "average_pool_include_pad or average_pool_exclude_pad"
  • 4 passed
• pytest -q tests -k flatbuffer_direct
  • 668 passed, 352 deselected, 3 warnings

4. Issue number (only if there is a related issue)

• Performance regression on a GPU with flatbuffer_direct backend #907

2.3.6

[!IMPORTANT] Starting with onnx2tf v2.4.0, tf_converter will be deprecated and the default backend will be switched to flatbuffer_direct. With the v2.3.3 update, all backward compatible conversion options have been migrated to flatbuffer_direct, so I will only be doing minor bug fixes until April. If you provide us with ONNX sample models, I will consider incorporating them into flatbuffer_direct. I'll incorporate ai-edge-quantizer when I feel like it, but that will probably be about 10 years from now.

Summary

This PR advances the feat-torch2 line by making flatbuffer_direct-generated native PyTorch packages much more useful in Torch 2 workflows and by tightening native export/runtime behavior on models that previously fell back, misreported shapes, or failed secondary artifact generation.

From a functional perspective, the branch now goes beyond generating a PyTorch package and TorchScript artifact. It can also emit Torch 2-oriented artifacts directly from the generated native package, while keeping the flatbuffer_direct path aligned with real-model parity and validation needs.

<img width="1390" height="680" alt="image" src="https://github.com/user-attachments/assets/8a78b9f7-1557-4e5a-954d-cff9a4a74d04" />

Functional improvements

• Adds --flatbuffer_direct_output_dynamo_onnx (-fdodo) so a generated native PyTorch package can emit <model>_dynamo.onnx via torch.onnx.export(..., dynamo=True).
• Adds --flatbuffer_direct_output_exported_program (-fdoep) so the same package can emit <model>_ep.pt2 via torch.export.save.
• Exposes the same functionality in the Python API through flatbuffer_direct_output_dynamo_onnx and flatbuffer_direct_output_exported_program.
• Automatically enables flatbuffer_direct_output_pytorch when any secondary PyTorch artifact is requested, keeping the UX consistent with existing TorchScript behavior.
• Extends split export flows so split packages can also produce Dynamo ONNX and ExportedProgram artifacts, with counts and output paths surfaced in the returned metadata.

Native package and export robustness

• Adds dedicated generated-package export helpers for TorchScript, Dynamo ONNX, and ExportedProgram generation with subprocess isolation and structured metadata updates.
• Records success and failure details back into each generated package metadata.json, including example input shapes, dynamic-input state, and export errors.
• Reuses the existing example-input pipeline for Torch 2 exports, including support for shape_hints, test_data_nhwc_path, and per-input custom example data.
• Fixes a real blocker for Torch 2 export on native packages by materializing constant reduction axes directly in generated code instead of forcing tensor-to-Python integer extraction during export.
• Improves native codegen/runtime handling around layout-sensitive operations so generated packages stay exportable on more realistic models instead of only trivial cases.

flatbuffer_direct coverage and parity improvements

• Improves native/export handling for models using layout-sensitive softmax patterns, concat alignment edge cases, and additional channel-first/channel-last conversions.
• Refines 3D convolution and transposed-3D-convolution weight rewriting for native PyTorch codegen.
• Fixes TopK output shape/signature handling in the flatbuffer_direct lowering path, including cases where k is constant, stale, or runtime-derived.
• Expands runtime and evaluation support used by the generated packages so ONNX/TFLite/PyTorch comparisons remain viable on a broader set of outputs.
• Keeps user-facing reporting aligned by surfacing success/failure messages for direct and split Dynamo ONNX / ExportedProgram outputs from the CLI conversion flow.

Documentation and packaging

• Documents the new CLI and Python API options in README.md, including dynamic-input guidance and how shape_hints, -cind, and --test_data_nhwc_path apply to the new export modes.
• Bumps the package version to 2.3.6.
• Adds onnxscript==0.6.2, which is required by the Dynamo ONNX export path.

Test coverage

• Expands exporter tests for:
  • generated-package Dynamo ONNX output
  • generated-package ExportedProgram output
  • split PyTorch auxiliary artifacts
  • constant-axis reduction exportability
  • native package parity on more realistic model shapes and operators
• Expands direct-builder coverage around TopK, parity validation, and related flatbuffer_direct behavior.

Validation

• Ran pytest -q tests -k flatbuffer_direct
  • Result: 651 passed, 317 deselected, 3 warnings
• Verified the previously failing real-model workflow for pidnet_S_cityscapes_192x320.onnx now emits both:
  • pidnet_S_cityscapes_192x320_dynamo.onnx
  • pidnet_S_cityscapes_192x320_ep.pt2 instead of only writing warning entries to the generated package metadata.

Why this matters

Before this branch, the flatbuffer_direct Torch path was useful primarily for package generation and, in some cases, TorchScript export. After this change, the generated native package becomes a practical bridge into broader Torch 2 ecosystems: users can generate Dynamo ONNX and ExportedProgram artifacts directly from the same conversion output, while the lowering/runtime fixes reduce the gap between synthetic smoke tests and real deployment models.

What's Changed

• feat(flatbuffer_direct): expand Torch 2 export artifacts and native coverage by @PINTO0309 in https://github.com/PINTO0309/onnx2tf/pull/906

Full Changelog: https://github.com/PINTO0309/onnx2tf/compare/2.3.5...2.3.6

2.3.5

[!IMPORTANT] Starting with onnx2tf v2.4.0, tf_converter will be deprecated and the default backend will be switched to flatbuffer_direct. With the v2.3.3 update, all backward compatible conversion options have been migrated to flatbuffer_direct, so I will only be doing minor bug fixes until April. If you provide us with ONNX sample models, I will consider incorporating them into flatbuffer_direct.

Summary

This PR significantly expands the feat-torch work around the flatbuffer_direct pipeline, with a primary focus on native PyTorch package generation, direct evaluation, and end-to-end validation.

The branch moves PyTorch export from an experimental wrapper-heavy path toward a practical direct-codegen flow built on ModelIR, with better output readability, stronger validation, and broader compatibility for both ONNX-input and TFLite-input workflows.

What Changed

1. Added native PyTorch package generation from ModelIR

Added a path to generate PyTorch model source code from a LiteRT model without going through ONNX. This is a very experimental implementation and has only been tested on a few models.

<img width="1425" height="617" alt="image" src="https://github.com/user-attachments/assets/54e0a68f-2e00-484c-9d35-91484ed5b842" />

• Introduced direct PyTorch package export from flatbuffer_direct ModelIR.
• Added a dedicated PyTorch exporter and runtime:
  • onnx2tf/tflite_builder/pytorch_exporter.py
  • onnx2tf/tflite_builder/pytorch_package_runtime.py
• Native packages now generate a real torch.nn.Module-based model.py instead of always falling back to a wrapper package.
• Generated native packages support:
  • Model(...)
  • load_model(...)
  • state_dict.pth loading through standard load_state_dict(...)
• The generated state_dict.pth format was updated to be compatible with standard PyTorch state_dict() keys for native packages.

<details><summary>A sample PyTorch model model.py generated from LiteRT</summary>

from __future__ import annotations

from pathlib import Path
from typing import Any, Callable, Dict, Optional

import torch
import torch.nn.functional as F

from .runtime import (
    _align_tensor_to_target_shape,
    _apply_module_transpose_conv2d,
    _normalize_axes,
    _reduce_mean,
    _resolve_named_input_value,
    _shape_list,
    load_generated_weights,
)

PACKAGE_DIR = Path(__file__).resolve().parent
INPUT_NAMES = ['onnx____Reshape_0']
OUTPUT_NAMES = ['63']
class _Conv2dBlock(torch.nn.Module):
    def __init__(
        self,
        *,
        in_channels: int,
        out_channels: int,
        kernel_size: tuple[int, int],
        stride: tuple[int, int],
        padding: tuple[int, int],
        dilation: tuple[int, int],
        groups: int,
        bias: bool,
        pad: Optional[list[int]] = None,
        activation: str = 'none',
        negative_slope: float = 0.2,
        pad_mode: str = 'constant',
        pad_value: float = 0.0,
    ) -> None:
        super().__init__()
        self.conv = torch.nn.Conv2d(
            in_channels=in_channels,
            out_channels=out_channels,
            kernel_size=kernel_size,
            stride=stride,
            padding=padding,
            dilation=dilation,
            groups=groups,
            bias=bias,
        )
        self.pad = pad
        self.activation = str(activation)
        self.negative_slope = float(negative_slope)
        self.pad_mode = str(pad_mode)
        self.pad_value = float(pad_value)

    def forward(self, x: torch.Tensor) -> torch.Tensor:
        if self.pad is not None:
            x = F.pad(x, self.pad, mode=self.pad_mode, value=self.pad_value)
        x = self.conv(x)
        if self.activation == 'leaky_relu':
            return F.leaky_relu(x, negative_slope=self.negative_slope)
        if self.activation == 'relu':
            return torch.relu(x)
        if self.activation == 'relu6':
            return torch.clamp(x, min=0.0, max=6.0)
        if self.activation == 'relu_n1_to_1':
            return torch.clamp(x, min=-1.0, max=1.0)
        if self.activation == 'relu_0_to_1':
            return torch.clamp(x, min=0.0, max=1.0)
        if self.activation == 'tanh':
            return torch.tanh(x)
        if self.activation == 'sigmoid':
            return torch.sigmoid(x)
        return x

class Model(torch.nn.Module):
    const_ConvTranspose_12_transpose_conv_bias: torch.Tensor
    const_ConvTranspose_15_transpose_conv_bias: torch.Tensor
    const_ConvTranspose_18_transpose_conv_bias: torch.Tensor
    const_decoder_decoder_lin_0_bias: torch.Tensor
    const_onnx__MatMul_73: torch.Tensor
    const_onnx__MatMul_74: torch.Tensor
    def __init__(self, *, device: str | None = None, eval_mode: bool = True, load_weights: bool = True):
        super().__init__()
        self.input_names = list(INPUT_NAMES)
        self.output_names = list(OUTPUT_NAMES)
        self.conv_block_0 = _Conv2dBlock(
            in_channels=1,
            out_channels=64,
            kernel_size=(1, 9),
            stride=(1, 1),
            padding=(0, 4),
            dilation=(1, 1),
            groups=1,
            bias=True,
            pad=None,
            activation='relu',
            negative_slope=0.2,
            pad_mode='constant',
            pad_value=0.0,
        )
        self.conv_block_1 = _Conv2dBlock(
            in_channels=64,
            out_channels=128,
            kernel_size=(1, 5),
            stride=(1, 1),
            padding=(0, 2),
            dilation=(1, 1),
            groups=1,
            bias=True,
            pad=None,
            activation='relu',
            negative_slope=0.2,
            pad_mode='constant',
            pad_value=0.0,
        )
        self.conv_block_2 = _Conv2dBlock(
            in_channels=128,
            out_channels=256,
            kernel_size=(1, 3),
            stride=(1, 1),
            padding=(0, 0),
            dilation=(1, 1),
            groups=1,
            bias=True,
            pad=[2, 2, 0, 0, 0, 0, 0, 0],
            activation='none',
            negative_slope=0.2,
            pad_mode='constant',
            pad_value=0.0,
        )
        self.conv_transpose2d_0 = torch.nn.ConvTranspose2d(
            in_channels=256,
            out_channels=128,
            kernel_size=(1, 3),
            stride=(1, 1),
            padding=(0, 0),
            bias=False,
        )
        self.conv_transpose2d_1 = torch.nn.ConvTranspose2d(
            in_channels=128,
            out_channels=64,
            kernel_size=(1, 5),
            stride=(1, 1),
            padding=(0, 0),
            bias=False,
        )
        self.conv_transpose2d_2 = torch.nn.ConvTranspose2d(
            in_channels=64,
            out_channels=64,
            kernel_size=(1, 9),
            stride=(1, 1),
            padding=(0, 0),
            bias=False,
        )
        self._init_constants()
        if load_weights:
            load_generated_weights(
                model=self,
                package_dir=PACKAGE_DIR,
                device=device,
            )
        elif device is not None:
            self.to(device)
        if eval_mode:
            self.eval()

    def _init_constants(self) -> None:
        self.register_buffer('const_ConvTranspose_12_transpose_conv_bias', torch.zeros([1, 1, 1, 128], dtype=torch.float32), persistent=True)
        self.register_buffer('const_ConvTranspose_15_transpose_conv_bias', torch.zeros([1, 1, 1, 64], dtype=torch.float32), persistent=True)
        self.register_buffer('const_ConvTranspose_18_transpose_conv_bias', torch.zeros([1, 1, 1, 64], dtype=torch.float32), persistent=True)
        self.register_buffer('const_decoder_decoder_lin_0_bias', torch.zeros([1, 66, 1], dtype=torch.float32), persistent=True)
        self.register_buffer('const_onnx__MatMul_73', torch.zeros([66, 2], dtype=torch.float32), persistent=True)
        self.register_buffer('const_onnx__MatMul_74', torch.zeros([2, 66], dtype=torch.float32), persistent=True)

    def _device(self) -> torch.device:
        for parameter in self.parameters():
            return parameter.device
        for buffer in self.buffers():
            return buffer.device
        return torch.device('cpu')

    def _max_pool2d_same(self, x: torch.Tensor, *, kernel_size: tuple[int, int], stride: tuple[int, int]) -> torch.Tensor:
        pad_h_total = max(int(kernel_size[0]) - int(stride[0]), 0)
        pad_w_total = max(int(kernel_size[1]) - int(stride[1]), 0)
        pad_top = pad_h_total // 2
        pad_bottom = pad_h_total - pad_top
        pad_left = pad_w_total // 2
        pad_right = pad_w_total - pad_left
        x = F.pad(x, [pad_left, pad_right, pad_top, pad_bottom], mode='constant', value=float('-inf'))
        return F.max_pool2d(x, kernel_size=kernel_size, stride=stride)

    def forward(self, *args: torch.Tensor, **kwargs: torch.Tensor) -> Any:
        if len(args) > 0 and len(kwargs) > 0:
            raise RuntimeError('Use either positional inputs or keyword inputs, not both.')
        if len(kwargs) > 0:
            onnx____Reshape_0 = _resolve_named_input_value(kwargs, 'onnx____Reshape_0')
        else:
            if len(args) != 1:
                raise RuntimeError(f'Input arity mismatch. expected={len(self.input_names)} actual={len(args)}')
            onnx____Reshape_0 = args[0]
        input = torch.reshape(onnx____Reshape_0, [int(v) for v in _shape_list(torch.as_tensor([-1, 1, 64], dtype=torch.int32, device=self._device()))])
        Conv_2_conv1d_input_nchw2d = torch.reshape(input, [int(v) for v in _shape_list(torch.as_tensor([1, 1, 1, 64], dtype=torch.int32, device=self._device()))])
        Conv_2_input = torch.reshape(Conv_2_conv1d_input_nchw2d, [int(v) for v in _shape_list(torch.as_tensor([1, 1, 64, 1], dtype=torch.int32, device=self._device()))])
        Conv_4_input = self.conv_block_0(Conv_2_input.permute(0, 1, 3, 2).contiguous())
        Conv_6_input = self.conv_block_1(Conv_4_input)
        Conv_6_output = self.conv_block_2(Conv_6_input)
        input_31 = torch.reshape(Conv_6_output.permute(0, 3, 1, 2).contiguous(), [int(v) for v in [1, 66, 256]])
        input_35 = torch.relu(input_31)
        _tmp_x_11 = input_35
        _tmp_y_11 = self.const_onnx__MatMul_73
        _tmp_x_11 = _tmp_x_11.transpose(-1, -2)
        onnx____Add_49 = _align_tensor_to_target_shape(torch.matmul(_tmp_x_11, _tmp_y_11), [1, 256, 2])
        onnx____MatMul_50 = torch.add(torch.as_tensor([0.05643776059150696, -0.00908423587679863], dtype=torch.float32, device=self._device()).reshape([1, 1, 2]), onnx____Add_49)
        _tmp_x_13 = self.const_onnx__MatMul_74
        _tmp_y_13 = onnx____MatMul_50
        _tmp_x_13 = _tmp_x_13.transpose(-1, -2)
        _tmp_y_13 = _tmp_y_13.transpose(-1, -2)
        onnx____Add_52 = torch.matmul(_tmp_x_13, _tmp_y_13)
        onnx____ConvTranspose_53 = torch.add(self.const_decoder_decoder_lin_0_bias, onnx____Add_52)
        ConvTranspose_12_convtranspose1d_input_nchw2d = torch.reshape(onnx____ConvTranspose_53, [int(v) for v in _shape_list(torch.as_tensor([1, 1, 66, 256], dtype=torch.int32, device=self._device()))])
        ConvTranspose_12_output = _apply_module_transpose_conv2d(self.conv_transpose2d_0, ConvTranspose_12_convtranspose1d_input_nchw2d, target_shape=[1, 1, 68, 128], fallback_shape=[1, 1, 68, 128], fused='NONE')
        ConvTranspose_12_output_nhwc_cropped = ConvTranspose_12_output[0:1, 0:1, 2:66, 0:128]
        ConvTranspose_12_output_nhwc_bias = torch.add(ConvTranspose_12_output_nhwc_cropped, self.const_ConvTranspose_12_transpose_conv_bias)
        ConvTranspose_15_input = torch.relu(ConvTranspose_12_output_nhwc_bias)
        ConvTranspose_15_output = _apply_module_transpose_conv2d(self.conv_transpose2d_1, ConvTranspose_15_input, target_shape=[1, 1, 68, 64], fallback_shape=[1, 1, 68, 64], fused='NONE')
        ConvTranspose_15_output_nhwc_cropped = ConvTranspose_15_output[0:1, 0:1, 2:66, 0:64]
        ConvTranspose_15_output_nhwc_bias = torch.add(ConvTranspose_15_output_nhwc_cropped, self.const_ConvTranspose_15_transpose_conv_bias)
        ConvTranspose_18_input = torch.relu(ConvTranspose_15_output_nhwc_bias)
        ConvTranspose_18_output = _apply_module_transpose_conv2d(self.conv_transpose2d_2, ConvTranspose_18_input, target_shape=[1, 1, 72, 64], fallback_shape=[1, 1, 72, 64], fused='NONE')
        ConvTranspose_18_output_nhwc_cropped = ConvTranspose_18_output[0:1, 0:1, 4:68, 0:64]
        ConvTranspose_18_output_nhwc_bias = torch.add(ConvTranspose_18_output_nhwc_cropped, self.const_ConvTranspose_18_transpose_conv_bias)
        onnx____ReduceMean_60 = torch.reshape(ConvTranspose_18_output_nhwc_bias, [int(v) for v in _shape_list(torch.as_tensor([1, 64, 64], dtype=torch.int32, device=self._device()))])
        onnx____Squeeze_61 = _reduce_mean(onnx____ReduceMean_60, _normalize_axes(torch.as_tensor([2], dtype=torch.int32, device=self._device()), onnx____ReduceMean_60.ndim), True)
        t_63 = torch.reshape(onnx____Squeeze_61, [int(v) for v in _shape_list(torch.as_tensor([1, 64], dtype=torch.int32, device=self._device()))])
        return t_63

    def forward_named(self, *args: torch.Tensor, **kwargs: torch.Tensor) -> Dict[str, torch.Tensor]:
        return {'63': self.forward(*args, **kwargs)}

def load_model(device: str | None = None, eval_mode: bool = True) -> Model:
    return Model(device=device, eval_mode=eval_mode)

</details>

2. Made generated model.py more PyTorch-like

• Refactored code generation to favor a PyTorch-first style instead of exposing low-level export/runtime details.
• Simplified generated source for native packages by:
  • reducing wrapper/template output
  • moving low-level loading/runtime helpers out of model.py
  • preferring direct torch / torch.nn.functional expressions
  • introducing block-style generation for repeated patterns
  • splitting complex forward() methods into stage/helper structure where appropriate
• Improved readability for simple models such as fused_conv_relu.
• Extended the same philosophy to more complex models such as:
  • yolov7_tiny_head_0.768_post_480x640
  • yolox_s
  • lite_model_mobilebert_1_metadata_1

3. Expanded native operator/codegen coverage

• Added or improved native export/codegen/runtime support for a broad set of ops and graph patterns, including coverage needed by real models in target_models.txt.
• This includes work across layout propagation, recurrent handling, detection postprocessing, reshape/layout synchronization, and direct lowering for operators such as:
  • GATHER, GATHER_ND
  • SLICE, STRIDED_SLICE
  • RESIZE_*
  • LOGISTIC, SOFTMAX
  • DEPTH_TO_SPACE, SPACE_TO_DEPTH
  • ARG_MAX, ARG_MIN
  • AVERAGE_POOL_2D, MAX_POOL_2D
  • NON_MAX_SUPPRESSION_V4/V5
  • recurrent sequence ops and related state-shape handling
• Strengthened channel-first normalization and layout propagation so native export can stay closer to optimized PyTorch execution instead of bouncing through stale NHWC-style intermediates.

4. Added direct PyTorch accuracy evaluation and comparison reports

• Added PyTorch package evaluation support:
  • onnx2tf/tflite_builder/pytorch_accuracy_evaluator.py
• -tb flatbuffer_direct -fdopt -cotof now supports same-input comparison reporting for:
  • ONNX ↔ TFLite
  • ONNX ↔ PyTorch
• Added combined comparison reports that summarize both results side-by-side.
• Introduced task-aware pass handling for cases such as probability-map outputs where exact tensor equality is not the right semantic criterion.
• Added handling for ONNX Runtime inference failures so that PyTorch validation can continue when the user explicitly wants those exceptions ignored.

5. Added official -it + -fdopt support

• Formalized support for using --input_tflite_file_path together with:
  • -tb flatbuffer_direct
  • -fdopt
  • -cotof
• For -it + -fdopt + -cotof, the reference comparison is now TFLite ↔ PyTorch instead of ONNX ↔ PyTorch.
• Added report generation for direct TFLite-input workflows.
• Updated CLI behavior and README examples accordingly.

6. Added PyTorch bulk verification tooling

• Added a reusable bulk verification CLI:
  • onnx2tf-pytorch-bulk
  • implemented in onnx2tf/utils/pytorch_bulk_runner.py
• The bulk runner supports:
  • ordered processing of target_models.txt
  • duplicate preservation
  • skipping missing models
  • resume support
  • per-model logs and reports
  • PyTorch package smoke/accuracy evaluation
• This was used to drive real-model closure work across many ONNX models.

7. Improved flatbuffer_direct SavedModel/TFLite bridge behavior

• Added/expanded direct-export and bridge behavior for SavedModel and PyTorch outputs.
• Avoids materializing unnecessary hidden SavedModel bridge directories when native PyTorch export succeeds.
• Added missing SavedModel exporter kernels such as:
  • RANDOM_UNIFORM
  • REVERSE_SEQUENCE
• Kept flatbuffer_direct validation and report generation aligned across ONNX-input and TFLite-input paths.

8. Documentation and test coverage

• Updated README.md extensively for the new PyTorch export flow.
• Added examples for:
  • flatbuffer_direct PyTorch export
  • TFLite-input direct PyTorch export (-it + -fdopt)
  • comparison reporting with -cotof
• Added explicit documentation that generated native PyTorch packages are inference-oriented and should not be treated as training-ready models without further redesign.
• Added or expanded large test coverage across:
  • native codegen
  • accuracy evaluators
  • bulk runner
  • split/export flows
  • direct TFLite-input workflows

Why This Matters

This branch makes flatbuffer_direct much more practical as a direct export pipeline rather than only a TFLite writer. In particular, it enables:

• native PyTorch package output from direct-exported graphs
• direct validation against ONNX or TFLite references
• better generated PyTorch source quality
• larger real-model coverage
• direct TFLite-input to PyTorch workflows without falling back to the legacy converter path

Testing

Verified with focused tests during implementation and with the flatbuffer_direct suite:

pytest -q tests/test_tflite2sm_phase1.py tests/test_pytorch_exporter.py tests/test_model_convert.py tests/test_flatbuffer_direct_op_error_report.py tests/test_tflite_builder_direct.py

Result:

• 793 passed
• 4 warnings

Notes

• The generated native PyTorch packages are intended primarily for inference/package validation, not as drop-in training architectures.
• Existing public dependency constraints were not changed in pyproject.toml; this branch only bumps the package version.

What's Changed

• Improve flatbuffer_direct PyTorch export and validation workflow by @PINTO0309 in https://github.com/PINTO0309/onnx2tf/pull/905

Full Changelog: https://github.com/PINTO0309/onnx2tf/compare/2.3.4...2.3.5

2.3.4

[!IMPORTANT] Starting with onnx2tf v2.4.0, tf_converter will be deprecated and the default backend will be switched to flatbuffer_direct. With the v2.3.3 update, all backward compatible conversion options have been migrated to flatbuffer_direct, so I will only be doing minor bug fixes until April. If you provide us with ONNX sample models, I will consider incorporating them into flatbuffer_direct.

Summary

This PR significantly expands flatbuffer_direct builtin support and tightens runtime safety for the direct TFLite export path.

The main goal of this branch is to move a large set of README-listed ONNX operators from partial or custom-only handling into native builtin lowering, while also making the generated LiteRT models materially safer to execute.

In practical terms, this branch:

• broadens builtin ONNX operator coverage in flatbuffer_direct
• adds new validator-driven constrained lowerings for many previously missing ops
• hardens DeformConv so the builtin path no longer aborts LiteRT on invoke()
• expands direct-path runtime and coverage tests accordingly
• updates public package metadata and README support tables to reflect the new baseline

Why this change matters

The flatbuffer_direct backend is most valuable when it can emit TFLite builtins instead of relying on TensorFlow conversion fallback or custom ops. This branch improves that value proposition in two ways:

1. Coverage: many more README-supported ONNX ops now have builtin or builtin-decomposition paths.
2. Reliability: DeformConv now uses a LiteRT-safe lowering strategy for the supported standard case, instead of producing models that could serialize and allocate but still abort at runtime.

Key improvements

1. Large builtin coverage expansion

This branch adds or completes builtin lowering coverage across multiple builder families, including signal, reduction, elementwise, indexing, normalization, pooling, attention, and loss paths.

Notable areas improved include:

• random/window ops
  • Bernoulli, BlackmanWindow, HammingWindow, HannWindow, RandomNormal, RandomUniform, RandomUniformLike
• reduction and numeric ops
  • LeakyRelu, Mean, ReduceLogSum, ReduceLogSumExp, ReduceSumSquare, ThresholdedRelu, IsInf, IsNaN, Shrink
• shape/indexing/spatial ops
  • AffineGrid, CenterCropPad, Compress, ReverseSequence, Scatter, TensorScatter
• normalization / pooling / matrix ops
  • GroupNormalization, LpPool, GlobalLpPool, Det
• attention / signal / embedding / losses
  • Attention, DFT, STFT, MelWeightMatrix, RotaryEmbedding, NegativeLogLikelihoodLoss, SoftmaxCrossEntropyLoss
• ROI / unpool / deformable paths
  • MaxRoiPool, MaxUnpool, DeformConv

These additions are backed by registry-level dispatch/validation work and corresponding builder implementations.

2. DeformConv LiteRT runtime safety

DeformConv received a dedicated runtime hardening pass.

The new builtin lowering is intentionally constrained to the standard 2D float pattern:

• group=1
• offset_group=1
• rank-4 input/output/offset(/mask)
• FLOAT16/FLOAT32
• constant weights and optional constant bias

The previous path could generate a model that converted successfully but aborted the LiteRT interpreter during invoke(). The new path fixes that by:

• removing the GATHER(batchDims=1) dependency from the sampling path
• replacing it with a global-index gather approach
• reducing the high-rank grouped transpose chain
• lowering the sampling path to a LiteRT-safer rank profile

Unsupported grouped DeformConv patterns are still preserved as explicit custom-op candidates when custom ops are enabled and allowlisted.

3. Better registry, reporting, and docs alignment

The branch updates the source of truth for builtin support in op_registry.py, then propagates that state into:

• README builtin support table
• README custom-op candidate policy table
• op coverage tests and snapshots

The README builtin summary count is also refreshed to the current table contents.

4. Version and release metadata update

Package and container-facing version references are updated from 2.3.3 to 2.3.4 to match the functional expansion in this branch.

Testing

The branch includes substantial test expansion for flatbuffer_direct, including:

• direct conversion tests
• LiteRT runtime smoke/parity tests
• coverage report assertions
• fallback/custom-op policy tests

A notable addition is subprocess-isolated runtime verification for DeformConv, so native LiteRT abort regressions fail safely inside tests instead of taking down the full pytest worker.

Test result used for this branch

pytest -q tests/test_tflite_builder_direct.py tests/test_tflite_builder_op_coverage.py

Result:

• 646 passed, 2 warnings in 129.48s

Files of interest

The core changes are concentrated in:

• onnx2tf/tflite_builder/op_registry.py
• onnx2tf/tflite_builder/op_builders/conv.py
• onnx2tf/tflite_builder/op_builders/shape.py
• onnx2tf/tflite_builder/op_builders/reduce.py
• onnx2tf/tflite_builder/op_builders/elementwise.py
• onnx2tf/tflite_builder/op_builders/pool.py
• onnx2tf/tflite_builder/op_builders/index.py
• onnx2tf/tflite_builder/op_builders/norm.py
• onnx2tf/tflite_builder/op_builders/recurrent.py
• tests/test_tflite_builder_direct.py
• tests/test_tflite_builder_op_coverage.py
• README.md

Compatibility notes

This PR intentionally favors explicit constrained builtin support over broad but unsafe lowering.

The clearest example is DeformConv: the supported builtin path is now narrower, but it is meaningfully more correct because it survives actual LiteRT execution. Patterns outside that safe envelope still retain the existing custom-op fallback policy.

What's Changed

• feat: expand flatbuffer_direct builtin coverage and runtime safety by @PINTO0309 in https://github.com/PINTO0309/onnx2tf/pull/904

Full Changelog: https://github.com/PINTO0309/onnx2tf/compare/2.3.3...2.3.4

2.3.3

[!IMPORTANT] Starting with onnx2tf v2.4.0, tf_converter will be deprecated and the default backend will be switched to flatbuffer_direct. With the v2.3.3 update, all backward compatible conversion options have been migrated to flatbuffer_direct, so I will only be doing minor bug fixes until April. If you provide us with ONNX sample models, I will consider incorporating them into flatbuffer_direct.

Summary

This PR improves the flatbuffer_direct backend by adding ONNX-input support for several legacy conversion options that were previously only effective on the tf_converter path, while intentionally keeping input_tflite_file_path direct-import validation unchanged.

The main goal is to close backend behavior gaps without weakening the direct backend's architecture or introducing hidden fallback behavior.

What This PR Adds

1. Legacy option support for flatbuffer_direct ONNX input

This PR wires the following options through the flatbuffer_direct export and lowering path:

• disable_suppression_flextranspose
• disable_suppression_flexstridedslice
• optimization_for_gpu_delegate
• replace_argmax_to_reducemax_and_indices_is_int64
• replace_argmax_to_reducemax_and_indices_is_float32
• replace_argmax_to_fused_argmax_and_indices_is_int64
• replace_argmax_to_fused_argmax_and_indices_is_float32
• fused_argmax_scale_ratio

not_use_onnxsim was already effective for ONNX-input conversion before backend branching, so this PR treats it as an existing capability and adds documentation/tests to make that explicit.

2. Direct lowering support for Flex suppression control

flatbuffer_direct now honors:

• disable_suppression_flextranspose
• disable_suppression_flexstridedslice

Behavioral effect:

• When transpose suppression is disabled, the direct path emits a single builtin TRANSPOSE with the original rank instead of forcing rank compression/decomposition.
• When strided-slice suppression is disabled, the direct path emits a single builtin SLICE/STRIDED_SLICE instead of rank-compressing the operation.
• Existing identity/inverse-transpose fast paths remain intact.
• Existing compression knobs still remain the default path when the disable flags are not set.

3. GPU delegate-oriented rewrites in the direct backend

This PR adds direct-path handling for optimization_for_gpu_delegate in places where the TensorFlow path already had special-case behavior.

Key additions:

• Explicit broadcast materialization helper for direct binary arithmetic lowering.
• Reuse of that helper in broadcast-heavy direct arithmetic paths.
• GPU-delegate-specific normalization path for runtime negative Gather indices.
• GPU-delegate-specific Gemm bias handling for the dynamic BATCH_MATMUL fallback path.
• Direct BatchNormalization broadcast materialization improvements.

This keeps the implementation inside the ModelIR/direct-lowering pipeline rather than introducing any backend fallback.

4. ArgMax replacement modes in flatbuffer_direct

This PR adds direct backend support for the four legacy ArgMax replacement modes.

ReduceMax-based replacements

For:

• replace_argmax_to_reducemax_and_indices_is_int64
• replace_argmax_to_reducemax_and_indices_is_float32

The direct path now lowers ArgMax into a reduction-based subgraph that matches the legacy TensorFlow-path semantics, including:

• reverse-index tie handling
• keepdims support
• final output dtype conversion to int64 or float32

Fused ArgMax replacements

For:

• replace_argmax_to_fused_argmax_and_indices_is_int64
• replace_argmax_to_fused_argmax_and_indices_is_float32

The direct path now supports fused handling for ONNX Resize -> ArgMax 4D patterns.

The implementation:

• identifies the ONNX producer/consumer relationship directly in lowering context
• performs a reduced-resolution ArgMax path for the fused mode
• restores the requested spatial output size with nearest-neighbor resizing
• preserves output shape behavior and requested final dtype semantics

This intentionally does not attempt to add unrelated ScaleAndTranslate support.

Validation / Behavior Boundaries

This PR keeps existing validation philosophy intact.

Specifically:

• input_tflite_file_path still rejects these ONNX-lowering options because there is no ONNX graph stage in direct-import mode.
• Existing mutual-exclusion rules for the four ArgMax replacement options are preserved.
• Existing fused_argmax_scale_ratio validation is preserved.
• No hidden fallback to tf_converter is introduced.

Internal Changes

The implementation includes:

• option plumbing from convert() into export_tflite_model_flatbuffer_direct()
• lowering-context extensions for new direct-backend flags
• ONNX producer/consumer tracking for pattern-aware direct rewrites
• direct builder updates across transpose, slice, gather, gemm, batch normalization, elementwise broadcast, and ArgMax lowering
• README/API documentation updates to clarify ONNX-input vs direct-import support boundaries

Tests

This PR adds and updates tests covering:

• not_use_onnxsim=True on ONNX-input flatbuffer_direct
• transpose rank-compression bypass when suppression is disabled
• slice rank-compression bypass when suppression is disabled
• GPU delegate-specific Gather lowering
• GPU delegate-specific Gemm lowering
• GPU delegate broadcast materialization
• ReduceMax ArgMax replacement paths
• Fused ArgMax replacement paths
• direct-import validation for unsupported ONNX-lowering options

Full flatbuffer_direct-focused test execution was also run:

• tests/test_tflite_builder_direct.py
• tests/test_tflite2sm_phase1.py
• tests/test_flatbuffer_direct_op_error_report.py
• tests/test_model_convert.py

Result:

• 677 passed

Versioning

This branch also includes the package version bump to 2.3.3 and the matching uv.lock update.

Why This Matters

Before this change, flatbuffer_direct had a meaningful feature gap versus the legacy TensorFlow conversion path for several CLI options that users still rely on when debugging backend-specific issues, tuning delegate compatibility, or preserving legacy conversion behavior.

This PR narrows that gap while preserving the direct backend's core principles:

• explicit lowering
• no implicit converter fallback
• clear validation boundaries
• backend-specific behavior made visible in docs and tests

What's Changed

• Add flatbuffer_direct support for legacy conversion options by @PINTO0309 in https://github.com/PINTO0309/onnx2tf/pull/903

Full Changelog: https://github.com/PINTO0309/onnx2tf/compare/2.3.2...2.3.3

2.3.2

Starting with onnx2tf v2.4.0, tf_converter will be deprecated and the default backend will be switched to flatbuffer_direct.

Summary

This PR expands and hardens the flatbuffer_direct backend so that more conversion workflows stay on the direct ModelIR/FlatBuffer path instead of falling back to the legacy tf_converter path.

The branch focuses on improving functional parity, making interrupt/rewrite features work consistently in direct mode, and tightening the documentation/tests around the new behavior.

What changed

1. Keep more workflows on the direct path

flatbuffer_direct now keeps the direct export flow for cases that previously forced a fallback or were rejected:

• --output_h5
• --output_keras_v3
• --output_tfv1_pb
• --disable_model_save
• -it/--input_tflite_file_path direct-import workflows

Instead of dropping to tf_converter, these flows now use a ModelIR-derived SavedModel bridge when a non-TFLite artifact is required.

This means direct mode can now:

• produce .h5, .keras, and TFv1 .pb artifacts without leaving the direct backend
• run with --disable_model_save while still performing internal staging/validation and leaving no final artifacts in the requested output directory
• apply the same behavior to both ONNX input and -it input when appropriate

Validation was also tightened so incompatible combinations are rejected explicitly rather than silently changing execution mode.

2. ModelIR-based interrupt handling for -inimc / -onimc

flatbuffer_direct no longer depends on ONNX graph extraction for interrupt-based cropping.

Instead, it now crops the already lowered/imported top-level ModelIR using the requested boundary tensor names. This applies to:

• ONNX input
• -it/--input_tflite_file_path input

Behavioral impact:

• direct mode no longer relies on sne4onnx for this path
• the cropped model is the basis for downstream direct export, SavedModel bridging, split planning, and evaluation
• boundary names are validated against top-level ModelIR tensors and invalid requests fail explicitly

3. Direct-mode support for -dgc, -ebu, and -eru

Direct mode now supports these options through shared ModelIR rewrites instead of forcing the TensorFlow conversion flow:

• -dgc / --disable_group_convolution
• -ebu / --enable_batchmatmul_unfold
• -eru / --enable_rnn_unroll

The implementation unifies ONNX input and imported TFLite ModelIR handling by applying rewrites after lowering/import and before split planning / SavedModel bridging / final export.

In practice this adds:

• grouped CONV_2D decomposition in direct mode
• BATCH_MATMUL unfolding in direct mode
• direct recurrent unrolling support for the supported RNN/LSTM cases

If a requested rewrite cannot be applied safely, conversion now fails explicitly instead of behaving like a no-op.

4. Direct lowering for MeanVarianceNormalization with -me / --mvn_epsilon

This branch adds direct lowering support for ONNX MeanVarianceNormalization in flatbuffer_direct.

The new lowering expands MVN into builtin-friendly primitive ops:

• MEAN
• SUB
• MUL
• MEAN
• ADD
• SQRT
• DIV

mvn_epsilon is now threaded through the direct backend so -me / --mvn_epsilon affects both:

• tf_converter
• flatbuffer_direct

for ONNX input.

This removes one more feature gap between the two backends while keeping direct export fully self-contained.

5. CLI / README / API cleanup

This branch also updates the public surface and documentation so the new direct-path behavior is visible and consistent:

• add short option -esm for --eval_split_models
• fix broken formatting in the README CLI parameter section
• document the new flatbuffer_direct semantics for:
  • SavedModel-bridge based artifact generation
  • interrupt handling
  • ModelIR rewrites
  • --disable_model_save
  • direct MVN support
• update package/dependency metadata and lockfile sync in line with the branch changes

Why this matters

This work reduces the number of cases where users must understand or work around backend switches.

Before this branch, enabling certain artifact outputs or graph-manipulation options could unexpectedly move execution away from flatbuffer_direct, or reject flows that were conceptually still compatible with direct export.

After this branch, the direct backend is much more coherent:

• direct mode stays direct more often
• ONNX input and -it input behave more consistently
• interrupt/rewrite features are ModelIR-native
• more CLI options have explicit, documented semantics instead of fallback-driven behavior

Testing

I validated the branch with focused and backend-wide tests, including:

• targeted MVN direct-lowering tests
• direct-path interrupt / crop tests
• direct rewrite tests for grouped convolution, batch matmul unfolding, and recurrent unroll
• SavedModel-bridge / artifact-generation direct-path tests
• backend-wide flatbuffer_direct test sweep

Most recent backend-wide run:

pytest -q tests -k flatbuffer_direct

Result:

• 599 passed
• 141 deselected
• 2 warnings

The warnings were non-failing and already known:

• HDF5 legacy save warning from tf_keras
• float16 cast overflow warning in an existing negative-infinity broadcast test

What's Changed

• Improve flatbuffer_direct direct-path coverage and ModelIR rewrites by @PINTO0309 in https://github.com/PINTO0309/onnx2tf/pull/902

Full Changelog: https://github.com/PINTO0309/onnx2tf/compare/2.3.1...2.3.2

2.3.1

Starting with onnx2tf v2.4.0, tf_converter will be deprecated and the default backend will be switched to flatbuffer_direct.

Summary

This PR substantially expands the flatbuffer_direct split workflow and makes the direct path more coherent across export, validation, TFLite input import, and SavedModel output.

The main goal of this branch is to turn split handling into a first-class ModelIR-based feature for flatbuffer_direct, rather than relying on the legacy ONNX/GraphSurgeon split flow.

Main improvements

1. Unified split behavior under enable_auto_split_model

flatbuffer_direct now uses a single ModelIR-based split planner for direct export.

Key behavior changes:

• enable_auto_split_model=True is the public split entry point for flatbuffer_direct.
• Split planning now runs directly against ModelIR and produces the manifest-based artifact set.
• The legacy ONNX recursive split path is no longer used for flatbuffer_direct.
• Small models still succeed and emit a valid single-partition manifest instead of failing or falling back.

This makes split behavior deterministic and consistent with the direct backend’s internal representation.

onnx2tf \
-i deim_hgnetv2_s_wholebody28_ft_1250query_fixed.onnx \
-cotof \
-tb flatbuffer_direct \
-easm \
-asms 5MB # KB or MB or GB

<img width="827" height="554" alt="image" src="https://github.com/user-attachments/assets/22477679-7a70-4225-a0d5-66b132517325" />

<img width="2192" height="888" alt="image" src="https://github.com/user-attachments/assets/d2de788d-47f6-4558-9ca9-c97659c069c4" />

2. Cleaner split artifacts and partition boundaries

The partition builder was tightened so split artifacts are more faithful and easier to inspect.

Improvements include:

• embedded weights/constants are no longer exposed as partition runtime inputs
• dead branch operators are pruned from partition subgraphs
• split manifests now reflect only real runtime / cross-partition inputs
• partition TFLite graphs avoid the misleading “isolated constant/input” appearance that previously showed up in viewers such as Netron

This specifically addresses the issue where split partitions appeared to have many disconnected constants despite being otherwise executable.

3. Split SavedModel export and validation

The direct split pipeline now supports partition-level SavedModel output.

New capabilities:

• flatbuffer_direct + enable_auto_split_model + flatbuffer_direct_output_saved_model exports partition SavedModels
• split SavedModel validation runs partition-by-partition in manifest order and compares against the appropriate TFLite reference
• when split SavedModel output is requested, the workflow is aligned across ONNX-input and TFLite-input direct paths

This closes the gap between split TFLite output and SavedModel output for the direct backend.

4. -it / TFLite-input direct split support

TFLite-import (-it) can now participate in the direct split flow.

That means:

• imported TFLite can be lowered into ModelIR
• the same split planner can be applied to imported TFLite models
• split artifacts and optional split SavedModels can be generated from TFLite input as well

This makes the direct backend’s split functionality available beyond ONNX-originated conversions.

5. Evaluation and validation cleanup

Split evaluation and direct validation behavior were simplified and clarified.

Changes include:

• eval_split_models is now the single split evaluation interface and directly encodes the reference mode (onnx or unsplit_tflite)
• the separate eval_split_reference option was removed
• split evaluation now adapts NHWC partition inputs correctly, fixing false failures caused by input layout mismatches
• -cotof no longer emits a SavedModel inference warning when the workflow did not actually produce a SavedModel
• logs and reports now distinguish unsplit base accuracy evaluation from split-manifest evaluation more clearly

These changes reduce ambiguity in both CLI usage and generated reports.

6. External-data safety for ONNX shape inference

The branch also hardens ONNX handling by enforcing a repository-wide runtime policy:

• onnx.shape_inference.infer_shapes is not run when an ONNX model uses external_data
• the same policy is applied to the shared lowering helper and relevant test helpers
• symbolic fallback inference is also skipped in that case

This avoids unsafe shape-inference calls on external-data models while preserving the existing behavior for regular in-memory ONNX models.

Public-facing behavior changes

Notable interface changes:

• auto_split_tflite_by_size was removed as a public flatbuffer_direct split entry point
• enable_auto_split_model is the public split trigger for flatbuffer_direct
• auto_split_max_size remains the split target-size control
• eval_split_reference was removed
• eval_split_models now takes the reference mode directly (onnx or unsplit_tflite)

These changes intentionally reduce redundant split/evaluation options and make the direct backend easier to operate.

Testing

I ran the flatbuffer-direct-focused regression suite:

pytest -q \
  tests/test_tflite_builder_direct.py \
  tests/test_tflite_builder_op_coverage.py \
  tests/test_flatbuffer_direct_op_error_report.py \
  tests/test_accuracy_evaluator_input_layout.py \
  tests/test_accuracy_evaluator_name_map.py \
  tests/test_accuracy_evaluator_seeded_input.py \
  tests/test_accuracy_evaluator_subprocess.py \
  tests/test_tflite_split_planner.py \
  tests/test_tflite_builder_gridsample_validation.py

Result:

• 625 passed, 1 warning

The remaining warning is an existing float16 cast overflow warning during one direct-path test and does not fail the suite.

Why this PR matters

Before this branch, flatbuffer_direct split support was fragmented across multiple partially overlapping flags and code paths. This branch consolidates the split workflow around the direct backend’s own ModelIR, aligns TFLite and SavedModel outputs, improves artifact correctness, and removes several confusing validation/reporting edge cases.

As a result, feat-split turns split export from an experimental side path into a much more coherent and reviewable feature set for flatbuffer_direct.

What's Changed

• feat: expand flatbuffer_direct split export and validation workflows by @PINTO0309 in https://github.com/PINTO0309/onnx2tf/pull/901

Full Changelog: https://github.com/PINTO0309/onnx2tf/compare/2.3.0...2.3.1

2.3.0

Starting with onnx2tf v2.4.0, tf_converter will be deprecated and the default backend will be switched to flatbuffer_direct.

<img width="1869" height="770" alt="image" src="https://github.com/user-attachments/assets/d984cf61-062f-46f4-8472-3253e220f44c" />

1. Content and background

This PR introduces a complete SavedModel path for the direct LiteRT backend and aligns the CLI/runtime behavior with a no-fallback flatbuffer_direct policy.

Historically, SavedModel export behavior in direct-mode workflows depended on fallback/legacy conversion behavior. This branch makes SavedModel generation explicit and deterministic by using a dedicated ModelIR -> SavedModel exporter and by adding a TFLite -> ModelIR importer for direct TFLite input use cases.

The branch also adds machine-readable validation/reporting and a bulk verification runner to support large-scale model sweeps.

2. Summary of corrections

A. flatbuffer_direct SavedModel export

• Added a new exporter subsystem: onnx2tf/tflite_builder/saved_model_exporter.py.
• Exporter architecture:
  • tf.Module + @tf.function serving signature generation.
  • TensorSpec generation from shape_signature (-1 -> None) and TensorIR dtype mapping.
  • Graph execution via op-kernel dispatch over ModelIR operators.
  • Subgraph-aware execution and control-flow support (including WHILE).
• Added strict support gating:
  • Fail-fast on unsupported op types.
  • Explicit fail-fast on CUSTOM ops with detailed per-node diagnostics.
• Integrated into direct writer path in onnx2tf/tflite_builder/__init__.py and return payload now includes saved_model_path when generated.

B. Remove fallback behavior from direct backend

• Removed --flatbuffer_direct_fallback_to_tf_converter usage from CLI/runtime path.
• Direct path now fails explicitly on unsupported patterns instead of silently switching to TF converter.
• Updated docs and migration guidance to reflect explicit no-fallback operation.

C. New direct TFLite input mode (.tflite -> SavedModel only)

• Added input_tflite_file_path parameter to convert() and new CLI option:
  • -it, --input_tflite_file_path
• Added a new importer: onnx2tf/tflite_builder/tflite_importer.py:
  • Reconstructs ModelIR from TFLite FlatBuffer (main graph + subgraphs).
  • Restores tensor metadata (dtype, shape, shape_signature, is_variable, quant fields, constant data).
  • Restores operator metadata (op_type, version, builtin options normalization, CUSTOM metadata preservation).
  • Handles tensor naming normalization for empty/duplicate names.
  • Prefers serving_default SignatureDef boundary names when available.
• Added early tflite-mode branching/validation in onnx2tf/onnx2tf.py:
  • input_onnx_file_path, input_tflite_file_path, and onnx_graph are mutually exclusive.
  • ONNX-dependent options are rejected in TFLite direct mode.
  • disable_model_save=True is rejected in TFLite direct mode.
  • tflite_backend must be flatbuffer_direct for this mode.

D. SavedModel validation/reporting improvements (-cotof behavior)

• Added SavedModel-vs-TFLite validation path with structured JSON output:
  • <model_name>_saved_model_validation_report.json
  • Includes inference status, comparison pass/fail, matched/total outputs, max abs error, unmatched outputs, and overall pass.
• In TFLite direct mode, -cotof now runs SavedModel inference + comparison against input TFLite.
• In direct ONNX flow, SavedModel validation reporting is integrated so strict checks can be automated.

E. Bulk verification utility for large-scale sweeps

• Added onnx2tf/utils/tflite2sm_bulk_runner.py with CLI entrypoint behavior.
• Implements ordered list execution for model-list files, with:
  • Resume support (bulk_status.json state).
  • Per-model run directories + command/stdout/stderr capture.
  • Classification and strict-pass evaluation using machine-readable SavedModel validation reports.
  • Summary artifacts: bulk_summary.json and bulk_summary.md.
• Includes policy hooks currently used in this branch workflow:
  • model-skip policy for deformable_detr_one_input_simple
  • accepted-mismatch policy for mosaic-9

F. Accuracy evaluator improvement for seeded input generation

• Updated seeded random input distribution heuristic in accuracy_evaluator.py:
  • NCHW-like image tensors default to normal distribution.
  • NHWC-like image tensors keep non-negative uniform default.
• Added/updated tests for distribution behavior and non-finite metric stability.

G. Documentation and version updates

• README updates:
  • Added --flatbuffer_direct_output_saved_model / -fdosm descriptions in CLI and script-option sections.
  • Added --input_tflite_file_path / -it usage and examples.
  • Documented no-fallback behavior and constraints.
  • Added bulk-runner usage and SavedModel validation report details.
• Updated FLATBUFFER_DIRECT_MIGRATION_GUIDE.md for no-fallback migration guidance.
• Version bump to 2.3.0 (onnx2tf/__init__.py, pyproject.toml, uv.lock).

H. Tests

Added/updated test coverage across:

• tests/test_tflite2sm_phase1.py
• tests/test_tflite2sm_bulk_runner.py
• tests/test_accuracy_evaluator_seeded_input.py

Validated in this branch with:

• pytest -q tests/test_tflite2sm_phase1.py tests/test_tflite2sm_bulk_runner.py tests/test_accuracy_evaluator_seeded_input.py
• Result: 37 passed

I. Follow-up improvements in flatbuffer_direct

• Preserved direct-path error semantics in convert() by re-raising NotImplementedError from the flatbuffer_direct fast path (instead of always wrapping as generic RuntimeError).
• Expanded generic 2-input Einsum builtin handling so rank-2 equations that are not pure matmul-style can still lower via builtin ops.
• Kept the optimized matmul-style rank-2 path unchanged/preferred for ij,jk->ik-style equations.
• Confirmed builtin lowering for transposed-output rank-2 Einsum ij,jk->kj (no ONNX_EINSUM custom op).
• Added/updated tests:
  • direct-path test for builtin transposed-output rank-2 Einsum
  • custom-candidate fixtures switched to repeated-label unsupported equation ii,jk->kj
  • op-coverage snapshot extension for the new builtin dispatch case
• Additional validation run:
  • pytest -q tests -k flatbuffer_direct
  • Result: 564 passed, 108 deselected, 1 warning

3. Before/After (If there is an operating log that can be used as a reference)

Before

• Direct backend could depend on fallback behavior for certain flows.
• No dedicated .tflite direct-input mode for SavedModel generation.
• No standardized machine-readable SavedModel-vs-TFLite validation artifact for strict gating.

After

• Direct backend has explicit no-fallback semantics.
• SavedModel can be produced from:
  1. direct ModelIR in flatbuffer_direct flow (-fdosm), and
  2. direct .tflite input mode (-it).
• -cotof can produce strict SavedModel-vs-TFLite validation reports.
• Bulk sweep automation is available with resume + strict classification output.

4. Issue number (only if there is a related issue)

N/A

What's Changed

• feat: add direct SavedModel paths for flatbuffer_direct and TFLite input by @PINTO0309 in https://github.com/PINTO0309/onnx2tf/pull/900

Full Changelog: https://github.com/PINTO0309/onnx2tf/compare/2.2.2...2.3.0

2.2.2

Summary

This PR strengthens flatbuffer_direct conversion quality for DAMO-style graphs and broadens built-in operator coverage while keeping model contracts stable at graph boundaries.

Functional Improvements

• Added robust Float32 export path from FP16-heavy IR:
  • Introduced clone_model_ir_with_float32 to promote FP16 tensors/options into FP32.
  • Added prune_identity_cast_operators and optimize_redundant_transpose_operators to simplify generated graphs before writing TFLite.
  • Applied the same transpose cleanup to FP16 export and preserved reduced-precision metadata handling.
• Improved late-stage dynamic shape correctness:
  • _resolve_dynamic_reshape_shapes now supports prefer_runtime_inferable_from_onnx_raw=True to preserve ONNX runtime-inferable -1 templates instead of stale concretized shapes.
  • Extended dynamic-lineage preservation for RANGE outputs with runtime-dependent lengths.
  • Prevented aggressive reshape passthrough folding when preserveDynamicShape=True is set.
• Added LiteRT compatibility fallback for unsupported SPLIT input dtypes:
  • Rewrites unsupported SPLIT into SLICE chains (with optional cast in/out) so conversion succeeds for dtype combinations not accepted by LiteRT SPLIT.
• Added a new graph optimizer for DAMO-like normalization/padding layouts:
  • _optimize_transpose_flatten_globalnorm_pad_prepost_nhwc_chains removes redundant pre/post transpose wrappers and rewrites affected shape/pad metadata into NHWC-consistent form.

Operator Coverage Extensions

• Inverse:
  • Expanded built-in lowering from only 2x2/3x3 to generic square NxN (up to 16x16) by Gauss-Jordan-style tensor-op decomposition.
• GatherND:
  • Added lowering support for batch_dims > 0 by flattening batch prefix, synthesizing batch indices (RANGE + TILE + CONCAT), gathering, then reshaping back.
• Gemm/MatMul -> FULLY_CONNECTED path:
  • Added explicit I/O cast handling so FP16 interfaces can safely execute with internal FP32 compute.
• Bidirectional LSTM path:
  • Added FP16<->FP32 cast bridges to keep built-in kernel use while preserving external dtype contracts.
• Slice shape inference:
  • Improved static metadata resolution for strided slicing with known bounds; avoids incorrect oversized static lengths when runtime clipping may occur.
• GridSample:
  • Improved handling of partially-unknown shape metadata by merging with ONNX raw shape hints and inferring expected output dimensions when valid.

Validation/Registry Alignment

• Updated validators (Inverse, GatherND, GridSample) to match new lowering capabilities and constraints.
• Expanded op registry declarations for GatherND and shape/index helper ops (RANGE, SHAPE) required by new lowerings.

Tests

• Expanded tests/test_tflite_builder_direct.py with targeted regression tests covering:
  • FP16->FP32 IR promotion and cast/transpose cleanup
  • Inverse static 8x8 lowering
  • GatherND(batch_dims>0) and GridSample unknown-shape scenarios
  • FP16 FC cast bridges
  • unsupported-SPLIT fallback rewriting
  • reshape runtime -1 preference behavior
  • transpose/flatten/globalnorm/pad chain optimization
• Local test result:
  • pytest -q tests/test_tflite_builder_direct.py
  • 568 passed, 1 warning

Release Metadata

• Bumped package version to 2.2.2 (pyproject.toml, onnx2tf/__init__.py).
• Updated Docker tag examples in README to 2.2.2.

What's Changed

• Improve flatbuffer_direct coverage and DAMO conversion robustness by @PINTO0309 in https://github.com/PINTO0309/onnx2tf/pull/899

Full Changelog: https://github.com/PINTO0309/onnx2tf/compare/2.2.1...2.2.2

2.2.1

Summary

This PR delivers a broad flatbuffer_direct upgrade focused on three goals:

1. feature parity with the TF-converter path where practical,
2. safer dtype/shape handling for dynamic graphs,
3. stronger final-stage graph optimization for layout-heavy models.

It includes the two commits in fix-sgsch:

• d20dba8 Optimize redundant transpose chains in flatbuffer_direct
• e7641fa Update flatbuffer_direct converters, metadata handling, and tests

Key Improvements

1) FP16 accumulation metadata parity between TF-converter and flatbuffer_direct

• Added propagation of enable_accumulation_type_float16 into flatbuffer-direct export calls.
• Added model-level metadata plumbing to serialize reduced_precision_support in direct-written TFLite flatbuffers.
• Metadata now correctly reflects:
  • fp16accfp16 when -eatfp16 is enabled,
  • fp16accfp32 otherwise.
• This addresses mismatch cases where *_float16.tflite metadata stayed fp16accfp32 despite -eatfp16.

2) New/expanded builtin op coverage

• Added CumProd builtin lowering path (build_cumprod_op) using a TFLite decomposition:
  • RANGE, LESS/LESS_EQUAL, RESHAPE, TILE, FILL, SELECT_V2, REDUCE_PROD, optional REVERSE_V2.
• Added Unique builtin lowering path (build_unique_op) with validation constraints.
• Added CastLike builtin lowering path (build_castlike_op).
• Added Upsample dispatch entry to reuse Resize lowering/validation path.
• Added UNIQUE options serialization support in model writer (UniqueOptions).

3) Scatter/Index robustness upgrades

• ScatterND:
  • support numeric updates dtype (with casting to data dtype as needed),
  • normalize negative indices to ONNX semantics before scatter,
  • improve dynamic-shape compatibility for shape/signature handling.
• ScatterElements:
  • relaxed/expanded rank handling for indices/updates,
  • runtime shape handling for dynamic updates,
  • improved coordinate assembly for mismatched rank scenarios.

4) Dtype harmonization and 64-bit normalization improvements

• Added consistent INT64 -> INT32 and UINT64 -> UINT32 normalization in critical paths.
• Improved binary op dtype selection for mixed integer/float inputs and comparison ops.
• Added unary/reduce dtype harmonization helpers to prevent runtime mismatches after upstream dtype normalization.
• Improved Where/control-flow mux dtype normalization for signed/unsigned integer outputs.
• Added constant-input CAST folding and redundant INT64->INT32 CAST chain cleanup in post-lowering optimization.

5) Dynamic shape and reshape stability fixes

• Reworked dynamic reshape resolution behavior:
  • preserves runtime-driven newShape=[] intent when required,
  • uses safer template sanitization for 0 / multiple -1 cases,
  • avoids over-resolving shapes that should remain runtime-dynamic,
  • better treatment of onnxRawNewShape with existing concrete newShape.
• Improved replacement logic for EXPAND_DIMS / SQUEEZE to RESHAPE:
  • dynamic cases are preserved where static rewrite would be unsafe,
  • dynamic squeeze rewrite path uses runtime shape plumbing when valid.
• Improved static-shape reconciliation and signature sanitation for edge dynamic-lineage cases.

6) Layout/transpose optimization expansion

Added and integrated multiple strict/final-stage optimization passes in lower_from_onnx2tf.py, including:

• transpose-split mixed pre-concat consolidation,
• channel-shuffle reshape/transpose rewrite to gather,
• transpose-gather-transpose axis-remap cleanup,
• post-transpose InstanceNorm/bias/add chain simplification,
• affine chain folding (mul/add/mul forms),
• split-conv-concat bridge rewrite to single post adapter,
• concat/mul/add/transpose(+add/mean/reshape-tail) bridge reductions,
• additional terminal/final-pass reapplication for late-introduced patterns.

These reduce redundant transposes/casts and stabilize final graph topology after late rewrites.

7) GridSample and pooling support expansion

• GridSample:
  • expanded support to rank-5 border padding path (padding_mode in {zeros, border}),
  • refined indexing/clipping behavior and flattened indexing paths for border/zero modes.
• MaxPool1D:
  • expanded kernel/stride handling constraints (positive kernel/stride with more explicit indices-mode checks).

8) Accuracy evaluation and error-report runtime resilience

• Accuracy evaluator now supports safer dynamic-dimension defaults and layout-aware dynamic dimension inference.
• Added runtime TFLite input resize helper usage in evaluator and direct-op error report path.
• Prevents shape-signature incompatibility issues during validation/inference when model inputs are dynamic.

9) Control-flow handling improvements

• Extended supported If NMS-guard subpattern variant.
• Improved generic If branch mux dtype normalization.
• Strengthened Loop state output remapping/casting logic for dtype/shape consistency.

10) Documentation and release updates

• Updated README builtin coverage/constraints table to reflect current direct-lowering capabilities.
• Clarified that accumulation-type option applies to both tf_converter and flatbuffer_direct paths.
• Version bumped:
  • onnx2tf package version 2.2.0 -> 2.2.1
  • Docker tags in README updated accordingly.

Validation

Executed full direct-builder regression:

• pytest -q tests/test_tflite_builder_direct.py
• Result: 554 passed, 1 warning, 0 failures.

Also added/updated extensive tests for:

• CumProd builtin dispatch,
• Unique builtin lowering,
• ScatterND negative-index normalization and dtype casting,
• Upsample opset9 / Pad opset9 compatibility,
• dynamic reshape/signature preservation,
• transpose/layout optimization rewrites,
• integer dtype normalization in control-flow and elementwise paths,
• GridSample rank-5 border validation.

What's Changed

• flatbuffer_direct: add CumProd/Unique/CastLike, fp16 metadata parity, and shape/dtype optimizations by @PINTO0309 in https://github.com/PINTO0309/onnx2tf/pull/898

Full Changelog: https://github.com/PINTO0309/onnx2tf/compare/2.2.0...2.2.1

2.2.0

Summary

This PR consolidates the fix-rtmdet branch improvements focused on functional reliability, direct FlatBuffer export robustness, and conversion/typing stability across the ONNX->TFLite pipeline.

Compared to main, this branch contains 3 commits and broad updates across operator converters, the flatbuffer_direct backend, tests, and tooling.

Functional Improvements (Feature-level)

1. Stronger flatbuffer_direct lowering correctness

• Improved dynamic-shape handling for shape-sensitive graph patterns (notably in reshape/flatten/pad/reduce paths).
• Fixed edge cases where lowering metadata and runtime shape tensors could diverge.
• Tightened reduce-axis behavior so non-constant axes are rejected early in dispatch validation when required by flatbuffer_direct semantics.
• Stabilized dynamic PAD lowering flow by normalizing pads inputs through deterministic INT32 casting before reshape/transpose composition.

Impact:

• More predictable lowering behavior for dynamic ONNX graphs.
• Fewer latent runtime mismatches between static IR metadata and runtime tensor semantics.

2. Expanded and hardened operator conversion logic

• Updated many ONNX op converters in onnx2tf/ops/* to improve type/shape safety and runtime consistency.
• Replaced fragile tensor indexing/operator assumptions in multiple places with safer conversion patterns for dynamic tensors.
• Strengthened handling of optional values, dynamic dtypes, and tensor-vs-scalar branches.

Impact:

• Improved conversion success rate on real-world models (including RTMDet-related conversion paths).
• Reduced failure modes caused by ambiguous shape/dtype inference.

3. flatbuffer_direct coverage and validation alignment

• Aligned op coverage reporting and validation outcomes with actual builder constraints.
• Ensured unsupported reasons are surfaced consistently (for example, requires_constant_input for dynamic reduce axes where applicable).

Impact:

• Coverage reports now better reflect true exportability constraints.
• Debugging unsupported nodes is more actionable.

4. Benchmarking and developer tooling improvements

• Added benchmark_tflite.py for CPU/CUDA-oriented TFLite latency measurement workflows.
• Minor CLI/package/docs alignment updates (onnx2tf.py, __init__.py, README.md, etc.).

Impact:

• Easier performance characterization for converted artifacts.
• Better operational feedback loop when iterating on model conversion quality/performance.

Test and Quality Improvements

Added/updated direct-backend tests

• Extended tests/test_tflite_builder_direct.py with additional scenarios around dynamic pad/flatten/reshape behavior and related shape-signature expectations.

Validation status

Executed targeted flatbuffer_direct suites after fixes:

python -m pytest -q \
  tests/test_tflite_builder_direct.py \
  tests/test_tflite_builder_preprocess.py \
  tests/test_tflite_builder_op_coverage.py \
  tests/test_tflite_builder_gridsample_validation.py \
  tests/test_flatbuffer_direct_op_error_report.py \
  tests/test_tflite_split_planner.py

Result:

• 554 passed, 1 warning

Scope of Changes

• Large converter hardening across onnx2tf/ops/*.
• Core flatbuffer_direct updates in:
  • onnx2tf/tflite_builder/op_builders/shape.py
  • onnx2tf/tflite_builder/op_registry.py
  • additional related builder/utility modules.
• Benchmark tooling and docs updates.

Why this matters for fix-rtmdet

RTMDet export paths heavily depend on stable handling of dynamic tensor metadata, reshape/pad/reduce behavior, and consistent converter typing assumptions. This branch improves those exact stability points, reducing conversion-time ambiguity and increasing confidence in direct TFLite generation paths.

• Unable to convert RTMDet-ins to tflite #761

What's Changed

• Improve flatbuffer_direct reliability and converter robustness for RTMDet workflows by @PINTO0309 in https://github.com/PINTO0309/onnx2tf/pull/897

Full Changelog: https://github.com/PINTO0309/onnx2tf/compare/2.1.5...2.2.0

2.1.5

1. Content and background

This PR expands built-in coverage in flatbuffer_direct and improves stability for models with control flow and dynamic shapes.

In particular, it removes the ONNX_SLICE CUSTOM fallback that occurred while converting encoder-epoch-99-avg-1.onnx, allowing conversion to complete using built-in operators only.

It also includes version updates for the 2.1.5 release (pyproject.toml, onnx2tf/__init__.py, and Docker tags in README.md).

2. Summary of corrections

• Added generic branch-mux support for If in flatbuffer_direct.
• Added a safe branch-op subset for generic If branch lowering (arithmetic, comparison, shape ops, Slice, Conv, LSTM, etc.).
• Added branch-local value_info hint propagation to improve shape/dtype recovery.
• Added built-in lowering for ONNX Size (SHAPE + REDUCE_PROD (+ CAST)).
• Improved _NodeWrap and branch wrapping to preserve empty optional inputs ("").
• Extended Slice support for dynamic starts with constant ends/axes/steps on single-axis patterns.
• Fixed Slice starts/ends length checks so dynamic-start cases are validated correctly.
• Improved GatherElements handling for unknown-rank placeholders (shape=[1], signature<0) in both validator and builder.
• Added fallback rank/shape handling in Expand and Unsqueeze for unresolved placeholder ranks.
• Improved LSTM initial state (initial_h/initial_c) validation to be shape-signature aware.
• Added 3D reshape fallback for rank-missing LSTM state inputs.
• Relaxed some pre-validation paths for Reduce ops and delegated resolution to builder-side logic for dynamic-rank control-flow graphs.
• Added and updated regression tests in test_tflite_builder_direct.py for dynamic Slice, unknown-rank GatherElements, unknown-rank Unsqueeze, generic If branch mux, Size, and related paths.

3. Before/After (If there is an operating log that can be used as a reference)

Before:

• onnx2tf -i encoder-epoch-99-avg-1.onnx -cotof -tb flatbuffer_direct -kat x x_lens
• Built-in validation failed around Slice_699.
• Auto custom-op retry was triggered and ONNX_SLICE CUSTOM ops were introduced.

After:

• The same command completes through the built-in flatbuffer_direct path.
• Auto custom-op retry is not triggered.
• No ONNX_SLICE CUSTOM op is introduced.
• With allow_custom_ops=False, lower_onnx_to_ir completes with CUSTOM op count = 0.

Validation:

• pytest -q tests/test_tflite_builder_direct.py
• Result: 499 passed, 1 warning

4. Issue number (only if there is a related issue)

• vad model conversion shows the same error as previous fixed #782
• Unable to convert zipformer to tflite #785

What's Changed

• Release v2.1.5: expand flatbuffer_direct builtin coverage and control-flow lowering by @PINTO0309 in https://github.com/PINTO0309/onnx2tf/pull/896

Full Changelog: https://github.com/PINTO0309/onnx2tf/compare/2.1.4...2.1.5

2.1.4

Summary

Currently, flatbuffer_direct has better conversion stability than tf_converter mode.

This PR is a broad flatbuffer_direct upgrade focused on three goals:

• expand built-in lowering coverage,
• improve dynamic-shape correctness,
• harden optimization/runtime stability on large real-world models.

Functional Improvements

1. Built-in op coverage expansion

• Einsum

  • Added a specialized built-in lowering path for bchw,bnc->bnhw.
  • Added a generic 2-input Einsum decomposition path (for non-repeated-label equations) via built-in ops (SUM, TRANSPOSE, RESHAPE, BATCH_MATMUL, CAST).
  • Extended validation so unsupported equations fail early with clearer diagnostics.

• MatMul

  • Added low-rank built-in handling for vector-lhs, vector-dot, and scalar-multiply forms.
  • Improved behavior when metadata incorrectly collapses dynamic tensors to scalar-like shapes.

• Pooling

  • Added AveragePool rank-3 (1D) built-in lowering (EXPAND_DIMS -> AVERAGE_POOL_2D -> SQUEEZE).
  • Added rank-5 (3D) MaxPool/AveragePool lowering through staged reshape/transpose decomposition.

• ConvTranspose

  • Added grouped 2D ConvTranspose lowering using split-per-group + per-group transpose-conv + concat.
  • Added safer dilation and channel consistency handling.

• TopK

  • When k is scalar-constant, it is now constantized directly to INT32 for TOPK_V2.
  • Eliminates unnecessary constant-case runtime prep chains such as CAST -> SQUEEZE.

2. Dynamic-shape correctness improvements

• Reduce axis robustness

  • Improved rank resolution from signatures/metadata in dynamic paths.
  • Preserves raw negative axes (e.g. -1) when rank metadata is unreliable, preventing incorrect axis remapping.

• Reshape allowzero semantics

  • Added runtime rewrite for dynamic shape tensors so ONNX allowzero=0 dim-copy semantics are preserved in TFLite lowering.

• Reshape-chain safety

  • Prevented unsafe bypass of reshapes when shape vectors depend on input dimensions (0/-1 behavior).

3. Optimization and runtime stability hardening

• Added conservative guards and rollback logic in aggressive transpose/fanout rewrites to avoid introducing unbound runtime tensors.
• Extended unbound-input repair to include SHAPE consumers (not only reshape-related patterns).
• Improved channelwise constant layout repair to support rank-3 constants and apply rotation only when broadcast semantics stay correct.

4. Memmap/path safety improvements

• Added deterministic sanitize+truncate+hash filename generation to prevent path-component overflow (File name too long) in:
  • accuracy evaluator memmap outputs,
  • ORT output memmap files,
  • flatbuffer_direct op-error report input dumps.

5. Automatic scalar-Gather index normalization (without -rtpo)

• Added automatic normalization for Gather indices when the positions tensor is scalar-like:
  • constant scalar indices,
  • runtime scalar indices (metadata-normalized scalar signatures).
• The lowering now ensures indices are handled as 1D shape [1] for runtime compatibility, while preserving ONNX output-rank semantics by inserting a final shape restore where needed.

6. Version/docs alignment

• Bumped package version to 2.1.4.
• Updated README Docker image tags to 2.1.4.

Validation

• Full flatbuffer_direct direct-lowering test suite:

  • pytest -q tests/test_tflite_builder_direct.py
  • Result: 492 passed, 1 warning.

• Added/updated targeted regression tests, including:

  • AveragePool1D dynamic built-in lowering,
  • new Einsum built-in paths,
  • TopK constant-k constantization,
  • unbound SHAPE input repair,
  • channelwise broadcast-constant layout repair.

Expected Impact

• Fewer CUSTOM fallbacks for common patterns.
• Better numerical/shape consistency in dynamic models.
• Improved robustness against optimizer-induced invalid graph states.
• More stable large-model evaluation flows with safer memmap file handling.

What's Changed

• flatbuffer_direct: expand built-in lowering and harden dynamic-shape/runtime stability by @PINTO0309 in https://github.com/PINTO0309/onnx2tf/pull/895

Full Changelog: https://github.com/PINTO0309/onnx2tf/compare/2.1.3...2.1.4

2.1.3

Summary

This PR focuses on functional improvements in the flatbuffer_direct backend: broader builtin-op conversion coverage, stronger layout optimization (transpose elimination), and more stable evaluation/reporting behavior.

The goal is to reduce CUSTOM-op fallbacks, improve conversion fidelity on real models, and keep output checks reliable.

Functional Improvements

1) Builtin-op coverage expansion (CUSTOM fallback reduction)

This PR adds and stabilizes several builtin conversion paths so models that previously required CUSTOM lowering can stay on builtin TFLite operators.

Highlights:

• Einsum lowering improvements for practical attention-style patterns, with builtin decomposition paths.
• Variadic Sum support via builtin ADD chaining.
• Improved lowering behavior around complex conv/fused-conv paths and related shape/materialization logic, reducing unsupported cases and improving robustness for dynamic/placeholder shape scenarios.
• Additional operator-registry and validator coverage so more models can remain on direct builtin lowering.

2) 3D GridSample support improvements

• Added support/validation for rank-5 (3D) GridSample patterns.
• Tightened validation behavior for acceptable/rejectable grid formats.

3) Aggressive redundant transpose cleanup (major runtime/size win)

• Extended NHWC propagation and rewrite logic for difficult pre-concat subgraphs.
• Added handling for deeper Split/Add cascade patterns and nested add-chain cases.
• Eliminates transpose islands that were previously left in place in real workloads (including the best-352 style chains identified during debugging).

Net effect:

• fewer unnecessary TRANSPOSE ops,
• simpler graphs,
• better chance of preserving accuracy and runtime efficiency.

4) Evaluation / OP-error helper stability

• Improved flatbuffer_direct evaluation-helper behavior for edge cases in temporary input handling and intermediate comparison flow.
• Improved reliability of op-error reporting flow in scenarios that previously produced unstable helper-side failures.

5) Behavior/docs alignment for -dgc

• Documentation now reflects actual behavior: --disable_group_convolution is applied in both tf_converter and flatbuffer_direct flows.

What this means for users

• More models convert with builtin ops only (or fewer CUSTOM fallbacks).
• Better parity and stability when running -cotof -tb flatbuffer_direct.
• Cleaner TFLite graphs due to stronger transpose-chain elimination.
• More predictable debugging and op-level error analysis.

Validation

• Added/expanded targeted tests for:
  • builtin-lowering behavior (including Einsum/Sum-related paths),
  • 3D GridSample validation,
  • transpose-chain optimization regressions,
  • op-error helper/reporting edge cases.
• Local verification was performed on representative models used during regression debugging (including best-352) and output checks remained passing after optimization.

What's Changed

• Improve flatbuffer_direct lowering, built-in coverage, and transpose cleanup by @PINTO0309 in https://github.com/PINTO0309/onnx2tf/pull/894

Full Changelog: https://github.com/PINTO0309/onnx2tf/compare/2.1.2...2.1.3

2.1.2

1. Content and background

In the flatbuffer_direct path, we had recurring issues in practical use:

• Some ops such as TopK and MultiHeadAttention were not lowered as built-ins and ended up as custom ops, which made compatibility and accuracy validation unstable.
• In some models, broken shape_signature propagation and side effects from layout optimizations caused RESHAPE prepare errors and intermediate tensor comparison failures.
• During -cotof validation, memory usage could spike heavily, leading to SWAP thrashing and near-freeze behavior.
• Progress of the OP error helper was hard to observe, so long runs could look stalled.
• Some ORT contrib ops that should be treated as com.microsoft domain arrived as default domain, reducing runtime compatibility.

This PR addresses these issues together to improve built-in coverage, stability, and memory behavior in flatbuffer_direct.

2. Summary of corrections

1. Expanded built-in lowering coverage

   • Extended built-in lowering for TopK.
   • Implemented both largest=1 (descending) and largest=0 (ascending-equivalent) using TOPK_V2 plus auxiliary ops.
   • Added an explicit warning for sorted=0: TFLite always returns descending-sorted output, so index order may not exactly match ONNX reference.
   • Added built-in lowering for MultiHeadAttention with strict validation (3-input Q/K/V form, rank-3, FLOAT16/FLOAT32, num_heads constraints, etc.).
   • Extended lowering for MatMul vector-rhs and dynamic Slice.
   • Fixed DepthToSpace CRD mode lowering in built-in form (channel reordering + DEPTH_TO_SPACE).

2. Stabilized shape_signature and layout optimization behavior

   • Reinforced shape/signature propagation in index.py, shape.py, shared.py, and model_writer.py.
   • Reduced shape_signature inconsistency risk around TopK, Slice, and Reshape.
   • Hardened transpose optimizations in lower_from_onnx2tf.py.
   • Prevented unsafe Transpose -> Reshape rewrites in axis-semantic cases (e.g., rank-3), avoiding meaning drift in LogSoftmax/Transpose/Add paths.

3. Strengthened low-memory validation flow (memmap)

   • Enabled ONNX dummy-inference output memmap path by default (can be disabled via --disable_onnxruntime_output_memmap).
   • Added --onnxruntime_output_memmap_dir to control memmap storage location.
   • Updated accuracy_evaluator to pass worker inputs/outputs via memmap, reducing large IPC tensor copies.
   • Added fallback to in-memory outputs when ONNX output memmap is not possible due to dynamic output shapes.

4. Temp directory management and cleanup on abnormal termination

   • Added onnx2tf/utils/tempdir_cleanup.py.
   • Added managed tempdir cleanup with atexit and signal handlers (SIGTERM/SIGINT/SIGHUP/SIGQUIT).
   • Added stale tempdir cleanup using owner PID markers.

5. Improved OP error helper operability

   • Added a single-line spinner (| / - \) during helper execution.
   • Replaced timeout-centric behavior with clearer classification/logging for crash, prepare failure, and generic failure.
   • Unified seeded input generation in helper path for better reproducibility.

6. Improved ORT compatibility (Microsoft domain)

   • Added utility to supplement com.microsoft domain for selected contrib ops (including MultiHeadAttention) during conversion.
   • Applied temporary rewrites for runtime-check paths and added informative logs when applied.

7. Added standalone low-memory inference script

   • Added lowmem_test_infer.py.
   • Supports ONNX/TFLite low-memory random inference in isolated worker processes (single backend or both).
   • Intended for reproducible, low-memory test-inference workflows.

8. Test expansion

   • Added/updated regression coverage mainly in tests/test_tflite_builder_direct.py.
   • Representative local tests that passed:
   • test_flatbuffer_direct_min_topk_dynamic_k_lowering
   • test_flatbuffer_direct_matmul_vector_rhs_lowering
   • test_flatbuffer_direct_slice_dynamic_end_prefix_rank2_lowering
   • test_flatbuffer_direct_slice_dynamic_start_end_single_axis_lowering
   • test_flatbuffer_direct_logsoftmax_lowering

3. Before/After (If there is an operating log that can be used as a reference)

• Before (examples)

  • Custom ops lowered: ... op_types=[ONNX_TOPK] ...
  • Custom ops lowered: ... op_types=[ONNX_MULTIHEADATTENTION] ...
  • tflite/kernels/reshape.cc:94 num_input_elements != num_output_elements ...
  • OP error report generation was skipped ... helper process timed out

• After (behavior with this PR)

  • TopK and MultiHeadAttention now have built-in lowering paths.
  • TopK(sorted=0) now emits an explicit warning for potential index-order mismatch.
  • OP error helper shows live spinner progress.
  • Dynamic-shape cases that cannot use memmap safely fall back to in-memory path.
  • Tempdir lifecycle and cleanup behavior are strengthened for both normal and signal-exit paths.

4. Issue number (only if there is a related issue)

N/A

What's Changed

• flatbuffer_direct: expand builtin op coverage and stabilize low-memory evaluation paths by @PINTO0309 in https://github.com/PINTO0309/onnx2tf/pull/893

Full Changelog: https://github.com/PINTO0309/onnx2tf/compare/2.1.1...2.1.2

2.1.1

Summary

• Enhanced flatbuffer_direct graph rewrite passes to remove redundant transpose/reshape chains across a wide set of real-model patterns.
• Extended attention optimization coverage from QKV-specific paths to additional KV chain patterns.
• Added builtin lowering for ONNX ConvInteger in flatbuffer_direct (with validator/registry wiring), reducing unresolved custom-op failures.
• Added/updated regression tests in tests/test_tflite_builder_direct.py for new rewrite and lowering patterns.
• Bumped package/container version references to 2.1.1 (pyproject.toml, onnx2tf/__init__.py, README.md).

Why

• Multiple reported models still emitted redundant transpose chains and rewrite regressions.
• Some quantized ONNX models required ConvInteger; previously this path could fail as unresolved custom op.

Validation

• pytest -q tests/test_tflite_builder_direct.py
  • Result: 451 passed, 1 warning

Notes

• This PR is focused on flatbuffer_direct behavior/coverage and associated regression hardening.

What's Changed

• flatbuffer_direct: optimize transpose/QKV rewrites and add ConvInteger builtin by @PINTO0309 in https://github.com/PINTO0309/onnx2tf/pull/892

Full Changelog: https://github.com/PINTO0309/onnx2tf/compare/2.1.0...2.1.1

2.1.0

Summary

This PR significantly accelerates the flatbuffer_direct conversion path and improves operational visibility/stability.

Speed improvements (main point)

• Introduced a direct fast path for --tflite_backend flatbuffer_direct that avoids per-node TensorFlow graph building (op.make_node() loop).
• Reworked FlatBuffer serialization and removed the Python per-byte packing bottleneck.
• Replaced byte-by-byte PrependByte loops with Builder.CreateByteVector fast-path for large constant buffers.

Measured example (*_float32.tflite, same model):

• tf_converter: ~24.947s
• flatbuffer_direct: ~0.239s
• approximately 107x faster in this case

(Actual speedup varies by model/options/environment.)

https://github.com/user-attachments/assets/846f9c57-93ac-43a0-8cc2-9c989923c320

What changed

1. flatbuffer_direct fast path in convert()

   • Added early direct export route when TF-dependent outputs are not requested.
   • Added explicit blocker checks (output_h5, output_keras_v3, output_tfv1_pb, auto-split-model, disable_model_save).
   • Added controlled fallback to TF converter path when --flatbuffer_direct_fallback_to_tf_converter is enabled.

2. Removed runtime dependency on flatc for onnx2tf conversion flows (including -coion).

   • Bundled schema artifacts are now used:
     • onnx2tf/tflite_builder/schema/schema.fbs
     • onnx2tf/tflite_builder/schema/schema_generated.py
   • Updated CI/Docker/README accordingly to reflect non-flatc normal operation.

3. FlatBuffer writer performance overhaul

   • Added builder-direct serialization path in tflite_builder/model_writer.py.
   • Kept object-pack route as fallback; selectable via env:
     • ONNX2TF_FLATBUFFER_DIRECT_SERIALIZER
     • ONNX2TF_FLATBUFFER_DIRECT_SERIALIZER_FALLBACK_TO_OBJECT_PACK
   • Switched model IR sanitization from deep-copy to lightweight container copy to avoid large buffer duplication.
   • Added write/serialize timing breakdown and serializer mode reporting.

4. Progress and timing visibility

   • Added staged tqdm progress for direct lowering/export with dynamic_ncols=True.
   • Added detailed timing line:
     • total / serialize / sanitize / build / pack / output / write / size / mode

5. Report generation control

   • OP error report/CSV generation now runs only when -cotof (--check_onnx_tf_outputs_elementwise_close_full) is enabled.
   • Avoids expensive report generation in standard runs.

6. Lowering/operator coverage and robustness

   • Added Col2Im support in flatbuffer_direct dispatch (validation + builder path).
   • Improved Conv1D pad selection for static-output consistency.
   • Improved Pow handling (exact square path via MUL for reduced drift).
   • Added and hardened several transpose/layout optimization passes and guards.

7. Tests and docs

   • Expanded tests/test_tflite_builder_direct.py with new optimization test cases.
   • README updated with:
     • fast-path behavior,
     • fallback behavior,
     • performance claim (~107x case),
     • operational differences and notes.
   • Version bumped to 2.1.0; added tqdm dependency.

Validation

• pytest -q tests/test_tflite_builder_direct.py
  • 381 passed, 1 warning

Notes

• This PR prioritizes conversion-time acceleration and debuggability.
• Inference-time kernels are unchanged; runtime performance on edge devices is expected to remain equivalent for equivalent graphs.

What's Changed

• feat(flatbuffer_direct): fast path and 100x+ serialization speedup by @PINTO0309 in https://github.com/PINTO0309/onnx2tf/pull/891

Full Changelog: https://github.com/PINTO0309/onnx2tf/compare/2.0.27...2.1.0

2.0.27

Summary

This PR improves fix-yolo conversion quality and stability, with a focus on flatbuffer_direct parity and layout-optimization coverage.

1) Inverse handling and Flex fallback control

• Added Inverse builtin lowering in flatbuffer_direct (SLICE/MUL/SUB/ADD/NEG/CONCATENATION/DIV) for square 2x2/3x3 float matrices.
• Updated runtime/registry validation to gate unsupported shapes and dtypes clearly.
• Updated TF-side Inverse behavior to pseudo-lower by default (to avoid FlexMatrixInverse emission) and only keep tf.linalg.inv when -rtpo Inverse is explicitly set.
• README/CLI docs updated accordingly.

2) Microsoft domain and runtime-check robustness

• Expanded temporary microsoft-domain supplementation targets used in runtime checks / dummy inference to include additional contrib ops (GroupNorm, Inverse) so ORT graph validation remains consistent after graph rewrites.

3) Accuracy evaluator input-name mapping generalization

• Improved TFLite input/detail mapping to absorb SignatureDef-style prefixes/suffixes (including serving_default_* and generic suffix matching), reducing false input-shape mismatches during accuracy checks.
• Added dedicated unit test coverage for the name-map behavior.

4) flatbuffer_direct layout optimization improvements

• Added new fanout rewrite passes for transpose-heavy residual patterns:
  • pre-ADD fanout chains (MUL(const)->ADD(const)->post-TRANSPOSE)
  • pre-UNARY fanout chains (UNARY->MUL(const)->ADD(const)->post-TRANSPOSE)
• Added direct NCHW fallback bridging where applicable and hardened guard conditions to avoid partial graph mutation.
• Fixed bridge perm-constant collision by using per-bridge dedicated perm tensors.
• Result: residual transpose wrappers around intermediate branches are significantly reduced (including the reported heatmap-adjacent case).

5) Other flatbuffer_direct improvements

• MaxPool with indices: added constrained builtin lowering path and validation.
• Conv builder: added explicit compute-cast handling to stabilize dtype paths.
• Slice handling: fixed axes-based begin/end/strides expansion and negative-axis normalization.
• OP error report execution moved to subprocess-based invocation with clearer failure handling.

Validation

Executed flatbuffer_direct full test target:

uv run --with pytest pytest tests -k flatbuffer_direct -ra

Result:

• 373 passed
• 69 deselected
• 1 warning

Changed Areas

• Core: onnx2tf.py, ops/Inverse.py, ops/Slice.py, utils/common_functions.py
• Flatbuffer Direct: tflite_builder/lower_from_onnx2tf.py, op_registry.py, op_builders/*, accuracy_evaluator.py, preprocess rules
• Docs/Version: README.md, __init__.py, pyproject.toml, uv.lock
• Tests: tests/test_tflite_builder_direct.py, tests/test_tflite_builder_preprocess.py, tests/test_microsoft_domain_supplement.py, tests/test_accuracy_evaluator_name_map.py

What's Changed

• feat(flatbuffer_direct): improve yolo conversion stability and transpose elimination by @PINTO0309 in https://github.com/PINTO0309/onnx2tf/pull/890

Full Changelog: https://github.com/PINTO0309/onnx2tf/compare/2.0.26...2.0.27

2.0.26

Summary

This PR improves flatbuffer_direct conversion stability and builtin coverage, with a focus on string/optional handling and redundant layout-wrapper reduction.

Improvements in fix-str

• Added builtin lowering support for StringNormalizer under constrained runtime patterns.
  • Added no-op and stopword-filter paths for string tensors.
  • Added constant-input fold path including case conversion and stopword filtering semantics.
• Added builtin lowering support for determinable OptionalHasElement cases to reduce custom-op fallback.
• Extended Unsqueeze builtin axis normalization with output-rank semantics (covers opset8-style patterns).
• Expanded Pad builtin path to support non-zero constant padding via PADV2 (for non-quantized tensors).
• Improved dtype support in flatbuffer_direct tensor metadata handling (including STRING).
• Added/extended transpose and reshape optimization passes to remove redundant wrappers:
  • Transpose -> Reshape -> Transpose (including NHWC/NCHW channel-tail singleton variants)
  • redundant Squeeze -> Reshape identity chains
  • multiple SE/BiFPN-style layout wrapper cleanup patterns
• Improved op error-report/evaluation behavior around subprocess/in-process execution paths.

Validation

• pytest -q tests/test_tflite_builder_direct.py -k "string_normalizer or optional_has_element or unsqueeze or pad or transpose_se_conv_mul_prepost_nhwc_chain or squeeze_reshape_identity_chains_removed or transpose_reshape_transpose_to_expanddims_nhwc"
  • 28 passed
• pytest -q tests/test_accuracy_evaluator_subprocess.py
  • 2 passed

Notes

• README builtin-op coverage table and notes were updated to document new constraints/coverage.

What's Changed

• flatbuffer_direct: improve String/Optional builtins and transpose-reshape chain optimization by @PINTO0309 in https://github.com/PINTO0309/onnx2tf/pull/889

Full Changelog: https://github.com/PINTO0309/onnx2tf/compare/2.0.25...2.0.26

2.0.25

Summary

This PR improves flatbuffer_direct conversion reliability and coverage, with a focus on control-flow lowering and shape/runtime correctness.

What changed

• Added built-in control-flow lowering for ONNX Loop and improved branch handling for If in flatbuffer_direct.
• Registered and wired new control-flow builders across IR/model-writer/registry paths.
• Hardened runtime-check graph preparation in onnx2tf.py (preserving graph metadata such as domain/IR version) and improved eval fallback behavior.
• Fixed TF path random-like ops (RandomNormalLike, RandomUniformLike) to avoid None shape conversion failures.
• Added shape/permutation robustness fixes in several builders (Concat, ConvTranspose, Slice, control-flow related lowering paths).
• Added/updated direct-lowering tests, including loop-focused cases and transpose-optimization-related assertions.
• Updated README and version metadata (2.0.25).

Why

Recent conversion failures and regressions were observed around:

• unresolved ONNX_LOOP custom-op fallback,
• control-flow lowering edge cases,
• runtime-check invalid-graph handling,
• shape mismatch/transpose redundancy around direct-lowered graphs.

This PR addresses those issues by expanding built-in lowering coverage and tightening shape/runtime validation paths.

Validation

• pytest -q tests/test_tflite_builder_direct.py -k "loop_"
  • Result: 4 passed.

Notes

• This change is intentionally focused on flatbuffer_direct behavior and diagnostics, and keeps compatibility with existing conversion paths.

What's Changed

• flatbuffer_direct: add built-in Loop/If lowering and runtime-check hardening by @PINTO0309 in https://github.com/PINTO0309/onnx2tf/pull/888

Full Changelog: https://github.com/PINTO0309/onnx2tf/compare/2.0.24...2.0.25

2.0.24

Summary

This PR improves flatbuffer_direct conversion quality and operator coverage, with a strong focus on removing redundant transpose chains and reducing custom-op fallbacks.

What Changed

• Added a dedicated control-flow builder module for constrained If lowering:
  • NMS-guard pattern
  • axis0 Add-branch pattern
  • SequenceConstruct Add-branch pattern
  • nested ReduceMin/Add pattern
• Expanded built-in dispatch/validation coverage in flatbuffer_direct, including:
  • Log, Max, ReduceMin, ScatterElements, RoiAlign, LayerNormalization
  • improved If validation/dispatch rules for constrained patterns
• Added/extended non-max suppression handling:
  • support selecting NON_MAX_SUPPRESSION_V4 or V5 via --switch_nms_version (-snms)
• Introduced many transpose-chain optimization passes in lower_from_onnx2tf.py to remove redundant pre/post layout adapters and preserve required boundary adapters.
• Improved evaluation/reporting robustness:
  • safer handling for non-finite values in accuracy metrics/report generation paths.
• Updated README to reflect newly supported built-in patterns and current flatbuffer_direct behavior.

Tests

Executed:

• pytest -q tests/test_tflite_builder_direct.py tests/test_tflite_builder_op_coverage.py

Result:

• 351 passed, 1 warning

Warning observed:

• RuntimeWarning: overflow encountered in cast in float16 cast path during test_flatbuffer_direct_where_neg_inf_broadcast_no_nan.

What's Changed

• Improve flatbuffer_direct builtins and transpose optimizations by @PINTO0309 in https://github.com/PINTO0309/onnx2tf/pull/887

Full Changelog: https://github.com/PINTO0309/onnx2tf/compare/2.0.23...2.0.24

2.0.23

Summary

This PR brings the latest fix-var branch improvements into main and focuses on making the flatbuffer_direct path more practical for real ONNX models.

Key improvements

• Expanded builtin lowering coverage for multiple previously custom/failing paths, including additional pooling/reduction and recurrent/convolution-related cases.
• Added and refined operator builders/validators in the direct TFLite builder pipeline (op_registry, op_builders, and lowering passes).
• Improved transpose/layout optimization passes to remove redundant adapter chains and reduce graph bloat.
• Improved numerical consistency in several edge cases (including Softmax axis handling and shape/broadcast-sensitive paths).
• Added GlobalMaxPool builtin lowering (REDUCE_MAX) for flatbuffer_direct.
• Updated accuracy/coverage-related handling and added regression tests.
• Updated README documentation to reflect newly builtin-covered ops and behavioral changes.

Validation

• pytest -q tests/test_tflite_builder_direct.py (full suite): 307 passed
• Additional targeted conversion checks were performed during development for flatbuffer_direct models using --report_op_coverage.

Notes

• This PR is intentionally broad because fix-var accumulated tightly related direct-backend changes that were validated together.
• The changes are centered on flatbuffer_direct behavior and test/documentation alignment.

What's Changed

• Improve flatbuffer_direct builtin coverage and stability by @PINTO0309 in https://github.com/PINTO0309/onnx2tf/pull/885

Full Changelog: https://github.com/PINTO0309/onnx2tf/compare/2.0.22...2.0.23

2.0.22

Summary

This PR improves flatbuffer_direct lowering coverage and removes redundant layout conversion chains.

1) Builtin lowering coverage

• Add ONNX InstanceNormalization lowering to builtin TFLite ops (MEAN/SUB/MUL/ADD/SQRT/DIV chain)
• Add ONNX Dropout lowering (inference-time no-op + optional mask generation path)
• Register/import new builders and validators in dispatch registry

2) Reshape/shape robustness

• Allow Reshape dynamic shape input (INT32/INT64, rank-1) in validation/lowering
• Add static-template inference for dynamic reshape patterns built from CONCAT + SHAPE
• Avoid clobbering dynamic reshape options in runtime-shape cases

3) Layout/transpose optimization improvements

• Add/extend transpose-chain rewrites including:
  • transpose_slice_logistic_concat_reshape_tail NHWC rewrite
  • ShuffleNet-style transpose/shuffle chain rewrites
  • Pre-ADD / MUL-const / reshape-suffix rewrite refinements
  • New transpose_pre_unary_reshape_transpose_suffix rewrite
  • New transpose_logistic_muladd_prepost rewrite
• Re-run optimization sequences in recovery sweeps where needed

4) Shape reconciliation safety

• Prevent unsafe static rewrite of dynamic SLICE begin/size when input dimensions are dynamic

5) Tests and docs

• Expand tests/test_tflite_builder_direct.py with new coverage for:
  • Dropout, InstanceNormalization
  • Dynamic-slice reconciliation safety
  • New transpose optimization patterns
• Update README op coverage table and container/version references
• Bump version to 2.0.22 and sync dependency/lock updates (sne4onnx/sng4onnx 2.0.1)

Validation

• pytest -q tests/test_tflite_builder_direct.py -k "transpose_pre_unary_reshape_transpose_suffix_nhwc_chain_optimized or transpose_logistic_muladd_prepost_nhwc_chain_optimized"
• pytest -q tests/test_tflite_builder_direct.py -k "transpose_add_reshape_transpose_suffix_optimization or transpose_mulconst_add_reshape_transpose_suffix_optimization or transpose_pre_unary_reshape_transpose_suffix_nhwc_chain_optimized or transpose_pre_add_mul_add_prelu_nhwc_chain_optimized or transpose_logistic_muladd_prepost_nhwc_chain_optimized"
• Manual conversion check:
  • onnx2tf -i shadowformer_istd_160x240_split.onnx -cotof -tb flatbuffer_direct --report_op_coverage

What's Changed

• feat(flatbuffer_direct): add builtin coverage and reduce transpose chains by @PINTO0309 in https://github.com/PINTO0309/onnx2tf/pull/884

Full Changelog: https://github.com/PINTO0309/onnx2tf/compare/2.0.21...2.0.22

2.0.21

Summary

This PR expands builtin coverage in flatbuffer_direct and adds a broad set of transpose-chain optimizations to remove redundant NCHW/NHWC layout adapters.

Key Improvements

• Builtin coverage expansion
  • Min -> MINIMUM (multi-input is lowered as chained binary ops)
  • TopK -> TOPK_V2 (with explicit constraints for axis/largest/sorted/k shape/indices dtype)
  • DepthToSpace -> DEPTH_TO_SPACE (DCR) / RESHAPE+TRANSPOSE+RESHAPE (CRD)
  • HardSwish -> direct lowering to HARD_SWISH (removes pseudo-op expansion)
• NMS builtin enhancement
  • Supports class dim >1 without --output_nms_with_argmax via class-wise NON_MAX_SUPPRESSION_V4
  • Behavior aligned with default onnx2tf/ops/NonMaxSuppression.py
• Pad builtin enhancement
  • Supports dynamic pads (rank-1, length 2*rank) via CAST+RESHAPE+TRANSPOSE
• AveragePool builtin enhancement
  • Supports explicit pads / SAME_LOWER / count_include_pad in {0,1}
  • Applies divisor correction when count_include_pad=0 with non-zero effective padding
• Transpose optimization enhancements (terminal flatbuffer_direct rewrites)
  • Stronger NHWC propagation from Conv/DepthwiseConv outputs to avoid immediate redundant transposes
  • Leading transpose passthrough now includes CAST
  • Removes pre/post transpose wrappers around MEAN, PRELU, and HardSigmoid-expanded chains (including MUL+ADD+RELU_0_TO_1)
  • Added extra optimization patterns for ShuffleNet-style and SiNet-tail (Softmax-mask residual) chains
  • Folds single-path affine chain: CONV_2D -> MUL(const) -> ADD(const)
  • Canonicalizes MAXIMUM(0.0) -> MINIMUM(1.0) to RELU_0_TO_1
• Docs / version updates
  • Updated README with flatbuffer_direct support matrix, constraints, and optimization notes
  • Bumped version to 2.0.21

Regression Check

• pytest -q -k flatbuffer_direct
  • 254 passed, 53 deselected
• Conversion sanity check
  • onnx2tf -i sinet_320_op.onnx -cotof -tb flatbuffer_direct --report_op_coverage
  • Conversion succeeded, ONNX/TFLite output check is pass=True

What's Changed

• feat(flatbuffer_direct): builtin拡張とTranspose最適化を強化 by @PINTO0309 in https://github.com/PINTO0309/onnx2tf/pull/883

Full Changelog: https://github.com/PINTO0309/onnx2tf/compare/2.0.20...2.0.21

2.0.20

Summary

This PR now includes all current changes from fix-resize-cubic (not only Resize).

1) Resize(cubic) improvements

• Strict ONNX cubic lowering for rank-4 static spatial cases using:
  • RESHAPE + BATCH_MATMUL + RESHAPE + BATCH_MATMUL
• Preserves ONNX cubic behavior for:
  • coordinate_transformation_mode (align_corners, asymmetric, half_pixel, pytorch_half_pixel)
  • cubic_coeff_a
  • exclude_outside
• Improves batch-shape stability in cubic decomposition paths.

2) flatbuffer_direct lowering expansion

• Added/expanded lowering and dispatch coverage around microsoft-domain fused conv paths and related builder/registry wiring.
• Extended transpose/layout optimization passes in lower_from_onnx2tf.py to reduce redundant transpose insertion and improve NHWC propagation robustness.
• Added pattern fusions/rewrites for activation-related chains (including LeakyRelu-style decompositions).

3) Atan/ATAN2 path robustness

• ATAN2 RHS handling revised to avoid invalid rank mismatches while reducing unnecessary broadcast/materialization patterns.
• Boundary transpose elimination behavior improved for affected unary/binary chains.

4) Docs and tests

• README updated to reflect current Resize(cubic) behavior and constraints.
• Test coverage expanded in:
  • tests/test_tflite_builder_direct.py
  • tests/test_tflite_builder_op_coverage.py
  • tests/test_microsoft_domain_supplement.py

Scope

• Changed files: 16
• Commits: 2
• Branch: fix-resize-cubic -> main

Notes

• This PR intentionally contains the full branch delta requested, including non-Resize updates already developed on this branch.

What's Changed

• feat: include full fix-resize-cubic branch updates by @PINTO0309 in https://github.com/PINTO0309/onnx2tf/pull/882

Full Changelog: https://github.com/PINTO0309/onnx2tf/compare/2.0.19...2.0.20

2.0.19

Summary

This PR improves flatbuffer_direct conversion coverage and robustness in onnx2tf, with a strong focus on layout-safe lowering, grouped convolution behavior, and additional builtin operator support.

Key improvements

• Improve direct export retry behavior in convert():
  • Auto-retry with output_nms_with_argmax when NonMaxSuppression class-dimension constraints are detected.
  • Preserve grouped-convolution fallback path and propagate disable_group_convolution into direct lowering.
• Expand builtin lowering coverage:
  • Add builders/dispatch/validation for Tile, Erf, ScatterND, and GlobalAveragePool.
  • Add Einsum routing (FULLY_CONNECTED for const RHS, BATCH_MATMUL for non-const RHS).
  • Add UNIDIRECTIONAL_SEQUENCE_LSTM lowering for forward LSTM direction.
  • Add Conv/ConvTranspose rank-3 handling (Conv1D/ConvTranspose1D via 2D lowering).
  • Extend Slice support to legacy attribute form (starts/ends attrs).
• Improve conv-family behavior:
  • Better explicit-padding handling for Conv/QLinearConv/ConvTranspose.
  • Support grouped conv builtin path, plus split-conv-concat fallback when grouped-conv lowering is disabled.
  • Improve quantized global average pool lowering path and shape/signature reconciliation.
• Add/strengthen transpose and pattern optimizations in lowering:
  • New safe optimization passes for Swish/GELU/LeakyReLU and multiple concat/add/const transpose bridge patterns.
  • Improve terminal SOFTMAX / pre-ARG_MAX transpose handling and preserve layout boundaries when needed.
  • Make GELU pattern fusion robust for interleaved graphs; stop rewriting Gelu pseudo-op in wave1 to keep fused Gelu forms.
• Improve runtime input-layout adaptation for accuracy evaluation:
  • Add NCW/NCHW/NCDHW -> NWC/NHWC/NDHWC adaptation support.

Tests

• Added and updated coverage across direct lowering, op coverage, preprocess fusion, and accuracy evaluator input layout.
• Executed:
  • pytest -q tests/test_tflite_builder_direct.py tests/test_tflite_builder_op_coverage.py tests/test_tflite_builder_preprocess.py tests/test_accuracy_evaluator_input_layout.py
• Result: 227 passed

Version

• Bump package version to 2.0.19 (pyproject.toml, onnx2tf/__init__.py, README.md, uv.lock).

What's Changed

• Improve flatbuffer_direct lowering coverage and robustness by @PINTO0309 in https://github.com/PINTO0309/onnx2tf/pull/881

Full Changelog: https://github.com/PINTO0309/onnx2tf/compare/2.0.18...2.0.19

2.0.18

Summary

Expand flatbuffer_direct builtin lowering coverage and remove multiple CUSTOM fallbacks by adding direct dispatch/builders/validators for additional ONNX ops.

Main changes

• Added 49 ONNX ops to _DISPATCH_REGISTRY builtin path: Abs, Acos, Acosh, And, ArgMin, Asin, Asinh, Atan, Atanh, BitShift, BitwiseAnd, BitwiseNot, BitwiseOr, BitwiseXor, Ceil, Celu, Cos, Cosh, Elu, Equal, EyeLike, Floor, GRU, GatherND, Gelu, Greater, Hardmax, Less, LessOrEqual, Mish, NonMaxSuppression, NonZero, Not, Or, RNN, Range, ReduceL1, ReduceL2, Round, Selu, Sign, Sin, Sinh, Softplus, Softsign, Tan, Trilu, Where, Xor.
• Implemented new direct builders across categories:
  • elementwise/logical: trig/hyperbolic/softplus-softsign/celu-selu/mish/bitwise/where
  • index/selection: ArgMin, GatherND, Hardmax, NonZero, builtin NonMaxSuppression
  • recurrent: RNN, GRU
  • reduce: ReduceL1, ReduceL2
  • shape/indexing: EyeLike, Range, Trilu
• Recurrent constraints updated:
  • GRU: supports linear_before_reset={0,1} and direction in {forward, reverse, bidirectional} under flatbuffer_direct constraints.
  • RNN: direct builtin path with explicit constraints/validation.
• Added/extended FlatBuffer option writers in model_writer.py:
  • ArgMinOptions, NonMaxSuppressionV4/V5Options, SequenceRNNOptions
  • additional builtin op option routing for newly emitted ops.
• Wired output_nms_with_argmax through flatbuffer_direct lowering/report contexts.
• Updated README flatbuffer_direct builtin support section and constraints to match implementation.
• Version bump: 2.0.17 -> 2.0.18.
• Removed temporary implementation notes files: update-builder.md, update-builder2.md.

Validation

• python -m py_compile passed for all changed python modules.

Impact

• Reduces unresolved custom ops in flatbuffer_direct path.
• Improves direct conversion coverage for detection/postprocess and recurrent model patterns.

What's Changed

• flatbuffer_direct: expand builtin op coverage and recurrent lowering by @PINTO0309 in https://github.com/PINTO0309/onnx2tf/pull/880

Full Changelog: https://github.com/PINTO0309/onnx2tf/compare/2.0.17...2.0.18

2.0.17

Summary

This PR expands flatbuffer_direct builtin coverage and stabilizes int8 model conversion flows used on the imp-int8 branch.

Main changes

• Added/expanded builtin dispatch and lowering for:
  • DynamicQuantizeLinear
  • Shape
  • ConstantOfShape
  • FusedMatMul (including Microsoft-domain flow)
  • OneHot
  • MatMulInteger
  • Pow
  • Reciprocal
  • LRN
  • LogSoftmax
  • constrained LSTM path (BIDIRECTIONAL_SEQUENCE_LSTM lowering) <img width="454" height="569" alt="image" src="https://github.com/user-attachments/assets/cb4efac5-7acf-4b98-ab57-a248c0fad3e1" />
• Extended Resize builtin path:
  • Supports dynamic rank-1 integer sizes input (in addition to constant scales/sizes)
  • Prevents fallback to unresolved ONNX_RESIZE custom-op in affected models (e.g. FCN-resnet50 int8 case)
• Improved shape/signature reconciliation in lowering:
  • Propagation fixes for CAST, NEG, FLOOR_MOD, MAXIMUM, MINIMUM, BATCH_MATMUL
• Updated op coverage/report behavior and related validations
• Updated README builtin coverage/constraints to reflect new support

Validation

• pytest -q tests/test_tflite_builder_op_coverage.py
• pytest -q tests/test_tflite_builder_direct.py -k "logsoftmax_lowering or lrn_lowering or pow_lowering or reciprocal_lowering or onehot_lowering or matmul_integer_lowering or dynamic_quantize_linear_lowering or shape_lowering or constant_of_shape_lowering or fused_matmul_lowering or resize_dynamic_sizes_lowering or reconcile_cast_propagates_output_shape_signature or reconcile_neg_propagates_output_shape_signature or reconcile_floor_mod_propagates_output_shape_signature or reconcile_maximum_propagates_output_shape_signature or reconcile_minimum_propagates_output_shape_signature or reconcile_batch_matmul_propagates_output_shape_signature"

What's Changed

• flatbuffer_direct: expand int8 builtin lowering and graph reconciliation by @PINTO0309 in https://github.com/PINTO0309/onnx2tf/pull/879

Full Changelog: https://github.com/PINTO0309/onnx2tf/compare/2.0.16...2.0.17

2.0.16

Summary

This PR is focused on regression recovery after the 2.0.15 series, while keeping the TF backend stable and improving flatbuffer_direct robustness.

Regression-focused fixes

• Reverted high-risk TF backend layout heuristics in onnx2tf/ops/Conv.py and onnx2tf/ops/Transpose.py that caused channel/layout mismatch regressions.
• Added NMS layout canonicalization in onnx2tf/ops/NonMaxSuppression.py to normalize:
  • boxes: [batch, num_boxes, 4]
  • scores: [batch, num_classes, num_boxes] This addresses reported shape mismatch failures.

flatbuffer_direct stability and coverage improvements

• Added safer conversion fallback in onnx2tf/onnx2tf.py:
  • if TF conversion path fails, try flatbuffer_direct-only conversion directly
  • retry once with custom-op lowering enabled (when not explicitly requested)
• In flatbuffer_direct mode, continue processing when saved_model export path errors are irrelevant.
• Expanded builtin lowering/validation:
  • ArgMax, Cast, Expand, GatherElements, Mod, ReduceMax
• Added Div constant-denominator lowering:
  • Div(x, const) -> reciprocal precompute + Mul
  • non-float outputs are handled via CAST -> MUL -> CAST
• Added Conv/DepthwiseConv + (RELU/RELU6/TANH) fusion into fusedActivationFunction in flatbuffer_direct IR.
• Improved QLinearConv grouped/depthwise handling and custom fallback behavior for unsupported grouped cases.
• Updated opcode emission for builtin codes >127 compatibility (deprecatedBuiltinCode handling).

Documentation / version

• Updated README flatbuffer_direct capability matrix and notes.
• Bumped version to 2.0.16.

Validation performed

• onnx2tf -i convnext-det.onnx -cotof -tb flatbuffer_direct --report_op_coverage
  • conversion completed successfully
  • OP error report generation completed

What's Changed

• Fix major regression set in TF/flatbuffer_direct conversion paths by @PINTO0309 in https://github.com/PINTO0309/onnx2tf/pull/878

Full Changelog: https://github.com/PINTO0309/onnx2tf/compare/2.0.15...2.0.16

2.0.15

Summary

This PR significantly improves the flatbuffer_direct path, focusing on conversion stability, quantization-chain optimizations, shape/NHWC propagation, and redundant transpose reduction.

• Changed files: 37
• Diffstat: +7283 / -493
• Base commit (main): 9281976
• Head commit (fix-var): 2d9ca0a
• INT8 ONNX -> INT8 Flatbuffer
  • https://huggingface.co/opencv/text_detection_ppocr/resolve/main/text_detection_en_ppocrv3_2023may_int8.onnx?download=true

    onnx2tf \
    -i text_detection_en_ppocrv3_2023may_int8.onnx \
    -cotof \
    -tb flatbuffer_direct \
    --report_op_coverage
    

    INT8 ONNX	INT8 Flatbuffer
    <img width="298" height="634" alt="image" src="https://github.com/user-attachments/assets/c253a826-081e-4439-aba2-bf1444bfe19d" />	<img width="335" height="622" alt="image" src="https://github.com/user-attachments/assets/cefe6304-acb1-4bb9-8055-28b76cddd2c7" />
    <img width="1500" height="176" alt="image" src="https://github.com/user-attachments/assets/5142e47c-d005-4a71-8783-7aaa7849722d" />

Main Changes

1. FlatBuffer direct lowering and shape propagation

• Strengthened shape / shape_signature propagation, mainly in lower_from_onnx2tf.py.
• Reduced shape-freezing issues in dynamic-shape cases. <img width="716" height="492" alt="image" src="https://github.com/user-attachments/assets/d6f22ad5-7564-485f-937e-67ca48173054" />
• Expanded redundant Transpose-chain elimination around NCHW/NHWC bridges.
• Improved shape consistency around Resize*, Conv, DepthwiseConv, ConvTranspose, Mean, Squeeze, and Concat.

1. Quantized chain optimizations

• Added/extended folds for:
  • Dequantize -> HardSigmoid -> Quantize
  • Dequantize -> TransposeConv -> Quantize
  • Dequantize -> Reshape -> Quantize
  • Dequantize -> MaxPool -> Quantize
  • Dequantize -> Softmax -> Quantize
• Improved robustness for QLinear-family ops, including QLinearConv, QLinearConcat, QLinearAdd, QLinearMul, and QLinearSigmoid.

1. Preprocess pipeline improvements

• Added new rule: bn_fold_wave0 (preprocess/rules/bn_fold.py)
  • Conv/ConvTranspose + BatchNormalization fold
  • affine expansion for Dequantize + BatchNormalization
  • new fold for Conv -> Mul -> Add affine chains
• Added new rule: cleanup_unused_initializers_z9 (preprocess/rules/cleanup_unused_initializers.py)
  • removes unused initializers/value_info
• Improved preprocess report visibility so BN target/fold counts are easier to diagnose.

1. Runtime/logging and developer experience

• Filtered verbose Captures: lines during SavedModel export.
• Suppressed TensorFlow startup STDERR noise with:
  • ONNX2TF_SUPPRESS_TF_STARTUP_STDERR (default: enabled)
• Enhanced flatbuffer_direct OP error and tensor-correspondence reporting.

1. Docs/metadata updates

• Updated README OP summary.
• Updated package metadata/version files (__init__.py, pyproject.toml).

Validation

Validated mainly with real conversion runs (examples):

• text_detection_en_ppocrv3_2023may_int8.onnx
• human_segmentation_pphumanseg_2023mar_int8.onnx
• pose_estimation_mediapipe_2023mar_int8bq.onnx
• face_recognition_sface_2021dec_int8.onnx

Also confirmed:

• python -m compileall on key modified modules
• preprocess-only checks for bn_fold_wave0 and cleanup_unused_initializers_z9 statistics

What's Changed

• Enhance flatbuffer_direct stability: shape propagation, quantized fusion, BN/preprocess folding by @PINTO0309 in https://github.com/PINTO0309/onnx2tf/pull/877

Full Changelog: https://github.com/PINTO0309/onnx2tf/compare/2.0.14...2.0.15

2.0.14

Summary

This PR consolidates recent fix-lp work for flatbuffer_direct INT8 conversion quality, diagnostics, and reportability.

What Changed

• Expanded flatbuffer_direct INT8 lowering and registry support.
• Added tensor lineage tracking in IR lowering to preserve ONNX->TFLite correspondence through rewrites.
• Added automatic tensor correspondence report output (*_tensor_correspondence_report.json).
• Added automatic op error report generation during conversion:
• *_op_error_report.json
• *_op_error_report.csv
• Moved op error report utility to package path:
• from tools/flatbuffer_direct_op_error_report.py
• to onnx2tf/utils/flatbuffer_direct_op_error_report.py
• Added callable API generate_op_error_report(...) to reuse report generation from onnx2tf.py.
• Improved evaluation/accuracy tooling (layout alignment, compare mode handling, split evaluation integration).
• Updated quantized op implementations and shape handling across:
• QLinearAdd, QLinearConcat, QLinearGlobalAveragePool, Reshape, Resize
• op_builders/pool.py, op_builders/quantized.py, op_builders/shape.py
• Supplemented temporary Microsoft domain handling for selected QLinear ops, including QLinearSoftmax.
• Version bumped to 2.0.14 and README container tag references updated.

Validation

• Verified Python compile for updated modules.
• Verified flatbuffer_direct conversion path still works.
• Confirmed automatic generation logs for:
• op coverage report
• tensor correspondence report
• op error report JSON/CSV

<img width="1027" height="213" alt="image" src="https://github.com/user-attachments/assets/cbafca0f-73d5-4d02-a24d-4241a95b680f" />

Notes

• Scope is intentionally broad to capture the current stabilization and diagnostics work in one integration PR.

What's Changed

• Enhance flatbuffer_direct INT8 diagnostics and stability by @PINTO0309 in https://github.com/PINTO0309/onnx2tf/pull/876

Full Changelog: https://github.com/PINTO0309/onnx2tf/compare/2.0.13...2.0.14