TICO converts exportable PyTorch modules and saved
torch.export.ExportedProgram files into Circle models. Conversion itself is a Python
workflow; a Circle runtime is needed only when the generated model is executed.
- Prerequisites
- Installation
- Convert a PyTorch module
- Public conversion APIs
- Keyword arguments
- Dynamic shapes
- Compile configuration
- Convert a saved
.pt2file - Run a Circle model in Python
- Inspect and verify Circle artifacts
- Troubleshooting
- Next steps
- Python 3.10 or newer
- A supported PyTorch installation
- An inference-mode,
torch.export-compatible module
TICO's source installer keeps Torch 2.5 through 2.10 as legacy best-effort choices,
while the qualified stable support window is 2.11 through 2.13 and the default is 2.13.
Torch 2.14 is available as a qualification candidate. The package metadata keeps a
bare torch dependency so a normal pip install can coexist with a user-managed old
version. nightly is repository-pinned for reproducibility, while nightly-latest is
used as a moving early-compatibility signal rather than release support. See the
PyTorch Version Policy and
Development Guide for exact installation and CI details.
Important
Call eval() before conversion. TICO rejects training operators such as dropout in
the exported graph, and training-mode state updates can make PyTorch/Circle parity
checks unreliable.
pip install ticogit clone https://github.com/Samsung/TICO.git
cd TICO
python3 -m venv .venv
source .venv/bin/activate
# Installs a supported Torch build and TICO in editable mode.
./ccex installUseful source-install options include:
./ccex install --cpu_only
./ccex install --torch_ver 2.13
./ccex install --torch_ver 2.7 # legacy best-effort
./ccex install --torch_ver 2.11
./ccex install --torch_ver 2.14
./ccex install --torch_ver nightly
./ccex install --torch_ver nightly-latest
./ccex install --cuda_ver 12.8
./ccex install --index-url https://packages.example.com/simple--index-url is forwarded to every pip install run by ccex, including Torch,
the auxiliary requirements, and TICO itself. It replaces the automatically selected
PyTorch CPU, CUDA, or nightly index, so the supplied index must provide the requested
Torch and TorchVision wheels.
one-compiler and onert are not required to create or execute Circle files.
CircleModel.__call__() and the end-to-end tests use TICO's built-in reference runtime.
Install ONE or onert only when you explicitly want to compare against those runtimes
(CircleModel(..., runtime="circle-interpreter") or runtime="onert").
import torch
import tico
class AddModule(torch.nn.Module):
def forward(self, x: torch.Tensor, y: torch.Tensor) -> torch.Tensor:
return x + y
model = AddModule().eval()
example_inputs = (torch.ones(4), torch.ones(4))
circle_model = tico.convert(model, example_inputs)
circle_model.save("add.circle")example_inputs serve two purposes:
torch.export.export()uses them to capture the program.- Their shapes and dtypes define the default Circle input contract.
The result is a CircleModel object containing the serialized bytes in
circle_model.circle_binary.
TICO exports the following APIs from the top-level package:
import tico
circle_model = tico.convert(...)
circle_model = tico.convert_from_exported_program(...)
circle_model = tico.convert_from_pt2(...)Their current signatures are conceptually:
tico.convert(
mod,
args,
kwargs=None,
dynamic_shapes=None,
strict=True,
config=tico.get_default_config(),
)
tico.convert_from_exported_program(exported_program, config=...)
tico.convert_from_pt2(pt2_path, config=...)Use convert() when starting from an nn.Module,
convert_from_exported_program() when export has already been performed in memory,
and convert_from_pt2() for a saved exported program.
For large models, convert directly to a file to reduce peak memory from complete intermediate binaries:
tico.convert_to_file(model.eval(), example_inputs, "model.circle")
tico.convert_from_exported_program_to_file(exported_program, "model.circle")
tico.convert_from_pt2_to_file("model.pt2", "model.circle")These APIs return None and atomically replace the destination after conversion and
writing succeed. convert_to_file() accepts the same kwargs, dynamic_shapes,
strict, and config options as convert(); the other file APIs also accept
config. Circle O1 optimization remains enabled by default. The existing APIs
continue to return CircleModel objects containing complete bytes.
File conversion still keeps the source tensors and conversion graph in memory; see Large Circle export for layout selection and remaining memory limits.
Pass keyword example inputs through the kwargs argument:
import torch
import tico
class Affine(torch.nn.Module):
def forward(
self,
x: torch.Tensor,
*,
scale: torch.Tensor,
bias: torch.Tensor,
) -> torch.Tensor:
return x * scale + bias
model = Affine().eval()
args = (torch.ones(2, 3),)
kwargs = {
"scale": torch.tensor(2.0),
"bias": torch.tensor(0.5),
}
circle_model = tico.convert(model, args, kwargs=kwargs)At execution time, TICO binds positional inputs first and then keyword inputs by the serialized Circle input names. Use the same argument structure, dtype, and static dimensions that were used for export.
TICO forwards dynamic_shapes and strict to torch.export.export().
import torch
import tico
class RowSum(torch.nn.Module):
def forward(self, x: torch.Tensor) -> torch.Tensor:
return x.sum(dim=1)
model = RowSum().eval()
x = torch.randn(2, 4)
batch = torch.export.Dim("batch", min=1, max=8)
circle_model = tico.convert(
model,
(x,),
dynamic_shapes=({0: batch},),
)
circle_model.save("row_sum_dynamic.circle")A symbolic dimension is serialized with a Circle shapeSignature value of -1 and a
concrete placeholder dimension of 1 in shape. Static dimensions are preserved.
Dynamic-shape export and dynamic-shape execution are separate capabilities. The
built-in reference runtime executes dynamic models directly from the actual input
sizes, so circle_model(x_batch_3) and circle_model(x_batch_7) both work for the
model above. External runtimes may impose their own limits.
CompileConfigV1 controls selected conversion rewrites:
import torch
import tico
class Matmul(torch.nn.Module):
def __init__(self) -> None:
super().__init__()
self.register_buffer("weight", torch.randn(8, 4))
def forward(self, x: torch.Tensor) -> torch.Tensor:
return x @ self.weight
config = tico.CompileConfigV1()
config.convert_rhs_const_mm_to_fc = True
model = Matmul().eval()
circle_model = tico.convert(model, (torch.randn(2, 8),), config=config)Current version 1.0 fields are:
| Field | Default | Current behavior |
|---|---|---|
legalize_causal_mask_value |
False |
For recognized lifted attention-mask constants, replaces sufficiently negative mask values with -120. Enable only when the finite-mask assumption is valid for the target workflow. |
remove_constant_input |
False |
Omits exported ConstantArgument values from the Circle user-input interface. None inputs are omitted regardless of this option. |
convert_lhs_const_mm_to_fc |
False |
Enables eligible matrix-multiplication rewrites with a constant left-hand side. |
convert_rhs_const_mm_to_fc |
True |
Enables eligible matrix-multiplication rewrites with a constant right-hand side. |
convert_single_batch_lhs_const_bmm_to_fc |
False |
Enables the supported single-batch bmm rewrite with a constant left-hand side. |
convert_expand_to_slice_cat |
False |
Enables lowering of supported expand patterns to slice/concatenate operations. |
eliminate_rank_round_trip |
False |
Declared in CompileConfigV1, but the current conversion pipeline does not read this field; EliminateRankRoundTripRegion is currently enabled unconditionally. Do not rely on this field to disable that pass. |
preserve_embedding_quantization |
False |
Preserve explicit quantization boundaries in supported embedding paths during Circle export. Omission and explicit False retain the existing embedding export policy; see CPU embedding contracts. |
circle_optimize |
True |
Runs the verified Circle O1 pipeline on both bytes and file conversion paths. Set to False to disable Circle O1 while retaining the FX passes. |
Create a default configuration with either of the following:
config = tico.CompileConfigV1()
config = tico.get_default_config() # version 1.0Export and save a program with PyTorch:
import torch
model = AddModule().eval()
example_inputs = (torch.ones(4), torch.ones(4))
exported_program = torch.export.export(model, example_inputs)
torch.export.save(exported_program, "add.pt2")import tico
circle_model = tico.convert_from_pt2("add.pt2")
circle_model.save("add.circle")pt2-to-circle -i add.pt2 -o add.circleSupported options:
-i, --input Input .pt2 path
-o, --output Output .circle path
-c, --config Optional YAML compile configuration
-v, --verbose Enable verbose conversion logging
A YAML configuration must include version. Version 1.0 is currently supported:
version: "1.0"
legalize_causal_mask_value: false
remove_constant_input: false
convert_lhs_const_mm_to_fc: false
convert_rhs_const_mm_to_fc: true
convert_single_batch_lhs_const_bmm_to_fc: false
convert_expand_to_slice_cat: false
eliminate_rank_round_trip: false
preserve_embedding_quantization: false
circle_optimize: truept2-to-circle \
-i add.pt2 \
-o add.circle \
-c config.yamlUnknown YAML keys are not applied by CompileConfigV1.from_dict(). Treat misspelled
configuration names as errors in review even though the current loader does not reject
them.
CircleModel can be called directly:
import numpy as np
import torch
import tico
model = AddModule().eval()
inputs = (torch.ones(4), torch.ones(4))
circle_model = tico.convert(model, inputs)
output = circle_model(*inputs)
assert isinstance(output, np.ndarray)
print(output)Load an existing Circle file:
from tico.utils.model import CircleModel
circle_model = CircleModel.load("add.circle")
output = circle_model(torch.ones(4), torch.ones(4))Runtime behavior:
- A single Circle output is returned as one
numpy.ndarray. - Multiple outputs are returned as a list of NumPy arrays.
- Input count, names, dtypes, ranks, and static dimensions are checked before execution.
- Every operator result is validated against the serialized shape and dtype; dynamic
(
-1) dimensions follow the actual inputs. - The built-in execution path currently requires a one-subgraph Circle model.
- Quantized models execute with
ExecutionMode.FAKE_QUANTIZEthroughtico.quantization.evaluation.evaluate(..., BACKEND.CIRCLE); see Circle artifact tools.
Use tico-circle after serialization:
tico-circle inspect add.circle --tensors --operators
tico-circle verify add.circle
tico-circle extract model.circle --ops 20-64 -o region.circleverify checks the internal consistency of the Circle artifact: indices, dataflow,
buffers, graph interfaces, signatures, and control-flow references. It does not execute
the model, compare numerical outputs, or prove compatibility with a particular NPU
compiler.
See Circle artifact tools for the Python API, extraction semantics, cleanup passes, and CLI details.
Convert an evaluation-mode module:
model.eval()
circle_model = tico.convert(model, example_inputs)TICO validates every remaining call_function target against the registered Circle
serializer visitors. An unsupported operator is reported with the operator name and,
when available, its source stack trace. The usual fixes are to add a legalization pass,
add a serializer visitor, or change the model wrapper to emit supported ATen patterns.
Set the log level before starting Python:
TICO_LOG=4 python my_conversion.py
TICO_LOG=4 pt2-to-circle -i model.pt2 -o model.circleLog levels are:
| Value | Level |
|---|---|
1 |
fatal |
2 |
warning |
3 |
info |
4 |
debug, including graph/constant diffs where instrumented |
TICO_GRAPH_DUMP=1 python my_conversion.pyThe conversion pipeline writes images such as 1_after_decompose.png,
2_after_legalize.png, and 3_after_quantfold.png under a timestamped directory in
.tico_tmp/. Graph rendering requires pydot and a working Graphviz installation.
- System Design: conversion stages, invariants, serialization, and extension points
- PyTorch Version Policy: package installation, legacy and supported families, candidate versions, CI tiers, and promotion rules
- Development Guide: source setup, tests, formatting, and CI
- Quantization: quantize a model before Circle export
- Quantization examples: config-driven LLM and VLM workflows
- Circle artifact tools: post-serialization inspection and transformations