This guide describes the current source-development workflow for TICO. Repository
commands are routed through the ./ccex helper in the project root. It owns environment
setup, test and runtime commands, diagnostics, formatting, and CI reproduction.
The System Test Guide owns test strategy, harness behavior, and
correctness criteria.
- Prerequisites
- Create a development environment
- Torch selection
- Build and install a wheel
- Run tests
- Test organization
- Runtime selection
- Model tests
- Debug conversion
- Formatting and static checks
- Coverage
- Pull-request CI
- Change-specific guidance
- Linux development environment
- Python 3.10 or newer
git- A Python virtual environment is strongly recommended
Neither conversion nor the test suite requires ONE (one-compiler) or onert.
Converted Circle models are executed by the built-in reference runtime
(tico.circle.runtime). ONE's circle-interpreter and the onert package are
optional compatibility runtimes that are used only when selected explicitly.
git clone https://github.com/Samsung/TICO.git
cd TICO
python3 -m venv .venv
source .venv/bin/activate
# Install a supported Torch build and TICO in editable mode.
./ccex install
# Install formatter/static-check dependencies and test-only dependencies.
./ccex configureThe order matters. ./ccex configure test validates an already installed Torch package
and installs the matching TorchVision build; it does not install TICO or Torch. Run
./ccex install first. Test configuration also installs
test/requirements.txt and validates the package
environment. test/requirements_pre.txt is reserved
for pre-release test dependencies and is currently empty. Optional external runtimes
are installed separately; see Runtime selection.
Set up only one part of the environment when needed:
./ccex configure format
./ccex configure testThe source installer accepts a stable family, an exact version, a qualification candidate, a repository-pinned nightly, or the latest published nightly pair:
./ccex install --torch_ver 2.13
./ccex install --torch_ver 2.7 # legacy best-effort
./ccex install --torch_ver 2.11
./ccex install --torch_ver 2.13.0+cu132
./ccex install --torch_ver 2.14
./ccex install --torch_ver nightly # repository-pinned Torch/TorchVision
./ccex install --torch_ver nightly-latest # latest published nightly pairTICO keeps Torch 2.5 through 2.10 as legacy best-effort source-install choices,
qualifies 2.11, 2.12, and 2.13, and uses 2.13 as the default. Torch 2.14 is installable
as a qualification candidate but is not part of the release-support window. Family
requests resolve to a project-pinned patch rather than allowing pip to select an
arbitrary patch release. The package metadata itself keeps a bare torch dependency,
so a normal pip install does not reject a user-managed older version. nightly uses
the versions pinned under infra/dependency/; nightly-latest resolves Torch and
TorchVision together from one moving nightly index.
Compute-platform options:
# Force a CPU wheel.
./ccex install --cpu_only
# Override detected host CUDA capability when selecting a compatible wheel.
./ccex install --cuda_ver 12.8--cpu_only and --cuda_ver are mutually exclusive.
When no Torch version is explicitly requested and a configured stable Torch package is
already installed, ./ccex install preserves that installation if its compute platform
is compatible. This includes legacy best-effort families. ./ccex configure test then
installs the matching TorchVision package
and verifies the final package pair with pip check. Nightly selectors are deliberately
re-resolved: nightly restores the repository pin, while nightly-latest upgrades the
Torch/TorchVision pair together before test configuration validates it.
The source of truth for families, exact patches, CUDA wheel variants, and CI matrices is
tico/utils/compat/torch_version_policy.py.
See the PyTorch Version Policy for qualification,
promotion, and release-branch rules.
Build the distribution artifacts:
./ccex buildInstall from dist/ instead of editable source:
./ccex install --distA clean wheel workflow similar to CI is:
./ccex build
./ccex install --dist --torch_ver 2.13
./ccex configure test --torch_ver 2.13
pt2-to-circle -h./ccex testWith no filter, ccex runs unittest discovery under test/. Large model tests are
selected separately with -m and are not part of the normal model-independent suite.
./ccex test supports repository paths containing spaces.
./ccex test -k add
./ccex test -k ConvertMatmulToLinear
./ccex test -k quantization
./ccex test -k passes
./ccex test -k test_quantizer_registryQuote keyword patterns containing spaces to pass them to unittest as one argument.
The shorthand keywords op and net are expanded to the corresponding generated
module-test namespaces:
./ccex test -k op
./ccex test -k netUse the narrowest relevant test first, then expand to the owning subsystem and finally to the complete suite for cross-cutting changes.
./ccex test --all # Explicit full suite
./ccex test -i # Include internal-only tests
./ccex test -v # Set TICO_LOG=4 for this run
./ccex test -r reference # Built-in reference runtime (default)
./ccex test -r onert # Optional: execute with the onert package
./ccex test -p # Run performance benchmarks--all and --keyword cannot be combined. --model also cannot be combined with
--all or --keyword. -i sets RUN_INTERNAL_TESTS=1 for discovery. -p / --perf
selects the performance entry point instead of normal discovery. For example,
./ccex test -v -k add combines verbose diagnostics with a focused test; setting
TICO_LOG=4 explicitly has the same effect (see Log levels).
./ccex test --durations 10 # Collection, suite time, and slowest 10 full test IDs
./ccex test -k SimpleAdd --durations 0 # Report every executed test duration--durations N is opt-in, accepts a nonnegative count (0 means all), and works
with the default suite, -k, and -m. It does not apply to --perf. It uses the
standard unittest duration support on Python 3.12 and newer, with a small result
adapter on Python 3.10/3.11; selection, outcomes, and exit behavior stay with unittest.
Collection time starts in the timing worker before importing unittest and ends after
discovery (including discovered imports and generated test construction). It excludes
interpreter startup and the parent ccex script. Suite time surrounds unittest's runner,
including class/module fixtures and result output. Individual durations include test
setup, body, teardown, and cleanups; they exclude class/module fixtures and discovery.
Thus suite time minus individual durations is not a discovery measurement. For complete
command wall time, measure a fresh process externally, for example with
time ./ccex test -k SimpleAdd --durations 10; compare identical environment, selection,
and cache conditions across repeated fresh processes.
Small local CPU wrapper tests run in the default suite; their Transformers version
or model-class availability guards still apply. --include-internal is reserved for
external dependencies and costly CPU integration checks. It does not install their
prerequisites. Prefer selecting one test rather than enabling every internal test:
# Downloads Maykeye/TinyLLama-v0 and Salesforce/wikitext (wikitext-2-raw-v1).
# Requires the configured Transformers/datasets packages and Hugging Face access
# or an explicitly prepared local cache.
./ccex test --include-internal -k SmoothQuantTest.test_value
# Local CPU integration: large calibration tensors and Hessians; allow extra
# CPU time and RAM. This does not need a checkpoint or dataset download.
./ccex test --include-internal -k GPTQTest.test_net_on_zero_inputs
# Requires triv24-compile and triv24-ssinfer from the TRIV24 toolchain under
# /usr/share/one/backends/triv24/bin (separate from the reference runtime).
./ccex test --include-internal -k EvaluateTest.test_evaluate_simple_linearThe GPTQ test_model and introspection TestSmoothQuantPTQDiff checks use the same
remote model and dataset as the SmoothQuant example above. Other retained GPTQ/FPIGPTQ
CPU integration skips describe their calibration/Hessian costs in the test reason.
Model-directory tests keep their separate model selection path.
Local Gemma4 assistant integration requires GEMMA4_ASSISTANT_PATH (and
GEMMA4_TARGET_PATH for verification); run
python3 -m unittest -v test.quantization.recipes.integration.test_gemma4_assistant_integration
with those checkpoint directories set. CUDA observer checks require a CUDA-enabled
Torch build and a working GPU. The large Circle tests
document their independent memory/disk and ONE opt-ins. External runtimes keep the
explicit runtime selection requirements.
./ccex test -p runs the configured conversion-time and serialized-size threshold
benchmark:
./ccex test -pThe full Circle O1 scheduler comparison needs a caller-provided .circle artifact and
is invoked directly:
python3 -m test.performance.benchmark_circle_optimizer \
model.circle \
--repeat 3It compares the legacy RESTART scheduler with O1's round-based
UNTIL_NO_CHANGE scheduler, verifies both outputs, and requires byte-identical Circle
binaries. It is an opt-in diagnostic benchmark rather than a repository threshold.
See test/performance/README.md for all options.
Use focused unit tests for local behavior and module tests for PyTorch-to-Circle parity. Model tests and performance benchmarks are opt-in workflows. The test documentation index links the subsystem guides.
The System Test Guide owns the test layers, directory layout, and end-to-end harness flow. Consult its correctness criteria when deciding what a new regression must assert and its change-specific checklist when choosing coverage.
The default runtime for module parity tests, CircleModel.__call__(), and
tico.interpreter.infer() is the built-in reference runtime. It executes the
serialized Circle bytes with NumPy/CPU PyTorch kernels, supports dynamic shapes, and
needs no external package:
./ccex test -k add
./ccex test -r reference -k addTwo optional compatibility runtimes can be selected from the command line or the environment. They are never chosen implicitly, even when installed:
# ONE luci-interpreter through /usr/share/one/lib/libcircle_interpreter_cffi.so
pip install cffi # plus a ONE release providing the shared library
./ccex test -r circle-interpreter -k add
# onert Python package (pre-release wheels)
pip install --pre onert==0.2.0.dev250922
./ccex test -r onert -k add
CCEX_RUNTIME=onert ./ccex test -k addcircle-interpreter cannot execute dynamic-shape models; the harness runs a test tagged
use_onert with onert when that legacy runtime is selected. A negative test whose
expected error is runtime-specific (@test_negative(..., runtime="onert")) runs as a
normal parity test under the other runtimes.
See Circle artifact tools for the supported operator set, quantized execution modes, and debugging options.
Model tests are selected by directory name or shell-style pattern:
pip install -r test/modules/model/<model_name>/requirements.txt
./ccex test -m <model_name>
# Quote wildcard patterns so the shell does not expand them.
./ccex test -m "Llama*"The -m option sets CCEX_TEST_MODEL and runs the model-test loader. Model tests may
have additional dependencies and can be significantly more expensive than synthetic
unit or module tests. Do not use a full model test when a small deterministic graph can
cover the behavior.
Set TICO_LOG before the Python process imports TICO:
TICO_LOG=4 ./ccex test -k add
TICO_LOG=4 python examples/my_conversion.py| Value | Level |
|---|---|
1 |
fatal |
2 |
warning |
3 |
info |
4 |
debug |
Debug mode includes instrumented graph and constant-size diffs around passes and conversion phases.
TICO_GRAPH_DUMP=1 ./ccex test -k addThe main conversion pipeline writes these stage snapshots when applicable:
.tico_tmp/session_<timestamp>/1_after_decompose.png
.tico_tmp/session_<timestamp>/2_after_legalize.png
.tico_tmp/session_<timestamp>/3_after_quantfold.png
Graph rendering uses pydot and Graphviz. The output directory is session-scoped; do
not commit generated images.
tico-circle inspect model.circle --tensors --operators
tico-circle verify model.circle
tico-circle extract model.circle --ops 20-64 -o region.circle
tico-circle optimize model.circle --preset o1 -o model.o1.circleUse tico-circle verify for static Circle consistency. Use an end-to-end runtime test
for numerical parity and backend execution. CircleReferenceRuntime(...).run(inputs, trace=True) retains every materialized tensor value for debugging. Virtual full-table
dequantization values used by selected-row execution are omitted from the trace.
Before opening a pull request, apply formatting and run all static checks with one command:
./ccex formatThis runs UFMT (Black formatting and import sorting), Pylint, and mypy together. To check without applying formatter patches:
./ccex format --no-apply-patchesUse Python 3.12 and the development environment described above. Pylint and mypy need the project and test dependencies: missing PyTorch or different installed library versions can change inferred types without reporting an import error.
To reproduce CI in a fresh Python 3.12 virtual environment, install the policy-selected CPU Torch build and configure the same dependencies:
TORCH_FAMILY=$(python3 tico/utils/compat/torch_version_policy.py default-family)
./ccex install --torch_ver "$TORCH_FAMILY" --cpu_only
./ccex configure test --torch_ver "$TORCH_FAMILY" --cpu_only
./ccex configure format
python3 -m pip check
python3 -m pip list --format=freeze
./ccex format --no-apply-patchesUse a separate environment when developing against another Torch version or optional model dependencies. The static-analysis CI job records installed package versions to help compare environments.
To check tracked files changed from the local main branch, use --diff-only.
Add new files to Git before checking them:
./ccex format --diff-only --no-apply-patchesCI runs formatting and static analysis as separate jobs so formatting can finish
without installing PyTorch. Both jobs use the same lintrunner configuration as
./ccex format; developers do not need to select individual tools.
The formatter's transitive dependencies are pinned in
requirements-lintrunner.txt, including stdlibs
and libcst. Keep those pins when updating formatter packages: their versions can
change import classification and formatting even when Black, UFMT, and usort are
unchanged.
Run the discovery suite under coverage and print a terminal report:
./ccex coverageWrite a report under test/reports/cov/:
./ccex coverage -f txt
./ccex coverage -f xmlIf coverage is not installed, the current helper requests version 7.6.1.
Failed tests return a nonzero exit status while still generating the requested report;
report failures also return nonzero when the tests pass.
Coverage discovery and report generation support checkout paths containing spaces
and treat shell pattern characters in those paths literally.
The PR workflow targets main and rel/*. PyTorch versions are resolved from the
central policy module rather than duplicated in workflow YAML.
-
Commit-message check
-
Runs when the pull request is ready for review.
-
Requires at least one commit-body line in the form:
TICO-DCO-1.0-Signed-off-by: <NAME> <<EMAIL>>
-
-
Format check (
check-format)- Runs on Ubuntu 24.04 with Python 3.12.
- Executes
./ccex configure format, then runs lintrunner with--take UFMTwithout applying patches. - Does not depend on the package build or install PyTorch.
-
Package build
- Builds the TICO wheel once.
- Uploads one short-lived artifact reused by static analysis and versioned tests.
-
Static analysis (
check-static)- Runs on Ubuntu 24.04 with Python 3.12.
- Reuses the test-environment setup action with the default Torch family from the central policy, including CPU Torch, matching TorchVision, TICO, and test dependencies.
- Configures the analysis tools, checks package dependency consistency, records
installed versions, runs the Transformers/Pylint infrastructure regression tests
(
python3 -m unittest -v infra.style.tests.test_transformers_config_pylint), and runs lintrunner with--take PYLINT,MYPYwithout applying patches. - The existing Check style status (
check-style) succeeds only when both formatting and static analysis succeed, preserving existing required-check settings.
-
Versioned tests
- Runs the complete suite on the default qualified family, currently 2.13.
- Runs blocking export and quantization smoke tests on the oldest supported family, currently 2.11.
- Runs the same smoke tests non-blockingly on the qualification candidate, currently 2.14.
The smoke path covers a small torch.export/PT2/Circle conversion and a quantized CNN
Circle export. These focused tests detect version-contract breakage without running
the complete suite on every family for each pull request.
A separate compatibility workflow runs nightly-latest smoke tests on six days each
week and the complete supported/candidate/nightly-latest matrix on the remaining day.
Manual dispatch also runs the complete matrix.
It provides early warning without participating in PR branch protection. Official
package publication builds the wheel once and runs the full suite on every qualified
stable family before publishing; candidate and nightly selectors are outside that
release-support gate. See the PyTorch Version Policy
for the exact tiers.
The conversion/size threshold benchmark is available through ./ccex test -p.
The full Circle O1 scheduler benchmark is invoked directly with a caller-provided
artifact. Neither benchmark is part of the current PR matrix.
- Implement
PassBase.call()and returnPassResult(modified=...)accurately. - Preserve
ExportedProgramgraph-signature mappings while replacing nodes. - Update metadata for newly created or rewritten tensors.
- Add focused coverage under
test/unit_test/passes/. - Include both matching and structurally similar non-matching cases.
- Add an end-to-end module test when serialization or runtime behavior changes.
- Add or update a
NodeVisitorundertico/serialize/operators/. - Register every supported ATen overload with
register_node_visitor. - Add serializer/operator unit coverage and a module conversion/parity test.
- Do not leave an unsupported operator in the final graph and rely on a runtime to reject it; TICO validates supported targets before serialization.
- Preserve the
prepare -> calibration/statistics -> convertlifecycle. - Put generic algorithms and infrastructure in their owning quantization packages.
- Put model-family behavior in recipe adapters rather than generic stages.
- Prefer deterministic synthetic tests before full model evaluation.
- Update the relevant quantization README or configuration reference when behavior is user visible.
- Keep PyTorch-IR passes in
tico/passes/and serialized Circle-to-Circle passes intico/circle/passes/. - Verify index remapping, graph boundaries, buffers, signatures, multi-subgraph behavior, and cleanup contracts as applicable.
- Follow the dedicated pass guidance in Circle artifact tools.