All notable changes to this project are documented here.
The format follows Keep a Changelog, and this project adheres to Semantic Versioning.
fdu 0.5.0 adds ages and size colors to the tree and combines several disjoint roots in
one report. It changes the report schema to fdu.report/11 and includes Rust and Python
API changes. See the release notes for upgrade
details.
- The text tree shows each row’s age, between size and name: how long ago anything the
row counts last changed, the newest modification among its own entry and every entry
beneath it that it counts, files, directories, and symlinks alike.
The scan root’s own time never counts.
With no filter a directory’s tree age equals its
--longage; under a filter each row ages what it counts. An age is grayunknownwhere part of the subtree was not listed, a gray—on a row that counts nothing, and blank on the… and N more filesrow. - Machine output: tree nodes carry
mtime_ns,complete,age_ns, andmodified_at; list rows carrymodified_at; the envelope carriesage_reference_atbesideage_reference_ns. Each*_atfield is the RFC 3339 UTC rendering of its exact nanoseconds, andmodified_atis null wherever the time is only a lower bound. - Python:
TreeNodegainsmtime_ns,complete,age_ns, andmodified_at,FileRowgainsmodified_at, andReportgainsage_reference_at; eachdatetimeis timezone-aware UTC, derived from the nanoseconds and floored to the microsecond. - Rust:
TreeNodegainsmtime_ns,complete, andage_ns. - Several paths in one report:
fdu docs srcanswers whatfdu docsandfdu srcwould, added. Sizes, counts, rows, shares, and display bounds are taken over every root together, once. The tree shows a(total)row and each root as a row named by how it was given, every share of the total; flat listings print each path after its root’s label, asfind docs srcdoes. A root equal to or inside another is refused, naming both: by canonical path, which sees through a symlink, and on Unix by device and inode, which sees a root whose path runs through an alias of another. A root that is itself an alias into another’s tree, a bind mount or a macOS firmlink such as/usr/localbeside/System/Volumes/Data, is refused on Unix when the other root’s walk enters it, with exit status 2 as before the walk; Windows compares paths only. Every PATH is a directory; a file is refused naming the command that lists only the directories,fdu */, which is itself refused where one directory is a symlink to another (lib64 -> lib). Each root is walked separately and pays a walk’s fixed cost, so many small roots are much slower than one walk of their parent: 625 small directories took 1.72 s as roots against 0.23 s for their parent, about 7 times as long (exp-216, macOS, uncontrolled host);fdu --depth 1 --min-share 0% .shows every entry one level down in one walk, counting the hidden entries and files*/leaves out, and not walking the target of a symlink that*/names as a root. Each root’s snapshot lives where its own would, written once every root has been walked, on a bounded number of threads.--watch,--cache-status, and--cache-cleartake one PATH. - Machine output over several roots: the envelope’s
rootis null androotslists each root’slabeland canonicalpath; a tree section hastree: null, atotalrow, andtrees, one tree per root, each with its ownremainder; list rows, status errors, and.gitignorerefusals carryroot, the position of their root inroots, and keep their paths relative to it. A report over one root is unchanged. - Python:
fdu.reporttakes one path or a sequence of them. Over several rootsReport.rootisNoneandReport.rootsis a tuple ofReportRoot;TreeSectiongainstotal(TreeTotal) andtrees(RootTree);FileRow,OperationError, andRefusedControlgainroot. - Rust:
RootsandNamedRootname a report’s roots and validate them before any scan;RootsRequestshares one request across them, andRootsRequest::resolvevalidates paths, request, and delivery in the order one root always failed in;report_rootsreads one report from one index per root, refusing indexes of different scopes and, on Unix, an index that entered another root through an alias; andprepare_roots_report, with its progress and scan-diagnostics variants, runs each root’s plan and returns the roots’ pending saves in one handle, with each root’s telemetry (RootsPrepared).PerformanceSummary::sumtotals them,PendingSave::join_allnames each failed write,labelled_pathis the one rule text uses for a path under one of several roots, andReportSource::weakerranks sources.Delivery::cache_dirnames a directory in which each root’s snapshot is named, andDelivery::for_rootresolves it for one root; every route honors it.RequestErrorgainsNoRoots,RootsOverlap,RootsRepeated,RootReachedInside,RootScopesDiffer, andCacheLocationTwice.
- Breaking: the report schema is
fdu.report/11, for the fields above. - Breaking: ages past a month read in months of 30.44 days (
11mo) and past a year in years of 365.25 days (3y), in--longas in the tree; they previously read in days (4,382d). Time bounds still take days;--modified-before 2mois refused and the refusal names the days, as inuse 60d. - In a color terminal, the tree’s ages and sizes are colored, after kash’s file
listings. Ages by recency: under a minute bold bright red, under an hour bold yellow,
under a day bold bright green, under a month bold cyan, under a year blue, older gray.
Sizes by magnitude, with kash’s tiers moved up for larger trees: under 16 KiB gray,
under 1 MiB blue, under 1 GiB plain, under 100 GiB bold bright green, larger bold
yellow. Nothing else changes style; every other size keeps its style, and
--long, the other flat formats, and output without color are unchanged. - Breaking:
--sort mtimeon a tree orders rows by the age the column shows, their newest counted activity of any kind, where it ordered them by their newest regular file.newest_mtime_nskeeps its files-only meaning. - A watch measures each repaint’s ages, and
Session::reportand Python’sWatch.report()theirs, from the instant of that repaint, so a file written during the session no longer shows a negative age. The windows the query selects by stay the absolute bounds its request resolved when it was built. A repaint is still skipped when nothing a reader sees changed: ages rolling over repaint nothing, and any change in a row’s activity repaints. - A watch, an opened root’s observation, and a refresh re-read the directory a created,
removed, or renamed entry lives in, so that directory’s own time, and with it its age,
matches a one-shot report: deleting a directory’s newest file, or moving an old file
in, makes it young there too.
Under
--view filesthe record for such a change is now followed by anupsertrecord for its directory, unless it is the scan root or the selection leaves directories out; the record shape andfdu.stream/2are unchanged. - Rust:
TreeNodestruct literals need the three new fields. - Breaking (Rust):
Report::rootisOption<PathBuf>,Noneover several roots, beside the newReport::roots;TreeStatus::errorsholdsStatusIssue { root, issue }rather thanIssue;FileRowandRefusedControlgainroot;Section::Treegainsroots;ProgressSnapshotgainsroot, and over several roots a run’s phase starts over at each root while its counters keep adding. - Breaking (Rust):
Deliverygainscache_dir, so its struct literals name it. - Breaking (Python):
Report.rootisPath | None. - A PATH that is a file is refused before anything reads its snapshot, naming it as
given:
fdu README.md --stale-okand Python’sfdu.report("README.md", stale_ok=True)now fail because it is not a directory (exit 1,OSError) rather than because it has no snapshot. A delivery no route can carry is still refused first, as before:fdu README.md --cache off --stale-okexits 2 for its delivery. - The
PATHargument isPATH...; help readsUsage: fdu [OPTIONS] <PATH>.... The progress line names the root being walked and its position,src (2/3), and theperf:line sums every root’s walk and names each root’s tier when they differ. - On Linux the default one-shot tree stats each directory and symlink again, as every
other route does, so a directory’s age there counts its own time and equals its age
under
--longor a cached run. 0.3.0 had stopped those stats because the tree read none of their attributes; the age column reads their times. This is expected to give back most of that change’s gain, about 3% of the default tree on a directory-densenode_modulestree and little on the Linux v6.12 source tree: exp-197 measured the gain at 3.6%, and the version that shipped kept 83% of the stats it saved. The give-back itself is measured on Linux before this release. macOS lists every child’s attributes in bulk and is unaffected, except in a directory that falls back to the portable reader, which now stats them too.
fdu 0.4.0 is a breaking release about what a report asks for and what it says.
A content view now requests the analysis it shows: fdu . --view code,documents counts
lines of code by language and words by document type from one scan, with no --analyze.
A human report holds only the result on standard output, and every explanation follows
it on standard error as a consolidated note: or tip: line, with one runnable tip
that lifts every display limit.
documents percentages now add up to 100%. On the command line the breaking changes are
what --view=documents and --view=full run beside --analyze=lines or
--analyze=code; in Rust, Request gains a private field, RequestError becomes
#[non_exhaustive] with reshaped analysis refusals, and AxisNames gains a field.
No option was removed, and the report, cache, and stream schemas are unchanged.
The GitHub release text is
docs/project/release-notes/0.4.0.md.
- Rust:
ViewSpec::impliesandViewSpec::shows, the analyzers a view requests and the ones it displays;ViewSpec::defaults_for, the views a request shows its analysis in when the caller named none;AnalysisSet::union,named, andrequest_label;Request::implied_by;Request::read_opened, a read of an opened root that refuses in its words; andquery::BasisHolder, which the analysis refusals carry.
- Breaking: a content view requests the analysis it shows.
fdu . --view=code,documentsgives lines of code by language and words by document type from one scan, with no--analyze;--view=codeand--view=documentsgive either alone.--analyzenow adds analyzers beyond what the views imply: analysis shown in a metadata view (--analyze=code --view=languages), analysis run without display, or a wider set. Views with a metadata meaning (tree,languages,types,families,full) never imply analysis. In Python,fdu.report(root, Query(views=(View.CODE,)))runs code analysis; an index fromfdu.openorfdu.scanis never widened by a read, and a view it cannot answer is refused with the analyzer named.documentsnow requireswordsanalysis, so--analyze=lines --view=documentsis refused on a held index.request.analyzein structured output is the enabled set, named or implied; the report schema is unchanged. - Breaking:
--view=documentsbeside--analyze=linesor--analyze=codenow runs words analysis as well, becausedocumentsimplies it.--analyze=lines --view=documents --format=jsonreportsrequest.analyze: ["lines", "words"]andshare_metric: "document_words"where it reported["lines"]andraw_words; a script that read raw-word shares by document type readsdocument_wordsinstead. - Breaking:
--view=fullunder--analyze=linesor--analyze=codeno longer includes DOCUMENTS, which needs words analysis: the report lists it inomitted_viewsand saysnote: full omits documents without words analysiswithtip: include them: --analyze all. A script that read the DOCUMENTS section of such a report addswordsto--analyze. In Rust,ViewSpec::full_reportpartitions the same way, keeping a view only when the analyzers include everything it implies:full_report(AnalysisSet::LINES_ONLY)andfull_report(AnalysisSet::CODE_ONLY)now returnDocumentsamong the omitted views rather than the shown ones. - Breaking:
fdu_core::query::Requestgains a private field recording which analyzers the caller named and which views implied the rest. Rust code outside the engine crate builds a request withRequest::new,Request::read, orRequest::buildrather than a struct literal, and cannot destructure it exhaustively; its public fieldsbasis,query, andnoware unchanged.Request::implied_by()returns the views that implied analysis. - Breaking:
fdu_core::query::RequestErroris now#[non_exhaustive], so a later refusal is an additive change; code that matches it exhaustively adds a wildcard arm, and code that renders it withmessageorDisplayis unaffected. - Breaking:
RequestError’s analysis refusals change shape.ViewNeedsContent(ViewSpec)andNeedsAnalyzer { item, analyzer }are removed: a view the basis cannot answer isViewNeedsAnalyzer { view, held, holder }, and a metric sort without its analyzer isSortNeedsAnalyzer { metric, analyzer, held, holder }, whereanalyzerandheldareAnalysisSetvalues andholderis the newBasisHolder: whether a request it built, a basis supplied whole, a retained index, or an opened root fixed the analyzers.WatchContentbecomesWatchContent { named, views }, the analyzers the caller named and the views that implied the rest.AnalyzerNamedAsView::suggested_viewis aViewSpecrather than a&'static str;suggested_view.label()is the old string.ViewNeedsAnalyzer,SortNeedsAnalyzer,WatchContent, andAnalyzerNamedAsVieware each#[non_exhaustive], so a field one gains later is an additive change: code outside the engine crate matches them with.., as inViewNeedsAnalyzer { view, holder, .. }orWatchContent { .. }, and cannot construct them. - The messages around analysis name the exact remedy:
note: code analysis not shown by summarywithtip: show it: --view code, andnote: full omits code, documents without analysiswithtip: include them: --analyze all. Analysis that some selected view shows and some does not is named too:--analyze=code --view=documentssaysnote: code analysis not shown by documentswithtip: show it: --view documents,code. A refusal for missing analysis names a remedy its route accepts: an index names what it was opened with, an opened root says it runs no analyzer, and only a request that can add an analyzer is told to. A watch is the exception for now, because it checks a metric sort, and in Python a content view, before it refuses analysis:fdu . --watch --sort=code_linessaysadd --analyze code, andIndex.watch()says its index was opened withanalyze none, which sends the caller to reopen it with an analyzer that a watch then refuses. A refused watch names every axis that enabled analysis:fdu . --watch --analyze=words --view=codesays--analyze words and --view code need words and code analysis. - Human reports hold only the result: rows, column headings, and multi-view section
headers. Every explanation is a
note:ortip:line on stderr after it, including what used to sit inside the result: the percentage denominator (Percentage column: code linesis nownote: percentages are shares of code lines), share-floor omissions, a single view’s row bound,Ranked by, and the code overview’s coverage lines. - Grouped rows stack their measures.
In
types,families,languages, anddocuments, a row’s first line ends at its file count, and each further measure is a line of its own indented under it: lines with their breakdown, words with pages, the generated and vendored counts, and each reason files were not analyzed. A row with only a file count is unchanged. On the Linux kernel source the standard output offdu . --view code,documentsis now at most 88 columns wide, where its widest DOCUMENTS row was 154; theperf:line on standard error is still longer. Machine formats are unchanged. - Human grouped rows no longer show a documentation count (
4,063 documentation). It counts files under adoc,docs, ordocumentationdirectory or named like a README, CHANGELOG, or CONTRIBUTING file, which in DOCUMENTS is nearly every file, and it read as a measure of the documents. Machine output keeps it unchanged: each row’sdetection.flags.documentationcount and each file’sclassification.flags.documentation. - Notes and tips are consolidated: one note for what totals include, one listing every
display limit that hid something (
note: display limits: below 1% of root, depth 5), and one runnable tip that lifts them all (tip: show more: --min-share=0% --depth=all; in Python,min_share=0%, depth=all). Beside other views, a note names the section it is about (note: percentages are shares of code lines (CODE), document words (DOCUMENTS);2 rows below min share in TYPES), and sections that share a denominator are listed under it once (code lines (CODE, LANGUAGES)). The flat-format notes are shorter (note: result stale, incomplete). - Breaking:
fdu_core::query::AxisNamesgains a public field,setting_separator, that joins several settings in one suggestion (" "for flags,", "for keyword arguments).
documentspercentages add up to 100%, within display rounding. Each row’s share divided its document words by the total’s, but logical words are derived after pooling, so a pooled total of mixed formats differed from the sum of its rows in either direction: with a long-token format such as HTML beside ordinary prose the shares summed past 100% (101.6% on the Linux kernel), and with very short tokens they could sum well below it. Shares in every grouped view now divide by the sum of the rows’ values, which is whatshare.denominatorand the total row’ssharecarry in structured output; the total row’s owndocument_wordsandpagesare still the pooled count. Byte, code-line, and raw-word shares are unchanged, since their sums equal the total.
fdu 0.3.0 is a breaking release focused on Linux speed and stability.
On Linux the default fdu PATH tree is faster, and uses much less memory on a large
tree, and classifying entries against .gitignore is faster again.
On Linux a stat of a directory’s child no longer mounts an unmounted autofs trigger, on
any route. Four Rust API changes are breaking: counters::Counts gains three public
fields and is non-exhaustive, Error gains a variant, UnrepresentableTotal,
scan::ScanBackendDiagnostics gains three public fields and is non-exhaustive, and
content::CoverageReason gains a variant, TextOnly. A .gitignore that starts with a
byte-order mark, or holds a NUL byte inside a line, now reads as git reads it, which
changes the .gitignore semantics version: a snapshot written by an earlier release is
rebuilt rather than served.
A Markdown file over 64 MiB is counted as plain text under the words unit, and its row
and report say so; content analysis of a one-line source or a large Markdown file no
longer holds the file in memory.
No command-line option, report or cache schema, or Python API changed; a report’s
coverage map has one more possible key, text_only. The GitHub release text is
docs/project/release-notes/0.3.0.md.
FDU_COUNTERS=1reports three more rows in itscontrol stategroup:ignore patterns tested, the.gitignorerules whose glob ran against an entry;ignore bucket probes, the lookups of an entry’s name in an ignore file’s indexed rules; andignore bucket hits, those lookups that found a rule matching the entry.
- Breaking:
fdu_core::counters::Countsgains three public fields,ignore_patterns_tested,ignore_bucket_probes, andignore_bucket_hits, which carry the rows above, and is now#[non_exhaustive], so a later counter is an additive change rather than another breaking one. Rust code outside the engine crate that builtCountswith a struct literal, or destructured it exhaustively, must build it withCounts::default()and read or assign its fields instead; code that only reads its fields is unaffected. - Breaking:
fdu_core::scan::ScanBackendDiagnosticsgains three public fields,linux_dents_attempts,linux_dents_successes, andlinux_dents_fallbacks, which count the Linux native reader’s listings on every route that lists directories; they areNoneoff Linux and on a Linux build without glibc. TheFDU_SCAN_DIAGNOSTICS=1trace’sbackendobject carries them aslinux_dents_attempts,linux_dents_fallbacks, andlinux_dents_successes, ahead of the macOS keys, anddirs_readequals the native successes plusportable_directory_reads; no existing key changes meaning. The platform tuning guide documents them. It is now#[non_exhaustive]too, for the reasonCountsis: Rust code outside the engine crate can read its fields but can no longer build it with a struct literal or destructure it exhaustively, and a later backend field is an additive change. - Breaking:
fdu_core::ErrorgainsUnrepresentableTotal { path, counter }, which every route returns for a tree, orIndex::applyfor a batch, whose total nou64can hold (below); code that matchesErrorexhaustively must name it or use a wildcard arm. - Breaking:
fdu_core::content::CoverageReasongainsTextOnly, the coverage of a words record whose Markdown file was over the exact bound below and was counted as plain text; such an outcome carries a value, asAnalyzeddoes, andAnalyzerCoveragecounts it as analyzed. Code that matchesCoverageReasonexhaustively must name it or use a wildcard arm. These are the only public API changes. - On Linux the default tree is faster again on directory-dense trees: 8.9% faster on a
node_modulestree and 3.1% faster on the Linux v6.12 source tree, on top of the gains below, in one paired cell of 20 pairs per tree (exp-201). A one-shot tree report takes each directory’s and symlink’s kind from its parent’s listing and no longer stats it a second time, since the tree reads none of their own attributes;--one-filesystemkeeps the stat for the device number, and a listing stats its children until one stat succeeds, which proves the directory searchable (see Fixed). That removes thestatxof every directory and symlink except one a listing starts with: 7,886 of the 79,961 on thenode_modulestree (exp-197). The tree also decides from each child’s totals whether the child will show before building a row for it, and frees its index on a separate thread rather than the one that prints the answer (exp-199). No answer and no public API changed. - The default summary is faster on trees with many
.gitignorefiles: on Linux, 6.2% faster on the Linux v6.12 source tree, which has 358 of them, in one paired cell of 20 pairs (exp-198; 6.1% stacked with the other changes in exp-201), and unchanged on a tree with none. The summary compares each entry’s parent directory by its bytes rather than by path components, and finds the ignore files that apply to a directory through an index keyed by the directory’s bytes; each.gitignoreis read into a buffer sized from its length rather than grown from 32 bytes. No answer and no public API changed. - On Linux the default tree is 39% faster on the Linux v6.12 source tree (200 to 120 ms)
and 10% faster on a
node_modulestree (127 to 114 ms) than on 0.2.1’s engine, and the default summary 26% and 7% faster, each in one paired cell of 20 pairs (exp-194, exp-195). The changes below account for it. - When a one-shot tree report needs nothing but the tree, as the default
fdu PATHdoes, fdu no longer keeps every file in its index: it keeps every directory and only the largest files that could reach the smallest share the tree prints, and folds the rest into their directory’s totals. The answer is unchanged. On a generated million-entry Linux tree the default tree’s peak memory fell 79%, from 292 to 62 MiB, and its wall time 9% against 0.2.1’s engine, in a 12-pair screen (exp-194). - Classifying entries against
.gitignoreis faster again. Each ignore file’s rules are indexed by literal name, extension, and suffix, anchored rules are grouped by depth, and the rules left over are ruled out by cheap byte checks before any glob runs. On the Linux v6.12 source tree full glob evaluations fell from about 110 per entry to 0.0019 (exp-178), and what.gitignorehandling adds to the default tree fell from 52% of a walk without it to 1.6%. What the rules match is unchanged, and so isIGNORE_RULES_VERSION; fdu’s ignored set agrees withgit check-ignoreon every path of that tree. - On Linux builds against glibc, every walk, revalidation, reconciliation, and opened
discovery reads each directory with
getdents64into a reused buffer and stats its entries withstatxrelative to the directory, instead of through the standard library. A directory it cannot read that way is read the portable way; musl builds use only the portable reader. On the Linux v6.12 source tree the default command’sfstatcalls fell from 5,773 to 4, and the default summary was 9–10% faster on both real trees (exp-185, exp-186).
- A directory that is readable but not searchable (mode
0400) is reported the same way on every route: each of its children as an error, and none of them as an entry. In 0.2.1 the default--view summarycounted such a directory’s subdirectory as a directory and reported it a second time when it failed to open, and reported nothing at all for its symlink, because it took both kinds from the parent’s listing without the stat that fails there, while the full index reported each child once as an error. A listing now takes directory and symlink kinds from itself only once one of its children’s stat has succeeded, which proves the directory searchable; the saving above keeps all but one stat per listing that starts with directories or symlinks. - A tree whose apparent or allocated bytes sum past what a
u64can hold now fails withError::UnrepresentableTotalon every route, andfduexits 1 naming the file at which the total left the range. A filesystem can produce one: tmpfs, XFS, and btrfs let anyone create a sparse file that claims 8 EiB apparent and allocates nothing, so three of them in one directory are enough. Earlier releases panicked in debug builds and wrapped the total in release builds, sofdu PATHreported 8,191 PiB for that directory. A one-shot report, whether it folds a summary or builds a full or folded index, keeps one checked running total at the root, which bounds every directory’s total beneath it.Index::apply, and every other route that commits a batch of observations, refuses a batch that would carry a whole-tree total of files, directories, apparent bytes, or allocated bytes out of range before it applies any of the batch: the index, its clock, and its journal are as they were. A batch is applied in order, so the index must be representable after each of its operations; a replacement, a kind change, or a removal that makes room in the same batch counts. A snapshot whose recorded sizes sum pastu64is now refused as corrupt when it is loaded rather than summed. AScanReport’sbytes_walkedandallocated_walked, which measure the walk rather than answer it, saturate atu64::MAXrather than wrapping. - A
.gitignorethat starts with a UTF-8 byte-order mark now applies its first rule, and a pattern ends at the first NUL byte inside its line, as git reads both. Earlier releases matched the mark as part of the first rule, so it never applied, and kept the bytes after a NUL as part of the rule. The.gitignoresemantics version is 4, so a snapshot written under 3 is rebuilt rather than served. --analyze codeno longer holds a whole logical line in memory: the line classifier now runs in pieces over a 64 KiB window as a line arrives, carrying its lexer state and the facts the whole-line classifier read from the line, and a site whose lookahead the window’s edge cuts waits for more bytes rather than deciding on a cut token. A one-line minified or generated source therefore costs a worker the window, not the file. Every count is unchanged: a differential test holds the streaming scan to the previous whole-line classifier, kept verbatim as the oracle, for every supported language, comparing the cumulative counts after every line rather than only the file’s totals. It runs nine window sizes from one byte, which puts a piece edge at every byte, to the full 64 KiB; two-way splits of the source at every seventh byte, at windows of one and three bytes; and every two- and three-way chunking of a short mixed source, at a window of one byte. Its sources hold CRLF and lone CR, splices, byte-order marks, invalid UTF-8, Unicode whitespace, no trailing newline, and tokens longer than the window.--analyze wordsno longer holds a Markdown file of any size in memory to render it: a Markdown file over 64 MiB is counted as plain text instead, in the same 64 KiB chunks as any other text file, and its record says so. TheCommonMarkparser needs the whole source, because list tightness, headings, and link references resolve across the document, so an exact rendered count costs a worker the file and a parse beside it: a few hundred megabytes at most up to the bound. The largest Markdown files in common corpora are a few megabytes, so every real file is still rendered exactly, and the bound exists for the generated or concatenated file that would otherwise exhaust memory. Such a file counts every word as visible and each blank-line run as a paragraph; its row carriescounted as text, machine output carries it undertext_onlyin the words coverage map, and the report notes how many of the files its views show were counted that way and how. The bound is fixed: no option lifts it, which makes it the one bound without a flag. The design principles record it as a deliberate exception with that reason; lifting it needs an analyzer option in the content identity, which the request model does not have yet. A Markdown file at or under the bound is rendered exactly, as before, and a file of any other type retains at most its 16 KiB classification prefix and one chunk, which a test now holds it to. Themarkdown-prose-v1analyzer is version 2, so a cached words record from an earlier release is analyzed once more; the analysis sidecar’s format is unchanged.--analyzeno longer materializes every candidate file before its first read. A candidate holds two paths and a classification, so a million-file tree cost hundreds of megabytes of scheduling memory that the bounded worker channel then drained one at a time. Candidates are now walked in batches of 4,096 over one resumable walk of the index, which resumes a directory’s listing where the last batch stopped, and are read by one set of workers for the whole pass, from a queue of at most one batch, so no worker waits at a batch’s end for the slowest file in it; the count is taken first, without building any, so the progress denominator is exact as before, and every result is still applied conditionally on the revision and fingerprint its candidate carried. Answers, records, and counts are unchanged.fdu --watchno longer repaints its aggregate views when nothing a reader sees has changed: a touch that leaves a file’s size alone, or a change to an entry the selection leaves out, moves the index but printed the same tree again under a new timestamp. A repaint’s identity is what the format renders of the answer with its generation instant held fixed, plus its tree status, its source and freshness, and the notes, tips, and warnings written beside it on stderr, so machine formats still repaint when a modification time they carry moves, and a retained observation gap, a coverage change, or a new note, such as a.gitignorerefused for its line limit on an otherwise unchanged tree, repaints on every format. A session keeps a 128-bit digest of that identity rather than a copy of the rendered answer. Change records are never deduplicated. The rule is the engine’s,watch_session::Session::changed_report, which is new.- A C header is no longer classified as C++ because a C++ keyword appears inside an
identifier, a comment, or a string literal:
struct pid_namespace *holdsnamespace, and kernel comments discuss namespaces and templates freely. The probe now countsnamespaceandtemplateonly where they open a line andstd::andconstexpronly as whole tokens, outside comments and literals. On the Linux v6.12 source tree 168 of 25,308 headers (0.66%) were labelled C++; line counts are unchanged, only the language breakdown moves. - A modeline names a language as a whole token:
mode: conf-colonandmode: confno longer classify a file as C,ft=cssis notcs, andmode: gomodis notgo. Linux’sDocumentation/docutils.confwas counted as C code. Emacs’smode: shell-script, its documented name forsh-mode, is shell, and a Vim compound filetype names the language of its first component, as Vim applies it first:ft=c.doxygenis C. - On Linux, no stat of a listed child triggers an automount, on any route or with any
worker count: an unmounted autofs trigger directory (
/net,/misc, a systemd automount unit) is reported as the trigger, aslstat, GNUdu,dut, andbfsreport it, and a walk no longer mounts, or hangs on, a network filesystem it would not descend into. 0.2.1 and earlier mounted the trigger by statting it, because the standard library stats withstatxand withoutAT_NO_AUTOMOUNTon glibc. The walk root itself is resolved: listing it mounts it in any case, and--one-filesystembounds the walk to the filesystem it finds there. musl builds were never affected: the standard library stats withfstatatthere. The answer on a tree without automount triggers is unchanged (exp-196 records the non-regression screen). fdu --watchkeeps its progress line up while its first answer is built. The line stopped when the initial scan and its save returned, before the answer was built, so a heavy view such as--view fullover a large tree showed nothing for the seconds that build takes. The build is now drawn asSummarizing, as a one-shot report’s is, and the line is erased once the answer exists, before it or anything else is written. The entry point is the engine’s,watch_session::Session::changed_report_with_progress, which is new; the answer and every later repaint are unchanged.- Built from the source distribution inside an unrelated git repository, such as a home
directory kept in git,
fdu --versionand the installed skill’s “Generated by” line named that repository’s commit. A build now stamps a revision only when the repository tracks fdu’s own manifest.
fdu 0.2.1 is a patch release focused on Linux.
The default fdu PATH and --view summary are about three times faster on source trees
with many .gitignore files, the default summary reports its ignored share without
building an index, and building the full index is faster on trees of many small
directories. A .gitignore spelled in another case applies where git applies it, and a
--stale-ok answer always says it is stale.
No command-line option, schema, or Rust or Python API changed incompatibly.
The GitHub release text is
docs/project/release-notes/0.2.1.md.
- Python
Report.warningsand Rustreport_format::report_warningscarry what a reader must not miss about an answer, between its notes and tips. Today that is the stale-answer warning below; a verified answer carries none.
- A
--stale-okanswer now says it is stale on every format, plain text included: stderr carries awarn: stale answerline that names--stale-okas the way to a fresh answer, and--quietkeeps it. Plain text used to mark it only withcache onlyat the end of the performance footer, which--quietremoves. Paths and long output no longer print a separatenote: cache-only resultline, which the warning replaces. Machine output is unchanged;provenance.sourceandfreshnessalready said so. - The default
--view summaryno longer builds the full index to report its ignored share: it classifies each entry against.gitignoreas it counts it, keeping only the rules and the heads of ignored subtrees. The answer is unchanged, on every surface. On macOS peak memory fell 69% on a 137k-entry source checkout and 58% on a 77k-entry tree with no.gitignore, with wall time no worse. On Linux it took 19% less time and 97% less peak memory (314 to 8.6 MiB) on a million-entry tree with no.gitignore, and 7% less time and 23% less memory on the Linux v6.12 source tree. - On Linux, building the full index on a tree of many small directories is 7–11% faster:
the index builder returns each directory listing to the walker thread that allocated
it, so glibc no longer frees it across threads.
Measured on a million-entry generated tree (−10.6%) and a
node_modulestree (−8.6%); trees with few, larger directories are unchanged. - Classifying entries against
.gitignoreis about three times faster on trees with many ignore files, which is what the defaultfdu PATHand--view summarydo. Matching no longer allocates, and each directory’s governing ignore files are resolved once for all its entries instead of once per entry. On the Linux v6.12 source tree (358.gitignorefiles) the default tree fell from 590 to 211 ms and the default summary from 505 to 167 ms; answers are unchanged.
- A
.gitignorespelled in another case, such as.GITIGNORE, now applies where git applies it: on a case-insensitive volume (APFS and NTFS by default, an ext4 casefold directory), where opening<dir>/.gitignorefinds it. Before, fdu took rules only from a file listed exactly as.gitignore, so its gitignored shares differed from git’s on such trees. On a case-sensitive volume nothing changes: a.GITIGNOREthere is an ordinary file, as it is for git. Every route and surface agrees, including the default summary,--ignored,--watch, opened roots, and refresh; the rules are named.gitignorewherever they are reported. The engine fingerprint now mixes in the version of these rules, so nothing cached under the old rule is reused: each cached tree’s first run after upgrading scans cold, and its first analyzed run reads every file again.
fdu 0.2.0 makes the command people type cheaper and narrows what the cache does by
default. --cache takes auto, on, or off: under auto a one-shot metadata report
no longer writes a snapshot that no later one-shot report reads, and --stale-ok
replaces --cache only. Code analysis gains an aligned code table and population
control, tree output gains display bounds, multi-view reports resolve each file once,
and a large one-shot index is freed after the answer instead of before it.
Several flags, the report and cache-status schemas, and the macOS cache location changed
incompatibly; Upgrading from 0.1.0 says what to change.
The GitHub release text is
docs/project/release-notes/0.2.0.md.
- On macOS,
--watchand an opened root report a file held open for writing, such as a growing log or a SQLite write-ahead log, as of its last close. The kernel emits a content event for descriptor writes only when the last descriptor closes, and a watcher now reacts to the events it gets. Earlier, the whole-tree reconcile that every rename triggered also re-read such files; quiet trees already behaved this way. A one-shot report, or an explicit refresh from Rust or Python, still reads their current size. - Breaking:
--cachetakesauto,on, oroff, andautodepends on the kind of request. A one-shot metadata report underautono longer writes a snapshot, since no later one-shot report reads it; content analysis,--watch, and an opened index still read, revalidate, and write.--cache onalso writes after a one-shot metadata report. Python’sCachePolicyisAUTO,ON,OFF; Rust’sCachePolicyisAuto,On,Off, andPlan::persistsreplacesCachePolicy::writes.- A default
fdu PATHtherefore no longer leaves a snapshot for a lateropen,--watch, or--analyzerun of the same root, and that run starts with a walk unless an earlier run used--cache onor was itself one of those runs. A one-shot metadata report never read the snapshot: revalidating it stats every entry anyway, so it cannot make a metadata report cheaper, and on Linux it measured slower than a fresh walk (usage guide, platform tuning guide). What the missing snapshot costs the later run is small: on the million-entry Linux tree, anopenor watch that loaded one reached its first answer 0.22 s sooner (1.00 s against 1.22 s, one probe), and a repeated--analyzekeeps its savings in the content sidecar, which analysis runs still write.
- A default
- Breaking:
--cache onlyis now--stale-ok(Pythonstale_ok=True, RustDelivery::stale_ok), which answers from the snapshot without touching the tree. Leave a snapshot for it with--cache on. Rust callers that build aDeliveryby struct literal must now namestale_ok, or build it withDelivery::new(cache, cache_path)orDelivery::stale_ok(cache_path). - Breaking:
--ignored=include|exclude|onlycontrols population, replacing--exclude-ignoredand--only-ignored. Exclusion prunes safely ignored subtrees and skips their content analysis; only analyzes ignored bodies. - Tree output defaults to depth 5 and a 1% share of the selected root, with significant
file leaves.
--breadthbounds children;--limitbounds rows per section. - Code analysis defaults to a source-line overview with language and population totals, coverage, and metric sorting. The code table aligns per-language counts and a bold TOTAL row; display limits preserve complete totals. Correct multiline literals, heredocs, Rust lifetimes, and JavaScript regex handling; invalidate prior code-analysis records.
- Human output uses consistent names, primary totals, and gray parenthetical details. A code overview row’s parenthetical names only its gitignored share, and any share whose ignore status is unknown, never the non-ignored remainder. Performance includes ignore-file and rule counts plus total files/s and represented GiB/s.
- Breaking:
--workerssets content-analysis concurrency (zero selects available parallelism), replacing--analysis-workers. - Breaking: Cache data uses
<key>.metadata.binand<key>.analysis.bin. macOS and Linux default to~/.cache/fdu, so the macOS default moved from~/Library/Caches/fdu;--cache-dirandFDU_CACHE_DIRselect an exact destination. - Breaking: Report schema is
fdu.report/10; cache status isfdu.cache/3. Python exposes matching population, code overview, display limits, and cache destination controls. - A report with two or more unfiltered metric views (
types,families,languages,documents) resolves each regular file’s content record and path classification once and feeds every view from one pass, instead of once per view (H153). Output is unchanged; an exact oracle compares the combined result with separately built single-view reports. An exploratory macOS measurement found 100 four-view report constructions about 2.5 times faster on a 137,085-entry tree (exp-159). It is not a scan or end-to-end speedup, the default command and single-view reports do not take this path, and the change is retained provisionally until a quiet rerun settles its inconclusive major-fault gate. - A one-shot report frees an index of at least 65,536 entries on a detached thread,
named
fdu-index-release, after the answer is complete, instead of before returning it (H156). On the million-entry Linux tree that release was 95 ms of a 1.39 s--cache offreport, and the default tree job took 3.19% less wall time (95% interval 1.79% to 4.88% less, exp-160). Library callers ofprepare_reportor Python’sfdu.reportget the answer sooner and see the memory return moments after the call rather than before it. The release stays inline on Windows, withFDU_COUNTERS=1, for smaller indexes, and when the thread cannot be started. - The
fducommand installed from the Python wheel starts about 44 ms faster:fdu --versiontakes 22 ms rather than 65 ms on an Apple silicon Mac. The package imports its Python API on first use, so the command, which hands its arguments to the native command line, loads only the native module.
- Breaking:
--cache refreshand--cache read-only, with their Python and Rust counterparts. Each retired value is refused with its replacement named.read-onlyhas no exact replacement: reading a snapshot without ever writing one is gone. Underauto, a failed snapshot write is reported as a warning, so a read-only cache directory still answers;--cache offreads nothing.
- A rename no longer makes
--watchor an opened root reconcile the whole tree. The event backends report each side of a rename as its own event, and every one of them re-walked the root: on a busy 476k-entry agent-state tree on macOS, where atomic temp-file writes rename constantly, that was a full walk every 20 s, 48% of a core, and 6.9 GB of snapshot rewrites an hour. Each side is now verified as its own path, like a create or a remove: a vanished name is removed with its subtree, a present file is updated, and only a directory that arrives by rename is relisted, bounded by that directory. A rename that changes only case reconciles its parent. Kernel event loss, a rename of the root itself, and kqueue renames still reconcile the root. The removed root walks also happened to re-read files held open for writing, whose writes macOS reports only when they close; on macOS such a file is now current as of its last close.
- Command lines. Replace
--cache onlywith--stale-ok,--cache refreshwith--cache on, and drop--cache read-only(see Removed); fdu refuses each retired value and names its replacement. Replace--exclude-ignoredwith--ignored=exclude,--only-ignoredwith--ignored=only, and--analysis-workers Nwith--workers N. These three are refused as unknown arguments with exit status 2, and the parser’s suggestion does not name the replacement: for--exclude-ignoredit suggests--exclude, a different option. A script that relied on a one-shot report to leave a snapshot, for a later--stale-okread or a warmopenor--watch, adds--cache onto that report. - Machine output. A consumer pinned to
fdu.report/7moves tofdu.report/10, and one pinned tofdu.cache/2moves tofdu.cache/3;fdu.stream/2is unchanged. The machine-output reference describes the current shapes. - Cache files. 0.2.0 neither reads nor removes the files 0.1.0 wrote.
Each cached tree’s first 0.2.0 run walks it, as after every upgrade, because the
engine fingerprint mixes in the crate version.
On macOS without
XDG_CACHE_HOME, 0.1.0 wrote to~/Library/Caches/fdu, which 0.2.0 never looks at: delete that directory. Elsewhere the old files share 0.2.0’s directory under their old names,<key>.fduand<key>.content;fdu --cache-status=alllists them as unrecognized, andfdu --cache-clear=allleaves them in place. Delete them by hand, or run 0.1.0’sfdu --cache-clear=allbefore upgrading. - Rust.
CachePolicyisAuto,On,Off:OnlybecomesDelivery::stale_ok,RefreshbecomesOn, andReadOnlyis gone.CachePolicy::writesis replaced byPlan::persists. ADeliverybuilt by struct literal must namestale_ok, or useDelivery::neworDelivery::stale_ok. - Python.
CachePolicy.ONLYbecomesstale_ok=Trueonfdu.reportandfdu.open,CachePolicy.REFRESHbecomesCachePolicy.ON, andCachePolicy.READ_ONLYis gone.
The first release. fdu walks a directory tree once and answers, for every directory at
once, how big it is, how many files it holds, what changed most recently, and what kinds
of files it contains.
It caches that index between runs and can keep it live.
The same engine ships three ways: the fdu command line, the fdu Python package, and
the fdu-core Rust crate, which the fdu crate re-exports.
The GitHub release text is
docs/project/release-notes/0.1.0.md.
- Request model. The command line, the Python package, and the Rust library share
one typed
Request(Basis,Query,now), oneDelivery, the axis grammars, andRequest::DEFAULTS. Allocated size is the default everywhere, includingSizeMetric::defaultand an opened selection that names no metric. A refused request is a usage error: exit status 2 on the command line,InvalidArgumentError(aValueError) in Python.Query.axesis&'static AxisNames, so a refusal names flags or fields in the caller’s vocabulary. - Command line.
fdu PATHprints a size-sorted tree two levels deep with ten rows per directory: the defaultlistview intreeformat. Barefduprints help and scans nothing. Sizes are allocated bytes unless--size apparentasks for file lengths.- Every option belongs to one axis: scope (
PATH,--scan-depth,--one-filesystem), content (--analyze), selection (--include,--exclude,--min-size,--modified-since,--modified-before,--kind,--depth,--limit,--sort,--reverse,--size), view, format, and mode (--cache,--watch,--analysis-workers).--depthand--limitbound only what is rendered;--scan-depthbounds what is scanned. --viewtakeslist,summary,tree,families,types,extensions,languages,documents,largest,recent,files, orfull. Several views in one run share one walk.list, the default when nothing is analyzed, is every matching entry: text draws it as the directory tree, and JSON, JSON Lines, and YAML give it as flat rows.treeis the directory hierarchy, in machine formats too.largestandrecentare the 20 largest and most recently modified regular files;fileslists every matching entry in name order;fullis every view butlistandfiles.- Directories are selected like files.
--kind dir --include node_modules --modified-before 30dfinds stale build and environment directories: a directory’s size is the eligible regular-file bytes below it, and its modification time is the newest on it or any eligible descendant, directories and symbolic links included. Size and modification bounds test that subtree rather than the directory inode, so without--kinda time bound also matches every directory with activity in range. Exclusions win throughout a subtree, nested matching directories are each listed, and aggregate totals count covered contents once. --format text|tree|paths|long|json|jsonl|yaml, with--treeand--longas shorthands, and color decided by--color auto|always|never,NO_COLOR, andFORCE_COLOR.treeis the bounded directory tree;pathsprints every matching path, one per line;longadds each one’s size and signed modification age, measured against one instant for the whole report. A flatlistis complete and size-ranked unless--limitor--sortsays otherwise.treeapplies to thelist,files, andtreeviews, andpathsandlongto those and tolargestandrecent; grouped and mixed views, andfull, usetextor a machine format. Results go to stdout, warnings and errors to stderr, and a one-shot report intextortreeformat ends with a gray performance line.- A run at an interactive terminal that takes longer than half a second shows one
progress line on stderr: the root, the phase (
Loading,Scanning,Revalidating,Indexing,Analyzing,Saving,Summarizing), the counts so far in right-aligned columns, and elapsed time, erased before any output. Its bytes, like the performance line’s, are measured as the answer’s are: allocated unless--size apparent. It is never drawn when stderr is not a terminal,TERMisdumb(or unset, except on Windows), orCIis set to a non-empty value.--progress=auto, the default, also skips JSON, JSON Lines, and YAML;--progress=alwaysdraws it for those too, and--progress=neverturns it off. Ctrl-C while it is drawn erases it, printsfdu: interrupted, and ends the run as an unhandled Ctrl-C would: bySIGINTon Unix, withSTATUS_CONTROL_C_EXITon Windows. - Exit status 0 means a complete result, 1 a failed command, and 2 a partial result or
a usage error.
--allow-partialaccepts a partial result as success. fdu --docsprints the usage guide andfdu --skillprints a portable agent skill, both without aPATHand without scanning. The skill runs anfduonPATHand otherwiseuvx fdu@latest, names the persistent installs (uv tool install fdu,uv tool upgrade fdu,cargo install --locked fdu), and states the build that wrote it, asking to be re-installed whenfdu --versiondiffers.fdu --install-skillwrites the skill to.agents/skills/fdu/SKILL.mdand.claude/skills/fdu/SKILL.mdunder the git root of the current directory, or the current directory outside a repository;--agent-base DIRwritesDIR/skills/fdu/SKILL.mdinstead, for one agent’s user scope. It reports each file as installed, updated, or unchanged, and writes through a staged file and a rename. It replaces only files carrying its generated-by marker, and refuses aSKILL.mdwritten by hand with exit status 2 before writing anything.
- Every option belongs to one axis: scope (
- Machine output. Every report uses
fdu.report/7, watch changes usefdu.stream/2, and--cache-statususes the separatefdu.cache/2document. Reports carry the effectiverequest, structuralstatus, and per-tierprovenance.status.errorsretains bounded structured details andstatus.errors_omittedcounts the remaining failures. Each requested analyzer has its own coverage counts; metrics from unrequested analyzers stay absent. JSON, JSON Lines, YAML, and Python models share the answer shape, including lossless platform-tagged identities for paths that are not valid Unicode. A field change bumps the schema version.- Without
--viewor--analyze, a machine-format report is thelistview: afilesarray holding every matching entry, unless--limitbounds it.--view treegives the directory hierarchy instead. - A list row carries
path,kind,bytes,allocated, a directory’s descendantfilesanddirs,complete,mtime_ns,ignored, and a signedage_nsmeasured against the report’sage_reference_ns. A directory whose subtree was not listed in full, at a--scan-depthboundary or where the scan failed inside it, hascomplete: false, lower-bound sizes and counts, and a null age, and matches no modification bound.
- Without
- Content analysis, opt-in with
--analyze, which takeslines,code,words, a comma-separated set of them,none, orall.linescounts physical, blank, and nonblank lines and raw words;codeaddscode-sloc-v1code, comment, and blank lines for Rust, Python, JavaScript, TypeScript, Go, Java, C, C++, C#, Ruby, PHP, Swift, Kotlin, shell, and SQL;wordsadds normalized words, paragraphs, and derived pages, and reader-visible words for Markdown.- Binary files, unsupported encodings, invalid UTF-8, and code without a shipped SLOC analyzer are reported as coverage and do not make a run partial; an I/O failure or a file that changes while it is read does.
- Classification stays path-only for exact filenames and known extensions.
Only unresolved paths and
.hfiles receive bounded probes for shebangs, modelines, binary signatures, and generated-file markers, and every metric row reports how its files were detected and with what confidence. - A content sidecar keyed by the analyzer set keeps results between runs, so an unchanged file is not reopened; it answers only the analyzer set that wrote it.
--viewnever turns on an analyzer, and a view that displays none of what was read says so.
- Cache. By default a report or open saves a snapshot of its index under the user
cache directory:
$XDG_CACHE_HOME/fduwhen that is set, otherwise~/Library/Caches/fduon macOS,~/.cache/fduon other Unix systems, and%LOCALAPPDATA%\fduon Windows.--cache auto|refresh|read-only|only|offchooses how a run uses it, and--cache-status[=root|all]and--cache-clear[=root|all]inspect and remove snapshots without scanning.- A snapshot serves only the scan scope that wrote it, and a corrupt, truncated, or
unrecognized file is treated as absent.
Unix snapshots are created owner-only (
0600). - Cache status reports every file in the cache directory with a
state:current,stalefor a snapshot another fdu version or format wrote or a header this build cannot read,leftoverfor one of fdu’s own files left by an interrupted write,unrecognizedfor a file that is not fdu’s, andabsentfor a root with no snapshot. A stale row names why (older_format,newer_format,other_engine,unreadable) and a leftover row what it is. --cache-clearand--cache-clear=allremove stale snapshots as well as current ones, so an upgrade’s invalidated snapshots are reclaimed rather than deleted by hand, and--cache-clear=allalso reclaims fdu’s own leftovers: a staging file a killed writer never renamed, once it is older than the age its reaper uses, and a content sidecar whose snapshot is gone. A file is removed only if its name and its magic both say it is fdu’s; a symbolic link or a directory is never followed or removed, and an unrecognized file is always left in place and reported.- An unfiltered
--no-gitignore --view summaryis the one composition that retains no index and writes no snapshot. Reading.gitignore, which is the default, needs the index to classify entries, so an unfiltered--view summaryretains it and saves a snapshot like any other report. - Cache-only content restore avoids repeated roll-up, candidate, classification, and path work, streams records into the retained index, and repeated unfiltered metric views share one entry traversal. Individual development comparisons on exploratory, uncontrolled, warm-steady M1 Pro/APFS runs measured roughly 4% to 13% lower wall time for cache-only restore changes and 10% lower peak RSS from streamed restore. A separate workload that included setup plus 100 four-view reports measured 19% lower wall time. These results are not additive and do not measure an ordinary one-shot command or Linux; the performance evidence preserves each workload and interval.
- Watch.
fdu --watchrepeats the same query as the tree changes. Aggregate views repaint at most every--interval, which takes the age grammar, including200ms; the 2-second default is the engine’sWatchDelivery::DEFAULT_INTERVAL, which PythonWatchOptionsuses too.--view files --format jsonlemits onefdu.stream/2record per change. Events are verified by stat, and a backend overflow or rescan request becomes a reconcile of the affected subtree rather than a dropped event. An upsert carriesignored, and so does a removal a rule edit caused; an ordinary removal, an invalidation, and every record of a run that read no rules omit it. A watch is metadata-only on every surface: the command line refuses--analyzewith--watch, and a RustSessionor PythonIndex.watch()refuses an index opened with content analysis, asError::UnsupportedScanConfigorInvalidArgumentError. The request model also refuses a narrowed scan scope and cache-only, so a library or Python caller cannot ask for what the command line refuses. .gitignoreroll-ups, read by default on every surface. An index keeps ignored and unignored roll-ups for every directory beside the totals, and every report says how much of each size the tree’s own rules ignore.- Every
.gitignoreinside the scanned root governs its own directory and everything below it, and deeper files take precedence. Negation, anchored and directory-only patterns,**, and git’s bracket expressions follow git, pinned against verdicts recorded fromgit check-ignore. Matching is byte-exact and case-sensitive on every platform. - A watch or refresh that sees a
.gitignorechange re-reads its rules. fdu PATH,fdu --watch,fdu.report,fdu.open,fdu.scan,fdu_core::open,prepare_report, and opened roots all observe control state.--no-gitignoreon the command line, andread_controlsoff on RustScanConfigor PythonScanOptions, reads no rules for one request and is a snapshot scope of its own.- An index that did not observe control state says so rather than guessing:
Index::is_ignored,controls, and the partition accessors returnError::ControlStateNotObservedinstead of calling every entry unignored, and a row from such a run carriesnullrather than “nothing ignored”. - Text summary, tree, and extension rows end with the ignored part of their size, as
(128 B ignored), and leave it off a row with nothing ignored. The performance line counts the rule files read, asignore rules 1 file, or saysno ignore rules. - In machine formats summary and tree rows carry an
ignoredobject (files,dirs,bytes,allocated), extension rows one withoutdirs(files,bytes,allocated), and file rows anignoredflag; each isnullwhere no rule was read. --exclude-ignoredand--only-ignored,Selection::ignoredin Rust andSelection(ignored=...)in Python, report one side; sizes, ordering, and--min-sizefollow the entries shown. Either filter over a run that read no rules is refused, not answered with an empty report: a usage error on the command line,InvalidArgumentErrorin Python, andError::ControlStateNotObservedfromprepare_reportand a watch session.- A
--watchstream maintains the entry set its selection names, so a rule edit that moves an entry into--exclude-ignoredor--only-ignoredstreams the upsert that draws it and one that moves it out streams the removal, even though nothing about the file changed on disk. - A
.gitignorethat cannot be read makes the result partial, exit status 2 unless--allow-partial. - Two independent limits bound the rules an index retains, each a size or
unbounded, and each liftable without moving the other.
--gitignore-budget SIZE|allbounds the retained control state, 4 MiB by default;--gitignore-line-limit SIZE|allbounds one pattern, 16 KiB by default. They areControlLimits { budget, line_limit }onScanConfig::control_limitsandOpenOptions::control_limits, andcontrol_budgetandcontrol_line_limiton PythonScanOptionsandOpenedOptions. Both are part of the snapshot scope. - A
.gitignorepast either limit is refused rather than ending the scan: its rules do not apply, every size stays exact, and the result stays complete with exit status 0. An opened root completes the directory and keeps watching, and a watch keeps applying events. Identical.gitignorecontents are stored and charged once, so a tree of package checkouts spends a fraction of the budget it otherwise would. - A refusal is reported, not silent.
Reports carry
ignore_rulesin every machine format:nullwhen the run read no rules at all, otherwiselimits(budgetandline_limit, each bytes ornull), theappliedandrefusedcounts, andrefusals, each with apathand areasonnaming the limit that fired. A text note names the refused files’ directories and the flag that lifts each limit that fired.Index::control_coverageandReadDiagnostics::controlscarry the same, Python mirrors both, andEffectiveChange::ControlRefusalUpdatedreports a refusal recorded or lifted.
- Every
- Python package.
import fdugives typed, immutable options, reports, roll-ups, provenance, cache values, and change feeds.fdu.reportanswers one query while retaining the least state it needs;fdu.openandfdu.scanreturn anIndexwithtotal,rollup,children,provenance,report,refresh,since, andwatch.- Under the default
autocache policy,Index.refresh()saves the refreshed metadata and content, and iteratingIndex.watch()saves verified changes. A save that fails during a watch issues a Python warning rather than ending the feed. Report.as_dict()returns the command line’s JSON. A row’signoredis anIgnoredTallyonSummaryRowandTreeNode, anExtensionTallyonExtensionRow, and aboolonFileRow, eachNonewhere no rule was read.Status.ignore_rulesreports the limits, what was applied, and what was refused.Query.formatchoosesFormat.TREE,PATHS, orLONGbefore reading, andReport.render()serializes a report as the command line does. Rendering a folded tree as a flat list, or a flat list as a tree, raisesInvalidArgumentError; request another report from the index instead. A defaultQuerykeeps the directory tree, so itsas_dict()carries atree; setQuery.formattoFormat.JSONfor the flat rowsfdu PATH --format jsonprints.cache_path,cache_status,list_caches,clear_cache, andclear_all_cachesmanage snapshots. ACacheStatuscarriesstate, withstale_reason,format_version, andleftover_kindwhere they apply.clear_cachereturns whether it removed a snapshot, andclear_all_cachesreturns aClearSummarycounting the snapshots it removed and the leftovers it reclaimed.- Native calls are bulk, and open, scan, and refresh release the GIL. A failure that
stops an operation raises an
FduErrorsubclass, and one that makes a scan partial is reported onStatus. - The wheel installs an
fduconsole script that runs the native command line,--install-skilland--progressincluded. The script restoresSIGINTto the default disposition before entering the native CLI, so Ctrl-C interrupts--watchthe way it does acargo installbinary.
- Opened roots for interactive clients.
OpenedIndex::openin Rust, andfdu.opened.OpenedIndex.openin Python, return while discovery continues.read()answers several projections (Lookup,RollUp,Tree,Flat,Aggregate,Report,Continue,Diagnostics) from one engine version, with bounded pages and opaque continuations. One projection can be refused, asProjectionResult::Refusedor PythonRefusedResult, while the others answer.changes()resumes an exact change journal from a cursor.journal_capacity_bytesbounds it, 8 MiB by default and at least 64 KiB, and a consumer that falls further behind receives a reset outcome and re-reads.refresh()verifies named paths,prioritize()steers discovery, observation keeps the root live, andclose()joins every worker. A worker panic surfaces asError::OpenedWorkerPanickedin Rust and asOpenedIndexErrorin Python.- Selection inside a read matches the portable path, in which
%is written%25and a byte that is not UTF-8 is percent-escaped, so100%.txtis matched as100%25.txt. Theportable_pathaLookup,Tree, orFlatrow carries passes back as a filter unchanged. AReportprojection’s rows carry only the native path, so a name containing%or a byte that is not UTF-8, taken from one, does not. - Options cover hidden-name pruning with an allow list, special-file exclusion, a file budget, and a custom file-type registry (File Rollup registry schema 3 or 4).
- Rust library.
fdu-coreis the engine, andfdure-exports it alongside the command line.opentakes aBasis, what to scan and read, and aDelivery, how to carry it out: cache policy and location, partial-answer acceptance, workers, and traversal scheduling.planvalidates a request and delivery for one route into aPlan, which fixes how much state the run retains, whether it loads a snapshot, and whether it verifies the filesystem.openreturns anIndexwhose per-directory roll-ups come from pre-computed state, andprepare_reportanswers one report without retaining an index.- A
Progresshandle passed toprepare_report_with_progressorSession::start_with_progressobserves the phases and counts the command line’s progress line draws. report_format::renderreturnsResult<String>, refusing a conversion between a folded tree and a flat list that the report cannot answer.- Producers submit verified observations, and the index commits exact, clocked change
batches that
Index::sincereads. watchis the only build feature: off by default infdu-core, on by default infdu..gitignorehandling is always compiled in.- The minimum supported Rust version is 1.85.
- Published
fdu-corerequirements are caret ranges of the reviewed minimum.Cargo.lockstill pins exact versions for this workspace and forcargo install --locked.
- Packaging. The
fdu-coreandfducrates; thefduPython source distribution; and one CPython 3.12+abi3wheel for each of Linux x86-64 and arm64 (manylinux2014, glibc 2.17), macOS x86-64 and arm64 (macOS 11), and Windows x86-64. Wheels carry type information, license text, and a CycloneDX SBOM.
This applies only to anyone who ran fdu built from a development checkout.
- A cached tree can scan cold once. A snapshot serves only the engine and scope that
wrote it. The engine fingerprint mixes the crate version, the snapshot format (now
version 5), and the versions of the classification rules and of the per-entry facts
revalidation checks; the scope adds the type-rule and
.gitignoresettings. Development builds already carried version0.1.0, but the per-entry facts version was raised on every platform, not only Windows, when Windows validation gained change time and file identity. A snapshot from any development build without that change (anything built frommainbefore pull request #117 merged on 2026-09-23) therefore does not serve 0.1.0, whatever its format, and--cache-statuslists it asstale, witholder_formatorother_engineas the reason. Nor is a snapshot written under other type rules or.gitignorelimits, or one that read no.gitignore, for a run that reads it. That tree’s first run scans cold and replaces the snapshot when the run saves one. The content sidecar, now format 7, carries the same engine fingerprint, so the first analyzed run on such a tree reads every file again. The crate version moves at every release, so each later upgrade costs one cold run per cached tree. fdu --cache-clear=allreclaims the rest. A root that is never scanned again keeps a file this build cannot read, and clearing now takes it:--cache-clearand--cache-clear=allremove stale snapshots as well as current ones.fdu --cache-status=alllists each one asstalewith the reason first, so you can see what a clear would take before running it. Files that are not fdu’s are never removed, and are reported as left in place.- The
gitignorebuild feature is gone. A dependency declaringfeatures = ["gitignore"]fails to resolve; remove it.fdu’s default build features are["watch"]. - Update names a development build used. Flags, report schema versions, and Rust and
Python interfaces were renamed before this release, and
### Addedgives each under its released name. A consumer pinned to a development build’sfdu.reportversion moves tofdu.report/7, and a script that read the directory tree from a machine format without--viewadds--view tree, because the default there is thelistview. Rust callers open with aBasisand aDelivery, and handle theResultthatreport_format::renderreturns.
0.1.x may add fields and variants to public Rust types such as ReadProjection,
ProjectionResult, ProjectionRefusal, LimitedProjection, IssueKind,
ImpactDomain, Error, ReportRequest, TreePage, ReadResponse, RollUp,
Provenance, StateTransition, ReportSource, Attrs, Query, Basis, Delivery,
WatchDelivery, Route, ProgressPhase, ProgressSnapshot, and FileRow. Those
additions are breaking under Cargo’s semver rules for exhaustive types; they land in 0.2
rather than behind #[non_exhaustive] on 0.1.0.
- Memory. fdu builds an exact index that later questions reuse, and every report but
an unfiltered
--no-gitignore --view summaryretains it, so peak memory grows with the entries retained. The 2026-09-16 tool comparison, run on the release candidate over a generated 1,000,001-entry tree, measured fdu’s depth-one tree report, with its cache off, near 285 MiB of peak RSS, against 29 MiB for dumac, 21 MiB for dua, and 641 MiB for dust; each tool was invoked under a comparison contract that returned only a total.--no-gitignore --view summaryretains no index: on that tree it took 4.876 s at 15.0 MiB, against 4.942 s at 285.7 MiB for the default--view summary, with identical totals. The default summary retains the index because classifying entries against.gitignoreneeds it. On a 328k-file checkout with many.gitignorefiles, four runs of the same command peaked at 68, 68, 101 and 128 MiB, against 12 to 14 MiB without the index. The cost is the index’s, so it follows the entries retained and how the allocator grew on that run, and is a range rather than a fixed multiple. - Links. A symbolic link is listed as an entry but never followed, and adds nothing
to a directory’s totals; no command-line option follows links, and a
ScanConfigorOpenOptionswithfollow_symlinksset is refused. A file with several hard links is counted once for each path, whereducounts it once. - Cache retention. Nothing prunes snapshots of roots that are never scanned again,
or bounds the cache directory’s size.
--cache-cleartakes a root or the whole directory, so there is no way to clear only the stale snapshots an upgrade left; and because a snapshot in a newer format is stale to an older build in exactly the way an older one is, running an older build’s--cache-clear=allremoves a newer build’s snapshots. Status names every file and its state before anything is removed. - Cache scope. A snapshot that observed
.gitignorecan serve a request with--no-gitignoreby omitting control state. A request that needs rules cannot use a snapshot that never observed them. Projected reads preserve the stronger snapshot. Changing either.gitignorelimit invalidates a snapshot that observed controls. - Content analysis is one-shot on every surface: the command line refuses
--analyzewith--watch, a RustSessionand PythonIndex.watch()refuse an index opened with analysis, and a refresh reanalyzes after reconciling. SLOC covers 15 languages, with no embedded-language or syntax-tree metrics. Sidecars and coverage are scoped to the analyzer set, so a request for analyzers the stored set lacks reads the files again. - Analysis memory. Unknown-type detection uses a bounded prefix. Code analysis retains its longest logical line, and Markdown word analysis retains the whole input and parser state. Extremely long lines or large Markdown files can exhaust memory; these analyzers do not truncate input or silently skip it by size.
.gitignorefidelity..git/info/exclude,core.excludesFile, and.gitignorefiles above the scanned root are not read, and a nested repository is not a boundary..gitignorebudget. A.gitignoreinside a directory an ancestor’s rules already ignore is still read and charged against the budget, which git never does. Classification stays right, because an ignored parent settles its descendants, but a tree whose ignored directories hold most of its rule files can reach the 4 MiB default and see refusals for rules that could not have changed a verdict.--gitignore-budgetlifts it.- Ignored shares appear in summary, tree, extension, and file rows only;
types,families,languages, anddocumentsrows do not carry one yet, though--exclude-ignoredand--only-ignoreddo filter those views. - Watch backends. A watch uses the platform’s native event backend on every filesystem, with no polling fallback for network filesystems that do not deliver events.
- A watched row’s ignored bit. Under
--exclude-ignoredor--only-ignoreda stream keeps its entry set exact across rule edits. Under the default selection it keeps membership live but does not restate a row’signoredbit after a rule edit, so re-read a listing when the bit itself matters. - Opened roots. A
Treepage can exceed itsmax_workby the width of one directory level. The opened-root types (OpenedIndexand itsReadProjectionprojections,TreePage,ReadResponse, and thefdu.openedtypes that mirror them) are expected to change in 0.2, as the0.xrule allows. - Roll-up metrics are a fixed set; there is no interface for custom per-directory reducers.
- JSON integers. Fingerprints, option hashes, and nanosecond timestamps are JSON
numbers. Values above 2^53 lose precision in JavaScript
JSON.parseand any other IEEE 754 binary64 consumer. Read them as strings, or use a parser that preserves integers, if exact identity matters. - Performance evidence comes mainly from an M1 Pro MacBook with a local APFS SSD. Linux measurements are from virtualized hosts, Windows has none, and CI checks behavior rather than timing.
- Platform coverage. Linux arm64 wheels are cross-built and inspected, not executed, before release. There is no wheel for musl Linux or Windows arm64; those systems build from the source distribution with Rust 1.85 or newer.
- Free-threaded CPython (such as
3.14t) is not supported. It cannot install theabi3wheels, and an installer there falls back to building the source distribution; if uv selects a free-threaded interpreter, pass--python 3.14or--python 3.12.