Repository navigation
report source builds before installing; equip() installs rtools - #2413
Conversation
On Windows, restore() and install() now check up front whether any of the requested packages will be built from source when no Rtools installation is found, and report those packages along with the Rtools series matching the running R. Previously the install failed only on reaching the first package needing compilation. Add renv_platform_record(), a JSON-friendly description of the system R is running on: R version and platform, OS and Linux distribution, BLAS and LAPACK, extSoftVersion(), collation locale, Rtools (Windows), and installed versions of the lockfile's system packages (Linux). For now it is only reported by diagnostics(); where it should be persisted is still open.
On Windows with R >= 4.0, equip() now resolves the Rtools series for the running R from CRAN's Rtools index, reads the current installer link from that series' page (the file names carry build numbers), downloads it, and runs it silently through the shell so Windows can prompt for elevation. The legacy extsoft install is kept only for R < 4.0. Rtools discovery and compatibility now handle series newer than those listed, and the required series is the newest one not newer than R, matching how CRAN describes each release. The platform record gains a Tools section reporting pandoc, Quarto, Java and TeX when found.
Generalize the Windows-only Rtools preflight into a binary availability report: on platforms where binaries are in use, restore() and install() now list, before installing anything, the packages for which no binary of the requested version is available and which contain compiled code. On Windows without Rtools the report keeps its advice on the Rtools series required, now pointing at renv::equip(), and still prompts. Each record is classified as needing no build (cache, cellar binary, repository or P3M binary), a source build without compiled code, a compiled build, or unknown; only unknown records are reported when no toolchain is available, to avoid noise for pure-R packages installed from git remotes.
The shared test cache may already hold the test packages when these tests run after others, in which case the classifier correctly reports that no build is needed.
kevinushey
left a comment
kevinushey
left a comment
There was a problem hiding this comment.
Four issues should be fixed before merging; details are attached as inline comments.
Validation: targeted devtools::test() for install-preflight, platform, and rtools completed with 42 passed and 1 skipped. Additional isolated reproductions confirmed the P3M failure, incomplete preflight records, and cache-policy mismatch. The Windows installer was not exercised end to end.
| } | ||
|
|
||
| # a binary is available from P3M | ||
| if (renv_p3m_enabled() && !is.null(renv_retrieve_repos_p3m_date(record))) |
There was a problem hiding this comment.
[P2] Keep P3M lookup failures from aborting installation
When no repository binary matches, this call can propagate a P3M database refresh/download error out of preflight. Previously, retrieval caught that error and continued to source installation. I reproduced an available local source package failing here; bypassing the new preflight allowed installation to succeed. Catch lookup errors and continue classifying the source package.
| renv_install_preflight_unknown_source(records), | ||
| renv_install_preflight_permissions(library) | ||
| renv_install_preflight_permissions(library), | ||
| renv_install_preflight_binaries(records) |
There was a problem hiding this comment.
[P2] Run binary preflight on the resolved installation set
This runs before install() resolves dependencies and versions. For install() without arguments, preflight receives no records; for install('breakfast'), I confirmed it checks only breakfast although installation resolves four packages. Consequently, compiled dependencies can still fail without the promised warning. Run this check after dependency resolution and filtering to packages needing installation.
| path <- renv_cache_find(record) | ||
| if (nzchar(path) && file.exists(path)) | ||
| return("none") |
There was a problem hiding this comment.
[P2] Honor rebuild and cache settings before skipping packages
An existing cache path unconditionally classifies the package as requiring no build, even with rebuild = TRUE or cache use disabled. The actual installer ignores that cache entry in those cases, so source builds and missing-Rtools warnings disappear from preflight. Pass the installation policy into this check and apply the same cache eligibility rules as retrieval.
- install() now checks the packages left after dependency resolution and filtering, rather than only the explicitly requested records - the cache is only treated as satisfying a package when the installer would use it (cache enabled, no rebuild requested), via the same renv_graph_install_cache_find() policy - P3M database failures during preflight are caught, matching retrieval - resolved descriptions' NeedsCompilation is used when present
- skip records whose resolution failed via the resolution_failed attribute; the graph merges Package into the placeholder, so the previous check let them through - report packages whose need for compilation is unknown separately from those known to need it, since dev packages (e.g. from GitHub) rarely carry NeedsCompilation - list rtoolsNN directories from the root of the system drive, as a bare drive letter is relative to the current directory - prefer the newest compatible Rtools series when several are found - never fall back to a series newer than R itself - only report installed system packages; dpkg also reports versions for packages removed but not purged - ignore JVM 'Picked up' notices when reading a tool's version - share one extSoftVersion() call across the platform record, and skip the duplicate Rtools scan in equip()
- equip() respects an explicitly-requested Rtools installation rather than downloading one which renv_rtools_list() can't see - a tool which exits with an error reports no version - only installers for the requested Rtools series are accepted - the Rtools discovery test ignores installations on the host
the graph installer resolves versions missing from the repositories straight to the source archive, so a P3M binary for that version is never used; classifying such packages as needing no build hid exactly the failure the preflight is meant to report
This reverts commit ab67977.
the graph installer resolved versions missing from the repositories straight to the source archive, so P3M's historical binaries were never used for them. restoring an older lockfile on Windows or macOS therefore built such packages from source, where sequential retrieval had used the binary. this also makes the preflight's P3M check agree with the install
restore(strict = TRUE) only retrieves a record naming a repository URL from that repository; the P3M lookups in the graph and in the preflight now honor that, as the archive fallback already did
matches renv_graph_url_repository_p3m(); install() scopes the pkgType option too, so this only aligns the two call sites
- in strict mode, only the recorded repository is searched for a binary - P3M is only consulted when the repositories don't serve the requested version at all, as in renv_graph_url_repository() - the source-only test now runs with P3M enabled, so it can fail
the preflight re-implemented the resolver's choice between repository binaries, P3M, strict mode and the archive, and kept drifting from it. it now asks renv_graph_url_repository() directly and reads the resolved package type; the resolver's own behaviours are tested in test-graph.R
1 participant
Summary
Improves the experience of
restore()/install()when the requested packages need to be built from source but a build toolchain may not be available, and teachesequip()to actually provide that toolchain.restore()andinstall()now report which of the requested packages are not available as binaries and so will be compiled from source. On Windows, when no Rtools installation is found, the report also names the required Rtools version and points atrenv::equip(). Previously, such installs failed only after reaching the first package needing compilation.equip()installs Rtools. On Windows,equip()now looks up the Rtools series matching the running version of R, resolves the installer URL from the CRAN Rtools pages, and downloads and runs it when no Rtools installation is found. The external libraries previously installed byequip()are now only installed for R (< 4.0.0), since newer Rtools bundle them.equip()is no longer marked\keyword{internal}.diagnostics(). Newrenv_platform_record()(inR/platform-record.R) describes the platform R is running on: R version and platform, OS and Linux distribution, BLAS / LAPACK, external software R was built against, collation locale, the Rtools installation (Windows), versions of pandoc / Quarto / Java / TeX found on the system, and installed versions of the system packages required by the lockfile (Linux).diagnostics()now prints this record.Notes
diagnostics()for now. Whether it should also be recorded in the lockfile (or a separate file) is intentionally left for a follow-up.Testing
devtools::test()over the install, restore, retrieve, preflight, platform, rtools, equip, sysreqs, and diagnostics tests: 406 passed, 0 failed, 21 skipped (Windows-only cases).