Skip to content

fix(crunch): use datetime when products have no start or end datetime - #2441

Open
gmann14 wants to merge 1 commit into
CS-SI:developfrom
gmann14:fix-filter-date-planetary-computer
Open

gmann14 wants to merge 1 commit into
CS-SI:developfrom
gmann14:fix-filter-date-planetary-computer

Conversation

@gmann14

@gmann14 gmann14 commented Nov 7, 2026

Copy link
Copy Markdown

Your checklist for this pull request

Please review the guidelines for contributing to this repository.

  • Make sure you are making a pull request against the develop branch (left side). Also you should start your branch off our develop.
  • Check the commit's or even all commits' message styles matches our requested structure.
  • Check your code additions will fail neither code linting checks nor unit test.

Description

Fixes #2440

SearchResult.filter_date() kept every Planetary Computer product. FilterDate in eodag/plugins/crunch/filter_date.py only reads start_datetime and end_datetime, and skips a check when the value is missing. Planetary Computer's sentinel-2-l2a items only have datetime, so these products have neither property and nothing gets filtered out.

This used to work. Before #1730 the generic STAC mapping filled startTimeFromAscendingNode from $.properties.datetime. Now start_datetime comes from $.properties.start_datetime, which single-acquisition items don't have.

Changes:

  • FilterDate.proceed(): when a product has neither start_datetime nor end_datetime, its datetime is used as both. Products with either one set are filtered exactly as before.
  • FilterDate.sort_product_by_start_date() and FilterLatestIntersect.sort_product_by_start_date(): use datetime when start_datetime is missing, instead of sorting the product as 1970-01-01. FilterLatestIntersect had the same problem, so with Planetary Computer results it didn't sort by date at all.

Two notes on the example in the issue. With an end date of 2021-02-12T00:00:00, both products are dropped, not only the first, because the second was taken at 11:00 on the 12th. And filter_date() returns a new SearchResult without changing products, so the script needs filtered = products.filter_date(...) to show the new count. The Finished filtering products. N resulting products log line shows the real number.

How it was tested (Claude Code ran all of this):

  • New tests in tests/units/test_crunch.py cover FilterDate and both sort helpers with products that only have datetime, and check that start_datetime and end_datetime still take precedence. A new test in tests/units/test_search_plugins.py feeds a minimal Planetary Computer sentinel-2-l2a response through StacSearch and then calls filter_date. These 4 tests fail on develop and pass with the change.
  • The default suite with doctests (pytest -n 6 --ignore=tests/test_end_to_end.py --ignore=tests/test_benchmark.py): 1062 passed on Python 3.12, macOS.
  • black, isort, flake8 with flake8-eradicate, and pydocstyle, at the versions in .pre-commit-config.yaml, are clean on the changed files. mypy -p eodag reports the same 15 errors before and after, all in files this PR doesn't touch.
  • The search from the issue against the live Planetary Computer API. Before: filter_date('2021-02-10T00:00:00', '2021-02-12T00:00:00') kept 2 of 2 products. After: it keeps 0. With an end date of 2021-02-13T00:00:00 it keeps only S2B_MSIL2A_20210212T110059_R094_T30TXP_20210213T073201.

Not run: tox itself (no docs build, no Python 3.10, 3.11, 3.13 or 3.14) and the end-to-end tests.

Further comments

The fix is in the crunchers rather than the provider config, so it covers any STAC provider whose items only carry datetime. The other option is to fill start_datetime and end_datetime from datetime in stac_provider.yml, but then every STAC product would show properties the provider never returned. I can switch to that if you prefer it.

#2436 (ruff migration) also edits filter_date.py and tests/units/test_crunch.py, so whichever PR lands second will need a small rebase.

I made this fix with help from Claude Code, which also wrote and ran the tests above.

Products built from STAC items that only carry `datetime` (for example
planetary_computer sentinel-2-l2a) have no `start_datetime` or
`end_datetime`, so FilterDate kept all of them. FilterDate now uses
`datetime` as start and end date when both are missing, and the
FilterDate and FilterLatestIntersect sort helpers fall back to
`datetime` when `start_datetime` is missing.

Fixes CS-SI#2440

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Filter by date MPC search results

1 participant