fix(repo): stop reporting code scanning languages added by GitHub

docs/reference/organization/repository/index.md

@@ -20,7 +20,7 @@ Definition of a Repository for a GitHub organization, the following properties a
 | _auto_init_                               | boolean                                                   | If the repository shall be auto-initialized during creation                             | only considered during creation                                                                                                                                                            |
 | _code_scanning_default_setup_enabled_     | boolean                                                   | If the repo has default code scanning enabled                                           |                                                                                                                                                                                            |
 | _code_scanning_default_query_suite_       | string                                                    | The query suite to use for default code scanning                                        | `default` or `extended`, only taken into account when `code_scanning_default_setup_enabled` is set to true                                                                                 |
-| _code_scanning_default_languages_         | list[string]                                              | The languages to analyze with code scanning default setup                               | `actions`, `c-cpp`, `csharp`, `go`, `java-kotlin`, `javascript`, `python`, `ruby`, `swift`, `rust` or `typescript`, only taken into account when `code_scanning_default_setup_enabled` is set to true. The deprecated value `javascript-typescript` is translated to `javascript` and `typescript` |
+| _code_scanning_default_languages_         | list[string]                                              | The languages to analyze at least with code scanning default setup                      | `actions`, `c-cpp`, `csharp`, `go`, `java-kotlin`, `javascript`, `python`, `ruby`, `swift`, `rust` or `typescript`, only taken into account when `code_scanning_default_setup_enabled` is set to true. An empty list leaves the languages to GitHub, languages that GitHub analyzes in addition are kept. The deprecated value `javascript-typescript` is translated to `javascript` and `typescript` |
 | _custom_properties_                       | dict\[string, string \| list\[string\]\]                  | The custom properties to set for this repository                                        |                                                                                                                                                                                            |
 | _default_branch_                          | string                                                    | Name of the default branch                                                              |                                                                                                                                                                                            |
 | _delete_branch_on_merge_                  | boolean                                                   | If branches shall automatically be deleted after a merge                                |                                                                                                                                                                                            |

docs/userguide/guides/repository-security.md

@@ -75,11 +75,29 @@ of a public repository, see
 Default setup lets GitHub configure and run CodeQL. It is configured with:
 
 - `code_scanning_default_setup_enabled`: enables default setup
-- `code_scanning_default_languages`: the languages to analyze
+- `code_scanning_default_languages`: the languages to analyze at least
 - `code_scanning_default_query_suite`: `default` or `extended`
 
 The languages and the query suite are only taken into account when default setup is enabled.
 
+### Languages analyzed by GitHub
+
+Default setup analyzes the CodeQL languages that GitHub detects in the repository, and GitHub adds the languages
+that it detects later on, e.g. once Python code is pushed to the repository, see
+[About setup types for code scanning](https://docs.github.com/en/code-security/concepts/code-scanning/setup-types).
+The configured languages are thus the minimum set of languages to analyze:
+
+- without configured languages (an empty list), the languages to analyze are left to GitHub, `plan` and
+  `check-sync` do not report the languages that GitHub analyzes
+- a configured language that is not analyzed is added to the analyzed languages
+- languages that GitHub analyzes in addition to the configured ones are kept, they are not removed and not
+  reported as a difference
+
+!!! tip "Recommendation"
+
+    Leave the languages to GitHub, and only configure a language that GitHub does not analyze on its own, e.g. a
+    compiled language.
+
 ### Supported languages
 
 `code_scanning_default_languages` only accepts the language identifiers of CodeQL, not the names of the languages
@@ -122,9 +140,9 @@ The languages supported by CodeQL are listed in
 are shown in the sidebar of the repository: a language that is not detected, e.g. `python` for a repository without
 Python code, is reported as error, with the list of detected languages. Only `actions` is always accepted.
 
-As a consequence, languages can only be configured for repositories that already exist and contain code: create
-the repository first, then configure the languages in a second change. A new repository can enable default setup
-without languages:
+As a consequence, languages can only be configured for repositories that already exist and contain code. A new
+repository enables default setup without languages, GitHub then analyzes the languages it detects once code is
+pushed:
 
 ``` jsonnet
 orgs.newOrg('my-project', 'my-org') {
@@ -136,11 +154,11 @@ orgs.newOrg('my-project', 'my-org') {
 }
 ```
 
-1. Without configured languages, the languages to analyze are left to GitHub. Once the repository contains code,
-   configure the languages explicitly, so that the configuration shows what is analyzed.
+1. Without configured languages, the languages to analyze are left to GitHub, see
+   [Languages analyzed by GitHub](#languages-analyzed-by-github).
 
-Once the repository contains code, e.g. Java, Kotlin and TypeScript code as well as workflows, the languages are
-configured with their CodeQL identifiers:
+Once the repository contains code, e.g. Java, Kotlin and TypeScript code as well as workflows, languages to analyze
+at least are configured with their CodeQL identifiers:
 
 ``` jsonnet
 orgs.newOrg('my-project', 'my-org') {

docs/userguide/guides/troubleshooting.md

@@ -44,8 +44,9 @@ Its **Validate** button also has the configuration of an organization validated
 **`<repository> has 'code_scanning_default_languages' configured while the repository does not yet exist.`**
 
 : Languages are checked against the languages detected in the repository, which requires the repository to exist.
-  Create the repository first, with default setup enabled but without languages, then configure the languages in a
-  second change, see [Languages must be detected in the repository](repository-security.md#languages-must-be-detected-in-the-repository).
+  Create the repository with default setup enabled but without languages: GitHub analyzes the languages it detects
+  once code is pushed. Only configure a language in a second change if GitHub does not analyze it on its own, e.g. a
+  compiled language, see [Languages analyzed by GitHub](repository-security.md#languages-analyzed-by-github).
 
 **`<repository> has 'code_scanning_default_languages' configured with '<language>' but this language is not detected in the repository. Detected languages: ...`**
 
@@ -163,6 +164,12 @@ Its **Validate** button also has the configuration of an organization validated
   configuration is removed when deletions are applied. Add it to the configuration instead, see the warning in
   [Otterdog specifics](rulesets-vs-branch-protection.md#otterdog-specifics).
 
+**A language removed from `code_scanning_default_languages` is still analyzed**
+
+: The configured languages are the minimum set of languages to analyze: languages that GitHub analyzes in addition
+  are kept, and `plan` does not report them. To stop analyzing a language, change the default setup in the GitHub
+  web interface, see [Languages analyzed by GitHub](repository-security.md#languages-analyzed-by-github).
+
 **A rotated secret is not updated on GitHub**
 
 : Otterdog cannot detect that the value of a credential changed in the credential provider. Update the secret

otterdog/models/repository.py

@@ -963,6 +963,56 @@ def is_key_valid_for_diff_computation(self, key: str, expected_object: Self) ->
 
         return True
 
+    def get_difference_from(self, other: Self) -> dict[str, Change[Any]]:
+        diff: dict[str, Change[Any]] = super().get_difference_from(other)
+
+        languages_change = diff.get("code_scanning_default_languages")
+        if languages_change is not None:
+            languages_change = self._get_code_scanning_languages_change(languages_change)
+            if languages_change is None:
+                diff.pop("code_scanning_default_languages")
+            else:
+                diff["code_scanning_default_languages"] = languages_change
+
+        return diff
+
+    @classmethod
+    def _get_code_scanning_languages_change(cls, change: Change[Any]) -> Change[Any] | None:
+        """
+        Returns the change of the code scanning default languages that needs to be applied, if any.
+
+        Default setup analyzes the languages that GitHub detects in the repository and adds the languages
+        detected later on, the configured languages are thus the minimum set of languages to analyze:
+        - without any configured language, the languages to analyze are left to GitHub
+        - languages analyzed in addition to the configured ones are kept
+
+        The change goes from the configured languages that are already analyzed to the configured languages, so
+        that it only shows the missing ones. It does not contain the other languages of the current object, which
+        is the previous configuration when applying a merged pull request: the languages analyzed on GitHub are
+        only kept when the change is applied, see RepoClient._update_code_scanning_config.
+        """
+        expected_languages = change.to_value
+        if not isinstance(expected_languages, list) or len(expected_languages) == 0:
+            return None
+
+        current_languages = change.from_value if isinstance(change.from_value, list) else []
+        analyzed_languages = {cls._get_code_scanning_analyzed_language(x) for x in current_languages}
+
+        configured_languages = list(dict.fromkeys(expected_languages))
+        already_analyzed_languages = [
+            x for x in configured_languages if cls._get_code_scanning_analyzed_language(x) in analyzed_languages
+        ]
+
+        if len(already_analyzed_languages) == len(configured_languages):
+            return None
+
+        return Change(already_analyzed_languages, configured_languages)
+
+    @classmethod
+    def _get_code_scanning_analyzed_language(cls, language: str) -> str:
+        # CodeQL analyzes JavaScript and TypeScript together, analyzing one of them analyzes both
+        return "javascript-typescript" if language in cls._combined_code_scanning_languages else language
+
     def include_for_live_patch(self, context: LivePatchContext) -> bool:
         import fnmatch
 
@@ -1212,7 +1262,10 @@ async def get_mapping_to_provider(
 
         if "code_scanning_default_languages" in data:
             mapping.pop("code_scanning_default_languages")
-            code_scanning_mapping["languages"] = S("code_scanning_default_languages")
+            # without any configured language, GitHub analyzes the languages it detects in the repository
+            languages = data["code_scanning_default_languages"]
+            if is_set_and_valid(languages) and len(languages) > 0:
+                code_scanning_mapping["languages"] = S("code_scanning_default_languages")
 
         if len(code_scanning_mapping) > 0:
             mapping["code_scanning_default_config"] = code_scanning_mapping

otterdog/providers/github/rest/repo_client.py

@@ -698,6 +698,25 @@ async def _fill_code_scanning_config(self, org_id: str, repo_name: str, repo_dat
                 config["languages"] = [lang for lang in config["languages"] if lang != "javascript-typescript"]
             repo_data["code_scanning_default_config"] = config
 
+    async def _get_analyzed_code_scanning_languages(self, org_id: str, repo_name: str) -> list[str]:
+        _logger.debug("retrieving analyzed code scanning languages for '%s/%s'", org_id, repo_name)
+
+        status, body = await self.requester.request_raw(
+            "GET", f"/repos/{org_id}/{repo_name}/code-scanning/default-setup"
+        )
+        if status != 200:
+            # without the analyzed languages, the update would remove the ones that are not configured
+            raise RuntimeError(
+                f"failed to retrieve the code scanning config of repo '{org_id}/{repo_name}', "
+                f"needed to keep its analyzed languages: {status}: {body}"
+            )
+
+        config = json.loads(body)
+        if config.get("state") != "configured":
+            return []
+
+        return list(config.get("languages", []))
+
     @staticmethod
     def _to_default_setup_update_languages(languages: list[str]) -> list[str]:
         # GitHub reports 'javascript' and 'typescript' separately for the default setup, but the
@@ -714,6 +733,10 @@ async def _update_code_scanning_config(self, org_id: str, repo_name: str, code_s
 
         languages = code_scanning.get("languages")
         if isinstance(languages, list):
+            if len(languages) > 0:
+                # the configured languages are the minimum set of languages to analyze, keep the ones analyzed
+                # by the current default setup, e.g. added by GitHub when it detected a new language
+                languages = [*await self._get_analyzed_code_scanning_languages(org_id, repo_name), *languages]
             code_scanning = {**code_scanning, "languages": self._to_default_setup_update_languages(languages)}
 
         try:

otterdog/webapp/home/playground-reference.json

@@ -597,8 +597,8 @@
         },
         "code_scanning_default_languages": {
           "type": "list[string]",
-          "description": "The languages to analyze with code scanning default setup",
-          "notes": "`actions`, `c-cpp`, `csharp`, `go`, `java-kotlin`, `javascript`, `python`, `ruby`, `swift`, `rust` or `typescript`, only taken into account when `code_scanning_default_setup_enabled` is set to true. The deprecated value `javascript-typescript` is translated to `javascript` and `typescript`"
+          "description": "The languages to analyze at least with code scanning default setup",
+          "notes": "`actions`, `c-cpp`, `csharp`, `go`, `java-kotlin`, `javascript`, `python`, `ruby`, `swift`, `rust` or `typescript`, only taken into account when `code_scanning_default_setup_enabled` is set to true. An empty list leaves the languages to GitHub, languages that GitHub analyzes in addition are kept. The deprecated value `javascript-typescript` is translated to `javascript` and `typescript`"
         },
         "custom_properties": {
           "type": "dict[string, string | list[string]]",