Skip to content
Merged
Show file tree
Hide file tree
Changes from 3 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 9 additions & 1 deletion .github/workflows/L2-tests.yml
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,13 @@ jobs:
token: ${{ secrets.GITHUB_TOKEN }}
ref: ${{ github.head_ref || github.ref }}

- name: Check out shared L2 coverage tooling
uses: actions/checkout@v4
with:
repository: 'rdkcentral/docker-device-mgt-service-test'
ref: 'feature/l2docupdate'
path: docker-device-mgt-service-test

- name: Check out dependent repostiories
uses: actions/checkout@v4
with:
Expand Down Expand Up @@ -105,7 +112,8 @@ jobs:
- name: Generate L2_Coverage.md
if: (github.event_name == 'push' || github.event_name == 'pull_request') && steps.copy_tracefile.outcome == 'success'
run: |
python3 remote_debugger/test/functional-tests/generate_l2_coverage_report.py \
python3 docker-device-mgt-service-test/generate_l2_coverage_report.py \
--component "Remote Debugger" \
--tracefile /tmp/coverage.info \
--src-dir remote_debugger/src \
--features-dir remote_debugger/test/functional-tests/features \
Expand Down
41 changes: 32 additions & 9 deletions docs/rrd_l2_function_coverage_HLD.md
Original file line number Diff line number Diff line change
Expand Up @@ -183,7 +183,10 @@ The L2 suite terminates the daemon by signal during teardown in coverage mode. W

### 5.5 Coverage Report Generation

**Module:** `test/functional-tests/generate_l2_coverage_report.py` (as referenced by the workflow)
**Module:** `generate_l2_coverage_report.py` in the shared
`rdkcentral/docker-device-mgt-service-test` repository. The L2 workflow checks
out that repository alongside `remote_debugger` and invokes the shared script
with `--component "Remote Debugger"`. There is no module-local copy.

**Purpose:** Convert the raw coverage trace and the feature/test inventory into a human-readable summary and actionable gap analysis.

Expand All @@ -203,6 +206,22 @@ The L2 suite terminates the daemon by signal during teardown in coverage mode. W
**Outputs:**
- Markdown report with summary metrics, tables, and scenario coverage status

For local use, check out both repositories into the same parent directory and
run from that directory after collecting the tracefile:
Comment thread
Copilot marked this conversation as resolved.
Resolved

```bash
python3 docker-device-mgt-service-test/generate_l2_coverage_report.py \
--component "Remote Debugger" \
--tracefile /tmp/coverage.info \
--src-dir remote_debugger/src \
--features-dir remote_debugger/test/functional-tests/features \
--tests-dir remote_debugger/test/functional-tests/tests \
--output remote_debugger/test/functional-tests/L2_Coverage.md
```

The shared generator uses Python's standard library. See the shared repository's
README for reuse by other modules and report limitations.

## 6. Functional Workflow

### 6.1 Build Flow
Expand All @@ -225,10 +244,10 @@ The L2 suite terminates the daemon by signal during teardown in coverage mode. W
2. The Python report generator reads the trace file plus the feature/test directory metadata.
3. A Markdown report is produced with sections like:
- coverage overview
- direct and indirect coverage counts
- source function hit and miss counts
- mapped vs unmapped features
- orphan features and orphan tests
- gap analysis by behavior group
- scenario-to-test count gaps
4. On push-based workflow execution, the report is committed to the branch for review.

## 7. Data and Interface Model
Expand Down Expand Up @@ -256,13 +275,17 @@ The generator reads both and builds mappings to support consistent coverage narr
The generated report documents the following:

- total source-function estimate
- functions with direct L2 coverage
- functions with indirect coverage
- functions called at least once according to lcov
- functions with no L2 coverage
- active vs disabled tests
- active feature scenarios and expected gaps
- per-behavior coverage tables
- feature/test gap summary by area such as static profile handling, dynamic profile handling, upload flow, and harmful command detection
- test-definition counts and disabled-test estimates
- feature scenario counts and expected count gaps
- per-source-file line and function coverage
- unmatched features/tests and scenario-to-test count gaps

Static per-behavior coverage claims, RRD-specific infrastructure notes, and
hand-written priority recommendations are not generated by the shared script.
Keep such assessments in module-owned documentation and review them separately
from measured lcov coverage. Counts alone do not prove behavior coverage.

## 8. Error Handling and Boundary Conditions

Expand Down
Loading
Loading