[flyingdoubleG]

Summary

Three problems in delphi/log/result_analysis.py (run at the end of the pipeline with --verbose). The first two show up when several hookpoints are analysed.

1. Score files of other modules are loaded. load_data collects files with glob(f"*{module}*"). Score files are named f"{module}_latent{idx}.txt" (Latent.__str__), so with --hookpoints layers.1 layers.10 the pattern *layers.1* also matches layers.10_latent*.txt. Those rows are loaded a second time, labelled layers.1, and weighted by layers.1's firing counts. Residual-stream SAEs such as EleutherAI/sae-llama-3-8b-32x use hookpoints named layers.N.
2. frequency_weighted_f1 raises KeyError: 0 for every module after the first. counts[module][module_df["latent_idx"]] indexes a tensor with a pandas Series: torch reads the sequence positionally (series[0]), while pandas looks up the label. Only the first module's rows have labels starting at 0, so log_results crashes for any run with more than one hookpoint.
3. firing_count is mostly None. It is set only if latent_idx in counts[module], which tests whether some count value equals latent_idx, not whether the index is in range. This column feeds plot_firing_vs_f1.

Changes

• Match f"{module}_latent*", with / replaced by -- as when the files are written.
• Index the counts with torch.as_tensor(module_df["latent_idx"].to_numpy()).
• Check latent_idx < len(counts[module]).

Tests

Added to tests/test_log/test_result_analysis.py, using detection scores for latent 0 of layers.1 (all correct) and layers.10 (all wrong):

• test_load_data_keeps_modules_with_shared_prefix_apart: each module gets its own 2 rows, and firing_count is read by index. On main, layers.1 gets 4 rows.
• test_frequency_weighted_f1_with_several_modules: weighted_f1 == 0.5, the mean of the per-module F1s 1.0 and 0.0. On main this raises KeyError: 0.

Both fail on main and pass with this change. ruff check (with the repo's excludes) and black --check pass. Locally I ran the tests without tests/conftest.py because vLLM does not install on Windows.