[li-lizhe]

What does this PR do?

run.py records a score for a dataset only when dataset.evaluate() returns a dict or a
pandas.DataFrame; anything else is logged as skip_reason='evaluate_returned_none' in status.json.
Several datasets compute their summary, write it next to the prediction file and then discard it, so a
benchmark that ran to completion reports nothing at all (issue #1690).

This PR returns the summaries so the numbers reach status.json / the summary scripts:

dataset	before	after
olmOCRBench (#1690)	evaluator() returns None	returns {'overall': ..., '<jsonl category>': ...}
DUDE	accuracy table dumped, not returned	return DUDE_acc(storage)
SlideVQA	idem	return SlideVQA_acc(storage)
MMLongBench-Doc	idem	return MMLongBench_acc(storage)
MATBench	mean metrics dumped to CSV	mat_evaluate() returns the metric dict
OceanOCRBench	summary CSV written	oceanocr_evaluate() returns the ZH_/EN_/scene_ averages

Verification

Added tests/test_evaluate_returns_summary.py (stub-based, in the style of tests/test_logicvista.py,
so it needs neither the datasets nor a judge model):

# with this PR
$ python -m pytest tests/test_evaluate_returns_summary.py -v
tests/test_evaluate_returns_summary.py::...::test_dude PASSED
tests/test_evaluate_returns_summary.py::...::test_mmlongbenchdoc PASSED
tests/test_evaluate_returns_summary.py::...::test_slidevqa PASSED
tests/test_evaluate_returns_summary.py::...::test_olmocrbench PASSED
4 passed

# same tests against upstream main (vlmeval/ reverted to 54a063c)
4 failed  -> evaluate() returned None

Not tested: no full end-to-end benchmark run (olmOCRBench / DUDE / SlideVQA / MMLongBench-Doc /
MATBench / OceanOCRBench need their datasets and a judge model). The patch only plumbs back the object
that is already written to the *_score.csv / *_eval_summary.csv files; the returned values are
plain floats built from those rows, and flatten_summary_metrics already handles both dict and
DataFrame.

Fixes #1690

[li-lizhe]

Gentle ping on this one — it's been quiet for 8 days since the last activity, so I wanted to check whether there's anything else you'd like me to adjust or add before this can move forward. Happy to make changes if so. Thanks!

[li-lizhe]

@ssiq — would one of you be able to take a look at this when you get a chance?

dataset.evaluate() drops the summaries

evaluate() computes per-dataset scores and then returns nothing, so callers have to re-derive them from the log. It now returns the summaries while keeping the existing side effects. The diff is large mostly because several dataset modules are touched to thread the return value through.

Happy to adjust if the approach doesn't fit.