[sbryngelson]

Based on origin/master 80997da. One function changes (f_wall_face_sum in src/simulation/m_ibm.fpp); no parameter, default, or output changes.

Summary

f_wall_face_sum (used by s_ib_flux_wall_prepare, which runs three times per step, and by s_ib_flux_wall_setup) built per-thread arrays h(3) and c(3) and indexed the neighbour cells through them. It now sums the four (six in 3D) neighbour terms directly through a small helper, f_wall_face_term. Only the order of the floating-point additions changes.

On the MI300A (amdflang, OpenMP offload) the old kernel is very slow: s_ib_flux_wall_prepare takes 34% of all GPU kernel time of a reacting ib_flux_wall case. The cause is assumed to be that the compiler places the per-thread arrays in scratch (private) memory; I did not inspect the generated code or register/scratch usage, I only measured that removing the arrays removes the cost. On an A100 (nvfortran, OpenACC) the old kernel was not slow and the change is within noise.

Evidence

Case: examples/2D_ibm_reacting_surface physics (reduced GRI mechanism, carbon surface, reacting cylinder with surface_reaction = 1, thermal_bc = 1) as a periodic square array of one cylinder, area fraction 0.2, body-force driven, 96 cells per diameter (190 x 190 cells), 100 fixed steps, 1 rank, ib_flux_wall = T ("on") or default ("off"). Metric: Performance (ns/gp/eq/rhs) printed with run_time_info = T. Three repeats, interleaved (off, on-master, on-PR, off, ...) on the same node in one job. "master" is a build of the base commit; "PR" is this change.

MI300A (AMD MI300A, amdflang AFAR, --gpu mp), ns/gp/eq/rhs:

repeat	option off	on, master	on, PR	PR vs master
1	9.83	13.85	10.62	-23%
2	11.30	15.87	12.76	-20%
3	12.68	16.66	13.78	-17%
mean	11.27	15.46	12.39	-20%

The cost of the option over "off" falls from +37% to +10% (means). All three numbers drift upward from repeat 1 to 3 on this node, which is why the comparison is within each interleaved repeat.

rocprofv3 kernel trace (--kernel-trace --stats, same case, 100 steps, 300 calls of the kernel):

	master	PR
s_ib_flux_wall_prepare, ms per call	1.909	0.042
s_ib_flux_wall_prepare, total ms	572.8	12.5
share of total GPU kernel time	34.0%	1.05%
total GPU kernel time, ms	~1680	~1190

A100 (NVIDIA A100, nvfortran 25.11, OpenACC, --gpu acc, idle GPU), ns/gp/eq/rhs:

repeat	option off	on, master	on, PR
1	5.41	6.02	5.88
2	5.36	6.03	5.97
3	5.86	6.00	5.97
mean	5.54	6.02	5.94

About 1% (the repeat-to-repeat spread of "off" is 9%), so I make no performance claim for NVIDIA. No profile was taken on the A100.

Tests

Existing goldens, no regeneration; ./mfc.sh test --only <label> --no-build after a --dry-run build of each mechanism.

machine	IBM	Chemistry	Flux Wall (C1F0BCB1, the only test with ib_flux_wall)
MI300A (amdflang, --gpu mp)	85/85	37/37	1/1
A100 (nvfortran 25.11, --gpu acc)	85/85	37/37	1/1

No golden moved. ./mfc.sh format and ./mfc.sh precheck pass.

Limitations

• The scratch-memory explanation is an assumption consistent with the measurement; the kernel was not inspected at the ISA level.
• Measured on one case and one resolution (96 cells/D, 2D, 1 rank). 3D, multiple ranks, and the option's other kernels (s_ib_wall_src_buffers, whose per-call time varied between 0.038 and 0.067 ms between the two profiled runs) were not studied.
• Floating-point summation order changes, so results with ib_flux_wall = T can differ at round-off level from master; the existing test passes within its tolerance on both vendors.
• Not run: Cray/CCE builds, other AMD or NVIDIA devices, CPU-only tests, and the other ib_flux_wall configurations beyond the one test above.
• Timings on a shared pre-production queue node; the absolute numbers drift between repeats.

This PR was made with Claude Code (AI).

[github-actions]

Lines of Code

File	Lines	Diff
src/simulation/m_ibm.fpp	2507	-4

Directory	Lines	Diff
simulation	31169	-4
total	50501	-4

[github-actions]

Claude Code Review

Head SHA: 7ca55a5

Files changed:

• 1
• src/simulation/m_ibm.fpp

Findings:
No issues found. Checked in step 4: src/simulation/m_ibm.fpp. The refactor of f_wall_face_sum into f_wall_face_term is equivalent to the old loop. The 2D and 3D sums match, and 1D is excluded for IB by case_validator.py:829. The new function carries GPU_ROUTINE(parallelism='[seq]').