[ericnewcomer]

Helps search engines find every page of a help site, and the images its articles show.

• GET /sitemap.xml lists the site's home page, each section and each published article as absolute URLs on the host and scheme the site was reached on, in the sitemaps.org format. Sections and articles carry a <lastmod> - an article's is when it was last modified, a section's the latest of its own and its articles', since its page lists them.
• Each article's entry lists the images its body shows with Google's image sitemap extension: the src of each <img>, found with an HTML parser after uploaded images are resolved to where storage serves them (exactly as the article's page renders them), resolved against the article's address, each listed once in the order they appear. Fragments are dropped, since they're never part of what's fetched.
• GET /robots.txt allows everything and names the sitemap.
• Both are served at those exact addresses (no trailing-slash redirect), ahead of the section routes, with a 5 minute cache lifetime - pages themselves are rendered fresh on every request. A preview serves neither.
• The sitemap takes two queries whatever the size of the site: the sections, and a new LoadReadable for every readable article with its body.

[claude]

Performance: this is an unauthenticated endpoint. Every hit loads the body of every readable article and HTML-parses each one. Cache-Control: max-age=300 only helps clients that respect it; anything else can request it repeatedly and cost DB load and CPU in proportion to the site's size. Two options:

• Cache the generated XML in-process per site for a few minutes.
• Select only what's needed: the image srcs can't come from SQL, but body_html could be fetched without the other columns.

Also, there's no guard for the sitemaps.org limits (50,000 URLs / 50MB uncompressed). That's probably fine for help sites, but it's worth a comment.

[ericnewcomer]

Left as is. That's two queries and an HTML parse of each body, about what a busy page already costs (an article page loads every link target on the site). An in-process cache would also go stale across instances until it expired. Help sites are orders of magnitude below the 50,000 URL / 50MB limits.

🤖 Addressed by Claude Code

[claude]

Review summary

The approach is solid. The sitemap takes two queries regardless of site size, images are resolved the same way the article page renders them, and the root-file routes are registered ahead of the section pattern. Tests cover drafts, image deduplication, relative and absolute image URLs, non-http image sources, preview 404s and the no-trailing-slash behaviour.

I left three inline comments:

1. LoadReadable has no top-level parent check. It doesn't require p.parent_id IS NULL, so any nested article would appear in the sitemap at a URL that 404s.
2. Cost per request. The endpoint is unauthenticated, and every hit loads and HTML-parses every article body. Consider caching the output per site in-process.
3. r.Host is used verbatim. Case and port variants that still match the site end up in the <loc>s. Normalize the host, or build URLs from the site's domain.

Test gaps (minor):

• No test for a published article inside an unpublished section, i.e. that it is excluded.
• No direct unit test of ImageURLs edge cases: fragments, whitespace in src, //host scheme-relative URLs.

🤖 Generated with Claude Code