Home / SEO / KA-019

Triage a weekly site crawl into a ranked fix list with owners

Turn a weekly crawler export into a short, ranked fix list: status codes, redirects, canonicals, noindex and orphan pages, each with an owner.

Who it suits

Freelance SEO, small agency technical lead, in house marketer

Time saved

About 60 to 90 minutes per weekly crawl review (estimate)

Level

Medium

Updated

Oct 13, 2026

Why this task

A weekly crawl export often has thousands of rows, and most teams skim it or skip it. Real problems, like a key page set to noindex after a release, can sit unnoticed for weeks. An agent can sort every row into issue types, compare with last week and hand each fix to a named person. You then spend your time checking the top of the list.

You are a technical search engine optimization (SEO) analyst. Triage this week's crawl of [SITE DOMAIN] into a ranked fix list.

Inputs:
1. This week's crawl export: [CRAWL EXPORT THIS WEEK]
2. Last week's crawl export: [CRAWL EXPORT LAST WEEK]
3. Priority pages, most important first: [PRIORITY PAGE LIST]
4. Owner map: [OWNER MAP, e.g. developer = redirects and templates; content lead = titles and copy]
5. Known exceptions to ignore: [EXCEPTIONS, e.g. staging paths, intentional noindex pages]

Do this:
1. Group rows into issue types. Use: 4xx errors, 5xx errors, redirect chains or loops, and redirects to non 200 pages.
   Also use: broken or mismatched canonicals, noindex on pages meant to rank, noindex pages in the sitemap, and orphan pages with zero inlinks.
2. For each type, give the count this week, the count last week, and how many addresses are new.
3. Rank fixes. Put issues on priority pages first, then issues affecting many pages through one template, then the rest.
4. For each fix, write the issue, up to five example addresses, the likely cause and a suggested fix. Add the owner from the owner map and effort as Small, Medium or Large.
5. List anything you cannot classify under a heading called Needs human check.

Rules:
- Use only the data provided. Do not guess traffic, rankings or causes you cannot see in the rows.
- Never recommend noindex as a way to pick a canonical page.
- If an owner is unclear, write Unassigned instead of guessing.
- Keep the ranked list to [MAX ITEMS] items. Put the rest in a backlog table.

Return a summary table, the ranked fix list, the backlog table, and the Needs human check list.

Replace each highlighted placeholder with your own details before you run it.

Steps

1. Export the full crawl as a comma separated values (CSV) file. Include address, status code, redirect target, canonical, indexability, meta robots and inlink count.
2. Keep a short priority list of your most important pages, such as top landing pages from Google Search Console. The agent uses it to rank issues.
3. Keep last week's export beside this week's so the agent can report what is new, what is fixed and what is still open.
4. Write an owner map in plain text. For example: templates and redirects go to the developer, titles and copy go to the content lead, sitemaps go to the SEO.
5. Run the prompt below with both exports, the priority list and the owner map attached. Ask for counts first, then the ranked list.
6. Spot check five rows in each issue group against the live site. Crawlers can misread pages behind logins, scripts or rate limits.
7. Check each recommendation against Google's own pages before you assign it. Google's guidance on duplicate URLs (https://developers.google.com/search/docs/crawling-indexing/consolidate-duplicate-urls) says it does not recommend noindex for choosing a canonical.
8. Paste the approved list into your task tracker with owner and due date. Close last week's items the new crawl shows as fixed.

What you get

A one page summary with issue counts against last week, then a ranked list of ten to twenty fixes. Each fix shows example addresses, a likely cause, a named owner and an effort size. Anything the agent could not classify sits in a separate list for a person to review.

What can go wrong, and the check for each

Related playbooks