Blog / AI & search /

A review process for AI-generated recommendations

“High priority: fix canonical issues site-wide.”

Sounds decisive. It also might be wrong for the site in front of you.

AI recommendations need review because they usually bundle a diagnosis with an implied action. Before you implement one, ask what evidence supports it, what it assumes and what happens if the assumption is false.

Require a problem statement

A useful recommendation names the observed issue, affected pages, intended outcome and evidence. “Improve technical SEO” is too vague to approve. “These product URLs return a blocked response in the test environment” is something a developer can reproduce. Ask the tool to separate observed facts from interpretations and to cite the input it used. Then verify those observations independently, because I don’t act on a recommendation I can’t reproduce.

Check what the recommendation assumes about the site. A model may not know your release process, CMS configuration, commercial priorities or the history of a route. It may propose a common fix that conflicts with an existing requirement. If the recommendation depends on missing context, pause and gather that context rather than filling the gap with guesswork.

Assess risk and reversibility

A title rewrite is relatively easy to review; a site-wide canonical change or redirect migration has broader consequences. Consider reach, potential user impact, ease of rollback and confidence in the diagnosis. High-impact or hard-to-reverse actions deserve a technical owner and a test plan. Don’t treat an urgency score from a model as a substitute for this assessment.

Look for unintended side effects. Could a proposed block hide useful pages? Could a bulk content change remove legal wording or a necessary qualification? Could a script slow a page or alter tracking? Ask what should remain unchanged and how the team will detect regression. I’d rather make the smallest testable change that addresses a verified problem.

Decide, test and learn

For each recommendation, choose: accept, adapt, investigate or reject. Record the reason and owner. If accepted, define the expected result and how it will be checked. Use staging or a limited sample when possible, then inspect the live implementation. If rejected, keep a short note so the same suggestion does not return without new evidence.

A recommendation may be useful even when the proposed solution is not. It can point to a question worth checking. Ask the model for counterarguments or failure cases, but do not let that become another round of unverified output. The final decision belongs to the person accountable for the site.

Ask what the recommendation would change

Before accepting a recommendation, translate it into a specific proposed change. “Improve internal linking” could mean adding links from relevant articles, changing navigation, or generating automated related links. These actions have different costs and risks. Ask which pages are involved, why the change fits the reader’s journey and what success would look like. If the answer remains vague, the recommendation is not ready to implement.

Check the input quality. If the model got only a URL and a short prompt, it hasn’t seen the page, the analytics or the configuration. A recommendation based on incomplete context should be treated as a question for investigation. Give it relevant, non-sensitive evidence, and don’t pretend it saw anything you didn’t provide.

Keep a review trail

A decision record can be small: recommendation, evidence checked, decision, owner and follow-up. Note whether the action was changed, limited to a sample or rejected. If a similar recommendation returns later, the team can decide whether new evidence changes the previous conclusion.

For accepted changes, choose a verification method before implementation. A URL change can be checked with a crawl and direct tests; a content revision needs review against its brief and sources. Avoid choosing a success metric after seeing the outcome. If monitoring data shifts, document other changes and timing before attributing the movement to the AI-generated idea.

A human reviewer should challenge confident language and ask what counterexample would make the proposal fail. That isn’t suspicion of every suggestion. It’s making the reasoning visible enough to act on. The recommendation should make the next investigation clearer even when the final decision is to do nothing.

For recommendations involving content, review the page’s purpose and existing history before changing it. A model may suggest adding sections or keywords without knowing that the page serves an existing audience or supports another route. Check internal links and related pages so the proposed fix does not create duplication. For technical suggestions, inspect dependencies and the deployment path before editing production. The smallest verified change is usually easier to learn from than a broad set of simultaneous edits.

A recommendation may also need coordination with people who own other parts of the journey. A technical SEO change can affect product, analytics or support workflows. Before implementation, ask whether those owners need to review the plan. That doesn’t mean a committee for every edit. It just catches dependencies the prompt or audit output can’t see.

Once an action has been approved, keep the recommendation and implementation details linked but distinct. The recommendation explains why the team chose a change; the release record explains what actually shipped. If implementation differs from the original proposal, note the difference and the reason. That prevents later reviewers from evaluating an idea that was never deployed.

Review outcomes separately from the recommendation. If a change is followed by a metric shift, timing alone doesn’t prove cause.

Which part of the next AI recommendation would you verify before approving the action?