Why AI-generated diffs need a different kind of review
When an AI tool generates a pull request for a React app backed by Postgres, the diff can look “reasonable” while still hiding breaking changes. The risk is less about obvious syntax errors and more about subtle shifts: a renamed column, a query that becomes non-deterministic, a React state shape that drifts from what components expect, or an auth policy that changes who can read what.
This checklist is designed for reviewing AI-to-GitHub diffs before merge—especially for apps built on the modern React + Postgres pattern (often with Supabase). It’s also well aligned with workflows where the prototype is generated quickly and then synced to GitHub for disciplined review, such as projects built with lovable.dev.
Diff triage first: identify what could break
1) Classify the diff by “blast radius”
Before reading line-by-line, scan the file list and sort changes into three categories:
- Contract changes: database schema, API routes, shared types, auth rules, environment variables.
- State and UI changes: React components, form handling, routing, data fetching hooks.
- Infrastructure and build changes: migrations, seed scripts, CI, Docker, Supabase config.
AI often touches multiple layers in one PR. Treat any PR that includes contract changes as “needs extra checks,” even if the UI looks fine.
2) Look for “silent breaking” patterns
- Renamed keys or fields without a compatibility layer.
- Default values removed or changed.
- Nullable → non-nullable schema changes.
- Query logic that changes ordering, filtering, or paging semantics.
- New dependencies or upgraded major versions.
Database checklist: Postgres changes that commonly break apps
3) Migrations are present, ordered, and reversible enough
If the diff changes schema-related code, verify there’s an explicit migration (or equivalent schema change artifact) rather than “it works locally” edits. Confirm the migration order matches deployment order and that it won’t fail on existing data. For high-risk changes (dropping columns, rewriting constraints), require a staged approach or a backfill script.
4) Backward compatibility for existing data
AI-generated schema edits often assume a blank database. In review, ask:
- What happens to existing rows when a new non-null column is added?
- Are there defaults and backfills for historical records?
- Do new constraints (unique, foreign keys) match real-world data?
If you have API changelogs or public contracts, it helps to map impact explicitly; the approach in A 45-Minute Workflow to Map Backward-Compatibility Impact From API Changelogs is a useful mental model even when the “API” is your internal data layer.
5) RLS and auth rules don’t change the product by accident
In Supabase-style stacks, Row Level Security (RLS) is part of the app’s core contract. Review policy diffs like you would review payment logic:
- Do policies reference the correct JWT claims and role assumptions?
- Do they unintentionally broaden access (e.g., public read on a private table)?
- Do they become too strict and break existing screens?
Also confirm that any service-role usage is isolated to server-side code and not leaked into the client.
6) Query correctness and performance regressions
AI can produce queries that “work” but scale poorly. In the diff, look for:
- N+1 patterns introduced in new screens.
- Missing indexes for new filters or join paths.
- Changed sort order that breaks pagination stability.
When in doubt, request an EXPLAIN (ANALYZE, BUFFERS) snippet for any query now used in a list view or dashboard.
React checklist: catching UI breaking changes beyond TypeScript
7) State shape and component contracts
Even with TypeScript, AI-generated changes can subtly alter assumptions:
- Props renamed but consuming components not updated everywhere.
- Optional fields treated as required (or vice versa).
- Form payload structure diverges from what the backend expects.
Specifically review shared types, generated API clients, and any “mapper” logic between form state and persistence.
8) Data fetching and caching semantics
If the diff changes hooks or query keys (React Query, SWR, custom caches), verify:
- Cache keys still include all relevant parameters (filters, user ID, workspace ID).
- Mutations invalidate the correct queries.
- Error and loading states are still handled (no infinite spinners on auth failures).
AI sometimes removes “boring” loading paths to simplify code; that can turn intermittent failures into user-facing dead ends.
9) Routing, auth gates, and edge-case navigation
Review route diffs for:
- New pages that should be protected but aren’t.
- Redirect loops introduced by auth guards.
- URL param changes that break deep links shared by users.
Integration checklist: environment, secrets, and operational readiness
10) Environment variable contract changes are documented
AI-generated diffs may add new env vars for third-party services (email, payments, webhooks). Ensure the PR includes:
- Updates to example env files.
- Clear naming and scoping (client vs server variables).
- Reasonable fallbacks for local development.
For teams, this is where a pre-configured environment can save review time; Pairing warm-start protocol for sharing a pre-configured dev environment fast complements this checklist by reducing “works on my machine” ambiguity.
11) Dependency and build changes are intentional
Scan lockfiles and package manifests. Flag:
- Major version bumps.
- New heavy runtime dependencies added “for convenience.”
- Build config changes that could affect production bundles or SSR behavior.
Ask for justification in the PR description for any dependency not obviously required by the feature.
Testing checklist: a small set of high-signal gates
12) Minimal “breaking change” test suite
You don’t need exhaustive tests to catch most AI-introduced breaks. Require evidence of:
- Migration applied to a database with representative data (not empty).
- Sign-in, sign-out, and an auth-protected screen flow.
- Create → read → update for a core entity (including validation errors).
- A list view with pagination or filtering (if present in the app).
If the PR touches RLS, explicitly test at least two user roles (or two accounts) to ensure policies behave as intended.
How to run the review in practice
13) Make AI show its work in the PR
AI-generated PRs are easier to review when the author (human or AI) includes a short “contract impact” section:
- Schema changes (tables/columns/policies) with rationale.
- Client-visible API or type changes.
- Required env vars and deployment steps.
Tools that start from a working prototype and then sync to GitHub—such as lovable.dev—fit well here because the prototype speed doesn’t have to come at the expense of merge discipline. The key is treating the diff as the source of truth and applying consistent gates before shipping.
14) When to split the PR
If the diff mixes schema changes, UI refactors, and dependency upgrades, request a split. Smaller PRs make it much easier to detect breaking changes and to roll back safely if something slips through.
15) A final “break glass” check
Before merge, answer two questions explicitly:
- Can we deploy this without downtime or data loss?
- If we roll back the app code, will the database still be compatible?
If either answer is “not sure,” hold the merge until the migration strategy and compatibility story are clear.
