Security check for your vibe-coded app: is your data exposed?

Who this is for: you built an app with Lovable, Bolt, Cursor or another AI tool, it stores data in Supabase, and you want to know before launch whether strangers or other users can read your data or find your keys.

The honest headline: no free scanner we tested caught most of the holes we planted. We built a small Supabase app with 10 common security holes and ran five free tools on it (2026-10-16). The best caught 3. All five together caught 5. Every tool missed the other five: a policy that lets any logged-in user read and edit everyone’s profile, a storage bucket anyone could list and upload to, an Edge Function with no login check, user text rendered as raw HTML (XSS), and a known-vulnerable package. Check those five by hand (checklist below).

Our pick: Supabase Security Advisor (76/100). It’s built into every Supabase project, caught the most holes (3 of 10), and every finding came with a copy-paste SQL fix. Semgrep Community Edition (63/100) is the runner-up for your code: free, it runs on your machine, and it found both leaked secrets in our code. Use them together. They look at different layers, and between them they caught all five holes that any tool found.

Why Supabase Security Advisor

• What it caught: our notes table with RLS off, a view leaking every user’s email from auth.users, and a SECURITY DEFINER function anyone could call to delete other people’s notes. After we ran the SQL from its fix pages, a rescan came back clean for all three.
• What it can’t see: it only reads your database settings. Code, the JavaScript bundle, Edge Functions and dependencies are out of its reach.
• A surprise: Supabase documents checks for “always-true” policies (lint 0024) and listable public buckets (lint 0025), but neither fired on our project, which had both. Same result on a second run.
• When to add Semgrep: you have the code locally and can open a terminal. It’s the tool that caught secrets in code. Expect to ignore its warning about the public anon key, which is meant to be in the browser.

Vibe coding security checklist: five things every tool missed

1. Policies that check the owner, not just “signed in”. Create a second test user, sign in as them, and try to read or edit the first user’s rows (from your app or the Supabase API). If it works, your policy says using (true) or only checks authenticated. Policies should compare auth.uid() to the row’s owner column, per Supabase’s RLS docs.
2. Storage buckets. In Storage, every bucket should be private unless its files are truly public. Even public buckets shouldn’t allow anonymous listing or uploads: remove select/insert policies on storage.objects that are open to anon. To test, try listing the bucket and uploading a file while signed out.
3. Edge Functions and API routes. Call each function with no login (for example, open its URL or use curl without an Authorization header). It should refuse. Keep “Verify JWT” on, and inside the function use the caller’s token rather than the service-role key, so RLS still applies.
4. Raw HTML from users (XSS). Search your code for dangerouslySetInnerHTML, innerHTML and v-html. Anything a user typed should be rendered as text. To test, save a note containing <img src=x onerror=alert(1)> and see whether an alert pops up.
5. Known-vulnerable packages. Run npm audit and update what it flags. On our app, lodash 4.17.21 still had advisories; 4.18.1 cleared them. GitHub’s Dependabot didn’t alert us until our next push, so don’t wait for it.

Also check what the tools did catch: RLS on every table, no secret or service_role key in front-end code or a committed .env (if one ever shipped, rotate it; deleting the line isn’t enough, and it stays in git history), and no view or function exposing auth.users.

What we planted and what each tool caught

Scroll sideways for more columns →

Wrong all-clears count against a tool

A scanner that says “you’re fine” when you aren’t is worse than one that says nothing, because you stop looking. Before any re-run, we amended our test script (v1.1, 2026-10-16 02:03 ET): a false all-clear, where the report says a check passed while that planted hole was open, costs twice a false alarm. We applied it to every tool’s first-run evidence. Only Prodable had any (two: “Row-Level Security looks correct” while our notes table was open to anyone, and “API routes require authentication” while our admin function wasn’t). Its score fell from 66 to 57, which moved it from runner-up to “also tested”. Saying nothing about a hole counts as a miss, not an all-clear.

The other tools

• GitHub code scanning, secret scanning and Dependabot (60/100): free on public repos. Push protection blocked our first push because of the Stripe- and OpenAI-style keys in .env, with no setup at all, which is the best moment to catch a leak. But CodeQL’s default setup found nothing in our code, secret scanning didn’t flag the Supabase service-role key, and Dependabot raised no alert for the vulnerable lodash within our 60-minute limit, although GitHub’s dependency graph listed it. GitHub’s docs say alerts are created as soon as you turn Dependabot on; ours appeared only after the next push, by which time lodash was fixed (it then flagged real advisories in our vite and esbuild dev tools instead). On a repo you keep pushing to, it would probably have caught lodash on the next commit. We didn’t test that. On private repos, code scanning and secret scanning need paid GitHub Code Security or Secret Protection on a Team plan (Dependabot alerts stay free).
• Prodable (57/100): a free scan of your live URL. It found the service-role key in our JavaScript bundle within seconds (after you enter an email), with a correct fix and no false alarms. But that was its only catch, and it gave the two wrong all-clears above. Fuller guidance is a $27 Deep Scan.
• VibeScan (54/100, skip): couldn’t scan our app. Every attempt ended “blocked / too little evidence” with no findings.

Going live on your own domain next? See put your app on your own domain. For email confirmation and password rules, see add login to your app.

Supabase’s keys are changing

Supabase says it is deprecating the legacy anon and service_role keys “by the end of 2026”, in favour of publishable (sb_publishable_…) and secret (sb_secret_…) keys (API keys docs, checked 2026-10-16). The rule doesn’t change: publishable keys can be public, secret keys never can.

Scores

Scroll sideways for more columns →

Our rule calls for a second run when the top two are within 10 points. Before the v1.1 change, they were exactly 10 apart (76 vs 66), and we count that as within. So after the change we re-ran the top two, Security Advisor and Semgrep, on a fresh copy of the vulnerable app. Both returned identical findings, so the averages equal the first-run scores.

What it costs

• Security Advisor: included in every Supabase project, including Free.
• Semgrep Community Edition: free and open source (LGPL 2.1). Dependency scanning is part of Semgrep’s AppSec Platform, not the free CLI.
• GitHub: code scanning and secret scanning are free on public repos; on private repos they’re paid add-ons on a Team or Enterprise plan. Dependabot alerts are free everywhere.
• Prodable: the scan is free. An email unlocks the full list. Deep Scan is $27 one-time (shown on our report, 2026-10-16).
• VibeScan: the URL scan is free.

Caveats

• A clean scan is not a guarantee. It means the tool found nothing with the checks it ran.
• One small test app, planted holes. Real apps have more tables, functions and code paths. Tools may do better or worse on them.
• Each tool looks at a different layer (database, code, repo, live site). Our score asks which single tool does the most for a vibe coder, but the table shows how to combine them.
• Dummy keys: every planted key was a correctly formatted fake. A scanner that tests keys against the provider would find them invalid. This may have affected Prodable’s RLS probe and which keys GitHub flagged (we can’t see how either works). A real leaked key would be valid.
• GitHub run: we used a separate public copy of the app and let the dummy keys through push protection with the “used in tests” reason. The copy has been archived.
• Agent runs: an agent ran every tool through its official interface. For Security Advisor we used Supabase’s MCP server, which runs the same checks as the dashboard; for GitHub we used its official CLI and API, which flip the same settings as the dashboard.

Not tested

• Snyk: not tested, because its terms ban publishing benchmarks.
• Aikido (free tier): not tested yet. It needs the site owner’s GitHub account connected.
• Lovable Security Scan: not tested yet. Lovable describes a Security Scan and a Security Reviewer in its own blog (published June 9, 2025). Lovable runs on this site are human-tested only, and we haven’t done one.

This page has no affiliate links yet. Every link goes to the tool’s own homepage. Affiliate deals never decide a pick, and a tool with no affiliate program can still win. Full policy.