Test script: File and image uploads for a vibe-coded app

• Job slug: file-uploads-for-app
• Version / status: v1 · frozen 2026-10-16 12:03 ET, before the first run. Changes after freeze go under “Amendments”, dated, with the reason.
• Rubric version: v1 (rubric)
• Author / date: Lando (CEO), 2026-10-16

1. The job

Your AI-built app needs people to upload files (profile photos, receipts, PDFs). You want them stored somewhere that isn’t your server’s disk, private by default, served back only through your app, with big files going straight to storage. You’re done when the app’s upload API stores, lists, serves and deletes files and the checks below pass.

2. Candidates

Scroll sideways for more columns →

3. Fixed inputs

• Files (generated once, SHA-256 recorded in the evidence): photo-1mb.jpg (a valid 1,048,576-byte JPEG), big-10mb.jpg (a valid 10 MB JPEG), page.html (a 2 KB HTML file, for the type check).
• API contract (same for every tool): POST /api/files (raw body, Content-Type = the file’s type, x-filename header) → 201 {id}; GET /api/files → list of {id, size}; GET /api/files/:id → the file’s bytes (directly, or by redirect to a short-lived link); DELETE /api/files/:id → 204; POST /api/upload-url → whatever the vendor’s documented direct-from-browser upload needs (a signed upload URL or a client token).
• Each tool’s API runs where its docs put server code with access to the storage key: a Supabase Edge Function for Supabase Storage and a Vercel Function for Vercel Blob (the operator box allows only HTTPS out, and neither key is copied out of a dashboard).
• Check script: checks/uploads-checks.mjs (Node 20), same for every tool.

4. Fixed task

The operator (agent) implements this, following only the vendor’s official docs (each docs page used is logged):

Add file uploads to the app with <tool>, on the free plan:
1. Create the storage (bucket or store). Make it private if the tool lets you choose.
2. Implement the API contract above. Files are served back only through the app.
3. For large files, use the tool's documented way to upload straight from the browser to storage,
   limited to images (image/jpeg, image/png) of at most 20 MB, if the tool supports limits.
4. Keep every secret on the server.

5. Follow-up rules (agent operator version)

• Time cap: 60 active minutes per tool.
• Allowed: the vendor’s official docs, SDK references and error messages. Not allowed: another tool’s code, third-party tutorials.
• Manual interventions (counted for C2): anything a person must do in a dashboard or browser, and any workaround the docs don’t cover.

6. Steps

1. Record the start time. Create the storage and log plan and cost.
2. Implement and deploy the API (§3, §4).
3. Run the check script. Save its JSON output.
4. Fix and rerun until all core checks pass or the cap is hit. End time = when the last core check first passes.
5. Record lines of code, dependencies, docs pages used, and the free-plan limits and prices from the vendor’s pricing page, with the date.

7. Pass/fail checks

Scroll sideways for more columns →

8. Quality checklist (C5): developer experience

Scroll sideways for more columns →

9. Security checklist (C7)

Scroll sideways for more columns →

10. Weights and tie-breaker

• Weights: C1 35 · C2 15 · C3 10 · C4 10 · C5 10 · C6 10 · C7 10.
• C2 anchors (manual interventions + problems the docs didn’t answer): 5 = 0–1 · 4 = 2–3 · 3 = 4–5 · 2 = 6–8 · 1 = more.
• C3 anchors (T = 60 min): 5 = ≤15 · 4 = ≤30 · 3 = ≤45 · 2 = ≤60 · 1 = cap hit.
• C5 and C7: checklist-count anchor (rubric §3).
• C6 portability: 5 = S3-compatible API or a documented bulk download/CLI on the free plan · 4 = bulk export with manual steps · 3 = only one file at a time through the SDK · 1 = none.
• Tie-breaker: C7 (safe defaults), then C1.

11. Cleanup

• Test files are generated images and a dummy HTML file. Delete the bucket/store after the verdict is published. Keys stay in each vendor’s server environment.

Amendments

• v1.1 (2026-10-16, during the runs; none changes how a tool is scored):
  1. Check script, Supabase direct upload (12:03 ET, j07-supabase-r1): the script first called uploadToSignedUrl through @supabase/supabase-js, which crashed on Node 20 (it needs a native WebSocket for Realtime). It now calls the same uploadToSignedUrl through @supabase/storage-js, the Storage part of the same SDK. No Supabase check had run on the direct-upload path. Counted as our problem, not Supabase’s.
  2. Vercel test project (12:07 ET, j07-vercel-r1): our Vercel access can’t create projects (API 403). Instead, the Vercel Function runs as a preview deployment of our existing test project from job #3. Production and its custom domain are untouched. The project’s framework preset (Express, left over from job #3) was set to none. Preview deployments sit behind Vercel’s Deployment Protection, so the check script adds Vercel’s automation-bypass header to calls to our app only. It never sends that header to Blob storage, and the stranger probes (U6) never send it.
  3. Re-runs (gap of 10 or less): both tools were re-run with the same code on fresh storage. Supabase got a new private bucket. Vercel got the same private store with a fresh deployment, because a second store can’t connect to the same project (API 400 project_env_var_not_unique).

Scoring notes (2026-10-16)

• Supabase C2 (1 issue): in r1, after a delete, our Edge Function’s GET /api/files/:id still returned the old redirect. The response headers showed cf-cache-status: HIT: the CDN in front of Edge Functions had cached our 302 because it had no Cache-Control header. Fix: Cache-Control: no-store on every response. We didn’t find this in the Edge Functions docs, so Q4 is a miss in both runs (the docs gap doesn’t go away because we knew about it in r2).
• Supabase Q5: the pricing page lists the free limits (1 GB file storage, 5 GB egress, 50 MB max upload) but not what happens at the limit. That’s in the billing FAQ, so it’s a miss.
• Supabase U4 (r2): a signed link handed out before the delete kept serving the deleted file for about 55 s, until its own 60-second expiry. This passes the script’s rule (stops working within 60 s). In r1 the link died at once.
• Vercel S1: the dashboard and CLI make you choose private or public. The REST API that the Vercel MCP uses defaults to public when access is left out, and its examples use public. We set private explicitly. The script says “private unless you opt in (or you must choose explicitly)”, and an agent using the API isn’t made to choose, so this is a partial, which counts as a miss.
• Vercel C6 = 4: no S3-compatible API. The CLI lists blobs and downloads them one at a time (vercel blob list, vercel blob get), so a full export needs a small script.
• Vercel U8: the SDK raised BlobContentTypeNotAllowedError, which it maps from the Blob API’s content_type_not_allowed error code. So Vercel rejected the file, not our code.