VibeAudit · Claude Fable 5.1 reads a whole repo and grades it for launch. Free quick scan, $19 deep audit. Scan your own repo →
DEEP AUDIT · 2026-09-02
Nutlope/pdftochat
70–89 fix first
<70 not ready
PDFToChat is a Next.js 14 app where users upload a PDF (via Bytescale), it gets chunked and indexed into Chroma Cloud (or Pinecone/MongoDB), and they chat with it through Together AI. The core flows work and the delete route shows the right ownership pattern, but the two expensive API routes do not: /api/chat lets any signed-in user chat with any document by id, and /api/ingestPdf fetches an arbitrary user-supplied URL server-side and has no limits on PDF size. There is also no rate limiting anywhere, and the Chroma retriever creates a new collection for every unknown chatId, so a single script can run up LLM/Chroma bills. Fix the ownership check on /api/chat and the URL/size validation on ingest before letting real users in.
49 files reviewed · claude-fable-5-1 · deep audit
Findings (16)
`chatId` is read straight from the request body (line 36) and passed to `loadRetriever` (line 59), which builds a `ChromaRetriever(chatId)` / Pinecone namespace for that id. There is no `getAuth()` call and no `prisma.document.findFirst({ where: { id: chatId, userId } })` check. Compare with `app/api/document/[id]/route.ts` lines 10-26, which does it correctly. Authentication currently depends entirely on `middleware.ts` treating `/api/(.*)` as a protected route; the README's own TODO list (line 86) says API routes still need protecting.
Any signed-in user who obtains another user's document id (shared /document/{id} link, log line, or enumeration of cuid v1 ids, which embed a timestamp and counter) can ask questions against that user's private PDF and receive its contents in the answer and in the `x-sources` header. If the middleware matcher is ever loosened, the route becomes fully anonymous.
At the top of POST: `const { userId } = getAuth(req); if (!userId) return 401;` then `const doc = await prisma.document.findFirst({ where: { id: chatId, userId } }); if (!doc) return 404;` before building the retriever. Note this requires the Node runtime for Prisma, so drop the conditional `edge` runtime or move the ownership check to a lightweight store the edge can reach.
In app/api/chat/route.ts, add authorization to the POST handler. Import `getAuth` from '@clerk/nextjs/server' and `prisma` from '@/utils/prisma'. Right after parsing the body, call `const { userId } = getAuth(req)`; if there is no userId return `NextResponse.json({ error: 'Unauthorized' }, { status: 401 })`. Validate that `chatId` is a non-empty string, then run `const doc = await prisma.document.findFirst({ where: { id: chatId, userId } })` and return a 404 if it is null. Only then call `loadRetriever`. Because Prisma needs Node.js, change `export const runtime` to always be 'nodejs'. Reason: currently any authenticated user can chat with any other user's document by supplying its id.`fileUrl` comes from the JSON body (line 10) and is fetched unmodified (line 35). Nothing restricts it to the Bytescale upload host, checks `response.ok`, checks `content-type`, or caps the body size before `response.blob()` loads it into memory. The only MIME check is client-side in `dashboard-client.tsx` line 31 (`onValidate` at line 39 always returns undefined). The unvalidated URL is also persisted to `Document.fileUrl` (line 29) and later rendered in the PDF viewer.
A signed-in user can make your server issue GET requests to internal/cloud-metadata addresses or any third party, and can point it at a multi-GB file to exhaust the function's memory/time. They can also store any URL as a 'document', which your own UI will then load in the viewer.
Validate before fetching: parse with `new URL()`, require `https:`, and require the hostname to match your Bytescale account host (e.g. `upcdn.io` path for your account id). Check `response.ok`, `content-type` includes `application/pdf`, and reject if `content-length` (or the accumulated body) exceeds e.g. 10 MB. Wrap `request.json()` in try/catch and require `fileName` to be a string under ~255 chars.
In app/api/ingestPdf/route.ts, harden the POST handler. (1) Wrap `await request.json()` in try/catch and return 400 on invalid JSON. (2) Validate `fileUrl` with `new URL(fileUrl)`: require protocol 'https:' and hostname equal to the Bytescale CDN host used by our uploader (read an allowlist from an env var like `ALLOWED_UPLOAD_HOSTS`, default 'upcdn.io'); return 400 otherwise. (3) Validate `fileName` is a string with length 1-255. (4) After `fetch(fileUrl)`, check `response.ok` and that `response.headers.get('content-type')` includes 'application/pdf'; reject if `content-length` is missing or greater than 10 * 1024 * 1024. (5) Move the auth check above body parsing. Reason: the current code performs server-side fetches of arbitrary URLs (SSRF) and has no size limit.Each POST to /api/chat runs the history-aware rephrase LLM call plus the answer LLM call (utils/ragChain.ts lines 45-55) and a Chroma Cloud hybrid search that computes Qwen and SPLADE query embeddings. `messages` (line 35) is accepted with no cap on count or per-message length; the client's `maxLength={512}` in document-client.tsx line 217 is not enforced server-side. /api/ingestPdf likewise runs paid embedding for every chunk of an unbounded PDF. There is no per-user or per-IP throttle anywhere.
A single free account can loop requests with 100 KB messages and long fake histories and burn your Together AI and Chroma Cloud credit in minutes, or degrade the service for everyone. With the free 4-document allowance this is the primary abuse vector.
Add a per-user limiter (e.g. `@upstash/ratelimit` keyed on Clerk userId) on both routes: e.g. 20 chats/min and 5 ingests/hour. Server-side, reject `messages.length > 30` and any `content.length > 2000`, and truncate history to the last N turns before sending to the model. Cap ingested pages/chunks (e.g. reject > 200 pages or > 1500 chunks).
Add rate limiting and input bounds to app/api/chat/route.ts and app/api/ingestPdf/route.ts. Install @upstash/ratelimit and @upstash/redis and create utils/ratelimit.ts exporting a sliding-window limiter (20 requests/minute for chat, 5 requests/hour for ingest) keyed by the Clerk userId from `getAuth`. In /api/chat, after auth, return 429 when limited; also validate `messages` is an array with at most 30 entries, each with a string `content` of at most 2000 characters, and only pass the last 10 messages as `chat_history`. In /api/ingestPdf, return 429 when limited and after `loader.load()` reject PDFs with more than 200 pages (`rawDocs.length`) with a clear error. Reason: these endpoints call paid LLM/embedding APIs and currently have no throttle or size limit.
`_getRelevantDocuments` calls `getOrCreateDocCollection(this.docId)` (line 125), which uses `client.getOrCreateCollection` with a full schema including two embedding indexes (lines 97-103). `loadRetriever` in vector_store/index.ts line 41 passes the raw `chatId` from the request body. If `chatId` is missing, the collection `doc-undefined` is created. Combined with F1, any string a caller sends becomes a persistent collection in your tenant.
A script can create thousands of collections in your Chroma Cloud database, hitting collection quotas or billing limits and breaking ingestion/retrieval for real users. Chatting with a document whose ingest failed also creates an empty collection instead of a clear error.
In the retriever, use `client.getCollection({ name })` and translate not-found into a 404/empty result; only `getOrCreateCollection` in `ChromaVectorStore.addDocuments`. Validate `chatId` is a non-empty string matching `/^[a-z0-9]+$/` before use.
In app/api/utils/vector_store/chroma.ts, change `ChromaRetriever._getRelevantDocuments` to fetch the collection with `client.getCollection({ name: collectionName(this.docId) })` instead of `getOrCreateDocCollection`; if the collection does not exist, throw a descriptive error (e.g. 'Document index not found') that the chat route maps to a 404. Keep `getOrCreateDocCollection` only for `ChromaVectorStore.addDocuments`. In app/api/utils/vector_store/index.ts `loadRetriever`, throw if `chatId` is not a non-empty string matching /^[a-z0-9]+$/. Reason: reads currently create collections for arbitrary ids, letting callers pollute the Chroma tenant and hide ingest failures.Quick wins
- · Add `if (!user) redirect('/sign-in')` to both server pages so Prisma never receives an undefined userId (F8).
- · Add real HTTP status codes to /api/ingestPdf error responses (F13).
- · Return a generic error string from /api/chat instead of `e.message` (F12).
- · Flatten `loc.pageNumber` in sanitizeMetadata so source page buttons work again (F6).
- · Turn the mobile header buttons into Links (F14).
- · Export `maxDuration = 300` on /api/ingestPdf.
- · Change the quota check to `>= 3` with a named constant.
What's already good
- · /api/document/[id] DELETE does auth, ownership, and 404/403 correctly; use it as the template for the other routes.
- · Clerk authMiddleware with the standard matcher covers pages and API routes, so unauthenticated access is blocked by default.
- · Per-document Chroma collections give natural tenant isolation once the chatId ownership check is added.
- · Chroma metadata sanitization and per-chunk stable ids show awareness of backend constraints.
- · Secrets are read from env only; no hardcoded keys, and .env is gitignored.
- · Prisma client is correctly singletoned for dev hot reload.
Do this first
- Add auth + document-ownership check to /api/chat and make it Node runtime (F1); at the same time stop the Chroma retriever from creating collections on read and validate chatId (F4).
- Lock down /api/ingestPdf: allowlist the Bytescale host, check response.ok/content-type/size, page cap, maxDuration, and batch the Chroma add (F2, F9).
- Add per-user rate limiting and server-side message bounds on both paid routes (F3).
- Fix the ingest lifecycle so failed ingests do not leave orphan documents (F5, F11).
- Make delete actually delete: stop swallowing Chroma errors, add Pinecone/Mongo cleanup, and remove the Bytescale file (F7).
- Ship the small correctness fixes: page-number metadata, error statuses/messages, fail-closed server pages, mobile header links (F6, F8, F12, F13, F14).
Fixed things? Re-audit.
Run a new scan on the updated repo. Use a 5-pack key or pay per audit.