Nutlope/llamacoder

The stream route accepts any messageId, loads the whole chat, and calls together.chat.completions.stream with max_tokens 20000 and a 270s deadline (lines 166, 204-213). There is no authentication, no per-IP/per-chat throttle, no daily budget, and no check that a generation is already in flight for that message. /api/create-chat (lines 38, 122) additionally runs a vision model call per request and inserts rows. /api/generate-chat-title runs an LLM call for any chatId. Note that public share URLs are /share/v2/<messageId>, so every shared link is also a valid, unlimited generation capability for the underlying chat.

A single curl loop can burn your entire Together balance in hours, fill the database with junk chats, and exhaust Vercel function time. Because generations are 20k tokens and run up to 4.5 minutes each, even a modest script is expensive. Nothing in the code would tell you it is happening except the invoice.

Add a rate limiter (e.g. @upstash/ratelimit or Vercel KV/Redis) keyed by IP and by chatId in all three routes, return 429 when exceeded, and add a concurrency guard so one message can only have one in-flight generation. Consider a daily global spend cap using Braintrust/token metrics. Example: const { success } = await ratelimit.limit(`gen:${ip}`); if (!success) return new Response('Too many requests', { status: 429 });

In app/api/get-next-completion-stream-promise/route.ts, app/api/create-chat/route.ts and app/api/generate-chat-title/route.ts, add rate limiting before any database or Together AI call. Create lib/rate-limit.ts using @upstash/ratelimit with a sliding window (e.g. 10 generations / 10 minutes per IP for the stream route, 5 chats / 10 minutes per IP for create-chat, 20 / hour for title). Derive the IP from the x-forwarded-for header. Return a 429 JSON/text response when the limit is exceeded. Also in the stream route, add a per-messageId guard (e.g. a short-lived Redis SETNX key `gen-lock:<messageId>` with 5 minute TTL) so the same message cannot have concurrent generations. Reason: these endpoints spend real money on the Together account, are unauthenticated, and messageIds are exposed in public share links.

requestSchema only requires `model` to be a non-empty string (line 44). resolveModel (lib/constants.ts line 24) maps known aliases and passes anything else through untouched, so `resolvedModel` on line 158 can be any string the caller chooses and is sent straight to Together on line 206. The chat already stores its model (chat.model, written in create-chat line 125) but the route never reads it. create-chat (line 24) has the same problem: any model string is persisted.

Anyone can run generations against your account with the most expensive model Together offers, or a dedicated-endpoint model, regardless of what the UI shows. Combined with F1 this multiplies cost exposure. It also lets a caller make a chat unusable by persisting a bogus model in create-chat.

Select `model` from the message's chat and ignore the body value; in create-chat validate `model` against the visible MODELS list. Example: const resolvedModel = resolveModel(message.chat.model); and in create-chat: if (!MODELS.some(m => m.value === model && !m.hidden)) return 400.

In app/api/get-next-completion-stream-promise/route.ts, stop using the client-provided `model` from the request body. Extend the prisma.message.findUnique include on line 72 to also select `model: true` on the chat, then compute `const resolvedModel = resolveModel(message.chat.model)` on line 158. Remove `model` from requestSchema (or keep it optional and ignore it). In app/api/create-chat/route.ts, after parsing the body on line 23, validate that `model` is a string and exists in MODELS from @/lib/constants with `hidden !== true`; return a 400 JSON error otherwise. Update the client callers in app/(main)/chats/[id]/chat-box.tsx, app/(main)/chats/[id]/page.client.tsx and app/(main)/prompt-form.tsx to stop sending `model` to the stream endpoint. Reason: the model determines cost and the server should be the source of truth, not the browser.

createMessage(chatId, text, role, files) is a public server action with no ownership check and no runtime validation: `role` is only typed as 'assistant' | 'user' (line 9) but any string arrives at runtime, `text` is unbounded, and `files` is any[] serialised straight into a Json column (line 25). If a caller inserts a message with role 'system' or 'bogus', the stream route's `z.enum(['system','user','assistant'])` parse on lines 123-130 of app/api/get-next-completion-stream-promise/route.ts throws for every subsequent generation in that chat, so the chat is dead for its real owner. Callers can also inject fake 'assistant' turns (prompt-injecting the model) or store multi-megabyte JSON blobs.

Anyone who learns a chat id (chats are shareable URLs and are bound to leak) can corrupt it, poison the model context, or bloat the database; there is no way for the owner to recover.

Validate all inputs with zod (role enum, text max length ~50k chars, files as an array of {path,language,code} with size caps), and do not allow the client to create 'system' messages. Longer term, mint a per-chat secret in a cookie at create-chat time and require it here.

In app/(main)/actions.ts, add zod validation at the top of createMessage: chatId must match /^[A-Za-z0-9_-]{16}$/, role must be z.enum(['user','assistant']), text must be z.string().min(1).max(60000), and files must be z.array(z.object({ path: z.string().max(300), language: z.string().max(20), code: z.string().max(200000) })).max(30).optional(). Throw a plain Error (not notFound) with a clear message on validation failure. Also introduce lightweight ownership: in app/api/create-chat/route.ts set an httpOnly cookie `chat-owner-<chatId>` containing a random 32-byte token and store its sha256 hash in a new nullable `ownerTokenHash` column on Chat (add a Prisma migration); in createMessage read the cookie via next/headers and reject if the hash does not match. Reason: today any client can write arbitrary messages into any chat and a single bad `role` value makes the stream route throw forever for that chat.

The server streams tokens to the client but never stores the result; page.client.tsx waits for finalContent/abort/error and then calls the createMessage server action from the browser (lines 133-181). The server already has `finalText` in the superviseCompletionStream .then() in app/api/get-next-completion-stream-promise/route.ts line 245 but only logs it to Braintrust. Consequences: closing the tab, a flaky network on the action call, or navigating away mid-stream leaves the chat with an orphan user message and no assistant reply; the client also computes the cumulative `allFiles` from a `chat.messages` snapshot captured at effect time (line 145), and the server has to trust whatever the client says the model produced.

Users pay (you pay) for a 1-4 minute generation and get nothing persisted; refreshing shows a chat with a dangling prompt. Any later follow-up sends the model a conversation that is missing its own previous answer, degrading quality.

Persist the assistant message server-side when the stream completes (or aborts with partial content): in the stream route's .then(finalText) create the Message row with position = message.position + 1 and computed cumulative files, then have the client just router.refresh(). Alternatively return the created message id in a trailing SSE event.

Move assistant message persistence from the client to the server. In app/api/get-next-completion-stream-promise/route.ts, inside the superviseCompletionStream(...).then(async (finalText) => {...}) block (line 245), after computing finalText: sanitize with sanitizeAssistantOutput from @/lib/utils, skip if empty, compute cumulative files by merging extractAllCodeBlocks over all prior assistant messages in `messagesRes` plus the new text (same logic as page.client.tsx lines 145-163), and create a Message row with role 'assistant', position message.position + 1, content finalText, files. Also handle the .catch path: if latest partial content is available (track it via stream.on('content')), persist the partial. Then in app/(main)/chats/[id]/page.client.tsx, change persistResponse (lines 133-181) to no longer call createMessage; instead call router.refresh() and set the active message from the refreshed chat (e.g. by keying on the last assistant message). Keep the didFinalize guard. Reason: today closing the tab after a paid generation loses the result and the server has no authoritative record of what the model produced.