FundMeNot

Overview

FundMeNot is a hackathon project where you record a 30-second voice pitch and a panel of AI investors with exaggerated personalities respond, out loud, in their own voices. Under the comedy it is a compact multi-step AI pipeline: speech-to-text, persona-conditioned generation, sentiment scoring, text-to-speech, structured report cards and a shared leaderboard, all as typed Genkit flows called from Next.js server actions.

The problem

Most LLM demos are one prompt and one text box. I wanted to see how far a single model could be pushed across modalities in one request (audio in, persona text, structured scores, audio out) while keeping every step typed and individually replaceable, and to make it fun enough that people would actually try it.

What I built

• Seven Genkit flows in src/ai/flows, each with Zod input and output schemas: transcribe pitch, judge response, judge fight, glitched judge event, sentiment analysis, report card and TTS. All run on googleai/gemini-2.5-flash, with TTS on gemini-2.5-flash-preview-tts.
• A 30-second in-browser recorder using MediaRecorder (WebM) that sends the audio as a base64 data URI to a server action, which passes it to Gemini as media for transcription.
• Eight judge personas with rarity tiers (common, rare, glitch) and a fixed prebuilt voice each. Single mode draws a judge weighted toward commons; panel mode draws four non-glitch judges.
• Random events decided on the server: a 15% chance the judge is "hacked" and replaced by the Broken Judge with absurd, sometimes reversed speech, and a 40% chance in panel mode that the judges ignore the pitch and roast each other instead.
• TTS that converts Gemini's raw PCM output into a WAV data URI with the wav package so the browser can play it directly. TTS failures are swallowed so text feedback still arrives.
• A report card flow that scores Originality, Viability and Clarity (0-100 with letter grades) plus an overall roast level, and a Firestore leaderboard of the 20 most roasted pitches, written through the Firebase Admin SDK by anonymously signed-in users.

Architecture

The browser only records and plays audio; every model call happens in server actions that compose Genkit flows.

flowchart LR
  subgraph client["Browser"]
    rec["Recorder - MediaRecorder 30s"]
    ui["Judge panel UI and audio player"]
    lb["Leaderboard page"]
  end
  subgraph server["Next.js server actions"]
    act["actions.ts orchestrator"]
    roster["Judge roster and rarity"]
  end
  subgraph flows["Genkit flows"]
    stt["transcribeAudioPitch"]
    judge["generateAIJudgeResponse"]
    fight["generateJudgeFight"]
    glitch["createGlitchedJudgeEvent"]
    senti["analyzeSentiment"]
    tts["generateTTS"]
    card["generateReportCard"]
  end
  gem["Gemini 2.5 Flash and TTS"]
  fsdb[("Firestore leaderboard")]
  rec --> act
  ui --> act
  act --> roster
  act --> stt
  act --> judge
  act --> fight
  act --> glitch
  act --> senti
  act --> tts
  act --> card
  flows --> gem
  act -->|"Admin SDK write"| fsdb
  lb -->|"top 20 by roast level"| fsdb

System design

A single-judge round, from recorded audio to a spoken roast and a saved score.

sequenceDiagram
  participant U as User
  participant B as Browser
  participant A as Server actions
  participant G as Genkit flows
  participant F as Firestore
  U->>B: Record pitch up to 30s
  B->>A: handleAudioTranscription with audio data URI
  A->>G: transcribeAudioPitch
  G-->>A: transcript
  B->>A: getJudgeFeedback with transcript and judgeId
  alt 15 percent glitch
    A->>G: createGlitchedJudgeEvent
  else Normal
    A->>G: generateAIJudgeResponse with persona
  end
  G-->>A: response text
  opt Speech not reversed
    A->>G: generateTTS with judge voice
    G-->>A: WAV data URI
  end
  A->>G: analyzeSentiment
  A-->>B: response, sentiment, audio
  B->>A: generateReportCard
  A->>G: score originality viability clarity
  G-->>A: report card
  U->>B: Submit name
  B->>A: saveLeaderboardEntry
  A->>F: Write roast level and summary

Decisions & trade-offs

• Genkit flows with Zod schemas rather than raw SDK calls: every step returns typed output the UI can trust, and each flow can be run in the Genkit dev UI on its own. The cost is an extra framework and a bit of boilerplate per flow.
• One model family for everything (STT, generation, TTS) keeps credentials and latency characteristics in one place, but means a Gemini outage takes down the whole app.
• Randomness lives on the server, not the client, so glitch and fight events cannot be forced from the browser and the response shape stays consistent.
• Audio travels as base64 data URIs in server action payloads. Simple and stateless, but heavier than streaming or uploading to storage.

What I'd do next

• Run the per-judge TTS calls in panel mode in parallel instead of sequentially to cut wait time.
• Stream the judge text to the UI while audio is still being generated.
• Add rate limiting per anonymous user before the leaderboard write.