#2: Launch a SaaS like Fireflies.ai or Scribenote next week with Open Source

Dive into the growing market of AI scribes and note takers, and learn how to build your own with these resources!

Most successful SaaS products weren’t the first of their kind - think Slack, Shopify, Zoom, Dropbox, or HubSpot. They didn’t invent team communication, e-commerce, video conferencing, cloud storage, or marketing tools; they just made them better.

So what are AI scribes and note takers?

They’re AI-powered assistants that record, transcribe, and analyze conversations in real time. These tools will identify the speakers, summarize key points, extract insights, and trigger actions on your behalf. AI scribes and note takers eliminate the need for note-taking and processing, and enable you to focus fully on discussions - whether in meetings, lectures, interviews, or consultations!

Let's look at the market!

Utilizing a mix of speech recognition, speaker diarization, and (of course) LLMs; AI scribes and note takers started gaining traction in early 2023 and have seen consistent growth in market interest, which is currently at an all-time high (source):

Phrases like "scribe AI" and "AI note taker" see 10k–100k monthly searches (Google Keyword Planner). While AI “Note takers” and “Scribes” are technologically synonymous, they appeal to different audiences:

Note takers like Fireflies and Otter cater to broad markets, automating meeting notes and triggering workflows for sales, management, and recruiting. They also transcribe and analyze notes for educators, content creators, doctors, and other professionals. Fireflies and Otter have ~15M users each, with business plans around $30/seat.

Some note takers will target niche markets and use more specific terminology. For instance “Scribe”, an existing job title in healthcare, makes sense for healthcare note takers. Currently “AI medical scribe” gets 1–10k Google hits compared to just 1–10 for “AI medical note taker.”

There’s a rising market adoption for healthcare note takers, which help record clinical sessions and generate SOAP notes for therapists, vets, and physicians. For example, Scribenote is used by 1000+ Vets and charges ~$249/month, and Sunoh has over 60K physicians, starting at ~$1.25 per consultation.

Alright, so how can we build this quickly?

Most note-takers work with three layers:

1. Recording: Captures the conversation, either natively on the device (Mac/IOS/Android/Windows/linux all have native libraries for this) or via a microservice (e.g., via recall.ai) that records online meetings over Zoom, Google Meet, or Teams.

2. Speech Recognition and Diarization: Transcribes the speech, and labels the speakers (if omitted by recorder) in the conversation. This can be done either by combining an open source ASR model like Whisper-v3-Turbo with Pyannote for speaker diarization (Huggingface ASR list), or via API (Google Speech / Amazon Transcribe).

3. Text analysis: An LLM (e.g., Llama, ChatGPT) is prompted to analyze the entire transcript and generate relevant insights.

Here are some of the best open source projects to execute this pipeline:

• Meeting Bot Starter Kit by Zoom

• noScribe by Kaixxx

• Scriberr by Rishikanth C.

• Live Meeting Assistant Sample by AWS

Worried about building signups, user management, payments, etc.? Here are my go-to open-source SaaS boilerplates that include everything you need out of the box:

• SaaS Boilerplate by Remi Wg

• Open SaaS by wasp-lang

How will my SaaS stand out in the noise?

Here are a few strategies that could help you differentiate and achieve product market fit (based on the pivot principles from The Lean Startup by Eric Ries):

1. Personalize your UX for a niche audience: Design for professions which need Scribes such as Vets (Scribenote’s focus), Therapists, Dentists, Teachers, Lawyers, Recruiters & Researchers (for interviews) etc. Alternatively, target specific regions or industries with unique requirements for language, channel, or features.

2. Add unique features to increase switching cost: Exclusive sticky features could mean unique language support, unique meeting channels, industry specific reporting, and integrations with existing tools used by your audience.

3. Offer platform level advantages: You could ship native mobile/desktop apps for a more integrated, channel independent, UX. Additionally, if this is executed solely using a local, non api-driven, deployment (eg. combine llama+whisper+pyannote), then privacy could become a big selling factor and attract higher licensing fees.

TMI?

I’m an ex-AI engineer and product lead, so don’t hesitate to reach out with any questions!

P.S. I started this free weekly newsletter to share open-source / turnkey resources for recreating popular products (like this one). If you’re a founder looking to launch your next product without reinventing the wheel, please subscribe :)

---