Building a Meeting Transcription Tool with Whisper API

← Back to Tutorials

In the modern remote-work era, meetings are the lifeblood of corporate communication. Yet, retrieving information from a recorded hour-long Zoom call can be an absolute nightmare. Relying on manual note-taking often leads to missed action items and lost context. Enter OpenAI's Whisper API, a state-of-the-art speech-to-text model that is revolutionizing how developers build transcription services. Today, we will build a robust meeting transcription tool from the ground up, complete with audio chunking, translation capabilities, and production-ready Python code.

Whisper is uniquely powerful because it was trained on 680,000 hours of multilingual and multitask supervised data collected from the web. This massive dataset leads to improved robustness to accents, background noise, and technical jargon—making it the ideal candidate for transcribing messy, real-world corporate meetings.

Why Choose the Whisper API?

While there are numerous speech-to-text providers (Google Cloud Speech, AWS Transcribe, Rev), Whisper stands out for several reasons:

Zero-Shot Performance: It requires no fine-tuning to achieve human-level accuracy across various domains and industries.
Multilingual Support: It natively transcribes and translates audio from over 50 languages into English.
Punctuation and Formatting: Whisper automatically adds commas, periods, and capitalization, making transcripts highly readable out-of-the-box.
Affordability: Priced at $0.006 per minute, transcribing a one-hour meeting costs just 36 cents.

💡 Pro Tip: If data privacy is a strict concern for your enterprise, you can run the open-source Whisper model locally on your own GPU infrastructure instead of using the API. For this guide, however, we will focus on the highly convenient REST API.

The Challenge: The 25MB File Limit

The single biggest hurdle developers face when building with the Whisper API is the strict 25-megabyte file size limit per request. A standard one-hour meeting recorded in WAV format can easily exceed 500MB, and even highly compressed MP3s often breach the 25MB threshold. To solve this, we must implement an audio chunking strategy.

We will use PyDub, a popular Python library, to slice the audio file into smaller, overlapping chunks before sending them to the API. Overlapping ensures that words cut off at the edge of a chunk are captured accurately in the next segment.

Python Script: Chunking and API Integration

Below is a comprehensive script that takes a large audio file, chunks it intelligently, queries the Whisper API, and stitches the transcript back together.

import os
from pydub import AudioSegment
import openai

# Set your API key
openai.api_key = os.getenv("OPENAI_API_KEY")

def chunk_and_transcribe(file_path, chunk_length_ms=600000): # 10 minutes per chunk
    # Load the audio file
    print(f"Loading {file_path}...")
    audio = AudioSegment.from_file(file_path)
    
    # Calculate the number of chunks
    total_length_ms = len(audio)
    chunks = [audio[i:i + chunk_length_ms] for i in range(0, total_length_ms, chunk_length_ms)]
    
    full_transcript = ""
    
    for i, chunk in enumerate(chunks):
        chunk_name = f"temp_chunk_{i}.mp3"
        print(f"Exporting chunk {i+1}/{len(chunks)}...")
        chunk.export(chunk_name, format="mp3")
        
        # Open the chunk and send to Whisper API
        with open(chunk_name, "rb") as audio_file:
            print(f"Transcribing chunk {i+1}...")
            response = openai.Audio.transcribe(
                model="whisper-1", 
                file=audio_file,
                response_format="text" # We want raw text back
            )
            full_transcript += response + " "
            
        # Clean up the temporary file
        os.remove(chunk_name)
        
    return full_transcript

# Example usage
final_text = chunk_and_transcribe("company_all_hands.mp3")
with open("transcript_output.txt", "w") as text_file:
    text_file.write(final_text)
print("Transcription complete!")

In this script, we default to 10-minute chunks (600,000 milliseconds). Since a 10-minute MP3 file is typically around 10-15MB, it safely bypasses the API limits. The text is concatenated sequentially to form the final document.

Leveraging Translation Endpoints

In global organizations, meetings are frequently conducted in multiple languages. Whisper provides a dedicated translation endpoint that will consume non-English audio and output an English transcript. All you need to do is change the API call method from transcribe to translate.

# For translating Spanish/French/etc. audio directly to English
response = openai.Audio.translate(
    model="whisper-1", 
    file=audio_file
)

This is incredibly powerful for asynchronous global teams, allowing a Spanish-speaking engineering team to record a technical review, and seamlessly providing an English transcript for stakeholders in the US and UK.

Addressing Speaker Diarization

A common request for meeting tools is speaker diarization—identifying who spoke when. Currently, the Whisper API does not natively support diarization. To build a robust meeting tool, developers typically employ one of two workarounds:

Pre-processing with Pyannote: Use an open-source library like pyannote.audio to detect speaker changes and split the audio based on speaker segments before sending to Whisper.
Post-processing with LLMs: Pass the raw transcript to GPT-4 with a prompt asking it to infer speaker changes based on conversational context (e.g., "Summarize this and format it as a dialogue between Speaker A and Speaker B").

Performance and Accuracy Benchmarks

Understanding Whisper's accuracy across different languages is crucial if you are deploying this tool globally. The model is benchmarked using Word Error Rate (WER) — a lower WER indicates higher accuracy.

Language	Word Error Rate (WER)	Usability for Meetings
English	~4.2%	Flawless for production use
Spanish	~4.5%	Highly accurate, even with accents
Mandarin	~7.3%	Excellent, handles mixed dialects well
Arabic	~14.5%	Good, may require manual proofreading
Welsh	~35.2%	Not recommended for critical transcription

Extracting Action Items with GPT-4

A wall of text is rarely useful on its own. The final step in building an ultimate meeting tool is passing the generated transcript to a Large Language Model to extract insights. By feeding the transcript into GPT-4, you can automatically generate meeting minutes, action items, and follow-up emails.

summary_prompt = f"""
You are an executive assistant. Read the following meeting transcript and provide:
1. A 3-sentence executive summary.
2. A bulleted list of action items with assignees.
3. Key decisions made.

Transcript: {final_text}
"""

FAQ: Common Whisper API Questions

Does the API store my audio data?

According to OpenAI's data usage policies, data sent to the Whisper API is not used to train their models, and the audio files are deleted after the request is processed, making it relatively secure for corporate environments.

How fast is the API?

The API is highly optimized. A 10-minute audio chunk typically transcribes in less than 30 seconds, allowing you to process an hour-long meeting in under a few minutes with asynchronous requests.

Can I provide a custom vocabulary or dictionary?

You can use the prompt parameter in the API call to pass in a list of industry-specific jargon, acronyms, or employee names. The model will use this prompt as context to heavily bias its transcription towards your custom vocabulary.

By combining audio chunking, the Whisper API, and GPT-4 summarization, developers can deploy highly scalable and intelligent meeting transcription tools in a matter of hours, delivering immense value to productivity workflows.