Answer WhatsApp sales queries from text, voice, photos, and video with OpenAI and Gemini

Description

Quick overview
This workflow indexes a product brochure PDF into an in-memory vector store and then responds to incoming WhatsApp text, voice, image, and video messages using OpenAI and Google Gemini, with per-customer chat memory and retrieval-augmented answers from the catalogue.

How it works
Runs manually to download a product brochure PDF, extract its text, split it into chunks, generate OpenAI embeddings, and index everything into an in-memory vector store.
Triggers when a new WhatsApp message arrives and loads the workflow configuration (models, prompts, phone number ID, and vector store key).
Routes the message by type and converts non-text inputs into text by transcribing audio with OpenAI, describing images with OpenAI Vision, or describing videos with Google Gemini.
Normalizes the customer input (message text, captions, and sender number) into a single prompt for the sales agent.
Uses an OpenAI chat model with per-customer memory and a vector-store retrieval tool to answer questions grounded in the indexed product catalogue.
Sends the agent’s reply back to the customer via WhatsApp, or returns a predefined message for unsupported WhatsApp message types.

Setup
Create and connect credentials for WhatsApp Business Cloud (trigger + send/media access), OpenAI (chat, embeddings, transcription, and vision), and Google Gemini (video analysis).
Set a public direct URL to your product brochure/catalogue PDF in the knowledge base settings and run the manual indexing branch whenever the catalogue changes.
Fill in your WhatsApp Business phone number ID, choose your OpenAI/Gemini model IDs, and adjust the system prompt and limits in the configuration fields before activating the workflow.

Additional info
Build a WhatsApp Sales Agent that understands text, voice, photos and videos with RAG
📥 Open full documentation on Notion

Need help customizing?
Contact me for consulting and support : Linkedin

MY NEW YOUTUBE CHANNEL
👉 Subscribe to my new YouTube channel. Here I'll share videos and Shorts with practical tutorials and FREE templates for n8n.