VoxOrder - Universal AI Voice Order Bot
Overview
VoxOrder is a versatile AI-powered voice assistant that handles orders, appointments, and service requests across any industry. The bot interacts naturally with users through voice or text, collects requirements, confirms details, and finalizes orders. It uses a configuration-driven approach where each business defines their own products, services, pricing, and business rules. Key capabilities include: - Voice-first interaction with Web Speech API - Multi-industry support (restaurants, medical, salons, retail) - AI-powered conversation engine with intent recognition - Real-time streaming responses via WebSockets - Order confirmation and management system - Email and webhook notifications - Multi-tenant business configuration - Industry-specific adapters for custom logic
The Challenge
Businesses across different industries need automated ordering and appointment systems, but building separate bots for each industry is expensive and time-consuming. Traditional order-taking systems lack natural conversation flow and don't handle complex requests well. Customers expect conversational interfaces but most businesses only offer basic web forms or phone calls. The main challenges include: - High cost of developing industry-specific bots - Lack of natural language understanding in traditional systems - Difficulty handling complex orders with multiple options - No unified solution for different business types - Poor customer experience with rigid form-based ordering - Missed sales due to friction in ordering process
The Solution
VoxOrder solves these challenges with a universal AI voice assistant that adapts to any business through configuration. The system uses advanced AI to understand natural language, extract order details, and guide customers through a conversational flow. Businesses simply configure their products and services once, and the AI handles the rest. Technical Approach: 1. Multi-tenant architecture with business-specific configuration 2. AI-powered conversation engine using DeepSeek/OpenAI 3. Real-time WebSocket communication for streaming responses 4. Voice processing with Web Speech API and Whisper 5. Industry-specific adapters for custom business logic 6. Order management with status tracking and notifications 7. Webhook integration for external systems (POS, CRM) 8. Docker deployment with PostgreSQL and Redis
Technical Implementation
AI Conversation Engine with Intent Recognition
Implementation of the core conversation engine using DeepSeek AI. The system dynamically generates system prompts based on business configuration, extracts order information, and manages conversation flow with confirmation logic.
class VoxOrderEngine:
def __init__(self, business):
self.business = business
self.client = OpenAI(
base_url=settings.DEEPSEEK_BASE_URL,
api_key=settings.DEEPSEEK_API_KEY,
)
self.conversation_history = []
self.current_order = {'items': [], 'customer_info': {}}
def get_system_prompt(self):
products = Product.objects.filter(
business=self.business,
is_available=True
)
product_list = "\n".join([
f"- {p.name}: ${p.price} - {p.description}"
for p in products
])
return f"""You are VoxOrder for {self.business.name}.
AVAILABLE: {product_list}
Collect: name, phone, items, preferences
Confirm order before finalizing."""
async def process_message(self, message, session_id):
self.conversation_history.append({
'role': 'user',
'content': message
})
response = self.client.chat.completions.create(
model="deepseek-chat",
messages=[
{"role": "system", "content": self.get_system_prompt()},
*self.conversation_history
],
stream=True
)
async for chunk in response:
if chunk.choices[0].delta.content:
yield chunk.choices[0].delta.content
await self.extract_order_info()
if self.is_order_confirmed(response):
await self.finalize_order(session_id)WebSocket Consumer for Real-time Chat
Implementation of Django Channels WebSocket consumer that handles bidirectional communication between browser and server. Streams AI responses token by token and manages conversation sessions.
class VoxOrderConsumer(AsyncWebsocketConsumer):
async def connect(self):
self.business_id = self.scope['url_route']['kwargs']['business_id']
self.session_id = self.scope['url_route']['kwargs']['session_id']
self.room_group_name = f'voxorder_{self.business_id}_{self.session_id}'
self.business = await self.get_business()
self.engine = VoxOrderEngine(self.business)
await self.channel_layer.group_add(
self.room_group_name,
self.channel_name
)
await self.accept()
await self.send(text_data=json.dumps({
'type': 'message',
'content': self.business.welcome_message,
'sender': 'assistant'
}))
async def receive(self, text_data):
data = json.loads(text_data)
message = data.get('message', '')
async for token in self.engine.process_message(message, self.session_id):
await self.send(text_data=json.dumps({
'type': 'token',
'content': token,
'sender': 'assistant'
}))Voice Processing with Browser Speech API
Implementation of browser-based voice recording and speech recognition using Web Speech API and MediaRecorder. Provides fallback for browsers without speech recognition support.
class VoiceRecorder {
constructor() {
this.mediaRecorder = null;
this.audioChunks = [];
}
async start() {
const stream = await navigator.mediaDevices.getUserMedia({ audio: true });
this.mediaRecorder = new MediaRecorder(stream);
this.audioChunks = [];
this.mediaRecorder.ondataavailable = (event) => {
if (event.data.size > 0) {
this.audioChunks.push(event.data);
}
};
this.mediaRecorder.onstop = async () => {
const audioBlob = new Blob(this.audioChunks, { type: 'audio/wav' });
const formData = new FormData();
formData.append('audio', audioBlob);
const response = await fetch('/api/speech-to-text/', {
method: 'POST',
body: formData
});
const data = await response.json();
if (data.text) {
this.onTranscript(data.text);
}
};
this.mediaRecorder.start();
}
stop() {
if (this.mediaRecorder && this.mediaRecorder.state === 'recording') {
this.mediaRecorder.stop();
}
}
}
// Speech Recognition Alternative
const recognition = new (window.SpeechRecognition || window.webkitSpeechRecognition)();
recognition.lang = 'en-US';
recognition.interimResults = false;
recognition.onresult = (event) => {
const transcript = event.results[0][0].transcript;
sendMessage(transcript);
};
recognition.start();Multi-tenant Database Models
Django models for multi-tenant business configuration with support for different industries. Includes Business, Product, ServiceOption, and Order models with flexible JSON fields.
class Business(models.Model):
INDUSTRY_CHOICES = [
('restaurant', 'Restaurant / Cafe'),
('medical', 'Medical Clinic'),
('electrical', 'Electrical Services'),
('salon', 'Beauty Salon'),
('hotel', 'Hotel Booking'),
]
name = models.CharField(max_length=200)
slug = models.SlugField(unique=True)
industry = models.CharField(max_length=50, choices=INDUSTRY_CHOICES)
description = models.TextField()
welcome_message = models.TextField(default="Welcome! How can I help you?")
opening_time = models.TimeField()
closing_time = models.TimeField()
phone = models.CharField(max_length=20)
email = models.EmailField()
webhook_url = models.URLField(blank=True)
is_active = models.BooleanField(default=True)
class Product(models.Model):
business = models.ForeignKey(Business, related_name='products', on_delete=models.CASCADE)
name = models.CharField(max_length=200)
description = models.TextField()
price = models.DecimalField(max_digits=10, decimal_places=2)
category = models.CharField(max_length=100)
is_available = models.BooleanField(default=True)
duration_minutes = models.IntegerField(null=True, blank=True)
class Order(models.Model):
business = models.ForeignKey(Business, on_delete=models.CASCADE)
customer_name = models.CharField(max_length=200)
customer_phone = models.CharField(max_length=20)
items = models.JSONField(default=list)
total_amount = models.DecimalField(max_digits=10, decimal_places=2)
status = models.CharField(max_length=20, default='pending')
conversation = models.JSONField(default=list)
created_at = models.DateTimeField(auto_now_add=True)Industry-specific Business Adapters
Adapter pattern implementation for industry-specific business logic. Each adapter can override product formatting, order validation, and pricing calculations.
class BusinessAdapter(ABC):
@abstractmethod
def format_product_response(self, products: List) -> str:
pass
@abstractmethod
def validate_order(self, order_data: Dict) -> tuple[bool, Optional[str]]:
pass
@abstractmethod
def calculate_price(self, items: List) -> float:
pass
class RestaurantAdapter(BusinessAdapter):
def format_product_response(self, products):
lines = []
for product in products:
lines.append(f"🍕 {product.name}: ${product.price:.2f}")
if product.description:
lines.append(f" {product.description[:100]}")
return "\n".join(lines)
def validate_order(self, order_data):
if not order_data.get('items'):
return False, "No items in order"
return True, None
def calculate_price(self, items):
total = sum(item['price'] * item.get('quantity', 1) for item in items)
return round(total * 1.10, 2)
class MedicalAdapter(BusinessAdapter):
def format_product_response(self, products):
lines = []
for product in products:
duration = f" ({product.duration_minutes} min)" if product.duration_minutes else ""
lines.append(f"🏥 {product.name}{duration}: ${product.price:.2f}")
return "\n".join(lines)
def validate_order(self, order_data):
if not order_data.get('preferred_time'):
return False, "Please provide appointment time"
if not order_data.get('customer_info', {}).get('name'):
return False, "Please provide patient name"
return True, NoneOrder Confirmation Flow with AI
Implementation of multi-step confirmation flow that uses AI to validate order completeness and guide users through missing information.
class ConfirmationFlow:
CONFIRMATION_STEPS = [
('items', "Let me confirm: {items_summary}"),
('customer', "Customer: {customer_name}, Phone: {customer_phone}"),
('instructions', "Special instructions: {instructions}"),
('final', "Total: ${total:.2f}. Is this correct?")
]
def __init__(self, order_data):
self.order_data = order_data
self.current_step = 0
self.confirmed = False
def get_next_prompt(self):
if self.current_step >= len(self.CONFIRMATION_STEPS):
return None
step_name, prompt_template = self.CONFIRMATION_STEPS[self.current_step]
return prompt_template.format(
items_summary=self._format_items(),
customer_name=self.order_data.get('customer_info', {}).get('name', 'Not provided'),
customer_phone=self.order_data.get('customer_info', {}).get('phone', 'Not provided'),
instructions=self.order_data.get('special_instructions', 'None'),
total=self._calculate_total()
)
def process_confirmation(self, user_response):
user_lower = user_response.lower()
if any(word in user_lower for word in ['yes', 'correct', 'confirm', 'okay']):
self.current_step += 1
if self.current_step >= len(self.CONFIRMATION_STEPS):
self.confirmed = True
return True
return True
elif any(word in user_lower for word in ['no', 'wrong', 'change']):
self.current_step = max(0, self.current_step - 1)
return False
return False