Choose your response:
Listening...
No one selected
SMF Voice Agent - User Guide
Welcome to SMF Voice Agent
The SMF Voice Agent is a semi-autonomous conversational AI designed to help people with ALS and similar conditions communicate naturally. This guide will help you understand and use all the features effectively.
Key Features:
- Voice Recognition: Continuous recording with automatic speech detection
- AI Responses: Choose from three intelligent response options
- Eye Gaze Control: Fully accessible interface for eye-tracking users
- Multiple Speakers: Support for different users with voice profiles
- Knowledge Base: Store and access personal information and memories
- Internet Search: Get current information when needed
Getting Started
First-Time Setup
When you first open the app:
- Grant microphone permissions when prompted by your browser
- Click
to pick a voice from the voice list or try the default one
- Click
to select who you're talking to
Start a conversation
Write a message in the text box and
- click to speak your message or
- click to generate three responses or
- click to start recording a conversation
Voice Recording & Conversation
Starting a Conversation
You have three ways to communicate:
1. Voice Recording (Recommended)
- Click the button
- Speak naturally - the app detects when you start and stop talking
- The app automatically transcribes your speech
- Three response options will appear
💡 Tip: The Voice Activity Detection (VAD) automatically detects speech, so you don't need to click for each sentence!
2. Speak Text Message
- Type your message in the text box
- Click
- The app will speak your typed message aloud and auto record
3. Text Input
- Click in the text box at the bottom
- Type your instruction (e.g., "Tell me about the movie I watched with my family")
- Click
Choosing Responses
After you speak or type, the AI generates three response options:
- Mouse/Touch: Click on your preferred response
- Eye Gaze: Look at a response for 3 seconds (adjustable)
- Regenerate: Click for new options
- Cancel: Click to cancel and continue recording
Eye Gaze Controls
How Eye Gaze Works
The interface supports eye gaze control through emulated eye gaze (hover and hold):
- Look at any button or response option
- A progress indicator shows selection progress
- Keep looking for the set duration (default: 3 seconds)
- The item is automatically selected when the timer completes
💡 Note: This feature emulates eye gaze control through hover and hold, making it accessible for both eye-tracking devices and mouse users.
Adjusting Eye Gaze Settings
- Click
- Go to the "System" tab
- Adjust "Eye Gaze Duration" (1-5 seconds)
- Toggle "Eye Gaze Visual Feedback" on/off
💡 Tip: Start with 3 seconds and adjust based on your comfort. Shorter times are faster but may cause accidental selections.
People Management
Selecting or Adding a Person
- Click
- Click on a person's name to select them and click , or double-click a person
- Enter a new person's name in the text field
- Click
Switching Between People
- Click
- Click on the person's name button
- The current person shows in the header
Managing People
- Edit: Select a person and click
- Delete: Select a person and click
💡 Note: People represent who you're talking to, not who is using the app. The patient is the primary user guiding the assistant.
Knowledge Base & Memories
Knowledge Base Files
Upload documents for the AI to reference:
- Click
in the header
- Click the "Files" tab
- Click
- Select files (supports .txt, .pdf, .docx, .json)
Supported file types:
- .txt: Plain text files
- .pdf: PDF documents
- .docx: Word documents
- .json: Structured data
Creating Memories
Store important information as memories:
- Click
- Click the "Memories" tab
- Click
- Enter a title and content
- Click
💡 Tip: Use memories for personal information, preferences, medical details, or anything you want the AI to remember.
Settings Configuration
Access settings by clicking in the header.
Voice Settings
- Voice: Choose from available TTS voices
- Model: Select quality (Turbo for speed, Multilingual for accuracy)
- Speech Rate: Adjust speaking speed (0.5-2.0)
- Voice Settings: Fine-tune stability, similarity, and style
Recorder Settings
- Speech Thresholds: Adjust voice detection sensitivity
- Frame Settings: Control speech detection timing
LLM (AI) Settings
- Model: Choose AI model (GPT-4 or GPT-5)
- Temperature: Control response creativity (0-2)
- Max Tokens: Limit response length
System Settings
- Language: Set default language
- Eye Gaze Duration: Adjust selection time
- Visual Feedback: Toggle selection indicators
- Internet Search: Enable/disable web search
Troubleshooting
Microphone Not Working
- Check browser permissions (look for microphone icon in address bar)
- Ensure no other application is using the microphone
- Try refreshing the page and granting permissions again
Voice Not Detected
- Speak louder and clearer
- Check recorder settings and lower the speech threshold
- Ensure you've clicked "Start Recording"
Eye Gaze Issues
- Calibrate your eye tracking device
- Adjust the hover duration in settings
- Enable visual feedback to see selection progress
- Ensure good lighting and proper distance from screen
Responses Not Appearing
- Check your internet connection
- Verify API keys are properly configured
- Look for error messages in the conversation area
Performance Issues
- Close unnecessary browser tabs
- Use Chrome or Edge for best performance
- Clear browser cache if needed
- Reduce max tokens in LLM settings
💡 Need more help? Check the browser console (F12) for detailed error messages.