What to know

  • ChatGPT's Advanced Voice Mode now has the ability to see and understand what's happening around you.
  • Enable video in Advanced Voice Mode by tapping on the video icon in the bottom left corner.
  • The feature is currently available to Plus, Pro, and Teams users on ChatGPT's smartphone apps. Enterprise and Edu subscribers will get it in January. There's no word on when it'll be available to free users.

ChatGPT's Advanced Voice Mode has finally received the ability to 'see' and understand what's happening around you using your phone's camera. Thanks to this new visual ability, you can let ChatGPT see what you see and get real-time video analysis.

The multimodal feature was first previewed almost 7 months ago when OpenAI first revealed the Advanced Voice Mode. Here's how you can stream live video to ChatGPT AVM and have real-time voice and video interactions with it.

How to have real-time video interactions with ChatGPT's Advance Voice Mode

With the new video feature in Advanced Voice Mode, you don't have to capture and upload photos and screenshots to ask about them. You can simply turn on your camera in the app and ask ChatGPT about what's in frame.

Note: Advanced Voice Mode's new visual ability is currently available to Teams, Plus, and Pro users only. The feature is also limited to ChatGPT's smartphone app.

Step 1: Launch Advanced Voice Mode on the ChatGPT app on your smartphone.

Step 2: Once Advanced Voice Mode starts, you'll see a new video icon in the bottom left corner. Tap on it to start streaming live video.

Step 3: Give it the permission to use your phone's camera.

Step 4: Keep what you want to show ChatGPT within the frame. Then simply ask ChatGPT about it.

Step 5: ChatGPT will answer your questions based on what it sees. You can continue having the conversation hands-free.

Step 6: If the lighting's low, tap on the flash icon in the bottom left corner of the frame to illuminate the object. Use the flip camera icon in the bottom right corner of the frame to switch between rear and front camera.

Step 8: Whenever you need to stop sharing your video, simply tap on the video icon again and you'll return to voice-only chat.

Step 9: As before, once you exit the voice mode, you'll get a transcript of the conversation.

OpenAI announced Advanced Voice Mode's visual ability on Day 6 of OpenAI's 12 days of OpenAI event. The event has proved a smashing success thanks to several new updates and new tools like Sora - the text-to-video generator. Stay tuned for more AI updates.