Search Video & Audio
Our latest feature leverages intelligent search with conversational AI to surface content from video and audio. In this short demonstration, we index public media files from Nasa listed under a creative commons license. When we have a high confidence of a match to a question, we’ll return to the location in the video (or audio) where that answer resides. Using a simple query like ‘When was Nasa founded?‘, or ‘Who is Wernher von Braun?‘ returns meaningful results in the form of a video or audio file.
Users can play the file at the exact timestamp where the content is referenced directly in the chat window. Clicking the ‘More’ button will play the media in its own window from a chat or search bar. Many organizations have hours and hours of content locked away in files that never gets found. There are many use cases where you can improve. Searching lectures, public board meetings, and how-to videos are just a few examples of how to apply this solution.
How are you doing this?
We introduce some additional steps to index media content, but much of the process behaves the same as what we already do for text content. These are the steps that we follow to support this feature.
Place your media content in an S3 bucket to be indexed.
1. Transcribe the voiceover in the video and audio files from S3.
2. Apply the transcription to our knowledge-base that include the source (with timestamp) where that content can be found.
3. When a query returns a match on a media file, our chat and search widget detects the sources as video or audio and dynamically renders an html 5 player the user can click.
