Our musings on making video personal
Over the last 20 years, the web has evolved from static, manually written pages to computer generated pages, driven by data collected in databases, inputted by users and computed by algorithms. SmartVideo, to me, is the natural evolution of video into the dynamic era. Instead of manually producing every single video (such as the web once was), a system is now capable of generating, tailoring, personalizing, contextualizing and optimizing video based on data.
This vision is made possible through a combination of several technological building blocks, known as the SmartVideo technology stack. In this two-part post, which kicks off our SmartVideo Tech series, I’ll describe the centerpiece of this stack: video generation.
At the very core of SmartVideo is the ability to generate videos based on data. Just about any data available on the web can be used to generate SmartVideo: catalogs, user profiles, search results, user input and interactions, photos and videos, billing information, social connections – and more. The key enabler of SmartVideo is technology that creates millions of up-to-date, varied videos on the fly, at low cost, while maintaining creative freedom and video quality.
At SundaySky, our dynamic video generator is based on templates called Videolets™. Each Videolet specifies how to use input data to construct videos for a specific customer use case, along with a collection of all assets required to construct these videos. See the below graphic to understand how it all works together.
Though the video generator was the very first piece of technology developed at SundaySky, it has evolved substantially over the years. Our customers, partners and creative team continuously challenge SundaySky’s R&D group to enable more personalized, real-time, sophisticated videos at increasing numbers. Today, the video generator contains numerous creative capabilities, optimization tricks and operational features that push the envelope of what’s possible with SmartVideo.
Most features of our video generator address two primary aspects of SmartVideo. The first is content: the ability to generate a wide range of engaging videos using any input data. I’ll focus on this aspect of video generation in part two of this post next week. The second aspect is performance, including:
We’re not talking about two, ten or even hundreds of videos. We’re talking millions. Consider, for example, video bills explaining the bill for each and every AT&T customer, every month. Or, videos for every product in West Elm’s product catalog. Our video generator addresses this using a highly distributed, horizontal scale-out architecture that allows us to increase capacity by adding machines to our cluster.
#2. Up-to-Date and On-the-Fly
SmartVideos are data-driven and should be up-to-date with data changes. When your account balance changes your account summary video should reflect that. It’s one thing to generate a large number of videos once – it’s entirely another to do so continuously in the face of high-frequency changes.
With on-the-fly generation, videos are created at the moment they are being viewed – and not rendered in advance. On-the-fly video generation is a boon for up-to-date videos, and it opens up a large range of personalized, context-relevant, real-time use cases for SmartVideo. Our video generator supports this by employing a timeline-driven pipeline, where elements are rendered and combined in a streaming fashion, throughout the entire architecture, all the way to the user. This pipeline is optimized and scheduled so that video reaches the user with minimal delay and no buffering.
For SmartVideo to be useful, the cost of generating millions of up-to-date videos cannot be prohibitive. ‘Nuff said.
To minimize this cost, the work necessary to generate each video is optimized, squeezing as much performance as possible from each machine. We also use a hybrid cloud approach to increase our resource utilization: our baseline capacity is served from our owned infrastructure, while capacity spikes are served by scaling up and down in the cloud.
#4. Quality and Multiple Formats
Computers, tablets, mobile devices, connected TVs, and game consoles demand higher resolution videos (which are more costly to produce), at varying formats, for Flash, as well as HTML5. Our video generator creates multi-format videos intelligently for both pre-rendered and on-the-fly generation.
To deliver an effective, engaging message, SmartVideos contain various creative mechanisms, such as visual and audio effects, transitions, layers and blending, 3D, lighting, narration, music and beat synchronization, and more. This is challenging because complex effects require additional computational resources to render, increasing the cost of generating videos and the time it takes to render them.