SmartVideo Tech Series: How is SmartVideo Generated? (Part 1)

Yair Halevi September 20, 2012
SmartVideo Tech Series: How is SmartVideo Generated? (Part 1)

Over the last 20 years, the web has evolved from static,
manually written pages to computer generated pages, driven by data collected in databases, inputted by users and
computed by algorithms. SmartVideo, to me, is the natural evolution of video into the dynamic era.
Instead of manually producing every single video (such as the web once was), a system is now capable of generating,
tailoring, personalizing, contextualizing and optimizing video based on data.

This vision is made possible through a combination of several technological building blocks, known as the SmartVideo
technology stack. In this two-part post, which kicks off our SmartVideo Tech series, I’ll describe the centerpiece
of this stack: video generation.


At the very core of SmartVideo is the ability to generate videos based on data. Just about any data available on the
web can be used to generate SmartVideo: catalogs, user profiles, search results, user input and interactions, photos
and videos, billing information, social connections – and more. The key enabler of SmartVideo is technology that
creates millions of up-to-date, varied videos on the fly, at low cost, while maintaining creative freedom and video

At SundaySky, our dynamic video generator is based on templates called Videolets™. Each Videolet specifies how to use
input data to construct videos for a specific customer use case, along with a collection of all assets required to
construct these videos. See the below graphic to understand how it all works together.

SmartVideo Tech Video Generator Videolet

Though the video generator was the very first piece of technology developed at SundaySky, it has evolved
substantially over the years. Our customers, partners and creative team continuously challenge SundaySky’s R&D
group to enable more personalized, real-time, sophisticated videos at increasing numbers. Today, the video
generator contains numerous creative capabilities, optimization tricks and operational features that push the
envelope of what’s possible with SmartVideo.

SmartVideo Performance vs Content

Most features of our video generator address two primary aspects of SmartVideo. The first is content: the ability to
generate a wide range of engaging videos using any input data. I’ll focus on this aspect of video generation in
part two of this post next week. The second aspect is performance, including:


scalable video generation

#1. Scalability

We’re not talking about two, ten or even hundreds of videos. We’re talking millions. Consider, for example,
video bills explaining the bill for each and every AT&T customer, every month. Or, videos for every product
in West Elm’s product catalog. Our video generator addresses this using a highly distributed, horizontal scale-out architecture that allows us to increase capacity by
adding machines to our cluster.

Real Time Video Generation

#2. Up-to-Date and On-the-Fly

SmartVideos are data-driven and should be up-to-date with data changes. When your account balance changes your
account summary video should reflect that. It’s one thing to generate a large number of videos once – it’s
entirely another to do so continuously in the face of high-frequency changes.

With on-the-fly generation, videos are created at the moment they are being viewed – and not rendered in advance. On-the-fly video generation is a boon for
up-to-date videos, and it opens up a large range of personalized, context-relevant, real-time use cases for
SmartVideo. Our video generator supports this by employing a timeline-driven pipeline, where elements are rendered
and combined in a streaming fashion, throughout the entire architecture, all the way to the user. This pipeline
is optimized and scheduled so that video reaches the user with minimal delay and no buffering.

#3. Cost

For SmartVideo to be useful, the cost of generating millions of up-to-date videos cannot be prohibitive. ‘Nuff

To minimize this cost, the work necessary to generate each video is optimized, squeezing as much performance as
possible from each machine. We also use a hybrid cloud approach to increase our resource utilization: our
baseline capacity is served from our owned infrastructure, while capacity spikes are served by scaling up and down
in the cloud.

SmartVideo for multiple formats

#4. Quality and Multiple Formats

Computers, tablets, mobile devices, connected TVs, and game consoles demand higher resolution videos (which are more
costly to produce), at varying formats, for Flash, as well as HTML5. Our video generator creates multi-format videos intelligently for both
pre-rendered and on-the-fly generation.

#5.Creative Effects

To deliver an effective, engaging message, SmartVideos contain various creative mechanisms, such as visual and audio
effects, transitions, layers and blending, 3D, lighting, narration, music and beat synchronization, and more. This
is challenging because complex effects require additional computational resources to render, increasing the cost of
generating videos and the time it takes to render them.

In part two of this post, I’ll describe how the SundaySky video generator uses a Videolet to assemble dynamic
content based on data, and some of the challenges in doing so. Stay tuned next week!