Generating Highlight Videos of a User-Specified Length using Most Replayed Data

We propose a novel system that automatically generates highlights of any user-specified length. Our system leverages Most Replayed Data from YouTube, which identifies how frequently a video has been watched over time, to gauge the most engaging parts. It then optimizes the final editing path by adjusting internal segment durations, and finally makes the short edit of the original video that includes the most engaging moments. A, B, and C at the peak of the MRD represent key moments in the soccer match.

Abstract

A highlight is a short edit of the original video that includes the most engaging moments. Given the rigid timing of TV commercial slots and length limits of social media uploads, generating highlights of specific lengths is crucial. Previous research on automatic highlight generation often overlooked the control over the duration of the final video, producing highlights of arbitrary lengths. We propose a novel system that automatically generates highlights of any user-specified length. Our system leverages Most Replayed Data (MRD), which identifies how frequently a video has been watched over time, to gauge the most engaging parts. It then optimizes the final editing path by adjusting internal shot durations. We evaluated the quality of our system's outputs through two user studies, including a comparison with highlights created by human editors. Results show that our system can automatically produce highlights that are indistinguishable from those created by humans in viewing experience.

Supplementary Video

Results

Results with Various User-Specified Length

Results show how the segments differ when highlights are generated using different user-specified lengths for the same original video (Source: https://www.youtube.com/watch?v=rzj4FFi7wt8).

These are the results of various genres with different user-specified lengths (Source: (A) https://www.youtube.com/watch?v=DzcH9eNFVto (B) https://www.youtube.com/watch?v=aE4BdIP6bvc (C) https://www.youtube.com/watch?v=ZD1QrIe--_Y).

Results on User-Driven Customization

On top of the fully-automated function, we additionally allow user-driven highlight customization, enabling users to select the frames they want to include or exclude (Source: https://www.youtube.com/watch?v=rzj4FFi7wt8).

Comparison with Human-Editors

We conducted user studies comparing the highlights created by human editors. The results show that our method is comparable to that of the results created by human editors in terms of viewing satisfaction. For our method-generated highlights, we set the final highlight length to match the length of the corresponding human-edited highlight.