Video Frame Interpolation via Cyclic Fine-Tuning and Asymmetric Reverse Flow

Research output: Chapter in Book/Report/Conference proceedingArticle in proceedingsResearchpeer-review

203 Downloads (Pure)

Abstract

The objective in video frame interpolation is to predict additional in-between frames in a video while retaining natural motion and good visual quality. In this work, we use a convolutional neural network (CNN) that takes two frames as input and predicts two optical flows with pixelwise weights. The flows are from an unknown in-between frame to the input frames. The input frames are warped with the predicted flows, multiplied by the predicted weights, and added to form the in-between frame. We also propose a new strategy to improve the performance of video frame interpolation models: we reconstruct the original frames using the learned model by reusing the predicted frames as input for the model. This is used during inference to fine-tune the model so that it predicts the best possible frames. Our model outperforms the publicly available state-of-the-art methods on multiple datasets.
Original languageEnglish
Title of host publicationProceedings of 2019 Scandinavian Conference on Image Analysis
PublisherSpringer
Publication date2019
Pages311-323
ISBN (Print)9783030202057
DOIs
Publication statusPublished - 2019
EventScandinavian Conference on Image Analysis - Norrköpings Visualisering Center, Norrköping, Sweden
Duration: 11 Jun 201913 Jun 2019
http://ssba.org.se/scia2019/

Conference

ConferenceScandinavian Conference on Image Analysis
LocationNorrköpings Visualisering Center
Country/TerritorySweden
CityNorrköping
Period11/06/201913/06/2019
Internet address
SeriesLecture Notes in Computer Science
Volume11482
ISSN0302-9743

Keywords

  • Computer Science
  • Image Processing and Computer Vision
  • Pattern Recognition
  • Computer Graphics
  • Artificial Intelligence
  • Computer Communication Networks
  • Slow motion
  • Video frame interpolation
  • Convolutional neural networks

Fingerprint

Dive into the research topics of 'Video Frame Interpolation via Cyclic Fine-Tuning and Asymmetric Reverse Flow'. Together they form a unique fingerprint.

Cite this