Video Processing Benchmarks and Research

Benchmarks

• MSU Super-Resolution for Video Compression Benchmark 2022
  • H.264, H.265, H.266, AV1, AVS3 codec standards
  • More than 260 test videos
  • Visual comparison for more than 80 SR+codec pairs
  • Extensive subjective comparison with 5300+ valid participants
• MSU Video Quality Metrics Benchmark
  • Largest compression dataset in this area (2500+ streams)
  • 40 different codecs of 10 compression standards
  • 780.000+ subjective responds
  • Both VQA and IQA metrics
  • 20+ metrics without variations
  • 150+ metrics with variations
• MSU Video Alignment and Retrieval Benchmark Suite
  • 3 benchmarks with different time distortions
  • 560 test pairs in each Benchmark with a total duration of ~2 million frames
  • Combinations of 13 frequent distortions obtained due to human/machine video editing and processing
• MSU Video Super Resolution Benchmark
  • New metrics for detail restoration
  • The most complex content for restoration task
  • Different degradation types to lower the resolution
  • 14 algorithms
• MSU Shot Boundary Detection Benchmark 2021
  • 1287 minutes of video: 1964553 frames
  • 7501 cut transitions
  • 629 dissolve transitions
• MSU Deinterlacer Benchmark
  • #1 deinterlacer benchmark
  • 28 video sequences
  • 31 algorithms
• Global Data Compression Competition
  • #1 prize fund
  • 12 compression areas for research
  • 41 algorithms
• MSU Video Matting Benchmark
  • #1 matting benchmark
  • Five moving objects captured in front of a green plate and seven captured using the stop-motion procedure
  • 3 quality metrics
  • 6 algorithms
• MSU Video Completion Benchmark
  • 7 video sequences
  • 4 quality metrics
  • 6 algorithms

Papers

• Video compression dataset and benchmark of learning-based video-quality metrics
  • Largest compression dataset in this area (2500+ streams)
  • 83 codecs of different compression standards
  • 760.000+ subjective responds
  • Both VQA and IQA, NR and FR metrics
  • 25+ metrics without variations
  • 150+ metrics with variations
• Neural-Network-Based Detection Methods for Color, Sharpness, and Geometry Artifacts in Stereoscopic and VR180 Videos
  • 2 neural network based models for estimating 3 types of stereoscopic artifacts in VR180 videos
  • Simultaneously detecting color and sharpness mismatch between stereoscopic video views
  • 9,488 stereopairs of size 960 × 540 from 16 stereoscopic movies to train method for color and sharpness mismatch estimation
  • 22800 stereopairs with artificial distortions from 39 3D movies in the train dataset for geometry mismatch estimation method
  • Objective quality assessment of 100 VR180 videos from YouTube using proposed methods
• Machine-Learning-Based Method for Content-Adaptive Video Encoding
  • A new approach to predicting video codec presets
  • Predicted presets decrease bitrate of the x264 and x265 codecs by 17.8% and 7.9% compared to standard options
• Temporally coherent person matting trained on fake-motion dataset
  • A novel fake-motion algorithm for generating neural-network training video clips from a dataset of images with ground truth alpha mattes and background videos
  • A U-Net based deep neural network method with LSTM blocks and an attention module on skip connections
  • A motion estimation based method for improving the output's temporal stability
  • Better than 8 different matting methods according to subjective evaluation
• Stereoscopic quality assessment of 1,000 VR180 videos using 8 metrics
  • Analysis of stereoscopic quality for 1,000 VR180 YouTube videos
  • Detection and analysis of common 3D-Shooting artifacts
• Stereoscopic dataset from a video game: detecting converged axes and perspective distortions in S3D videos
  • Dataset with a synthetic set of frame sequences from GTA V video game
  • Suitable for stereoscopic video analysis and processing
  • 4000 frames for the training set and 500 for the test set
• Hacking VMAF and VMAF NEG: Vulnerability to Different Preprocessing Methods
  • Increase VMAF by up to 218.8% and VMAF NEG by up to 21.9%
  • Comparation of 8 different preprocessing methods
  • Results verification on encoded streams
• Hacking VMAF with Video Color and Contrast Distortion
  • We consider different color corrections of compressed videos which increase the values of full-reference metric VMAF and almost don’t decrease other widely-used metric SSIM
  • The proposed video approach shows the metric in-applicability in some cases for video codecs comparisons, as it may be used for cheating in the comparisons via tuning to improve this metric values.
• Machine-Learning-Based Method for Finding Optimal Video-Codec Configurations Using Physical Input-Video Features
  • 9-20% bitrate savings against x264 standard presets
  • Faster by 10 times than other existing solutions
  • More than 350 videos and 1300 presets of x264 codec in the developed dataset
  • 13 considered video physical features

Datasets

• MSU Video Saliency Dataset (SAVAM)
  • Eye-tracking device: SMI iViewXTM Hi-Speed 1250 (20 fixations per frame)
  • 43 fragments, 19760 frames
  • 50 observers

Video Quality

• MSU VQMT
  • Quickest metrics
  • Contributors such as Netflix, Walt Disney, Intel etc.
  • Flexible Functionality
  • HDR metrics
  • Objective Metrics Support
  • Metrics Visualization
  • Professional Bit Depth Support
• MSU PVQT
  • Easy subjective video testing organization for companies and universities
  • 300+ Video and all Main Image Formats
  • Real-Time result observation
  • Bradley-Terry model
  • Training questions
  • Fast debug
• Subjectify.us
  • Crowd-sourced subjective comparisons for the comparison of images, video, and sound processing methods
  • Automatic filtering cheaters and bots
  • Easy to use platform with detailed analysys of the results

#1 Codec Comparisons

• MSU Video Codecs Comparison 2020 Part 1: FullHD, objective
  • 50 sequences
  • 20 video encoders
  • 1 fps & 30 fps
• MSU Video Codecs Comparison 2020 Part 2: Subjective report
  • 8 video sequences
  • 6,100+ unique observers
  • 11 codecs of HEVC/AV1/AVC
  • 1 fps & 30 fps
• MSU Video Codecs Comparison 2020 Part 3: 4K report
  • 12 UHD videos
  • Section with comparison on 10-bit videos
  • 1 fps & 30 fps
• MSU Hardware Video Codecs Express Comparison 2020
  • 4 hardware-accelerated and 4 software encoders
  • 50 FullHD video sequences in objective comparison
  • 60 fps encoding
• MSU UGC Subjective Express Comparison 2020
  • 8 encoders
  • 10 FullHD video sequences from YouTube UGC
  • 1 fps encoding
• MSU Video Codecs Comparison 2019 Part 1: Main report
  • 100 FullHD video sequences
  • Three encoding use cases
• MSU Video Codecs Comparison 2019 Part 2: Subjective report
  • 11 codecs
  • 732 unique observers
  • 5 video sequences
• MSU Video Codecs Comparison 2019 Part 3: 4K report
  • 12 codecs
  • 11 4K video sequences
  • Two Encoding Use Cases
  • 9 metrics
• MSU Video Codecs Comparison 2019 Part 4: High-quality encoding report
  • 7 codecs
  • 6 FullHD video sequences
  • Special Encoding Use Case
  • 9 metrics
• MSU Video Codecs Comparison 2018 Part 1: Main report
  • 28 video sequences
  • 14 codecs
• MSU Video Codecs Comparison 2018 Part 2: Subjective report
  • 10 codecs
  • 473 unique observers
  • 5 video sequences
  • 6 metrics
• MSU Video Codecs Comparison 2018 Part 3: 4K report
  • 6 codecs
  • 10 4K video sequences
  • Special Use Case
  • 5 objective metrics
• MSU Video Codecs Comparison 2018 Part 4: High-quality encoding report
  • 7 codecs
  • 5 FullHD video sequences
  • Special Use Case
  • 10 objective metrics