Our plan is to develop #1 benchmark collection in Video Processing area
Benchmarks
-
MSU HDR Video Reconstruction Benchmark 2022
- Comparison of 16 methods of HDR video reconstruction
- A new private dataset for testing. 20 different scenes
- 10 metrics for restoration quality assessment
- HDR video player for self-assessment of quality
- Subjective comparison
-
MSU Super-Resolution for Video Compression Benchmark 2022
- H.264, H.265, H.266, AV1, AVS3 codec standards
- More than 260 test videos
- Visual comparison for more than 80 SR+codec pairs
- Extensive subjective comparison with 5300+ valid participants
-
MSU Video Quality Metrics Benchmark
- Largest compression dataset in this area (2500+ streams)
- 40 different codecs of 10 compression standards
- 780.000+ subjective responds
- Both VQA and IQA metrics
- 20+ metrics without variations
- 150+ metrics with variations
-
MSU Video Alignment and Retrieval Benchmark Suite
- 3 benchmarks with different time distortions
- 560 test pairs in each Benchmark with a total duration of ~2 million frames
- Combinations of 13 frequent distortions obtained due to human/machine video editing and processing
-
MSU Video Super Resolution Benchmark
- New metrics for detail restoration
- The most complex content for restoration task
- Different degradation types to lower the resolution
- 14 algorithms
-
MSU Shot Boundary Detection Benchmark 2021
- 1287 minutes of video: 1964553 frames
- 7501 cut transitions
- 629 dissolve transitions
-
MSU Deinterlacer Benchmark
- #1 deinterlacer benchmark
- 28 video sequences
- 31 algorithms
-
Global Data Compression Competition
- #1 prize fund
- 12 compression areas for research
- 41 algorithms
-
MSU Video Matting Benchmark
- #1 matting benchmark
- Five moving objects captured in front of a green plate and seven captured using the stop-motion procedure
- 3 quality metrics
- 6 algorithms
-
MSU Video Completion Benchmark
- 7 video sequences
- 4 quality metrics
- 6 algorithms
Papers
-
Video compression dataset and benchmark of learning-based video-quality metrics
- Largest compression dataset in this area (2500+ streams)
- 83 codecs of different compression standards
- 760.000+ subjective responds
- Both VQA and IQA, NR and FR metrics
- 25+ metrics without variations
- 150+ metrics with variations
-
Neural-Network-Based Detection Methods for Color, Sharpness, and Geometry Artifacts in Stereoscopic and VR180 Videos
- 2 neural network based models for estimating 3 types of stereoscopic artifacts in VR180 videos
- Simultaneously detecting color and sharpness mismatch between stereoscopic video views
- 9,488 stereopairs of size 960 × 540 from 16 stereoscopic movies to train method for color and sharpness mismatch estimation
- 22800 stereopairs with artificial distortions from 39 3D movies in the train dataset for geometry mismatch estimation method
- Objective quality assessment of 100 VR180 videos from YouTube using proposed methods
-
Machine-Learning-Based Method for Content-Adaptive Video Encoding
- A new approach to predicting video codec presets
- Predicted presets decrease bitrate of the x264 and x265 codecs by 17.8% and 7.9% compared to standard options
-
Temporally coherent person matting trained on fake-motion dataset
- A novel fake-motion algorithm for generating neural-network training video clips from a dataset of images with ground truth alpha mattes and background videos
- A U-Net based deep neural network method with LSTM blocks and an attention module on skip connections
- A motion estimation based method for improving the output's temporal stability
- Better than 8 different matting methods according to subjective evaluation
-
Stereoscopic quality assessment of 1,000 VR180 videos using 8 metrics
- Analysis of stereoscopic quality for 1,000 VR180 YouTube videos
- Detection and analysis of common 3D-Shooting artifacts
-
Stereoscopic dataset from a video game: detecting converged axes and perspective distortions in S3D videos
- Dataset with a synthetic set of frame sequences from GTA V video game
- Suitable for stereoscopic video analysis and processing
- 4000 frames for the training set and 500 for the test set
-
Hacking VMAF and VMAF NEG: Vulnerability to Different Preprocessing Methods
- Increase VMAF by up to 218.8% and VMAF NEG by up to 21.9%
- Comparation of 8 different preprocessing methods
- Results verification on encoded streams
-
Hacking VMAF with Video Color and Contrast Distortion
- We consider different color corrections of compressed videos which increase the values of full-reference metric VMAF and almost don’t decrease other widely-used metric SSIM
- The proposed video approach shows the metric in-applicability in some cases for video codecs comparisons, as it may be used for cheating in the comparisons via tuning to improve this metric values.
-
Machine-Learning-Based Method for Finding Optimal Video-Codec Configurations Using Physical Input-Video Features
- 9-20% bitrate savings against x264 standard presets
- Faster by 10 times than other existing solutions
- More than 350 videos and 1300 presets of x264 codec in the developed dataset
- 13 considered video physical features
Datasets
-
MSU Video Saliency Dataset (SAVAM)
- Eye-tracking device: SMI iViewXTM Hi-Speed 1250 (20 fixations per frame)
- 43 fragments, 19760 frames
- 50 observers
Video Quality
-
MSU VQMT
- Quickest metrics
- Contributors such as Netflix, Walt Disney, Intel etc.
- Flexible Functionality
- HDR metrics
- Objective Metrics Support
- Metrics Visualization
- Professional Bit Depth Support
-
MSU PVQT
- Easy subjective video testing organization for companies and universities
- 300+ Video and all Main Image Formats
- Real-Time result observation
- Bradley-Terry model
- Training questions
- Fast debug
-
Subjectify.us
- Crowd-sourced subjective comparisons for the comparison of images, video, and sound processing methods
- Automatic filtering cheaters and bots
- Easy to use platform with detailed analysys of the results
#1 Codec Comparisons
-
MSU Video Codecs Comparison 2020 Part 1: FullHD, objective
- 50 sequences
- 20 video encoders
- 1 fps & 30 fps
-
MSU Video Codecs Comparison 2020 Part 2: Subjective report
- 8 video sequences
- 6,100+ unique observers
- 11 codecs of HEVC/AV1/AVC
- 1 fps & 30 fps
-
MSU Video Codecs Comparison 2020 Part 3: 4K report
- 12 UHD videos
- Section with comparison on 10-bit videos
- 1 fps & 30 fps
-
MSU Hardware Video Codecs Express Comparison 2020
- 4 hardware-accelerated and 4 software encoders
- 50 FullHD video sequences in objective comparison
- 60 fps encoding
-
MSU UGC Subjective Express Comparison 2020
- 8 encoders
- 10 FullHD video sequences from YouTube UGC
- 1 fps encoding
-
MSU Video Codecs Comparison 2019 Part 1: Main report
- 100 FullHD video sequences
- Three encoding use cases
-
MSU Video Codecs Comparison 2019 Part 2: Subjective report
- 11 codecs
- 732 unique observers
- 5 video sequences
-
MSU Video Codecs Comparison 2019 Part 3: 4K report
- 12 codecs
- 11 4K video sequences
- Two Encoding Use Cases
- 9 metrics
-
MSU Video Codecs Comparison 2019 Part 4: High-quality encoding report
- 7 codecs
- 6 FullHD video sequences
- Special Encoding Use Case
- 9 metrics
-
MSU Video Codecs Comparison 2018 Part 1: Main report
- 28 video sequences
- 14 codecs
-
MSU Video Codecs Comparison 2018 Part 2: Subjective report
- 10 codecs
- 473 unique observers
- 5 video sequences
- 6 metrics
-
MSU Video Codecs Comparison 2018 Part 3: 4K report
- 6 codecs
- 10 4K video sequences
- Special Use Case
- 5 objective metrics
-
MSU Video Codecs Comparison 2018 Part 4: High-quality encoding report
- 7 codecs
- 5 FullHD video sequences
- Special Use Case
- 10 objective metrics