DevNews

FFmpeg fruc_vulkan: interpolated frames and tradeoffs

On this page
  1. Check the actual implementation
  2. Two controls change different parts of the work
  3. Evaluate the complete pipeline
  4. Sources

FFmpeg’s fruc_vulkan synthesizes intermediate frames from estimated motion. It can change a video’s output cadence, but it does not recover frames that a camera never captured, and generic Vulkan support is not enough to run it.

On a fictional 16×8 image, flow grids spaced at 1, 2 and 4 pixels have 128, 32 and 8 positions. These are geometric counts, not rendered video, measured memory or a speedup prediction.
On a fictional 16×8 image, flow grids spaced at 1, 2 and 4 pixels have 128, 32 and 8 positions. These are geometric counts, not rendered video, measured memory or a speedup prediction. Chart : PeopleAreGeek. Data source.
View full-size image

Check the actual implementation

The filter manual requires the NVIDIA VK_NV_optical_flow extension and describes forward and backward flow between input frames. The tuning commit adds perf and grid. Code merged into development does not guarantee that your packaged FFmpeg contains it.

These commands inspect the installed binary without processing or overwriting media:

ffmpeg -version
ffmpeg -hide_banner -h filter=fruc_vulkan

An unknown-filter response means this binary lacks that filter. Finding its help does not prove the active GPU driver supports the required extension or your chosen formats.

Two controls change different parts of the work

perf selects slow, medium or fast optical-flow estimation, with slow the default. grid selects the flow-field spacing; auto chooses the finest supported size, while explicit choices include 1, 2, 4 and 8. Unsupported sizes are rejected. Neither setting is a universal real-time guarantee.

For a 3840 × 2160 frame, a 1 × 1 grid has 8,294,400 positions; 2 × 2 has 2,073,600; 4 × 4 has 518,400. Those counts follow the grid dimensions. They are not measurements of GPU memory or total processing time: bidirectional fields, buffers and other filter work still contribute. The drawing shows the same relationship at a smaller illustrative resolution.

A coarser field cannot represent the same local detail at every pixel. Compare moving text, thin objects, occlusion edges and camera cuts rather than judging only a broad pan. Keep the original video when the distinction between recorded and synthesized frames matters.

Evaluate the complete pipeline

Record GPU, driver, FFmpeg revision, input pixel format, resolution and target cadence. Start from the documented Vulkan upload and download requirements, then verify that neighbouring filters and the encoder accept the chosen surfaces. A Vulkan filter does not automatically make a mixed decode/filter/encode pipeline free of copies.

Use a short duplicate clip, compare one setting at a time and inspect both playback and individual frames. Measure end-to-end processing speed and artifacts together. Our former claim that medium plus grid 2 was the documented answer for all 2160p24 content was not supported by the manual checked here and has been removed.

Sources

Verified the filter documentation and tuning commit; removed the unsupported universal 4K setting, distinguished Vulkan extension support from generic Vulkan and added a grid calculation.