AI video upscaling workflow
Video models often output at 512p or 720p — sometimes as low as 256p for early image-to-video tools — which looks soft on a modern screen. Upscaling to 1080p or 4K with Topaz Video AI or Real-ESRGAN plus FFmpeg fixes that, but the order of operations matters. This guide builds a correct denoise → upscale → interpolate → encode plan for your source, target and motion level.
How it works
The tool computes your scale factor from source and target resolution and flags when it exceeds a safe single-pass 4× (suggesting two passes instead). It chooses a denoise strength and decides whether frame interpolation is worth it based on the motion level — high-motion clips gain the most from added in-between frames but need gentler denoising to avoid smearing. Finally it emits an FFmpeg encode command with a codec suited to delivery or further editing.
The correct order of operations
The sequence matters significantly and is easy to get wrong:
Step 1 — Denoise. AI-generated video typically has subtle temporal flicker (brightness variations frame to frame) and compression artifacts from the model’s output codec. Upscalers treat every pixel as signal and will sharpen noise into visible block artifacts if you skip this step. A light denoise pass first gives the upscaler a cleaner source.
Step 2 — Upscale. With the source cleaned, the AI upscaler can apply its neural model to generate plausible high-frequency detail. Topaz Video AI’s Proteus or Iris models work well on AI-generated content. Real-ESRGAN’s realesr-animevideov3 or realesrgan-x4plus are common open-source choices. The scale factor should be 2× or 4× in a single pass; larger multipliers in one pass produce over-sharpened, “plastic” results.
Step 3 — Interpolate (optional). Frame interpolation adds synthetic in-between frames to increase the apparent frame rate (for example, 24fps source → 60fps output). It works best at higher resolution, which is why it comes after upscaling. High-motion clips benefit most. Low-motion or near-static AI video often does not need interpolation.
Step 4 — Encode. The final output codec depends on the destination. For delivery, H.264 at CRF 18 (libx264 -crf 18) is universally compatible. H.265/HEVC at CRF 22 produces a smaller file. For re-editing or further passes, use a near-lossless intermediate (ProRes 422, DNxHD, or FFV1) to avoid accumulating compression artifacts.
Scale factor reference
| Source resolution | Target 1080p | Target 4K | Passes needed |
|---|---|---|---|
| 512×512 | ~2.1× | ~4.2× (borderline) | 1 or 2 |
| 512×896 (portrait) | ~2.1× | ~4.2× | 1 or 2 |
| 720p | 1.5× | 3× | 1 |
| 1080p | — | 2× | 1 |
For scale factors above 4×, run two sequential 2× passes rather than a single extreme pass. The intermediate is discarded after the second pass.
Tips and notes
- Always denoise first. Upscalers sharpen whatever they are given, including flicker and compression noise from the source model.
- Interpolate last, at the higher resolution. Motion estimation is more accurate once the frames are already sharp and the image is full-sized.
- Cap single-pass upscaling at 4×. For bigger jumps, chain two moderate passes for a more natural, less artificial-looking result.
- Pick the codec for the job. libx264 CRF 18 for delivery, ProRes or DNxHR for further editing.
- Check motion blur before interpolating. If the source AI video has visible motion blur (which many models add), interpolation can double the blur artifact. Use interpolation only on clean, low-blur sources.