Frame interpolation for animations within a tight deadline (per third party software)?

[Micha]

Hi,

has someone experience with interpolation frames for an animation? At my current project I render stills of an interior, but finally my client would like to get a fly through too. I ask me, could it be possible to render the fly trough for example at 5 fps and interpolate is per third party software to 24fps? Did someone tried it?

-Micha

[fixeighted]

5fps to 24 might be a bit too much to expect but you could very likely get away with 15...definitely worth testing the below software, as it's free; who knows?, it may well work with 5fps though I'd be seriously amazed if so
Interpolation working the way it does, it largely depends on the sort of shot, speed etc. So may work better in some than others.

https://www.core77.com/posts/101838/...-Interpolation

EDIT: Some general chat about it, with good points made https://hackaday.com/2020/09/20/boos...-fps-using-ai/
Twixtor and similar is what I used in the past, with varying success, exactly for some of the reasons mentioned....cuts are an issue, fast changing motion can be an issue...other similar odd stuff. For a flythrough it 'should' work better, as you can control some things better before you start, e.g. separate shots that don't vary too much...edit later.
Also you can output velocity channel to help (in Twixtor and others)...results can vary also.

I'd love to see how far you get

[Micha]

Thank you for your suggestions. I downloaded the tool from https://grisk.itch.io/dain-app and did some tests. Finally I found it works quite nice if the pixel information doesn't make big jumps. Small choppy jumps are smoothed. But it doesn't work for bigger changes at the screen. So, it looks like it's good for smoothing 15fps to 30fps, but it doesn't work stable. I think, this technology need some more years to mature. My impression is that the KI is working with 2D image information only and instead a solution would be if the KI would create a 3D representation in "her mind" like humans did it.

Here a quick test with a screenshot recording at 15fps and interpolation to 30fps. The artefacts at the end are to strong.

https://www.simulacrum.de/temp/29fps_Int-15.mp4

[fixeighted]

Yeah that is as I expected it'd be. Realistically I don't expect it to be any better than this in the future, as, like you noticed, it can only work with what it sees - one frame to another - and has
no spatial knowledge. With the added problem of lots of layers of depth it will always have issues.

What you could try is breaking up the scene, where you see it is going wrong, to help it out, so e.g. the 'you look great' sign has issues - so render only that and it may work, then comp it.