Much iteration later
I can't believe it's been 4 months since I wrote about the first version of motion already. Time ticks on.
I haven't stopped iterating, and with the newer coding models released since, and the things I've learned, my iteration speed has even accelerated.
One big improvement to my quality of life has been a WebGPU inspector Chrome extension. It has really nice UX to show every resource made each frame via WebGPU, as well as the ability to inspect individual frames. Really great, can't recommend enough for debugging.
Direction
There has been this big shadow looming over the entire project, and the name of the project is Touch Designer. It has always been possible to replicate all video effects made on my little toy in Touch Designer, so (I asked myself sometimes) why bother, why not just play with TD?
And honestly I don't know. The web has its charm, it's accessible to anyone with a browser. I think I enjoy building this thing more than I would enjoy getting good at Touch Designer. Maybe I'm a masochist...
But I was heavily inspired by Touch Designer's UX. A node based layout was always a magical far off goal for me to one day have in my toy. And I'd love to say Gemini did it all for me, but I did have to think really hard about how the WebGPU backend is going to work together with the node based UI, and how to be conservative with texture creation and things like that.
But as soon as I saw a light at the end of the tunnel, which came in the form of connecting the video input node to a flip node, and seeing the screen display the flipped video, I knew it would all work out. I smiled all day after that.
Today
There are now enough effects in this thing that I have tried a tiny minority of all combinations of pipelines. It would be great to see what other people can come up with.
I have no guidance to give around GPU pressure though. I have an M1 Mac and I have definitely managed to warm it up with this, although it's several hundred times more efficient than it used to be, and my Mac was also fine back then.
Thoughts on Vibe Coding
Once each node was a self contained little video processing step, Gemini took off. I wrote the first 10 effects half with Gemini, half on my own, but Gemini wrote all the rest. Now, all I need is an idea, and my prompts are:
@main-files, implement an effect that allows users to remap color and alpha channels.
This will produce the entire effect, with any required settings, in 1 shot, 95% of the time. The other 5% are effects that require a different paradigm for implementation (like needing compute shaders instead of vertex+fragment shaders).
I think more and more I'm vibe coding this. And it's great. I'm still learning a lot, although it is true that despite now having a Kuwahara filter, I would not be able to explain the operation of applying it without looking at the code. I am willing to pay this price for the joy the project brings me.
I know vibe coding stretches far beyond a gap that size, towards building a fullstack application without knowing what an http request is. I still think it's mostly fine and a lot of people are gatekeeping, but I do agree that not knowing why your API keys don't belong in your html is going to earn you a hard lesson the hard way. Let's see how this evolves, I suppose.