A summer with GSoC, FFmpeg and Netflix

19 Aug 2024

Hey! GSoC about to end, and I’m in the final stage - to submitting my work. I had a chance work with incredible mentor(s) this year, big thanks to _Kyle Swanson_ and _Thilo Borgmann_. I worked on FFmpeg and Netflix codebases. This was my first big codebase experience and to be honest I was little bit scared at the beginning. The people in the mailing list are very talented and were talking about things that I don’t understand most of the time, but I believe this experience improved my skills a lot and gave some confidence to me. So If you are feeling same feelings with me be confident and passionate!

What was my project?

My project was adding a public API to libmvaf(vmaf) to allow per-frame metadata propagation. This explanation might be little abstract let me explain;

Libvmaf can make feature extractions and fuse those extracted features to calculate/predict model feature(s) (vmaf itself is a predicted). You can see that extracted features by setting output file in vmaf cli or FFmpeg, but problem is you can only see that file when process is finished, in other words you can’t see extracted features while doing feature extraction or prediction.

My project was propagate those features to a user defined callback function when features are ready. In short I make program able to show extracted features when they extracted instead of waiting to process finish.

Problem / Requested Changes

I need to make 2 changes to complete this program.

How I approached initially?

My initial approach was far from ideal as you can see. I made bunch of new files and added extra API for that calling metadata function but that was not the requested changes.

How changed my approach?

After couple of discussion sessions with my mentor Kyle, he suggested a solution that integrated to feature_vector_append function, we planned as propagate data while appending to feature vector, and it worked very well. Current implementation lays on that.

Challenge(s)

  • Standarts of FFmpeg’s and Netflix’s codebases are very high, catching those standarts requires couple of iterations generally.
  • Before applying FFmpeg as GSoC contributor, I worked with different codebases and most of them you need to convince community somehow to your patch is good-written, has no bugs. IF that patch is a feature then you also need to convince them to they need that feature. This is to be honest way more harder than writing some code.

Conclusion

As I submit my final work and reflect on this journey, I’m reminded of the importance of confidence, passion, and perseverance. When faced with uncertainty or self-doubt, it’s essential to take action, learn from mistakes, and seek guidance from experienced mentors. As the famous quote goes, “Do the best you can until you know better. Then when you know better, do better.” That was the my journey so far, I’m proud of what I’ve accomplished.

Thanks for reading.