Help regarding OpenCV

Hey there!
I’m new to vvvv so please bear with me. I wanted to make face filters (like snapchat) and made two “filters” with Skia (bubbles and hearts).
Now I wanted to make one with openCV (f.e. backroundblur or blur the face (in a rectangle f.e.)) but no idea how to interact with the blur and the camera.
I basically used the “detect faces” example of opencv and only added the blur…
Help please!

(I tried uploading it, but it won’t let me as my account is new)
OpenCV part: Imgur: The magic of the Internet
rest filters: Imgur: The magic of the Internet

1 Like

Hi @ChiikiBreeki, welcome to the forums!

So I am not sure if I am understanding your intent correctly, but to give you a bit of an idea of how things are working:

The face detection mechanism in OpenCV only gives you a rectangle surrounding the detected face, which is probably not detailed enough for you to pull off a face filter like the ones you’ve seen on snapchat and others.

For a better result, you probably want to look at the help patch titled: “Detect Face Landmarks” in the OpenCV section. That will give you a much more detailed set of face points which you can then combine with something like what is shown in “Find the convex hull surrounding a pointcloud” to get a polygon that fits on the face, and use that as a mask to blur/unblur etc. (“Apply a mask to an image”).

Hope that sets you on the right track and of course, come back and share your results when you are done :)

Cheers!

Tyvm @ravazquez!
You perfectly got my question, I managed to make the convex hull around my face (took a while to find after which node etc. to bind it) but I can’t manage the mask-part as even with an image, as soon as I put my hull and f.e. an image, I get an error…
Error: Imgur: The magic of the Internet
Whole nodes: Imgur: The magic of the Internet

It doesn’t matter what mask I try out but maybe I am just thinking wrongly… do you maybe have an idea?

Hey @ChiikiBreeki ,

You are on the right path, and as I just discovered OpenCV did not give you everything you needed in the version you had.

I added a new node called FillPolygon in the Drawing category which will let you draw a full polygon instead of just the outline, which is what you need to create a mask on the face hull.

I made a simple example where I am blurring my face:

image

And here is the creepy result:

image

I hope that helps!

Ah, you will need to update your version of VL.OpenCV. I will push a new nuget now :)

Oops, I cannot push out a new nuget for now, technical issues I am afraid.

In the meantime, you can try to generate the mask in Skia instead.

I would expect to be able to generate a new nuget tomorrow (9/10/2022)

Thanks again very much!!
I have been struggeling for so long with this…!

Sadly I don’t know how to combine Skia and OpenCV :( wanted to ask that too later on since I wanted to know if I could put the image somehow on the Skia renderer

Hello again @ChiikiBreeki,

The new VL.OpenCV nuget (2.5.1) is now up so you should be able to duplicate my screenshot above once you update.

As for sharing stuff between OpenCV and Skia, you need to use the “Image” nodes. There are a few of them for different occasions, so have a look. They generally have names like FromImage, ToImage and so forth.

Cheers!

@ravazquez Thanks again very much! I forgot to answer to be honest :( I removed the blur as I was tweaking what I liked, now you are just a floating head :D!

Abgabe_KEYBOARD.vl (204.5 KB)

2 Likes

This topic was automatically closed 365 days after the last reply. New replies are no longer allowed.