Problem with affine transformation using OpenCVSharp Nuget

Hello everyone!

This question might be slightly out of place since it’s not strictly a gamma question - apologies for that.

Regarding my problem: I am trying to build a face-swapping application that will take an input image of a person, detect and extract the face in that image. Afterward, the extracted face should be imposed onto the face of another person’s image in a somewhat convincing way (taking into account varying angles and such). Basically, I am following the procedure outlined here:, but I am rebuilding it in vvvv.

I used the DlibDotNet NuGet package to detect and extract faces using dlib’s 68-landmark feature, which worked fine. Then, I split the face image into various triangles using Delaunay triangulation and the 68 landmark points. Now comes the crucial step: each extracted triangular image from image 1 has a corresponding triangle in image 2, but they are, of course, of different sizes and have different angles. This is where the getAffineTransform method comes in (provided by the OpenCVSharp4 NuGet package), which generates a transformation matrix that can be used to apply an affine transformation onto the triangles from image 1 so they fit the triangles in image 2.

However, the triangles returned after applying the transformation are not as expected.

The input looks like this:

Screenshot 2023-04-27 113510

and this is the output i get:

Screenshot 2023-04-27 113411

The output is a mirrored image (which can be accredited to the border extrapolation method being mirrored) but i am not that well versed in opencv or linear algebra to understand the underlying issue here.

This video shows a bit better what kind of output I am trying to achieve:

So I was hoping someone here might be able to give me a hint of what might be going on here.
Any help is greatly appreciated!

This is my patch:

Edit: I was told @ravazquez might be able to help?

i want to have ago at the problem, but I am missing fillpolygon, are you using some specific version of opencv?

Hi @StiX thanks for the reply. I use the vl.OpenCV Nuget for that. Latest version 2.5.1 if that helps.

hm that is the one I use, but I am still missing that node

Hmmm. Should be the one. Can you just replace it with the one you have?

there is fillpoly and fillconvexpoly in gamma 5.0, but they dont have output. This node works in gamma 4.12, so there is something going in between vl versions, that is making problems for me

I think fillpoly and fillconvexpoly are part of the OpenCVSharp Nuget, they didn’t work for me either. That’s why I used the FillPolygon Method featured in the VL.OpenCV wrapper. I am using vvvv 5.0. Also shouldn’t the referenced Packages be independent from the vvvv version? I am at a loss why this is happening with your version. What does the projects menu say about missing dependencies?

Edit: Are you using stable or preview version of vvvv? Maybe this could be a problem.

I just had to reinstall the opencv nuggets, and now its ok. Had to delete all folders.

1 Like