I’m trying to play with openCV, I would develope kind of face filters
From the ObjectDetector I take the data with GetItem(spread) and then Split(Rectangle) to let the various shapes track the face
But doesn’t work really good (patch attached), is there any other method? for example with Yolo?
Plus I was wondering how to put filters only in a determined area of a recognized area, for example removing static background or extracting contours
Hi @chronoPan, for face filters what you usually want is something called FaceMarks. This is sadly currently not supported in VL.OpenCV. It has been recently added to the underlying library OpenCVSharp, but in order to integrate those changes we need to move other things around. It will come but I cannot say exactly when.
In your example I noticed the application was hanging when trying to GetItem on Spreads with no slices in them.
I added a couple of if regions to prevent that form happening and tweaked the ObjectDetector’s Min Neighbors and its working relatively well for me.
Forgot to mention, if your camera has a properties/settings dialog (try enabling the VideoIn’s ShowProperties pin) make sure you have anything related to low light compensation or auto brightness disabled since these features are known to cause lower camera framerates.
I guess you could try to use ObjectDetection and HaarCascade files to get eyes, mouth, etc., then calculate each rectangle center and use those points to make a Voronoi triangulation. That should already be somewhat close to a rudimentary form of FaceMarks but to be honest, tuning such a system in sounds complicated.
Another option would be to combine ObjectDetection with contour detection to get more accurate readings on eyes, mouth, etc, but this also requires a lot of tuning for the specific scenario, lighting, camera, etc.
If you are going for a realtime solution I would suggest using the Kinect2 Face or FaceHD nodes instead, these already give you more information on face specific features which you can use to further develop your solution. The drawback with this is that it will only work realtime (you will not be able to use old footage or pictures for example).