Hi. I’m completely new in the field of image tracking, so i beg your patience here ;-)
I am trying to do fiducial tracking on a new project. I tested the “FiducialTracker” example and it seems to be working alright when i use the reactivision fiducial patterns (image 1)
now my question is: how can i use other fiducial patterns, like this one (image 2) ?
how would i create a “map” or “dictionary” for the tracker to recognize patterns like these ? and how would i feed it to the “FiducialTracker” node, so that it uses them ? is that even possible with this node ?
or on a more general level: are there other probably better tracking options in vvvv for my custom patterns, as shown in image 2 ?
I don’t have a good answer for you at the moment but its something I\m looking into to.
For now you’ll need to create a Haarscade xml file, which is a process you’l probably have to do with a seperate tool. There is a tutorial here for how to produce one, but it isn’t for the faint hearted.
Thanks a lot for your explanations. Will look into it, but on first sight this seems to exceed my programming abilities ;-) plus it seems to be directed towards face recognition, which is not what i am after, i would need something much more simpler.
but then: even if i would succeed in creating such a .xml file (or use an existing one as a reference): how would i let the “FiducialTracker” node use it ? if the fiducial patterns that this node recognizes are hardcoded (so it seems), it wouldn’t be worth the effort.
So, again a question to the forum: are there other pattern/object tracking methods (not color or light !) in vvvv that i am not aware of ? like loading an (image) file with patterns that are then recognized by a node and mapped to IDs (like the “fiducial” node - only with customized patterns ?)
fiducial trackers are a very nice topic. short answer: yes it is possible to create your own trackers for your own patterns, will require some patching though.
i’ve started an implementation for the original fiducials in VL.OpenCV. not perfect still, but could give you an idea of how to approach your pattern.
meanwhile: where is that pattern from? do you have any link to the source of some documentation about it?
the pattern is from a stop-motion-film hardware from a company that has gone bankrupt. the hard- and software is still in use though and i am somehow trying to reverse-engineer it. a google image search has met no results, so i really don’t know if there’s any documentation about it at all.
it’s not really important to stick with those patterns though, the original fiducials will do as well. i just wanted to know in general if those patterns are hardcoded to the “fiducial tracker” node and how much of a hassle it would be to change them. as much as i have learned by now it’s much easier to stick with the original (reactivision) patterns.
======================================================================== @joreg wrote: yes it is possible to create your own trackers for your own patterns, will require some patching though
like - vvvv patching ? can you give me some hints or starting points ?
i will look into VL.OpenCV as well …
as for ARUCO: nice, but the same question as above: how would i let the vvvv node use them ? plus I am looking for circular pattern recognition.
Google searches have not yet been helpful in getting an insight into the fiducial theory (or maybe it’s over my head) as it only tells me that there ARE several fiducial patterns (square and circular) around, but no info on how to make your own or how to make software look for it.
i tried “REACTIVISION” - it uses simple text based files called “*.trees” to load a specific fiducial “dictionary” to compare against the captured camera image, then sends the detected IDs and coordinates via UDP. will investigate further …
still, it would be nice to do this with VVVV, without another software running in the back (and occupying the video cam !)
Anyway, thanks a lot for all your suggestions. probably will be back soon ;-)
VL.OpenCV FiducialTracker is the starting point. it is completely patched, so you can see how it is done and probably adapt to any similar pattern. for the theorie behind how they work, read these papers.
aruco also only works with VL.OpenCV and comes with a helppatch.
i’d be happy to help you further, but before digging into it, would be nice to have some paper about the exact fiducials you want to use. that would save some time figuring out everything yourself.
Hi. Joreg !
I’m on holidays for a week, so: sorry for late replies. Thanks a lot for all infos, will check them out carefully when i’m back. ATM i’m doing just research for a project that i’m not even sure will get commissioned. But i’m very interested in the subject nonetheless.
As for the pattern i sent you: i couldn’t find any paper on it, so i’m pretty sure it was some closed-source company thing back then. If this project comes to life, i think i will use fiducials that are open-source and documented, rather than to really reverse-engineer everything.
I did quite a lot of fiducial tracking in the past, did some research to understand the concept and to answer your original question - yes it is easily possible to make your own fiducials that even can be used with vanilla reactivision.
It has been some years and i hope i remember everything right: The fiducial concept used in reactivision is actually quite simple: the tracker searches for regions of a color that contains a certain number of other regions of the other color. take this fiducial you posted: it is made up of the following
black region (outer ring) → contains one white region → contains 8 black regions → {[contains 4 white regions], [contains 1 white region],[contains 2 white regions], [contains 3 white regions]}
you get the concept? check out the “.tree” files in the reactivision folder, they basically contain these hierarchies for each marker.
this one would be w0122221222122121111 (which is fiducial with id7)
w: starting with a fully contained white region (surrounded by the black ring)
0: the first white region
1: a black region
2222: containing 4 white dots
1: another black region
222: containing 3 white dots
1: black
22: 2 white dots
1: black
2: 1 white dot
1111: four black regions (dots)
it is quite easy to try out yourself, just grab an edding marker and draw some shapes that follow this hierarchy. by creating a .trees textfile you can specify which pattern correspondends which marker id (from line number).
position is somehow calculated from the center of mass of the regions, to make orientation calculation possible the patterns need to be somehow asymmetric.
also take care that the patterns are easy to recognize by a cam. your marker posted above will probably get some problems in the edge-region, as the border is very thins and the cam might not be able to clearly resolve these regions as separate → inner and outer black region become one → hierarchy is messed up and will not be recognized. afaik the original fiducial marker set were “designed” by a genetic algorithm, that optimized distances between the regions to be as large as possible.
your own tree files can be fed to reactivision.exe via commandline switch or config file. i also do recommend using reactivision.exe. it is superstable and configurable. also its load will not influence your vvvv patch, just use its TUIO receivers. this app could also run on a different computer and send the TUIO data via network.
Wow!
Thanks a big bunch for this incredibly useful and informative tutorial. I was always wondering about the .tree files and how the coding exactly works. As soon as i am home i will try some stuff (and post results if useful). My own fiducials would indeed be much more raw, as i only need about 40 in whole (for which i would do 20 “normal” ones and then invert them).
