Aruco Marker Tracking and Displaying image or video

Hello,
can anyone give me (am beginner in vvv and joined some nodeforum workshops) help concerning aruco tracking? probably just a simple question, but not really simple for a beginner like me…

Depending on the different Aruco tags, I would like to display different images or videos over the entire area on the renderer / screen. I can display a picture (but not over the whole area). The marker ID is also recognized. But how do I display a different image/video with sound over the entire area depending on the (Aruco) ID?

Patch is attached…

Thank you for your response.

1 Marker detection-JB.vl (24.6 KB)

You should have the rights now.

1 Like

hey Jo,

this patch shows how you map the marker IDs to a spread of images and select always one image to draw over the camera image, when a marker is detected.

1 Marker detection-JB2.vl (26.6 KB)

let us know if you need further help.

1 Like

hi joreg

thanks for your reply. I arranged the patch a little and get it… nearly…

only thing i do not get:

the spread of images needed to display the images depending from the marker ID consists of slices containing images?

Is this correct?
If so: How do i create the „image spread“ with the slices?

I put some jpg´s (1-5.jpg) into the Folder E/Download/xyz but this does not display any image in the render window.

I found in the help tutorial (F1) that i can manage items in a spread… but maybe this is not the right direction?

Would be very happy for some further tips – also if videos w sound can also be displayed additional to the images?

thanks a lot!

jo

1 Marker detection-JBx.vl (23.3 KB)

yes, in general we say: a spread consists of slices (its individual entries).

hm… the patch should actually work as is. the spread is created by the foreach loop. if your dir contains .jpgs you should see them when hovering the output splicer of the foreach loop:

grafik

true, this is not what you need here.

for videos, you’ll use the VideoPlayer node.

hi, just checked again. unortunately it does not work…cannot see any files. but it seems that the id does not get until the spread thing (the number in [0] stays zero). the aruco meanwhile is detected by the cam and the marker detector node.

i attach a screenshot and the patch again:

1 Marker detection-JBx.vl (23.3 KB)

Thank you!!!

the only explanation i can assume here is that the directory does not contain any *.jpg files then. what does the output of the Dir node say? have you tried hitting the Refresh bang?

Just test it. I create this(E:\DOWNLOAD\xyz) path on my computer, after clicking refresh i get all jpg images from that directory.

DO YOU USE CAPITAL LETTERS IN download directory name?
remark^ Just test it with name “download” also work for me. Try use simple names like: E:\1\1

Mb you have “.jpeg” images?

Hey joreg&poligrafp,

checked it again and it is working now. refresh and creating new jpg files helped.
Thanks a lot. Love vvvv and will play around more. and really clean vvvv gamma patch in comparison to the one(s) i used 2008 for the same task… (which are not working in vvvv gamma)

Thanks!!!

Hi,

now trying to display videos instead of images. I guess the “Image Reader” and “Draw Image” node have to be replaced to display videos in a directory?
Is there some kind of “Video Reader” (instead of Image reader) or similar Node which can be used to access the files from the directory? And the Videoplayer Node is the one replacing the draw image node?

or am I going in the wrong direction?

You just need to replace ImageReader with a VideoPlayer node, the DrawImage stays where it is. You will need to use a ToImage (OpenCV.Conversion) to be able to connect the output of VideoPlayer to DrawImage so add it inside the foreach right after the VideoPlayer.

Hope that helps

Thank you for the quick reply and tip - it works!

Can the video be played from start instead of looping ?
There is probably no possibility to get the audio of the video audible with the draw image or renderer, right?
Attached the last patch which shows video but without playing from start and playing sound…1 Marker detection-video-JBx.vl (30.6 KB)

You can use https://www.nuget.org/packages/VL.mediafoundation/ for sound and video
For Gamma 2020.2.2 use Version 0.0.6
For 2020.3 (preview with stride) you can use the latest version.

ok, thanks!

there is a videoplayer[Media Foundation] node within the vl.mediafoundation nuget.
seems that it is possible to set loop start time/loop end time and also volume there.

is this the right one for playing video sound and defining start/end time of displayed video?

unfortunately this node has an url as input and not a local directory.
if this is the right node can i define a directory instead of the url input ?

or is the direction of my thoughts wrong? (would be also a revelation ;) )

it takes both a remote or local file url.

SImple DIrectory 1 file player.vl (14.0 KB)
This is a project that will play 1 file from chosen directory with chosen index.

thank you.
it was not possible for me to connect the videoplayer[Media Foundation] node within the foreach.
but i integrated poligrafp´s simple directory player patch.
and: sound is playing and video start/loop time etc. can be edited.

it nearly works!

only thing is that there is still a frame of a video showing when new marker is detected.

any tip on how to “hide” this frame or where it comes from??

1-Marker-detection-video-starttime-sound.vl (21.1 KB)

If i understand you correct, before showing video that belongs to marker it show last video in folder. It happens, because IndexOf when got 0 at input “item” it result output -1. -1 is the last file in folder. Then, when program detect’s a marker, it change output, but player start play the last file (because it had it on input earlier) and then start playing a correct file.
Small switch will help:
1-Marker-detection-video-starttime-sound (1).vl (22.5 KB)