Forum

Guitar Fretboard tracking

vvvv beta question
#1

Hi guys,

I’m a student from Germany in the 8. semester of Media Systems at HAW Hamburg.
Currently I am working on my Bachelor of Science.
Therefor I need to know how to track the pattern of a guitar fretboard (that’s the ‘neck’ of a guitar).

So I have chosen vvvv for my project because it is a simple and powerful tool in comparison to other programming languages.

Now my question: Can you tell me which nodes you would use for the pattern tracking of the guitar fretboard? I have already informed myself about contour, blob tracking and detect object but I am running out of time and need the right solution fast.

Please help!

Best wishes Friiky

#2

this is quite advanced. big problem is knowing when someone is pressing their finger or just hovering above the string. Bar chords (is that what their called?) will be tricky to detect.

how much time do you have?

Have you tried different colour paper stuck to each finger and blob tracked them? Can coloured paper be obscured in some finger positions?

Speed of moving fingers will require fast camera.

The more fret board you want to track the less resolution you’ll have with the camera and tracking will be harder. Lots of cameras could help with this but that brings lots of other challenges.

Research? Has anyone else done this before? Sensor gloves - are they an option? Make your own sensors on the end of fingers?

I’d say you’d need to fix the camera to the guitar so the fret board is always static in the camera view.

I’m sure there is more to this. As said it’s a pretty advanced project. Maybe think of simplifying it somehow.

#3

yes i would also reconsider this whole thing, btw with keyboard you would have it super easy :)) othervise, i would go with some kind of arduino capacitance sensing, camera is rly not good for this

only logic that actually comes to my mind is to take snapshot every time signal from guitar reaches a critical point (i mean checking volume, so u know when u strike the strings)

#4

If your bachelor thesis is not about the optical tracking, but rather about some processing of the result you might consider using one of those midi pickup thingies.

#5

hey it is pretty easy

is a string pressed? -> just connect the string to an arduino and see if the value changes - im sure it does - when you touch it

where is the string pressed? -> camera can do simple tracking with background substraction so you know where the hand is. Another solution would be a ultrasonic distance sensor that would also give you the position of the hand.

those two together are enough to make an approximation of which half tones on the guitar neck are pressed.

if you know the tuning of the guitar and the system - by the way - you can also make everything from only the sound!

#6

Make a digital connection to each fret (you need to salvage a neck of course), do the same for each string (and remember to isolate the strings from the bridge) , there you go you need about 18x6 digital matrix to know which note is playing on each string. It may not be perfect but …

@tekcor I tried once that but it s pretty difficult to get the position of each finger

#7

First of all thanks for the fast answers but they depart from my thoughts.

What I want is:

vvvv should recognise the pattern of the fretboard. So when I hold my guitar fretboard in front of the cam the program should track it because it recognises the grid pattern.
After this it should mark a position on the grid pattern which is defined by myself. (When a specific tone on the guitar is played the marker should switch on a new position on the grid.)

So it is all about the grid pattern which should be recognized by vvvv.
Any solutions?

Best wishes Friiky

#8

somehow i have the feeling that this thread is more about cv and haar cascade training than vvvv. maybe give VVVV.Packs.Image a try.

#9

If you need a general solution that will work with every guitar then as m4d pointed out probably some form of CV training is needed (not really a patching thing that).

If you want it to just work with your guitar for your project a much easier way could be to just get some info about the neck pose, and reconstruct the grid from that. You could have a few markers on the guitar for example that give you a transformation you can apply to your (fret) grid points.

#10

I think this is going to be very tricky, as your fingers could occlude each other, hand positions can vary, the pattern matching isnt the hard bit, its getting the pattern in the first place. Maybe a leap motion could track your fingers, but this isn’t a trivial concept, the most robust is the matrix of sensors, or a button under every string and fret…
If you succeed you can probably sell it to roland for a guitar synth…

© vvvv group