3

I'm aiming to translate sounds played from the guitar to guitar tabs (accurately).

I've had a look at solutions for detecting which string is being plucked: Detect string plucking in electric bass guitar?

Problem is that the sound waves won't reflect which fret the strings are being played from as different tabs could produce the same sound frequencies. So I concluded purely relying on sound frequencies won't do the job (accurately). I can't have any hardware that interferes with the string touching the fret board as it could effect the performance of the guitar. But I need a solution around it, a way to detect when the string has touched the fret board (and which fret on the board as well) + how long the string has been on the fret (for accurate tabs generated).

I think the ability to detect all of those would allow me to generate accurate tabs. Does anyone have solutions as to how I can achieve this? I don't think a pure software solution would work and would be great to hear your ideas.

thatguyjono
  • 131
  • 1
  • It's a great idea. I don't know of a solution, but I can suggest some thoughts. Playing the same C on the 3rd and 4th string should result in different ***relationships*** of the magnitudes of their harmonics -- particularly the higher ones -- because of the string length (and perhaps mass per unit length) and the order of supportable standing waves on it. If I were doing this, I'd get set up to digitize and then analyze these relationships to see if this hypothesis holds up (or else see if someone else has done the work for me.) – jonk Aug 29 '19 at 00:28
  • 1
    Just detecting which string is vibrating at which note should be enough...would take less hardware and installation space than searching through each fret. I think designing a pickup for 6 individual strings is easier than designing a fret detector for every fret while trying not to interfere – DKNguyen Aug 29 '19 at 00:37
  • 1
    I Agree with @DKNguyen if you use metal string, you could try to have a coil detecting the vibration for each string (six coils in total), that could possibly be placed near the bridge and wouldn't get in the way of playing the guitar – jDAQ Aug 29 '19 at 00:40
  • This is a good start https://electronics.stackexchange.com/questions/82452/which-sensor-to-use-for-detecting-nylon-string-vibration – jDAQ Aug 29 '19 at 01:02
  • Detecting vibration won’t work. Takes too long for the pluck displacement to reflect off the fret and fully establish a clear standing wave. Until then, any vibration is likely at the wrong pitch (that of an attack transient). – hotpaw2 Aug 29 '19 at 01:02
  • 1
    @hotpaw2 I dont see how that is an issue when software can just register the time of attack and assign the attack to a note after the fact – DKNguyen Aug 29 '19 at 01:07
  • 1
    If the detection doesn't need to output the result instantly (with all the processing in the Arduino) you could apply better signal processing to detect the beginning of the notes and which note it was. And, I disagree it "takes too long" for the string to go into a steady-state, after all, the microcontroller operates in the timescale of 1 us and the finger of the guitarist are plucking at ~200 ms, so, by at most 200 ms the notes should be as clear as they get, or the fastest notes would be unrecognizable. – jDAQ Aug 29 '19 at 01:17
  • any reason you can't use a MIDI guitar pick-up (like the Fishman "Triple Play"), or similar? –  Aug 29 '19 at 01:42
  • 1
    Maybe if you electrified each guitar string (assuming they are metal strings) with a low voltage/current. .5v for string 1, .7v for string 2, .8v for string 3, etc. Then use the metal frets connected to an ADC to determine which voltage is connected to the fret. Some creative math and you could probably detect multiple strings on each fret... Not sure what kind of access you have to the fret/neck... – Ron Beyer Aug 29 '19 at 01:46
  • Thanks for the suggestions and comments guys. Anyone know if the line type laser from @jDAQ comment would be more accurate than coil detection? I think I can only go with one solution. – thatguyjono Aug 29 '19 at 01:56
  • well laser works well on any opaque surface (nylon and metal string) but the coils only work on magnetic metals. – jDAQ Aug 29 '19 at 02:14
  • @Ron Beyer that's a great solution if the guitar neck can be altered, in fact, you could read the frets pressed as if it was a 6x12 matrix keypad (no different voltages but some sort of scanning/polling). But the wiring of the frets would have to be done from the inside of the guitar neck, for most players use the whole neck of the guitar and it would hinder their performance. – jDAQ Aug 29 '19 at 02:27
  • 1
    @jDAQ Maybe, you should be able to machine out the back of the neck and put the wires in, then a wireless/battery transmitter box that you attach near the headstock. You can then fill in the machined-out area with epoxy, shouldn't have anything hindering the guitarists hands from sliding up and down the neck. Could even drill down to the fret and connect it from the back. – Ron Beyer Aug 29 '19 at 02:33
  • I wouldnt use a laser since you might look right into it given its location. Infrared or eve visible LEDs would work fine. Its the photodiode or phototransistor that matters. IR has a tendency to pass straight through plastics though. Unsure about nylon specifically, but nylon string are clear. Probably a higher parts count too. – DKNguyen Aug 29 '19 at 02:33
  • @DKNguyen if it's accurate enough that's fine, I didn't really life the fact that I needed an image sensor. I feel like the less bulky hardware on the guitar would be a decent solution. I also can't do any hard refactoring (drilling etc) on the guitar, my goal was to have an easy to install solution. – thatguyjono Aug 29 '19 at 03:49
  • 1
    @thatguyjono Have you seen the "individual string pickups" that are for sale? You might not need to build anything at all other than the processing hardware. An optical pickup seems like it could be made thinner though but you would have to test to see how crisp the signal is as the string passes in front of the center of the optical sensor since narrow field of view works better but thinner optical sensors tend to have a wider field of view. – DKNguyen Aug 29 '19 at 04:17
  • Perhaps this question is a partial dup of this one: https://electronics.stackexchange.com/questions/450178/guitar-to-midi-conversion/450186#450186 – hotpaw2 Aug 29 '19 at 05:48
  • 2
    @thatguyjono: See my answer to [Expresiv MIDI guitar implementation](https://electronics.stackexchange.com/questions/259364/expressiv-midi-guitar-implementation/259686#259686) which discusses the fretboard side of the problem. – Transistor Aug 29 '19 at 06:15
  • Thanks @Transistor it looks very interesting. Based on what people have said, it looks very accurate. – thatguyjono Aug 29 '19 at 19:14

2 Answers2

1

A good MEMS accelerometer can operate throughout the full audible range (https://www.analog.com/media/en/technical-documentation/data-sheets/adxl1005.pdf). If you strategically placed a few of these on the guitar I bet you could pick out the phase differences based on the vibration of the guitar.

TuxRobotics
  • 84
  • 3
  • Yes. With 2 mems, you might be able to measure when the first traveling wave from a pluck reflects off the fret, allowing you to determine determine which fret (by distance). – hotpaw2 Aug 29 '19 at 04:10
0

If it's possible to build a special neck, a strain sensor on each of the fret bars could detect a press on the fretboard.

Another solution would be to use machine vision approach to watch the player's fingers. Here's a paper about that: https://pdfs.semanticscholar.org/1aad/47eaa4e6c9e2b5b69c4d7d7f4d0c87d03e1a.pdf

Another: http://joey.scarr.co.nz/pdf/autotab.pdf

Here's yet another approach using CNN (Convolutional Neural Networks) to detect tabs: https://towardsdatascience.com/audio-to-guitar-tab-with-deep-learning-d76e12717f81

hacktastical
  • 49,832
  • 2
  • 47
  • 138