-1

I'm making a piece of software that is able to recognize Magic the Gathering Cards from a picture or the actual camera.

The codebase scans for rectangular shapes (cards) in an image and calculates 64bit perceptual hashes for them, which get compared to a precompiled database of approximately 40.000 hashes for all the magic cards. The hashes are a perceptual hashing algorithm i can explain further (kinda like JPEG with DCT).

The database also gets compiled with an average hamming distance between all the individual card hashes and standard deviations for that. The average hamming distance is a difference of 24bits of the hashes and the standard deviation is around 5.3.

There are several cards in whole database, which only have 2 different bits when I compare. And a few with 1 bit difference. This means that it will be hard to discriminate all the cards from each other. It will be really hard to discriminate an actual picture of a card.

I'm wondering whether I could use other or better approaches to narrow down the potential candidates for recognition. I've considered a second step, perhaps with a bit of textrecognition, but it seems a bit cumbersome and probably not viable since textrecognition will be hard on blurry camerashots. I've considered using a database of hashes based on higher contrast images (currently the database is from original pictures from scryfall)

Do you guys have ideas for other algorithms or adjustments I could make for better discrimination between so many similliar objects?

Let me know if you want me to post the steps of the algorithm.

candied_orange
  • 102,279
  • 24
  • 197
  • 315
Martin
  • 21
  • 3
    This is a fairly specific topic but the question is written as if it's basic knowledge. It's also written in a very narrative style that doesn't lend itself to the kind of answers you will get here. I would suggest starting by adding inline links to things like 'hamming distance' and 'DCT'. – JimmyJames Aug 10 '23 at 20:39
  • maybe this link would be useful? google image search seems to be able to find the two cards in CO's answer : https://alibaba-cloud.medium.com/google-image-search-explained-30af8ba9cbea – Ewan Aug 10 '23 at 22:59
  • @Ewan : That's exactly what i do.. More or less! – Martin Aug 11 '23 at 06:15
  • @JimmyJames: Alright, I never seem to figure out where on earth I can ask such questions in regards to computer science. I seldomly have specific questions. Everyone wants a simple question of sorts - chatgpt is too dumb to know what it's talking about. It's somewhat frustrating – Martin Aug 11 '23 at 06:19
  • @Martin I can understand your frustration. I think you could get some help here, but you will need to put a little more work into the question. Links to references at a minimum. I do think adding some psuedo-code or listing the steps would help too. – JimmyJames Aug 11 '23 at 15:47

1 Answers1

2

Two suggestions:

  • Try a different perceptual hashing algorithm
  • Crop out features common to every card

Different algorithms will react differently to commonalities. A good test is worth 1000 expert opinions.

enter image description here enter image description here

A MtG card has a lot of features identical to the other cards. Especially if they're from the same color school. If they're from the same production run even the colored frame will be the same.

Fortunately the art is unique to each card type. Try cropping down to just that and see if it gets you what you want.

If that’s not enough to avoid collisions on its own consider hashing other unique features as well. Each hash you add may contribute unique bits.

The risk is that different scans of the same card start to look different. So don’t trust. Test.

candied_orange
  • 102,279
  • 24
  • 197
  • 315
  • you could also try to separate other features that would narrow the search, the colour and the cost could be grabbed separately from the image hash – Ewan Aug 10 '23 at 23:13
  • @ewan better now? – candied_orange Aug 11 '23 at 00:15
  • @candied_orange: Thank you for your suggestions! I do use tests alot, since it's a lot of cards and don't know how much different methods change my data! Your post made me think of splitting the card in two and processing the art/upper part. But this doesn't help in terms of "fullart" cards. Maybe if i cranck up the contrast for the database and even more so for the searched card it will get better at discriminating – Martin Aug 11 '23 at 06:24
  • @Ewan: Thank you! Maybe i could make a combined hash of sorts. – Martin Aug 11 '23 at 06:25
  • @Martin if [this](https://customizedmtg.com/wp-content/uploads/2021/06/Arid-Mesa.png) is what you mean by "fullart" then cropping out common features is still worth a try. The idea is to create a situation more like what the creators of the image hashing algorithm had in mind. Getting rid of the text may help do that. – candied_orange Aug 11 '23 at 09:04
  • @candied_orange: I'll try it out, maybe make a combined hash of sorts! So far applying a CLAHE to the database images makes them a lot more distinct, the average difference in the hashes go up and the standard deviation goes down. In a smaller sample it actually only found relevant matches. If i go down that path the next headache will try to get an image from a camera to best match the database. – Martin Aug 11 '23 at 09:29