-1

The idea of programming katas is kind of widespread at this point. A lot of people know what that is, not a lot of people actually practice it.

Furthermore, you can find a lot of resources about coding katas on the internet, majority saying that katas can improve code quality, programming skills and etc. Still, there's a wide range of different approaches or methods around, each one proposing different problems, different ways to tackle them and other details.

In the end I struggle to find concrete evidence about this matter. That's why my question: is there scientific evidence showing that katas work?

  • 2
    see [Is empirical evidence off topic here?](https://softwareengineering.meta.stackexchange.com/a/7573/31260) – gnat Nov 25 '19 at 09:42
  • I'd never heard of the concept until now. I'd also be sceptical of a method described by Wiki as based on "practice and repetition", when the most intractable elements of code quality usually relate to the design, not the rote activities. – Steve Nov 25 '19 at 10:03
  • I've honestly *never* heard this word before, so I think your premise in the first sentence is wrong. A quick Google search makes me believe it is the usual unscientific hot air, though. I'm not surprised to see the name "Robert C. Martin" pop up in the corresponding Wikipedia article. – Christian Hackl Nov 27 '19 at 08:02

2 Answers2

4

I am a seasoned developer and I never heard of the phenomenon so I am not so sure about the popularity of so called katas. I had to look up the term and found they are just programming exercises.

Can there ever be proof of exercises improving skills? It would be a pretty pointless experiment. If you do something a lot you will likely get better at it but software development is not a one dimensional skill.

The benefit will be closely related to the similarity of the exercises on one hand and the problems you want to get better at solving on the other hand.

If you play a lot of scales you will get better at playing scales. Will it make you a better musician? Maybe, maybe not. Some will get better because it helps them fit the pieces together, others will just develop motor skills.

From some point on just doing random exercises is not likely to help you get anywhere. Any exercise should be targeted at making you take the next step in the direction you want to go.

Martin Maat
  • 18,218
  • 3
  • 30
  • 57
3

The challenge with coming up with valid empirical evidence for anything is dealing with dependent variables. Dependent variables are things that tend to be linked in the subject under test. For example, many people who smoke tend to do little exercise. So when trying to determine whether smoking increases the risk of heart attacks, I have to turn exercise into an independent variable, eg by ensuring my group of smokers and my control group do the same amount of exercise. That way, I ensure that any differences between the two groups are related to smoking only.

What does this have to do with the question? Well the same applies to code katas. It might be the case that people who tend to do code katas are also the sort of people who listen to podcasts, read blogs, attend conferences etc. So were I to try and scientifically measure whether there was a significant difference between coders who do katas and those that don't, I'd have to factor all those dependent variables out. Even then, it might be a case that the kata group and more likely to drink decaf coffee and caffeine levels might affect coding skills, so I have to factor that out too. And so the list would go on. There are so many dependent variables around coding that it would be hard to deal with all of them.

And even if I could, how would I measure whether katas make someone a better coder. A "good coder" is too subjective to measure in this regard, not least because there is no universally agreed "best practice" to writing code that fits every development scenario.

So the answer to your question, "is there scientific evidence showing that katas work?" is very likely "no". It would be a very difficult study to set up and anyone that did would struggle to have their data set accepted as reliable as the assumptions they'd have to make around removing those dependent variables and the definition of what's being measured would be all too easy to criticise as false assumptions.

David Arno
  • 38,972
  • 9
  • 88
  • 121