140

Occasionally I see questions about edge cases and other weirdness on Stack Overflow that are easily answered by the likes of Jon Skeet and Eric Lippert, demonstrating a deep knowledge of the language and its many intricacies, like this one:

You might think that in order to use a foreach loop, the collection you are iterating over must implement IEnumerable or IEnumerable<T>. But as it turns out, that is not actually a requirement. What is required is that the type of the collection must have a public method called GetEnumerator, and that must return some type that has a public property getter called Current and a public method MoveNext that returns a bool. If the compiler can determine that all of those requirements are met then the code is generated to use those methods. Only if those requirements are not met do we check to see if the object implements IEnumerable or IEnumerable<T>.

That's cool stuff to know. I can understand why Eric knows this; he's on the compiler team, so he has to know. But what about those who demonstrate such deep knowledge who are not insiders?

How do mere mortals (who are not on the C# compiler team) find out about stuff like this?

Specifically, are there methods these folks use to systematically root out such knowledge, explore it and internalize it (make it their own)?

Robert Harvey
  • 198,589
  • 55
  • 464
  • 673
  • 11
    I think this is particularly where open source software shines. It's nice to be able to step into the framework/system/libraries all the way down. I used to have a way better understand of framework internals back when I worked with Qt than when I was working with WinForms. – Vitor Py Jul 11 '11 at 20:46
  • 2
    When would you need to know this specific example, other than not looking dumb in front of a special crowd? They idiot-proofed this. Other than that, the Effective C#, Java, C++, etc. series might have some cool things in it. Eric Lippert's blog is a good source as well. In general, we often do not know what do not know, so as they say "live for 100 years, learn for 100 years and die a fool". – Job Jul 11 '11 at 20:46
  • @Vitor Braga, even if the source code was available to you (and it is for the libraries) ... if the 'foreach' always worked when you implemented `IEnumerable and IEnumerable`, then why would you bother climbing up the ladder? http://freekvermeulen.blogspot.com/2008/08/monkey-story-experiment-involved-5.html – Job Jul 11 '11 at 20:49
  • @Job Yes, I agree that source availability is not the (only) answer to this question (this is why I commented on it, not answered). I just believe that having source code available makes it easier to have a deeper understanding of a library/framework/so on. I agree with you that this particular example would be hard or impossible to find when debugging, even with the source at hand. Others might be more feasible. – Vitor Py Jul 11 '11 at 20:53
  • The IEnumerable example is just that; an example. Whether it's good practice or not, knowing this gives you power; you can actually write a class that works with `foreach` without implementing `IEnumerable`. Who knew? Perhaps the ones who understand this stuff are the ones who know how to bend the design without actually breaking it. There are use cases for writing such interface-less code; Linq is one such example (see later in Eric's blog entry for this). – Robert Harvey Jul 11 '11 at 20:56
  • 26
    Is it worth the effort? I am bilingual and trying to learn a few other spoken languages. I have taken some math classes but not enough of them. I would like to learn how to play tennis half-decently and learn to swim using butterfly stroke. I would like to travel more. I want to learn some Clojure. What I do not want is to be expert in one language, to have a PhD in math, to spend 30 hrs per week in a pool like Michael Phelps, etc. Lippert's and Skeet's knowledge is due to the fact that they put a lot of effort in one (or a few) things while missing out on other experiences. Maybe change job? – Job Jul 11 '11 at 20:57
  • 10
    "I can understand why Eric knows this; he's on the compiler team, so he has to know." - chances are he knows this *because he thought it up in the first place*. I doubt he had to 'find out' that it works like this :) – Alex ten Brink Jul 13 '11 at 11:13
  • 10
    @Alex: I've actually only worked on C# since we started actually building the implementation of C# 3. The "foreach" specification was written over six years before that. I still find out crazy historical things about the language every day. For example, I learned today that for delegates, ((A + B) + C) - (A + C) = A + B + C, but ((A + B) + C) - (B + C) = A. Weird! – Eric Lippert Jul 15 '11 at 06:08
  • That *is* weird. Are delegates combined like stacks and does delegate removal silently fail if the smaller stack doesn't match the top of the other stack? – Alex ten Brink Jul 15 '11 at 21:07
  • You'd be surprised at how simple jQuery seems once you understand how to apply monads in JavaScript. – Evan Plaice Feb 19 '14 at 02:25
  • Spend one hour pondering a question, then 9999 hours learning the answer. When I was 7, I asked my older brother (this was in the early 70s) "What are computers *for*? What do they *do*?" 30 years later, I had been to school, held a programming job, and finished being interested in my original question. Along the way I learned a *lot* about TCP and socket programming, multi-threading, data replication and other complex topics. And got paid some of that time. You learn by being interested enough to really go deep in to whatever area you are asked to work on. –  Mar 28 '17 at 18:00

10 Answers10

172

First off, thanks for the kind words.

If you want to get a deep knowledge of C# it is undoubtedly an advantage to have the language specification, ten years of design notes, the source code, the bug database, and Anders, Mads, Scott and Peter just down the hall. I'm certainly fortunate, no question about it.

However, even without those advantages it is still possible to get a deep knowledge of the subject.

Back when I started at Microsoft I was working on the JScript interpreter that shipped with Internet Explorer 3. My manager at the time told me something that was some of the best advice I've ever gotten. He said that he wanted me to become the recognized expert at Microsoft on the syntax and semantics of the JScript language, and that I should go about this by seeking out questions on those aspects of JScript and answering them. Particularly answering the questions I didn't know the answers to, because those are the ones I would learn from.

Obviously StackOverflow and other public Q&A forums are like drinking from a firehose for that sort of thing. Back then, I read comp.lang.javascript and our internal Microsoft "JS User" forums religiously and followed my manager's advice: when I saw a question that was about the language semantics that I didn't know the answer to, I made it my business to find out.

If you want to do a "deep dive" like that, you've got to choose carefully. I to this day am remarkably ignorant of how the browser object model works. Since I have been concentrating on becoming the C# language expert these last years, I am remarkably ignorant of how the various classes in the base class libraries work. I am fortunate in that I have a job that prizes specific deep knowledge; if your job or your talents are more in line with being a generalist, going deep might not work for you.

Writing a blog is also tremendously helpful; by requiring me to explain complex topics to other people, I am forced to confront my own inadequate understanding of various topics all the time.

Eric Lippert
  • 45,799
  • 22
  • 87
  • 126
  • 15
    Not to drag this off-topic, but after reading this answer I am curious about why you haven't asked any questions here or on Stack Overflow. Are your colleagues, blog, etc. sufficient for you at this point? Are there better resources than SO which we should know about? – Matthew Read Jun 01 '12 at 22:38
  • 7
    Perhaps you misunderstood what he's saying. Counter-intuitively, he wasn't asking questions to learn things, he was *answering* questions. – jhocking Nov 01 '13 at 13:01
  • 1
    @jhocking, "learning by answering" must be one of the most profound things I've heard this year. Providing *answers* to questions put by others is indeed an extremely effective method of learning - and you don't just learn about the topic yourself, you learn about how other people think about the topic, and how to explain it to others. – Steve Mar 28 '20 at 18:18
66

Having been on the "guru" side of the conversation once or twice, I can tell you that a lot of times what you perceive as "deep knowledge" of a programming language or system is often the result of the "guru" recently struggling for a month to solve the exact same problem. That's especially true on a forum where people can choose which questions they will answer. Even the likes of Jon Skeet and Eric Lippert had to learn hello world at one point. They pick up their knowledge one concept at a time, same as anyone else.

Karl Bielefeldt
  • 146,727
  • 38
  • 279
  • 479
  • 1
    A very good point. I often find when embarking on long periods of research that I find questions I can now answer due to things I learned earlier in the day. – Matthew Read Jun 01 '12 at 22:11
47

Paraphrasing Yogi Bhajan:

"If you want to learn something, read about it; if you want to understand something, write about it; if you want to master something, program it."

Programming is like the ultimate teaching challenge. Teaching computer do something requires, that you know your stuff really well -- or you will learn to master it.

For example, if you want to learn physics, write a physics engine. If you want to learn chess, program a chess game. If you want to learn deep C# knowledge, write a C# compiler (or some other tool).

Maglob
  • 3,839
  • 1
  • 24
  • 27
  • 2
    Programming is also a modest attempt at writing (to be read by people, of course) in a most unambiguous way. – vpit3833 Jul 12 '11 at 23:09
  • 4
    That quote sounded really deep until I read the chess example. Unfortunately programming a chess AI will not make you a better chess player (it's basically a search in a Min-Max tree). Still +1 – bughi Aug 23 '12 at 15:04
  • 1
    @bughi Maybe you can master the rules :D – Julio Rodrigues Oct 18 '12 at 18:50
  • @bughi, 'program it' is very broad term not always related to writing code!! Just think a little out of the box. – Nitesh Verma Aug 24 '13 at 09:28
25

As far as I know, the ways to learn this are:

  • Read about it from someone like Eric Lippert
  • Experience and then solve the problems first hand.

The second way may take much longer but will probably result in a deeper understanding (but not always).

Michael K
  • 15,539
  • 9
  • 61
  • 93
FrustratedWithFormsDesigner
  • 46,105
  • 7
  • 126
  • 176
24

I would say do the following:

After learning a relatively useful stack of languages (the ones you need for a real job) at the level where you can do most common tasks, stop learning more languages until you have studied at least one in depth. Part of the problem in our industry right now, in my opinion, is that people only learn the first 5-10% of the language before moving on to some other language. Once you have the capability to do most common tasks in a job, then start looking at one thing in depth. (You can go back to getting breadth after you get some depth, then go back and forth between the two.)

Volunteer for the more complex, harder tasks, the ones that make you have to go in depth to solve the problems. If there are none where you work, look for open source tasks to do or start working on a personal project that will make you have to go in depth. If your job has no interesting problems, consider looking for a more challenging job.

Read the advanced books on one language (for SQl Server for instance this would include reading about performance tuning and database internals) instead of the learn X in 30 days type of books.

Read the interesting questions here and other places where they are asked and try to solve some yourself. If you want to learn, try to solve some without reading the other answers first. Even if the question has already been answered, you will learn more if you find the answer yourself. You might even find a better answer than the one the question had.

Ask a few of the harder questions. Evaluate the answers you are given, don't just use them. Make sure to understand why the answer would or would not work. Use those answers as a starting place to research.

Find some good technical blogs from known experts in the field and read them.

Stop throwing away your knowledge after you are finished with it. Learn to retain. Most experts don't have to look up the common syntax. They don't have to reinvent the wheel every time they face a problem because they remember how they approached a simliar problem before. They can connect the dots and see how problem X that they did two years ago is similar to problem Y that they have right now (it amazes me how few people seem able to make connections like that). Consequently, they have more time available to spend researching more interesting subjects.

HLGEM
  • 28,709
  • 4
  • 67
  • 116
  • A nice answer. But I'm left wondering, how do I get better at retaining knowledge and connecting the dots? –  Oct 03 '12 at 15:02
  • I suggest you take notes of whatever you learn. I began doing this in my Evernote, and over a few years, I found that I can live off my notes. Slowly, I have also come to the point where my notes can be made into presentations, which I'm ready to present at a moment's notice. – Shivasubramanian A May 10 '19 at 12:10
10

You can start by deeply studying the language specifications of the ones that you seek to be an expert off. For example:

Marcelo
  • 609
  • 6
  • 11
  • 3
    Good answer - for example, section 15.8.4 of the linked C# specification covers the implementation of `foreach`, and spells out the behaviour described in the quoted blog post from Eric Lippert. If anyone ever finds themselves thinking something like "I wonder how foreach *really* works.." this would be a good place to start looking. – Carson63000 Jul 12 '11 at 01:06
6

Get Reflector or any other decompiler (since it's paying now), and start opening up some of the most used .NET libraries to learn how the internals work. Combined with a book like CLR via C# you'll get quite deep (deeper than most of us will go on their regular job).

Bart
  • 748
  • 4
  • 10
4

Deep knowledge and programming expertise means being comfortable at all abstraction levels. I.e.

  • libraries and APIs
  • language semantics
  • compiler optimizations
  • compiler internals and code generation
  • runtime and garbage collector behavior
  • architectural and instruction set issues

Everything I've seen in the last 15 years has shown that only if you really can get into the compiler and runtime do you have a chance of becoming deeply proficient. You may have to force yourself to take the step and start reasoning (and building) software at the next level of abstraction lower in the stack, but its the only way to expertise.

All we have is language for abstraction. You must understand how programming languages are designed and built to really know what the machine is doing.

Don Stewart
  • 1,177
  • 6
  • 8
4

I developed that kind of knowledge in C++ by hanging out in comp.lang.c++.moderated for a couple years, even though I wasn't really working that hard to code in it at that point. I'm not sure how guru I can say I am, though.

I think there's two kinds of knowledge that one can pick up about a programming language:

  1. Knowing trivia about the language, and knowing how to avoid pitfalls.
  2. Knowing how to solve problems effectively.

Number 2 can only be achieved by programming in the language and looking at other people's code, but number 1 can be achieved by taking lots of time to read about the language on its discussion forums, seeing what kinds of questions people ask, and what the answers are. StackOverflow is a good place for that too.

Ken Bloom
  • 2,384
  • 16
  • 20
3

Read The Fine Manual This is not particularly deep knowledge. It's published in the C# language specification section 8.6.4. You should get in the habit of at least skimming the specifications for the languages that you use, as well as skimming the documentation for all the builtin libraries.

Anyway, this not my idea of deep knowledge; it's just an uninteresting implementation detail. It could be more interesting if the designer explained why it was done in this more dynamic way, instead of just checking that the object implements Iterable.

kevin cline
  • 33,608
  • 3
  • 71
  • 142
  • 1
    I don't think there is such a thing as "skimming" the C# language specification. – Robert Harvey May 28 '12 at 15:50
  • @RobertHarvey: you can skim through most of the formal language covering things you already know, like operator precedence and declaration syntax, and focus on the unexpected but useful details, like the exact behavior of C# foreach or Java enum constructors. – kevin cline May 28 '12 at 20:34
  • You can buy an annotated version of the standard. It is a bit dated now but the comments are still very interesting for the parts of the language which are covered. – Jørgen Fogh Mar 30 '16 at 10:04