52

I stumbled upon this quote from Bill Gates: "You should definitely send me a resume if you can read the whole thing." He was talking about The Art of Programming books. So I was pretty curious and want to read it all. But honestly, I don't understand it.

I'm really not that intellectual. So this should be the reason why I can't understand it, but I am eager to learn. I'm currently reading Volume 1 about fundamental algorithms. Are there any books out there that are friendly for novices/slow people like me, which would help to build up my knowledge so that I can read Knuth's book with ease in the future?

Rho
  • 673
  • 1
  • 6
  • 6
  • Humm let's say I would understand it (and you too I guess), but it would take a lot of time and there are easier books nowadays.. – Nils Feb 08 '11 at 13:32
  • 22
    They may be hard to read, and there may be easier books, but you should read them anyway. I have only gotten through the first book so far, and I don't understand everything in that! but it's well worth it. Computer Science is *hard*. The sooner you realize that you aren't smart enough to understand it, the sooner you can start learning how to learn it. – Michael K Feb 08 '11 at 14:37
  • 2
    They are math inclined for math-y programmers. CS has evolved in latest years, and there are ton of beneficial areas that are not math-y. Which direction do you want to develop further? If it's algorithms and such, they are probably good read, if it's other areas, they will expand your horizon, but just as much as reading a book on some biology topic. So, depending on your area, they can be anywhere, from mission critical to almost useless. – Coder Jun 25 '12 at 17:51
  • 1
    Since when do programmers consider Bill Gates authoritative? – Giorgio Aug 29 '12 at 05:44
  • 2
    [Coursera](https://www.coursera.org/) has 6 free courses on Algorithms (Algorithms Parts 1 and 2, Algorithm Design and Analysis Parts 1 and 2, and Analytic Combinatorics Parts 1 and 2). – Anthony Aug 29 '12 at 08:57

8 Answers8

56

Be sure to read all of Gates's quote including this:

"It took incredible discipline, and several months, for me to read it. I studied 20 pages, put it away for a week, and came back for another 20 pages. If somebody is so brash that they think they know everything, Knuth will help them understand that the world is deep and complicated."

They are not easy books and they're not intended to be. Remember that one of Knuth's goals was to bring mathematical rigor to computer science. That's great if you want to prove something about an algorithm, but not-so-great if you just want to know how it works.

Michael Dorfman has a few good tips for reading the books in his answer to the (now deleted) question on Stackoverflow about What Can I Get From Reading the Lot?. If you don't have 10k rep, you can still view the question and his answer on the wayback machine.

What will you get from reading the lot? An excellent foundation in Computer Science. You'll understand how computers operate, from logic gates on up through compilers. You'll think about problems you never really knew were problems (i.e., what is the fastest way to multiply?) and see algorithmic connections between things that you never thought were related (riverbeds, RNA and nested parentheses, for example.)

I completely disagree with the folks who say "build software instead of reading about building software"-- there is a difference between the disciplines of Software Engineering and Computer Science. TAOCP is about the latter.

If you haven't yet begun, I have a few recommendations.

First, you may want to begin with Volume 4. It's some exciting material, very up-to-date, and Knuth's sense of humor shines through. Plus, there are videos available (on the Stanford SPCD site, or the Stanford iTunes) where Knuth discusses various sections. These videos are highly recommended. Fascicles 0, 1, 2, 3 and 4 of Volume 4 are available as separate paperbacks. Together, the published V4 material is larger than any of the first 3 volumes, but sliced up into bite-sized treats. (I wonder if Volumes 1-3 would seem less terrifying to people if each volume had been published in single-chapter paperback form...)

Depending on your math background, I'd recommend you skim Chapter 1 the first time through, and return to it as needed. In fact, you'll probably want to read each section (at least) twice-- quickly the first time, just to get the intuition and the gist of the arguments, and then slowly, carefully, understanding each step.

Be sure to read Volume 1, Fascicle 1 on MMIX instead of the old sections on MIX. MMIX is better in many ways, and you'd be better off converting the MIX in the text to MMIX as you go along than trying to straddle both worlds.

A general rule: don't skip the exercises. There's a lot of good material in the questions (and answers). Do as many exercises as you can; but read them all (and read the answers, once you've taken a shot at the problem, or decided to give it a pass).

Finally, if you really catch the bug: read the index. Lots of great jokes hidden away there.

Naturally, StackOverflow would be a good place to post specific questions on the text, if they should happen to arise....

For other resources, I've found that browsing the syllabuses of well-regarded computer science schools is useful. For example, textbooks for beginning algorithm classes:

Community
  • 1
Corbin March
  • 8,104
  • 3
  • 37
  • 39
  • The link to SO was broken, but I found a post that I imagine is similar: http://stackoverflow.com/questions/1022167/advice-on-how-to-approach-the-art-of-computer-programming – asjohnson Jun 25 '12 at 15:31
  • +1 for the original post on the Way Back Machine and the book list from those universities – Anthony Aug 29 '12 at 09:04
  • +1 for recommending to start with Volume 4, Fascicles 0, 1, 2, 3 and 4, and also for reading about MMIX instead of MIX. As a result, I am going to start with Volumn 1, Fascicle 1, because it covers MMIX. – Shaun Luttin Jan 29 '14 at 18:33
  • As a Java/c programmer, I feel very easy to understand & quite practice. – Eric Feb 03 '15 at 08:56
39

Even I think Knuth's book is a bit advanced and difficult to understand. Those books are definitely for research level algorithmists IMHO.

So are there any books out there that are friendly for novices/slow people like me?

Introduction to Algorithms by CLRS is much simpler.

EDIT :

Still if you want to read Knuth's book you should first go through Concrete Mathematics. Knuth wants his students to be aware of the basic mathematical portion of algorithms analysis.

anujm
  • 103
  • 1
Prasoon Saurav
  • 1,018
  • 11
  • 14
  • 5
    Algorithm design manual is even easier / more accessible - it comes close to being fun to read – Martin Beckett Feb 08 '11 at 15:21
  • Intro to algorithms is a great book. We used this for our first CSE algorithms course and I loved it. Though I must say I spent many iterations going through the same examples to really say I understand them. – Chris Feb 08 '11 at 18:56
  • 4
    @SidCool: There's a [CLRS 3rd edition](http://www.amazon.com/Introduction-Algorithms-Third-Thomas-Cormen/dp/0262033844/ref=dp_ob_title_bk) with a dozen new sections. – Bill the Lizard Feb 09 '11 at 02:11
  • 10
    Knuth's books are War & Peace of programming. They are good, but are mostly used to make bookshelves look impressive. – Gaurav Feb 09 '11 at 17:32
  • 6
    "Even I think Knuth's book is a bit advanced"? modesty eh :) – occulus Jun 25 '12 at 09:10
  • I don't agree with CLRS. I find it still too hard to understand for autodidacts and very advance esp. for people like me who likes to read cover to cover –  Apr 06 '15 at 11:13
29

Knuth is the most revered, quoted, talked about, and highly respected computer science author in history. His books adorn the bookshelves of all serious software developers, and are referred to with the same level of respect people give the Bible and Art of War.

I've even heard that some people have actually read portions of Knuth's books.

Most people just intend to.

Personally, I'm saving them for my retirement

Steven A. Lowe
  • 33,808
  • 2
  • 84
  • 151
  • 22
    That's why well worn used copies are worth more than new ones! – Martin Beckett Feb 08 '11 at 15:20
  • 13
    If you understand The Art of War, you'll realize you only need to make people think you understand Knuth when you don't, and conversely that you don't understand Knuth when you do. If not, you don't. And if you understand book 5 of the Book of Five Rings, you'll not even need to speak of Knuth. And if you've read ESR's The Art of Unix Programming and understand the koans, you won't even need Knuth, because you'll have transcendented the barrier of complexity. – Christopher Mahan Feb 14 '11 at 16:51
20

Knuth's books changed the field of algorithms forever. He himself has said that '2 pages in my book is somebody's entire career work' and that his books were difficult to read. The book contains condensed material from years of work in Computer Science.

You should not feel bad if you cannot understand it.

As Prasoon said, CLRS is a simpler book to read.

You also have Algorithms by Rajasekaran, Sahni et al which is easy to understand.

Community
  • 1
Arjun J Rao
  • 351
  • 1
  • 5
  • thats amazing to hear.. i thought im the only one having trouble reading this book.. thank you very much – Rho Feb 08 '11 at 13:37
  • 7
    @Raymond Ho: I don't think anyone really likes reading Knuth's books. I know at least one person who has them on the bookshelf just for the sake of making the shelf look impressive. – FrustratedWithFormsDesigner Feb 08 '11 at 14:53
12

When I first graduated I picked up the first three volumes of TAOCP as a graduation gift to myself and attempted to read them straight through. Never managed it. These days I've made it through maybe 1/3rd of the first three volumes (thought in no particular order). The material is definitely dense, but there are three tips I've learned that helped a lot.

First, don't try to read from cover to cover. TAOCP is really as much of a reference work as anything, and I've found it's best to read a section when it's relevant to a problem you're trying to solve. Like many things in the world, understanding the solutions is much easier once you've encountered the problems they are trying to solve.

Next, that flowchart in the front of the book, it's not just a bit of humor but actually quite a useful tip. Read the sections you're working on iteratively, starting first with just the overarching concepts and then gently getting deeper into the math.

Finally, keep some good old fashioned paper and a pencil handy to work through the algorithms as they are described, and work through a few of the easy problems. It goes a long way to help re-enforce what you're reading.

Cercerilla
  • 1,969
  • 11
  • 14
10

Don't worry, most people don't understand The Art of Computer Programming (TAOCP). So don't think yourself as slow or a novice for not understanding-- you're just like the other 99.99% of us who don't get it.

You're pretty ambitious if you want to get to the level where you can read The TAOCP with ease. I myself only have paged through the books before putting them away. There's probably only a handful of people on this planet that understand TAOCP.

Check out the post: Books Programmers Don't Really Read by Bill the Lizard.

There are plenty of other books listed there that are quite readable, understandable, and you can benefit from right away.

I personally like:

Thomas Owens
  • 79,623
  • 18
  • 192
  • 283
spong
  • 9,361
  • 6
  • 44
  • 58
8

I stumbled this quote from Bill Gates: "You should definitely send me a resume if you can read the whole thing." He was talking about The Art of Programming books.. So I was pretty curious and want to read it all but honestly, I don't understand it at all.. I'm really not that highly intellectual being.. So this should be the reason why I can't understand it, but I am eager to learn.. I'm currently reading volume 1 about fundamental algo.. So is there any books out there that are friendly to novice/slow people like me? So I can build up myself and hopefully in the future I can read Knuth's book at ease..

if you define yourself as not a highly intellectual being then you are setting yourself to low expectations. You got to break that mentality if you want to do something worthwhile the trouble. There should be no doubt in your mind that you can achieve something. Also, achieving it doesn't mean that you will achieve it easily.

Things worth pursuing are the ones that are difficult... and that's not a cliche. In software, engineering, on in life in general, if you want to achieve something, you got to go for the things that are hard, the things people avoid, and not settle for the lowest common denominators of things.

First, it is not clear what your CS background is. Knuth's book require a degree of maturity. Few people with a CS degree can go through it with ease. I would not expect a CS student that has just finished his first course in algorithms to actually be able to go through even a single of Knuth's book. The maturity needed to get it is just not there, and that has nothing to do with the student's mental capacity.

You need to have your basic of algorithms down cold and clear, and you need to have a fair amount of programming (work and/or scholastic) under your belt - I'd say, 40 credits on programming at least. You also need to have your CS mathematics on firm ground.

You can't go far ahead without having a good grasp of discrete mathematics (and possibly theory of computation.)

It's not that you will need that knowledge to work on Knuth's problems, but you need a maturity to be able to go through that type of material.

First pick one book and one book only (CLRS' book as previously suggested), and work it from start to finish. When possible do programs implementing the algorithms. Don't use Java or C#, not even C++. Go to bare bones C and get the feeling of building things from bare-bone metal scraps.

Also get Knuth's book on "Concrete Mathematics" if you have not done a course in discrete mathematics and computation theory. Would be good for you to go through that book as well.

Then tackle Knuth's encyclopedia, one tome, one chapter at a time. Don't go to another chapter without having a good grasp of the first one.

I would suggest you go through volume I (fundamental algorithms) first, then volume III (searching and sorting). Those should be your immediate goals. Then, later (much later), tackle volume IV (Combinatorial Algorithms) and then Volume II (Semi-Numerical Algorithms.)

Don't feel bad if you don't get it at first. I've been trying to get through volume I and III for years (10 years now).

And you should not put so much weight on it either. Don't do it to prove something to somebody or yourself. Do it because you are intellectually interested in doing so. You can become proficient in algorithms by simply using the CLRS' book (or any of the fine grad-level books out there.)

Be pragmatic and give yourself a break. Treat going through Knuth's book as a long-term personal ambition, not as an immediate proof that you are CS material ;)

There are other more important things (career-wise) to kill yourself for ;)

luis.espinal
  • 2,560
  • 1
  • 20
  • 17
  • 2
    Note: You refer to Volume III twice and never Volume II, naming Volume III two different names. – alternative Feb 13 '11 at 12:14
  • Thank you for letting me know (+1). The order I meant was as follows: first volume I (fundamental algorithms), then volume III (searching and sorting), then volume IV (combinatorial algorithms) and then volume II (semi-numerical algorithms.) – luis.espinal Feb 14 '11 at 12:25
  • 6
    +1 _It is not because things are difficult that we do not dare; it is because we do not dare that they are difficult._ -- Seneca – mouviciel Jun 25 '12 at 07:13
4

Before even starting on Knuth I had to pace my way through four different books. The first two are the Sedgewick books on Algorithms. These overview most algorithms and data structures in an actual implemented form, so you can see what they are and how they work. These books come in different language versions - I read the ones in C, but iirc they were originally written in Pascal, and there are C++ and Java versions out there.

After that I worked through a good portion of the Cormen book on Algorithms and used Sedgewick and Flajolet's Introduction to the Analysis of Algorithms as a complimentary text, as it is more in the vein of Knuth's mathematical rigor than the Cormen book. I still have yet to finish either of these, mostly cherry-picking the parts that I feel I need.

After reading those, and getting a degree in Mathematics I can read through some of TAOCP, but it is a difficult read. That isn't to say it is not useful. TAOCP are some of the greatest algorithm reference manuals around, but thinking you can use them to "fully" understand anything is somewhat problematic.

Justin Hamilton
  • 137
  • 5