For a startup, what are the pros and cons of building a software product predicated on "Throw away code" vs "an extendible codebase"?
In reality, if you phrase the question as "either one or the other", it might reduce your options significantly. I feel like you kind of answered already the pros and cons in the question, so I'll try to exercise the real question:
I don't know of a single startup success story where people take their time to build a good system. Time to market is king, and like you said, going against it can increase significantly the risk of going out of business.
"Doing right" has serious problems in the first moments of a startup. One, it requires experienced engineers (which requires money); two, it requires unbounded time (depends on the complexity of the product and the experience of the engineers with this kind of system). Finally, we all know you don't need good design to be in business, you need something that runs, and whose problems are hidden from your customers.
Pretty much any startup experience I know of (and probably every single famous one) took big advantage of this fact: if part of your product is poor quality, but that part is invisible to the customer (or their potential desire is not significantly affected by it), don't waste time on improving it, just ship it.
But that is "time 0" in the time scale. In time t > 0
you begin to learn what most startups learn the hard way: the poor quality of those invisible parts is constraining your current opportunities. That is, the team is requiring much more time to implement new features, these are being shipped half baked, bugs are bugging the customers and making them angry, etc.
tangent: at this point, hopefully the manager knows the capital sin of software engineering and would not fall into the myth of the man-month.
The difference that makes the difference here is to know at which time t
you should pause, hold the horses, and put some effort into cleaning the house before you sink (and how much time dedicated to and how much cleaning should be done).
In other words, this is not solved by selecting either "Throw away code" vs "an extendible codebase" once at the beginning, closing your eyes and running in the chosen direction. You can solve it by putting the "time" in the equation and acknowledging there is a question you should make from time to time, and from time to time, it might require opposing answers.
In summary, instead of having a static "time vs. cost vs. quality" pick-two-triangle, put these in a time scale:

(source: metareload.com)
Now, take into consideration that, at any t
, cost
and time
for developing a feature depends on the resulting quality
of the product at t-1
. In other words, how much quality of design you chose to invest right now is telling how cheap or onerous the future developments will be (so, quality
is the variable that tells you the cost of your future).
If you are a manager, this point of view can give you more clues about how to manage the tension between design quality and time to market. And that is something to be managed constantly.
Finally, experienced engineers are optimized for design. And if you have very good ones, you can push time constraints on them and they will make a significant effort into selecting critical parts of the system to dedicate to design and testing, and will let less-critical parts to be of lesser quality -- they know these have messes that are more easily manageable and the trade-off for quick delivery pays off. You can also count on them to put some breaks at the incoming demands and tell you that this is the t
moment they should stop and work on the code base to make room for new maneuvers. In other words, what you're doing is "amortizing" the total cost of good design (and risks) by spreading it through prioritized parts of the system and through time.