Introducing functional programming into your development environment

Question

This is a long story but I will try to summarize it as best I can. We are a .NET shop that writes software for a variety of different types of funds. For example, we write health claims management software, pension management software, 401(k) software and some other financial-type things.

Now we are entering a new realm: derivatives and quantitative analytics type of work, which seems very exciting. This is something that is coming down in the next 8-12 months I've been told.

Now, I have learned some functional programming on my own, mainly just superficial glances at various languages via this book but never really dug deep into it. Now, since we are a .NET shop I was thinking F# might be a good choice, because we could potentially leverage some .NET libraries and existing knowledge possibly.

My question is, where to start? I am having an exceedingly difficult time trying to figure out if I should go with Haskell, Erlang, Scala, F#, etc. They all seem very interesting and capable and quite frankly this might be a good chance to break out being so Microsoft dependent.

Has anyone had a similar situation? If so, what was your experience making the jump to functional and what did you choose and why? I know this is kind of a big question but I don't know any developers that are using functional methods right now so I have nowhere else to turn other than Googling incessantly and finding flame wars everywhere on the best functional language.

Answer 1

Prototype, prototype, prototype! Take one bit of business functionality that you think requires functional programming and try the various languages out and see if they really do give you the benefits and the interoperability that you're looking for.

Answer 2

The first question you really need to answer truthfully is why you are considering using a functional language. If you can't justify a business reason for the switch then you shouldn't do it. In other words, wanting to introduce a new framework, language or other new technology into your work environment just because you want to learn it or because it seems like the next "cool" thing is definitely a mistake. So you need to first truthfully ask yourself what the motivation is.

If you really feel you need a functional language to solve a particular problem, and assuming most of the mainstream functional languages will meet your requirements, then I would go with the most mature and the one with the largest user community. Erlang is a good choice and meets both of these requirements, however, in a purely ms/.NET environment, I can understand using F#.

Answer 3

I'd strongly agree with F# for a shop with an existing .Net codebase, much as I would strongly agree with Scala for a shop with an existing Java codebase.

Functional programming is a tool like any other. Used well, and integrated with how you're already developing code, it can make you more productive by making it easier to reason about what your code is doing. Switching languages is far from free, however, so your best bet is a solution that lets you use as much of your existing code as possible for as long as possible into the transition. The surest way to fail to introduce a new language into your environment, after all, will be to tell your coworkers that they have to rewrite everything they've got so far in order to see the benefits of a change that, at this point, you're still trying to sell them on.

Answer 4

I would recommend starting functional programming without also learning a new language right away. It just makes it more difficult if you're trying to learn a new paradigm when you're also trying to get to grips with the syntax of a new language.

Of course, languages that have been specifically developed to do functional programming will have some advantages (such as having specific constructs such as comprehensions and making data structures immutable by default), but in general the biggest step to make is change your thinking to a functional style. C# is great for doing that.

Basically, you just stop changing the state of your code. I've done this using Java and it's even easier with C# because you have lambdas. Once you get the hang of that style and a feeling for what it's good for, it will be very easy to pick up a functional language (regardless of whether it's F# you choose or Erlang) and be immensely productive with it.

Answer 5

If all you want is to learn and understand functional programming, then just install IronPython and focus on the functional features in Python. At worst, you will learn a tool that can be integrated with C# to cut down the number of lines of code in an application and help you deliver more bugfree products ahead of schedule.

Have a look at DaBeaz's presentations about generators for an example of how functional approaches in Python can simplify complex things http://www.dabeaz.com/generators/

Other than that, I think that you would be wise to invest some time with Scala. It runs on .NET in beta mode, so you can install it and use it today for learning purposes, and by the autumn, it will be in release mode for .NET. This means that you can write code in Scala that is portable across the JVM and .NET. And, since Scala is based around Actors and message passing, it is very, very easy to build an application made out of several separate programs running on several separate machines. When you add the .NET/JVM portability to the mix, then there is another aspect to consider. You could have one application that leverages both 3rd party Java libraries and 3rd party .NET libraries without messing around with developing protocols to make them communicate. Both processes would be written in Scala, and would use Scala remote messaging (remote actors) to communicate. Check out the Akka, library which seems like it will eventually become part of Scala's standard library judging by what Typesafe.com are doing.

Answer 6

I certainly agree with the main thrust of the accepted answer, and given that you basically are diving into things headlong by giving Erlang a try it sounds like you likely have a pretty good understanding of how you learn and just needed a little nudge in the right direction, so clearly it was a good answer for you... But, I think I would approach the question a little differently, since I see that this answer wouldn't have helped me much at all; I always code to learn! So, here are my thoughts...

(BTW, I tend to go on in detail suited more for, say, chapters in a book, and I'm sure I can't stifle my instincts completely here, but I'm going to try a different tack; I'm going to just summarize my thoughts here, and in case someone would like more detail on anything, or thinks somethings I say is misleading in this format, I will do my best to remember to go through response reminders...)

To try to keep myself on task here's my understanding of the thrust questions asked in the OP; I'll keep sidelines to a minimum as necessary to explain....

First the quick answers:

was I in a similar situation? It was at least similar; I was in a lead position doing many varied projects that were nonetheless related... (CRM/Web/DB/Data Integration/etc.)

how/why did I make the jump to functional?" I saw some LINQ examples and although I had definitely dreamed of some sort of integrated, statically typed, query language [as I had primarily used statically typed languages (primarily C++, and later C#)] throughout my career... But, it was a fairly rapid-fire environment I was in, and although I had often managed to see what was coming in the past, I had never really thought it through so I never foreseen that it could/would allow operations on Plain Old Objects (lol!) just as easily; when I saw it I knew I had to have it... so, that's the why and the start of the how: I focused on learning to understand LINQ.

Four Thoughts ...errr, no that's not right... Forethoughts

When I read Martijn Verburg's answer, the mention of business functionality draws my mind immediately to starting to implement code for whatever I'm working on... Depending on exactly how shallow that initial experimentation was I might answer differently, but assuming it was just dipping your big toe in, I am not sure that I had anything directly for work (or soon-to-be work) that would have provided me the excitement/passion I think you should probably have for the first few problems you start to tackle...

I was grounded almost entirely in statically-typed languages + OOP metaphors and patterns + whatever I ended up wrapping my brain around accidentally while solving my actual problems over the years... Point being that, if you're at all like I was, you likely have a lot of brain dedicated to stuff that isn't going to help you that much with LINQ/FP.

I think of it kinda like pure procedural vs. OO programming: there's a lot of procedural stuff that you will end up using in OOP, but those who come to C++ from C without making it a priority to grok/ken/"get" OO end up being very very bad at C++. I actually have interviewed a hell of a lot (15+) long-time firmware and device driver developers who really thought they knew C++, but who, at most, had a terribly basic/textbook understanding of C++, with practically no understanding of or experience in OOP—because they never actually did OOP... They wrote singleton multi-purpose classes with static members and static functions with some amount of non-static/non-singleton classes that were used as structs.

And FP has some similarly mind-bending (for those not raised on the paradigm) concepts and other things that go along with it, and (although I have found the hybridization of lots of techniques to be ideal for me) I do understand more and more as time goes on exactly how limited my thinking was prior to getting real functional abilities into my toolset; I had managed to implement many things in the past in a way that was a bit more creative that perhaps most OO programmers, but as I use the concepts how stunted my thinking was prior... There are whole classes of problems that can be solved in a few lines that used to take a hell of a lot of contorting to do in C++/C#.

suddenly you find yourself...

In A "Too Long" Post

You are in a maze of twisty "didn't read"s, all alike.

You see a teleconference in the near future, rapidly approaching.
>terse
Uhhuh. Sure. Okay, we'll say "terse" mode is "on". 

You see a teleconference in the near future, rapidly approaching.
>what's that supposed to mean?
Just sayin'. Didn't you have to do something this morning?

You see a teleconference in the near future, rapidly approaching. 
>hey, is that a grue behind you?
What!? WHERE?! [runs away screaming]

>Sorry, I did not understand WHAT WHERE, rephrase?
[continued running and screaming, sarcasm unnoticed]

So... What should I learn, dear PSE, dear PSE?

I personally started in C# with LINQ. It allowed me to introduce a few concepts at a time, and while I was constantly reading about FP and its concepts, and more LINQ, and the relationship between the two, it gave me a way forward while still doing productive work. I added in a few things at a time, the data queries quickly became a useful tool to me without having to understand a ton.

Now, looking back on it, I kind of wish I had been able to do my next project (tackled about a year later) first; I got moderately familiar with F# (which, incidentally, gave me a great head start on learning ML (metalanguage) and it's other derivatives (e.g., OCaml as well.)

Basically, I think a decent answer to 'what' depends on finding a good pairing of some programming problem that interests you, but of course it must pair with a bit of FP that you want to learn... (and you can rinse/repeat/lather after knocking something off the list...) And, of course, you always end up learning a bit more than the main thing you set out to do; at first I took some baby steps, but then I ended up doing bigger things and letting the smaller stuff fall into place while I did those.

First, what floats your boat? Especially at first it's best to have something fun and exciting (to you) and that will keep your interest enough to make it worthwhile for you. So, IOW problems to work on, and techniques that address those problems... LINQ and inline data querying for me at first. Recursion was another one for me, including tail recursion, I dig the GodelEscherBach-ness of it; and I had been reading about tail recursion. Around this time, the things I was working on were put on hold, and I ended up having a large block of time, so I was able to keep at it for a good long while. It was easier with less interruptions, but because I chose things that I thought were fun, it wasn't all that hard even with the interruption of work. :)

And although I didn't come up with anything cool I can tell you about like self aware Crab Program, I managed to get pretty good at it.

And... With what shall I learn it, dear PSE, dear PSE, with what?

For this, I used various algorithms I just happened to be interested in anyway, plus various things I wondered if you could do in F#, and when I would run out of ideas I'd tackle things like 99 bottles of beer and the Project Euler problems...

I'm sure you can find lots of things that interest you that FP is relevant to; it offers everything from enhancements to OOP that help you state things a bit more succinctly all the way to twisting the way you would have thought about it before into a shape you don't recognize and likely didn't even have the mental model to express before.

But... There's a hole in my model, dear PSE a hole...

And this is why it's important that, especially at the beginning, but really throughout your time learning (and isn't it always true when you're learning something?) you at least take a bit of time between problems to read things unrelated but still FP and time to read source code written by experts, preferably solving the same or similar problems; as well as their explanations of things...

And all the while, you have to be straining your brain to understand it all, not from your old perspective but from within FP itself... At some point for me, it clicked, and ... the thing I can best relate it to is becoming more-or-less fluent in Dutch; at some point I managed to put myself enough in the mindset (I did it via immersion, which is basically what I've described here; immersing yourself in FP rather than trying to learn it solely through "book larnin'"...

And, eventually I did; I managed to internalize everything and twist my brain around until out started popping FP/LINQ without me even having to make an effort to translate it back into OOP. (Yes I did that; I had to have something to hang my skirt on. Hat. Whatever.)

Final thoughts...

My, my, it seems you're losing your ability to think of your signature clever
section titles. What a shame.

The teleconference is still rapidly approaching. It appears much larger now.
>yes, yes, well.. terse and all. I do see you managed to lose that grue.
 close call, that... well, yes, I am being ter... OH NO IT'S BACK!
AAAAAAAHHHHHH!! [runs away screaming, once again]

The teleconference is still rapidly approaching.
It somehow resembles Sir Lancyjohn Cleezewiz.
It is almost upon you.
>hey, hurry back! I have an experiment to try with you! (and a po-ta-to)

You can find a wealth of info online about FP, naturally. Main thing is to understand basic concepts, and then learn to apply them. For instance, learn about immutability, and why it's important/useful for FP. I highly recommend learning a little theory to go along with everything, like how pure FP can be much more amenable to formal proof. This was a driving force behind F#'s progenitor, ML. YMMV of course you may be someone bored to tears in which case, lots of examples, lots of trial and error to see exactly why the techniques used are what they are will help you have that 'Aha!' lightbulb moment.

So, I'll leave it at that for now. Hope this is of some use to someone. I wanted to get a bit more into specific things to learn but I'm out of time for the moment. I hopefully will find some time to get back to it soonish although it's looking to be a long week for me, so probably will be at least the weekend.

<3 "Introducing GRUEBOL, a stopgap; just until SnozzML is finished. It should be soon; I got a huge committee helping me this time." --Grace Hopperwit Egghead, Famous Last Words, XX97 G.U.E. <3