Ways of integrating legacy applications

Question

My company is one which has grown up rapidly over recent years in multiple territories. We are not a software company per se, but a technical one who have developed various bespoke applications inhouse. Some of these are now sold as SAAS.

I am personally responsible for 3 of these applications: all internal web applications, one written in MySQL/Python/Django, one written in SQL Server/Python/Django and one in MySQL/PHP/Symfony. My developer resource includes expertise in all of these, as well as some Java and Adobe Flex. None of us have any significant experience in Microsoft technologies beyond SQL Server.

However, all the other departments in my company that have developers all work using the Microsoft stack - all our other webapps are C#/asp.NET etc etc etc.

I have been asked to evaluate integration of our depts products with others in the company at a data-sharing and presentational level.

I see 3 possible routes:

1. We bite the bullet and move to the Microsoft development stack, and this becomes a company-wide policy. Pros: standardisation, ability to pool resource, integration can be done at any level. Cons: massive licensing costs we can't really afford right now, complete retraining of all developers required, having to use IIS instead of nice Ubuntu/Apache servers, and general philosophical opposition (we're mostly open-source evangelists in my dept).

2. Middleware. We've been looking at Adobe Flex in order to handle data subscriptions between services. Pros: relatively cheap solution, existing in-house experience, can use whatever underlying tech we choose. Cons: it took ages to get a quote for Adobe data services, it isn't clear to me how futureproof this route is - are we potentially buying into a tech that's going extinct?

3. DIY. We stick with open-source all the way, and custom-write web service APIs between all the apps. Pros: cheap on licensing, open-source. Cons: additional time required, more testing, core issue of techology proliferation not tackled

I'd be interested to know what other approaches there might be here. Are there other Middleware ways of going about this? Am I hopelessly out of date with my views? What's a safe bet for the future?

Thanks

Answer 1

There are probably those in the company that see too many disparate systems that are duplicating effort and making it harder to include data in the same report. The head of IT needs to focus on design and not be so myopic to think everyone using the same language is the panacea.

Consolidating to the MS stack may make it easier to share developers and knowledge. SQL Server is the most cost prohibitive part of doing this. There are other licensing issues for Windows Server and Visual Studio. If you can find an ROI for business operations and/or the development costs, you can justify the purchase.

Data integration is not going to be a pure technical problem just because you're on different stacks. You could do this through web services.

With presentation, you need a consistent ui design. A web app user could be moved to a different technology when they go from one module/department app to another without even knowing it. You need to decide what makes sense to consolidate. Are there separate databases holding employee data that are manually updated to match? HR and Sales have this in common. A good test is to see how difficult it is to have an employee leave, get hired, or have a name change due to marriage.

You should be providing some input from your department, but someone higher up in the company needs to be seeing the bigger picture.

Answer 2

You have at least two different issues here that you are conflating into one.

1. Toolset / stack
2. Communication

For the first, you may or may not want to migrate towards fewer technologies. Having one of everything under the sun can make maintenance difficult, and does limit your ability to share certain parts of your applications.

Communication between modules/applications/systems is the second part. You mention data subscription, do you really have real-time push-type requirements. If you do, then you should look at message-bus type technologies like RabbitMQ, ActiveMQhere and others (there are many questions covering these on this and other sites).

If on the other hand, you merely want to centralize some data, simple web services (be they SOAP or REST based) may well be your best bet. The .NET teams can easily create them, and all your other technologies can merely call them, and vice versa. This may well be your best short-term bet, allowing the various siloed apps to communicate while you work out a bigger plan.

Answer 3

Have you read anything about Enterprise Integration Patterns? There is an entire book on this subject and a variety of frameworks out there to help.

http://www.eaipatterns.com/

Answer 4

I think one question is how much resource/budget your department gets. You alluded to budget issues. Where I work now (until Friday) the department gets no resource, and they use Microsoft technologies. This leads to really poor tooling because no one wants to pay licensing costs. It leads to all sorts of frustrating things like remote desktopping to a slow overloaded computer in order to use SQL Server Management Studio (because they only wanted to pay for one copy), etc... Or not having a Dev/Test environment at all due to not wanting to spend any money on licensing. Whereas with PHP/Python/Java just install the server and go. Another issue is libraries. Python/Ruby/PHP have way more web frameworks available and free. Microsoft has only just released their MVC library. Even things like Java had MVC frameworks for a much longer time, and things like Hibernate too were ahead. Additionally many of the greatest .NET libraries are not free at first. I know one prior employer was paying a fortune for infragistics web controls. And whenever I wanted to write excel without Office Installed, the most useable libraries were not free (although now you can just use NPOI). But even beyond money many Microsoft departments are not willing to accept third party libraries which also limits you. Whereas most PHP/Java/Python/Ruby/Perl shops I have seen are using third party libraries up the wazoo...

But that being said Visual Studio 2010 looks like it has a lot of features and Microsoft has been catching up on some libraries (Entity Framework and LINQ from an ORM perspective and more, MS MVC for web framework, F# for functional programming, etc..). Unfortunately there are still many places using .NET 2003 or .NET 2005 which do not have most of these features.... Then there are other shops who upgrade to the latest/greatest ASAP (usually with a lot of PRO MS corporate support). So it's hard to say. But definitely LINQ and Entity Framework (and even nHibernate) came out way after most other languages had ORMS. MS MVC came out way after DJango, Struts, Catalyst, etc.... So they will be behind in many of the latest/greatest discoveries.

The problem is not so much developer training. Most developers can pick up C# in a blink if they have been exposed to another C like object oriented language. I knocked out my first .NET application in less than a week at my first Job (it was VB.NET but the time was spent not in learning VB.NET but rather the .NET framework, page lifecycle of asp.net web forms, etc..). The problem is the "fun" factor. Many developers enjoy Python/PHP/Ruby a lot more than C# or Java (well more Python/Ruby than PHP). Some of them may just leave. Also the additional frameworks/libraries of the other language make you more productive than straight C# beyond just the extra productivity of a scripting language.