13

The place I'm working at is trying to establish some ground rules, and the debate we're having now is local libraries vs web services for code reuse. Web services seem to be the popular pick in most companies, and that's what most of the developers here are leaning toward.

I just can't see how you can effectively use web services for any serious work. How can I safely execute multiple service calls if I can't use a transaction?

Let's say I have a cron job that grabs customers from our database who meet a certain condition that they need to be notified of. They are sent a fax, an email, and a ticket is created to track the issue internally. That is 3 different service calls that would happen for each customer in a for loop.

If an error occurs anywhere in there, it's possible that, for example, a fax and email is sent to the customer, but a ticket is not created. Or worse, this cron job could contain a bug on that causes it to fail at the same point every time, and it repeatedly emails the same customer. If the libraries were all local, everything could just be wrapped in a transaction, and none of that would happen. But we're using web services in this example.

Note that the email and fax methods actually insert the data into database-backed queue tables, which in turn are handled by a separate cron job process. So the calls to the "send email" and "send fax" service methods could be aborted side-effect free if needed.

An option is to put this entire chunk of code in the web service itself, so the web service itself would call the email, fax, and ticket creating methods in a transaction. But then we're creating a web service method just for the use of a transaction; there is no valid reason we would ever actually need to call this method from anywhere except this one cron script.

How would you generally handle this method?

ryeguy
  • 267
  • 1
  • 7
  • In order to provide a full answer, I'd need more information. There are a lot of things to consider when moving to a service-based architecture. For example, do you have a heterogenous (different languages and platforms) or homogenous (single language and platform) environment. Do you have a service bus system in place? If not do you plan to implement one? How do you plan to access your services (single location Intranet, multi-location wan, distributed client hitting a public service api). There are a number of variables that impact an "optimal" solution. – Michael Brown Feb 14 '12 at 14:46
  • @MikeBrown: The services would all be written in one language, but consumed by multiple platforms. No idea about implementing an EBS, we have not even begun any kind of services work, so anything is possible. The services would be mostly internally consumed over our local network, but there would be some that would need to be public facing for mobile applications. – ryeguy Feb 14 '12 at 14:52
  • "If the libraries were all local, everything could just be wrapped in a transaction, and none of that would happen". False. The error could occur **after** the email was sent but before the final database update. A transaction doesn't retroactively prevent the email or the fax. – S.Lott Feb 14 '12 at 16:14
  • @S.Lott: It would, because the email and fax service calls are actually just inserting them into a queue, which get delivered by a different process. If the transaction I described above happened, the queue insertion would be aborted. – ryeguy Feb 14 '12 at 16:29
  • 1
    @ryeguy: Please **update** the question. Please don't add comments to the question. This is an important part of your architecture. Please disclose it in the question. – S.Lott Feb 14 '12 at 16:33
  • @S.Lott: It's in there, and has been in there since the question was posted in its original form... – ryeguy Feb 14 '12 at 16:45
  • @S.Lott: I'm not trying to be an ass about it, what did my comment provide you that is not said in the 5th paragraph? – ryeguy Feb 14 '12 at 17:02
  • @S.Lott: I understand, sorry about that. I'll add further clarification in the question. Thanks for your help. – ryeguy Feb 14 '12 at 17:19
  • "database-backed queue tables"? Not an actual queue at all? Just tables with queue-like use cases? – S.Lott Feb 14 '12 at 18:07
  • @S.Lott: Yes, I suppose you could say that. – ryeguy Feb 14 '12 at 18:34
  • @ryeguy: It seemed like a "yes/no" question. I'll try again, since it's not clear to me. Is it a queue (MQ-series, MS MQ, JMS, or other actual Queue)? Is it a database table that has queue-like semantics imposed by application programs? Also. If this is a database table, is this all in one database with a single (simple) transaction? Or is this in multiple databases with more complex two-phase commit among the databases? Please help us by being **specific**. Phrases like "I suppose you could say that" are non-committal and vague. – S.Lott Feb 14 '12 at 19:42
  • @S.Lott: Sorry, yes, it is simply a database table with queue-like semantics. Enqueuing an item is inserting a row; popping an item is deleting a row. The fax and email queue tables are in different databases. – ryeguy Feb 14 '12 at 20:30
  • @ryeguy: So you have a distributed two-phase commit among these separate databases? Is that correct? Why would implementing web services change this architecture? – S.Lott Feb 14 '12 at 20:53
  • @S.Lott: Yes, it's two transactions right now, and they're both local library calls. It works now because I can begin all transactions at once, and end all transactions at once. At least with my naive understanding of web services, a transaction begins and ends with the HTTP request. If I want to call my email service, then call foo(), and then call my fax service, I'd like to roll back the email if foo() unexpectedly errors out. – ryeguy Feb 14 '12 at 21:50
  • Two transactions is not necessarily two-phase commit. You have two separate transactions in two separate databases A and B? And your application makes sure that A commits before B? And if A commits even if B eventually fails, then, what? How does this "work"? Can you be **precise** in your question? Please don't add a lot of comments. Please **update** the question to clarify what you actually have working. – S.Lott Feb 14 '12 at 23:01

5 Answers5

5

What you are describing is in fact a distributed transaction implementing two-phase commit. Some enterprise messaging platforms include transaction managers to support that kind of thing, but concrete products are platform / language dependent. I don't have concrete experience with such tools, but hope these pointers help.

Péter Török
  • 46,427
  • 16
  • 160
  • 185
3

It's interesting that as I'm participating in this particular Q&A, there is a similar thread about services supporting multiple platforms on the DDD/CQRS mailing list that I'm participating in. I can re-iterate some of my advice here.

One option for supporting transactions in a heterogeneous environment is using a transport mechanism that supports transactions and is supported on all platforms from which it will be used. The Advanced Message Queue Protocol (AMQP) does support transactions and there is a native API for almost every language that is commonly used today. RabbitMQ is a server that implements AMQP and has been vetted in the industry as a robust solution.

Leveraging a RabbitMQ based system puts you on a path to having a full ESB should you need to grow into it. You publish messages to a channel and subscribe to a queue. Where that gets really powerful is that between the channel and queue, you can perform a lot of interesting things. A channel can feed multiple queues (pub/sub) a queue can be fed by multiple channels, you can route messages to different queues based on content, etc. etc.

I was just reading about alternatives to transactions (which come with overhead and turn an async operation into a blocking op). RabbitMQ supports what's called a publisher confirm. Basically it allows you to register a callback for a published method to handle a failed transaction. In your case it could undo the email/fax requests and delete the ticket.

Of course the rabbit hole (pardon the pun) goes even deeper from there. You can use Rabbit to do complex orchestrations with both internal and external web services.

For your public facing webservices, it becomes dead simple. Your service (be it SOAP, REST, or JSON) just publishes a message to the appropriate service queue and let your internal system handle it from there.

There's also functionality for creating a request/response message for those scenarios where you expect information back quickly.

Michael Brown
  • 21,684
  • 3
  • 46
  • 83
2

The keywords you are looking for are 'web service choreography'.

Check out Wikipedia's article about it.

Jalayn
  • 9,789
  • 4
  • 39
  • 58
Dibbeke
  • 2,514
  • 1
  • 16
  • 13
1

The way I handled this in a services app I wrote was to create a wrapper to handle the necessary transactions. In my case, the user request, made by web site, desktop app or Windows service, had to query a web service and, based on the result and the user's options, it had to update a local DB and, optionally, a remote one via a web service. Then it had to generate a report to be returned immediately, emailed and/or faxed. I had control over the local DB, email and report generation but none on the web services or fax server.

Creating a wrapper allowed better transaction control and error handling. It also allowed better security by controlling access to internal network services from outside sources. In general, I see the need for transactions and management of the service as a valid reason to create an appropriate wrapper for a single solution as long as code is reused properly (no cut-n-paste coding).

jfrankcarr
  • 5,082
  • 2
  • 19
  • 25
1

I just can't see how you can effectively use web services for any serious work. How can I safely execute multiple service calls if I can't use a transaction?

You can't.

The question you should be asking is: how do I implement transactions with web service framework X? Right now, you're just assuming that it's impossible.

Mike Baranczak
  • 2,614
  • 16
  • 16