Background Story Blog

Aug 30, 2014 at 7:24 PM
Hi Steven,
Been rereading your blogs and was curious if you'd be able to do a blog on this background story:
<BackgroundStory>This last one is a very interesting one. Years ago I worked on an application that used a database table as queue for commands that would be executed in the future. We wrote business processes (commands by themselves) that sometimes queued dozens of other (sub) commands, which could be processed in parallel by different processes (multiple Windows services on different machines). These commands did things like sending mail or heavy stuff such as payroll calculations, generating PDF documents (that would be merged by another command, and sending those merged documents to a printer by yet another command). The queue was transactional, which allowed us to -in a sense- send mails and upload files to FTP in a transactional manner. However, We didn't use dependency injection back then, which made everything so much harder (if only we knew).</BackgroundStory>
How would you have solved it with simple injector. Anything you could share from that experience? It sounds pretty amazing and advanced and would love to see how you approached the solution and what that pseudo code would have looked like. Especially with SimpleInjector. :)
Aug 31, 2014 at 9:40 AM
As a matter of fact, I'm thinking about writing a blog post about publishing and handling events. I did some consulting lately with two teams where I advised how to implement this technically. The thing is, that those queued commands in that background story were in most cases actually business events that described things that happened in the system where the business was interested in, but we didn't know about the concept of business events back then. Technically the difference is quite slim, but functionally there's obviously quite some difference between a command and an event.

But to be honest, there's no real challenge in implementing this with Simple Injector. From the perspective of a DI library, the solution will be almost identical to registering command handlers and query handlers (with the difference that one event could be processed by multiple event handlers). And even the queueing of those events is just a matter of serializing such event to XML or JSON and inserting them in the queue table (or sending them to some message bus).

In the case of the background story however, the solution was a bit more tricky, because we had performance and reliability requirements which forced us to have multiple services reading from the queue simultaneously and handling different events of the same type in parallel. We had to ensure that it was impossible for multiple services to accidentally process the same event. We ended up using SQL's READPAST query hint to skip command records that were locked by a different transaction. This was tricky because adding and changing indexes on the queue table influenced the behavior. So this solution is not one I suggest unless you have the same requirements. For most systems it will probably be enough to have one single-threaded Windows service that processes the events from the queue one by one.

But don't expect any blog post about this particular background story :-)
Marked as answer by dot_NET_Junkie on 9/28/2014 at 4:18 AM