Wednesday, July 15, 2015

Beginning with

A few months ago I saw mention of on some blog post. It talked about "actor-based systems" and Erlang, so I kinda ignored it. Until a colleague started talking about it. I read up on it and let the information simmer for a bit.

Some background: at work I'm responsible for a couple of applications that provide technical provisioning capabilities to multiple order management systems. We started out with one system, but eventually it grew into three systems and a few tools on the side. Since there is some overlap in technical responsibilities I've been thinking about merging them into one.
I'd also like to take this opportunity to make architectural changes to the system. Right now we work with 'requests'. A request fails, we send an error back. I'd like to work with 'orders'. We accept an order and handle it. If there is a technical problem (database down, network failure) we'll keep trying until the order is handled. This would also mean we'd handle these orders asynchronously and we'd need queuing capabilities. Queuing is also useful for the many data migration scenarios we run. Giving customers new services, or changing services on customers. These migrations always cause a lot of pain and I'd like to simply queue them for all customers and have the system work it's way through them. Lastly I'd like throttling capabilities. Many of the back-end systems my systems use are slow and don't handle many requests at once well, so I'd like to throttle these in case of a burst of orders. And last, we have simple load-balanced instances. So scaling up means installing a new instance and adding it to the load-balancer. Even if it's only a single order type that causes the load. I'd like to have better scaling capabilities.

With these things in  mind, my first idea was to use a system based around a message queue server and Windows Workflow. Windows Workflow has nice persistence and waiting capabilities. So I can simply save an order, restart the system and resume processing, or in case of an error, unload the order and try again at a later time. I can also push my requests to the throttled systems through a system that makes sure not too many requests are let through and in the mean time the workflows can simply unload while they wait for their data via a callback.
Now I have experience with Windows Workflow. Albeit an earlier version. Some of the main problems we ran into still exist though: versioning with persisted workflows, tooling for viewing information about the workflows. These problems can be solved, but take a lot of work.

Then I started thinking about
I watched a few videos, read a few articles and it occurred to me I could also use to solve some of my problems. I will cover on how will help me in future posts as I intent to document my learning process with

I would really encourage you to read about on their website:
It's still under heavy development and at the time of writing version 1.0.3 is the latest. Version 1.1 will be the one I'll be waiting for, but until then I will play with it for one-off tooling projects. And I would also suggest you do the same if you really want to use it. There is a lot to learn and I've noticed actor-based systems kind of collide with how I think about program logic. So it takes a lot of getting used to for me, but I'm positive it'll be worth it.