Saturday, January 31, 2009

Seeing a workflow in the database doesn’t mean it’s persisted

At work we are building a big application that uses Windows Workflow. We are delivering versions to Test at certain milestones so we don’t have to build everything first before we can see how good we built it.

We ran into an interesting issue I want to share. But first let me tell you about some technical details. As I said, we use Windows Workflow to use workflows in our application. The application receives an order (through a webservice), does dome checking and authenticating and then finds the correct workflow to handle the order and start that workflow. The workflows we make inherit from a base class. This base class contains 2 or 3 dependency properties that every workflow should have (such as the order data) and also contains some methods that every workflow can use. We also have a base class for some activities, but these mainly contain methods that most other activities will need. A workflow also needs to implement a certain interface, so we can load all the workflows with our Dependency Injection container and can determine which interface we need for which order.

So far so good, aside from some other problems I want to blog about soon it all seemed to work perfectly. Our first milestone was a couple of simple workflows that started, did some stuff and ended. Nothing special and they worked perfectly. If you used Workflow Monitor to look in the database you’d see the workflow. So to us that meant the workflow was successfully persisted.

Then for the second milestone we created a bunch of more complex workflows. These workflows did a call to a webservice and had to wait on the call-back from that webservice. In workflow this is handled very easily using the ExternalDataExchangeService and the HandleExternalEventActivity. We coded it up and started testing.

When the call-back came in, some stuff happened like validation, and eventually the call-back data came to the part where we’d ask the workflow runtime to give use a pointer to the workflow instance that’s waiting on the data and then use the ExternalDataExchangeService to deliver the data to the workflow so it can go on with it’s process. But instead of just working, the workflow runtime told use that the requested workflow couldn’t be found in the persistence store. We checked with the Workflow Monitor and we saw the workflow. It’s status set to running (or idle). But the runtime insisted it was not there.

Huh??

I did some digging and finally noticed an error in our log files. Some error about an interface not being serializable which caused the persistence to fail. Turns out that what we see in the Workflow Monitor is Workflow Tracking and not Workflow Persistence. In fact, unless you explicitly tell it to do so, workflows will not be persisted. You can either explicitly tell the Workflow Runtime to persist your workflow, or you can use some activity that will cause it to be persisted. Which is what (as far as I know) most activities do that will cause your workflow to be paused. So in our first milestone nothing caused the workflow to be persisted, but in our second milestone we caused the workflow to pause at the HandleExternalEventActivity which meant the workflow had to be persisted, which then failed.

So what was the problem? Well, all fields and get+set properties on a workflow or activity have to be serializable. So it must have the [Serializable] attribute tacked onto its class or interface and onto every class or interface that is inside that class or interface. Since we set up some references to some logic classes (Logging Service, Order Service, etc) in the base class, they would also need to be serializable.

We solved the problem by refactoring all the service fields into read-only properties (‘get’ properties) and not store the reference in the workflow or activity, but rather ask our Dependency Injection container each time we need one through a static method.
We also removed some other classes that where stored as private fields and just instantiate them within a method as they are needed and a few classes were made serializable.

The biggest downside of this is that Windows Workflow demands a very strict versioning scheme, so this means that we now have a few more assemblies that require careful versioning.

Friday, January 16, 2009

Application configuration in the database

Recently K. Scott Allen posted an article called App Configuration and Databases.

His conclusion is:

The database should be one of your last choices as a repository for configuration information. Configuration in code is easily tested, versioned, changed, and refactored. As Jeff said – only a specific requirement should drive you to a more costly alternative like the database.

I humbly disagree. We used to put are configuration into the web.config or our projects. This was easy, but had two down sides:

  1. Whenever we release a new version it’s best our operations guys overwrite to old config, because we might have changed some values in there that they should not mess with. Things like configuration for our DI containers, or WCF settings. They do need to copy the settings from the old config into the new one and that takes time and there is chance of error, especially if you have large config files.
  2. It’s impossible to do “hot-configuration”. If you change the web.config, IIS will restart the web application. This is not a big issue, but did cause some problems once or twice.

As for configuration in code I see the following problems:

  1. We use OTAP (separate environments for Development, Test, User Acceptance and Production), this mean that we’d either have to create different “configuration DLLs” for our different environments or that we’d need to have code that detected what kind of environment the application ran in and use that to determine the correct configuration.
  2. If you’d need a configuration change, you’d need a developer to check-out code, change it, check it back in and deliver the code to test. Test would need to run al kinds of tests and when all done deliver the application to operations for installation. Configuration in a separate file, or database is something the operations guy can do himself.
  3. We, as developers, don’t have access to production machines. Security regulations dictate we should not know passwords to production environments, so we couldn’t even create the configuration DLLs.

Of course these concerns can all be addressed, but we decided it was better to use a database.

  • If the configuration is in a database, the application (and it’s associated web.config file) can be overwritten without problems. (Of course some values would need to be copied, but they would only be a few).
  • We can provide the application with default configuration values and the tester can change them for the test environment. The guy from operations can change them into the production values.
  • If you’re doing load balancing on the application it’s possible to reuse the same configuration database (might not be practical for some systems, but it is a possibility).
  • It’s easy to add a configuration page to an administration web application that we’ll probably need to build anyway.

So all-in-all it seems like the best solution for us (for the moment), but that’s largely because of the way we work. Your company may use other methods which might not make configuration in the database the best choice, but I don’t think it should be the last choice.

Tuesday, January 6, 2009

SOAP logging for webservices

If you wish to log incoming SOAP requests to your ASMX webservice, you can write a SOAP extension that does just that.

You’ll need to create a class that inherits from SoapExtension and you should override the methods GetInitializer, Initialize, ChainStream and ProcessMessage.
It’s not that hard, you can find an working example at the MSDN site: How to: Implement a SOAP Extension. It works for both the server (who accepts incoming SOAP requests) and a client (who sends SOAP requests to servers).

All you need to do is add this to your web.config (or app.config) and it will work:

<system.web>
    <webServices>
        <soapExtensionTypes>
            <add type="MySoapLogger, MyNamespace.SoapLogger" priority="1" group="High" />
        </soapExtensionTypes>
    </webServices>
</system.web>

If you wish to create a SOAP logger for a WCF webservice, that’s both easier and harder. It’s simpler to write the extension for, but now you must write different things for a server and a client. And making it configurable requires some more work then the ASMX version.

In WCF you don’t write a SOAP Extension, but rather a Message Interceptor. And you won’t need to inherit from a base class, but you’ll need to implement certain interfaces to make it work.
To be able to see incoming (server) messages you’ll need to implement IDispatchMessageInspector and for outgoing (client) messages you’ll need to implement IClientMessageInspector. It's worth mentioning that nothing prevents you from implementing these interfaces both on the same class. That's what I did.

Both interfaces require you implement two functions:

object IDispatchMessageInspector.AfterReceiveRequest(ref Message request, IClientChannel channel, InstanceContext instanceContext);
void IDispatchMessageInspector.BeforeSendReply(ref Message reply, object correlationState);

And:

object IClientMessageInspector.BeforeSendRequest(ref Message request, IClientChannel channel);
void IClientMessageInspector.AfterReceiveReply(ref Message reply, object correlationState);

For IDispatchMessageInspector you have AfterReceiveRequest for when the request is received, before your normal code will run and BeforeSendReply, which is directly before the response will be send to the requester, after all your normal code has run.
For IClientMessageInspector you have BeforeSendRequest which is right before the request is send to the server, after all your normal code has run and AfterReceiveReply for when the response from the server has been received, before your normal code will run.

Basically you can do anything to the request you want at those points, even alter the data. One of the examples I found was to validate incoming requests to an XSD. Another was to do some translations for backwards compatibility sake. But in this case we just want to log the data.

So what we can do in each of these functions, is the following:

try
{
    // Only try to parse the message if it is not empty.
    if(message.IsEmpty == false)
    {
        string xml = message.ToString();
        XDocument xmlDocument = XDocument.Parse(xml);

        StringBuilder sb = new StringBuilder();
        XmlWriter writer = XmlWriter.Create(sb, new XmlWriterSettings { Encoding = Encoding.ASCII, Indent = true });
        xmlDocument.WriteTo(writer);
        writer.Close();

        sb.Append("\r\n");
        this.soapLogger.Log(sb.ToString());
    }
}
catch(Exception e)
{
    if(System.Diagnostics.Debugger.IsAttached == true)
    {
        System.Diagnostics.Debugger.Break();
    }
}

In this case the "message" variable contains the SOAP message. I wrote one function to do the logging and have it called from the 4 functions mentioned above. Also the "soapLogger" class variable is the logger I use. It's not particulary important which logger this specifically is, but if you want to know, I use Log4NET.

The use of the XDocument and the XmlWriter is absolutely not necessary, but I use them to make the XML look pretty (indented) in the log. It is a lot slower then just directly logging it to the logfile, but that would result in the whole XML message to appear on 1 line.
The stuff in the try/catch clause is a neat trick I use to catch the unexpected exceptions.

If you want the most simple solution, you can also do it like this:

// Only try to parse the message if it is not empty.
if(message.IsEmpty == false)
{
    string xml = message.ToString();
    this.soapLogger.Log(xml);
}