Friday, February 20, 2009

Error loading workflow

Have you ever seen an error message like this when developing a Windows Workflow application?

ErrorLoadingWorkflow

I have. Lots of times.

I’m not sure what is causing this. We have a project that contains the workflow and we have several other projects that contain the activities we use. Also we do a little inheritance, where all workflow inherit from a base workflow and all activities inherit from a base activity. The only thing they inherit are properties and functions. We do not have a basic workflow with activities which other workflow then add to. Although I would really love to have that!

So far I discovered two possible solutions workarounds:

  1. Rebuild the projects containing the workflow classes.
  2. Restart Visual Studio.

Solution #1 is preferable, because otherwise, if you have a large solution, the time it takes for Visual Studio to load your solution can get really annoying. So what I did was create a simple small MSBuild script that cleans and then builds all my workflow project. It starts by cleaning the projects and then builds the projects. First building the projects containing the activities and then the projects containing the workflows. Most of the time that worked for me. And you don’t even need to close anything. You can even keep the workflow designer open with the error message showing. After the build it will refresh and if successful the workflow will be shown.

But sometimes that does not work. Then I would have to quit Visual Studio, run my build script and open Visual Studio again.

It’s also very possible that you have a genuine problem or bug. It is pretty easy to create a situation where you end up being unable to open the workflow in the workflow designer and you’ll have to edit the code-behind file by hand.

Now I’m sure that our inheritance aggravates some of the problems as inheritance isn’t officially supported in Windows Workflow (although I heard they might have something for that in WF 4.0, but I haven’t checked that out yet), but I suspect you’ll run into these problems every now and then even if you do not use inheritance.

Saturday, January 31, 2009

Seeing a workflow in the database doesn’t mean it’s persisted

At work we are building a big application that uses Windows Workflow. We are delivering versions to Test at certain milestones so we don’t have to build everything first before we can see how good we built it.

We ran into an interesting issue I want to share. But first let me tell you about some technical details. As I said, we use Windows Workflow to use workflows in our application. The application receives an order (through a webservice), does dome checking and authenticating and then finds the correct workflow to handle the order and start that workflow. The workflows we make inherit from a base class. This base class contains 2 or 3 dependency properties that every workflow should have (such as the order data) and also contains some methods that every workflow can use. We also have a base class for some activities, but these mainly contain methods that most other activities will need. A workflow also needs to implement a certain interface, so we can load all the workflows with our Dependency Injection container and can determine which interface we need for which order.

So far so good, aside from some other problems I want to blog about soon it all seemed to work perfectly. Our first milestone was a couple of simple workflows that started, did some stuff and ended. Nothing special and they worked perfectly. If you used Workflow Monitor to look in the database you’d see the workflow. So to us that meant the workflow was successfully persisted.

Then for the second milestone we created a bunch of more complex workflows. These workflows did a call to a webservice and had to wait on the call-back from that webservice. In workflow this is handled very easily using the ExternalDataExchangeService and the HandleExternalEventActivity. We coded it up and started testing.

When the call-back came in, some stuff happened like validation, and eventually the call-back data came to the part where we’d ask the workflow runtime to give use a pointer to the workflow instance that’s waiting on the data and then use the ExternalDataExchangeService to deliver the data to the workflow so it can go on with it’s process. But instead of just working, the workflow runtime told use that the requested workflow couldn’t be found in the persistence store. We checked with the Workflow Monitor and we saw the workflow. It’s status set to running (or idle). But the runtime insisted it was not there.

Huh??

I did some digging and finally noticed an error in our log files. Some error about an interface not being serializable which caused the persistence to fail. Turns out that what we see in the Workflow Monitor is Workflow Tracking and not Workflow Persistence. In fact, unless you explicitly tell it to do so, workflows will not be persisted. You can either explicitly tell the Workflow Runtime to persist your workflow, or you can use some activity that will cause it to be persisted. Which is what (as far as I know) most activities do that will cause your workflow to be paused. So in our first milestone nothing caused the workflow to be persisted, but in our second milestone we caused the workflow to pause at the HandleExternalEventActivity which meant the workflow had to be persisted, which then failed.

So what was the problem? Well, all fields and get+set properties on a workflow or activity have to be serializable. So it must have the [Serializable] attribute tacked onto its class or interface and onto every class or interface that is inside that class or interface. Since we set up some references to some logic classes (Logging Service, Order Service, etc) in the base class, they would also need to be serializable.

We solved the problem by refactoring all the service fields into read-only properties (‘get’ properties) and not store the reference in the workflow or activity, but rather ask our Dependency Injection container each time we need one through a static method.
We also removed some other classes that where stored as private fields and just instantiate them within a method as they are needed and a few classes were made serializable.

The biggest downside of this is that Windows Workflow demands a very strict versioning scheme, so this means that we now have a few more assemblies that require careful versioning.

Friday, January 16, 2009

Application configuration in the database

Recently K. Scott Allen posted an article called App Configuration and Databases.

His conclusion is:

The database should be one of your last choices as a repository for configuration information. Configuration in code is easily tested, versioned, changed, and refactored. As Jeff said – only a specific requirement should drive you to a more costly alternative like the database.

I humbly disagree. We used to put are configuration into the web.config or our projects. This was easy, but had two down sides:

  1. Whenever we release a new version it’s best our operations guys overwrite to old config, because we might have changed some values in there that they should not mess with. Things like configuration for our DI containers, or WCF settings. They do need to copy the settings from the old config into the new one and that takes time and there is chance of error, especially if you have large config files.
  2. It’s impossible to do “hot-configuration”. If you change the web.config, IIS will restart the web application. This is not a big issue, but did cause some problems once or twice.

As for configuration in code I see the following problems:

  1. We use OTAP (separate environments for Development, Test, User Acceptance and Production), this mean that we’d either have to create different “configuration DLLs” for our different environments or that we’d need to have code that detected what kind of environment the application ran in and use that to determine the correct configuration.
  2. If you’d need a configuration change, you’d need a developer to check-out code, change it, check it back in and deliver the code to test. Test would need to run al kinds of tests and when all done deliver the application to operations for installation. Configuration in a separate file, or database is something the operations guy can do himself.
  3. We, as developers, don’t have access to production machines. Security regulations dictate we should not know passwords to production environments, so we couldn’t even create the configuration DLLs.

Of course these concerns can all be addressed, but we decided it was better to use a database.

  • If the configuration is in a database, the application (and it’s associated web.config file) can be overwritten without problems. (Of course some values would need to be copied, but they would only be a few).
  • We can provide the application with default configuration values and the tester can change them for the test environment. The guy from operations can change them into the production values.
  • If you’re doing load balancing on the application it’s possible to reuse the same configuration database (might not be practical for some systems, but it is a possibility).
  • It’s easy to add a configuration page to an administration web application that we’ll probably need to build anyway.

So all-in-all it seems like the best solution for us (for the moment), but that’s largely because of the way we work. Your company may use other methods which might not make configuration in the database the best choice, but I don’t think it should be the last choice.

Tuesday, January 6, 2009

SOAP logging for webservices

If you wish to log incoming SOAP requests to your ASMX webservice, you can write a SOAP extension that does just that.

You’ll need to create a class that inherits from SoapExtension and you should override the methods GetInitializer, Initialize, ChainStream and ProcessMessage.
It’s not that hard, you can find an working example at the MSDN site: How to: Implement a SOAP Extension. It works for both the server (who accepts incoming SOAP requests) and a client (who sends SOAP requests to servers).

All you need to do is add this to your web.config (or app.config) and it will work:

<system.web>
    <webServices>
        <soapExtensionTypes>
            <add type="MySoapLogger, MyNamespace.SoapLogger" priority="1" group="High" />
        </soapExtensionTypes>
    </webServices>
</system.web>

If you wish to create a SOAP logger for a WCF webservice, that’s both easier and harder. It’s simpler to write the extension for, but now you must write different things for a server and a client. And making it configurable requires some more work then the ASMX version.

In WCF you don’t write a SOAP Extension, but rather a Message Interceptor. And you won’t need to inherit from a base class, but you’ll need to implement certain interfaces to make it work.
To be able to see incoming (server) messages you’ll need to implement IDispatchMessageInspector and for outgoing (client) messages you’ll need to implement IClientMessageInspector. It's worth mentioning that nothing prevents you from implementing these interfaces both on the same class. That's what I did.

Both interfaces require you implement two functions:

object IDispatchMessageInspector.AfterReceiveRequest(ref Message request, IClientChannel channel, InstanceContext instanceContext);
void IDispatchMessageInspector.BeforeSendReply(ref Message reply, object correlationState);

And:

object IClientMessageInspector.BeforeSendRequest(ref Message request, IClientChannel channel);
void IClientMessageInspector.AfterReceiveReply(ref Message reply, object correlationState);

For IDispatchMessageInspector you have AfterReceiveRequest for when the request is received, before your normal code will run and BeforeSendReply, which is directly before the response will be send to the requester, after all your normal code has run.
For IClientMessageInspector you have BeforeSendRequest which is right before the request is send to the server, after all your normal code has run and AfterReceiveReply for when the response from the server has been received, before your normal code will run.

Basically you can do anything to the request you want at those points, even alter the data. One of the examples I found was to validate incoming requests to an XSD. Another was to do some translations for backwards compatibility sake. But in this case we just want to log the data.

So what we can do in each of these functions, is the following:

try
{
    // Only try to parse the message if it is not empty.
    if(message.IsEmpty == false)
    {
        string xml = message.ToString();
        XDocument xmlDocument = XDocument.Parse(xml);

        StringBuilder sb = new StringBuilder();
        XmlWriter writer = XmlWriter.Create(sb, new XmlWriterSettings { Encoding = Encoding.ASCII, Indent = true });
        xmlDocument.WriteTo(writer);
        writer.Close();

        sb.Append("\r\n");
        this.soapLogger.Log(sb.ToString());
    }
}
catch(Exception e)
{
    if(System.Diagnostics.Debugger.IsAttached == true)
    {
        System.Diagnostics.Debugger.Break();
    }
}

In this case the "message" variable contains the SOAP message. I wrote one function to do the logging and have it called from the 4 functions mentioned above. Also the "soapLogger" class variable is the logger I use. It's not particulary important which logger this specifically is, but if you want to know, I use Log4NET.

The use of the XDocument and the XmlWriter is absolutely not necessary, but I use them to make the XML look pretty (indented) in the log. It is a lot slower then just directly logging it to the logfile, but that would result in the whole XML message to appear on 1 line.
The stuff in the try/catch clause is a neat trick I use to catch the unexpected exceptions.

If you want the most simple solution, you can also do it like this:

// Only try to parse the message if it is not empty.
if(message.IsEmpty == false)
{
    string xml = message.ToString();
    this.soapLogger.Log(xml);
}

Monday, December 29, 2008

XML element order in WCF

Call me old-fashioned, but I’ve always held to the belief that in XML nesting is very important and order should never be important. I’ve always found it unbelievably stupid when some application depended on the order of elements in an XML document. To me XML is all about representing structured data, not about a rigorous document layout.

Imagine my surprise when I started testing my new WCF webservice with a request I wrote by hand. All of a sudden a member was left null no matter what I did. Because it was the last element in my hand-written request that remained null, my first though (after extensively verifying I didn’t make any typo’s) was that, maybe, for some obscure reason, the last member didn’t get deserialized correctly. So I put another member last and indeed it was also null. Then I put another member last and al of a sudden all where filled, except for the first one.

After a few hours of WTF?!! I found out that the ‘default’ serializer for WCF, the Microsoft favourite, the “Data Object Serializer” has a dependency on XML element order. After changing the serializer to the “XML serializer” (just remove the “DataContract” attribute from the classes and the “DataMember” attributes from the members of those classes and replace then with the “Serializable” atttribute and/or any of the “Xml*” attributes from the “System.Xml.Serialization” namespace and tack the “XmlSerializerFormat” attribute on the class that contains your WCF methods) it all just worked! Order wasn’t important anymore, as it should be.

What I don’t get is why Microsoft would promote the “Data Object Serializer” as the preferred methods for WCF. The “XML Serializer” is much more flexible if you want to control what the resulting XML document should look like and XML element order doesn’t matter! The only advantage to using the “Data Object Serializer” is that it doesn’t serialize members of classes that do not have the required attribute (opt-in instead of the opt-out method of the “XML Serializer”) and that you can serialize private members, which can come in handy from time to time. But it has a dependency on XML element order!!

Wednesday, December 3, 2008

Terribly busy

It's been a while since I last posted anything. I've been extremely busy at work. We're currently nearing the first milestone of a very ambitious project. I'm not a 100% sure we'll make the deadline, but we're doing the best we can. We're currently using WCF and Windows Workflow. While the WCF is very basic, just a webservice to accept requests to do stuff, the WF stuff is more interesting, so I hope to be able to post more about that later. Ccurrently I'm trying to implement a SOAP logger. We had one for our ASMX webservices, but since we use WCF now we can't re-use it. According to some blogs I found on the subject creating one is easier then it was for ASMX webservices. But I haven't got it running yet. As soon as I have, I'll detail it here.

Tuesday, September 2, 2008

LINQ is faster then it looks

I was recently working on a LINQ query. I needed to get some information from a database table and I needed to know if certain information regarding the information I just retrieved from that database table was available in one of two other database tables. I wasn't interested in the information itself, just whether or not there was information. Basically I needed to return some stuff and two boolean's.

I couldn't get it to work in one single query, so as always I turned to my colleague who knows a helluva lot more about database and SQL than me and asked him for help. Since he doesn't know LINQ he asked me the SQL that LINQ creates for the query so he could use that as a starting point. So I fired up LINQPad and generated the SQL.

The SQL LINQ produced for my particular query was a lot of selects. It was selects within selects within selects within selects with a "Case-Then" with selects. So, understandably, my colleague expressed his disgust with the SQL and then proceeded to give me a very lean-and-mean query that did exactly what I wanted. I updated my existing LINQ query to use some of the SQL stuff he used in creating the query and my LINQ query gave the exact results I wanted. I was happy, but my colleague wasn't.

My colleague wanted the final SQL as produced by the LINQ query that I had now, so I went into LINQPad again, generated the query and send it to him. This time LINQ had added another level of selects and after seeing the query my colleague went on a rant about how bad LINQ was and how much performance we'd lose if we'd use it. I wasn't worried. This particular project I was working on was not a high-profile application, but rather an internal tool for supporting some testers. So as long as I got an answer from the SQL Server within 1 second I was happy. My colleague wasn't. He wanted to know exactly hoe much performance was lost on this LINQ nonsense.

My colleague took the SQL query he handcrafted into perfection over 30 minutes time and took the bloated SQL Query that LINQ produced for the query I created in less then 5 minutes and ran them both through the SQL Server Profiler. Then his heart stopped. According to the SQL Server Profiler both queries took exactly the same time, in fact both queries resulted in exactly the same execution plan!

Again this was just a reminder of something I mentioned before: Don't guess about performance. Measure! And also, the SQL Server Query Optimizer is pretty bad-ass :-)