XML element order in WCF

Call me old-fashioned, but I’ve always held to the belief that in XML nesting is very important and order should never be important. I’ve always found it unbelievably stupid when some application depended on the order of elements in an XML document. To me XML is all about representing structured data, not about a rigorous document layout.

Imagine my surprise when I started testing my new WCF webservice with a request I wrote by hand. All of a sudden a member was left null no matter what I did. Because it was the last element in my hand-written request that remained null, my first though (after extensively verifying I didn’t make any typo’s) was that, maybe, for some obscure reason, the last member didn’t get deserialized correctly. So I put another member last and indeed it was also null. Then I put another member last and al of a sudden all where filled, except for the first one.

After a few hours of WTF?!! I found out that the ‘default’ serializer for WCF, the Microsoft favourite, the “Data Object Serializer” has a dependency on XML element order. After changing the serializer to the “XML serializer” (just remove the “DataContract” attribute from the classes and the “DataMember” attributes from the members of those classes and replace then with the “Serializable” atttribute and/or any of the “Xml*” attributes from the “System.Xml.Serialization” namespace and tack the “XmlSerializerFormat” attribute on the class that contains your WCF methods) it all just worked! Order wasn’t important anymore, as it should be.

What I don’t get is why Microsoft would promote the “Data Object Serializer” as the preferred methods for WCF. The “XML Serializer” is much more flexible if you want to control what the resulting XML document should look like and XML element order doesn’t matter! The only advantage to using the “Data Object Serializer” is that it doesn’t serialize members of classes that do not have the required attribute (opt-in instead of the opt-out method of the “XML Serializer”) and that you can serialize private members, which can come in handy from time to time. But it has a dependency on XML element order!!

Terribly busy

It's been a while since I last posted anything. I've been extremely busy at work. We're currently nearing the first milestone of a very ambitious project. I'm not a 100% sure we'll make the deadline, but we're doing the best we can. We're currently using WCF and Windows Workflow. While the WCF is very basic, just a webservice to accept requests to do stuff, the WF stuff is more interesting, so I hope to be able to post more about that later. Ccurrently I'm trying to implement a SOAP logger. We had one for our ASMX webservices, but since we use WCF now we can't re-use it. According to some blogs I found on the subject creating one is easier then it was for ASMX webservices. But I haven't got it running yet. As soon as I have, I'll detail it here.

LINQ is faster then it looks

I was recently working on a LINQ query. I needed to get some information from a database table and I needed to know if certain information regarding the information I just retrieved from that database table was available in one of two other database tables. I wasn't interested in the information itself, just whether or not there was information. Basically I needed to return some stuff and two boolean's.

I couldn't get it to work in one single query, so as always I turned to my colleague who knows a helluva lot more about database and SQL than me and asked him for help. Since he doesn't know LINQ he asked me the SQL that LINQ creates for the query so he could use that as a starting point. So I fired up LINQPad and generated the SQL.

The SQL LINQ produced for my particular query was a lot of selects. It was selects within selects within selects within selects with a "Case-Then" with selects. So, understandably, my colleague expressed his disgust with the SQL and then proceeded to give me a very lean-and-mean query that did exactly what I wanted. I updated my existing LINQ query to use some of the SQL stuff he used in creating the query and my LINQ query gave the exact results I wanted. I was happy, but my colleague wasn't.

My colleague wanted the final SQL as produced by the LINQ query that I had now, so I went into LINQPad again, generated the query and send it to him. This time LINQ had added another level of selects and after seeing the query my colleague went on a rant about how bad LINQ was and how much performance we'd lose if we'd use it. I wasn't worried. This particular project I was working on was not a high-profile application, but rather an internal tool for supporting some testers. So as long as I got an answer from the SQL Server within 1 second I was happy. My colleague wasn't. He wanted to know exactly hoe much performance was lost on this LINQ nonsense.

My colleague took the SQL query he handcrafted into perfection over 30 minutes time and took the bloated SQL Query that LINQ produced for the query I created in less then 5 minutes and ran them both through the SQL Server Profiler. Then his heart stopped. According to the SQL Server Profiler both queries took exactly the same time, in fact both queries resulted in exactly the same execution plan!

Again this was just a reminder of something I mentioned before: Don't guess about performance. Measure! And also, the SQL Server Query Optimizer is pretty bad-ass :-)

Problems with Visual Studio 2008 SP1

I installed VS2008 SP1 the week after it came out. I used the prepare tool first, then installed the service pack and everything seemed to work perfectly. I haven't really checked out all of the new features, but I really like the fact that all the TODOs in your solution now show up, in stead of only the TODOs in the files you currently have opened. And I also like the VB-esque background compiling (or however they implemented it) and showing you selected compiler errors before you compile. If you create a function that returns a value, but haven't put in the return statement yet, the IDE will already inform you that the function doesn't return a value.

So all was fine and perfect, until I tried to debug an ASP.NET application. Then the IDE would freeze and prompt me with a window that told be VS2008 had an internal error and must close. I could then close VS2008 or close it and look for a solution online. Whichever I would choose, the window would disappear, but VS2008 would remain, taking up 100% CPU on one core. The other core would be pwned by WerFault.exe and it would not end until I'd manually end-tasked DevEnv.exe.

I had my ASP.NET development set up to use IIS, so I switched so Cassini (the internal ASP.NET development server), but that didn't work. I then tried to run the application from IIS again and attach the debugger. Still hanging.
I then searched on Google for a bit and tried some work-arounds for similar problems I found there, I also tried the repair option in the VS2008 installer, but debugging still hung the IDE. So I reported it at Microsoft Connect and went home (I was at work at the time).

That evening at home I completely removed VS2008 and everything related from my laptop and reinstalled it. I verified debugging worked and when it did I installed SP1 again. But this time debugging kept working. So I have no idea what was wrong, but it's all working again now. So if anyone runs into similar problems, just uninstall and install again.

Visual Studio 2008 Service Pack 1

VS2008 SP1 is out and you should download and install it. It's got major improvements. And it *finally* has the feature I requested somewhere around Visual Studio beta 2:

"In earlier versions of Visual Studio, the task list is populated by using the ToDo tasks that are specified in open files. In this service pack, the Visual C# IDE extends this functionality. The Visual C# IDE populates the task list by using the ToDo tasks that are specified in the whole solution."

Of course I'll have to see it before I believe it :-) I'm currently downloading the VS2008 SP1 ISO image.

Oh, and if you're going to install SP1, don't forget the run the Service Pack Preparation Tool first. This might safe you some hassle of conflicting hot fixes or betas.

Update: Don't forget the .NET 3.5 Service Pack 1!

Open source is not more secure by default

This week a rather interesting announcement was made in light of the Black Hat security conference. Some researcher claimed to be able to bypass all of Windows Vista enhanced security features through Internet Explorer by using Active-X, Java or .NET. They also claimed that Microsoft couldn't do anything to fix it, because it abused core architectural assumptions made by Microsoft.

This is pretty disturbing, but I'm waiting for an official Microsoft response. I recall something similar in the past and then Microsoft was able to patch it up. I just can't for the life of me remember what that was, so maybe I'm just imagining things.

This message also spurred some interesting discussions on forums around the world. A lot of people are expressing their "expert" opinion on the matter. And on public forums everyone is a die-hard kernel developer of course. The most interesting discussion that came up again was the notion that open source is more secure, because everyone can look at the source and fix bugs.

I've always found this idea to be naive. First of all, just because everyone can look at the code, doesn't mean everyone will. I'm a programmer, but I never look at any of the source code provided by open source applications, unless I want to see how they implemented a certain feature so I can use it too somewhere. But sure, there are bound to be enthusiasts who will review the code. And they will find an occasional bug, but will they find all the security bugs? Of course not, why should they? Are they all software security experts? No. I think these "extra reviewers" are more likely to find typos then actual security bugs. I'd rather put my faith on static analysis tools to detect common errors and security researchers that try to break an application then a hobbyist security "guru".

I think the strength of open source lies in the fact that everyone interested can pitch in, but not because every user is a security reviewer.

SecuROM strikes again

I've posted about DRM before and I've mentioned SecuROM before. So today I was uninstalling a program that left a shortcut on my desktop. So I wanted to right-click on it and delete it. But when I right-clicked on it, Explorer crashed. Thinking it might be a fluke I tried again and Explorer crashed again. Hmmm...so I figured that it might be the shortcut itself, so I selected it and pressed "delete", but...Explorer crashed again. So I opened a command prompt and deleted the shortcut from there. That worked.

Then I right-clicked another shortcut and Explorer crashed again. Clicked another shortcut and Explorer crashed. So I rebooted Windows to solve the problem, only it wasn't solved. Explorer still crashed if I right-clicked a shortcut.

I suspected it might have something to do with Shell Extensions. And when I googled the problem I found a program called ShellExView. This program shows you all the Shell Extensions that are currently loaded and let's you disable them. So I started the program, selected all the "Context Menu" type extensions and tried to disable them. Nothing happened. I figured it might be because I wasn't running the program as administrator.

So this was interesting. I couldn't run the program as administrator by right-clicking on it (that would just crash Explorer), the CTRL-Shift-Enter method of starting an elevated program would also cause Explorer to crash, so I had a problem. But then I remembered something about how Vista determines when to elevate a program even when it's not asked to. You see for installers you'd need to run them elevated or they wouldn't be able to install anything, but older installers won't have a manifest that Vista can use to determine they need elevation. I read on one of the many developer blogs I read, that Vista can also decide to elevate based on the name of the executable. So I renamed "shexview.exe" to "shexviewsetup.exe" and I got a nice elevation prompt.

So I disabled all the "Context Menu" extension and tried to right-click a shortcut. No crash. Great! But which extension caused the problem. I wanted to enable them all one-by-one when I saw an extension that I immediately suspected of being the culprit. The extension was called "CmdLineContextMenu Class" and it's description read: "SecuROM context menu for Explorer."

Why would I need a SecuROM extension installed? What does it do? And why was it installed without my consent? The only thing I installed right before I started noticing crashes was the new Space Siege demo. But why would you want a freely available demo to have copy protected? Oh wait, it's not the first time they've done that. But I already uninstalled Space Siege so why wasn't SecuROM removed? Why would companies want to leave a rootkit behind on my system? Then again, I'm not absolutely sure it came from the Space Siege demo (UPDATE: it didn't, see below), but it was the last thing I installed before Explorer started crashing. I tried looking in the installer files of the demo, but those are InstallShield cabinet files and I can't look into them.

Aside from being some nasty piece of DRM, the extension was installed in a very peculiar location. Not in "Program Files" or maybe the "Windows" directory like you'd expect. No, it was installed in the temp directory of my user profile! It's called "CmdLineExt.dll" and when I looked in the temp directory I also noticed the file "drm_dyndata_7370010.dll" of which the file details also mentioned being part of SecuROM.

Ofcourse the question is, why did it crash? Now I'm happy it did, otherwise I wouldn't have found out, but I'm still curious as to why it crashes Explorer whenever I right-click a shortcut.
I'm a big fan of Mark Russinovich's blog and his "The case of..." series of blog post. I have WinDBG installed, however I'm mainly a C# developer and have not done any real Win32 C++ programming for years and even when I did, it's wasn't at the low level that Mark understands. But, I did do some basic stuff with WinDBG.

I attached WinDBG to Explorer, I enabled the extension and I made Explorer crash. So now I'm in WinDBG and it's telling me Explorer has crashed. I run the !analyze command and I get the following feedback: "Probably caused by : heap_corruption ( heap_corruption!heap_corruption )".

I get the callstack and I see these as the last lines:

77b39790 04800fd8 00250000 ntdll!DbgBreakPoint
c0000374 77c4c030 000ee1ac ntdll!RtlReportCriticalFailure+0x2e
00000002 77b39754 00000000 ntdll!RtlpReportHeapFailure+0x21
00000008 00250000 04800fd8 ntdll!RtlpLogHeapFailure+0xa1
00250000 00000000 04800fe0 ntdll!RtlFreeHeap+0x60
04800fe0 000ee408 00000000 kernel32!GlobalFree+0x47
000ee240 00000001 04800fe0 ole32!ReleaseStgMedium+0x124
WARNING: Stack unwind information not available. Following frames may be wrong.
03d41cdc 00000000 0030db98 CmdLineExt!DllUnregisterServer+0x3c1c
02640e60 0030db98 00000004 SHELL32!HDXA_QueryContextMenu+0x1b5
055c2ee8 00f902ef 00000000 SHELL32!CDefFolderMenu::QueryContextMenu+0x38b

Looks about right, although I can't be absolutely sure since I don't have de debug symbols for the SecuROM DLL.

So looking at the stacktrace I get the idea the SecuROM extension is trying to free some object COM twice. I don't know what object and I don't know why, my low-level knowledge ends about here. I'm just happy I found the problem and make my system right-clickable again.

Oh and one interesting thing I've learned tonight. When you have Explorer.exe crashed in the debugger, you can't use ALT-TAB (which wasn't unexpected), but you can use WIN-TAB (Flip3D). That made switching back and forth between windows a whole lot easier.

UPDATE:
It apears it wasn't the Space Siege demo that infected me with the DRM rootkit, it was the Mass Effect 1.01 update that did. I installed it, but installed the Space Siege demo right after it. So even though I did't get infected by their rootkit the first time, I was stupid enough to fall for it anyways.