Wednesday, 08 July 2009

Hi all

2 and a half year ago, I wrote a blog post about looping around elements, which can be found here: http://blog.eliasen.dk/2006/11/05/LoopingAroundElementsOfAMessage.aspx. In this post, I mention two ways to do something per element in a message. One is to use the XML (or flat file) disassembler to split the message into lots of messages and have them handled individually, and the other is to do it inside an orchestration.

Since then, BizTalk has been updated such that you can now call a receive pipeline from within your orchestration, which gives you the another way of looping through elements inside an orchestration.

Often times, in the forums, users are suggested to loop through elements of a message inside an orchestration. It is a fairly common approach if you want the subelements of a message handled in the correct order, for instance.

This post is to make sure people know that looping around a message inside an orchestration can have a serious performance impact.

Say you receive a large message – 100MB. You start looping around it inside an orchestration and split it up into, say, 1000 messages of 100k each. Then, for each message you call a web service or send out the message to some system. What will happen is, that the orchestration will persist its state after each send shape, meaning that the 100BM is written 1000 times to the messagebox. Persistence points are something you always need to be aware of when designing your orchestrations, and with messages of 100MB, even more so.

SO, don’t just loop around a message inside an orchestration because you can… do it when you absolutely have to.

Thanks

--
eliasen

Wednesday, 08 July 2009 22:07:07 (Romance Daylight Time, UTC+02:00)  #    Comments [2]  | 

Hi all

Upon reading a forum post today, I stopped to think about the usage of custom XSLT to completely replace the mapper, custom scripting functoids and the built-in functoids.

Some people can’t be bothered using the mapper and go straight to custom xslt whenever a mapping is to be created. Others use the built-in functoids whenever possible. And the rest go for some path in the middle, with functoids and the occasional scripting functoid whenever the built-in functoids either can’t get the job done or seem to be too troublesome to use.

The advantages of custom XSLT

  1. Performance. It seems rather obvious that automatically generated XSLT can’t perform as well as custom XSLT. So if you really need high performance, use your own XSLT.
  2. Maintainability. I will be using this argument against custom XSLT as well :-) but in essence, some people think that having a map with perhaps 200 functoids is too difficult to maintain. Sometimes you need lots of functoids in order to complete some mapping task that can be easily done in XSLT. In that case, the XSLT can be more readable and hence maintainable than using functoids.

The advantages of using the built-in functoids

  1. Overview. Using the functoids can give a developer an overview of what functionality a map provides, whereas custom xslt needs to be gone through and “parsed” by a developer before he can understand what is happening.
  2. Maintainability. (See, there it is again – in the opposite direction). A BizTalk developer knows about maps and functoids. He does snot necessarily know about xslt – and even if he does, he is not guaranteed to very proficient in it. By staying with the functoids you give a BizTalk developer the best possible opportunity to understand a new map he is going to maintain.

The advantages of using functoids and the scripting functoids

  1. Everything is possible :-). Basically you can do what you want. Use built-in functoids as much as you prefer and throw in scripting functoids if you think the solution either can’t be done with the built-in functoids or is too troublesome to do.

My preference

Well, those that read my blog regularly wont be surprised when I say that my preference is to use the built-in functoids whenever possible. I turn to XSLT only when I cannot solve my problem with the built-in functoids. And when I need XSLT, I always use custom scripting functoids to supplement the other functoids, so I let as much of the job as possible be handles by built-in functoids. I never use custom XSLT to replace the map. My main reasons for this are:

  1. Maintainability. I truly believe that it is easier for a new BizTalk developer to troubleshoot and/or change an existing map if only built-in functoids are used.
  2. Overview. If a BizTalk developer opens up a map and sees lots of scripting functoids, he needs to open them all up and examine the code inside it in order to know what the map does. And after looking at 5 different scripting functoids, he needs to go back and look at the first again. Really annoying!

If I am doing a rally complex map where the number of custom scripting functoids is larger than the number of built-in functoids, I might consider using a custom XSLT script… but this haven’t happened yet :-) And also, if i run into really serious performance issues, I will consider it as well, off course.

Just my thoughts.

--
eliasen

Wednesday, 08 July 2009 21:36:39 (Romance Daylight Time, UTC+02:00)  #    Comments [2]  | 
Tuesday, 07 July 2009

Hi all

When trying to help the guy I wrote about in http://blog.eliasen.dk/2009/07/07/DistinguishedFieldNotWorking.aspx getting his distinguished field working, another guy suggested that he just used he xpath function instead of trying to get the distinguished field working. Now, he actually ended up using the xpath function because he had a reserved word in the xpath statement to the field he needed to access the field, but I thought I’d just write a post about why the xpath function should be avoided and perhaps generalize the post to the usage of promoted properties, distinguished fields and the xpath function. When to use what and why…

Why use distinguished fields over the xpath function

  1. Readability. Inside an expression shape, it is much easier for a BizTalk developer to look at “Message.MyElement.MyField” than it is to look at “xpath(Message, “string(/*[local-name()=’MyElement’ and namespace-uri()=’http://mynamespace.com/something/somethingmore’]/*[local-name()=’Myfield’ and namespace-uri()=’’])”)”
  2. Maintainability. If you need to change a schema at some point, the distinguished field is automatically updated to correspond to the new xpath expression that points to the relevant field (if you are using the BizTalk Editor, that is). If you use the xpath function, you need to find ALL occurrences of the xpath function in your entire solution that has an xpath expression that needs to be updated.
  3. Performance. If you use the xpath expression to get values form a message, then the entire message needs to be loaded from the database and the xpath expression is then evaluated. Distinguished fields, on the other hand, are kept in the context of the message and is therefore loaded quickly at runtime.

Why use promoted properties over distinguished fields

  1. Routing. Only promoted properties can be used for routing in the internal publish/subscribe engine if you need to route based on the content of a message.
  2. Correlation. If you need to correlate a received message into a specific instance of your orchestrations, then you need to do this using promoted properties, as these are the only properties that can be added to a correlation type. This makes sense when you think of it, because correlation is really just runtime routing - setting up the correct subscriptions at runtime that will make the message hit the correct orchestration instance. And routing can only be done with promoted properties, hence these are needed for correlation.
  3. Tracking. You cannot track distinguished fields – for tracking, you need promoted properties.

Why use the xpath function

  1. Distinguished fields cannot be used because there is a reserved word in the path of the xpath expression
  2. Distinguished fields cannot be used because you need to set or read a value from a reoccurring element

Why use distinguished fields over promoted properties

  1. Performance. You want to keep the number of promoted properties as low as possible, since the messaging engine really needs as few promoted properties as possible to evaluate when finding out which subscriptions match an incoming message that is published to the MessageBox.

So, to sum up: Use distinguished fields whenever possible, switching only to xpath function or promoted properties if really needed.

--
eliasen

Tuesday, 07 July 2009 23:36:10 (Romance Daylight Time, UTC+02:00)  #    Comments [0]  | 

Hi all

A guy on the forums posted a very small schema, in which he had promoted an element as a distinguished field.

His schema was this:

<?xml version="1.0" encoding="utf-16" ?>
<xsd:schema xmlns:b="http://schemas.microsoft.com/BizTalk/2003" attributeFormDefault="unqualified" elementFormDefault="qualified" xmlns:xsd="http://www.w3.org/2001/XMLSchema">
  <xsd:element name="outbound_task">
    <xsd:annotation>
      <xsd:appinfo>
        <properties xmlns="http://schemas.microsoft.com/BizTalk/2003">
          <property distinguished="true" xpath="/*[local-name()='outbound_task' and namespace-uri()='']/*[local-name()='task' and namespace-uri()='']/*[local-name()='task_id' and namespace-uri()='']" />
        </properties>
      </xsd:appinfo>
    </xsd:annotation>
    <xsd:complexType>
      <xsd:sequence>
        <xsd:element name="task">
          <xsd:complexType>
            <xsd:sequence>
              <xsd:element minOccurs="0" name="task_id" type="xsd:decimal" />
            </xsd:sequence>
          </xsd:complexType>
        </xsd:element>
      </xsd:sequence>
    </xsd:complexType>
  </xsd:element>
</xsd:schema>

His issue was, that when he tried using the distinguished field in an orchestration, the task_id field just didn’t show up in intellisense. And if he just entered the complete value in the expression shape (like this: Message.task.task_id) then he got a compile time error.

I messed around with it a lot, trying all sorts of stuff, but ended up with a quite simple solution: the word “task” is a reserved word. For a complete list of the reserved words, take a look at http://msdn.microsoft.com/en-us/library/aa547020.aspx. So, basically, renaming the “task” element to “Task” or something completely different (but still avoiding any reserved words) will work.

Hope this helps.

--
eliasen

Tuesday, 07 July 2009 22:27:16 (Romance Daylight Time, UTC+02:00)  #    Comments [0]  | 
Monday, 06 July 2009

Hi all

A long time ago, I had a post about the toolbox in Visual Studio keeping old functoids on it and crashing and stuff.

The post can be found here: http://blog.eliasen.dk/2006/12/05/RemovingFunctoidFromToolbox.aspx (Note the comments).

With this post I just wanted all to know that the path to the toolbox* file that should be deleted on a Windows Server 2008 with visual Studio 2008 is C:\Users\<User>\AppData\Local\Microsoft\VisualStudio\9.0

Hope it helps

--
eliasen

Monday, 06 July 2009 21:11:22 (Romance Daylight Time, UTC+02:00)  #    Comments [0]  | 

Hi all

So, yesterday I had to loop through all properties on a message inside a pipeline component.

When you program a general pipeline component, you get an Execute method that looks like this:

public IBaseMessage Execute(IPipelineContext pContext, IBaseMessage pInMsg)

Now, in order to loop through the properties, the common solution is like this (and you can find this in many many blog posts around the great internet):

for (int i = 0; i <= pInMsg.CountProperties; i++)
{
  string name;
  string ns;
  object value = pInMsg.Context.ReadAt(i, out name, out ns);
  // Do something with the vale, name and namespace
}

Now, this is quite all right as it is, but there is one small pitfall that I don’t think most people realize; The CountProperties property is a uint, meaning that it can contain values up to 2^32 = 4294967296. Unfortunately, the first parameter to the ReadAt method is an int, which only holds values up to (2^31) – 1 = 2147483647. So, if there are, say 3000000000 properties, then the CountProperties property will return the correct number of properties, but there is no way of calling the ReadAt method to get the property.

This is kind of silly.

Now, to be fair, I think that the most properties I have seen on a message ever may have been around 40-50 properties. So there is a looooong way to 2 billion properties :-) So in real life, I don’t expect anyone to run into this limitation – if you do, I’ll buy you a beer! :-)

BUT, just to be sure, what you should do to be absolutely correct in your code is this:

uint counter = pInMsg.CountProperties;
string name;
string ns;
if (counter > int.MaxValue)
{
  counter = int.MaxValue;
}
for (int i = 0; i < counter; i++)
{
  object value = pInMsg.Context.ReadAt(i, out name, out ns);
  // Do something with the vale, name and namespace
}

And then, off course, you should somehow notify someone that there were more properties than you could handle – like a Debug statement, an error in the eventlog or the such… perhaps even throw an exception BEFORE trying to enumerate the properties, sine it would be useless, the result you get.

Anyway… hope this is somehow a help to someone… not sure how, though :-)

--
eliasen

Monday, 06 July 2009 19:20:41 (Romance Daylight Time, UTC+02:00)  #    Comments [0]  | 
Friday, 03 July 2009

Hi all

During my years on the forums, I have seen plenty, plenty and plenty more posts about the SQL Server Adapter schema generator closing unexpectedly, without any errors or any artifacts created.

Today I ran into it myself for the very first time with a BizTalk 2009 installation. I did some searching and found this post: http://support.microsoft.com/kb/917847 – which only applies to BizTalk 2004 and BizTalk 2006.

This blog post is just to let everyone know that it applies to BizTalk 2009 as well.

Good luck out there…

--
eliasen

Friday, 03 July 2009 22:24:56 (Romance Daylight Time, UTC+02:00)  #    Comments [0]  | 
Thursday, 02 July 2009

Hi all.

So, I am very proud to be able to publish that I have been awarded the MVP title again as of July 1’st. This, naturally is a great honor and I am rally happy about it.

Not sure I am as happy about it as Randal van Splunteren, though – because he promised some weeks ago, that if I didn’t get reawarded he would eat his shorts, shoes, hat or something – can’t remember which one, but it is in my log! :-)

The MVP title has lots of advantages, naturally – the title, the prestige, the gifts, the monthly webcasts and what not, but mainly I am happy that I do not have to bother removing the MVP logo from my blog, web site, business cards, et cetera .-)

Anyway, this is my third MVP title – going for the fourth in a year! :-)

--
eliasen

Thursday, 02 July 2009 06:35:50 (Romance Daylight Time, UTC+02:00)  #    Comments [7]  | 
Tuesday, 30 June 2009

Hi all

A friend and colleague of mine has just release SolZip version 1.1 on CodePlex – Find it here: http://solzip.codeplex.com/.

Basically, it is a nifty way of zipping a Visual Studio 2008 C# solution. The utility is pointed towards the .sln file and then zips all files in the solution and projects into one zip file.

To me this is really nice, because as of BizTalk 2009, the project files are just specialized C# projects and therefore, it seems to work just fine for BizTalk 2009 solutions as well. I can use it when blogging to quickly zip up a solution to attach to a blog post without having to zip the entire folder and then deleting the large dll files and other silly stuff. SolZip just takes what is necessary.

I was the very first downloader of the 1.1 version – you should go download it now! :-)

--
eliasen

Tuesday, 30 June 2009 22:02:39 (Romance Daylight Time, UTC+02:00)  #    Comments [0]  | 
Sunday, 28 June 2009

Hi all

Recently, I posted a post about handling mapping to and from a comma separated list. You can find it at http://blog.eliasen.dk/2009/06/22/HandlingCommaSeparatedValuesInsideAMapPartI.aspx

I came to think that this was actually also a great opportunity for me to develop my first cumulative functoid. The functoid is then supposed to take the values as input and output the list of values with a comma in between them.

The functoid has been developed and added to my functoid library at http://eebiztalkfunctoids.codeplex.com – free to download. Also another functoid in the library, which has been there for a while is the “CSV Extract” functoid which extracts a specific position from a separated list.

--
eliasen

Sunday, 28 June 2009 00:58:34 (Romance Daylight Time, UTC+02:00)  #    Comments [0]  | 

Theme design by Jelle Druyts