Hi all

So, the other day, a guy asked a question on the online forums, and another guy tried helping out by stating, among other things, that the maps on receive ports are executed before the receive pipeline. This isn’t true, and I posted a post, where I tried to explain how things work. This ended up being slightly wrong, so I posted a correction, but now it seems I need to post another correction… and I ended up writing this post to explain how stuff works.

First of all, let me set one thing straight; When a message arrives on a receive location it is first sent through the receive pipeline that is specified on the receive location. This is needed before the map for several reasons, including converting the input to XML and promoting the MessageType, so the receive port can choose the correct map to execute. The receive pipeline also promotes all the distinguished fields and promoted properties that are specified on the schema.

Now, after the map has finished executing, the transformation engine will look up the schema for the output and it will instantiate the XMLDisassembler with this particular schema, so the disassembler doesn’t have to find the correct schema itself. After instantiating it, it will call it, so the XMLDisassembler will read the output from the map and promote all distinguished fields and promoted properties to the context of the message. Also, before doing the promotion, it will copy all the context from the original message, so you get all the properties from the adapter and so on copied to the destination message.

Now, there are a couple of issues to this, which most people don’t realize – mostly because they will only affect you on very very rare occasions:

Distinguished fields
I found that if you have an input message with a field marked as a distinguished field and then look at the context of the output from the map, then the output message also has the distinguished field from the input message in its context which really doesn't make sense, since you can't use it in any way. This has NO influence at runtime and NO influence at design time, so you will go through your life not realizing it. Also, usually we don’t set distinguished fields on the external schemas, because we don’t want to change them and because we don’t want to use schemas exposed to trading partners in our business processes, which is the only place we can use the distinguished fields.

Promoted Properties
If you have promoted a field from both input and destination schema to the same property, the value from input schema in the properties is overwritten by the value from the destination schema after the map. For the same reasons as for the distinguished fields, we rarely have promoted properties on the external schemas, and therefore you probably will never have an issue with this.

Envelopes
IF the schema for the destination message is marked as an envelope, the message will fail. This is because the disassembler will recognize the schema is an envelope and it will then debatch the message into several messages. Only the first is returned though, since the mapping engine which is calling the disassembler assumes this is not an envelope and therefore will only call the GetNext method once, whereas normally the GetNext emthod is called until it returns null. This first message is then looked at, but in the properties for the disassembler, the transformation engine has all ready set the only allowed schema, which was the envelope schema. So the disassembler only has the envelope schema as a possible schema, and the instance that comes out after debatching is not the output from the map anymore, meaning that it will fail with the standard error message “Details:"Document type "http://MyNameSpace.com#Record" does not match any of the given schemas."” As with the first to issues about distinguished fields and promoted properties, this should practically never happen, since you are most likely mapping the incoming message to some internal schema, for which there is usually no reason to mark as an envelope.

Conclusion
So, basically, when a message arrives, the receive pipeline is executed, then the map is executed and at the end, the XML disassembler is executed by the transformation engine to get all promoted fields in the destination message promoted. There are a couple of known issues with this, but they are either totally unimportant or very unlikely to occur.

I hope this helps someone out there.

--
eliasen

Hi all

Just a quick book review, since I have just read the “Who moved my Cheese” book.

It only takes like an hour to read, and it is the story of two mice and two people who are searching for cheese, where the cheese is a metaphor for something you really want to have. At some point, they run out of cheese, and the four characters take very different approaches to the changes in the environment. The point of the story is, naturally, that you can most likely identify yourself as one of the characters and after reading the book, perhaps you have learnt something about how you react to changes and how you should react – and hopefully improve your life.

Easily read with lots of points. Read it…

--
eliasen

Hi all

Just to let you all know, I have updated the documentation for my pipeline component library at http://eebiztalkpipelinecom.codeplex.com/ and the documentation for my functoid library at http://eebiztalkfunctoids.codeplex.com/

Happy reading

--
eliasen

Hi all

Saravana Kumar has launched version two of his great site http://www.biztalk247.com – it has loads of nice information about BizTalk, so go check it out.

Also, while you are checking it out, take a look at his other new site: http://blogdoc.biztalk247.com/ which is collecting information from lots of blog entries about BizTalk.

--
eliasen

Hi all

Today I encountered something I have never seen before, when creating a map.

The issue occurs because my customer had a schema that imports two other schemas, both of which have an element called “metadata” – but naturally the two schemas have different target namespaces.

The main schema imports both, and has two records just below the root, and these two records reference each of the two metadata elements in the two imported schemas.

So the first schema could look like this:

And the second schema could look like this:

So both have an element named “metadata” but one is in the namespace “http://TwoElementsDifferentNamespace.MetadataOne” and the other is in the namespace “http://TwoElementsDifferentNamespace.MetadataTwo”.

After that, I create the schema that impors both:

As you can see it imports the first two schemas, and has to elements that reference each of the metadata elements form the two first schemas.

Also, I crated an output schema that just has three elements and then I created this map:

Pretty simple. Now, the issue comes when compiling, because I get this error:

Exception Caught: The map contains a reference to a schema node that is not valid.  Perhaps the schema has changed.  Try reloading the map in the BizTalk Mapper.  The XSD XPath of the node is: /*[local-name()='<Schema>']/*[local-name()='Root']/*[local-name()='metadata']/*[local-name()='Field2']

o the first try was to reload the schema – but that didn’t help – it just broke one of my links.

In he end I found out, that the issue is that the links are stored like this in the .BTM file:

<Link LinkID="1" LinkFrom="/*[local-name()='&lt;Schema&gt;']/*[local-name()='Root']/*[local-name()='metadata']/*[local-name()='Field1']" LinkTo="/*[local-name()='&lt;Schema&gt;']/*[local-name()='OutputRoot']/*[local-name()='Field1']" Label="" />

<Link LinkID="2" LinkFrom="/*[local-name()='&lt;Schema&gt;']/*[local-name()='Root']/*[local-name()='SomeFields']/*[local-name()='Field3']" LinkTo="/*[local-name()='&lt;Schema&gt;']/*[local-name()='OutputRoot']/*[local-name()='Field2']" Label="" />

<Link LinkID="3" LinkFrom="/*[local-name()='&lt;Schema&gt;']/*[local-name()='Root']/*[local-name()='metadata']/*[local-name()='Field2']" LinkTo="/*[local-name()='&lt;Schema&gt;']/*[local-name()='OutputRoot']/*[local-name()='Field3']" Label="" />

So. basically, the .BTM file saves links as XPath expressions WITHOUT the namespaces. So naturally, this has to go wrong, when there are two “metadata” elements on the same level in the schema.

The way to solve this is to choose the properties of the map and disable the “Ignore Namespaces for Links” like this:

After setting this property, the links change having namespaces inside the .BTM file and everything is just fine.

One might wonder why the namespaces are not enabled as the default, since they do make the solution more robust. Well, the reason is simple; If the namespaces are in all the links, and you change for instance the namespace of the root node, then ALL links in the map gets broken. So actually, not having the namespaces in the links make the solution more robust.

So… I hope this can help someone.

You can download the solution here:

--
eliasen

Hi all

Just as i have started developing a functoid library (found at http://eebiztalkfunctoids.codeplex.com/) I have also started developing a pipeline component library. Right now it contains three components:

• DevNull. This pipeline component is quite simple. It will "swallow" everything that comes as input. This enables performance testing of stuff without concerns about adapter transport time at send port for instance.
• SearchAndReplace. This component will perform a search and replace on the incoming stream, replacing some string with some other string. Optionally, you can decide to let the input string be a regular expression and replace based on that instead of normal string search and replace.
• Promote. This component has three parameters, the name of a property, the namespace of the property and an XPath expression. The component will read in the value that corresponds to the XPath expression at runtime and promote it to the property given by the name and name space. This enables you to promote reoccurring elements.

You can find it at http://eebiztalkpipelinecom.codeplex.com/ – the url is weird, I know. But there is a limit to the length of the urls at codeplex, unfortunately.

--
eliasen

Hi all

So, first I had a post about promoting reoccurring elements where I discussed 4 options for promoting an element that can occur multiple times in the input. Then I had a post about how you can pseudo do it in BizTalk 2000/2002.

This post is about two things:

1. Another way of doing it that doesn’t work
2. Writing the custom pipeline component that was my second suggestion in my first post.

First issue

As discussed in my first post, if you change the XPath of the promoted property in the schema (either by clicking the ellipsis as I wrote about or manually editing the XSD) to add the “[1]”, you get this compilation error:

Node "ElementWhereNumber1IsPromoted" - The promoted property field or one of its parents has Max Occurs greater than 1. Only nodes that are guaranteed to be unique can be promoted as property fields.

But, since I couldn’t get the “[1]” to work in BizTalk 2000/2002 when editing my second post I used the position function instead. So i started wondering if that might work in BizTalk 2006 R2. It doesn’t, though. It compiles just fine, and I really had my hopes up, but at runtime you get this error:

There was a failure executing the receive pipeline: "Microsoft.BizTalk.DefaultPipelines.XMLReceive, Microsoft.BizTalk.DefaultPipelines, Version=3.0.1.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35" Source: "XML disassembler" Receive Port: "ReceivePort3" URI: "C:\Projects\Blog Entries\PrommotingReoccurringElementsEditXSD\Instances\In\*.xml" Reason: Unexpected XPath format:

So i had to drop that idea again.

Second issue

I decided to write the custom pipeline component that I discussed in my first post.

The main code is easy – just three properties:

1. Name of the property
2. Namespace of the property
3. XPath

You can download it at http://eebiztalkpipelinecom.codeplex.com/ where I have just made my initial release.

--
eliasen

Hi all

I have had posts about the context accessor functoid here and here.

Just a couple of notes about the context accessor functoids (plural - because there are two functoids at codeplex):

• One of the functoids will only work when called from a map that is executed inside an orchestration.
• The other functoid will only work when called from a map in a receive port AND only if the pipeline component that ships with the functoid has been used in the receive pipeline.

As you can see, creating a map based on either of these functoids makes your map impossible to use in either an orchestration or a receive port based on which functoid you chose. So you are creating a pretty hard coupling between your map and where it should be used. This can be ok, but if other developers mess around with your solution in a year or so, they wont know that and things can start breaking up.

My self: I am a user of the functoids - I would use them instead of assigning values inside an orchestration using a message assignment shape.. but this discussion is pretty much academic and about religion

Anyway, beware the limitations!

--
eliasen

Hi all

I had a post about one of the context accessor functoids which can be seen here: http://blog.eliasen.dk/2009/04/01/TheContextAccessorFunctoidPartI.aspx

This post is about the other one – the one that can only be used in a map that is used in a receive port.

Basically, the functoid takes in three inputs:

The first is the name of the property and the second parameter is the namespace of the property schema this property belongs to. The third parameter is an optional string that is returned in case the promoted property could not be read.

This functoid only works in a map that is called in a receive port and only if the receive location uses a pipeline that uses the ContextAccessorProvider pipeline component that is included in he same DLL as the functoids.

What the pipeline component does is, that it takes the context of the incoming message and saves it in a public static member. This way, the functoid can access this static member of the pipeline component and read the promoted properties this way.

Good luck using it.

--
eliasen

Hi all

I had a post about promoting reoccurring elements in BizTalk. As we all know, this isn’t possible…. BUT… actually… well…

Basically, you promote properties for three reasons:

1. You need it to route based on the value
2. You need it for correlation (which is basically just a specialization of routing)
3. You need to either read or set the value inside an orchestration (Don’t do that! Use distinguished fields instead)

So, dealing with number 1, routing, I started thinking back at BizTalk 2002, which was my first BizTalk experience (I’l bet ALL BizTalk developers/architects think of BizTalk 2000/2002 every now and then… no? ). I seemed to recall that you could do something fancy with routing back then, so I fired up an old BizTalk 2002 on windows 2000 Professional to test it. It turns out, that on a channel you can enter a filter which is an XPath expression. The text field is editable, so you can change the XPath expression all you want – it will complain if you try to leave it with an invalid XPath expression (syntactically – not semantically).

Given this XML:

<MyRoot>
  <myReoccuringRecord Myfield="42" /> 
  <myReoccuringRecord Myfield="2" />
  <MySecondRecord MySecondField="jan" /> 
  <MyThirdRecord>
    <MyThirdField>1</MyThirdField> 
  </MyThirdRecord>
</MyRoot>

I can have a channel with this filter:

• /*[local-name()='MyRoot' and namespace-uri()='']/*[local-name()='myReoccuringRecord' and namespace-uri()=''][position()=1 and @Myfield = 42]

Basically, documents will only go through this channel, if the value of the “Myfield” attribute of the first “myReoccuringRecord” element has the value 42.

So, it isn’t promoting as such – BizTalk 2002 doesn’t have this concept, but it allows us to route based on the value of a specific occurrence  of a reoccurring element.

On a side note; If you leave the filter like this:

• /*[local-name()='MyRoot' and namespace-uri()='']/*[local-name()='myReoccuringRecord' and namespace-uri()=''][@Myfield = 42]

it will accept the incoming document no matter what the position of the “myReoccuringReord” is. Can’t make up my mind if this is a good thing or not

Now, as we all know (?), manually editing the XSD in BizTalk 2006 solution to make sure the XPath evaluation will only return a single XmlNode doesn’t work. You either get a compile time error:

• The promoted property field or one of its parents has Max Occurs greater than 1. Only nodes that are guaranteed to be unique can be promoted as property fields.

or you get a runtime error:

• There was a failure executing the receive pipeline: "Microsoft.BizTalk.DefaultPipelines.XMLReceive, Microsoft.BizTalk.DefaultPipelines, Version=3.0.1.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35" Source: "XML disassembler" Receive Port: "ReceivePort3" URI: "C:\Projects\Blog Entries\PrommotingReoccurringElementsEditXSD\Instances\In\*.xml" Reason: Unexpected XPath format:

Anyway… not that many BizTalk 2000/2002 installations till in production, I assume, so this post is merely for informational purposes. It’s just funny discovering functionality that is doable in earlier versions of BizTalk and not in the latest versions.

--
eliasen