Sunday, 14 January 2007

Hi

Just read that the 70-630 exam in Office SharePoint Server 2007 and the 70-631 exam in Windows SharePoint Services 3.0 are now available.

Microsoft haven't got the http://www.microsoft.com/learning web site quite up2date yet, but probably it wont take long.

Also, no MCTS certification is listed on the MCTS page yet. But since the two exam-pages refer to TS exam status, I suppose the certifications are on their way.

--
eliasen

Sunday, 14 January 2007 13:44:19 (Romance Standard Time, UTC+01:00)  #    Comments [0]  | 

Hi

I was trying to help someone in the BizTalk newsgroups, creating a flat file schema, and I ran into this very annoying error message: "Unable to match the data in the input stream.". I got the error message after having created the schema and trying to validate an instance, using VS.NET 2005.

I searched and searched...

  • Someone needed to set the "parser_optimization" to "complexity" - I had allready done that.
  • Someone else needed to set the "lookahead_depth to 0 - I had also tried that.
  • Someone needed to set the input type to "Native" instead of XML. Had that been my problem, I wouldn't have blogged about it - I'd be too embarassed :-)

No, the problem was that I had created parts of the schema using the flat file wizard. It had set one of my records to have "minOccurs" = 18 and "maxOccurs" = 18 as well. Since my test instance didn't have that many records in it, I got this error message. Changing the two 18's to the default 1 solved my problem.

I hope Microsoft considers this and perhaps in future releases find a better error description :-)

In order to test it, you can just download the working schema: Exolit_FlatFileSchema.xsd (91.59 KB) and the test instance: Instance.txt (.72 KB) - it will validate. But try to change the minOccurs and maxOccures of the "Invoices"-record to "18" - and revalidate.

Anyway... Hope this helps someone in the future.

--
eliasen

Sunday, 14 January 2007 13:41:33 (Romance Standard Time, UTC+01:00)  #    Comments [3]  | 
Friday, 05 January 2007

Hi

Should you ever run into this warning in the eventlog, when using the SQL Adapter for BizTalk:

your problem is that the SQL Adapter is receiving a result set instead of XML.

There are two possibilities:

  1. As we all know, when using the SQL Adapter to call a stored procedure, you must write "for xml auto, xmldata" (or some variant - I like the "elements" way of things) in your Stored Procedure in order to let the schema generator generate a schema for you. After the schema has been generated, you need to remove the ", xmldata" or else schema data will be returned every time the SP is called instead of real data. Anyway, sometimes people remove too much from the SP, ie. they remove the entire "for xml auto, xmldata" instead of just ", xmldata".
  2. Sometimes, someone forgets to write "for xml auto" after a select statement in the receive location.

Both will cause the above mentioned error.

Hope this has helped someone.

--
eliasen

Friday, 05 January 2007 23:07:27 (Romance Standard Time, UTC+01:00)  #    Comments [4]  | 
Friday, 22 December 2006

Hi

I have just played quickly with receiving flat files.

Often, people create a receive pipeline for each flat file messagetype that must be received. This is done because the flat file disassembler needs to know exactly which document schema is to be used to parse the incoming message, as opposed to the XML disassembler, which will find the schema at runtime. So instead of having several flat file disassemblers in one pipeline, often, many pipelines are created.

BUT, I'd just like to point out that with BizTalk 2006, you can change properties of the pipeline components on a per instance level. So double click on your receive location, click the ellipsis on the pipeline and go to the "DocumentSpecName" property of the flat file disassembler. Why they haven't called it "Document Schema" as it is called in the pipeline designer, I don't know!

Anyway, here you can specify which document schema this pipeline instance should use. You must specify the fully qualified .NET typename of the schema, however. And it is a textbox, not a drop down, as in the designer. You can get the .NET typename by clicking on the .xsd in solution explorer, and taking it from the "Fually Qualified Name" property.

Hope this was helpful

--
eliasen

Friday, 22 December 2006 21:47:24 (Romance Standard Time, UTC+01:00)  #    Comments [3]  | 
Sunday, 17 December 2006

Hi

When generating a schema based on the SQL Adapter, there are some options for manipulating the XML.

I always use the "for xml auto, elements" version, because I'm an "element-kind-of-guy". Others use the "for xml auto" for several reasons:

  1. It is the default
  2. Attributes create smaller XML
  3. They like attributes

None the less, I'm an "element-kind-of-guy" and I do what I want! :-)

Anyway, the schemas that are the result of this leave my schema with loads of records instead of elements, meaning that I can not promote the result to distinguished fields.

There is a way of changing this, though... allthough I really don't consider it to be a nice solution :-)

As an example, I have the following table in a database:

And I have this stored procedure to get the values:

CREATE PROCEDURE [dbo].[Get_eliasen]
AS
BEGIN
 -- SET NOCOUNT ON added to prevent extra result sets from
 -- interfering with SELECT statements.
 SET NOCOUNT ON;

    -- Insert statements for procedure here
 SELECT * from eliasen for xml auto, elements, xmldata
END

Basically, it returns all rows from the table. The schema generated from this stored procedure will look like this:

The "seqno", "field1", "field" and "field3" are records and not elements. If I want to change them to be elements instead, I need to open the XSD in a text editor and do this manually:

As an example, I will change the record "seqno" into an elements.

The original XSD looks like this:

<?xml version="1.0"?>
<xs:schema attributeFormDefault="unqualified" elementFormDefault="qualified" targetNamespace="http://eliasen.dk/SQLAdaterRecordsTest" version="1.0" xmlns:xs="http://www.w3.org/2001/XMLSchema">
  <xs:annotation>
    <xs:appinfo>
      <msbtssql:sqlScript value="exec [Get_eliasen]" xmlns:msbtssql="http://schemas.microsoft.com/BizTalk/2003" />
    </xs:appinfo>
  </xs:annotation>
  <xs:element name="SQLAdaterRecordsTestRequest">
    <xs:complexType>
      <xs:sequence>
        <xs:element minOccurs="0" maxOccurs="unbounded" name="eliasen" xmlns:q1="http://eliasen.dk/SQLAdaterRecordsTest" type="q1:eliasenType" />
      </xs:sequence>
    </xs:complexType>
  </xs:element>
  <xs:complexType name="eliasenType">
    <xs:choice minOccurs="0" maxOccurs="unbounded">
      <xs:element name="seqno" xmlns:q2="http://eliasen.dk/SQLAdaterRecordsTest" type="q2:seqnoType" />
      <xs:element name="field1" xmlns:q3="http://eliasen.dk/SQLAdaterRecordsTest" type="q3:field1Type" />
      <xs:element name="field2" xmlns:q4="http://eliasen.dk/SQLAdaterRecordsTest" type="q4:field2Type" />
      <xs:element name="field3" xmlns:q5="http://eliasen.dk/SQLAdaterRecordsTest" type="q5:field3Type" />
    </xs:choice>
  </xs:complexType>
  <xs:complexType name="seqnoType">
    <xs:simpleContent>
      <xs:extension base="xs:long" />
    </xs:simpleContent>
  </xs:complexType>
  <xs:complexType name="field1Type">
    <xs:simpleContent>
      <xs:extension base="xs:string" />
    </xs:simpleContent>
  </xs:complexType>
  <xs:complexType name="field2Type">
    <xs:simpleContent>
      <xs:extension base="xs:string" />
    </xs:simpleContent>
  </xs:complexType>
  <xs:complexType name="field3Type">
    <xs:simpleContent>
      <xs:extension base="xs:string" />
    </xs:simpleContent>
  </xs:complexType>
</xs:schema>

And I will change it into this:

<?xml version="1.0"?>
<xs:schema attributeFormDefault="unqualified" elementFormDefault="qualified" targetNamespace="http://eliasen.dk/SQLAdaterRecordsTest" version="1.0" xmlns:xs="http://www.w3.org/2001/XMLSchema">
  <xs:annotation>
    <xs:appinfo>
      <msbtssql:sqlScript value="exec [Get_eliasen]" xmlns:msbtssql="http://schemas.microsoft.com/BizTalk/2003" />
    </xs:appinfo>
  </xs:annotation>
  <xs:element name="SQLAdaterRecordsTestRequest">
    <xs:complexType>
      <xs:sequence>
        <xs:element minOccurs="0" maxOccurs="unbounded" name="eliasen" xmlns:q1="http://eliasen.dk/SQLAdaterRecordsTest" type="q1:eliasenType" />
      </xs:sequence>
    </xs:complexType>
  </xs:element>
  <xs:complexType name="eliasenType">
    <xs:choice minOccurs="0" maxOccurs="unbounded">
      <xs:element name="seqno" type="xs:long" />
      <xs:element name="field1" xmlns:q3="http://eliasen.dk/SQLAdaterRecordsTest" type="q3:field1Type" />
      <xs:element name="field2" xmlns:q4="http://eliasen.dk/SQLAdaterRecordsTest" type="q4:field2Type" />
      <xs:element name="field3" xmlns:q5="http://eliasen.dk/SQLAdaterRecordsTest" type="q5:field3Type" />
    </xs:choice>
  </xs:complexType>
  <xs:complexType name="field1Type">
    <xs:simpleContent>
      <xs:extension base="xs:string" />
    </xs:simpleContent>
  </xs:complexType>
  <xs:complexType name="field2Type">
    <xs:simpleContent>
      <xs:extension base="xs:string" />
    </xs:simpleContent>
  </xs:complexType>
  <xs:complexType name="field3Type">
    <xs:simpleContent>
      <xs:extension base="xs:string" />
    </xs:simpleContent>
  </xs:complexType>
</xs:schema>

So what have I done?

  1. I Found the definition of the record "seqno". It was based on the type "q2:seqnoType".
  2. I found where the "seqnoType" type was declared, and noted that it extended "xs:long".
  3. I changed the type of "seqno" from "q2:seqnoType" to be "xs:long".
  4. I removed the q2 namespace declaration from the seqno element, as it was no longer needed.
  5. I removed the "seqnoType" type declaration, as it was no longer needed.

That's it. Save the XSD and reload it in your schema editor - you will get this:

But it sure would be nice to have the option of just changing it in the properties of an elements in the schema editor...

Hope this helps someone.

EDIT on the 4'th of march 2007: Remember also that the record just below the root node must be set as "maxOccurs=1" instead of unbounded. Otherwise, you can not promote the fields below the record, obviously.

--
eliasen

Sunday, 17 December 2006 23:42:35 (Romance Standard Time, UTC+01:00)  #    Comments [4]  | 
Saturday, 09 December 2006

Hi

I ran into this issue today. It turned out that the error occured in an expression shape that used xpath to get a value from a message into a variable.

Having been using xpath expressions inside my orchestrations for a long time, I am really not used to having them fail on me :-) And this particular xpath expression was copied from the "Instance XPath" property of the element in the schema editor. So it shouldn't fail. I did add a "[1]" to the xpath because I needed to access the first element of a re-occuring element, but still - not something I hadn't done plenty of times before.

Actually, it took quite some time before I discovered what was wrong. My expression was:

DeliveryDate = xpath(OrderMessage, "/*[local-name..........");

and I needed to change it to

DeliveryDate = xpath(OrderMessage, "string(/*[local-name..........)");

Again, something I have done a million times... this time I just forgot it. And honestly, I really don't think the error message gives a very good idea of what is wrong...

Hope this gets indexed and someone later on will find it helpful :-)

--
eliasen

Saturday, 09 December 2006 23:24:28 (Romance Standard Time, UTC+01:00)  #    Comments [13]  | 

Hi

On my current project, I ran into a small issue. In my orchestration, I created a record in a table in a database. The stored procedure that created the record also returned the ID of the record. This ID was to be emailed to a trading partner in the body of an html email. The trading partner could reply, and I needed to correlate on the ID of the record in the database.

But how to do that? The body of the email is one long rawstring with the ID embedded in the HTML.

Off course, there was the obvious way; Just use a property that is never used in some property schema. I happen to know that my solution will never actually use the POP3.Subject property, and therefore I can use it for my purposes. So when I create the Message to be sent out by email to the trading partner, I put the ID inside the POP3.Subject property. When the message is sent, I initialize a correlation set that is based on this single property. I have then also promoted some element of the reply from the trading partner to the same property, and this works just great.

BUT, I am not a fan of misusing properties like that. I find it a bit ugly. So I looked for alternatives, and then I found the /dev/null adapter written by BizTalk MVP Tomas Restrepo. Baiscally, the adapter discards anything it receives. Tomas writes that it can be used to subscribe to anything in order to avoid getting the "No subscribers found" errors, but I found another use for it.

So I created a dummy schema that only had one element in it and promoted that element to a property of my own property schema. I also promoted the appropriate element from the reply from the trading partner to the same property. In my orchestration, I then created a correlation set based on this property, and I added a send port that sends out an instance of the dummy schema just after sending out the email to a port using the /dev/null adapter. The message doesn't go anywhere, but the correlation set gets initialized and I can follow it later on in my orchestration.

See my solution at CorrelationAfterSMTPSend.zip (94,37 KB)

Remember that you must download and install the /dev/null adapter before trying this.

And see the solution without using Tomas' adapter here: CorrelationAfterSMTPSendWithoutDevNull.zip (78,18 KB)

Pros and cons... Well, the first obvious solution is much simpler, as it doesn't require a dummy schema, it doesn't require a property schema and the orchestration doesn't need to create and instance of the dummy schema and send it out. On the other hand, it really isn't pretty to use the properties like that.

The second solution requires installing an adapter, registering it, and all the legwork described above. But the solution uses my own properties and the POP3.Subject property can be use later on, should it be necessary.

Oh well, use whichever you like best :-)

Hope this was helpful to someone.

 

Updated on the 10'th December 2006:

Well, had I taken the time to think it over again before writing the blog entry, I would have realized that my first obvious approach, which is using the POP3.Subject property doesn't actually have to use an existing property in an existing property schema. I could just as easily have created my own property schema and used that instead.

The reason I abandoned this approach was, that after creating the property schema and creating a property inside the property schema, to which I promoted fields in the schemas, I couldn't assign values to it in message assignment shapes. The property just didn't show up using intellisense in the message assignment shapes.

So I went ahead and used the POP3-Subject property instead. BUT, I discovered that I can actually assign values to my own properties. I just need to change a property of the property element in the property schema. The property element needs to be marked as "MessageContextPropertyBase" in the "Property Schema Base" property of the property element in the property schema.

So, I have abandoned the /dev/null adapter, although I liked the idea :-) And am now using my own property. The best of the two worlds! :-)

--
eliasen

Saturday, 09 December 2006 23:17:36 (Romance Standard Time, UTC+01:00)  #    Comments [0]  | 
Tuesday, 05 December 2006

Hi

Updated at the bottom with a solution.

I have this very weird problem. I have created a functoid, but silly, as I am, I created an icon for it that was way to big. So I wanted to remove the functoid from the toolbox and deploy another functoid instead.

This turned out to be quite a difficult task. I have removed the functoid from GAC and I have removed it from the "Mapper Extensions" folder of the BizTalk installation folder. I have then reset the toolbox. The functoid is STILL there. Actually, I can do whatever I want, even boot the server, but the functoid stays in the toolbox. If I move my mouse over it in the toolbox, VS.NET 2005 will crash very hard, so this isn't desirable :-) If I am very quick, I can go to the functoid, right click on it and delete it from the toolbox. Yiepee, it is gone. BUT, resetting the toolbox makes the functoid reappear :-(

Another guy has the same problem, and at this moment, a support incident with Microsoft is being evaluated. Microsoft have reproduced the behaviour, they claim, so I am awaiting the answer to this peculiar problem any time now.

I will post the solution here, off course.

Update with a solution posted on 9'th December 2006:

Solution 1:

Log on to Windows, using a username other than that, under which this error occurs. The ser must have administrative priviledges.

Right click on "My Computer", Select Properties, Advanced Tab, User Profiles -> Settings and delete the profile that is experiencing the problem.

Logout, and login as the user that had the problem. The problem should now be gone.

This solution has a great drawback, though - the profile is deleted (duh!) This means that alll your settings for all kinds of programs are removed and you need to recreate them. For instance, VS.NET 2005 thinks it is the first time it is started the next time you start it.

Solution 2:

Microsoft stated another solution that should work, but this one I haven't tested:

  1. Ungac the Assembly
  2. Remove the Assembly from Mapper Extension Folder.
  3. Choose Items on the Toolbox and Unselect the offending items
  4. Then do a Reset Toolbox

--
eliasen

Tuesday, 05 December 2006 21:59:06 (Romance Standard Time, UTC+01:00)  #    Comments [3]  | 

Hi

A guy on the microsoft.public.biztalk.something newsgroup is having problems splitting an incoming flat file into several files. The dataformat he has is this:

HH06SGLN00084CR
31102006~06~Miss~ABD~DEF GHI ~ZZZZZ~F~31111111~~BD46A~233435~NUTRITION & FOOD SCIENCE~1~88888888~ AN ADDRESS SOMEWHERE~AB18ZZ~AAAA33333333T~04444.00~01200.00~00000.00~00000.00~F~Y~P~Y~N~N~N~
01112006~06~Miss~ASODIFN~DSFJ ~BSODIFJSDF~F~55555555~~Q190E~444444~ENGLISH & POPULAR CULTURE~1~66666666~ ANOTHER ADDRESS ~CF51NT~EEEE94857463H~01200.00~01200.00~00000.00~00000.00~F~Y~F~Y~Y~N~N~
TT000024

The first line is a header, with identifier "HH". The last line is the trailer with identifier "TT". In between are many body lines, one per line.

He would like to create schemas for the header, the body and the trailer and use them in a custom receive pipeline, using the flat file disassembler to split it up, so he gets one XML document per body line.

Unfortunately, this doesn't seem to be doable. What seems to happen is that when parsing the document, first the header schema is used. It describes a single line, and this line in the incoming file is then parsed. Then, the body schema is used for parsing the rest of the incoming document. Since the body lines don't have tag identifiers, though, it seems that BizTalk will continue to parse the document, and this includes parsing the last line, which is the trailer. BizTalk doesn't know when to stop parsing for body lines. Therefore, this error appears in the eventlog:

-- BEGIN ERROR
Source: "Flat file disassembler" Receive Port: "ReceiveFlatFile" URI: "C:\Projects\BTS 2006\NewsgroupHelp\BodyWithoutTagIdentifierFlatFile\Instances\In\*.txt" Reason: Unexpected data found while looking for:
'~'
The current definition being parsed is BodyRoot. The stream offset where the error occured is 404. The line number where the error occured is 4. The column where the error occured is 0.
-- END ERROR

Baiscally, the flat file disassembler can't find the ~ character on the last line, which isn't supposed to be there, since this line is the trailer. So BizTalk gives up and fails.

What I have come up with isn't actually pretty, but it does seem to work :-)

I have created a schema for the entire flat file. And I have created a schema for the entire flat file without the trailer. Then, I created a schema for the header, and a schema for the body. I have made heavily use of the flat file schema wizard, because there are many elements in the body lines :-)

Then, I created a map between the two main schemas, effectively removing the trailer from the input.

I also created three custom pipelines:

  1. A pipeline for receiving the complete flat file
  2. A pipeline for sending out the flat file without its trailer
  3. A pipeline for splitting the flat file without trailer into several body documents, using the header- and body schemas.

So the solution is:

Let BizTalk read in the complete flat file, and use a map on the receive (or send) port to convert it into the same structure without the trailer. Then output it to a file. Let another receive location pick the new file up, and use the pipeline with body- and header schemas to split it up into several documents.

Pitfalls are: Remember to use different combinations of rootnode and target namespaces for each schema. After copying a schema it is easy to forget to change it. Also, change the .NET typename of the schema after copying it. The compiler will remind you of that if you forget it, though :-)

I really wanted to not use a flat file for the intermediate step and use XML instead, but I couldn't get it to work. I would have to have a schema for the output and another schema for the input, which was an envelope with an "Any"-element inside it, and these two schemas would need to have the same rootnode and target namespace. So I dropped it, and stayed with the flat file, allthough I hated it :-)

My .NET project can be found here: BodyWithoutTagIdentifierFlatFile.zip (99,28 KB)

I hope this has been of some help. The conclusion basically is that it can't be done, splitting an incoming flat file up using header, body and trailer schemas, if the body doesn't have a tag identifier.

UPDATE on 5'th December 2006: Greg Forsythe has another solution to the problem, which he has written in the newsgroup. I quote:

-- BEGIN QUOTE

There is another way of doing this:
Create a document schema with an optional trailer record.
This will debatch each record, with the last document having a trailer
record. This can be removed/ignored in the first map

--  END QUOTE

I haven't tried it, but it makes sense, and why didn't I think of that?

--
eliasen

Tuesday, 05 December 2006 00:14:35 (Romance Standard Time, UTC+01:00)  #    Comments [0]  | 
Sunday, 05 November 2006

Hi

A guy on the microsoft.public.biztalk.orchestration newsgroup has asked about looping around elements inside an orchestration, and I thought I'd just write some lines about the issue here, for everybody to see in the future.

The problem is, that he has the following document structure:

<Employees>
<Employee title="mgr">
</Employee>
<Employee title="vp">
</Employee>
<Employee title="ceo">
</Employee>
</Employees>

And he wants to do something with each employee, based on what the title is. As I see it, there are two options:

  1. Loop through the Employee-elements inside the orchestration
  2. Use an envelope to split the incoming Employees-message into several Employee-messages

Suggestion 1

I have proposed the following two schemas:

 for the big message that arrives and  for the individual Employee. Note, that I have promoted "title" as a distinguished field.

I then create an orchestration, that takes an instance of the Employees schema as input, and loop around the employees. The orchestration looks like this:

It receives the incoming message, and then initializes a couple of variables. The first expression shape, named "Initialize Loop variables" contains the following three lines:

empCount = xpath(InputMessage, "count(/*[local-name()='Employees' and namespace-uri()='']/*[local-name()='Employee' and namespace-uri()=''])");
counter = 1;
counterStr = "1";

Basically, I get the number of employee-elements, and initialize the counter. The stringcounter is used to build xpath expression later, as you will see.

Then, I loop. The loop condition is "counter <= empCount" - so I want to loop as long as there are employees.

Inside the scope, I have declared a message employeeMessage, which is of the type of a single employee.

In the message assignment shape, I do this:

EmployeeMessage = xpath(InputMessage, "/*[local-name()='Employees' and namespace-uri()='']/*[local-name()='Employee' and namespace-uri()=''][" + counterStr + "]");

It takes the employee from the big incoming message that corresponds to the counter and assigns it to the message that is to be constructed.

In the next expression shape, I just write the content of the distinguished field "title" of the employee to the eventlog.

And in the final expression shape, I increment the counter variables:

counter = counter + 1;
counterStr = System.Convert.ToString(counter);

So by doing this, I am looping over the employees inside the employees-message, and I have access to all the values inside the single employee. I didn't have to have schema number 2, describing a single employee, I could have just used xpath all the way down, or maybe declare an XmlNode variable to hold the employee-element instead. But I like this solution better.

Another option I have now is that I can add a decision shape, and based on the title value, I can call different orchestrations, that will handle a specific employee-type. Or perhaps I could just send the employee message to a direct-bound send port and have other orchestrations subscribe to the employee message type. This would require the title to be a promoted property instead of a distinguished field, though, in order to route on it.

Suggestion 2

I propose using an XML Envelope to split the incoming file into several employee-messages and let the orchestration handle them individually.

To do this, I have created two schemas:

 for the envelope and  for the employee. Note: I have changed the names of the root nodes in both schemas. Naturally, in real life you wouldn't do this. The only reason I did this is to be able to have both my examples deployed at the same time. If I hadn't done it, I would have had multiple schemas deployed with the same combination of target namespace and root node, which we all know is BAD. On the schema for the envelope, I have clicked on the "<Schema>"-node and in the properties windows, I have set "Envelope" to "Yes". Then, I clicked on the "EmployeesEnvelope"-node and in the properties window, I set the "Body XPath" property to point at the "EmployeesEnvelope"-element.

Then, I created an orchestration, that takes an employee as the input - not the employees-type, but the single employee. It looks like this:

So here I have a much smaller, simpler, and faster orchestration than the one from suggestion 1.

After the solution is deployed, I create a receive location, and remember to use the default XMLReceive pipeline. The disassembler stage will look at the incoming message, determine the message type, see it is an envelope, split the message into smaller messages, and publish them individually with their own message type.

If I need different orchestrations depending on the value of the title attribute, I can promote it to a promoted property instead of a distinguished field, and add it to a filter on the receive shape in each orchestration.

Examples:

I have my two samples here: LoopAroundEmployee.zip (68,66 KB) and here: LoopAroundEmployeeEnvelope.zip (48,64 KB)

I hope this has been useful for someone. If you have any questions, just ask.

--

eliasen

Sunday, 05 November 2006 14:04:11 (Romance Standard Time, UTC+01:00)  #    Comments [13]  | 

Theme design by Jelle Druyts