Envelopes, maps and pipelines

I was in the process of posting to one of the newsgroups this afternoon when I saw that someone else had beaten me to it, and posted on the exact same topic, Envelope schemas and mapping.

It's a fairly common scenario - a message needs to be split, with common 'header' information being included in each message. I had thought that it would be possible to do this in a pipeline, and I've found various posts referring to this situation (see end of post for references), none of which really answered the question.

To demonstrate this in action I'll use the classic customer / order example. I have a starting message which contains header and body information, and an orchestration that uses the individual order messages, with the customer data being inserted into each message, as shown below:

Original message:
<CustomerOrdersResponse>
 <Customer>...</Customer>
 <Orders>
  <Order>...</Order>
  <Order>...</Order>
 </Orders>
</CustomerOrdersResponse>

Messages as required by Orchestration:
<CustomerOrder>
 <Customer>...</Customer>
 <Order>...</Order>
</CustomerOrder>

I have defined three schemas to accomodate this - Customer, Order and CustomerOrder:
Customer:
<Customer>
 <Firstname>John</Firstname>
 <Lastname>Doe</Lastname>
 <Age>30</Age>
</Customer>

Order:
<Order>
 <Item>1</Item>
 <Qty>2</Qty>
</Order>

CustomerOrder:
<CustomerOrder>
 <Customer>...</Customer>
 <Order>...</Order>
</CustomerOrder>

The first thing to do try is a simple splitting of the messages using an envelope schema. Create a new schema, set its Envelope property to true, and the Body XPath of the root node to the xpath that points to the root of the collection - in this case to the <Orders/> node. The sample xml message below shows the relationship between envelope and body parts:
Envelope
<CustomerOrdersResponse> (Set Body XPath property of this node to xpath of Orders node below)
 <Customer>...</Customer>
- - - - - - - - - - - - - - - - -
Body
 <Orders>
  <Order>...</Order>
  <Order>...</Order>
 </Orders>
- - - - - - - - - - - - - - - - -
</CustomerOrdersResponse>

When a message that conforms to an envelope is processed in an (XML) pipeline, the envelope is discarded, and the repeating body records are split out into separate messages. The message above would be converted into two <Order/> messages. So far, so simple.

The problem now is how to get a CustomerOrder message out of the above, rather than the Order alone.

My first idea was to use a map to convert the original message to a collection of CustomerOrder messages, and redefine the envelope schema to use the new message:
<CustomerOrders>
 <CustomerOrder>
  <Customer>...</Customer>
  <Order>...</Order>
 </CustomerOrder>
 <CustomerOrder>
  <Customer>...</Customer>
  <Order>...</Order>
 </CustomerOrder>
<CustomerOrders>

I figured that if I put this map into a send pipeline, then I could send the original CustomerOrderResponse message out through this pipeline, converting it into a CustomerOrders message, which would then be picked up by a receive location which would extract the individual CustomerOrder messages.

Which is where it all goes wrong.

If you try and do this, you'll get an exception thrown by the send pipeline saying that "Document type "CustomerOrder" does not match any of the given schemas." I looked around, and found that someone else had had a similar issue, and concluded that mapping to an Envelope schema is not possible within a pipeline. Which brings me back to where I started, as this is exactly the issue that Duncan Millard and I collided on whilst posting to microsoft.public.biztalk.general.

You can map to Envelope schemas within an orchestration, and it turns out that this is how we are getting around the problem - the original CustomerOrdersResponse message is mapped to CustomerOrders within an orchestration, sent out through a send pipeline (without further mapping), then picked up by a receive location that splits out the CustomerOrder messages.

There is an alternative method - using XPath within a loop / expression combination. This may well be better in certain situations, but it seems a shame not to use the inherent envelope functionality. I believe it may also be possible using some custom pipeline magic, and Stephen Thomas' blog posting (see below) has some interesting stuff on property promotion / demotion tricks, but none of it seems very simple.

Does anyone know of an easier way to do all of this in one pass?

References:

How to split an XML message in BizTalk 2004 using Document & Envelope Schemas (Jan Tielens)

BizTalk Server will split up your documents for you. (Scott Woodgate)

Property Promotion and Demotion in BizTalk 2004 (Stephen Thomas)

Looping around Message Elements (Darren Jefford)