Wednesday, December 18, 2013

Consuming EBS-EDT SOAP service from WCF

@YaronNaveh

If you want personal guidance with EBS-EDT feel free to mail me at yaronn01@gmail.com

A while ago the Ontario Ministry of Health and Long-Term Care published this document, which explains how to consume their new SOAP web service. (In favor of Google the exact title is "Technical Specification for Medical Claims Electronic Data Transfer (MCEDT) Service via Electronic Business Services (EBS) Ministry of Health and Long-Term Care"). I have received over a dozen of questions about how to consume this service with WCF. Unfortunately it is not a simple task since the service uses a complex configuration which is not available in any of the built-in WCF bindings. However it is possible to do it with some custom code. Bellow I describe the general scheme for this to work. I know some community members are preparing a simple wrapper for this so I will publish it here once ready.


The Errors
Depending on which path you chose for implementation, the most common error message you are likely to receive is the dreadful:

The incoming message was signed with a token which was different from what used to encrypt the body. This was not expected.

There are other possible errors as well or some consumers may not know where to start.

The Solution
1. Since the client needs to send both username token and an X.509 certificate (and sign with the latter) we need to write a code binding:


One thing you want to notice in this code is that it contains the username and password, so change them according to your credentails.
Another thing to notice is that the client certificate is loaded from disk. You could change that to the windows certificate store if you wish. As for the server certificate, you could put any dummy certificate there, including the same one as the client certificate (it will not be used but WCF needs something in this setting).
Also note the EnableUnsecuredResponse=true. It is a key for the next steps.

2. Since the request needs to be signed only (not encrypted) let's configure the contract in reference.cs with the ProtectionLevel attribute:


3. WCF is reluctant to decrypt the response. For this reason we need to do the decryption manually. This is the hardest part but I give most of the code here so hopefully it will be easier. You need to implement a custom message encoder and configure the binding above to use your encoder instead of text message encoder. Read here on how to implement an encoder.

4. You need to override the ReadMessage method of the encoder and decrypt the response message in it.

This code shows how to decrypt a message (not necessarily in the context of an encoder):


This code needs access to your private key so it could extract the session key in the message and it also needs some elements from the response. Once you get the decypted message you can replace the encypted body part in the message provided by the encoder with the decrypted message.

5. The last mission to accomplish in the encoder is to delete the <security> element (and all of its child nodes) from the response message before you return it to WCF. Otherwise WCF will try to decrypt the message which is redundant since we just unencrypted it now (WCF decryption would fail anyway). Remember the EnableUnsecuredResponse flag from step #2? It tells WCF not to expect any security, so stripping the elements out is safe.

Information on some possible errors in this process is available here.

MIME Attachments

Hopefully by now you have a working client. Some of the operations also receive an attachment from the service. This attachment in SwA (Soap with Attachments) which is a MIME format a little different than the MTOM whcih WCF knows about.To extract this attachment you could use some kind of a mime parser library as the first step of your encoder (apply it over the raw bytes from the network). Copy the first MIME part to the Message object (this is the SOAP). The second part will be the attachment which you can keep on the custom encoder as a property or on some other context available to your application code.

Fault Contract
Since there is no formal fault contract in the WSDL you should inspect any incoming soap fault using a custom message inspector.

To sum up, consuming EBS-EDT from WCF is not easy but doable, good luck!

@YaronNaveh

What's next? get this blog rss updates or register for mail updates!

20 comments:

Milton said...

Thank you very much Yaron. I was almost ready to desist.

Cheers from Bolivia

John said...

Your approach with the CustomTextMessageEncoder works perfectly (thank-you!) to get a response and decrypt the body inside ReadMessage. Any advice on how to alter the body with the decrypted message? I've replaced the encrypted body with the decrypted value (base64 encoded) and removed the security element, but proxy call always returns a null value (no exception).

Yaron Naveh (MVP) said...

Hi John

Maybe you have overriden the immediate node bellow the body element. You should keep that node and only change its childs. Log the message your encoder returns and compare it to a sample decypted message or to the schema. If you are not sure how to do it drop me a mail yaronn01@gmail.com

Yaron

John said...

You are correct, the decrypted text starts with element "return" which needs to appear as below (and no need to base64 encode). The "getTypeList" call now works with this XML hierarchy returned from ReadMessage:

Envelope
..Header (now empty)
..Body
....getTypeListResponse
......return
........data
........etc

WCF now returns a proper typeListResult object. Thanks again!

Лисовский Виктор said...

Thanks for the informative article. But I can not upload files over 16K. OHIP says "You are sending inline content and you should be sending MTOM attachments". I do so:
public override ArraySegment WriteMessage (Message message, int maxMessageSize, BufferManager bufferManager, int messageOffset)
{
var encoder = (new MtomMessageEncodingBindingElement(MessageVersion.Soap11, Encoding.UTF8)).CreateMessageEncoderFactory().Encoder;
return encoder.WriteMessage (message, maxMessageSize, bufferManager, messageOffset);
}
But it does not work.
Could you give me any clue?

Yaron Naveh (MVP) said...

Hi Лисовский Виктор

I am not sure it is possible to do it with the MTOM encoding element (though I heard of someone that claimed to do it). One alternative way is to implement the attachment mechanism yourself with a MIME library. Anyway first you need to see with Fiddler how your request looks like (full HTTP request, with all parts). If you want you can mail me it and I will tell you what you should do differently yaronn01@gmail.com

Thanks,
Yaron

John said...

I can verify Yaron's strategy of doing the MIME encoding without the help of WCF. I have got it to work by removing the SOAP body "content" value and replacing it with an XOP Include element which references the MIME attachments. Bit of work, but doable.

The upload seems to work without encrypting the files. I'm going to start turning encryption on as I don't think they'll allow that in production (?) For the response, we're decrypting this ourselves (both the SOAP body and the file attachments).

Yaron, for the request, any idea whether we should be doing our own encrypting (e.g. upload attachments), or can WCF finally start pulling it's weight :-)

Yaron Naveh (MVP) said...

Hi John

Based on the sample SOAPs that the service authors have provided I don't see a need to encrypt the request attachments. You should be able to pass compliance without it and no reason to change anything after. The request is secured using SSL.

Kori Francis said...

Don't forget everyone that I've released a wrapper for the EBS/MCEDT service here: https://ebsconnect.io

I've passed conformance for all services, so if you're getting tired of fighting with the service - please get in contact with us.

Nick Varpness said...

Thank you very much. After reading this I am going to open a POC that has gotten the better of me for too long. This has been an extremely frustrating issue. I hope it works for me.

Yaron Naveh (MVP) said...

Hi Nick

Feel free to contact me directly if you need more tips: yaronn01@gmail.com

Yaron

James said...

Hi Yaron,

I'm working on developing the interface for the new Ontario EBS. Using your site I have managed to get all the calls working, except for the Upload when I have large files. As mentioned by someone else if the files are under a certain size they can be directly embedded and work. I am working with the SoapWithAttachments encoder sample project to create a MIME message to upload the file. I am getting internal errors as a response from the EBS although my MIME format seems ok.

It may be a lot to ask, but if you find yourself with some free time, my email is jbrierley@opto.com. I can send you code samples and the files I generate if you want the challenge :)

Thanks

James said...

Hi Yaron,
I have managed to get the interface almost completely operational with the help of your site. I am hitting the same bottleneck as a previous posted mentioned, being when Upload files exceed a certain size. I am using the CodePlex SoapWithAttachments project as a guide (http://wcfswaencoder.codeplex.com/SourceControl/latest#Microsoft.Austria.WcfHelpers.SoapWithAttachments/SwaEncoder.cs), and seem to get a MIME message constructed, replacing the content with an Xop reference, but am getting an internal error from the EBS.

If you have some free time, my email is jbrierley@opto.com. I can send you code extracts and the xml stream I am generating before the encryption is added on.

Thanks in advance,
James

Eugene said...

Hi James,

I find myself in the same boat. I got everything working except uploading large files. Did you ever figure out this problem? If you did would you mind sharing the answer?

Thanks a bunch,
Eugene.

Yaron Naveh (MVP) said...

Eugene - make sure your content-type header is similar to the sample in the documentation. Feel free to send me a mail if you still have issues.

Nachiket Suryawanshi said...

Hi Yaron,
Thank you so much for this blog article. I was in the dark with ministry's service before reading this article.

I am facing trouble uploading the documents to ministry using uploadMethod.
I get this error message "https://204.41.14.78:1441/EDTService/EDTService: cvc-particle 2.1: in element upload of type {http://edt.health.ontario.ca/}uploadData, found (in default namespace), but next item should be resourceType"

Can you please help ?

Thanks,
Nachiket

Yaron Naveh (MVP) said...

Hi Nachiket

You should compare the XML your client sends to the sample xml provided by the ministry. If you are unsure what exactly to do feel free to send me a mail anytime yaronn01@gmail.com

Nachiket Suryawanshi said...

Hey Yaron,
Thank You so much for your reply.
I can see the problem now. I was missing the resourceType field in the uploadData object.

Thanks again,
Nachiket

Nachiket Suryawanshi said...

Wow, its been a month, (looking at my previous comment) .

I was able to get the client almost working. Downloads with attachments are working fine. But haven't found a solution for the Uploads of bigger files.

Yaron, James, Eugene could you please share what needs to be done to get the bigger files uploaded to EBS.
Please give me some pointers.

-Nachiket

Yaron Naveh (MVP) said...

Hi Nachiket

Large attachments must use SwA / MTOM and not plain text encoding. However I do not recommend to use the WCF MTOM encoder directly.

The best approach IMO is to implement the attachments by yourself. You can use a library to serialize the MIME (a good one is http://wcfswaencoder.codeplex.com/). So the general scheme would be:

1. Configure WCF to use your custom encoder
2. in WriteMessage you serizlize the WCF Message object to string and then load it to some XmlDocument
3. using xpath you loop over all the "content" xml elements, convert them from base64 to binary and:
3.1 using the above library add them as attachment to the output message
3.2 in the soap remove the base64 element and replace it with an xop:include element in the MTOM format with the id of the attachment you gave to the MIME library
4. in the end configure the MIME library to use the altered SOAP as the main part

While doing this be careful not to change the SOAP too much (esepcially white spaces) since it is already signed. Consider to use PreserveWhitespace = true when you work with the .Net XML objects.

Just maybe, instead of all this you could inline the certificate after the MTOM encoder does its work, but I always recommend to have as much control as possible over the process.

Good luck!