It is quite common to have a few service environments, for example one for testing and one for production. One way to switch a Wcf client from one environment to another is by changing the address in app.config:
However sometime we need to dynamically change the address from code due to some logic.
The naive approach would be to do something like this:
The reason this is naive is that app.config may contain additional information on the endpoint, namely its identity and headers:
when we create the proxy with a different endpoint in the constructor we override the identity information. This may result in this error:
The solution
Create the proxy normally. Then separately assign the new endpoint address keeping the identity and header values:
A few years ago a white paper on web services interoperability would start with "turn off secure conversation" or "avoid the following types". A recent set of white papers published by Microsoft shows the today all vendors are much more mature and ready for real world interoperability.
Abu Obeida and Mike Champion from Microsoft announced recently the below articles. I recommend this reading for those of you who need to write interoperable web services. Of course not each article is relevant for each interop scenario so guess what? I've read them for you so you can decide which ones are relevant for you.
Data types interoperability between .NET and Java Christian Weyer and Buddhike de Silva have listed most types you might ever want to use with web services and verified their interoperability status. This is a must read for every public facing web service developer.
One thing that I am missing here though is the Axis2 java stack which is very popular. Also I'm confused by the check on WebSphere - as an application server you can use various stacks on it so I'm not sure which one was used. I assume Wcf was used on the .Net side (".Net 4" is specified) and I'm very curious if the data contract was able to cope with all scenarios or was xml serialization required in some cases. Generally data contract are optimized to enforce "best practice" which is sometimes counterproductive for interoperability.
The conclusion is very promising - good interoperability exists in almost all scenarios. One important reminder is the lack of interoperability with Hashtable and some of the generic types.
Jesus Rodriguez wrote a great article on ws-security interop between Wcf and Oracle WebLogic. Naturally the article dives into the ws-security details and is very technical. It proves that X.509 message security is a good option for interoperability. While jesus shows how to achieve federation interoperability via SAML and WS-Trust, it is clear that there is still a way to go here as a holder-of-key scenario is still not possible. I have not tested it myself but hopefully bearer tokens should also work well with these parties. The article does not mention it as it deals with message security while bearer tokens are a mix of message and transport.
Jesus adds another good one and this time on interoperability between Wcf and WebSphere. The username and x.509 scenarios work pretty well. The article mentions that WS-Trust interoperability is possible, but does not give the details, so I'm not sure if it is a mistake in the upload or if it really required some tweaks which were out of scope. Either way this is yet another proof that there is still a way to go with federation interoperability. Such security scheme will be very popular in the cloud era so hopefully all vendors will keep investing in it. As it stands now Wcf+Wif have great support for the related scenarios so I'll report on any news from the other vendors.
As a side note, While MTOM interoperability is possible here, I have seen in the past a few gotchas so hopefully a future paper will cover that up (I'll try to write about it myself actually).
Standards-Based Interoperability between SAP NetWeaver and Microsoft .NET Framework
A rich set of scenarios are working in this case, including security and MTOM. It is a little disappointing that a basic scenario like custom headers only partially pass, but NetWeaver is an opinionated framework that expose its users only what is relevant for them.
Metro to WCF Interoperability
The combination of Metro and Wcf shows off the richest set of working interoperability scenarios. This is not surprising considering the fact that one of the goals of WSIT (Metro's security stack) is interoperability with Wcf.
To conclude, all articles are very interesting and show that interoperability is a reality.
Yesterday Microsoft had announced Silverlight 5 Beta to be released in the first half of 2011. Sure, there are plenty of great features there, but if you are readers of my blog I'm sure you are especially interested in a particular one. Here is a hint (click to enlarge):
Yes, more of the WS-* stuff is making its way into SL.
While this is all good news, the announcement is missing more details as for what exactly will be supported. This post will analyze the status of SL & WS-Trust in SL 4 + following unofficial projects, and based on the missing gaps I will try to estimate what the upcoming release *might* include.
Background - Federation and WS-Trust
This post is not aimed to be an introduction to federation or to claims based programming, but as a quick reminder here is the general flow:
Our environment contains:
Application Service. This service contains some operations that clients want to consume. The service will only accept client requests that embed a token gotten from the STS (usually a Saml token, which is an Xml token format). One reason might be that the service does not want to deal with authentication and leaves this mission to the STS which is developed by a dedicated team.
STS. STS role is to authenticate clients and generate tokens that the client can present to the service. This is a little oversimplified, mainly since there are some STS that only aim to provide client identity, and some that provide protection over the resource (service), and in many cases both of them will be used.
Client. The client is any application (client or server) that wants to consume the application service and needs to get a token from the STS first. The client is of course not necessarily using Silverlight.
Needless to say is that Wcf is a great framework for such scenarios.
How Silverlight fits this picture?
A lot of organizations already have security infrastructure that contains STS. They would like SL applications to work with it also, mainly so SL applications can call other web services which expects a Saml identity.
What is the current status of SL & WS-Trust (SL4+ unofficial projects)
Until the beginning of 2010 there was no built-in support for such scenarios, and some creative solutions were used. Earlier this year Caleb Baker had disclosed a project that brings WIF support to SL as part of the identity developer training kit. WIF is (among other stuff) a set of extensions to WCF that makes working with claims easier. Since Saml is very aligned with the claims model, and STS usualy emits Saml over WS-Trust protocol, this effectively brought WS-Trust support for SL for the first time.
What *might* SL5 support?
In other words we need to ask what is missing after the mentioned WIF support.
As far as I know the mentioned support was not official but a part of a training kit. Many organization will not use it until it is official.
WIF generally work on XP only. I say "generally" because the WIF/SL seems to be standalone so I'm not sure if that's the case with it also. Anyway since SL is not windows-only we will probably get this subset of WIF on all platforms.
WIF works well, but it is a different programming model than the Wcf WSFederation binding. Possibly Microsoft will consider supporting the native Wcf model as well.
STS is not limited to bearer tokens as the current WIF implementation is. This is actually an SL limitation since it only supports a subset of the Wcf bindings and most of the X.509 certificate settings are not supported. Microsoft might consider to support more bindings which might be used to communicate with the STS / application service.
The current training kit does not natively support multi leg federation (e.g. more than one STS), although dominick had published the a way to fix it.
Again, these are just my guesses / wish-list, but either way this is great news for every silverlight developer.
Last week was an important moment for web services interoperability: WS-I has approved the final versions of Basic Profile (BP) 1.0 and 2.0 and of Reliable Secure Profile (RSP) 1.0. These profiles are a set of rules which web services vendors and consumers are expected to comply with in order to support interoperability. The original profile goes back to 2004 and was highly successful in improving web services interoperability.
So how do we test our web services for WSI profiles compliance?
The WSI site contains a deliverables section which has a few command line utilities to check the services. Using these command lines directly has some overhead like a learning curve for all the options, preparing the input wsdl / soap files in the specific log format, and repeating this manual task every time.
I have written the WsiHero utility to make this validation simpler.
How to become a WSI hero?
If you want to easily validate your service for WSI compliance and become a WSI hero then Download WsiHero. It is a GUI application which saves you the need to deal with the WSI command line and its strict log format.
Installation 1. Ensure Microsoft .Net framework 2.0 or higher is installed (http://www.microsoft.com/downloads/en/details.aspx?FamilyID=0856eacb-4362-4b0d-8edd-aab15c5e04f5&displaylang=en).
2. Ensure Java 5 or higher is installed (http://www.java.com/en/download/index.jsp).
3.Download WsiHero and extract it to some folder, for example c:\program files\WsiHero.
4. Since it is not legal for me to distribute the WSI command line utility, please download it by yourself as follows:
Do not extract them, just put them as .zip in the WsiHero folder (c:\program files\WsiHero). Do not change their names.
In case the direct links do not work for some reason here is how to get the above files from the WSI site: From the "deliverables" section of the site download "BP 1.2 and 2.0 Test Tools package" and "Reliable Secure Profile Delivery Package".
Usage 1. Run WsiHero.exe from its folder.
2. Paste a Wsdl or a Soap file location into the text box and press "Add".
Notes:
Location can be a local disk path or a Url.
If the file contains Soap it must have the .Xml extension. Otherwise it is assumed to be a Wsdl.
For example any of the following can be put in the text box:
Recently I had to call a Java web service from Wcf. The service was secured with a 256-bit ssl certificate. Every request I made was rejected with a 401 Unauthorized response.
I then used this site to check what encryption strength my machine supports:
It seemed my pc is configured to only allow 128 bit connection.
Luckily I have found this link which explains how to configure windows to use 256-bit Ssl (Vista only). It seems the problem is in the windows defaults and not in Wcf. This changed the result in IE:
and also the Wcf client worked. The strange is that the setting that needs to be changed should only affect the order of the supported algorithms. So if the server requires a 256-bit OR a 128-bit key, the default setting would suggest 128 first. But if the server only allows 256-bit I would expect this to work since this is a supported configuration, just not the first in order. Either way the fix made it work.
Update: Silverlight is alive and well! Microsoft clarifies that Silverlight is a strategic technology. Scott Guthrie promises that future features and tools will be revealed soon. I will update about anything which has relevancy to Wcf. This post is still very relevant to show the increasing importance of javascript clients, side-by-side with Silverlight clients.
Everybodytalk about the "unofficial" announcement of Microsoft embracing Html5 over Silverlight. And while this was never approved (unless you read between the lines) the below may be a hint.
Yesterday Microsoft had published an "early access" codeplex project with some features from the next Wcf version. The two main themes seem to be native Http support and jQuery. And not a word about Silverlight. Will there ever be security in the SL netTcp transport? Will we see more WS-Security coverage from the Wcf stack? It seems jQuery improvements got a higher priority for this release. Is the Wcf early access project a hint for a focus shift from SL to Html5? (update: more SL features will be revealed soon, hopefully they include the above).
The planned jQuery support includes code generation for javascript clients, working with untyped json objects, Html forms submission and more.
So how does the Html5 thing affect Wcf?
This is definitely a big push to the Rest / Json / Anti WS-* approach (not that it needed it anyway). Silverlight did not support the full Wcf stack in version 4, but in theory nothing prevented it from growing into it in the next versions. Browsers in contrary cannot work with WS-Security and feel much more at home with Json over Xml. Also Rest is more natural for the web anyway.
I have writtenabout Wcf performanceissuesbefore, but this one seems to be the biggest. Valery had published in the Wcf forum an interesting performance issue. In short, a WCF client tries to consume a non-WCF service where the contract looks something like this:
In soap, byte arrays are encoded as base64 strings so it can look like this:
or with line breaks after each 73 characters, like this:
both options are valid according to the base64 RFC:
Ok so it does not really advocate this... But it is a fact that many soap stacks still use this MIME-originated format and also Wcf supports it.
So what is the problem?
It seems that when Wcf gets a message which contains base64 with CRLF, the processing is slower in a few seconds(!). A drill down shows that the problem is in the DataContract serializer. Take a look at this program:
For those of you who are interested to test this, the files are here and here.
The output is:
This clearly reveals a performance problem.
Why does this happen?
While debugging the .Net source code, I have found this in the XmlBaseReader class (code comments were in the source - they are not mine):
So the data contract serializer tries to read the base64 string, but for some reason succeeds only if the string does not have white spaces inside it (we can further debug to see how that happens but it is exhausting for one post :). The serializer then removes all the white spaces (which requires copying the buffer again) and tries again. This is definitely a performance issue.
Notes:
This happens with both .Net framework 3.5 and 4.0.
This is a DataContract specific issue - it does not happen when you use other .Net mechanisms such as Convert.FromBase64String
I have reported this in Microsoft connect, you are welcome to vote this issue up.
Workarounds
There a few workarounds. The trade-offs are generally convenience of API (or "where you prefer to put the 'dirty' stuff").
1. As Valery noticed, you can change the contract to use String instead of byte[]. Then Convert.FromBase64String will give you the byte array.
2. Change your contracts to use the XmlSerializer instead of DataContract serializer. The former does not experience this issue. The XmlSerializer is generally slower (when base64 does not appear that it) so this is what you loose. You get a better API here as clients do not need to manipulate the base64 string.
3. The best of course is to change the service implementation to return base64 without line breaks. Also if large binaries are returned anyway it may be a better idea to employ MTOM.
4. A Wcf custom encoder can strip the spaces from the message before it is deserialized. However this also involves copy of strings and this is beneficial only in rare cases.
UPDATE: seems like I'm late to the party in a few years...
This one really surprised me. I have built a utility which saves WSDL files locally. Now a WSDL can reside in exotic locations such as "http://server/service/?q=wsdl" or "http://service/...[900 chars]..." which do not imply a natural candidate for the local file name (the latter also violated the 256 characters path limit). But I wouldn't want the utility to use opaque names such as "1.wsdl" either. So I use this code snippet to generate a meaningful name from a url, which is cool. Usually. The other day a user complained that the utility does not work with his wsdl which resides here:
Seems pretty straight forward. Until you do this:
this yields the below:
And if you try to rename a file directly from explorer you face:
Two weeks ago I announced the Msdn ultimate giveaway contest. A lot of people published great comments to the post which made it a really hard decision for me. I finally had to decide and I have based my decision on the following criteria: Relevancy to web services interoperability, amount of implementation details, and presence of rationale behind the details.
It was a tough decision and a very close call. I learned a lot from your responses and thank you all for participating.
This one is rare but may happen when you get a malformed x.509 certificate.
Upon sending a request your WCF proxy / server will throw this:
The reason is that the SubjectKeyIdentifier extension in the X.509 certificate is invalid. It is not empty (which is ok) and does not contain a legal value either. It is just malformed.
The only solution here is to use a different valid certificate.
Well, the title makes it sound bigger than it really is.
Sometimes a soap request or response contains an X.509 certificate encoded in base64 string:
It may be useful to create the actual certificate that this encoded string represents, usually for the purpose of debugging.
This code snippet will do the trick:
now the certificate is ready in the designated path:
The below is a common error with Wcf clients in security interoperability scenarios:
What does it mean?
When a signed response comes back from the server it has two ways to reference the signing certificate.
Option A (key identifier):
Option B (direct reference):
The above error means that the response has key identifier but the client is configured to require a direct reference.
How to fix it?
On your client, configure allowSerializedSigningTokenOnReply to true:
An alternative can be to build a custom message encoder which changes the response from option B to A. This is possible since we know what is the certificate (using the reference) so we can create the binary token. Of course this alternative is much harder and in the general case the former should be preferred.
A lot of Wcf users encounter the below error when consuming a secured non-Wcf web services:
Understanding the problem
When we configure the service x.509 certificate in the client the latter sees this certificate as the service identity. So whenever we interact with this service the client verifies that this certificate is still the service identity, and if not it assumes some sort of phishing occur and drops the connection.
example
This is our client config:
We have declared the above certificate here since we want Wcf to use it for encryption and to verify signature. Wcf does all this and also something else: it assumes this is the service “identity”. We’ll see how this afefcts us later.
This is our client request:
The client uses the service public key to create a session key which is used to encrypt the request body. Only the owner of the matching private key knows how to decrypt this and since this owner is the server the client gets the authentication it wants.
This is the service response:
The service signs the response message. Anyone who has the service public key can now verify that the service is the one who sent this response and not some MITM. And since our client has the server public key (it’s the one we configured in the config, which is the same one the client used for encryption) the client can now validate the signature source.
So when do we get this “signed with a different token” error?
We can see in the service response a “binarySecurityToken” element. The service uses it to notify the client which certificate it used for signature. The content of this element is just a base-64 encoded certificate, so the client can easily extract it and compare to the certificate it has configured for the service. If they are not the same we get this exception.
Why must they be the same?
Remember that when we configure the service certificate on the client, Wcf treats it as the service identity. When a response comes back the client expects the service to prove its identity by signing the response with its key. If the response is signed – but using a different key – Wcf throws this authentication error since service identity has not been proved.
I am not using any encryption. Why do I get this error?
Even if you use ProtectionLevel.Sign, Wcf still requires you configure a service certificate. Since the service signs the response Wcf will compare the signing token to the certificate you configured.
Solution
If there is indeed a requirement for the service to use a different signing and encryption certificates, what we would like to do is to configure both of them on the client, so Wcf would know that both of them can authenticate the service. Unfortunately Wcf only allows us one slot of service certificate per configuration. So we need to look in other places.
If we use ProtectionLevel.Sign, meaning we only sign messages but do not encrypt them, then the service certificate which we configure on the client is only used for the purpose of validating the response signature. So we are free to change it to be the actual signing certificate. In some cases we may not know what this certificate is. Apart from asking the service author to provide it we can examine the response (as above) and extract the certificate from the binary security token (which may not always exist though).
Option 3
This is a workaround that reduces the security level of the communication. Use it only if you have other means to verify the service apart form the signature or if you are willing to have reduced security.
Step 1: Implement a custom message encoder which removes from the response the Envelope/Header/Security element all together.
This is a Wcf 4.0 new feature but it is also available in a patch to Wcf 3.5 SP1. Strive to use 4.0 if you can since some users have problems with the patch.
This solution requires to use a custom binding. If you need help converting your wsHttpbinding to a custom one use the binding converter.
The implication of this workaround is that your client will not verify the service identity at all – this is usually not desired from security perspective.
We're just 8 days away from your chance to win a $11899 MSDN Ultimate subscription.
All you need to do is leave a comment to the original post describing (in detail) a project you were involved with where web services interoperability was used. There are 3 prizes waiting so you have pretty good chances...
Once again, it's time to renovate my blog layout. This time the motivation is not only visual - the previous layout left no space for long code samples.
Ido had noticed a bottleneck in Wcf services which use the Asp.Net compatibility mode. Asp.Net synchronizes by default requests that come from the same session, so such requests are executed sequentially instead of concurrently. If this behavior is not desired by your service you should consider turning the compatability mode off or apply one of the other workarounds.
This package is worth $11,899. And you will get it for free.
How do I get a $11,899 MSDN Ultimate for free?
Publish a comment to this post describing a project you were involved with where web services interoperability was required. The three most detailed descriptions by 8 October 2010 will grant their author a $11,899 MSDN Ultimate for free!
Edit: detailed = in quality, not length :) chosen by me.
The "padding oracle" Asp.Net vulnerability will surely affect many production web sites. Does this have any effect on web services and Wcf?
To put this in concrete terms, can a web service become a "padding oracle"?
Without getting too much into details, I believe it is possible in some situations. The best way to protect against this is to ensure all your messages are digitally signed. With Wcf this behavior is actually built in to the framework and you cannot send messages which are encrypted but not signed. With older frameworks such as Wse, and some non-MS soap stacks, this is possible, so you might want to check any relevant production application.
Regardless of this it is always a best practice to sanitize customer facing error messages.
I have actually did a little test. I have configured a Wcf service to emit debug information in error responses (this is the most verbose response Wcf is capable of). I have then created a valid encrypted request, manually altered the encryption to make it not valid, sent it to the server and got this soap fault:
No secret data is enclosed here. I have then changed my server implementation to throw a logical exception:
I initiated a legal request and this time got this response:
So Wcf did return a full stack trace with an application exception. This means that Wcf shields us from verbose errors when security exceptions are thrown.
So are we safe?
You never are. For this reason take these precautions when using Wcf:
Disable debugging information in responses
Even though we saw Wcf tries not to disclose security information you should still protect from such information leaking into application level exceptions (or Wcf bugs). Just make sure your service has this behavior:
Protect your trace files at all cost
They may contain the detailed exceptions hackers look for.
What about Asp.Net web services?
Asp.Net web services (asmx) are inherently insecure (WRT message level) unless Wse is used with them. You should apply the same technique and sanitize your Wse error messages as explained in this walkthrough.
Edit: Some information in the original version of this post was not correct. I have fixed it.
A few hours ago Microsoft had publicly disclosed a critical Asp.Net security vulnerability. In short, a hacker can get sensitive data from an Asp.Net application in a few minutes. This information includes any file on the server (e.g. web.config) and any viewstate data even if it is encrypted. To block this attack, site owners should immediately direct all errors to a single page (custom errors redirect).
If you have a production Asp.Net system read this now and go fix it.
How does the vulnerability work?
Without making this a walkthrough to my hacker readers, this is the general flow:
Some common encryption algorithms (among them TripleDes and AES) divide the encrypted data to blocks of equal size, where the last block is padded with additional data to make all blocks equal. A few years ago a security researcher had published an article explaining how to hack such an algorithm provided that a "padding oracle" exists. What is a padding oracle? It is an application (or some abstract... oracle) which tels us if the padding of an encrypted data is in the correct format.
Well, it turns out that Asp.Net uses TripleDes, and that some specific feature of Asp.Net turns it into a "padding oracle", you get the idea...
How to avoid this?
1. The single most important rule is this:
When there is an error in your application, never let the end user know what it was. In particular never allow an end user to distinguish between 404 and 500 status codes.
The way to do it is to redirect all errors to one page. ScottGu's post explains how to do it.
2. The hacker that found this hole wrote his in this twitter account:
error message setting is irrelevant. no error? there's always HTTP status. always the same HTTP status? there's always big timing different
He means that even though we hide from him the real Http status (by redirecting to a single error page) the real status still "leaks" by computing the response time.
This means that adding a random wait before sending the error page can help us. ScottGu's post has the details too.
If you are really interested in the bits & bytes (&blocks) of this kind of attacks this is a nice introduction.
Apple's announcement yesterday on relaxing the strict AppStore SDK policy is good news for web services interoperability. Why is that? We need to get some background first:
With the rising popularity of iPhone development, people seek for web services support.
One known Mac framework is wsdl2objc (wsdl to objective-C, the iPhone development language). It give the basic functionality of creating a proxy and calling the service.
If your web service is relatively simple and you can afford to build the soap payload by hand, you could also consider using the native Http library CFNetwork or a the ASIHTTPRequest wrapper.
The latter option also enables transport level security (ssl).
Message level security
This is a known pain point. I have yet to find a WS-Security implementation for the iPhone. If you are doing simple stuff (mainly username token) you can get away with manually pushing these headers. But more advanced scenarios (involving X.509) are harder and Apple does not seem to develop a ws-security implementation soon.
So why doesn't any third party develop one?
Until yesterday, apple banned the usage of code generators for AppStore applications. While a soap stack may be developed in pure objective-C, the realm of web services highly relies upon code generation and libraries.
Yesterday, Apple announced that it now allows any development tools to be used for AppStore applications. While the media mainly mentioned the implication on Adobe Flash, the announcment also positively affected the MonoTouch framework which enables C# development for iPhone. And while MonoTouch WCF support is still experiential and does not include advanced WS-*, there is now hope to have all the advanced stuff in a future version.
UPDATE 1: It seems MonoTouch does support the Wcf Silverlight stack. I assume username token is in but not sure about binary encoding and tcp transport. Looking at Mono WCF support page shows that Wcf is still not fully supported there, which may be an indication.
UPDATE 2:Carlos mentions that Silverlight is supported on Mac. After yesterday's announcement there is a reason to believe that SL will be officially supported in iPhone either which makes standalone SL applications a prospect too.
@YaronNaveh
