A few years ago a white paper on web services interoperability would start with "turn off secure conversation" or "avoid the following types". A recent set of white papers published by Microsoft shows the today all vendors are much more mature and ready for real world interoperability.
Abu Obeida and Mike Champion from Microsoft announced recently the below articles. I recommend this reading for those of you who need to write interoperable web services. Of course not each article is relevant for each interop scenario so guess what? I've read them for you so you can decide which ones are relevant for you.
Data types interoperability between .NET and Java Christian Weyer and Buddhike de Silva have listed most types you might ever want to use with web services and verified their interoperability status. This is a must read for every public facing web service developer.
One thing that I am missing here though is the Axis2 java stack which is very popular. Also I'm confused by the check on WebSphere - as an application server you can use various stacks on it so I'm not sure which one was used. I assume Wcf was used on the .Net side (".Net 4" is specified) and I'm very curious if the data contract was able to cope with all scenarios or was xml serialization required in some cases. Generally data contract are optimized to enforce "best practice" which is sometimes counterproductive for interoperability.
The conclusion is very promising - good interoperability exists in almost all scenarios. One important reminder is the lack of interoperability with Hashtable and some of the generic types.
Jesus Rodriguez wrote a great article on ws-security interop between Wcf and Oracle WebLogic. Naturally the article dives into the ws-security details and is very technical. It proves that X.509 message security is a good option for interoperability. While jesus shows how to achieve federation interoperability via SAML and WS-Trust, it is clear that there is still a way to go here as a holder-of-key scenario is still not possible. I have not tested it myself but hopefully bearer tokens should also work well with these parties. The article does not mention it as it deals with message security while bearer tokens are a mix of message and transport.
Jesus adds another good one and this time on interoperability between Wcf and WebSphere. The username and x.509 scenarios work pretty well. The article mentions that WS-Trust interoperability is possible, but does not give the details, so I'm not sure if it is a mistake in the upload or if it really required some tweaks which were out of scope. Either way this is yet another proof that there is still a way to go with federation interoperability. Such security scheme will be very popular in the cloud era so hopefully all vendors will keep investing in it. As it stands now Wcf+Wif have great support for the related scenarios so I'll report on any news from the other vendors.
As a side note, While MTOM interoperability is possible here, I have seen in the past a few gotchas so hopefully a future paper will cover that up (I'll try to write about it myself actually).
Standards-Based Interoperability between SAP NetWeaver and Microsoft .NET Framework
A rich set of scenarios are working in this case, including security and MTOM. It is a little disappointing that a basic scenario like custom headers only partially pass, but NetWeaver is an opinionated framework that expose its users only what is relevant for them.
Metro to WCF Interoperability
The combination of Metro and Wcf shows off the richest set of working interoperability scenarios. This is not surprising considering the fact that one of the goals of WSIT (Metro's security stack) is interoperability with Wcf.
To conclude, all articles are very interesting and show that interoperability is a reality.
A few months ago I have written a post about replay attacks in Wcf. One of my readers had sent me a mail mentioning that while that post had some interesting information on replay attacks in general, it did not really discussed the Wcf part. In particular he asked me about the Wcf nonce cache and its difference from the Wse one. Also he was interested in cases where a replay attack may become a DOS attack. Here is what I wrote him back:
Yes, Wcf has a replay cache. One difference from the Wse cache is that in Wcf only the primary signature value is used for cache and not a username nonce (Wcf does not emit these anyway). You might want to take a look at some of the relevant Wcf security settings, for example ReplayCacheSize.
If you are really interested you can open reflector in System.ServiceModel.Security.ReceiveSecurityHeader.ProcessPrimarySignature() and see how the primary signature is validated againt the nonce cache:
As for DOS, the cache comes to solve the problem of replay attackes, e.g. an attacker sends a message a second time which makes a transaction presumably run twice and cause business damage. DOS attacks overloads the server with many requests until it stops from effectively surveying clients. While these attacks are not related, the cache used to prevent replay attacks can become very big until the server is out of memory. This can happen with or without a DOS taking place. For this reason the cache is limited in size and after X minutes a timestamp is used instead. If an attacker can send many non-replay requests in a short time to fill the cache he might be able to actually replay a message before the time windows is closed. However this will not be considered a DOS (even if it is just semantics). Regardless, I am not aware of any way for a web site to protect against a massive DOS attack where bots from around the globe act together.
WCF Binding Box is on air for over six months now. Hopefully it will stay there for a while, at least until Azure will close the free trial period. For those of you who forgot, the binding box allows to convert any WCF binding (e.g. WSHttpBinding) to a CustomBinding.
Around 1500 users have converted their bindings already. This seems like a good time to analyze some WCF security usage patterns.
Message Security Type
The first aspect I analyzed is the security type people use. Results show that over 60% of users apply some message level security (possibly together with ssl). One possible explanation to this result is that message level security is the default with Wcf (in contrary to most legacy soap stacks). Another explanation is this: People who work with message level security have much more available configurations, so they seek the binding box in order to get a custom binding. This means the measurement here may be a little biased.
(click image to enlarge)
Client Credential Type (Message)
The second question I tested is which message credentials are most popular. The results show that username authentication is by far the most popular authentication mechanism. This is expected considering that usernames are the easiest to work with and offer the best interoperability. We can tie this together with the huge interest in ClearUsernameBinding. Windows authentication gets a relatively low share which may indicate that most binding box users wrote internet and not intranet applications.
(click image to enlarge)
Client Credential Type (Transport)
Finally I have tested which are the most popular client authentication types with transport security. Here domain based authentication (windows, ntlm) is used by almost 40% of the users. Basic authentication is only 15% which is surprising taking into consideration that the most popular message level authentication mechanism is a UserName token. A quarter of the users chose the Certificate authentication type. We can add to this the fact that client certificates have a similar message level share, and that server side certificates are used in all of the transport modes and in most message level modes. This shows that most organizations deal with X.509 certificates this way or the other.
When we install an X.509 certificate in a windows store, it is valid only if we can build from it some valid certificates chain. For example, if its issuer's certificate is in the trusted root store.
However it seems there is a meaningful change between XP and Vista. In XP, when we install a certificate in the "trusted people" store it is not valid unless such chain exist:
In Vista, once we put the certificate in the TrustedPeople store it is automatically valid even if no chain exist:
This change makes sense: If we trust someone then we do not need to trust its issuer also. However there is a backward compatibility risk: non valid certificates on XP become implicitly valid on Vista which is a potential security hole.
One important attack against web services is a "replay attack": a malicious man-in-the-middle captures a valid message from a client to a server and resends it. This can cause duplicate transactions or denial of service (DOS).
Typically the way to protect here is to have a timestamp on the message and some expiration policy on the server. This is a very simplified description though, here are all the details.
The sending party attaches this timestamp to the SOAP message:
The receiving party can then decide based on some policy how much after the message was created it is suspected as a replay.
But what is this service policy? We can see an "expires" element in the client message itself, then why does the service need an additional policy?
The answer is that there is a duplicate protection against replay attacks. The client can decide its own policy and express it inside the message. The server should respect this policy and also adds its own policy. For example:
In this case the message is considered valid by the service, but the service is expected to respect the client policy and reject the message. the situation gets more interesting when we add real network latency and clocks synchronization into the picture.
In Wcf there are two somehow confusing settings to control the expiration:
There are various other settings which relate to tolerance for clock differences and I will write a separate post on them some other time.
Recently I had to consume an Axis2 web service using a Wcf client. Axis2 used the rampart module for message security where a mutual X.509 certificate was required. Since I had control over both client and server I was able to fine tune them to reach perfect interoperability.
On the Axis side, I have found the WS-Policy flavor configuration better to work with (interoperability-wise) .
This is the Axis configuration:
and this is the Wcf configuration:
Here is the full Axis2 service (which is based on the rampart SDK sample):
and the wcf client:
You will also need certificates.
Client certificate:
Server Certificate:
If prompt for a password use "adminamdin".
This is the Java key store:
and the password is "adminadmin".
If you want to use your own certificate just make sure the certificates have an SKI.
An important web services security requirement is nonrepudiation. This requirement prevents a party from denying it sent or received a message. The way to implement this is using Xml Digital Signatures. For example, if I sent a message which is signed with my private key, I cannot later deny that I sent it.
A common mistake is to think that every web service that require an X.509 certificate ensures nonrepudiation. This goes without say for web services that only require server certificate - in these services clients are either anonymous or username/password identified, which is considered weak cryptographically material.
However, also when a client X.509 is involved, nonrepudiation is not always guaranteed. For example, let's examine a Wcf service which uses WsHttpBinding with TransportWithMessageCredential and clientCredentialType="Certificate":
This is how the client request looks like:
The message body is not signed! This practically means anyone who has this message (for example the server) can extract the signed parts and resend them with a bogus body.
So without any further preparations let's see how to use ClearUsernameBinding.
Step 1: Download latest release
Download it here or go to google codegithub.
Then extract the zip to some folder, let's say C:\program files\ (the ClearUsernameBinding subfolder will be created when extracting the zip).
Step 2 (optional) - Run the sample project
It can be useful to run the sample application.
Run the server:
And now the client:
And if everything went smoothly you have just seen ClearUsernameBinding in first action!
Step 3 (optional) - Investigate the sample project source code
The best way to learn a new (and very simple in this case) technology is by looking at existing projects. Just open with VS 2008 the solution file:
And look at the source of the projects TestClient and TestService. These two projects are just normal WCF projects configured to use ClearUsernameBinding. In other words, making a WCF client/service use ClearUsernameBinding is just a matter of changing web.config and does not require coding. We will see in the next steps how to do it from scratch.
I'll probably have a separate post on the binding implementation itself. It is pretty straight forward and the handling of security is as I learned from Nicholas Allen's blog.
Step 4 - Creating your own service
For this step just create any normal WCF web site or a self hosted service.
Step 5 - Configure the service to use ClearUsernameBinding
Add your project a dll reference to
Then open web.config and register the ClearUsernameBinding under the system.ServiceModel section:
Finally configure your endpoint to use ClearUsernameBinging and its configuration:
Step 6 (optional) - Configure the message version
If you need to use a specific message version configure it in the "messageVersion" attribute in the above configuration. Valid values are: Soap11WSAddressing10, Soap12WSAddressing10, Soap11WSAddressingAugust2004, Soap12WSAddressingAugust2004, Soap11, Soap12, None, Default.
Example:
Step 7 - Configure the username authentication
This one needs to be done in any username/password authenticated service and not just one that uses ClearUsernameBinding. By default your server will authenticate the users against your active directory domain. If you want to do your own custom authentication you need to create a new class library project with a class that implements System.IdentityModel.Selectors.UserNamePasswordValidator
The class can look like this:
Don't forget to add dll reference to System.IdentityModel and System.IdentityModel.Selectors or the project will not compile. Then add this project as a project reference to your service project/website and configure the latter to use this custom authenticator:
Step 8 - Run the service
Yes, the service is now ready to be activated, so run it when you are ready (run it directly from VS, just press F5).
Step 9 -Build a client
A service is worth nothing if there are no clients to consume it.
Create a new console application.
Right click the "References" node in the solution explorer and choose "Add service reference". Specify the WSDL of the server. If you are running the server from the given sample then the wsdl is in http://localhost:8087/SampleService/?WSDL. If you used your own server just run it and get the wsdl.
Now add some client code that uses the proxy to call the service. Don't forget to specify your username/password. For example:
Step 10 - Configure the client
Configuring the client is as simple as configuring the service.
Here is the full client app.config:
Step 11 - Done, Done, Done!
That's all. You can now run your client and see how WCF can be used to access a service with a cleartext username/password. Use a tool like fiddler to verify that indeed a clear username is sent (I've shorten some low-level stuff from bellow message):
Conclusion
Sending username/password on the clear is not available out of the box with WCF (for reasons mentioned above). If such a scenario is required then ClearUsernameBinding needs to be used.
A lot of people use WCF default settings on production. In many cases changing these defaults can gear up the service throughput dramatically.
Let's look at the following use case:
The WsHttpBinding implicit defaults can be explicitly written as bellow:
Let's simulate a load on this service by employing many virtual users who constantly call the service one time and immediately disconnect. The number of users should be large enough such that service will use its max capacity. The results are:
Note: I didn't use a super strong server here but as we can see below it shouldn't matter for our needs. Also a load of just a few minutes was enough to prove our theory.
Those are not very good results of course.
Now let's tweak the configuration a little bit:
And with the same amount of virtual users we get these results:
That's 3.5 times faster!
So, what happened here? Since the only change we did is in two settings we need to analyze each of them.
negotiateServiceCredential This setting determines whether the clients can get the service credential (e.g. certificate) using negotiation with the service. The credentials are used in order to authenticate the service and to protect (encrypt) the messages. When this setting is set to "true" a bunch of infrastructure soap envelopes are sent on the wire before the client sends its request. When set to "false" the client needs to have the service credentials out of band.
The trade off here is better performance (using "false") versus more convenience (using "true"). Setting "false" has its hassles as we now need to propagate the service credential to clients. However, performance wise, setting "negotiateServiceCredential" to "false" is always better.
Take a look at how many infrastructure messages are exchanged when negotiateServiceCredential is "true":
While when not negotiating life is much brighter:
establishSecurityContext This setting determines whether WS-SecureConversation sessions are established between the client and the server. So what is a secure conversation anyway? In a very simplified manner we can say that a normal secured web service request requires one asymmetric encryption. Respectively, normal N requests require N asymmetric encryptions. Since asymmetric encryption is very slow, setting up a secure conversation is usually a good practice: It requires a one-time asymmetric encrypted message exchange in order to set up a session; Further calls in the session use symmetric encryption which is much faster.
Now remember that in our case we assume that clients call the service just one time and disconnect. If a secure session is established the message exchange will look like this:
If we do not use secure session we have:
So it is clear that we're better off in the latter case.
With secure sessions there isn't really any trade off and the decision is quite scientific: When only one client request is expected set establishSecurityContext to "false".
Summary Wisely changing WCF defaults can yield a significant improvement in your service performance. The exact changes need to be made and their exact effect are dependent in the scenario. The example above showed how to speed up a certain service 3 times faster.
When you use SSL with WCF or .Net 2.0 web services you might get this exception:
This means that the X.509 certificate that the server presented is not valid according to your client's trust chain. the trust chain is the list of certificate issuers that you trust. In many cases this means you should not trust this web service. In some cases you decide to trust it anyway, due to the fact that you know the author or are still in early testing stages. You have two options:
Option 1 - Make the certificate valid You would need to have the server certificate or its issuer's certificate in the trusted certificates store of your client. This can be done using WinHttpCertCfg.exe or using the "mmc" console.
Option 2 - Configure WCF/.Net 2.0 to not validate the certificate With WCF, a common gotcha is to try and achieve that by setting:
However this will not work with transport level security (SSL) but only with message level security. The correct way to do it is the same one as with .Net 2.0 web services. Just add this code before calling the service:
When a custom binding is used in WCF it is possible to configure the value of requireSecurityContextCancellation. The name of this property doesn't exactly indicate in what scenarios it should be used.
When the client and server use WS-SecureConversation (by setting authenticationMode to "SecureConversation") the client get some token from the server in the initial handshake and needs to send it in every connection attempt. The server can extract this token and find the relevant data of this session in its memory. But what happens when the server/IIS is restarted? The memory is cleared and the session becomes invalid. The way to overcome this is to set requireSecurityContextCancellation to false (the default is true). In this mode the session key is saved at the client side and it sends it to the server in every request. This means that even if the server is restarted the session key is available in the request. This also means that the size of each request is a little bigger.
Since with this mode the server does not need to save the session key it does not need to be informed when the session is over and from here the property name requireSecurityContextCancellation.
Recently I had an interesting case with WCF "raw message" contract:
The message I used on the client side contained some SOAP headers. Since this is an untyped message these headers were neither encrypted nor signed. That was a show stopper for me.
However the request message was still not being encrypted nor signed. I struggled with this issue for some time until I was lucky enough to find Pedro Felix's message in the WCF MSDN forum:
In other words the OutgoingSignatureParts always relates to server outgoing messages while client outgoing stuff is always considered as "incoming". After fixing this the code worked like a charm. I probably should have proactively RTFM myself but at least I felt good for not being the only one who fell for this...
First attempt - using basicHttpBinding with MessageClientCredentialType of "Username". Unfortunetelly this would yield the following exception:
Second attempt - using basicHttpBinding with TransportWithMessageCredential mode.
Since this mode implies that we need to secure the transport we get any of these exceptions, depending if we are on the client or the server side:
Third attempt - using wsHttpBinding with MessageClientCredentialType of "Username".
Depending on several other settings, and wheather we're on the client or the server, we would get any of these exceptions:
Forth attempt - using wsHttpBinding with TransportWithMessageCredential mode.
Similarily to the second attempt we get:
OR
Fifth attempt - using customBinding with httpTransport and security element with authenticationMode of UserNameOverTransport
This time we get:
So it really seems like Microsoft is trying to (im)politely convince us not to use clear username/password. But what can we do for cases where this behaviour is really required?
The solution
The solution s to use ClearUsernameBinding. This binding seamlessly integrates with WCF and allows us to use clear username/password.
Many web services authenticate clients using username/password. In the soap message it looks like this:
You can notice that the username/password are clear (not encrypted). This means that everyone that can see this message can steal the password.
There are typically 3 ways to overcome this.
1. Sending the hashed password only:
Where the digest is calculated from the password, a timestamp and a nonce. However this is not really secure. A hacker can use a dictionary attack to extract the password.
2. Protecting the username with SSL. This is a valid option but is limited to HTTP web service only and denies us from some of the rich WS-Security options. There are some other transports which are inherently secured (like SSL passthrough of load balancers as F5's BIG-IP) but I will not discuss them here.
3. Protecting the username with message-level X.509 certificate. This is another valid option but sometimes more complex to implement.
In practice if you want to use username/password you would need to decide between options 2&3. Some frameworks, like Microsoft's WCF, even prevents you from using option #1 at all. Nevertheless there are a few exceptions where option #1 is valid:
Your network is inherently secured so you are not afraid of password stealers.
You are required to interoperate with a service that requires to send clear username/password
When you try to import a WCF-created WSDL from a Java client such as Metro (WSIT) you may get the following error:
The reason is that the WCF WS-Policy contains this section:
This means that your WCF service uses X.509 certificates negotiation. In such a scenario, clients use the server X.509 certificate for encryption. The unique here is that clients are not required to have this certificate out of band (as in most cases), rather they get this certificate using a SOAP-level negotiation. This is implemented as an extension over WS-Trust. While this is not strictly a proprietary Microsoft solution, Microsoft was the only one to implement it so far. In short - X.509 negotiation (SslContextToken) is not interoperable. Unfortunately(?) the default configuration of a WCF service is to use this negotiation. To turn it off either update your WsHttpBinding configuration:
Or choose the correct scenario in your CustomBinding:
You can also use the equivalent cases for username authentication. Note that any clients (including WCF ones) will now need to have the service certificates defined out of band.
A client that uses a server certificate in order to encrypt a message needs to reference this certificate in code. Inside the message the reference would typically look like this:
Or this:
Or this:
Or in any of a few other formats.
The required format can be declared in WS-Policy. For example:
However many services do not have WS-Policy so it's not a must.
There are various ways to get this SKI from a certificate. Some of them are:
The certificate might contain an out-of-band generated SKI inside it as an "extension".
An SHA1 thumbprint is calculated out of the certificate raw data.
The certificate serial number and issuer name are used
The certificate subject name is used
The SKI needs to be calculated from the certificate using some algorithm
Guess which option causes us most trouble? Right, the last one. The problem is that it is not concrete enough. There were a few ways to generate this SKI and so there were misunderstandings between Java and Microsoft applications and even between different versions of the same frameworks.
Let's take a sample certificate as an example. First convince yourself that it does not contain SKI extension:
Now look at the format WebSphere 6 server would expect:
WSE2 would generate:
And WSE 3 generates this one:
With WCF you would get the following exception:
As WCF does not work with certificates that do not have SKI when the latter is explicitly required - and that's what I did for this sample.
So by all and all we've got 4 different SKI's out of one certificate!
Out of these 4 frameworks there is a way to interoperate using SKI and X.509 that does not contain SKI extension between the following:
Note: It is possible to interoperate between these platform in various other ways. If you now design your system you should use these ways. The information here is provided for those who are already working with a given systems.
So how can we reach this interoperability?
To interoperate with WebSphere all you need to do is to change WSE2 configuration to:
And it will generate a WebSphere-like SKI (which is more standard all around). To interoperate between WSE2 and WSE3 you need to leave the default WSE2 configuration as is and change WSE3:
Note: even though WSE 3 allows skiMode="RFC3280" as WSE2 does the calculation is different and will not interoperate.
As for the rest - you would have to use a certificate with an SKI extension or use another type of reference instead.
What's next? get this blog rss updates or register for mail updates!
@YaronNaveh