Security interop gotcha: Empty signature transformation

@YaronNaveh

Markó had challenged me recently with a security interoperability case between Wcf and WSIT (metro). A mutual X.509 protection was involved, and the Java server was returning the following soap fault:

com.sun.xml.wss.XWSSecurityException: Encryption Policy verification error: Looking for an Encryption Element  in Security header, but found com.sun.xml.wss.impl.policy.mls.SignaturePolicy@3657517

I have not seen this error before, but I have followed the book and asked for a sample working soap message:

<S:Envelope xmlns:S="http://schemas.xmlsoap.org/soap/envelope/" xmlns:wsse11="http://docs.oasis-open.org/wss/oasis-wss-wssecurity-secext-1.1.xsd" xmlns:wsse="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-secext-1.0.xsd" xmlns:wsu="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-utility-1.0.xsd" xmlns:xs="http://www.w3.org/2001/XMLSchema" xmlns:ds="http://www.w3.org/2000/09/xmldsig#" xmlns:xenc="http://www.w3.org/2001/04/xmlenc#" xmlns:exc14n="http://www.w3.org/2001/10/xml-exc-c14n#">

  <S:Header>

    <To xmlns="http://www.w3.org/2005/08/addressing" wsu:Id="_5006">http://server.com</To>

    <Action xmlns="http://www.w3.org/2005/08/addressing" wsu:Id="_5005">someAction</Action>

    <ReplyTo xmlns="http://www.w3.org/2005/08/addressing" wsu:Id="_5004">

      <Address>http://www.w3.org/2005/08/addressing/anonymous</Address>

    </ReplyTo>

    <MessageID xmlns="http://www.w3.org/2005/08/addressing" wsu:Id="_5003">uuid:203aa9ea-eea6-4f96-b0b2-16575411e9a2</MessageID>

    <wsse:Security S:mustUnderstand="1">

      <wsu:Timestamp xmlns:ns19="http://schemas.xmlsoap.org/ws/2006/02/addressingidentity" xmlns:ns18="http://docs.oasis-open.org/ws-sx/ws-secureconversation/200512" xmlns:ns17="http://www.w3.org/2003/05/soap-envelope" wsu:Id="_3">

        <wsu:Created>2010-12-22T14:25:10Z</wsu:Created>

        <wsu:Expires>2010-12-22T14:30:10Z</wsu:Expires>

      </wsu:Timestamp>

      <wsse:BinarySecurityToken xmlns:ns19="http://schemas.xmlsoap.org/ws/2006/02/addressingidentity" xmlns:ns18="http://docs.oasis-open.org/ws-sx/ws-secureconversation/200512" xmlns:ns17="http://www.w3.org/2003/05/soap-envelope" ValueType="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-x509-token-profile-1.0#X509v3" EncodingType="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-soap-message-security-1.0#Base64Binary" wsu:Id="uuid_87b326f6-54fe-42d5-a3a5-aea7cb12f66b">PO7sWakqK...</wsse:BinarySecurityToken>

      <xenc:EncryptedKey xmlns:ns19="http://schemas.xmlsoap.org/ws/2006/02/addressingidentity" xmlns:ns18="http://docs.oasis-open.org/ws-sx/ws-secureconversation/200512" xmlns:ns17="http://www.w3.org/2003/05/soap-envelope" Id="_5002">

        <xenc:EncryptionMethod Algorithm="http://www.w3.org/2001/04/xmlenc#rsa-oaep-mgf1p" />

        <ds:KeyInfo xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:type="KeyInfoType">

          <wsse:SecurityTokenReference>

            <wsse:KeyIdentifier ValueType="http://docs.oasis-open.org/wss/oasis-wss-soap-message-security-1.1#ThumbprintSHA1" EncodingType="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-soap-message-security-1.0#Base64Binary">TzX5OGaS9Ftsw1t+eGyfBmJblWc=</wsse:KeyIdentifier>

          </wsse:SecurityTokenReference>

        </ds:KeyInfo>

        <xenc:CipherData>

          <xenc:CipherValue>WJU7hIO...</xenc:CipherValue>

        </xenc:CipherData>

      </xenc:EncryptedKey>

      <xenc:ReferenceList xmlns:ns19="http://schemas.xmlsoap.org/ws/2006/02/addressingidentity" xmlns:ns18="http://docs.oasis-open.org/ws-sx/ws-secureconversation/200512" xmlns:ns17="http://www.w3.org/2003/05/soap-envelope">

        <xenc:DataReference URI="#_5008" />

      </xenc:ReferenceList>

      <ds:Signature xmlns:ns19="http://schemas.xmlsoap.org/ws/2006/02/addressingidentity" xmlns:ns18="http://docs.oasis-open.org/ws-sx/ws-secureconversation/200512" xmlns:ns17="http://www.w3.org/2003/05/soap-envelope" Id="_1">

        <ds:SignedInfo>

          <ds:CanonicalizationMethod Algorithm="http://www.w3.org/2001/10/xml-exc-c14n#">

            <exc14n:InclusiveNamespaces PrefixList="wsse S" />

          </ds:CanonicalizationMethod>

          <ds:SignatureMethod Algorithm="http://www.w3.org/2000/09/xmldsig#hmac-sha1" />

          <ds:Reference URI="#_5003">

            <ds:Transforms>

              <ds:Transform Algorithm="http://www.w3.org/2001/10/xml-exc-c14n#">

                <exc14n:InclusiveNamespaces PrefixList="S" />

              </ds:Transform>

            </ds:Transforms>

            <ds:DigestMethod Algorithm="http://www.w3.org/2000/09/xmldsig#sha1" />

            <ds:DigestValue>q9TWoUKb25xlel3AIA8bFUkl0hw=</ds:DigestValue>

          </ds:Reference>

          <ds:Reference URI="#_5004">

            <ds:Transforms>

              <ds:Transform Algorithm="http://www.w3.org/2001/10/xml-exc-c14n#">

                <exc14n:InclusiveNamespaces PrefixList="S" />

              </ds:Transform>

            </ds:Transforms>

            <ds:DigestMethod Algorithm="http://www.w3.org/2000/09/xmldsig#sha1" />

            <ds:DigestValue>5Ab1ebo4/FraGgck/A8iDx1J9+I=</ds:DigestValue>

          </ds:Reference>

          <ds:Reference URI="#_5005">

            <ds:Transforms>

              <ds:Transform Algorithm="http://www.w3.org/2001/10/xml-exc-c14n#">

                <exc14n:InclusiveNamespaces PrefixList="S" />

              </ds:Transform>

            </ds:Transforms>

            <ds:DigestMethod Algorithm="http://www.w3.org/2000/09/xmldsig#sha1" />

            <ds:DigestValue>ipPKrD6igfYF3tC10tsurnoHSks=</ds:DigestValue>

          </ds:Reference>

          <ds:Reference URI="#_5006">

            <ds:Transforms>

              <ds:Transform Algorithm="http://www.w3.org/2001/10/xml-exc-c14n#">

                <exc14n:InclusiveNamespaces PrefixList="S" />

              </ds:Transform>

            </ds:Transforms>

            <ds:DigestMethod Algorithm="http://www.w3.org/2000/09/xmldsig#sha1" />

            <ds:DigestValue>cMG2PvlZKNLoAWehtNhxruuRl9Y=</ds:DigestValue>

          </ds:Reference>

          <ds:Reference URI="#_5007">

            <ds:Transforms>

              <ds:Transform Algorithm="http://www.w3.org/2001/10/xml-exc-c14n#">

                <exc14n:InclusiveNamespaces PrefixList="S" />

              </ds:Transform>

            </ds:Transforms>

            <ds:DigestMethod Algorithm="http://www.w3.org/2000/09/xmldsig#sha1" />

            <ds:DigestValue>rRhCJKKcNv2gfh+Hpi62wDEirbE=</ds:DigestValue>

          </ds:Reference>

          <ds:Reference URI="#_3">

            <ds:Transforms>

              <ds:Transform Algorithm="http://www.w3.org/2001/10/xml-exc-c14n#">

                <exc14n:InclusiveNamespaces PrefixList="wsu wsse S" />

              </ds:Transform>

            </ds:Transforms>

            <ds:DigestMethod Algorithm="http://www.w3.org/2000/09/xmldsig#sha1" />

            <ds:DigestValue>OUm9PNmaQgH1CQ4JitIqVX5eY+g=</ds:DigestValue>

          </ds:Reference>

          <ds:Reference URI="#uuid_87b326f6-54fe-42d5-a3a5-aea7cb12f66b">

            <ds:Transforms>

              <ds:Transform Algorithm="http://www.w3.org/2001/10/xml-exc-c14n#">

                <exc14n:InclusiveNamespaces PrefixList="wsu wsse S" />

              </ds:Transform>

            </ds:Transforms>

            <ds:DigestMethod Algorithm="http://www.w3.org/2000/09/xmldsig#sha1" />

            <ds:DigestValue>r5dVsFsteThPivLkpsO+Fme5FIs=</ds:DigestValue>

          </ds:Reference>

        </ds:SignedInfo>

        <ds:SignatureValue>8PLYt+ddRqRV2i7tX2R7PgIEJU=</ds:SignatureValue>

        <ds:KeyInfo>

          <wsse:SecurityTokenReference wsu:Id="uuid_ba14a75f-0bee-4ec6-ae77-2646ab277e2f" wsse11:TokenType="http://docs.oasis-open.org/wss/oasis-wss-soap-message-security-1.1#EncryptedKey">

            <wsse:Reference URI="#_5002" ValueType="http://docs.oasis-open.org/wss/oasis-wss-soap-message-security-1.1#EncryptedKey" />

          </wsse:SecurityTokenReference>

        </ds:KeyInfo>

      </ds:Signature>

    </wsse:Security>

  </S:Header>

  <S:Body wsu:Id="_5007">

    <xenc:EncryptedData xmlns:ns19="http://schemas.xmlsoap.org/ws/2006/02/addressingidentity" xmlns:ns18="http://docs.oasis-open.org/ws-sx/ws-secureconversation/200512" xmlns:ns17="http://www.w3.org/2003/05/soap-envelope" Id="_5008" Type="http://www.w3.org/2001/04/xmlenc#Content">

      <xenc:EncryptionMethod Algorithm="http://www.w3.org/2001/04/xmlenc#aes128-cbc" />

      <ds:KeyInfo xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:type="KeyInfoType">

        <wsse:SecurityTokenReference wsse11:TokenType="http://docs.oasis-open.org/wss/oasis-wss-soap-message-security-1.1#EncryptedKey">

          <wsse:Reference URI="#_5002" ValueType="http://docs.oasis-open.org/wss/oasis-wss-soap-message-security-1.1#EncryptedKey" />

        </wsse:SecurityTokenReference>

      </ds:KeyInfo>

      <xenc:CipherData>

        <xenc:CipherValue>+VCg/Fg9BR...</xenc:CipherValue>

      </xenc:CipherData>

    </xenc:EncryptedData>

  </S:Body>
</S:Envelope>

This looks a little intimidating, but I was actually able to create a Wcf custom binding which generates this exact message:

private static CustomBinding CreateCustomBinding(EndpointAddress address)
{

            var res = new CustomBinding();

            SymmetricSecurityBindingElement sec =
                (SymmetricSecurityBindingElement)SecurityBindingElement.CreateMutualCertificateBindingElement(

                    MessageSecurityVersion.
                       WSSecurity11WSTrust13WSSecureConversation13WSSecurityPolicy12BasicSecurityProfile10);

            res.Elements.Add(sec);

            sec.SetKeyDerivation(false);

            sec.MessageProtectionOrder = MessageProtectionOrder.SignBeforeEncrypt;

            sec.DefaultAlgorithmSuite = SecurityAlgorithmSuite.Basic128;

            sec.EndpointSupportingTokenParameters.Signed.Add(sec.EndpointSupportingTokenParameters.Endorsing[0]);

            sec.EndpointSupportingTokenParameters.Endorsing.Clear();

            sec.EnableUnsecuredResponse = true;            

            res.Elements.Add(new CustomTextMessageBindingElement());

            res.Elements.Add(new HttpTransportBindingElement());

            return res; }

At this point everything was smelling like roses – I was able to get a real response from the server instead of the error above.

However from a reason unknown to me at the time the Wcf client was rejecting the response with this error:

Message security verification failed.

"End element 'Reference' from namespace 'http://www.w3.org/2000/09/xmldsig#' expected. Found element 'ds:DigestMethod' from namespace 'http://www.w3.org/2000/09/xmldsig#'. Line 1, position 2075."

Here is the relevant response section:

<ds:Signature Id="_1">

    <ds:SignedInfo>

       ...

        <ds:Reference URI="_5002">

           <ds:Transforms />

           <ds:DigestMethod Algorithm="http://www.w3.org/2000/09/xmldsig#sha1" />

           <ds:DigestValue>2jmj7l5rSw0yVb/vlWAYkK/YBwk=</ds:DigestValue>

        </ds:Reference>

       ...

    <ds:SignedInfo>
</ds:Signature>

This is the response signature, which seems pretty straight forward. Why would Wcf throw an exception?

Analysis

If you take a closer look at the response above, you can see how it contains an empty “transforms” element. A non empty one looks like this:

          <Reference URI="#_0">

            <Transforms>

              <Transform Algorithm="http://www.w3.org/2001/10/xml-exc-c14n#" />

            </Transforms>

            <DigestMethod Algorithm="http://www.w3.org/2000/09/xmldsig#sha1" />

            <DigestValue>jcLdOx6w1477YMoCJR8TJQRJuKw=</DigestValue>

          </Reference>

This is the framework code that parses the signature transforms element:

        public void ReadFrom(XmlDictionaryReader reader, TransformFactory transformFactory, DictionaryManager dictionaryManager)

        {

            reader.MoveToStartElement(dictionaryManager.XmlSignatureDictionary.Transforms, dictionaryManager.XmlSignatureDictionary.Namespace);

            reader.Read();

 
            while (reader.IsStartElement(dictionaryManager.XmlSignatureDictionary.Transform, dictionaryManager.XmlSignatureDictionary.Namespace)) 
            {

                //handle transformation
            } 
            reader.MoveToContent();

            reader.ReadEndElement(); // Transforms              ...
        }

There are 3 cases that this code is supposed to handle:

• In case the transforms element has childs (as in the last snippet before the above code), then the Read() call would put the cursor on the first child element (“transform” in singular). The while loop will end at the closing “transforms” element which is aligned with the following ReadEndElement() call.

• In case there are no childs but there is a separate closing element:
  
  
  
  
  

  
  

            <Reference URI="#_0">
  
              <Transforms>
  
              </Transforms>
  
              <DigestMethod Algorithm="http://www.w3.org/2000/09/xmldsig#sha1" />
  
            </Reference> 

  
  

  
  
  
  then Read() would put us on the closing element, and since we will not enter the loop ReadEndElement will be right again.

• In case of an empty element:
  
  
  
  

  
  

   <Reference URI="#_0">
  
                <Transforms />
  
                <DigestMethod Algorithm="http://www.w3.org/2000/09/xmldsig#sha1" />
  
    </Reference> 
  
  
  
  
  the Read() would advance us behind the transformations tag! The next tag is the opening DigestMethod which of course fails for ReadEndElement().

So the case of an empty transforms element is responsible for the interoperability bug.

Workaround

If you are in a position to change the output of the Java server then various workarounds may apply. But what if you can’t?

The only real solution seems to be implementing the relevant ws-security portion by yourself. Here I show how to do the bare minimum of such a scheme – use the wcf security stack for the sending of the message and decrypting the response by ourselves. The missing piece will be the signature validation, which is very important in production but we can continue with development even without it as at least we get a readable response.

My solution involves using a custom message encoder. I have used the Wcf SDK message encoder sample as a template. When the response comes back to the encoder it decrypts it and remove the security from it before passing it to the upstream channel.  To make sure the security channel will not fail (due to the now unsecured response) we verify our binding has EnableUnsecuredResponse  set to true.

Here is how the encoder code looks like:

        public override Message ReadMessage(Stream stream, int maxSizeOfHeaders, string contentType)

        {
            var sr = new StreamReader(stream);

            var wireResponse = sr.ReadToEnd();

            var logicalResponse = GetDecryptedResponse(wireResponse);

            logicalResponse = String.Format(

                    @"<s:Envelope xmlns:s=""http://schemas.xmlsoap.org/soap/envelope/"">


                            <s:Body>


                                {0}


                            </s:Body>


                        </s:Envelope>",

                    logicalResponse);

            XmlReader reader = XmlReader.Create(new StringReader(logicalResponse));

            return Message.CreateMessage(reader, maxSizeOfHeaders, MessageVersion.Soap11);
        }

        private string GetDecryptedResponse(string encryptedResponse)
        {
            var doc = new XmlDocument();

            doc.LoadXml(encryptedResponse);

            var cipherNode = doc.SelectSingleNode("//*[local-name(.)='Body']//*[local-name(.)='CipherValue']");

            byte[] cypher = Convert.FromBase64String(cipherNode.InnerText);

            byte[] key = GetEncryptingKey();

            byte[] iv = GetIV(cypher);

            var AesManagedAlg = new AesCryptoServiceProvider();

            AesManagedAlg.KeySize = 128;

            AesManagedAlg.Key = key;

            AesManagedAlg.IV = iv;

            var body = ExtractIVAndDecrypt(AesManagedAlg, cypher, 0, cypher.Length);

            return UTF8Encoding.UTF8.GetString(body);
        }

So we find the ciphered portion of the message and decrypt it. We then use it to construct a new unsecured message which we will pass along in the pipeline. Here this is done in a dirty way which hard codes the soap11 envelope.

But where do we get the security key to decrypt the message from?

This key is encrypted in our request:

      <e:EncryptedKey Id="uuid-7674d043-9a35-495d-a4dc-27daee955237-1" xmlns:e="http://www.w3.org/2001/04/xmlenc#">

           …

        <e:CipherData>

          <e:CipherValue>HQ0UzEZnc9lq...</e:CipherValue>

        </e:CipherData>       

      </e:EncryptedKey>

We cannot decrypt it since it is encrypted with the server public key and we do not have the private one. However the security channel must keep reference to this key somewhere so that it will be able to decrypt the response (which it would do anyway unless the empty transformation bug). We need to use some reflection tricks to get the session key:

        private void SaveEncryptionKey(Message message)
        {
            var secHeaderType = message.GetType().GetField("securityHeader", 
                                              BindingFlags.NonPublic | BindingFlags.Instance);

            var secHeader = secHeaderType.GetValue(message);

            var encTokenType = secHeader.GetType().BaseType.BaseType.GetField("encryptingToken", 
                                              BindingFlags.NonPublic | BindingFlags.Instance);

            var token = (WrappedKeySecurityToken)encTokenType.GetValue(secHeader);

            var securityKey = (InMemorySymmetricSecurityKey)token.SecurityKeys[0];

            var symmetricKey = securityKey.GetType().GetField("symmetricKey", 
                                               BindingFlags.NonPublic | BindingFlags.Instance);

            this.key = (byte[])symmetricKey.GetValue(securityKey);
        }

The full source code is here. It will not run since for privacy reasons I did not use the real wsdl / certificates. But it can be used as a reference in case you encounter a similar situation.

@YaronNaveh

What's next? get this blog rss updates or register for mail updates!