Michel Barneveld
To unpathed waters, undreamed shores
Kernel-mode authentication performance benefits

 

2 Weeks ago I wrote a post about kernel-mode authentication and SharePoint 2010. One of my questions was: What is the performance benefit of using kernel-mode authentication?

I did some testing last week and now I had time to analyze the results and create some graphs out of those results.


The test setup 

I defined some requirements for my tests:

  • Test the authentication with a minimum of overhead
  • Test without IIS or client caching
  • Automated and repeatable tests
  • Test with multiple users
  • Test with multiple threads
  • Test with different amounts of group memberships
  • Test with separate client and web server
  • Test with full stressed web server and not full stressed client
  • Test with different application pool identities
  • Test Kerberos and NTLM


Based on those requirements I used the following setup:

Users & Groups:

  • Created 750 AD users (TestUser0001 to TestUser0750)
  • Created 100 AD groups (TestGroup0001 to TestGroup0100)
  • TestUser0001-0250 are member of 100 groups
  • TestUser0251-0500 are member of 50 groups
  • TestUser0501-0750 are member of 0 groups
  • Created account for application pool identity (using Kerberos AES 256 bit encryption).

So I have 3 sets of users with different group memberships (0, 50 and 100 groups)

Web server:

  • Hyper-V VM
  • Single CPU with limit of 20% CPU usage on host*
  • windows 2008R2
  • 4 sites with 4 application pools
  • each application pool with a different authentication setting
    • Negotiate:Kerberos (Kerberos only provider and can only work with kernel-mode off)
    • Negotiate with kernel-mode off
    • Negotiate with kernel-mode on (SPN registered on machine account)
    • Negotiate with kernel-mode on (SPN registered on app pool identity + useApplicationPoolCredentials setting)
  • 1 single HTML file in the root**

*) The web server was way more efficient in decrypting Kerberos tickets than the client was in encrypting them. I needed to lower the CPU speed of the web server to make sure it was at 100% CPU usage while the client was not completely stressed out. Setting it to 20% in Hyper-V slowed down the web server enough to get it at 100% CPU usage, while keeping the CPU usage of the client far enough from 100%.

**) To isolate the performance of authentication I wanted to eliminate the overhead of the .Net framework so I didn't use any aspx files but a plain .HTML file.


Client:

  • Windows 2008R2 in the same domain (I know that the 10 connection limit were removed in Vista SP2 and Windows 7, but just to be sure I used a server OS as client)
  • Physical machine with dual core CPU
  • Custom written .Net test application
    • Used a queue of 5000 requests
    • Used 32 threads to fire those 5000 requests (I tested with 1,2,4 and 8 threads first, but that couldn't get the server at 100%)
    • Used the credentials of 30 different users (I wanted to use the whole set of 250 users, but using more than 40 users made the client app using 100% CPU, so with 30 users I was on the safe side)
    • Each thread would ask the queue for the next request and based on the request number (using modulo) it used a different user for the request: User = RequestNumber % 30 (examples: RequestNumber 21->User21, RequestNumber 33 -> User3, etc)
    • Request are of form: http://site1.mb.local/test.html?<GUID>  (The GUID makes the request unique so prevent client or server side caching)
    • The time was captured to complete those 5000 requests
    • Each test was run 6 times. The first run was discarded. In the presented results the average of the 5 runs was used and converted to requests per second.
    • The user set was configurable, so the tests could be batched with different group memberships.

 

Test 1 - Kerberos authentication using the ApplicationPoolIdentity

For the first test I configured the 4 application pools to use the built-in ApplicationPoolIdentity as the identity of the application pool. And I registered the SPN's of the 4 URLs to the machine account.

Result:

2 Observations from above graph:

  1. Using kernel-mode authentication (green and purple bars) has a significant improvement over the disabled kernel-mode bars (blue and red)
  2. When the users are member of more groups the overall performance decreases. That is expected because the Kerberos ticket size will increase since each group membership is included in a Kerberos ticket and with more tickets there is more data to decrypt.

Note:
This scenario is not compatible with web farm scenario's. When you have multiple web servers you must use domain accounts for the application pool identity and register the SPN's to that account.

 

Test 2 - Kerberos authentication using a domain account

For the second test I configured the 4 application pools to use a custom domain account as the identity of the application pool. And I registered the SPN's of 3 URLs to that domain account. The 4th SPN was registered to the machine account, because the Kerberos tickets are decrypted by the machine account when using kernel-mode authentication without the useAppPoolCredentials setting.

The domain account used as the application pool identity didn't have any delegations configured to other services.

Result:

*) Kernel-mode on (the green bar) is not compatible with web farm scenario's, because the SPN is registered to the machine account.

 3 Observations from above graph:

  1. Also with this test there is more performance when using kernel-mode.
  2. And also the performance decreases with more group memberships.
  3. The performance of the kernel-mode enabled scenario's is the same as in the first test. However the performance of the kernel-mode disabled scenario's is significant less compared to test 1!

I also captured the request lenght and noticed something odd: The request length of the green bar scenario was much larger (almost doubled) compared to the other 3. The only difference was that this scenario has the SPN registered to the machine account instead of the domain account. After inspection of the Kerberos tickets I discovered the difference: The ticket for the machine account was almost twice as large compared to the ticket for the domain account and there was also an other difference: it has the ok_as_delegate flag. That flag means that the user credentials can be delegated to any other service (TRUSTED_FOR_DELEGATION).

So tickets for SPN's registered to the machine account has the ok_as_delegate flag set. I didn't confirm if it actually allows to delegate to any service (the normal TRUSTED_FOR_DELEGATION setting). But I did test what happens when I gave my domain account the same "delegate to any service" setting. The Kerberos tickets for the domain account became twice as large, just like for the machine account!

So when a service account has the "Delegate to any service" setting set, the Kerberos tickets for SPN's registered to that account will get twice as large. That's an interesting finding!
To put this in perspective: The Kerberos tickets at my current customer are around the 7KB in size (yep large organization with a lot of groups),  So 90% of the web request size is this Kerberos tickets. And if you trust this domain account to delegate to any service it is even doubling the request size. That has impact on performance when dealing with large volumes of clients. But since we are only using constrained delegation and don't have any accounts with the delegate to any service option, we don't have these larger tickets.

To see if that delegate to any service has an impact on the performance of the authentication I used a 3rd test:

Test 3 - Kerberos authentication using a domain account with delegate to any service

For the third test I used the same configuration as the second test with the exception that the domain account was trusted for delegation to any service.

Result:

*) Kernel-mode on (the green bar) is not compatible with web farm scenario's, because the SPN is registered to the machine account.

 3 Observations from above graph:

  1. Same results as previous tests (kernel-mode is faster and being a member of more groups is slower)
  2. The performance of the green bar scenario is the same as in Test 2. Remember that this scenario has the SPN registered to the machine account. The ticket size remains the same as in Test 2, so this is expected behavior.
  3. The performance of the 3 other scenario's is decreased! So the change to the domain account being trusted for delegation has increased the ticket size and effected the performance, which is expected when you have to decrypt a larger ticket.   

 

Test 4 - NTLM authentication using the ApplicationPoolIdentity

Kerberos was tested in the first 3 tests. But will NTLM authentication have the same result? That's what test 4 and 5 is about.

For test 4, I used 2 sites with 2 application pools and used the NTLM authentication provider. One site is using kernel-mode enabled the other one is disabled. Both application pool use the built-in ApplicationPoolIdentity as application pool identity for this test.

Result:

2 Observations from above graph:

  • Kernel-mode authentication has also improved performance for NTLM.
  • The group membership doesn't effect NTLM authentication performance. Which is expected because the groups are not included in the NTLM authorization header as is the case with Kerberos.

 

Test 5 - NTLM authentication using a domain account

So is there a difference using a domain account as the application pool identity as we saw in the Kerberos tests? This test will give the answer.

The application pools from test 4 were now configured to use a domain account.

Result:

1 Observations from above graph:

  • Using a domain account as the application pool identity doesn't effect the authentication performance. It's the same result as with test 4.

 

Conclusions

  • Using Kernel-mode authentication can give up to a 20% to 25% performance boost.
    Note: In real-world scenario's this will most likely be much lower, because above test scenario's were optimized to show the impact of kernel-mode authentication.
  • Using an application pool identity account that is trusted for delegation will lower the performance boost. Use constrained delegation for maximum performance ( and is more secure btw)
  • The more groups users are member of the lower the performance of Kerberos authentication.
  • Using the Kerberos only authentication provider is a good way to enforce the use of Kerberos authentication, but it can't profit of the performance boost since it's not compatible with kernel-mode authentication.
  • When you can't use kernel-mode authentication (like in the Kerberos only scenario), you will get the most performance with Kerberos using the built-in ApplicationPoolIdentity as the application pool identity instead of a domain account. However this is only possible for single web server scenario's. Web farm Kerberos scenario's require domain accounts as application pool identities.

 

 


Posted 12-02-2009 10:30 PM by Michel Barneveld
Filed under: , ,

Comments

WMicka wrote re: Kernel-mode authentication performance benefits
on 01-26-2010 6:09 PM

Hi, great article!

The only thing I miss is another comparison between NTLM and Kerberos authentication performance Impact.

Thanks

Wojciech

Michel Barneveld wrote re: Kernel-mode authentication performance benefits
on 01-28-2010 2:02 PM

Hi WMicka,

you are correct in some way. ;-)

It's missing the conclusion from above test results that Kerberos benefit a bit more from kernel mode authentication than NTLM. (Kerberos max 20%-30% versus NTLM 15%-20%). Note that above tests were done in Lab conditions. So real life benefits are expected to be lower.

Comparing the performance between NTLM and Kerberos is a different story. That is more dependent on other things, like network latency, the average number of groups users are member of, location of AD controllers, etc. Getting some metrics around that is on my to do list ;-)

SHOX R3 wrote re: Kernel-mode authentication performance benefits
on 08-05-2010 6:28 AM

Getting some metrics around that is on my to do list <a href=www.r3tn.com/.../1048.html> SHOX NZ </a> Getting some metrics around that is on my to do list

Blog du Tristank wrote Configuring Kerberos for SharePoint farms - a generic gotchas list
on 10-27-2011 10:05 AM

Recently, I worked on a Kerberos configuration issue with a customer; these are my notes from the visit

Add a Comment

(required)  
(optional)
(required)  
Remember Me?

  Enter code:
Copyright © 2003 - 2009 Michel Barneveld
Powered by Community Server (Non-Commercial Edition), by Telligent Systems