How to configure ISA 2006 with FBA for OWA and NTLM for Outlook Anywhere and Autodiscover in Exchange 2007

Configuration of Exchange 2007 with Outlook Web Access (OWA), Outlook Anywhere (OA), ActiveSync and Autodiscover can add grey hair to any system administrator or IT consultant. Then also trying to get different authentication schemes and ISA 2006 to play nice is not making it any easier.

Most Administrators have a wish to configure their environments used externally as securely as possible, including using SSL certificates with HTTPS instead of no encryption with HTTP, and using NTLM authentication instead of Basic authentication. But security is not everything, a userfriendly interface like Forms Based Authentication (FBA) is a must to avoid user iritation and support calls.

However getting FBA and NTLM to work together in ISA with Exchange 2007 can be quite scary, so lets go deeper and find out what we need to be aware of to get it working.

Why FBA and NTLM together fails

The reason Forms Based Authentication and NTLM doesnt go hand in hand, is that when a listener in ISA is setup to use FBA, and we usually want FBA on Outlook Web Access (it just look better), it will automatically fall back to Basic authentication when the client fails to authenticate with FBA. Outlook when connecting will refuse the FBA authentication, so it will automatically recieve a Basic request instead, and answer that correctly. But because Exchange is configured to use NTLM, the basic request is not accepted, since it was anticipating an NTLM response.

Example with Forms Based authentication and Exchange configured to NTLM Authentication

1. Outlook connects to ISA 2006 and recieves an FBA response
2. Outlook refuses and gets the option to use Basic authentication
3. Outlook prompts the user with a username and password prompt and authenticates with Basic authentication
4. The request fails even if correct, because the wrong authentication protocol (Basic instead of NTLM) was used.
5. Outlook prompts the user again, resulting in the very anoying never ending password prompt in outlook!

Example with Forms Based authentication and Exchange configured to Basic Authentication

1. Outlook connects to ISA 2006 and recieves an FBA response
2. Outlook refuses and gets the option to use Basic authentication
3. Outlook prompts the user with a username and password prompt and authenticates with Basic authentication
4. The request succeeds, and all is well – most administrators use this option due to its much easier setup.

So the simple solution is simple, just use Basic authentication instead… but if You are anything like me, that is not a solution but a workaround – and we dont like that.

How to get FBA and NTLM to work at the same time

The following solution works for me, I set up Exchange to use NTLM and do the rest of the changes on the ISA 2006 firewall.

By configuring the Outlook Web Access on a seperate listener(1) with Forms Based Authentication and placing all the other websites like Outlook Anywhere and Autodiscover on a another seperate listener(2) using NTLM(/Basic) authentication where compatible, – note ActiveSync wants Basic.

This solution requires two IP addresses, since each listener must have its own IP address.

So we setup our ISA like this

Exchange 2007 Outlook Web Access publishing rule #1

• External fqdn: owa.sole.dk
• Listener1 using IP1 and Forms Based Authentication
• Directs to Exchange 2007 CAS server for Outlook Web Access

Exchange 2007 Outlook Anywhere/Autodiscover/etc publishing rule #2

• External fqdn: autodiscover.sole.dk
• Listener2 using IP2 and NTLM authentication
• Directs to Exchange 2007 CAS server for Outlook Anywhere
• note some services might require Basic Authentication like ActiveSync

To make my life easier and usually save a lot of time I use a SSL SAN Certificate with the following domains for full Exchange 2007 compatibility.

• mail.sole.dk and/or owa.sole.dk
• autodiscover.sole.dk
• autodiscover.other-email-domains-i-would.use
• internal-servername1
• internal-servername2

If only one e-mail domain is used a Domain Validated SSL certificate can be used, if more than one e-mail domain is to be supported a Organisation validated SSL certificate is needed. Remember most SSL certificates require licenses for each server You install the certificate on, including ISA and CAS servers, some issuers allow more than one installation per license.

To test if everything works correctly i use www.testexchangeconnectivity.com an excellent and slightly hidden Microsoft tool to test external access to Exchange 2007 services. Outlook can also be used to test the AutoDiscover response by right click the little Outlook icon in the minibar (usually bottom right) and selecting “Test automatic configuration of e-mail”.

Hope this saved you some minutes, if there is a need for a more elaborate guide with screenshots, make sure to comment letting me know this, and I will add it later.

Related posts:

1. How to install a SSL certificate backup file on Exchange 2007 and still have time for facebook The easiest way (I love easy!) to order and install...
2. How to publish a website with both Anonymous and Forms Based Authentication in ISA 2006 We had a customer asking for the main part of...
3. How to use OWA for both Exchange 2003/2007 during migration and extra things to consider A customer asked me if it was possible during migration...
4. How to fix Exchange 2007 RPC over HTTPS/Outlook Anywhere on Windows Server 2008 A small but anoying bug in Microsoft Exchange 2007 in...
5. How to fix missing PPTP Interfaces from RRAS console in ISA 2006 and stop a memory leak in the process All PPTP VPN interfaces in ISA 2006 (sp1) disappeared from...