How to configure ISA 2006 with FBA for OWA and NTLM for Outlook Anywhere and Autodiscover in Exchange 2007

Configuration of Exchange 2007 with Outlook Web Access (OWA), Outlook Anywhere (OA), ActiveSync and Autodiscover can add grey hair to any system administrator or IT consultant. Then also trying to get different authentication schemes and ISA 2006 to play nice is not making it any easier.

Most Administrators have a wish to configure their environments used externally as securely as possible, including using SSL certificates with HTTPS instead of no encryption with HTTP, and using NTLM authentication instead of Basic authentication. But security is not everything, a userfriendly interface like Forms Based Authentication (FBA) is a must to avoid user iritation and support calls.

However getting FBA and NTLM to work together in ISA with Exchange 2007 can be quite scary, so lets go deeper and find out what we need to be aware of to get it working.

Why FBA and NTLM together fails

The reason Forms Based Authentication and NTLM doesnt go hand in hand, is that when a listener in ISA is setup to use FBA, and we usually want FBA on Outlook Web Access (it just look better), it will automatically fall back to Basic authentication when the client fails to authenticate with FBA. Outlook when connecting will refuse the FBA authentication, so it will automatically recieve a Basic request instead, and answer that correctly. But because Exchange is configured to use NTLM, the basic request is not accepted, since it was anticipating an NTLM response.

Example with Forms Based authentication and Exchange configured to NTLM Authentication

  1. Outlook connects to ISA 2006 and recieves an FBA response
  2. Outlook refuses and gets the option to use Basic authentication
  3. Outlook prompts the user with a username and password prompt and authenticates with Basic authentication
  4. The request fails even if correct, because the wrong authentication protocol (Basic instead of NTLM) was used.
  5. Outlook prompts the user again, resulting in the very anoying never ending password prompt in outlook!

Example with Forms Based authentication and Exchange configured to Basic Authentication

  1. Outlook connects to ISA 2006 and recieves an FBA response
  2. Outlook refuses and gets the option to use Basic authentication
  3. Outlook prompts the user with a username and password prompt and authenticates with Basic authentication
  4. The request succeeds, and all is well – most administrators use this option due to its much easier setup.

So the simple solution is simple, just use Basic authentication instead… but if You are anything like me, that is not a solution but a workaround – and we dont like that.

How to get FBA and NTLM to work at the same time

The following solution works for me, I set up Exchange to use NTLM and do the rest of the changes on the ISA 2006 firewall.

By configuring the Outlook Web Access on a seperate listener(1) with Forms Based Authentication and placing all the other websites like Outlook Anywhere and Autodiscover on a another seperate listener(2) using NTLM(/Basic) authentication where compatible, – note ActiveSync wants Basic.

This solution requires two IP addresses, since each listener must have its own IP address.

So we setup our ISA like this

Exchange 2007 Outlook Web Access publishing rule #1

  • External fqdn: owa.sole.dk
  • Listener1 using IP1 and Forms Based Authentication
  • Directs to Exchange 2007 CAS server for Outlook Web Access and ActiveSync services

Exchange 2007 Outlook Anywhere/Autodiscover/etc publishing rule #2

  • External fqdn: autodiscover.sole.dk
  • Listener2 using IP2 and NTLM authentication
  • Directs to Exchange 2007 CAS server for Outlook Anywhere
  • note the ActiveSync service requires Basic authentication work 100%, it will not work with a NTLM listener.

To make my life easier and usually save a lot of time I use a SSL SAN Certificate with the following domains for full Exchange 2007 compatibility.

  • mail.sole.dk and/or owa.sole.dk
  • autodiscover.sole.dk
  • autodiscover.other-email-domains-i-would.use
  • internal-servername1
  • internal-servername1.internaldomain.local

If only one e-mail domain is used a Domain Validated SSL certificate can be used, if more than one e-mail domain is to be supported a Organization validated SSL certificate is needed. An Organization validated SSL requires the same owner is listed in whois for the domains. Remember most SSL certificates require licenses for each server You install the certificate on, including ISA and CAS servers, some issuers allow more than one installation per license (GlobalSign allows 3 servers pr. license).

To test if everything works correctly i use www.testexchangeconnectivity.com an excellent and slightly hidden Microsoft tool to test external access to Exchange 2007 services. Outlook can also be used to test the AutoDiscover response by right click the little Outlook icon in the minibar (usually bottom right) and selecting “Test automatic configuration of e-mail”.

Hope this saved you some minutes, if there is a need for a more elaborate guide with screenshots, make sure to comment letting me know this, and I will add it later.

Update: I have written a more comprehensive and slightly more technical blog about how to do this, take a look at How to configure Exchange 2007 and ISA with FBA and NTLM, without loosing ActiveSync on the way

4 Responses to “How to configure ISA 2006 with FBA for OWA and NTLM for Outlook Anywhere and Autodiscover in Exchange 2007”

  • shahid:

    hello
    i am trying to setup autodiscover in exchange2007 , i tried several scenario ,
    my setup as exchange 2007 backend ,frontend ,Isa 2006. i have 1 certificate with san name owa.domain.com, autodiscover.domain.com,exch…..
    The above scenario as you mentioned,
    the solution requires two IP addresses, since each listener must have its own IP address ????
    Internal IP or External IP ? if internal on ISA or Exchange ?

    Thanks and Regards,

  • Sole:

    Hi Shahid,
    Just came back from vacation, so sorry for the late reply.
    Because You have 1 SAN certificate with all the domains included, you only need 1 public IP address, but if You only use one IP address You will not be able to use both Forms Based Authentication and NTLM authentication at the same time for that listener. If you configure Exchange 2007 to use Basic authentication this is no problem at all and will work just fine.

    However if You configure your Exchange 2007 to use Forms Based Authentication for OWA and NTLM for autodiscover, Outlook Anywhere, etc. then You will need to create seperate listeners with seperate public IP addresses. The IP address on your CAS can be just one address with no problems.

    Let me know how you fare and if you get it working, what the problem was.

    -Sole

  • Sole:

    I have extended on this post, with information on configuration of Exchange 2007 and ISA for this setup with FBA & NTLM & Basic authentication in this post http://www.sole.dk/post/how-to-configure-exchange-2007-and-isa-with-fba-and-ntlm-without-loosing-activesync-on-the-way/?p=207

  • John Harris:

    Sole – Mange tak !

    I had wondered if the dual listener thing would work. Thanks for saving me a lot of headache. The SAN cert is a perfect solution.

    JH

Leave a Reply