The Mystery of the IP-HTTPS Listener, an Outlook Client and an IPv4 Only Network

Overview
A Problem with Outlook or DirectAccess?
Why is Outlook Asking for Credentials?
The Mystery of the Enabled IP-HTTPS Adapter
Putting Together the IP-HTTPS Pieces
What’s with the Slow Link Detection?
Summary
Discussion
FOR MORE INFORMATION:

Overview

A customer presented the DirectAccess team with an interesting problem that brought together many pieces of how a DirectAccess works, and how things might not work in certain circumstances. Because the problem was an interesting one, and because it highlights how some features of DirectAccess work, we thought it might be a good idea to share our experiences.

The customer was using a UAG DirectAccess solution that included only the “manage out” capabilities. By “manage out” I mean that they’re currently only interested in the ability to manage the DirectAccess clients, regardless of where those DirectAccess clients are located. At this time they aren’t allowing their users to connect to the corpnet using what we call the “intranet tunnel”. While this isn’t a solution we’ve documented, it is possible to configure things to support a “manage out only” solution. For more information on the manage out (always managed) option, please see Deep dive into UAG DirectAccess (Tweaking the GPOs) at http://blogs.technet.com/edgeaccessblog/archive/2010/02/18/deep-dive-into-uag-directaccess-tweaking-the-gpos.aspx

There were no problems with remote management of the DirectAccess clients and in fact they found this ability to keep remote clients in the same security state as on-network clients a fantastic advantage. In addition, they found that their “manage out only” solution enabled users to change this passwords using the same methods that on-network clients use – CTRL-ALT-DEL and click “Change Password”. If you’ve had to deal with password changes using a VPN or other solution, you’ll quickly appreciate just how nice it is that the users can change their passwords in the same why they always do, without having to fire up a VPN connection.

A Problem with Outlook or DirectAccess?

Prior to deploying DirectAccess, external Outlook users used RPC/HTTPS (Outlook Anywhere) to connect to Exchange when they were outside the internal network. When the users were on the internal network, they connected over MAPI/RPC. When the users connected from an external location using RPC/HTTPS, Outlook would ask the users for user name and password, but when on the internal network, Outlook did not ask. This is typical behavior and the solution was working well for them.

The problem seemed to come after they deployed DirectAccess. For some reason, Outlook users were being asked for their passwords when they were on the internal network. This wasn’t normal behavior for Outlook and led the customer to ask if the Outlook problem might be related to DirectAccess.

During the information gathering process it was discovered that:

The customer had an IPv4 only internal network – there was no IPv6 connectivity available at all. Neither native IPv6 nor ISATAP was available.
The customer’s current DNS infrastructure enabled users on the internal network to resolve the name that external users use to connect to the RPC/HTTPS proxy
There was a path that enabled users on the internal network to connect over RPC/HTTPS to the RPC/HTTPS proxy.
Internal could resolve the name of the IP-HTTPS listener and there was a path that allowed internal clients access to the IP-HTTPS listener on the external interface of the DirectAccess server.

Why is Outlook Asking for Credentials?

The answer to that question came quickly. The reason why the internal network users were being asked for passwords was that the Outlook client was no longer using MAPI/RPC to connect to the Exchange Server; it was using RPC/HTTPS.

What didn’t make sense about this is that the Outlook clients were configured to RPC/HTTPS if a slow link is detected. That is to say, the option On slow networks, connect using HTTP first, then connect using TCP/IP (figure 1) is selected. For some reason, the Outlook client detected that it was on a slow network.

Figure 1

At this point you might wonder why the RPC/HTTPS (Outlook Anywhere) connection was being established, even if the a slow link was detected, since in most cases, the name of the publicly accessible proxy used to publish RPC/HTTPS wouldn’t be available to hosts on the internal network. As it turns out, the name was resolvable to both internal and external clients and a path was available for the internal Outlook clients so that they could loop out through the Internet and back into the internal network to access Exchange. Not very efficient, but it did explain why the Outlook clients didn’t try to use MAPI/RPC.

But the team still couldn’t figure out why Outlook determined that there was a slow link between itself and the Exchange Server. There was no significant network congestion, so a fast link should have been detected and MAPI/RPC should have been used.

The Mystery of the Enabled IP-HTTPS Adapter

This got the team wondering if the DirectAccess client assumed it was on the Internet. That didn’t seem to be the case. When they ran

netsh namespace show effectivepolicy

it indicated that the DirectAccess client was on the internal network and that the Name Resolution Policy Table (NRPT) wasn’t being used to resolve names. The fact that clients appeared to have normal corporate connectivity (with the exception of the Outlook issue) confirmed that the DirectAccess clients didn’t think they were on the Internet.

However, the team did notice that the IP-HTTPS adapter on the DirectAccess clients was still active. That shouldn’t have been the case since DirectAccess clients use Corporate Connectivity detection to determine when to enable the IP-HTTPS adapter, and corporate connectivity should be successful.

The status of the IP-HTTPS adapter is a function of:

Initialization of the IP-HTTPS adapter
The results of Corporate Connectivity detection

IP-HTTPS Adapter Initialization

The IP-HTTPS adapter on the DirectAccess client initializes (also known as entering the Enabled state) if any of the following conditions are met:

The DirectAccess client detects that it’s behind a Web proxy (note that web proxy detection is not automatic; the user must configure IP-HTTPS to use a specific web proxy)
The DirectAccess client detects that it’s behind a port or protocol blocking firewall
Policy requires that the IP-HTTPS link is always on (this is not configured by the UAG DirectAccess wizard)
The user manually enables the IP-HTTPS adapter

Corporate Connectivity Status Detection

Corporate Connectivity detection is a little more complex. There are two possible states for Corporate Connectivity:

Corporate Access
No Corporate Access (default)

The DirectAccess client uses active and passive analyses to determine Corporate Connectivity.

There are two methods used for active probing:

DNS probing. The DirectAccess client performs a DNS query to resolve an admin configured internal host name. This name is the Corporate DNS Probe Host Name. The Corporate DNS Probe Host Name is stored in the DirectAccess client Registry at HKEY_LOCAL_MACHINE\SOFTWARE\Policies\Microsoft\Windows\NetworkConnectivityStatusIndicator\CorporateConnectivity\DnsProbeHost
Web probing. The DirectAccess client sends requests to an admin configured internal Web site. Any successful retrieval of a Web page is an indication of success. The Web probe URL is stored in the DirectAccess client’s Registry at HKEY_LOCAL_MACHINE\SOFTWARE\Policies\Microsoft\Windows\NetworkConnectivityStatusIndicator\CorporateConnectivity\WebProbeUrl. The reason for the Web probe is that DirectAccess clients may be located behind a Web proxy server that performs DNS queries on the behalf of the DNS client. Since the DNS client service doesn’t perform DNS queries for the destination host name for Web proxy clients, this alternate method is required.

There are also two methods used for the passive probing:

Path reachability monitoring. DirectAccess clients have a set of IPv6 prefixes of hosts on the corporate network. If any of the hosts are reachable, then Corporate Connectivity is confirmed. If all paths are unreachable, then Corporate Connectivity enters a Suspect state. If Corporate Connectivity remains in the Suspect state for three consecutive polls for an active connection to a host on the configured IPv6 prefixes, an active probe (either a DNS probe or a Web probe) is initiated to determine if Corporate Connectivity still exists. The list of acceptable prefixes is defined in the DirectAccess client’s Registry at HKLM\Software\Policies\Microsoft\Windows\ NetworkConnectivityStatusIndicator\CorporateConnectivity\SitePrefixes
New IPsec Main Mode Security Association (SA) establishment. Corporate Connectivity can also be confirmed by the establishment of a new IPsec SA between the DirectAccess client and a destination with a corporate IPv6 prefix. Only Main Mode SAs are checked. Note that Main Mode SAs remain active as long as the interface is up. Therefore, Corporate Connectivity can only be determined by looking for the establishment of a new Main Mode SA. The DirectAccess client (NCSI) records the SA identification number (ID) of the last known SA for each interface. When a new interface with a larger ID is seen for an interface, Corporate Connectivity is confirmed. The list of corporate IPv6 prefixes for Main Mode SA establishment is stored on the DirectAccess client’s Registry at HKLM\Software\Policies\Microsoft\Windows\ NetworkConnectivityStatusIndicator\CorporateConnectivity\SitePrefixes

As you can see, passive reachability monitoring is performed continuously in the background. Also note that active probing takes place only after path reachability monitoring indicates Corporate Connectivity failure.

The Corporate DNS Probe Host Name is configured automatically by the UAG DirectAccess wizard and registered in DNS. This is registered in DNS as a quad-A (AAAA) record with the host name UAGDirectAccess-corpConnectivityHost and the IPv6 address ::1 (which is the local host address in IPv6). Note that the DirectAccess client only needs to resolve the name, not connect to the address mapping to the name, hence the reason why we use the local host address.

The UAG DirectAccess Wizard does not configure a URL for the Web probe active probe.

Determining the State of the DirectAccess Client IP-HTTPS Adapter

Whether or not the IP-HTTPS adapter is enabled depends on IP-HTTPS adapter initialization and the results of Corporate Connectivity detection:

If the IP-HTTPS adapter is initialized, Corporate Connectivity is confirmed, and the DirectAccess client has an IPv6 enabled adapter other than the IP-HTTPS adapter, then the IP-HTTPS adapter deactivates.
If the IP-HTTPS adapter is initialized, and Corporate Connectivity detection fails or there is no IPv6 enabled adapter other than the IP-HTTPS adapter, then the IP-HTTPS adapter remains active.

To understand how Corporate Connectivity for the DirectAccess client works, it helps to think about the adapters used by the DirectAccess client to connect to the DirectAccess server:

6to4 adapter. The 6to4 adapter is activated when the DirectAccess client is assigned a public IP address. (There are other requirements, but this is a key requirement).
Teredo adapter. The Teredo adapter is activated when the DirectAccess client is behind a NAT device, but deactivates when domain location determination identify that the client is on the internal network and can communicate with a domain controller. (There are other requirements, but this is a key requirement).
IP-HTTPS adapter. Used by the DirectAccess client when no Corporate Connectivity is detected.

What happens if the DirectAccess client tries to use 6to4 or Teredo to perform Corporate Connectivity detection? If the client is able to connect using either the 6to4 or Teredo adapter, and confirm Corporate Connectivity, the IP-HTTPS adapter is deactivated.

What happens if the DirectAccess client fails to confirm Corporate Connectivity using either the 6to4 or Teredo adapter? Then the IP-HTTPS adapter remains activated.

Note that IP-HTTPS activation may take place before Corporate Connectivity detection is complete. That explains why you will sometimes see both 6to4 and IP-HTTPS or Teredo and IP-HTTPS activated at the same time.

Putting Together the IP-HTTPS Pieces

Let’s summarize what we know so far:

The customer is running an IPv4 only network (no native IPv6 and no ISATAP).
The Outlook clients appear to be detecting a slow link between themselves and the Exchange Server.
The 6to4 adapter doesn’t initialize because the clients are assigned private IP addresses.
The Corporate Connectivity check is successful because the clients are able to resolve the Corporate DNS Probe Host Name that was automatically registered in DNS by the UAG DirectAccess Wizard.
The Teredo adapter initialized, but then is disabled because Corporate Connectivity was confirmed because of the clients were able to successfully resolve the Corporate DNS Probe Host Name.
The IP-HTTPS adapter is initialized and remains active because it doesn’t meet the requirements for deactivation (Corporate Connectivity was confirmed, but there are no other adapters other than the IP-HTTPS adapter that are IPv6 enabled).
Network Location Detection is successful as demonstrated by the fact that the Name Resolution Policy Table (NRPT) is not being used.

Now it’s clear why the IP-HTTPS adapter was still active – the customer was running an IPv4 only network. In order to deactivate the IP-HTTPS adapter, we need to confirm Corporate Connectivity and have an adapter other than the IP-HTTPS adapter assigned an IPv6 address.

What’s with the Slow Link Detection?

Now that the team solved the IP-HTTPS problem, the next step was to determine if there was a relationship between the active IP-HTTPS adapter and Outlook’s slow link detection. Maybe the DirectAccess client on the internal network was using the DirectAccess IPsec infrastructure tunnel to connect to the DirectAccess server and “loop back” into the corpnet over the DirectAccess IPsec infrastructure tunnel connection – maybe that’s why there was a slow link detected?

This turned out to not be the case because in order to establish the DirectAccess IPsec infrastructure tunnel from the DirectAccess client on the corpnet and the external interface of the DirectAccess server, you would need:

A path from the DirectAccess client on the corpnet to the external interface of the DirectAccess server. In our customer’s case, there was such as path, so we considered an IPsec tunnel between the Outlook client and the DirectAccess server’s external interface to be a possibility
Windows Firewall with Advanced Security firewall rules to support the DirectAccess IPsec connection
Windows Firewall with Advanced Security Connection Security Rules to support the DirectAccess IPsec connection

The second and third bullet points made it clear to the team that there was no traffic moving over a DirectAccess IPsec tunnel. The reason for this is that in order to establish the IPsec tunnels to the DirectAccess server, the DirectAccess client needs to enable either the Public or Private Profile in Windows Firewall with Advanced Security. When the Domain Profile is enabled, the firewall rules and Connection Security Rules required to establish the DirectAccess connection aren’t available – hence, even though the DirectAccess client could potentially connect to the IP-HTTPS listener on the DirectAccess server because there was an existing path, the IPsec tunnels could not be created – with the result that no data could move through the IP-HTTPS adapter.

The team confirmed that the Domain Profile was enabled on the DirectAccess clients on the internal network, which is consistent with the fact that the clients were able to connect to the Network Location Server and a domain controller. The reason why I say that this is consistent with being able to connect to the Network Location Server and a Domain Controller, is that these are these are the two requirements for activating the Domain Profile.

So, what was causing the slow link detection and causing the Outlook clients to use RPC/HTTPS instead of MAPI/RPC? From further investigations, it turned out that the IP-HTTPS adapter was reporting a link speed of around 100Kbps, slow enough for Outlook to decide that there is a slow link. At this point in time, we don’t have an answer for why Outlook decided to use this link speed value instead of the value provided by the NIC.

Summary

Let’s finish up with a summary of the technical issues that were encountered:

The DirectAccess clients on the internal network were able to confirm Corporate Connectivity because they were able to resolve the Corporate DNS Probe Host Name.
The 6to4 adapter is always enabled when the client has a public IP address. It is not enabled when the clients are assigned a private IP address, so the DirectAccess clients on the internal network had their 6to4 adapters disabled.
The Teredo adapter initializes when the client is assigned a private IP address, but then is disabled because of a successful Corporate Connectivity check.
ISATAP wasn’t being used on the network, so there was no ISATAP assigned IPv6 address.
An IPv4 only host that has no IPv6 enabled interfaces other than the IP-HTTPS adapter will not disable the IP-HTTPS adapter because the client must be able to confirm Corporate Connectivity and have an IPv6 address on an adapter that is not the IP-HTTPS adapter before it disables the IP-HTTPS adapter.
The IP-HTTPS adapter therefore remained active on the internal network DirectAccess clients.
The IP-HTTPS adapter reported a slow link speed, which is then used by the Outlook client to determine that it is on a slow link, which caused it to use RPC/HTTPS preferentially – resulting in users getting authentication prompts.
The NRPT is not in effect because Network Location Server detection was successful.
DirectAccess IPsec tunnels were not established between the DirectAccess client and the DirectAccess server because the clients on the intranet were able to connect to domain controllers and the Network Location Servers on the intranet, and therefore the Windows Firewall with Advanced Security Domain Profile was active. DirectAccess clients must use the Private or Public Profile to enable the firewall rules and Connection Security Rules required to establish the DirectAccess client/server link.

Discussion

There were a few reasons why we decided to share these experiences with troubleshooting the Outlook problem this customer encountered after deploying DirectAccess. The problem was solved by changing their DNS configuration so that internal network clients were not able to resolve the name of the RPC/HTTPS proxy –which made the initially observed problem with users needing to enter credentials go away. However, we still haven’t figured out why Outlook was using the link speed reported by the IP-HTTPS adapter, so that’s not the reason why I wanted to tell this story.

Instead, in the process of trying to solve this problem, the team ended up with a better understanding how the IP-HTTPS adapter works, and how it’s treated as the “IPv6 adapter of last resort” for DirectAccess clients. It also helped the team understand the importance of Corporate Connectivity detection, how the IPv6 transition technology interfaces are activated and deactivated, as well as the importance of Network Location Awareness and how that controls the Profile used by Windows Firewall with Advanced Security.

While all the lessons learned were good and valuable, this scenario called out a potential issue we might have with IPv4 only networks. Since we now know that DirectAccess clients on IPv4 only networks will not disable their IP-HTTPS adapter, the result is always going to be that the IP-HTTPS adapter is always be enabled on machines configured as DirectAccess clients on these IPv4 only networks.

What are the implications of having the IP-HTTPS adapter always enabled? It depends on how the network is configured. The team saw with the customer discussed in this article that the result was no traffic could flow out the IP-HTTPS adapter because Network Location Awareness determined that the DirectAccess client was on the corporate network because connectivity to the Network Location Server and a domain controller could be established. Therefore, the Domain Profile was enabled in Windows Firewall with Advanced Security and no IPsec DirectAccess tunnels could be established.

But what if we had the following scenario:

There is an IPv4 only network or the client is on a part of the intranet where there are no routable IPv6 addresses configured.
Therefore the IP-HTTPS adapter will always be enabled on DirectAccess clients
Also, suppose there is a failure to connect to the Network Location Server
The failure to connect to the Network Location Server would result in a failure to detect that the client was on the corporate network
The client then would be assigned either a Private or Public Network Profile in Windows Firewall with Advanced Security
The client would also would continue to use the NRPT, due to the failure to connect to the Network Location Server – so name resolution requests for most or all internal network resources would be forwarded to the DNS proxy on the UAG DirectAccess server
Private or Public Profiles enable the firewall rules and Connection Security Rules to enable DirectAccess IPsec tunnels to be established to the DirectAccess server
The corporate firewalls block outbound access to UDP 3544 (used by Teredo), but allow anonymous outbound proxy access, or even non-proxied access to TCP port 443 on the external IP addresses assigned to the external interface of the UAG DirectAccess server
The DirectAccess client establishes a DirectAccess connection to the UAG DirectAccess server and now is able to loop back into the corporate network by going out the corporate firewall and back into the network through the DirectAccess server. The response path would be the same path but in the opposite direction.
All DirectAccess clients on the internal network will connect to internal network resources through the Internet connection used by the DirectAccess server, which would have significant performance impact for all internal network hosts that need Internet access, even though who are not configured as DirectAccess clients

While it appears that it would take an unholy confluence of events to enable the DirectAccess client on the internal network to actually loop back through the UAG DirectAccess server to reach internal network resources, it’s likely that you’ve seen similarly unlikely events take place in the past.

This brings up the question as to whether or not you should enable this kind of scenario on purpose. For example, it’s possible that your Network Location Servers might become unavailable at some point in time, even if you have configured them to be highly available. If you enable the scenario described above, your computers configured as DirectAccess clients will still be able to connect to internal network resources, even though it will be through a very inefficient link. If you don’t enable the above scenario, then the DirectAccess client will not be able to establish DirectAccess tunnels to the UAG DirectAccess server, the NRPT will still be active, and the DNS queries sent to the UAG DirectAccess server’s DNS proxy will fail. The end result is that the DirectAccess clients will not have network connectivity if they need to access a resource by name.

The answer to that question is based on what scenario you prefer. Would it be better to bring your Internet connection to its knees for the period of time it takes to get the Network Location Servers up again? Or would it be better to “brick” your DirectAccess client computers until you can get the Network Location Servers going again?

To help you plan for such an eventuality, you can use the blog post When Good Network Location Servers Go Bad – Preparing Against NLS Failure at http://blogs.technet.com/tomshinder/archive/2010/04/06/when-good-network-location-servers-go-bad-preparing-against-nls-failure.aspx