Here is an interesting problem with DNS service. I have a new domain with domain controller running Server 2008 R2. I’ve noticed that the server is using more memory than expected. The DNS service is using 605MB of RAM which way too much.

I’ve checked on the other domain which have Server 2003 R2 domain controller and DNS service on these domain contoller are using ~20MB.
After spending some time reading forums and blogs I’ve found that theis is related to KB953230 .
Here is a nice blog about that KB http://msmvps.com/blogs/acefekay/archive/2009/09/03/the-dns-cache-poisoning-vulnerability-microsoft-kb953230-patch-and-ports-reservation-explained.aspx

The amount of memory consumed is relatated to the amount of UDP ports opened which is 2500.
If you have IPV6 and IPV4 running on the host then there will be 5000 ports opened, so simply by disabling IPV6 the memory usage dropped to 300MB but that is still too high.
Also the more cores you have the more memory will be consumed(don’t know how DNS  is related to CPU cores). In my case I have 2 quad core CPUs and also Hyper Threading enabled. So simply by disabling Hyper Threading in the BIOS the memory usage dropped by half to 150MB.
The last option is to set the SocketPool size from 2500 to a lower number and this will drop the memory again but I don’t think that is a good option from a security point as the less port you have the bigger the risk.
here is the command to lower the SocketPool to 1000 ports:
Dnscmd /Config /SocketPoolSize 1000

I’ll keep digging to find more information about this problem.

Update: OK here is a small update regarding this issue. I’ve been in contact with Microsoft support and was told that the DNS service works as designed. By default there are 2500 random UDP ports assigned for DNS. As I already wrote because 2008R2 comes with IPv6 enabled out of the box then we have a total 5000 UDP ports allocated. Now memory has to be allocated for UDP packets per each logical CPU. So the more logical CPUs you have the more memory DNS.EXE will use.
As it stands the solutions are as I already wrote:
1. Disable IPv6 if not used.
2. Disable Hyper Threading (I don’t think there is much use of it on domain controller)
3. If the server is not facing the Internet then you can reduce the SocketPool size