Avoiding Proxy Port Exhaustion. Symantec Blue Coat ProxySG.
Let’s go troubleshoot or avoid port exhaustion on Proxy. This post is about Symantec Blue Coat ProxySG product.
Well, in an usual proxy deployment there are two main options regarding the way that Proxy connects to outside. It is able to send traffic with the original client IP address, or it is able to send traffic to outside with its own IP (virtual or not) which is basically a source NAT.
From my point of view, the second way is the most extended, in fact, it ensures traffic incoming will follow the same return path, and do not go directly to the client (bypassing Proxy). Anyway, in case we want to know what is the original client IP address, there is another method we can use called x-forwarded-for e.g.
Now, taking the second way as more usual, all of client originated traffic will leave from proxy with only one outbound natted IP. Every connection that clients will do, will leave with the same outbound natted IP but with different port.
Symantec Blue Coat Proxy has this settings by default:
tcp-ip inet-lowport 49152
tcp-ip tcp-randomize-port enable
tcp-ip tcp-2msl 120
As you may know there is 65.535 ports available (which the first 1024 are reservated) so keeping in mind the default configuration, there are 65.535 – 49.152 = 16.383 ports available per outbound natted IP address in a Symantec Blue Coat ProxySG in order to manage outbound sessions.
It manage this 16.383 ports in a random selection and the time that a session will be in a time wait state will be by default 120 seconds.
That´s it that a Symantec Blue Coat Proxy is able to reach 16.383 concurrent sessions (in a 120 seconds period) by default per outbound natted IP. In case our environment (because of the number of users or the sessions volume) generates more traffic sessions that 16.383 we will face a port exhaustion problem.
What to do if that is our case? Well, we can perform several config changes I detail below:
Increase the range of available ports:
tcp-ip inet-lowport "value" > between 1024 and 49152.
For example, if you set 1024 as the value, you will have 65.535 – 1024 = 64.511 ports available per outbound natted IP.
Disable randomize port:
tcp-ip tcp-randomize-port disable
It ensures that Symantec Blue Coat Proxy will use the available ports sequentially ensuring it will use all of them one by one, not randomly.
Decrease the session time wait period to close sessions before:
tcp-ip tcp-2msl "value" > between 1 and 120
It decrease the time that a sessions still being in a time wait state (keeping an available port used till the session end) So as soon as the session is closed, an used port is released and available again.
Add a new outbound natted virtual address:
It consist in use, not only one, but two or more outbound natted IP addresses. So doing that, you are doubling the available ports as you are doubling outbound natted IP used by Symantec Blue Coat Proxy.
It consist in two steps in VPM policy:
Configure a new Virtual IP (VIP).
Menu > Configuration > Network > Advanced > VIPs > New.
Enter the new VIP and > OK > Save.
Configuring Policy to reflect those outbound natted IP addresses:
Add or edit a Forwarding Layer.
Select LAN users range as source in the Source Column.
Create a reflect-ip object in Action column pointing to the new VIP.
Save and install.
In CPL policy will be as follows:
<forward> client.address=192.168.20.0/25 reflect_ip(10.10.10.1) client.address=192.168.20.128/25 reflect_ip(10.10.10.2)
Imagine you have a LAN network 192.18.20.0/24. In that case you can e.g. subnet your LAN into two subnets and sends one subnet with one outbound natted IP and the other subnet with the other outbound natted IP.
Well, these are four methods to avoid port exhaustion on a Symantec Blue Coat ProxySG. Of course all of them are related and must be reviewed together, likewise using them is under your own risk and you must to have a deep understanding of your environment in order to fine tune these settings at your convenience.
