Previously, throughput of an appliance was the only performance-numbers presented by CheckPoint and it was close to useless in terms of determining if the appliance is powerful enough to support your needs.
The reason for that was due to the fact the performance-numbers were based on lab conditions using only firewall security and one firewall rule. In the real world however, you get different types of traffic, different blades based on your need and most certainly a bigger firewall policy base – and all this summed up to you not giving a bullocks about the provided performance number and rather had an educated guess about which appliance was best suited to meet the needs of your customer.
Now there is a new benchmark metric in town and it is called SecurityPower! *Que awesome Rocky theme-like music*
The idea and concept behind SecurityPower is, without doubt, brilliant;
You basically use a calculator and say “Right, I’d like to run IPS, Application Control and VPN on my appliance, the management will reside on the box and I expect the throughput to be close to 1Gbps” and you are provided with a certain number of Security Power Units your appliance will need to have in order to achieve desired performance and through-put.
Although the calculator still has not been released, CheckPoints product specification documents is updated with a new line called SecurityPower near the old-timers “Firewall-, VPN- and IPS-throughput”.
We can for instance read that the UTM100 series has 50 SPU, while its big brother,the UTM3000-series, has close to 300 SPU.
Power 5000-series has 600SPU and the 11000-series has up 1200SPU.
Daddy 21400-appliance has 2900 SPU when installed with an acceleration card.
But even the 21400-appliance pales in comparison to CheckPoint 61000 Security System which can deliver anything from 3000 to 14600 SPU!
So we understand the metric will be a central component in choosing CheckPoint appliances, but how solid are their testing methodology?
CheckPoint has released some information on how the new testing is conducted and can be summarized as follows:
They run two setups, one standalone- and one distributed deployment.
The rule base is now 100 rules, all set to log.
They run Static/Hide NAT rules covering all internal hosts.
The testing use different traffic protocols such as HTTP (68%), HTTPS(10%), SMTP(13%), DNS(6%), FTP(1%), POP3(1%) and telnet(1%).
The results represent a CPU utilization of 50%.
They have the following configuration for blades:
IPS-blade enabled – pre-defined profile Recommended_Protection / Protect internal hosts only.
VPN-blades enabled – VPN-tunnel with one additional host – permanent tunnels enabled
2 Remote users connected running ICMP-traffic.
Mobile Access blade enabled – 2 Remote users logged in and generating activity in portal every few seconds.
URL-filtering enabled with up-to-date signatures and logging blocked requests
AV & Anti-malware enabled with up-to-date signatures.
– All traffic to be scanned based on file direction.
– SMTP in stream mode. Scan incoming and outgoing to DMZ and internal networks. Scan passing between internal networks
– POP3 in proactive mode, scanning incoming Pop3 to DMZ and internal networks.
– FTP in proactive mode, scanning incoming files to DMZ and internal networks.
– HTTP in stream mode. Scan incoming and outgoing to DMZ and internal networks. Scan passing between internal networks
Anti-spam & E-mail security blade enabled with Content based and IP reputation An ti-spam protection level set to “High” and logging for spam and suspected spam.
– Both POP3 and SMTP-scans are set to incoming mail to DMZ and internal hosts.
IA-blade enabled, with firewall rules with Access role objects for each protocol in the test-traffic stream.
App Control blade enabled with Default rule and monitor mode.
DLP enabled with provided Out of the box DLP-policy. Applied protocols is SMTP.
Although SecurityPower is, without a single ounce of doubt, a step in the right direction I do wonder if it will accurate.
For instance, how many firewall administrators can say they have users just connecting to the VPN-gateway, idling and sending the occasional ICMP-packet down the tunnel or have SSL VPN users just randomly fiddle around the portal when connected?
CheckPoint has neglected to mention anything about traffic within the VPN-tunnel.
Surely real-world scenarios with a lot of decryption / encryption of a packet stream will have some performance impact.
Nor have they mentioned anything about realworld traffic in terms of small UDP-packets (Be it ICMP or VoIP).
Some of you may think they are not included application control properly in their testing methodology, seeing how they run AppCtrl with only the default rule and monitoring. This testing should be sufficient as the big “performance impact” lies in determining the application and not matching the application in the policy.
In conclusion we’ll just have to wait and see how the calculator looks and wether or not the testing methodology will be updated.
And hopefully they will include desired VPN- and IPS-throughput as well as ordinary throughput – and in time, perhaps desired Application Control throughput.
All we know for now, is that this will be a helpful tool in determining which appliance your customer should go for.
A selection tool will be available in 2011 Q4: https://store.checkpoint.com/pricelist/selectPath.htm
Testing methodology: https://store.checkpoint.com/pricelist/TestingMethod.html