This is a question I get often : When implementing Deep Security do we (as the client) need to purchase vShield Manager or vShield Endpoint for the solution at extra costs ?

Well Simple answer is NO !

vShield Endpoint is for free with ESXi Standard, Enterprise and Enterprise + editions. Here is the link to the VMware ESXi Editions comparison site. I took a screenshot of the section of the page that indicated the inclusion of vShield Endpoint :

vshield

 

The next question is around vShield Manager. vShield Endpoint is a function of vShield Manager, thus you need to deploy vShield Manager in your environment. If you are entitled to ESXi and vCenter as a licensed product you will be able to download vShield Manager also. There is no charge for vShield Manager (That I know of!). If you need to use vShield App or Edge you will need to only purchase the needed number of those (App/Edge) licenses and you will receive vShield Manager also.

I was at a client yesterday that were nearly 100% virtualized, talking about our IPS and Firewall Solutions using Deep Security on ESXi. One of the topics we talked about was how to create firewall rules and what do you was them on if you have an environment that have multiple application communicating to each other. The easy answer for me was that the client could use VMware vCenter Infrastructure Navigator. I have been using this in my lab also to give me insight to what VM and application is communicating to what other VM’s. The information returned by Navigator is very useful in the sense that is presents you with a map of a VM and the following information :

  • Details of the VM
  • Services that runs inside the OS (Like SQL, Remote Desktop..)
  • Listing Ports and process that listen on those ports.

Below is my SQL Server in my Lab. As you can see there is 5 incoming connections on port 1433 and 1434. One can now start to create Firewall rules based on this information.

Ops Navigator

I changed the view to a “Table View” and by doing this the Incoming Dependencies and Outgoing Dependencies is clearly shown. You can now create the rules in such a way that you can define incoming  and outgoing rules.

Ops01

In the second part of this series I want to take the the time to explain how Agent-Less Security works using Deep Security with VMware ESXi and vShield Manager. Agent-Less Security have been out for a while now and at my days at VMware I did some testing with Agents installed and without. These tests were just in a lab with stop watches but even then we could see a 10-20% speed increase using Agent-Less(Note : TESTS WAS DONE IN LAB..NOT OFFICIAL).

So lets start of with how the implementation is done to use Agent-Less. From a Deep Security perspective you need the following software :

  • Deep Security Manager (DSM)
  • Deep Security Virtual Appliance – The DSVA is deployed as a Virtual Machine to each host. It is responsible for scanning all data that is send to it via the Filter Driver(below) to confirm for threats.
  • Filter Driver – The Filter Driver is installed to each ESXi host in the cluster. This is the driver that will interact between the ESXi network layer and DSVA. The Filter drivers “re-directs” traffic from the ESXi networking layer to the DSVA to scan for treats. It is also responsible for sending over the connection states of a VM to the destination host when a VM is vMotion to anther host.

From a VMware Perspective you need the following :

  • vCenter
  • vShield Manager – Is needed to install the Endpoint Agents to each ESXi host
  • (NSX – Deep Security 9.5 will support NSX also.)
  • Inside each VM you need to load VMtools and select the vShield Driver option.

The scanning for threats (Malware/IPDS/Firewall) is done by either disk or network access. Disk based treats is typically Malware or Virus type activity. Network Based threats are IPDS or Firewall related. Lets look at how each work differently and independently and how they are used to detect treats.

Disk based detection method

Malware and Virus are the typical disk based treats. For these to be detected we use the vShield Driver inside the OS to redirect the Disk IO hash to the DSVA for scanning. There is a few settings that will can affect the scanning in large environments. The typical Deep Security Policy settings to look out for is the following :

  • Scan Caching : Is used by the Virtual Appliance to maximize the efficiency of Malware and Integrity Monitoring Scans of virtual machines. Scan Caching improves the efficiency of on-demand scans by eliminating the unnecessary scanning of identical content across multiple VMs in large VMware deployments. A Scan Cache contains lists of files and other scan targets that have been scanned by a Deep Security protection module. If a scan target on a virtual machine is determined to be identical to a target that has already been scanned, the Virtual Appliance will not scan the target a second time. Attributes used to determine whether entities are identical are creation time, modification time, file size, and file name. In the case of Real-time Scan Caching, Deep Security will read partial content of files to determine if two files are identical. There is an option setting to use a file’s Update Sequence Number (USN, Windows only) but its use should be limited to cloned virtual machines.
  • Max Malware Real-Time Scan Cache Entries : Determines the maximum number of records that identify and describe a file or other type of scannable content to keep. One million entries will use approximately 100MB of memory on the DSVA.
  • Max Concurrent Scans : Determines the number of scans that the Virtual Appliance will perform at the same time. The recommended number is four. If you increase this number beyond eight, scan performance may begin to degrade. Scan requests are queued by the Virtual Appliance and carried out in the order in which they arrive. (Quoted the manual on these 3 information )

Network based detection method

Any traffic that would have to go through the vNic of the Virtual Machine will be redirected to the DSVA for scanning by the Filter Driver. Thus no traffic enters the Virtual Machine before it have not been scanned. The Scan Engine have two modes that it can operate in namely Inline and Tap Mode. Here is an explanation on how this works.

  • Inline : All traffic is scanned for treats as the scan engine is in Inline mode to the vNic. Stateful tables are also maintained in this mode.
  • Tap : All traffic is allowed to pass to the vNic but a copy of the data packet is send for inspection. The “live” packet is not modified at all. Any detection or modifications are done on the duplicated packet.

Below we can see a Virtual Machine status that is protected by an Appliance(DSVA) only :

Appliance

There is some rules around this so let me explain.

  1. You do not have to have the vShield Driver installed to be able to protect a Virtual Machine. If you don’t have the vShield Driver installed you will be able to do all Network based scanning but no In Guest. Thus no Malware (and no Recommendations scan with regards to the IPDS…Part 3).
  2. You can enable just Malware(Disk based) and no Network based scanning. This is Policy driven
  3. You can have Agent with Agent-Less. This means you have the Deep Security Agent installed and configured with Appliance based Protection. This is called Coordinated Protection. The rules here is that Agent based will always work first and if the Appliance detect no heartbeat from the Agent then the Appliance protection takes over.

Conclusion

Know how the different methods of detection works and how they apply in your environment. Test the settings that effect Scan Cache in your environment.

In the next part I will explain more around the IPDS function and what we can do with this module.

Monster VM’s and Business Critical VM’s is a very interesting topic especially from a design perspective. I have been reading a lot from Michael Webster and Sunny Dua about design considerations on these type of VM’s. You should check there blogs out as they really good points of reference.  What I thought I wanted to add on is from a Security perspective around adding Security to these VM’s. Topics that came to mind was around Patching and Firewalls and Malware and how to protect these VM’s with the minimal impact to performance and downtime.  The product that I will be referring to is Trend Micro’s Deep Security.

Agent-Less Protection

This is an important technology that enabled Agent -Less scanning of Malware (and a few other modules) inside a VMware Virtual Machine. If you think about it, if you run a Security Agent inside a VM it will consume CPU/MEM cycles from the VM’s assigned resources. Thus the VM’s resources is taxed with scanning for Malware or any other Security related functions. If this scanning is offloaded to the Hypervisor in some why then the VM will have more in guest resources available.

Intrusion Prevention/Detection

Most Application and Operating System vendors bring out security related patches on a weekly/Monthly bases. Most of the time to install these patches you need downtime of some sort to install and reboot if needed. What if this risk could be mitigated by scanning at the hypervisor for these threats and blocking them before they even get the the vNic of the VM ? Thus you know there is a network related exploit for you application but you cannot install as yet.

Firewall

Firewall at the vNic level. Thus you can setup rules for each VM from a single console and manage them with policies. In addition when there is malicious activities like port scans, OS Fingerprint or TCP Null Scans from inside or outside your network you will be alerted and you can block the traffic.

Over the next few weeks I will cover in a 4 part series how to Secure your Monster VM’s using Deep Security. I will explain how Agent-Less works and the benefits of not having in guest agents, how the IPDS works at the Hypervisor level and then explain the Firewall configurations that can be used for protection. All this is done by using Trend Micro Deep Security and Agent-Less Integration with VMware ESXi.

 

So what do I mean with this ? While I was doing the Webinar on Compliance and Backups I got the idea to see if I can protect a VM that is being recovered by Veeam’s Instant Recovery using VMware’s vShield Manager and Agent-Less AV ability. So lets look at the product set we going to use for this :

Deep Security

Deep Security have the ability to protect VMware Virtual Machines from Malware, Web Threats, Firewall and Intrusion Detection/Prevention using Agent-Less technology. Thus there is no in guest agent needed to protect the VM from all these threats.

Veeam

Veeam have this cool feature’s that is called vPower and Instant Recovery. Basically with vPower Veeam can mount the backup storage as a datastore to an ESXi host via NFS and then power up the VM that you want to restore from withing this NFS mounted backup storage. Once the VM is up and running you can then use a Storage vMotion to migrate the VM to a VMware Datastore.

VMware vShield Manager

VMware vShield Manager provided the ability to Deep Security to protect VM’s using Agent-Less technology.

The Use Case….

So lets put this all together as a use case. You have a VM that needs to be recovered that might have Malware on it. Or just a VM needs to be started up from backup that have out dated AV pattern files, or no AV at all. The basics here is that we will need to restore a VM fast and at the same time provide immediate up to date security (Malware/AV/Web/Firewall/IPS) protection to this VM. This is how I did it in my lab.

In my Lab I have my AD01-Server that is being backed up by Veeam. See below the VM screenshot in vCenter (all 5.5)

Inst01

This VM is being backup up by a task in Veeam on an daily basis.

Inst02

Next we have to create an Event Based Task in Deep Security that will enable the Agent-Less option and set a default Policy on any VM that is added to vCenter. The policy have the following settings set :

  • Activate Computer Delay = 0 min
  • Assign Policy = Lab Policy (See screenshot of my Lab Policy below)
  • Assign Relay Group = Default Relay Group (Or you relay group name)
  • Conditions : vCenter Name = *  (or if you have multiple vCenters the correct name)

Inst03

Some notes on the rest of the config before we go on :

  • You can define VM Names.. I just used a * for testing
  • You can select more Conditions, for my test I only needed a vCenter Name. I did not want to add Physical servers also.
  • 0 Min will activate immediately. If you want a delay you can add ‘x’ min and the task for activating the VM will be delayed for ‘x’ min.
  • My MGMT Cluster do not have Agent-Less security deployed and as such the AD01-Server has no protection.
  • I also have Deep Security installed and working in a VMware environment. Screen shot below of my environment with ESXi hosts prepared and DSVA’s deployed.

Inst04

Restore Time!

Next step is to do an Instant Restore of the AD01-Server that is in my MGMT-Cluster to a Resource Cluster called “Cluster01” and provide Instance Protection. From the Veeam Console I do an Instant Recovery of the VM to a different Location namely Cluster01 ->ESX01. Below Screenshot of the Instant Recovery we can see that the current host is MGMTESX01 (My MGMT Cluster host) and target host is ESX01 (Cluster01 host).

Inst05

I also set the VM to Power on Automatically with NO Network Connectivity. I want the VM to startup with no network and then do a Malware scan on the server using Agent-Less Malware Scan.

When the Veeam Instant Recovery Job starts it will create the new VM. The Event Based Task in Deep Security will detect the new VM and apply the base Policy to it. See below Screenshot of the activation process and once complete the immediate protection that is enabled.

Inst06

 

Inst07

From the ESX01 Host side we can see that the VM is started up from the VeeamBackup_VEEAM01 NFS Datastore.

Inst08

Now that the VM is running we can do our Malware Scan from Deep Security. This done from the Deep Security console.Inst09

 

To Recap what we did :

  1. Have a VM with no security (AV) protection that we needed to restore and do a full scan on
  2. Did a Instant VM Recovery using Veeam
  3. Have an Event based task in Deep Security that will detect need VM’s, Activate them and apply new Policy to protect them
  4. Started up the VM in a Network Disconnect state
  5. Was able to start a Malware Scan using no in Guest OS Agents
  6. All done in about 2min!

If you did not know, the Deep Security 9.5 Beta code is out. I have been waiting for this for a while now. One of the new features is the NSX support that we have. I did a vMug Session a while ago on this. Some of new features listed below :

vSphere 5.5 Support (9.0 with the latest SP does support 5.5 also)

  • Security for Software-Defined Data Center NSX for both Network Security features and Anti-malware features
  • Multiple Supported Deployment Models (vSphere 5.5 and vSphere 5.1)

Smart Agent

  • Minimal Installer
  • Security Modules install based on Security Policy
  • Automatic new Linux Kernel Support

Trend Micro Control Manager Enhancements

  • More Widgets with drill down
  • Full Events
  • Command and Control Communication Prevention

Linux Support

  • New Distributions: Cloud Linux, Oracle Unbreakable
  • On Demand Anti-Malware Scan for all Distributions
  • Real Time Anti-Malware Scan for Red Hat and SUSE

If you want to register here is the link : Register Link

 

Myself and Eric (from Veeam) are doing a Webinar on the 27 Feb 2014 on an interesting topic. This is all around Compliancy and the POPI act that will come into affect this year in South Africa.

For those that don’t know the POPI act is all around Protection Of Personal Information. During the Webinar we will cover the following topics :

  • What is the POPI Act
  • What is defined as Personal Information
  • Obligations to protect Personal Information
  • Challenges we see in this
  • How Veaam can help with these challenges
  • Veeam vPower
  • Veeam Instant Recovery
  • Veeam SureBackup
  • Product Demo

Please join us 27 Feb 2014 at 9h30 GMT for this event. You can register here.

Update (3-3-2014)

The Webinar have been posted here.

Sure by now everyone knows about vMUG. This is the second vMUG in Johannesburg. I am glad to say that I am presenting at this vMUG on Deep Security and VMware Design Considerations. This session is all about how to design Deep Security in a VMware environment. Topic’s include :

  • HA, DRS, vMotion and Storage Design Considerations
  • DSVA : How our Agent-Less security works
  • LogInsight Usage
  • And if we have time SRM and Deep Security

Other presenters are :

  • Veeam – Can your VM do this ?
  •  Intel – Next Gen Datacenter

You can click here to register for the event.

The last event was really awesome and we had a good turnout of people. Lets all support this event again !

I have been wanting to test this for a while now in my lab. Backups is always the easy thing to do…its the Restore that is important. So I got my NFR key from Veeam and loaded Veeam Backup and Replication. I am using vCloud Director 5.1 in my Lab. The test that I wanted to do was as following :

  • Create vApp with 2 VM’s. Set startup and shutdown order. Add some Metadata. Then backup vApp -> Delete vApp -> Restore vApp.
  • Verify all config is still the same after the restore.

First I had to add my vCloud Director instance to my Veeam backup server. This can be done as follow :

Click on Backup Infrastructure -> Right Click Manage Servers -> Add Server -> VMware vCloud Director

veeam01

Next we need to add the vCloud Director DNS Name, Credentials, vCenter Servers. Some notes on this :

  • Make sure you can resolve the vCloud Director DNS name correctly
  • The vCloud Director credentials can either be : veeam@system or just veeam. You will need to create an Administrator account in System for this to work. in my case the username is “veeam”.
  • The vCenter Servers that is connected to vCloud Director is discovered. If you already have them in your inventory list dont add them again. You will have a duplicated vCenter Server object

Once the vCloud Director Server have been added we can create a vApp with 2 VM’s to backup.  The vApp was configured as follow :

  • Set lease to expire on vApp to 7 Days (Default was 30)
  • Description – “Test vApp”
  • Starting/Stopping VM’s as follow:

veeam02

  •  Added a user with Read/Write Access Level to vApp : ShareUser
  • Added Metadata tag : TestMeta with value “Test Data”

For the VM’s I did the following:

  • Enabled Guest OS Customization and Specified a password
  • Enable Virtual CPU Hot add

Next I setup a backup job in Veeam to backup this vApp. When you create the backup job you have to select the vCloud option. I create the job as follow :

  • Name : vCloud vApp Backup
  • I added the Organization name as the object to backup. This will backup all vApp’s in the Org.
  • I selected the option to backup the job after the job have been created.

Once the backup was completed I deleted the vApp and started the restore. The restore process is rather simple. You either have the option to restore a vApp or VM. In my case I selected to restore the complete vApp. I added the vApp I wanted to restore and selected “Restore to original location”. You can select to restore to different location which allow you to select a new vApp Network and Datastore.

After the restore I verified that all of the above settings was still the same for the vApp. Screenshots below :

veeam03

veeam04

veeam05

Conclusions

  • Veeam Backup can restore the Metadata of the vApp and not just the core construct of the vApp
  • Integration with vCloud Director is really easy to setup
  • Ideal for Clients and ISP’s that offer vCloud Director Services and want to backup a Client (or Organization in vCD) Virtual Machines and/or vApps

 

One of the futures of Deep Security is to integrate Agent Less with the VMware ESXi Hypervisor. With the Use of vShield Manager EndPoint Protection this is possible. In this post I aim to explain how this integration works with the Deep Security components.
The requirements from a Deep Security side is as follow :

  • Deep Security Installation files ( download from here, select “Product Patch” to get latest patches)
  • Deep Security DSVA and Filter driver that is needed for VMware Integration ( download from here, select “Product Patch” to get latest patches)
  • vSphere 5.x
  • ESXi 5.x (Note ESXi 4.1 is supported. please check Installation Manual for specific’s)
  • vShield Manager 5.0, 5.1, 5.5
  • VMtools with vShield Driver option installed within the OS

Lets looks at how the Agent Less protection work. There is two basic methods that is used :

  • Malware / AV – This is disk IO based
  • Web Reputation /Firewall /IPS – This is Network based traffic

Malware / AV

Malware and AV activity is disk IO based. The disk reads and writes is “captured” by die vShield Driver installed inside the OS and passed to the DSVA for scanning. Once scanned the results is returned. The file that was scanned is either committed to disk or delete (if Malware was found).  The key here is that for Malware / AV Deep Security used the vShield Driver installed inside the OS.

Web Reputation / Firewall / IPS

All this traffic is network based and is detected by the filter driver that is installed inside the Hypervisor when the ESXi host was “Prepared” in the DSM console. The Filter driver will pass the traffic to the DSVA appliance for scanning and based on the rules that was applied for that VM via the policy will either allow or deny the traffic.

(Note that Log Inspection cannot be done by Agent Less. an Agent is needed for this. Read below on Coordinated Protection)

The key message for this is to understand that different Deep Security modules rely on different methods of of filtering.

The next question I get often is that can an Deep Security Agent be installed along side Agent Less. The answer is yes and this is called Coordinated Protection. Here is the rules around this:

  • The VM will be protected by the Agent installed inside the OS. If this OS Agent goes off line the Host based Agent will take open the protection.
  • There is no “Double Scanning”. The Host based agent(Filter Driver on ESXi) will allow all traffic to pass. (Until it have to take over)
  • It provides mobility for Cloud based VM’s. Thus you can move the VM to different providers and still keep the same security settings.
  • Allow for the implementation of Log Inspection.