I was busy testing some functions on a project that I am working on. I needed about 1000 VM to test this function on. So I wrote this simple Powershell script that will do the following:

  • Connect to vCenter
  • Clone my VM a few times (while I sleep)
  • Start the Cloned VM’s
  • Disconnect from vCenter

The VM that I used for cloning was a No-VMDK VM that booted from a ISO image. So the cloning process was not to slow as there was no VMDK disks to be copied. But I thought I would share the script I created:


# *******************************************************
# ** Written by : Hugo Strydom                         **
# ** Email : hstrydom@virtualclouds.co.za              **
# *******************************************************

#-----------------
#Login to vCenter
#-----------------

#Get vCenter Details : vCenter name, User, password
Write-Host "Please enter the vCenter Host Name :"
Write-Host " "
$vCenterName = Read-Host vCenter Host Name
$Username = Read-Host Username
$SecurePassword = Read-Host Password -AsSecureString

#Convert Secure Password to Plain Text
$PASS = `
[System.Runtime.InteropServices.Marshal]::SecureStringToBSTR($SecurePassword)
$PlainPassword = [System.Runtime.InteropServices.Marshal]::PtrToStringAuto($PASS)
#Connect to vCenter
Connect-VIServer -Server $vCenterName -User $Username -Password $PlainPassword

#Remove Password from Session
Remove-Variable PlainPassword
Remove-Variable SecurePassword
Remove-Variable PASS

#------------------
# Do Cloning Tasks
#------------------

#Define all the Variables for Clone VM

$DSKT = "DSKT-"                     # This defines the VM Prefixed that will be used for the name of the VM
$CloneVMName = "DSKT-001"           # This is the VM name that must be used for cloning
$ResourcePool = "VDI-Desktops"      # The Resource Pool where the new VM's will be placed
$DataStore = "DSCL-Resource"        # The Datastore name where the VM's should be pleased 
$FolderLocation = "VDI-Desktops"    # Folder where the VM's must be placed into
$x = 100                            # This the start suffix number that will be used for the VM name (thus DSKT-100 will be the first VM Name)

#-----------------
# Cloning Command
#-----------------

do
{
$NewVMName = $DSKT + $x

Write-Host "Creating new VM : $NewVMName "

New-VM -name $NewVMName -VM $CloneVMName -ResourcePool $ResourcePool -Datastore $DataStore -Location $FolderLocation
Start-VM -VM $NewVMName              # This will Power On the New VM
$x = $x + 1                          # Increment $x by 1
} until ($x -eq 1000)                # Can change the 1000 to own number. This defines the number of VM's that will be created
#--------------
#Clean up tasks
#--------------
Write-Host "Disconnecting from vCenter Server : $vCenterName"
Disconnect-VIServer -Server $vCenterName -confirm:$false

Before I start with this post…Disclaimer : This post is not to show how to hack systems. Rather it is to be used to learn on how to test your IPS Systems that you have deployed. Now that we have that out of the way…

I was giving training a while ago on Trend Micro Deep Discovery. During the training I ask the question if any one ever test their IPS deployments. The answers I got was that they hope it works…Sadly not a good answer…
So should you test you IPS rules ? My answer is YES ! You should as a Security Professional be able to do some basic tests to see if the rules that you have implemented is working. Simple.
I have some basic tests I do when I do implementations of IPS rules. The most basic one I use is MS12-020. This is a RDP exploit that is present in Windows 7 SP1 and Windows 2008R2. Here is the How To.

The How to…
I use KALI Linux for all my testing. The application inside KALI that I use is Armitage. Armitage uses the MetaExploit rule sets. It is really easy to install. The basics is that you download the ISO, install from the ISO, do an apt-update and then apt-upgrade.
For the OS I used Windows 7 SP1 with no Service Packs installed. Ensure you have RDP enabled but do not select NLM Authentication for RDP Sessions. Also ensure the firewall on the Windows side is disabled. Make sure you not using a production machine…if you run this exploit against an unpatch machine it will Blue Screen the OS!

On the IPS side I used Trend Micro Deep Security. I ensured that I have added the rule set for MS12-020 to my VM as seen below:
MS12-020
Next from KALI I start Armitage, search for MS12_020. Make sure you double click on the Auxiliary ->dos->windows->rdp->ms12_020_maxchannelids exxploit. This will open a window. All you have to do is to enter the IP address of the OS that you want to run the exploit against. Click “Launch”. You can see in the console window in the back that the exploit was run against the host.
IPS Test01
If all goes well you should not have a OS Blue Screen and you should see a event in the IPS events of the VM as below:
IPS Test02
As you can see the IPS Module in Deep Security have done a RESET on the connection and have blocked the exploit.

Conclusions
Learn how to test IPS Systems. It took me a while to learn the basics around using the application and tools sets that is out there. YouTube is your friend in this matter. There is a lot of free tools that you can use….KALI Linux being the best out there that I know of. With regards to IPS systems, I use Deep Security as I know the product well (Use to work for Trend Micro). Happy Testing!!

I am sure by now everyone knows about CVE-2014-6271 aka “Bash Bug”. This article will explain how to protect against this vulnerability by using Trend Micro Deep Security. Deep Security Agents can be deployed in two ways depended on the environment. The first is by using an In-Guest Agent. The other option is Agent-Less which is only available on VMware Hypervisors. Regardless of the type of agent you have this IPS rule will protect your OS against this vulnerability. Note that Deep Security IPS Agents is Host based (Not Perimeter). Thus each Host/OS will have this protection enabled if you apply this IPS rule.

Adding the IPS rule to your Base Policy
In my lab I have a top level base Policy named HomeLab Policy. I added the following IPS rule to this Policy :
Shell02
Once you have added the IPS rule you will see that it will also add the HTTP Protocol Decoding Rule set. Once done you should have two additional rules. See below :
Shell01
Things to consider

  • For the IPS rule to be enforced you must place the Policy in “Prevent” mode (Intrusion Prevention Behavior)
  • You can apply the rule to individual VM’s manually or by doing an “Recommendation Scan”.
  • In my Lab I applied the rule to my top level Policy, thus ensuring all OS’s will get this rule applied regardless of the OS type.

Conclusion
Clients that is using Deep Security with the IPS module can use this IPS rule to provide protection until such time that they can install the needed patches in the OS’s.

I created a Resource Cluster in my Lab a while ago. The use case for this is to place all my Desktop OS VM and vCloud Director Resources into this Cluster. The issue I started to see is that most of the Virtual Desktops would have an Yellow exclamation mark and the vCPU Mhz usage would be 0 Mhz.

This initial did not bug me at first and ignored it as i was not using my Desktops that much. But I was doing a POC on Deep Security as a Service where I used my Lab to connect to the ISP Deep Security Instance to test some policies. During the testing this worked well…but the next morning I got connection errors from these Desktops. So back to my Desktops and I could see they had this Yellow Exclamation error. Initially I thought there was a issue on the ESXi host :-) but that was not the case.

Troubleshooting the event logs the following stand out :

VMStandby

As you can see the VM have entered Standby Mode. After change the Windows 7 Power options not to go into Standby this problem went away.

In additional to this I also changed my Win7 Template Power state settings.

I also found this KB article on VMware View and PCoIP disconnects that is related.

This topic comes up a lot on our internal and at client conversations. How to ensure the DSVA is available ? For those that don’t know, some points on the DSVA :

  • DSVA = Deep Security Virtual Appliance
  • One DSVA per ESXi host
  • Is used for the offloading of the Malware scanning (the basic function)

Below is some design considerations that I would have taken into account  in a VMware environment to ensure the DSVA is available and alerts is send out. Some of these options involve placing the host into MA mode, some restarts the DSVA and some just send out an alert.

DSVA Heartbeat

The DSVA have a default policy that you can modify. The options to review is on the DVSA Heartbeat back to Deep Security Manager(DSM). These can be found as below :

DSVA_HA_01

This policy works as follow using the options above pointed out in Red :

  • The Heartbeat interval is set to 2 mins
  • Number of Heartbeats to miss before alarms is raised in DSM is set to 2
  • Thus if the DSVA have missed two Heartbeats you will get an Alarm raised in DSM. Emails can be setup to send these alarms to administrator.

vSphere Alarms

vSphere Alarms can be setup to monitor if the DSVA power state. This have happened that the DSVA is powered off accidentally. By creating and Alarm that monitors the DSVA power state with an action to power on again one can assure that the the DSVA is always powered on. Below screenshots of the Alarm that can be created :

DSVA_HA_02

DSVA_HA_03

HA Monitoring

VMware HA Monitoring can also be used to ensure the DSVA is available. HA Monitoring uses the VMtools Heartbeat from inside the DSVA to determine if the machine is still responding to in OS heartbeats. You can setup rules for the DSVA to have a High (30 sec) monitoring setting. Thus for High if the Heartbeat is not received 3 times in a 30Sec interval the VM will be restarted.

Just a design consideration, when you enable HA Monitoring it is done for all the VM’s in the cluster. you will need to disable for the VM’s you don’t need it.

Performance Stats and vCOps

The DSVA is not disk intensive but can be Memory and CPU intensive if not sized correctly(The Deep Security Best Practice Guide have these sizing guidelines). You can use vCOps to monitor the Performance to gain insight to the usage of CPU and Memory over time. The Risk tab can provide guidelines on these metrics.

vSphere alarms can also be configured to trigger when the DSVA CPU and Mem usage is over 80%. This could be an indication that the DSVA VM sizing should be re-looked at.

Powershell Script to place Host into MA Mode

It is possible to create a Powershell Script that when a DSVA is Off the vSphere Alarm will run this script and place the DSVA into MA mode.

Key takeaways from this

There is many options to ensure the DSVA is available or to send alerts if there is an issue with the DSVA. It will form part of the design that you should review the availability options and implement as part of the design options that will fit the client and the design needs.

The above options is for the architect to consider and made aware of. Again, all my views and considerations I would have taken into account.

In the last 2 weeks I have been ask by a few clients about the memory allocation for the Deep Security Manager(DSM). There is two things to this :

  • Memory allocated the the DSM Operating system (Lets assume Windows for this blog)
  • Memory allocated to the DSM JVM

DSM Operating System Memory

Depending on you deployment size and modules that you will be using the DSM OS memory should be around 8-12Gb. The DSM service is a JVM so if you running DSM inside a VM(vs Physical server) you can consider to set reservations (I am not advocating the use of reservations…but do read up on JVM in a virtual environment).

DSM Memory configuration

The DSM JVM by default will have about 4Gb memory allocated. You can verify this in the DSM console. The Memory Metric that you looking for is “Maximum Memory. My DSM Server have 8.5Gb memory configured and as you can see only 3.56Gb available to DSM :

DSM_Mem_01

To increase the memory allocated to DSM do the following :

  • For Windows the default Installation folder is : C:\Program Files\Trend Micro\Deep Security Manager.
  • In this folder you need to create the following file : “Deep Security Manager.vmoptions” (I have tested and the case of the letters does not matter)
  • Edit the file and add the following line : -Xmx8g (Just open the file and add this line, then save..nothing more..)
  • The above will increase the Memory allocation to 8Gb memory. (If you want more just change the 8. Make sure your OS have more also)
  • Stop / Start the DSM process.

In my lab the new DSM Maximum Memory is now the following :

DSM_Mem_02

Takeaway points

  • Know that the DSM JVM memory can be increased
  • Typically when you have a lot of IPS Recommendation Scans you would increase this setting, again, based on your environment.
  • Monitor this setting. You will also see the following alerts in the DSM logs : “he DSM has reached the maximum memory that’s been allocated for the application”.

When I started with this “Home Lab Storage Upgrade Project” of mine I did no have the HP VSA in mind at all. I was looking for “something” that would give me better speed/io response times from a disk perspective. Not so much space (Space I can get easily with larger disks). Looking at my Lab config I did not realize at the time was the number of HP Equipment I have in my Home Lab. Here is just a few :

  • HP DL 380 G7 Server (74gb memory installed) – Management Host/Cluster
  • 2x HP Micro servers – my “storage units”
  • 6x HP Dual Port Intel Nic’s

So back onto the HP VSA. As part of the vExpert Program we can get a HP VSA NFR Lic  for 3 year for free. I started chatting on email to Calvin Zito (@HPStorageGuy) on some of the options of the VSA. I was looking into using the Adaptive Optimization option. Basically using tiering in your disk pool by using SSD drives and other type disks. In my case I used SATA disks and had 1x 120Gb SSD drive available to use.

The HP Microserver that I had in mind to use already had 4x 500Gb 7200rpm drives installed. Thus all the bays were full. I realised that the MicroServer had a CD-Rom and a seperate SATA connection. I mounted the SSD drive on a bracket and placed inside the CD-Rom bay. Now I had 4x 500Gb Sata Drives and 1x 120Gb SSD drive.

The VSA is installed on ESXi. I my case I used 1 of the 500Gb drives from my ESXi install and location of the VSA Virtual Machine.

The VSA Manual is very specific on how to configure the disks. You need to start adding the VSA VMDK storage on SCSI 1:0 and the mode needs to be Independent and Persistent.VSA_01

I have allocated all the storage, Powered up the VSA and added the Disks to the VSA Cluster. I only have a single Node Cluster in my configuration (The NFR Lic that you will get is a 3 node Lic).

I created 3x 400Gb Luns as Tier 1 Storage. I defined my SSD storage as Tier 0:

VSA_02

Next was to enable my Luns to use Adaptive Optimization (Tier 0 Storage, SSD) :

VSA_03

Here we can see that the Luns are using the Tier 0 Storage : VSA_04

On my VMware Cluster side I have 2 hosts in the cluster that I use for my VDI Desktops(Not using VIEW…just cloning Desktops). I created a Storage DRS Pool that only used Disk Space optimization so that the space on the Luns can be used “equally” and saves me the effort to decide where to place the next VDI Desktop. Below Storage DRS doing its thing very nicely!

VSA_05

Fom me the use case was to give some increase performance to my VDI environment as I use the VM’s a lot for testing of Trend Micro Software. My initial tests on cloning and installing application was that the speed did improve and that I spend less time waiting in my lab :-)

I had a few questions over the last few months on Deep Security Deployment options, Sizing and General Configuration options. I thought that I would publish the link to our Trend Micro Deep Security Best Practice Guide. The guide have been recently updated to reflect all the new enhancements.

Here is an extract of the Guide :

Deep Security provides a single platform for server security to protect physical, virtual, and cloud servers as well as
hypervisors and virtual desktops. Tightly integrated modules easily expand to offer in-depth defenses, including anti-malware, web reputation, intrusion prevention, firewall, integrity monitoring, and log inspection. It is available in agent-less and agent-based options that can all be managed through a single console across physical, virtual, and cloud server deployments.
This guide is intended to help users to get the best productivity out of the product. It contains a collection of best
practices which are based on knowledge gathered from previous enterprise deployments, lab validations, and lessons learned in the field.
Examples and considerations in this document provide guidance only and do not represent strict design requirements. The guidelines in this document do not apply to every environment but will help guide you through the decisions that you need to configure Deep Security for optimum performance.

The guide can be downloaded here.

As part of the vExpert Program we get some free NFR Lic from vendors. I must say I never actually did took advantage last year of all the NFR Lic that we get on the vExpert Program but this year that will change. One of the Companies that gave us a NFR lic is Devolutions. They gave us a 1 year Remote Desktop Manager License. I have been looking for a Remote Desktop Manager that will give me more that just RDP session management. My “Business Needs in my Lab”, I f I can call it that, was to have session management on the following :

  • SSH (Putty)
  •  Radmin
  • FTP/SCP
  • TeamViewer
  • VNC
  • and RDP..

What I also saw on the list is X Window and will have to try that out as some point.  I took a screenshot of some of the Visualization sessions that is supported. Yes, VMware Console is also listed and needs PowerCLI to be installed

RDM

If you are on the vExpert Program and want a free 1 year NFR License have a look at the vExpert Communities page as the details is there.

This is a question I get often : When implementing Deep Security do we (as the client) need to purchase vShield Manager or vShield Endpoint for the solution at extra costs ?

Well Simple answer is NO !

vShield Endpoint is for free with ESXi Standard, Enterprise and Enterprise + editions. Here is the link to the VMware ESXi Editions comparison site. I took a screenshot of the section of the page that indicated the inclusion of vShield Endpoint :

vshield

 

The next question is around vShield Manager. vShield Endpoint is a function of vShield Manager, thus you need to deploy vShield Manager in your environment. If you are entitled to ESXi and vCenter as a licensed product you will be able to download vShield Manager also. There is no charge for vShield Manager (That I know of!). If you need to use vShield App or Edge you will need to only purchase the needed number of those (App/Edge) licenses and you will receive vShield Manager also.