HOWTO setup @Veeam Cloud Connect Backup

Introduction

This is the first on a series of blogs on how to setup and configure Veeam Cloud Connect.  We will go through the basic steps of, first of all setting up a Cloud Repository for Backup to the Cloud and then we will look into Cloud Connect Replication.

In this, the first post we will look at how to configure Cloud Backup.  Specifically, the steps which you would go through in order to, from a technical perspective, set yourself up a a Cloud Backup Provider, or Veeam Cloud Service Provider (VCSP).

What is Cloud Backup?

Being able to backup to the cloud is a useful way of conforming to the 3:2:1 rule for backups.  I have being working with many different backup vendors over many different years and the 3:2:1 rule is an essential part of any backup strategy.  For those, not in the know, the 3:2:1 rule goes as follows:

• 3 copies of your data
• 2 types of media
• 1 of which is off-site

So, for a ‘classic’ backup configuration you would create a primary copy of your data (in Veeam terminology a ‘Backup Job’) which will be stored in local storage close to the source.  For your typical SMB this will probably be a HP DL380 or Dell R720 server with sufficient disk space, whereas larger customers would look at something more like a Cisco UCS C3160.  The advantages of this is that you have backs close to the source on fast media which can be restored quickly if required.  Features such as Instant VM Recovery (The ability to bring up a VM direct from a backup file) would use this form of storage.

However, what would happen if your site was flooded?  You would lose not only your production storage, but also your backup storage too.  This is where the ‘2’ and the ‘1’ come in.  You create a secondary copy, a ‘backup of a backup’ or ‘Backup Copy Job’ in Veeam terminology to a secondary site which has some physical separation from the primary. So, if you lose your primary site due to a disaster you still have your data.  But what if you do not have a secondary site?  Well, this is where solutions like Tape and the Cloud come in.  Tape has it’s advantages: It’s cheap, very portable but tapes need to be tracked and can break.  Cloud provides most of the advantages of tape but without the disadvantages. i.e. it gives you physical separation but without worrying about losing it.  Veeam supports both and both have their place.

Veeam Cloud Connect allows a service provider to provide existing customers with a cloud repository which they can backup to (or more realistically backup copy to) to fulfill this offsite requirement.  This data can be recovered at any time and allow customers to recover their data in the event of a site failure.  For the service provider it provides a revenue stream as they can charge customers for how much data they are storing (quota) and for how long they are storing it (lease).

We will now look at the steps in configuring yourself as a Veeam Cloud Service Provider (primarily from a technical perspective) and consider some of the sizing considerations.

Procedure

There are two parts to the configuration:

• The End-User with an on-premesis Veeam solution
• The Service-Provider with the Cloud Repository

Important in this consideration is the fact that, from the end-user perspective, the cloud repository is just like any other repository, with the odd exception, features such as Instant VM-Recovery are not supported.  Of course, this is fairly reasonable, the cloud repository is not really there to provide the primary backup copy but is very much utilised as a secondary backup.

The key steps involved in configuring Cloud Connect for Backup are as follows

1. Install and configure the Service Provider (SP) Veeam Backup Server
2. Setup SSL certficates
3. Create one or more Cloud Gateways
4. Configure one or more Cloud Repositories
5. Register Tenant Accounts
6. Connect the Tenants

We will now look at each of these in turn.

1. Configuring the SP Veeam Backup Server

This is a Windows Server  with the Veeam Backup Server Software installed which is running the Veeam Cloud Connect Service.  Think of this as a Veeam Backup Server which is dedicated to the provision of Veeam Cloud Connect Services and is not used for local backups.  Let us go through the steps for installation:

1. Obtain a Veeam Cloud Connect (or Hosting) License. Note that this is not the same as that for a Veeam on-premise backup server license.  This is a server which will be used for Cloud Connect, NOT on-premise backups (take a look at the following KB article if you want to do both: https://www.veeam.com/kb2139).  Importantly, this is licensed per managed VM. This is defined by the number of VMs with restore points within the last 31 days and is owned by the service provider.  It is then up to the service provider to bill the tenant for the resources uses (primarily lease and quota).
2. Install the Veeam Backup Server in the usual way.  The is just the normal installation process, just select the Cloud Connect License when prompted.  Once you have installed this you should see some thing similar to the following:

A few words on sizing here: The requirements are somewhat lighter than that for a standard Veeam Backup as it will not be controlling local backups, so although it will still need a 64-bit Windows OS it should be fine as a VM with 2x vCPUs and 4GB RAM for both the Veeam Backup Service and the MS SQL database.

 

2. Set up SSL certificates

An important concept to grasp with Veeam Cloud Connect is that we do not need to setup any dedicated VPN tunnels.  All of the communication between the tennants and the Service Provider is performed using SSL encryption over port 443.  This greatly simplifies the set up but does require an SSL certificate.  There are three options for doing this:

1. Generating and using a Veeam Backup and Replication self-signed certificate. This is quick, easy to do and can be performed using the Veeam Backup Console.
2. Select an existing SSL certificate from the Microsoft Windows certificates store.  Here a certificate is selected from the local certificate store of the server.
3. Import an SSL certificate from a file in the PFX format.  This is the recommend method and uses a certificate signed by a trusted CA.

Let us quickly go through the steps required to generate an SSL certificate in PFX format as this is the most likely scenario for a Service Provider.  In my example I have used openssl in Linux with my own CA certificate.  The relevant commands which I issued are as follows:

openssl req -x509 -nodes -newkey rsa:2048 -keyout ca.key -out ca.cer -days 1095

openssl req -newkey rsa:2048 -nodes -keyout veeam.key -out veeam.csr

openssl x509 -req -days 1095 -in veeam.csr -CA ca.cer -CAkey ca.key -CAcreateserial -out veeam.cer

openssl pkcs12 -export -in veeam.cer -inkey veeam.key -out veeam.pfx

Note the last command.  This is because openssl by default uses the x.509 .cer format but Veeam requires the file in .pfx format.

Once you have obtained this enter the Cloud Connect View:

Once here you will need to click the Manage Certifcates button in the middle of the screen:

Ensure that the Import certificate from a file is selected and click Next.  You should receive the following:

Browse to the location of the .pfx file and then enter any password if you have entered one. Next!

Check the summary to ensure that everything is OK and click Finish.  Note that if this certificate cannot be verified you will receive the error message Failed to assign certificate to the Cloud Connect Service:

This was rectified in my system by ensuring that the correct CA certificate was installed and that the name was resolveable. It is worth mentioning at this point that, in the real world, the tennants which connect to you will need to trust this certificate so it is probably best for them to be signed by a root CA.  If not then you should copy and paste the Thumbprint and send it to the tennants so they know that they can trust it.

3. Create the Cloud Gateway(s)

The Cloud Gateway is network service which routes traffic from the tennant backup server and the service provider’s cloud repository.  The service must be installed on a Windows platform such as Windows 2012R2.  As compression and deduplication is all performed on the tennant side it it not particularly resource hungry and 1GB RAM should be sufficient for around 2000 concurrent connections, though an additional 1GB should be provisioned to allow for the requirements of the underlying OS.

In the real world a service provider will probably deploy a number of these and they will automatically load-balance between themselves providing high-availability.  This is something built-in to the cloud gateway service and does not need an external load balancer in order to achieve this.  Although DNS Round Robin can be used it is only to balance the initial connection.  Once a tennant’s back-up service connects to a cloud gateway it automatically receives data on all available gateways and may be instructed to use others which have more spare resources.

The steps to create the Cloud Gateway are as follows:

In the Cloud Connect view click the Add Cloud Gateway link.  You should see the following page:

Note that in my case the Veeam Backup Server is selected.  In the real world you would probably select a different windows server for this role.  Note the port, this is used for the intial connect.  Next.

The networking configuration needs a certain degree of thought.  If the server has a publically routeable IP address then the top option, This server is connected directly to the internet, should be selected.

If you are behind a NAT device then you must enter the unique public IP for the Gateway.  Note that each cloud gateway needs to have its own public IP address, regardless of whether it is directly configured on the gateway itself (direct) or with a firewall in front of it (NAT).

Here, we are just using the backup server on the same network so we are just going to click Next.

After a prompt that the Cloud Gateway service will be installed click Next to begin the process.  You should eventually receive the following:

Click Next and then Finish to complete the installation.

4. Configure the Cloud Repositories

If you are familar with Veeam Backup and Replication then this step should be fairly straightforward, though there are some important considerations.

Choosing the repository

Although you have the same choices as for an on-premesis solution you would normally want to go for either a Windows or Linux repository.  Why? Well, the tennant may encrypt their data at source which would not be good for a deduplicating storage appliance. Also, if you use an SMB share you cannot deploy a Veeam data mover directly but have to go through a Gateway appliance.

Note that at the time of writing, you cannot use a Veeam Scale Out Backup Repository as a Cloud Repository.

Sizing the repository

From a disk point of view this should be done in a similar manner to a standard repository with thoughts on number of tennants, how much data they wish to backup and for how they wish it to be retained.

Compute considerations are also fairly straightforward.  A single active job uses one core plus 2GB of memory plus an additional 175MB for each TB backed up (assuming the default block-size). These figures will need to be scaled up for the amount of data and number of tennants with sufficient overhead for the OS.

Architecting the repositories

There are two broad approaches to this:

The first is what is referred to as a Pod design. Here, the ‘pod’ is a single, fixed-size repository with limited or no expansion capability. Customer data is located on a single pod and will require manual intervention when space becomes exhausted. Expansion here is achieved by adding more pods.

The second design is a Single Namespace Scale-out Design. Put simply, this is a single, large repository, where space is added dynamically, in the backend when needed. As Veeam’s own Scale-Out Backup Repositories are not not allowed, as yet, for Cloud Connect then a third party solution is required, such as Windows Storage Spaces. The key point here, though, is that a single namespace is presented to the customer and any storage expansion is done in the background, transparent to end-users.

Basic procedure

Create a backup repository in the normal way.  i.e. in Backup Infrastructure view click Add Repository:

 

Select the type:

Select the server and location (note that if the path does not appear click Populate):

At the respository step, review items such as concurrent tasks and adjust depending on resources:

Configure any mount server (usually the mount server or repository):

The nen click Next again followed by Finish.

If in doubt, click Yes when asked about the configuration backup location.

5. Register Tenant Accounts

In order for clients or ‘tennant’s to be able to backup their data to your cloud repository they must be first registered.  To do this you must launch the New Tennant wizard.  The easiest way to do this is by going to the Cloud Connect view and clicking Add Tenant:

Specify a username and password, for Cloud Connect for Backup tick Backup storage (cloud backup repository) (We will cover Replication in a later post) and specify the contract expiry time (Lease).  Note that the options on the left change with what is selected under Assigned resources. Next

If desired specify a network throttle. Next

At the Backup Resources step you need to select your backup repository (created in the previous step), give it a Cloud repository name and assign the user quota. If you have a WAN accelerator you can assign that too. Click OK and then Next when prompted.

At the summary screen review the options (note the option to copy the password to the clipboard – useful if you had it generated) and click Finish.

You should see the tennant under Cloud Connect -> Tenants as shown:

6. Connect the Tenants

This final step is very straightforward and is performed by the tenants themselves at the client side.

Importantly the tenant needs to know the following:

1. Username and password for the created account
2. Full DNS name or IP address of the Service Provider’s cloud gateway
3. (For self-signed certificates) Certificate thumbprint
4. (optional) External port to connect to if not the default

At the tenant side

Type in the IP address of name of the Service Provider and click Next.

Here we need to install the certificate and add the credentials.  As we are using a CA-signed certificate we do not need to verify it with a thumbprint.  Just click the link next to certificate to view it.  In our case we see the following:

Click OK on Certificate Information and then, on the Credentials page click Add to input the credentials for the service provider.  Once input click Next.

After a brief pause you should see the following:

Click Next.  You will then arrive at the following:

Click Next and then Finish when prompted.

Browsing to the Backup Repositories under the Backup Infrastructure view should show the new backup repository as type Cloud as shown below:

Congratulations, you have just installed and configured Veeam Cloud Connect for Backup!  The tenant can now send backup and backup copy jobs to the cloud repository just like any other repository.

Summary and Conclusion

We have shown how to setup Veeam Cloud Connect for Backup and how to create and register tenants.  The main steps are as follows:

key steps involved in configuring Cloud Connect for Backup are as follows

1. Install and configure the Service Provider (SP) Veeam Backup Server
2. Setup SSL certficates
3. Create one or more Cloud Gateways
4. Configure one or more Cloud Repositories
5. Register Tenant Accounts
6. Connect the Tenants

In our next post we will look at configuring Veeam Cloud Connect for Replication and the ability to issue full- and partial-  failover to the service provider cloud.

 

 

 

 

 

 

Creating an On Demand Sandbox for Storage Snapshots using @Veeam Backup and Replication v9

Introduction

Version 9 of Veeam Backup and Replication. which came out in January, brought in a number of exciting new features.  Many of these were publically announced prior to v9 release and are now well known.  Out of those which were not announced prior to launch, On Demand Sanbox for Storage Snapshots is one of the more significant features as it enables users to quickly spin up a test environment directly from the snapshot of a supported storage system rather than having to use either a backup file or a replica.  This is very useful for both testing and development purposes.  In this post we will look at how to setup and configure this for a simple configuration running on NetApp storage.

You Will Need

• An Enterprise Plus licensed version of Veeam Backup and Replication
• A VMware host
• A supported storage system. At the time of writing these are:
  • NetApp DataONTAP
  • EMC VNX or VNXe
  • HPE StoreServ or StoreVirtual
• SAN connectivity between the Veeam Proxy and the underlying storage with permissions correctly assigned

Procedure

1. Configure the storage

In this post we will not go into the configuration of the underlying storage in any great detail.  I am using an iSCSI LUN with NetApp but any other supported system will work just as well.  In any case, if you have a supported storage system chances are that you will be using Backup from Storage Snapshots so the storage side of the configuration should already be in place, with the Veeam proxy having access to the LUN and the correct permissions configured in the storage world, so you can probably skip this part.  For my environment I have added the IQN of my proxy server to the initiator group on the NetApp LUN and have also ensured that the proxy has the Microsoft  iSCSI initiator configured correctly.

 

2. Add the storage system using the Veeam Backup Console

Again, if you have already configured Backup from Storage Snapshots you can safely skip this step.  If not, then navigate to Storage Infrastructure and then click ADD STORAGE.

After clicking Add Storage you should see the following:

 

You can see the supported storage systems.  Click the relevant system and then fill in the required information. In this example, selecting NetApp gives us the following:

Fill in the IP information and then click Next:

At this point click Add to enter the required admin permissions then click Next followed by Finish.  You should then see something similar to the following:

You should be able to see your storage device with any presented LUNs.

3. Configure the snapshots

At this point we have Veeam Backup and Replication configured with our associated integrated storage appliance.  As already mentioned, if you have already configured BoSS this is your actual starting point.  Whilst the underlying storage system can be configured to create snapshots directly we will do this step within Veeam (after all this is a post on Veeam rather than NetApp!).  It is worth considering that, just as you can configure a backup job to create a primary backup on a local, fast disk, you can also create a backup job to create a snapshot.  As for a normal backup, Veeam will perform all necessary application-aware processing but the advantage here is that you can then also configure a secondary destination (i.e. a backup copy) to then perform an actual backup.  This gives a tiered approach to your backup strategy by providing a very fast restore point on a local snapshot but also backing up the data in a traditional sense which can protect you against RAID or controller failures or even the loss of the entire site, providing you have an off-site copy.

To begin the process click the big Backup Job button on the top left of the Console to start the New Backup Job wizard:

Give it a name and then click Next:

Here, select a virtual machine which is located on your storage appliance in the normal way and click Next.

Now this is where it gets interesting.  Rather than selecting a backup repository, select (in the case of NetApp) NetApp SnapShot. Note that you can also configure secondary destinations (i.e. backup copy jobs) too and in the real world this is probably what you would do.

Click Next

This is the page where you configure application-aware processing.  Whilst this is not mandatory it makes the difference between creating a crash- and application- consistent snapshot.  The latter gives seamless integration with Veeam Explorer for Storage Snapshots which we will have a quick look at during the next step.  Configure this in the usual way, as for a normal backup job then click Next followed by Create.

Once you have created the job, right-click it and select Start.

 

4. Explore the snapshots

OK, this isn’t exactly part of the procedure but a) It is a good way to check the snapshot process has worked and b) It gives us a good excuse to show how we can recover individual files and items from storage snapshots.

Navigate to Storage Infrastructure and expand the LUNS.  You should see something similar to the following:

We can see our snapshot and the VM within it.  Right click the VM.  You should see a menu similar to the following:

As you can see, you have the option here to perform a number of different restores, including Instant VM Recovery, file-level recovery, or even application-level recovery in a similar manner to traditional backups.  The latter option is what is generally referred to as Veeam Explorer for Storage Snapshots and could be used to say, recover a database table or an individual email.  Remember, this is all agent-less – very cool!

At this point we could continue and restore some items but we will now move on and create an on-demand sandbox.

5. Create an on demand sandbox

If you have already played with SureBackup in the past then this should prove to be a very familiar process.  The steps are:

1. Create an application group
2. Create a virtual lab
3. Tie it all together with a SureBackup job.

5.1 Create an application group

To do this, navigate to Backup Infrastructure and click Application Groups:

As you can see there are a few here already.  Click the big Add Group button at the top, then select VMware:

Give it a name and click Next:

If you click Add you should see a menu asking you where you wish to add the VM from.  The top two entries are for SureBackup and SureReplica but we need to click From storage snapshots:

As you can see, the snapshot we created is visible here so expand it, select the VM and click Add.

Once the virtual machine has been added, click Edit and then, if necessary, select the role.  Mine is a web server. Note that, for SureBackup, this is where we configure the tests.

Note that we could now add other VMs to the group, especially if they worked together, such as a 3-tier web app.  For now, click OK and then Finish when prompted.

5.2 Create a Virtual Lab

The purpose of a virtual lab is to create an isolated environment for the virtual machine to run in. Still in the Backup Infrastructure view click Virtual Labs then click the big Add Virtual Lab button and click Vmware:

.

Give the lab a name and then click Next:

Select the host you wish to run the lab on and click Next:

Here you will need to select the datastore for the metadata. Next:

Here you will need to click Configure to set the external IP address of the proxy appliance (think NAT box) used to isolated the virtual lab from the real world.

Next:

Here you always want to select Advanced single-host as it allows you to customise the network configuration. Next:

At this stage you will need to click Add and then specify the network mapping.  You will need to map the port group in the virtual lab to the port group in the production environment.

This requires some explanation.  What we have configured here is the IP address of the proxy within the virtual lab (in this case 192.168.244.1) and the Masquerade IP address on the outside, production, network.  Remember that the VMs will be started in the virtual lab with their original, production IP addresses.  This is fine as the proxy appliance will isolate them from the production world but we still need to connect to them.  This is done using the Masquerade IP which is changing the third octet in the IP address in our example from 244 to 245.  This means that the proxy appliance will be listening on 192.168.245.0 and will translate packets it receives destined to this subnet to the 192.168.244.0 subnet.  A temporary static route will be placed on the backup serve, routing traffic to 192.168.245.0 to our virtual appliance at, in our case, 192.168244.53.  The upshot of which is that our virtual lab can now be contacted from the production environment.

Next!

We can, if we wish, do a static NAT translation.  My VM has the IP address 192.168.244.152.  What I have configured here is a translation, so that the proxy will listen for and forward on to my VM in the virtual lab any packets destined for 192.168.244.150.  This means that I can now contact the VM in the virtual lab without any need to a static route – cool.

Click Next and when prompted Finish.

5.3 Create the SureBackup Job

OK, coming in to land!

The (almost!) final part of this task is to create the SureBackup job.  This effective ties everything together and enables us to place the application group into our virtual lab.

Go back to the Backup and Replication view and then, in the Home tab click SureBackup Job:

Give it a name and click Next:

Select the virtual lab. Next!

You will need to select your application group from the pull down menu.  Now, very importantly, click Keep the application group running after the job completes.

Click Next a few more times, to complete the wizard.  You may want to configure notifications or a schedule but that is very much up to you.

You should finally arrive at the following:

I have selected Run the job when I click Finish. Click Finish to start the job and create the on demand sandbox.

You should be able to view the progress of the job by clicking on Jobs -> SureBackup and clicking the SureBackup tab.  You should see something similar to the following:

You can gain more detail by double-clicking the job:

You should also be able to view the process from within vSphere:

What we are looking at here is the Virtual Lab and the copy of the VM from the snapshot running within it.

We can also test it.  The VM is actually just a little webserver with the IP address of 192.168.244.152.  Now, we configured the proxy appliance to translate this into 192.168.245.152 (see above).  If, on my VBR server, I open up a browser and point this to that address I should be able to access it within the virtual lab:

Fantastic, it works! We have just configured an on demand sandbox from a storage snapshot.

Summary

Configuring an on demand sandbox from storage snapshots is very similar to performing the same task from backup jobs but only now you can use your integrated storage appliance, saving both time and space, allowing a users to quickly and efficiently fire up a testing environment leveraging the performance of their production arrays.

 

 

 

Remote Powershell Scripting in @Veeam Backup and Replication v9

Introduction

Version 9 of Veeam Backup and Replication was released in Janurary 2016.  In this series of blogs we have been looking at some of the cool, yet mostly unknown, new features of version 9.

In this post we will look at the new enchancements to the Veeam Powershell Snapin.  The major one here is the ability to execute Powershell commands remotely without being logged on the Veeam Backup Server and without executing a Remote Powershell Session.  The way this works is that the Powershell Snapin is installed automatically when you install the Veeam Backup Console.  This is a big departure from earlier versions when this was an optional component which needed to be selected upon install.

Procedure

On a machine with the Backup Console installed, open up a Powershell Window and type the command: Add-PSSnapin VeeamPSSnapin:

We now need to use the command Connect-VBRServer to connect to the Veeam Backup Server.  The syntax for this is:

Connect-VBRServer -User <username> -Password <password> -Server <IP address or name>.

Once you have connected you can now issue cmdlets on the backup server.  For example:

Congratulations, you can now script and automate tasks to your hearts content!

Another additional feature in version 9 is the ability to restore MS SQL databases using powershell Cmdlets.  For more information on this see the following link: https://helpcenter.veeam.com/backup/powershell/sql_item_restore.html

 

 

1-Click File Restore using the @Veeam Enterprise Manager

The Veeam Enterprise Manager is an optional, free tool which can be used to manager multiple Backup Servers.  Put simply it is a web-based interface to your data protection environment, allowing you to edit and run existing jobs.   It also adds additional functionality such as delegated restores, a REST API, role-based access, encryption password loss protection and, in the context of this post, 1-Click File Restores.  Most of these features require an Enterprise license and the REST API an Enterprise+ license but since the vast majority of Veeam installations are one of these this should not be a problem.

1-click restore enables users to quickly browse or search for a deleted item and then quickly restore it.  With version 8 this required the guest to be backed up with Guest File System Indexing enabled:

One of the very cool, but widely unrecognised new features in v9, is that this is no longer required for browsing, only searching.  File system indexing gives an additional overhead in terms of the backup so is not generally enabled during the backup as file and object restore from the Backup Console has never required it.  With version 9 it is not required for the Enterprise Manager either.

To perform a file restore using the Enterprise Manager.  Naviate to HTTPS://<name or ip or server>:9443 then login and and click the FILES tab.  You should see the following:

At this point we need to select the virtual machine we wish to restore from so enter its name in the textbox or click the link or pick VM from list and select the relevant VM.  You should see the following:

The VM I have selected does not have indexing enabled.  With version 9 we can mount the backup file directly, using the mount server on the repository so select Mount Backup.

After a short delay and some messages describing the progress you should eventually see something similar to the following:

Expand the tree and select the file you wish to restore.  In the example here we will navigate to the FONTs:

Select the file you want to restore and then right-click it:

We have a number of options here.  Restore will simply restore the file to it’s original location (hence the name 1-click restore) but for now we will simply click Download.  After confirming that you really want to download the file you should see something similar to the following:

The file is now ready to download to you local machine.  Ensure that the LOG tab is selected and then just click the download link near the bottom of the screen.  If this is not available you will need to ensure that the restore entry in the list has been selected. After clicking this link you should receive the following diagolgue box:

Save the resulting ZIP file to a convenient location and then double-click it to open it.  We have the following:

Which shows the following when opened:

Now, let us just recap what we have done here.  Using Enterprise Manager we hae successfully restored a file quickly and easily using a web browser using 1-click restore.  With version 9 we did not need to enable indexing on the backup job in order to do this, as the v9 mount server mounts the backup file directly without this need, allowing us to browse the backup job.  Note that this is all from the same agentless, image-level backup of the virtual machine.  Better still, Enterprise Manager is available at no extra cost and the 1-click restore availability is included with the Enterprise license, which most customers already have, making this effectively a free tool which can greatly easy the restore of individual items without the need to use the Console.

 

 

vSphere Tag Backup and Restore with @Veeam Availability Suite v9

Introduction

This is the second of a series of blogs on some of the cooler, yet more unknown features of the Veeam Availailability Suite v9.  Here we will be looking at the backup and restore of vSphere tags.

 

What are vSphere tags?

Introduced in vSphere 5.5, a tag is a label that you can apply to a VM, or indeed any object, in the vSphere inventory.  By creating a tag you can assign it to a category which allows you to, say, tag your virtual machines by guest OS and then create a category called “Operating Systems”, is applied only to virtual machines.  You could then tag your VMs as “Windows”, “Linux”, etc.  You could also create a category called RPO and assign different tags depending on the required backup schedule.

 

So how does Veeam use tags?

Well, since version 8 of the Availability Suite, Veeam supports and understands vSphere tags.  This is very useful when creating backup jobs as you can select the VMs to backup based on tags, with VMs being automatically added to the job when they are assigned a particular tag.  At first glance this appears nothing special but applied correctly this lends itself to a policy-based data protection strategy.  For a more thorough discussion of this see Luca’s excellent whitepaper available here: https://www.veeam.com/wp-advanced-policy-driven-data-protection-using-vsphere-tags.html

With version 9, Veeam has also added support for backup and restore of these tags in addition to using them to dymanically select VMs.  You can think of the tags as just being backed up as any other VM property.  When it comes to performing a full VM restore you have the option of whether to restore the tags or not as shown:

Conclusion

Veeam’s support for vSphere tags ensures that not only can they be used for policy-driven data protection, but that they are also protected with the backup and can be restored as wel..

 

 

BitLooker -HOWTO exclude files and blocks from Backups in @Veeam v9

1. Introduction

In January this year Veeam release v9 of their Availability with a raft of new features which can be found here: https://www.veeam.com/availability-suite-features.html

However, in addition to the publicized ‘big’ features such as Cloud Connect Replication and EMC support for Backup of Storage Snapshots (BoSS) there are also a number of other features which equally cool and add functionality to the Veeam Availabiilty Suite.  The first of these which we are going to look at is BitLooker.  In this, fairly short Blog entry we will look at what BitLooker is, why do we need it and then have a look at how we actually configure it.

2. So what is BitLooker and what is it for?

Put simply, Bitlooker enables dirty blocks, swap files and even user-specified files and folders to be excluded from the image-level backup of the VM.  So, we are still talking about an image-level backup but within this, the backup proxy will exclude certain files and blocks from the backup, saving both storage space and bandwidth for the backup, which has to be a good thing.

 

3. How to configure it

Veeam is inherently intuitive and easy to use so it should come as no surprise that the BitLooker feature is realatively straightforward to use as well.  Note that if you are upgrading from v8 then it will be disabled by default (no point in changing an existing job) but will be enabled for new jobs.

I am going to create a new backup job on my VBR server to backup the machine called webserver1.  Note that for this to work the guest OS must be running.  Now in this VM which I am about to backup I have a folder called stuff which contains the following files:

Now, what I want to do is backup the VM but NOT the mp3s.  This is how I do it:

1. Create the backup job.  Just create the job in the usual way but let us pause at the step where we select the VM:

Notice that it is powered on.  Very important otherwise this will not work.

2. After you have selected the machine and clicked Next you should obtain the following:

3. You should click the Advanced button here and then navigate to the Storage tab:

Note the options Exclude swap file blocks (recommended) and Exclude deleted file blocks (recommended). We can see that BitLooker is enabled by default on this new backup job.

Before we move on to the next step let’s have a look at what is going on here.  By swap file we mean pagefile.sys and hiberfil.sys, the swap and hibernate files respectively.  The deleted blocks are blocks on the VM which contain data for files which have been deleted by the guest OS, sometimes called dirty blocks.

4. After clicking OK and Next you should eventually arrive at the Guest Processing window:

This is a very important part of the backup process because, although Veeam backs up whole virtual machines, it also ensures that any applications within them are backed up in an application-consistent state and can therefore be restored at the application level too.  For example the restore of an email or the retore of an Oracle database, all done through the one, image-level agentless backup.  This is a topic for a later date but is worth keeping in mind.

5. Check the box Enable application-aware processing and then click the Applications button:

6. Highlight the VM and then click Edit and click the File Exclusions tab:

Click the radio button next to Exclude the following files and folders and then click Add and at the prompt type *.mp3 to receive the following:

7. Click OK a couple of times to return to the Guest Processing window and then add in the relevant Guest  OS credentials.  You should be at the following:

8. Almost there! Click Next, configure any schedule and click Create when prompted to give the following:

Either run this later or do what I have done and click Run the job when I click Finish and click Finish.  The backup job will start.

 

4. Browse the files to verify the exclusions

Start the Guest File (Windows) restore wizard and then select the latest restore point and then in the BACKUP BROWSER navigate to the stuff folder.  You should see something like the following:

 

Notice that there are no mp3s to restore as they have not been backed up.  We could have used wildcards to select other files in addition if we had wished to but note that the number of entries in the list influences the job performance — the more files are included or excuded from the backup or replica, the more time Veeam Backup & Replication requires to process these files.

 

5. Conclusion

With v9 of the Veeam Availability Suite it is possible to exclude blocks, swap files, specific files and folders or even files matching a particular wildcard pattern. This results in reduced backup storage and time and enabled users to filter out non-business oriented files.

 

Global Load Balancing using the @Riverbed SteelApp Traffic Manager

Global Load Balancing allows the continuation of service for organisations with large, distributed infrastructures whilst at the same time ensuring users are directed to the site most appropriate to their location. For example, imagine a company with two large sites, one in York and the other in New York. For users accessing their web-site from the UK they will be automatically directed the the web-server in York, in the North of England. Users on the Western side of the Pond can be directed to the server in New York. Better than that, if one site goes down then all users can be redirected to the other. So, we have the benefit not only of intelligently being able to direct users to their most appropriate site but also of redundancy and continuity of service.

So, how does SteelApp achieve this? Well, fundamentally, this is done via the Traffic Manager intercepting DNS requests, determining the help of the sites and the location of the user and then making a decision as to where the request will be directed.

Will will now look at the configuration steps involved in setting this up. We will be using a demo environment, running on VMware Workstation on a laptop, though the steps will be pretty much the same in a real world scenario.

Procedure

First of all you need to have DNS and your services configured. In the case here I have two, non-clustered traffic managers both serving the same site (actually it really it exactly the same site) but with a small traffic script response rule indicated which site served the request. We have a URL of http://www.bredon.net and this is being served by two traffic managers, one with an IP address of 192.168.186.10 (York, via STM1) and the other with an IP of 192.168.186.14 (New York ,via STM2). We have a DNS server configured such that these two IP addresses are delivered in a round robin fashion. This last point is important.

Now, if I point my browser to http://www.bredon.net I will obtain one of the following:

      or    

 

The traffic script rule is placing the name of the traffic manager in the title so you know which one is handling the request.

OK, now let’s configure GLB for this.

The first step is to create the locations.  This is done under Catalogs > Locations.  We are going to add two locations here, York and New York.  Simply enter the names and then click Add Location, you should see the following:

We will not specify a location here, though in the real world you should do this.  This is because of SteelApp’s GeoIP database which can look up the location of the client based on IP address.  Do this for both York and New York to receive the following:

Next we need to create monitors for these locations.  Navigate to Catalog -> Monitors and create a HTTP monitor for each location as follows:

 

Note that the scope needs to be Pool/GLB and do not forget to include the port.  This must be done for each location you are load balancing for.  In the example here I also created one for New York but with the IP to monitor of 192.168.186.14:80.

OK, now we need to create the Virtual Servers. Navigate to Services > Virtual Servers and create a Virtual Server as shown below:

Note that I have already created a Pool containing my DNS server on port 53.  Click Create Virtual Server to create it.  You will see the following:

Don’t forget to enable the virtual serer!  You now need to click GLB Services.  This will take you to the catalog where you will need to create a GLB service by clicking Manage GLB Services in catalog:

Create the new GLB service by specifying the locations and the domain that you are load balancing for then click Create GLB Service:

In the page that follows do not forget to Enable it and then click Locations and Monitoring:

Ensure that you have specified the IP address of the service local and the correct monitor from the pull-down menu and then click Update.  This must be done for each location.

Now, go back to your GLB Service and select Load Balancing.  You should see something similar to the following:

Now, let us pause here.  In the real world you would probably want to select Geographic.  However, this is a demo environment so we will do this as Active/Passive. As a native Yorkshireman, I will select York here by clicking Set master next to York.  You can also drag and drop to place the master on top.

Ensure that you are pointing your DNS clients to the SteelApp DNS virtual server you created earlier and then enter the URL of your service, http://www.bredon.net in our case.  You should obtain the following:

The Master site is York, so this is working successfully.

To test it, disable the main site.  In our case we did the following:

In the case of our test environment, I simply returned to the browser and refreshed the page:

As you can see, we are now receiving the page from New York – it worked!

In the real word we would be using Geographic load balancing rather than simple Active/Passive which would also serve pages locally and then fail over to a remote site if the local one was down with real geographic awareness in addition to failover.

 

Summary

Global Load Balancing, provided through Riverbed SteelApp Traffic Managers provided not only location awareness for content delivery, but also the ability to fail over to other sites if the local one is not available.  With web services and applications becoming increasingly essential to the operation of the 21st century business this is a feature that not only enhances performance but provides resilience, serving the global business in the digital age.

 

 

 

 

 

 

 

Optimising encrypted MAPI and SMB-signed connections using @Riverbed SteelHeads

Not many years ago, installation of Riverbed SteelHeads was very straightforwards.  You plugged them in, ran the jumpstart wizard and, hey presto, you were optimising all important traffic on the network.  In recent years, however, more and more configuration is required in order to optimise modern applications.  Today, MS Exchange requires an encrypted connection and Riverbed Professional Services is quickly becoming much more lucrative due to the need to perform often lengthy configurations to optimise today’s more complex protocols which Microsoft seem to be updating on an almost daily basis.

Here we will look at the steps required to optimise both encrypted MAPI and also SMB-signed and SMB3 traffic.  The steps are pretty much the same for all of these so once you have configured one you will have pretty much configured the rest.  There is also a very neat little auto-auth wizard now available on the SteelHead which nearly does it all for you, making the process somewhat simpler than it used to be.

 

Procedure

Firstly you will need to be running RiOS version 8.5.3 or later.  I am currently recommending 8.5.3  to my customers as it is stable and supports most of the current protocols they would likely want to optimise.  If you are running version 8.6+ then the steps are pretty much the same but this may change when v9 is released.

There are three options for configuring encrypted optimisation.  These are:

1. Delegation Mode

2. NTLM Transparent Mode

3. End-to-End Kerberos authentication

All three of these require joining the server-side SteelHead to the domain.  Furthermore, you do not want to be using delegation mode any more and need to be looking at end-to-end Kerberos authentication.  This falls back to NTLM Transparent in any case so it is this which we will be looking at.  We will describe in detail the manual configuration steps, these are not too onerous and pretty much guarantee a working configuration.

1. Join the Server-Side SteelHead to the domain.

In order to do this you must ensure that you have prepared the following:

• DNS.  Ensure that the SteelHead has the root and all child-domains specified as search suffices as well as a valid DNS server (obviously).  The join is all done using the SteelHead’s Pimary IP address so that should also be configured in DNS.
• NTP. Both NTLM and Kerberos are time sensitive.  For Kerberos, the SteelHead’s clock must be within 30 seconds of the DC’s clock so make sure you have this configured.
• Make sure you have an account of sufficient privileges so that the the SteelHead can be joined as a RODC.  This will usually be a domain admin but does not necessarily need to be one.
• Know your AD structure!  This is the hardest part in many ways.  If you have a mult-domain forest I would recommend joining the SteelHead to the root domain.

Provided that you have done the above you should navigate to Configure -> Networking -> Windows Domain.  Here you should fill in the relevant fields and, unless you are in a really old network select Active Directory integrated (Windows 2008 and later).  This works well with Windows 2012 domains and adds the SteelHead as a Read Only Domain Controller (RODC).  Note that the SteelHead will not service logins, this is purely so that it has the permissions needed to obtain certain keys which are needed to optimise SMB-signed and encrypted MAPI connections.  Once you are here fill in the field as shown below:

 

Click Join and you should see the following:

If you have issues at this point then navigate to the lower part of the screen and try running some of the built-in diagnostics:

Note that I left the Domain Controller Name(s) blank in the above page.  From experience it is nearly always better to let DNS work these out for you.  Ironically, specifying these can cause more problems than solutions.

Once you are satisfied that the SteelHead is part of the domain (and a RODC no less!) then go and make yourself a cup of tea and get ready for the next step.

2. Configure the Replication User

This is needed for End-To-End Kerberos authentication to work correctly.  First of all create a user in AD and make sure that the password is both secure and does not expire.  Then, for every domain that you have a server in which you need to optimise with you must first right-click, as follows, and select Delegate Control :

This starts the following wizard:

 

Over the next few steps, select the user you created earlier and click Next.  Once you have selected the user you should arrive at the following window:

Ensure that you have selected to create a custom task and then click Next.

Choose the option as shown above and click Next.

Choose Replicating Directory Changes and then scroll down and also select Replicating Directory Changes All.  Ensure that you have selected both of these and then click Next.  You should see the following summary:

Examine this to ensure that you have select both options and then click Finish.

Importantly, you need to do this for every child-domain for which you have a server (Exchange or signed-SMB) which you will need to optimize with.

 

3. Add the Replication User to the server-side SteelHead

Navigate to Configure › Optimization › Windows Domain Auth and scroll to the Replication Users section.  In the below example the replication user is replicating for the following domains:

• scalby.sborough.net
• newby.sborough.net
• ayton.sborough.net

Click Add but note the following: We are using the wildcard to specify all sub-domains that we have configured in the earlier step for the replication user.  This is important or it won’t work!

3. Enable Optimisation features

This is the easiest part, just ensure that you click Apply when needed and save your config.  You will need to configure SMB signing only on the server-side Steelhead but MAPI encryption need to be enabled on both:

And, for Exchange:

You will need to restart the service after configuring these so you will have to do this out-of-hours for a production network.

4. Verify Connections and Perform Troublshooting

Navigate to Reports -> Networking -> Current Connections.
You should see the following:

Hoorah – you have done it!  The SMB2-SIGNED label indicates latency optimisation of SMB2-signed connections.  Time for a cup of tea…

5. Troubleshoot if Necessary

If you are unlucky you might see the following

Note the red triangle.  This means that there is a protocol error and the SteelHeads are not giving you latency optimsation (though SDR/LZ will still be reducing the traffic).  Don’t worry, this does not always work first time.  Click the magnifying glass next to the connection for a clue as to the problem and then go to the following page:

If the tests here prove successful restart the service on both SteelHeads and try again, eventually you should eliminate the source of the problem and the annoying red triangle.  The systems logs are also very useful here.

 

Conclusion

Encrypted MAPI and SMB-signed connections are now relatively straightforwards to configure and any issues are usually easy to troubleshoot provided the server-side SteelHead has been successfully joined to the domain.  Note that if you have sites with local servers, then the SteelHeads here will also need a domain join as they will be acting as server-side SteelHeads for these connections.

If you want to use the Domain Auto Auth feature then simply create a standard user with a password which does not expire and then add this user as a replication user on the server-side SteelHead.  All you need to do next is run the Auto Auth wizard with domain credentials, specify the protocols you wish to optimise and the wizard will do the rest, including assigning the appropriate replication rights to your specified user.  Things have come on a long way since RiOS 6.0…..

 

 

@Riverbed Application Performance Platform in the digital age – some thoughts

We are going through a digital transformation. Only a few years ago IT existed to replace paper and facilitate administration. We wrote letters on word-processors rather than typewriters, sent emails rather than paper mail, advertised and displayed information about our business on the internet rather than in the newspaper, stored information in databases rather than filing cabinets but our basic workflows were pretty much the same as they were 20 years earlier.

In the meantime we had a whole generation being brought up with technology, texting, tweeting, using apps, talking over Facetime and Skype, a generation that was now entering the workplace and more importantly a generation with a disposable income. This new generation, used to working digitally came expecting to be able to do at work what they could do at home. This ‘generation Y’ expect to be able to access their work data as easily at home as they can at work (ATAWAD – Any Time Any Where Any Device). This has had a number of consequences:

1. Increasing demands for mobility. I want to access corporate information wherever I am on whatever device I have.  This means data and applications need to be available via HTTP via any device, browser or app which needs to access it with the corresponding complexities in our applications and delivery mechanisms in order to achieve this.
2. Increased use of video and other collaboration tools. This just leads on from the previous point. Think about it, with mobility comes flexible working but human beings like to see each other when they are talking (a lot of communication is non-verbal) so collaboration has become much more important
3. IT becoming part of the business solution rather than simply a replacement for paper. In the 1970’s the businesses of our manufacturing based economy relied on machines to run their business and if a particular widget-making machine stopped working for a day we potentially lost a day’s worth of productivity (lots of £££s!). Now it is increasingly our IT systems which our businesses rely on, we lose our critical applications for a day and, again, we have lost lots of £££s.
4. All of the above are putting strains on our network like never before. It is true that bandwidth has reduced massively in cost over recent years the demands we are putting on it have increased correspondingly. Just think about phones, our networks now have the extra burden of our calls which are increasingly becoming video, they are stretched as never before. In order to cope with these changes, we are putting increasing load on our WAN-links which are struggling to cope with the increased demands.  This means turning to alternative solutions such as hybrid networking which utilises backup links and internet-VPNs  for bulky, low-priority, transactional data, leaving our low-latency, high-performance MPLS circuits for our mission critical applications.

So what does all of this mean?

Businesses need to adapt to the changing, digital world, delivering the services that both customers and users expect.  The consequence of this, though is that our businesses are more reliant than ever on our IT business systems with entire workflows becoming dependent on them.  At the same time, these systems are increasingly distributed and complex with many inter-dependencies. Users might connect via mobile devices over 3G into our cloud which then talks to a virtual web-server and then, via a virtual network, into a series of other applications and databases.  Monitoring, securing and maintaining these solutions is more difficult and complex than ever before, yet the consequences of them failing is correspondingly serious to the health of the business.  A comprehensive Application Performance Management solution is now an almost essential part of any IT infrastructure along with the standard back and business continuity tools.  The Riverbed SteelCentral suite of products, giving visibility on end-user-experience, network analytics, infrastructure health, database insights, and detailed transaction profiling is an ideal and increasingly necessary solution for the digital age.

Protecting Web Applications with the @Riverbed SteelApp Web Application Firewall

Introduction

It seems only a few years ago that web-pages served static content via HTML, hence the term web PAGE. These days, however, a web-page is merely the front-end to a web application. This will typically have multiple tiers with a database at the back-end. For these to work effectively you must allow end-users to submit information to your site, effectively modifying it. By doing this you have opened yourself up to a variety of threats at the application layer which traditional virus scanners and firewalls cannot protect against. You will hear terms such as Cross-Site Scripting and SQL Injection. These relatively modern threats exploit vulnerabilities in the web-application, either by running malicious code or by making changes to the back-end database. To make things worse, more and more applications are being served using HTTP creating lots of new juicy targets for attackers.  Often these applications are essential to the operation of a business so protection of them is vital.

Riverbed’s Web Application Firewall (WAF) gives protection against these types of attack by sitting between the user and the front-end web-server.  It is available as a standalone product for your web server but also comes as a license option for the Traffic Manager and this is the variant we will be looking at here.

We will give a short tutorial on how to set it up and protect against a dummy site. I have chosen the OWASP Broken Web Application which can be downloaded free from the internet and is available on SourceForge.

As for the other posts on SteelApp, we are running in a virtual environment using VMWare Workstation. We have one traffic manager, set up in front of a virtual machine running the OWASP Broken Web Site.

Procedure
We will browse (using Firefox) to the the front-end IP address of the web site by typing the front-end of the Traffic Manager into our browser. This gives the following:

Clicking Vicnum (game) brings us to the following:

It is important to understand that this site has been set up for be vulnerable.  We can test one of the (many!) vulnerabilities in this application by typing a very short script in this box.  For example, in the box type: <script>alert (“GOT YOU!!!”);</script>

Click PLAY and you should see the following:

Now we have established we are vulnerable let us use the WAF to protect us.

On the Traffic Manage, navigate to System -> Application Firewall and click the radio button afm_enabled following by update (ensure the confirm checkbox is checked).

Return to your virtual server and Enable the Application Firewall for it.  This is an important step, the firewall must be enabled on an individual virtual server basis -and don’t forget to click Update.  This is shown below:

 

Now click the Application Firewall tab on the top:

 

Remember that this is an application firewall so we now need to go in to Application Control and then click Create Application with Application Creation Wizard.

The wizard will start so give the application a name (OWASP in our case):

 

Continue!

You can choose between Protection or Detection mode (the latter does not stop any threats but puts an entry in the log).  For our example we are going to select Protection.

You will then be asked for a customer key.  We will just use the default:

The next step is very important and I have sometimes had to add this later then re-run the wizard in order for it to work.  You must select the hostname of the service you are protecting – this is what users will put into their browsers!

Ensure that you select your hostname and then click Add this hostname. Continue!

At the next step disable reduced logging for a testing/training environment as it helps you learn what is going on.  When requested enable full request logging.

You will finally find yourself here:

Now, at this point you need to be connected to the internet or everything will go tits up!  This is where the WAF will download the baseline signatures which it will use to protect your application.  If you have a test environment which is not connected to the internet (like me!) you will need to download them manually first.  This is very easy, just go to splash.riverbed.com and perform a search for ‘Application firewall signature’.  Download the ZIP file, extract it and then add them on the baseline management page – simples!

Click Continue and you will arrive at the following:

Not much more!  Click Finish to see the following:

 

Now, let’s test it.  Go back to the Vicnum page and enter the little script you typed before in the text box and then press PLAY.  You should receive the following on your browser:

If you receive something similar to the above, give yourself a pat on the back/nice cold beer as you have successfully configured the application firewall to protect your applications!

Just to check, go back to the application firewall as it was in the previous step and click the Monitoring tab for your application.  You should see a report on the event and how you have been protected:

 

To Summarise

The SteelApp Web Application Firewall provides real-time protection of your critical web applications and comes with a fairly straightforward wizard in order to configure it.  You can use detection mode to begin with until you are happy with the configuration before changing to protection mode to actively protect against attacks.  As an aside you can also use Traffic Script in the Traffic Manager itself to protect against a number of attacks if you do not have the firewall license, though this is labourious and not as powerful as the WAF.