Page 1
Page 2
Quantum esXpress™ Restoration and Disaster Recovery Guide, 6-66625-01 Rev B, August 2009, Made in USA.
Quantum Corporation provides this publication “as is” without warranty of any
kind, either express or implied, including but not limited to the implied
warranties of merchantability or fitness for a particular purpose. Quantum
Corporation may revise this publication from time to time without notice.
COPYRIGHT STATEMENT Copyright 2009 by Quantum Corporation. All rights reserved.
Your right to copy this manual is limited by copyright law. Making copies or adaptations without prior written authorization of Quantum Corporation is prohibited by law and constitutes a punishable violation of the law.
TRADEMARK STATEMENT
Quantum, the Quantum logo and StorNext are registered trademarks of
Quantum Corporation and its affiliates. DXi is a trademark of Quantum
Corporation. esXpress is a trademark of PHD Virtual Technologies, Inc. All
other trademarks are the property of their respective owners.
ii
Page 3
Table of Contents
Introduction ...................................................................................................................... 1
Quantum esXpress........................................................................................................... 1
Quantum Branding...................................................................................................... 1
Documentation............................................................................................................. 2
Webpage URLs.............................................................................................................. 2
Restoration and Disaster Recovery Overview .......................................................... 2
Flexibility Under Any Circumstance ......................................................................2
FULL and DELTA Backup Archives Explained....................................................... 3
Delta vs. Full Restore ............................................................................................... 4
Portable Self Extracting Delta Block Backups Explained ................................... 5
Restorations: Three Different Types...................................................................... 5
File Level Restorations: Two Different Types ....................................................... 6
File Level Backups Restores..................................................................................... 6
File Level Restores from VMDK Backups .............................................................. 6
Other Restore Considerations ................................................................................ 7
Getting Started................................................................................................................. 8
Planning......................................................................................................................... 8
Restoring A Virtual Machine .......................................................................................... 9
Restoration of a Virtual Machine to a VMware ESX Host via the Command
Line ................................................................................................................................. 9
Full Backup Archives ................................................................................................ 9
Delta Backup Archives............................................................................................. 9
Restore This Index Backup and create a new VMDK .......................................12
Verify Delta Blocks in this INDEX BACKUP File.................................................. 13
Validate FULL BACKUP as correct........................................................................ 14
Validate NEW VMDK as correct........................................................................... 15
Enter ‘A’ or ‘auto’ to restore VMDK with default options ............................. 15
Configure FTP.......................................................................................................... 19
VMX Menu .............................................................................................................. 19
Create VMDK Stub File.......................................................................................... 20
Best Practices .......................................................................................................... 21
Command Line Restore Help File......................................................................... 22
Restoration of a Virtual Machine to a Non-ESX Host Via the Command Line.... 26
Restoration of a Virtual Machine via the esXpress Console Menu ...................27
Text Menu Restores – Searching using Filter..................................................... 41
Checking the Status of Background Restore Jobs................................................ 44
L – BG Lock Status –...............................................................................................45
D – Delete all Completed/Error Jobs –................................................................ 46
R – Remove Pending (Waiting Jobs ) –............................................................... 46
Restore Queue Readme......................................................................................... 47
Automatic Mass Restorations of Virtual Machines.................................................. 49
Restoring Your Datacenter in 10 Easy Steps..................................................... 49
iii
Page 4
Simple Replication/Mass Restore............................................................................. 51
Default Options ...................................................................................................... 51
Replication Actions Menu......................................................................................... 51
L – BG Lock Status - <Current Status> ............................................................. 52
R - Run Replication Manually Now .....................................................................53
E - Edit ‘vmdks’ Restore file.................................................................................. 55
F - Load ‘vmdks’ file with distinct VMDK filenames ........................................ 57
V – View ‘/etc/phd/restored.log’.......................................................................... 58
D – Delete ‘/etc/phd/restored.log’ ....................................................................... 59
A – Abort to PHD Main Menu.............................................................................. 59
Appendix A...................................................................................................................... 60
Simple Replication/Mass Restores Readme .......................................................60
Appendix B...................................................................................................................... 70
Known Issues .............................................................................................................. 70
Opteron CPUs and Checksum Errors .................................................................. 70
Appendix C...................................................................................................................... 71
Installation of CYGWIN ............................................................................................. 71
Appendix D...................................................................................................................... 79
FAQ ............................................................................................................................... 79
Technical Support .......................................................................................................... 79
Contacts....................................................................................................................... 80
Getting More Information or Help ......................................................................... 80
iv
Page 5
Quantum esXpress Restoration and Disaster Recovery Guide
Introduction
Quantum esXpress
Quantum esXpress Backup Software provides high-availability virtual appliances
for protecting VMware virtual infrastructures and data to Quantum’s DXi-series
deduplication systems. Quantum esXpress Backup software combined with
Quantum DXi-series deduplication and replication appliances provides a
comprehensive, scalable but simple and complete data protection solution for a
VMware environment. esXpress has revolutionized data protection for virtual
environments by using the virtual environment itself to back up more data in
less time. esXpress uses “virtual backup appliances” (VBAs) – small virtual
machines – to perform autonomous, fault-tolerant backup and restoration of
your virtual environment directly to a DXi-series with no additional hardware or
software required, and minimal impact to VMware servers, the service console
or network performance. It scales easily across an enterprise’s entire virtual
infrastructure.
Important: esXpress is designed for the VMware administrator who has
passed the VCP (VMware Certified Professional) exam or has equivalent
experience. Installation and administration of the esXpress Backup
software requires that the administrator have a core understanding of
ESX server configuration, virtual machines, basic networking and
VMNET, and using VMware Virtual Center. This product is meant to be
customer installable assuming the customer is a VMware administrator.
The VMware administrator is for the purpose of this document the
“user” and or “customer”.
Quantum Branding
Quantum has modified esXpress work specifically with the DXi deduplication
appliance. In order for the Quantum DXi data deduplication and replication
appliance to function most efficiently, it has been designed to process data
that is unencrypted and uncompressed. As a result, Quantum esXpress passes
data images and deltas directly and unaltered to the DXi-Series system. For best
results, the DXi performs optimal deduplication and compression within DXi
itself. When replication is in use, the DXI performs encryption during the data
transfer. To ensure economy of scale, the DXi is designed to be the central
repository for all virtualized and traditional data center needs. esXpress as
delivered by Quantum is intended for use with the DXi family of products as
the target storage device. Other storage targets are not supported.
1
Page 6
Quantum esXpress Restoration and Disaster Recovery Guide
Documentation
There are 5 main documents available for the esXpress 3.5 release. These
documents are available for download from the www.quantum.com/esXpress
web site.
esXpress Release Notes (6-66627-01)
esXpress Quick Start Guide: (6-66582-01)
esXpress User's Guide: (6-66583-01)
esXpress Configuration and Deployment Guide (6-66737-01)
esXpress File Level Backups (6-66624-01)
Webpage URLs
• esXpress product page: www.quantum.com/esxpress
• registration/activation: www.quantum.com/esxpress/activation
Restoration and Disaster Recovery Overview
By using virtual machines, your entire server is encapsulated within a VMDK or
virtual machine disk file. While your virtual machines may look and feel like real
servers, they are in fact just files. They are no different from a word processing
document, spreadsheet or a picture. Simply copy the file and you have copied
the server. esXpress provides you 100% recoverable copies of these files.
esXpress backups are portable with the software required to restore each
virtual machine built into the backup file itself. The archives are self executable.
In an emergency, you only need your backup files. esXpress is not require to
restore backups. esXpress backups are portable and can be extracted on
VMware GSX and VMware Server as well as ESX 2.x and VI3 platforms.
esXpress simplifies the complexities of VMware backup and recovery while
ensuring your business is always protected.
Flexibility Under Any Circumstance
Probably the most important feature of any backup product is the ability to
restore a backup. While all products can restore a backup under ideal
conditions, it is the ability to restore under any condition that differentiates a
backup tool from a disaster recovery tool.
2
Page 7
Quantum esXpress Restoration and Disaster Recovery Guide
esXpress provides for the maximum flexibility and choices when restoring
virtual machines. For example you can restore to an ESX host in your datacenter or to your Windows laptop running VMware Server.
There will be many times when it will be necessary to restore one or more
virtual machines. Most of these will be in the course of daily business, such as
restoring a virtual machine in its entirety, or just a single file.
There may also be a time when you experience a catastrophic interruption like
fire or natural disaster, or something less dramatic, like simply having to
evacuate your building due to a derailed rail car or a gas leak in the building
next door. Either way, your business needs to continue to operate and with
the esXpress Auto/Mass restore feature, your entire virtual environment can be
restored and running in hours.
Virtual machines backups contain everything that makes up a virtual machine,
the operating system, the data and applications that make the data
meaningful. While this is a tremendous convenience, unless your backups are
encrypted, it is also one of the most dangerous conveniences.
No matter what the reason or circumstances, whether restoring a single virtual
machine or hundreds, esXpress is designed to assist the administrator as
quickly and reliably as possible.
FULL and DELTA Backup Archives Explained
When run, esXpress will create (2) types of archives, FULL and DELTA. A FULL
archive is created either, the first time a virtual machine is backed up by
esXpress, the virtual machine or host was scheduled to run FULLS, or the DELTA
threshold has been exceeded.
A FULL archive is nothing more than an archive of a VMDK file. This archive can
be restored on any Windows, Linux or ESX platform. You DO NOT need the
esXpress software to restore a FULL archive created with esXpress.
If the conditions for making a FULL have not been met, a DELTA archive is
created by default. A DELTA is a true block level differential of the last FULL
backup. Only (1) FULL and (1) DELTA file are required to restore any virtual
machine.
DELTA archives also contain the virtual machine configuration (.vmx) and nonvolatile RAM (.NVRAM) files. When esXpress creates a FULL archive, it will also
create an empty DELTA archive. Empty meaning it will contain no delta blocks,
but it will contain the .vmx and .nvram necessary to rebuild the entire virtual
machine.
3
Page 8
Quantum esXpress Restoration and Disaster Recovery Guide
So, assume for example that your backup schedule is creating a FULL archive on
Sunday, and DELTA archives Monday through Saturday. To restore to
Thursday’s backup would require Sunday’s FULL and Thursday’s DELTA.
Also, esXpress maintains an index map of the FULL backup, meaning no access
to the original FULL archive is required in order to create a DELTA archive. This
is both efficient and allows you to purge archives to tape.
Delta vs. Full Restore
When restoring backups, it is always preferable to restore a DELTA backup
instead of a FULL backup. While restoring a FULL backup is faster than restoring
a DELTA, DELTA restore have advantages over the FULL.
If you are restoring a FULL backup, it is simply restored as a plain file as it is
uncompressed. The system will write out the new file one block at a time, same
as the copy or tar command would. Because the VMDK is not pre-allocated,
restoring multiple virtual machines to the same VMFS file system
simultaneously would effectively interleave the two VMDKs causing severe
fragmentation and definitely affect the performance of the virtual machine.
When esXpress restores a DELTA archive, it knows the exact size of the VMDK,
as this information is stored in the DELTA archive. This allows esXpress to create
the VMDK file on the VMFS first, then import the backup archive into that preallocated VMDK file. This is the proper way to write to the VMFS. Because of
this import, you can restore
multiple VMDK files to the same VMFS at the same time with no risk of
fragmentation.
When esXpress creates a FULL archive, it always creates an empty DELTA archive
also, even in the free version. This DELTA backup contains the metadata
information about the backup including the original VMDK file sizes, along
with VMX file and the index maps. Restoring a FULL using the empty DELTA will
allow you to perform multiple FULL restorations against the same VMFS with
no risk of interleaving.
When esXpress DELTA backups are restored, the restored file is checked, block
by
block on restoration against the index map. If there are any problems, like
insufficient free space, or a checksum error, the restoration will be aborted. If
you were to lose a FULL archive, and try to rename a previous FULL to replace
it, it will not succeed. The checksums will not match the index map and the
restore process will be aborted.
4
Page 9
Quantum esXpress Restoration and Disaster Recovery Guide
Portable Self Extracting Delta Block Backups Explained
While the technology has a complicated name, the concept is a simple one;
create an archive file that is completely portable between all VMware platforms
and operating systems while maintaining security.
This self extracting archive is what differentiates esXpress from other products
and why it is truly a disaster recovery tool as much as a daily backup utility.
Archives created with esXpress are actually executable programs containing all
the logic necessary to restore and register a virtual machine without the need
for the esXpress software or license key.
esXpress Delta archives contain not only the virtual machine disk files (VMDKs),
but also the configuration and NVRAM files, everything you need to completely
restore a virtual machine. This allows for effortless restorations regardless of
circumstance or platform.
It also allows for easily and securely sharing virtual machines with co-workers
who might not have access to the esXpress software of VMware Infrastructure
3 platform.
Restorations: Three Different Types
Flexibility, business demands it. Misfortune happens and you are tasked to
react quickly. For that reason we have designed esXpress with multiple ways to
perform restores to handle most any circumstance or situation.
Command Line
Console Menu
Automatic Mass Restore
Command line restorations are ideal for disaster recover where you may not
have the resources or time necessary to build an environment capable of point
and click restore. It is also great for those that are comfortable with the
Linux/ESX command line.
Console menu restorations allow you to restore virtual machines using a text
menu interface. This is great for when you are in the data-center or physically
logged on to the host. The console menu is also available via any SSH
connection (via PUTTY for example). This option requires minimal bandwidth
and can performed remotely even across dial-up connections.
5
Page 10
Quantum esXpress Restoration and Disaster Recovery Guide
Automatic Mass Restores are repeated, scheduled restorations. This option is
for use in a disaster recovery situation where the administrator needs to restore
dozens to hundreds of virtual machines with little or no user interaction.
File Level Restorations: Two Different Types
With esXpress there are 2 different types for doing file level restores. In the 3.1
product the primary method would be to run the esXpress File Level Backup
Feature and then perform File Level Restores from those separate backup
archives. If you are running Delta/Full vmdk backups then it is also possible to
do File Level Restores although esXpress does not have any specific built in text
menu or GUI functions for it.
File Level Backups Restores
File Level Restores from VMDK backups
File Level Backups Restores
With esXpress file level backups it is simply creating a gzip or tgz file on a
network share. Because of that the backups can be simply restored by the end
users and therefore there isn’t any built in esXpress restore feature at this time.
What we recommend is that you identify the correct dated FLB backup and use
whatever tools you are comfortable with to uncompress the file ( for example
WinZip ). Then extract the needed file or files from that archive and copy them
to you associated virtual machine.
File Level Restores from VMDK Backups
For file level restores with esXpress from vmdk backups you will need to restore
the vmdk image and mount that vmdk to restore the file or files from it. There
are a couple of different options we recommend,
On your backup server (if backing up to FTP for example), install VMware Server
on that server, restore the VMDK on the backup server, mount it in a helper VM
and extract the files. (If using Windows, you need CYGWIN installed to restore
the delta backups).
Or on a DEV ESX host, restore the VMDK there, boot it up in host only mode, or
again mount the VMDK in another VM and extract the files from there.
6
Page 11
Quantum esXpress Restoration and Disaster Recovery Guide
It is planned that in the next generation of product of esXpress to be able to
pull files directly from the esXpress Delta or Full image vmdk backups.
Other Restore Considerations
If restoring to VMware Server or GSX on Windows you will need CYGWIN. See
section “CYGWIN Installation and Configuration Procedures”. Full instructions
for doing this install can be found on the documentation page of esXpress web
site,
‘Restoring esXpress Delta Backups under Windows using Cygwin’
http://www.esxpress.com/cygwin/index.php
7
Page 12
Quantum esXpress Restoration and Disaster Recovery Guide
Getting Started
Planning
esXpress can restore the archived data over the network, from attached storage
(SAN, iSCSI, NFS, local), or in the case of a DELTA restore, from both media
simultaneously.
If restoring esXpress backup archives from a network server, make sure you
know the following about the server:
• Name/IP Address of FTP/SSH Server
• Port
• User ID
• Password
• Path to Backup Folder
• The FTP/SSH user must have complete access to the share, it must be
able to create folders and files, rename and delete files.
As a security precaution, you must have root access to the VMware host to
install and execute the esXpress software.
*NOTE: Quantum DXi series are the only supported storage targets at this time.
8
Page 13
Quantum esXpress Restoration and Disaster Recovery Guide
Restoring A Virtual Machine
Restoration of a Virtual Machine to a VMware ESX Host via the Command Line
With esXpress you don’t need to have the software installed on your host to
restore your Virtual Machine’s vmdk files. For example, in a DR scenario you
may be recovering to a new host without esXpress installed on it. It is
important to note that esXpress is a vmdk restoration product, so each vmdk is
restored separately along with optionally the vmx. In a DR scenario we
recommend pre-creating your Virtual Machine from the VI3 client first and then
start the esXpress restores. In this way the vmx is setup correctly and the
correct folders to restore the vmdk into already exist.
Full Backup Archives
Every time esXpress creates a Full Backup it also creates an empty Delta archive.
You can use this empty delta archive to restore your Full Backup. This method
enables the built in menu within the delta and allows for less manual steps. It
also will verify the blocks in the backup archive. This process is described under
the Delta Backup Archives section below.
Example directory listing showing empty delta backup with Full
drwxr-xr-x 15 ftp ftp 4096 Apr 29 11:41 ..
-rw-r--r-- 1 ftp ftp 2097171 Apr 29 11:54 00-RedHat_VM1.vmdk.delta-2008.04.291134-080429-1134.phd
-rw-r--r-- 1 ftp ftp 1000739843 Apr 29 11:54 00-RedHat_VM1.vmdk.gz-080429-
1134.phd
To restore the Full backup in the example above you could run:
sh 00-RedHat_VM1.vmdk.delta-2008-04.29-1134-080429-1134.phd
Delta Backup Archives
All esXpress Delta backups are self extracting executable Files. To restore the
Delta backup archive you would shell the backup.
Example:
sh 00-RedHat_VM1.vmdk.delta-2008.04.29-1157-080429-1134.phd
9
Page 14
Quantum esXpress Restoration and Disaster Recovery Guide
This will launch a mini esXpress restore Menu which will walk you through the
restore process.
The top section of the menu shows the various information regarding the Delta
archive including, the host it was initiated from, the vmdk file, and the total
Delta Blocks. (
Figure sh-1
).
Figure sh-1, Top Portion of Menu
The second section of the menu shows the Full Backup Archive that this Delta is
using. For Delta restores you need the matching Full Backup so make sure you
restore the matching full as well to the new esx host and it is accessible.
sh-2).
(Figure
Figure sh-2, Matching Full Backup
The third section is the Various Menu Options for the Delta restore (
These include restoring the Delta backup, validating the new vmdk as well
3).
Figure sh-
as the Full archive among other options.
10
Page 15
Quantum esXpress Restoration and Disaster Recovery Guide
Figure sh-3, Menu Options
Table sh-1, esXpress sh Delta Restore Menu Options
Option Description Function
R
D
M
N
A
F
V
C
H
Q
Restore This INDEX BACKUP
and create a new VMDK
Verify Delta Blocks in this
INDEX BACKUP file
Validate FULL BACKUP as
correct
Validate NEW VMDK as correct Validate new VMDK against backup
Enter ‘A’ or ‘auto’ to restore
VMDK with default options.
Configure FTP Set up FTP
VMX Menu Show VMX Restore Menu
Create VMDK Stub File Create Stub file for GSX/Server
Help esXpress Restore ReadMe
Quit Quit the PHD esXpress application.
Restore backup and create a New
vmdk
Verify the Delta Blocks in the backup
archive
Validate and check the Full archive as
the match to the Delta
Auto Restore Backup using all defaults
11
Page 16
Quantum esXpress Restoration and Disaster Recovery Guide
Restore This Index Backup and create a new
VMDK
.
Choose this option to restore this backup file and create a new VMDK file.
You are asked for the new name to restore as, but it defaults to the current
VMDK name. When a backup is being restored the FULL backup is pulled
directly from the FTP server (unless it's locally on the same drive as this backup
file) and a new VMDK file is created. If you are restoring this backup on ESX,
then you can safely do more the one restore at time, as we create the new
VMDK using vmkfstools, then we import the backup directly into this new
VMDK file. Otherwise restoring more the one backup on the same file system
will created severely fragmented files.
The first thing you need to do is select which Full backup archive to use. The
menu will present a default if it finds one either local or from ftp if configured.
You can accept the default by just hitting Enter or provide the full path and file
name.
Your Command? R
***************************************************
* Restore a DELTA INDEX BACKUP to a NEW VMDK File *
***************************************************
To restore an INDEX BACKUP I need to know the location
of the FULL BACKUP. Pull from FTP not set.
Enter the full path of the FULL BACKUP File here, 'q' to quit
or Enter to accept the default of
'../2008.04.07-RedHat_VM1.564d1e34-4364-1808-23fd-e56565add965.FULL/00RedHat_VM1.vmdk.gz-080407-1110.phd'
What FULL Backup do you want to use?
The second step is to define the new VMDK name included its Full Path. Hit
Enter to accept the default or provide the new name and full path.
To create a new NEW VMDK file you need to enter the fully
pathed name such as: /vmfs/folder/name.vmdk
Enter 'q' to quit or
Press Enter to accept default './RedHat_VM1-flat.vmdk'
Create what fully pathed VMDK file?
12
Page 17
Quantum esXpress Restoration and Disaster Recovery Guide
The last step is to confirm your choices and proceed with the restore. You must
enter ‘yes’ to continue, enter ‘q’ to quit.
************************************************
* Ready to create a NEW VMDK from INDEX Backup *
************************************************
Accessing FULL BACKUP from File
../2008.04.07-RedHat_VM1.564d1e34-4364-1808-23fd-e56565add965.FULL/00RedHat_VM1.vmdk.gz-080407-1110.phd
Create NEW VMDK File: './RedHat_VM1-flat.vmdk'
Ready to continue (Enter 'yes' to continue, 'q' to quit)?
Verify Delta Blocks in this INDEX BACKUP File
This option will verify the Delta blocks in this INDEX backup. When you are
doing a RESTORE, only the first 3,000 blocks will be checked, so choose this
option if you want to verify all blocks beforehand.
Figure sh-4, Verify Delta Blocks
If the verify is correct you will see the following message:
CHECKSUMS ALL GOOD IN DELTA FILE
13
Page 18
Quantum esXpress Restoration and Disaster Recovery Guide
Validate FULL BACKUP as correct
This option will validate a VMDK as the FULL Backup that was used when this
INDEX backup was made. If the file is local it will be used. If FTP is configured,
then the FULL backup will be pulled and verified through FTP.
Note: Quantum does not support FTP at this time.
Your Command? M
Verify FULL Backup as correct for this Delta.
Default: '../2008.04.07-RedHat_VM1.564d1e34-4364-1808-23fd-e56565add965.FULL/00RedHat_VM1.vmdk.gz-080407-1110.phd'
Press Enter to accept default or 'q' to quit.
Verify what FULL Backup?
******************************************
* Verify a FULL Index to a FULL Backup *
******************************************
Verify FULL BACKUP from
File: '../2008.04.07-RedHat_VM1.564d1e34-4364-1808-23fd-e56565add965.FULL/00RedHat_VM1.vmdk.gz-080407-1110.phd'
Ready to continue (Enter 'yes' to continue, 'q' to quit)? yes
Starting at: Wed Apr 30 16:42:44 EDT 2008
Verifying FULL BACKUP File, 20479 Blocks in file:
../2008.04.07-RedHat_VM1.564d1e34-4364-1808-23fd-e56565add965.FULL/00RedHat_VM1.vmdk.gz-080407-1110.phd
+++++++++++++++++++++++++++++++++++++++++++++++++++
+++++++++++++++++++
This is just a verify, you can safely abort (hit ^C).
Master Verify: 14%, 700 mb of 5120 mb, at 63 meg/sec, Total Seconds: 10
14
Page 19
Quantum esXpress Restoration and Disaster Recovery Guide
When the Full verification is complete you should see a message similar to the
figure (
Backup VMDK verify is complete. If an error is shown there was a problem with
the Full Backup that needs to be looked into.
Figure sh-5, Verify Delta Blocks
Figure sh-5
) below showing the Master is 100& verified and that Full
Validate NEW VMDK as correct
This option will validate a VMDK and compare it to the INDEX Backup that
was used to make this INDEX backup. This is also done when a RESTORE is
completed.
Enter ‘A’ or ‘auto’ to restore VMDK with default options
The auto restore option will use all the Default settings when restoring the
vmdk. For example the default name, the original location, etc. When you
choose this option you will be prompted to confirm if you wish to continue (
Figure sh-6
). Enter ‘yes’ to confirm or ‘no’ to abort the restore.
15
Page 20
Quantum esXpress Restoration and Disaster Recovery Guide
Figure sh-6, Auto Restore Confirmation
Once entering yes, you will see the default values being used for the restore.
The answer ‘Autorun Restore’ is automatically set for each question for the
restore. They include, the correct Full Backup and the path for the new vmdk.
Before choosing the auto run restore make sure the correct Full Backup is
accessible either by configuring FTP or copying the matching Full Backup to the
same directory as the Delta you are restoring.
Do you want to continue? (Enter 'yes' or 'no')? yes
Auto Restoring Backup with Defaults
***************************************************
* Restore a DELTA INDEX BACKUP to a NEW VMDK File *
***************************************************
To restore an INDEX BACKUP I need to know the location
of the FULL BACKUP. Pull from FTP not set.
Enter the full path of the FULL BACKUP File here, 'q' to quit
or Enter to accept the default of
'../2008.05.02-Fedora_VM1.564d1e34-4364-1808-23fd-e56565add965.FULL/00Fedora_VM1.vmdk.gz-080502-1113.phd'
What FULL Backup do you want to use? Autorun Restore
########################################################
#
To create a new NEW VMDK file you need to enter the fully
pathed name such as: /vmfs/folder/name.vmdk
Enter 'q' to quit or
Press Enter to accept default './Fedora_VM1-flat.vmdk'
Create what fully pathed VMDK file? Autorun Restore
16
Page 21
Quantum esXpress Restoration and Disaster Recovery Guide
The below steps show the new vmdk being created using vmkfstools which the
restore backup will be imported into.
************************************************
* Ready to create a NEW VMDK from INDEX Backup *
************************************************
Accessing FULL BACKUP from File
../2008.05.02-Fedora_VM1.564d1e34-4364-1808-23fd-e56565add965.FULL/00Fedora_VM1.vmdk.gz-080502-1113.phd
Create NEW VMDK File: './Fedora_VM1-flat.vmdk'
Ready to continue (Enter 'yes' to continue, 'q' to quit)? Autorun Restore
*******************************************
* Checking if Import VMDK is available. *
*******************************************
vmkfstools -c '5242880k' -a lsilogic './Fedora_VM1.vmdk'
New VMDK: './Fedora_VM1.vmdk'
New Flat: './Fedora_VM1-flat.vmdk'
Successfully Created.
Backup VMDK Size: 5368709120
-rw------- 1 root root 5368709120 May 5 14:10 ./Fedora_VM1-flat.vmdk
-rw------- 1 root root 376 May 5 14:10 ./Fedora_VM1.vmdk
Backup will be imported into new VMDK file.
Because esXpress v3 imports the backup into the New VMDK, you can do multiple restores at
once.
Starting at: Mon May 5 14:10:34 EDT 2008
The next step in the autorun restore is the verification of the Delta blocks in the
backup.
+++++++++++++++++++++++++++++++++++++++++++++++++++
+++++++++++++++++++
Verifying 255 Delta Blocks
+++++++++++++++++++++++++++++++++++++++++++++++++++
+++++++++++++++++++
1 255
===================================================
===================
Blocks found 255, CHECKSUMS ALL GOOD IN DELTA FILE
===================================================
===================
17
Page 22
Quantum esXpress Restoration and Disaster Recovery Guide
The last messages you will see is the running status of the backup restore. In
this example 69% of the Delta backup has been processed so far and 45% of
the Full has. The estimated time remaining for the restore is 3 minutes and 21
seconds.
+++++++++++++++++++++++++++++++++++++++++++++++++++
+++++++++++++++++++
Delta Blocks Found: 255
Restoring VMDK file: ./Fedora_VM1-flat.vmdk, 20480 Blocks in file
+++++++++++++++++++++++++++++++++++++++++++++++++++
+++++++++++++++++++
Delta: 69% Full: 45%, 2300 mb of 5120 mb, at 14 meg/sec, Elapsed: 02:41s Remaining:
03:21s
Upon the successful completion of the Autorun Restore you will see messages
similar to the following, showing a good checksum and the successfully created
vmdk files.
Delta: 100% Full: 100%, 5100 mb of 5120 mb, at 14 meg/sec, Elapsed: 06:03s Remaining: 01s
Total Blocks: 20480, Processed: 20480
===================================================
===================
CHECKSUMS ALL GOOD IN VMDK FILE:
../2008.05.02-Fedora_VM1.564d1e34-4364-1808-23fd-e56565add965.FULL/00Fedora_VM1.vmdk.gz-080502-1113.phd
===================================================
===================
+++++++++++++++++++++++++++++++++++++++++++++++++++
+++++++++++++++++++
Skipping Verify of NEW VMDK file
VMDK successfully created: './Fedora_VM1-flat.vmdk'
-rw------- 1 root root 5.0G May 5 11:02 ./Fedora_VM1-flat.vmdk
VMDK successfully created: './Fedora_VM1.vmdk'
-rw------- 1 root root 376 May 5 14:10 ./Fedora_VM1.vmdk
Completed at: Mon May 5 14:16:41 EDT 2008
Thank You for using PHD esXpress v3 Backups, www.esxpress.com
18
Page 23
Quantum esXpress Restoration and Disaster Recovery Guide
Configure FTP
Note: Quantum does not support FTP to the DXi at this time.
From within the restore menu you can set up a FTP server if needed to go and
download the matching Full backup for the Delta backup being restored.
Your Command? F
This DELTA backup can pull the FULL backup from your FTP host.
What FTP Server ('q' to quit, Enter to accept []): 192.168.1.110
What User Name ('q' to quit, Enter to accept []): ftpuser
What Password ('q' to quit, Enter to accept []): ftpuser
What /Path/File ('q' to quit, Enter to accept []): /backups/esXpress
You Entered the following FTP information.
Server: 192.168.1.110
User: ftpuser
Pass: ftpuser
File Path: /backups/esXpress
Is this correct (Enter 'yes' or 'no')?
VMX Menu
This menu (
logs from when this backup was made. You can save it to any path, and it will
ask you if you want to register the VMX file. Do not use an ESX VMX file with
GSX/Server. Make a new one, but use the old one as a guide.
Figure sh-5
) will allow you to restore the VMX file, NVRAM and
19
Page 24
Quantum esXpress Restoration and Disaster Recovery Guide
Figure sh-5, VMX Menu
Create VMDK Stub File
You can use your ESX VMDK files directly with other VMware products. There
is almost nothing different between an ESX VMDK vs GSX/Server Pre-Allocated
disk. If you create a STUB file that points to the ESX VMDK then you can use it
directly on your Backup server or any machine. When you configure an 'Existing
Virtual Disk' just point to the STUB file. Make sure the VMDK and the STUB file
are in the same folder if you move them.
Your Command? C
*******************************************************
* Create Stub File for using ESX VMDK with GSX/Server *
*******************************************************
The Size of this VMDK Backup is: 5368709120
The most recently restored VMDK
File is:
Size is:
Your Creation Options are as Follows:
1. Point to a different file and make a Stub file.
20
Page 25
Quantum esXpress Restoration and Disaster Recovery Guide
What do you want to do? ('q' to Quit)?
Best Practices
esXpress is a vmdk restore based product. With this in mind we recommend
that in a DR scenario or when you are restoring to an ESX host without the
esXpress software installed to Pre-Create the Virtual Machine from within the
VI3 client.
While esXpress can restore the vmx file from the original backup we do not
make any changes to it. So with a new host when they could be different
settings in the vmx ( for example, VM location, different datastore, possible
duplicate vmdk names, network differences, VM memory allocation and others
) we don’t attempt to update the vmx file directly and cause possible additional
issues.
This is why we recommend creating the VM upfront from the VI3 client.
When creating the VM you would set the following settings as normal :
- VM Name and location
- DataStore
- Resource Pool if applicable
- CPUs
- Memory
- Network
- Virtual Disk (s) - create the virtual disk the same size as the original
vmdks from the backups.
Now you will have a fresh VM, set up correctly for the new host and then
can just restore the esXpress vmdk backup directly over the blank vmdk you
created with the VM. This makes for a much cleaner restore and recovery.
** Note – If you do choose to restore the VMX from the backup, this will also
work but you need to go to the VI3 client and make any necessary changes to
the VM as a result of the new esx host environment.
21
Page 26
Quantum esXpress Restoration and Disaster Recovery Guide
Command Line Restore Help File
** This is the online file from the delta shell restore **
PHD ESXPRESS DEFINATIONS:
A FULL Backup - This is a complete copy of a VMDK file, it may be
compressed.
A DELTA Backup - This is only the blocks that we're different when compared
to the original FULL backup. The backup file is also
an executable program that will rebuild the backup from
the Delta blocks in this backup and with the FULL backup.
An INDEX Backup - Is a different name for the DELTA backup. We say an INDEX
backup of DELTA blocks.
When you run the backup files, you can pass it parameters.
AUTORUN - This will activate the AUTORUN features in the restore program.
RESTORE - The backup will try to restore itself using default parameters,
only works on local space, no FTP yet.
DELTA - Used with AUTORUN, a Delta block verify will execute automatically.
VERIFY - Used with AUTORUN, the default is not to Verify a restored VMDK.
This option enables the Verify option.
IGNORE - Set to Ignore errors. Keep restoring or verifying even with ERRORS.
NOABROT - Do not allow aborts from restores.
EXIT - Exit out of Restore menu on errors when AUTORUN. Normal action is
to stay at the menu in case of an error.
MENU OPTIONS:
P. Set Passwords - PASSWORDS SUCCESSFULLY SET
P. SET PASSWORDS
This option allows you to set the password for this backup file
and the password for the FULL backup. If you are restoring from
the EsXpress Text menu, then the passwords will be passed from the menu
to this restore program. You can also set Environment variables.
R. Restore this INDEX BACKUP and create a new VMDK
Choose this option to restore this backup file and create a new
VMDK file. You are asked for the new name to restore as, but it
defaults to the current VMDK name. When a backup is being restored
the FULL backup is pulled directly from the FTP server (unless it's
locally on the same drive as this backup file) and a new VMDK file
is created. If you are restoring this backup on ESX, then you can
safely do more then one restore at time, as we create the new VMDK
using vmkfstools, then we import the backup directly into this new
VMDK file. Otherwise restoring more then one backup on the same
file system will created severely fragmented files. This applies to
22
Page 27
Quantum esXpress Restoration and Disaster Recovery Guide
every restore method out there, except for esXpress and vmsnap.pl
D. Verify Delta Blocks in this INDEX BACKUP file
This option will verify the Delta blocks in this INDEX backup. When
you are doing a RESTORE, only the first 3,000 blocks will be checked,
so choose this option if you want to verify all blocks beforehand.
M. Validate FULL BACKUP as correct
This option will validate a VMDK as the FULL Backup that was used when
this INDEX backup was made. If the file is local it will be used. If
FTP is configured, then the FULL backup will be pulled and verified
through FTP.
N. Validate NEW VMDK as correct
This option will validate a VMDK and compare it to the INDEX Backup that
was used to make this INDEX backup. This is also done when a RESTORE is
completed.
F. Configure FTP
You can configure the FTP server options for where to get the FULL
backup from.
V. VMX Menu
This will allow you to restore the VMX file, NVRAM and logs from when
this backup was made. You can save it to any path, and it will ask you
if you want to register the VMX file. Do not use an ESX VMX file with
GSX/Server. Make a new one, but use the old one as a guide.
C. Create VMDK Stub File.
You can use your ESX VMDK files directly with other VMware products.
There is almost nothing different between an ESX VMDK vs GSX/Server
Pre-Allocated disk. If you create a STUB file, that points to the ESX
VMDK, then you can use it directly on your Backup server or any machine.
When you configure an 'Existing Virtual Disk' just point to the STUB
file. Make sure the VMDK and the STUB file are in the same folder if you
move them.
H. Help
This file you are reading.
Q. Quit
Exit this menu and go back to a prompt.
IGNORE - Ignore Checksum errors, verify/restore anyhow.
Type in 'ignore' to toggle between on and off. Default is OFF.
ENV VARIABLES:
If you are doing multiple verifies/restores on your backup server and you
23
Page 28
Quantum esXpress Restoration and Disaster Recovery Guide
don't want to type in the password to un-encrypt your backups, you can set
the following environment variables. This backup restore program will
automatically try the passwords passed to it.
export USE_SPASS="the system passphrase"
export USE_MPASS="the master password phrase"
export USE_DPASS="Password for Delta"
export USE_FPASS="Password for Full"
The Folder to use when doing automatic restores.
export USE_RPATH="/vmfs/LOCAL"
Or a complete file path.
export USE_RFULL_PATH="/vmfs/LOCAL/file.vmdk"
And the FTP path can be set by:
export
PHD_LYNX="ftp://user:password@ftpsite/pathname/to_backuo/file_name.FILL.gz"
RESTORE PROGRAM OPTIONS
Try these examples on your backup server. We're using dft01 in our examples.
Don't actually enter a '#', that's there to simulate a shell prompt on your
Linux/ESX/CYGWIN(Windows) server.
Run the esXpress Restore Menu in the Backup File. Which gives you the ability
to restore this backup.
# sh 1001-dft01.vmdk.delta-2006.04.19-2001-060402
Automatically verify the Delta Blocks in the Backup File, less then 10 errors.
# sh 1001-dft01.vmdk.delta-2006.04.19-2001-060402 auto run delta
Automatically verify the Delta Blocks in the Backup File, and accept no errors.
# sh 1001-dft01.vmdk.delta-2006.04.19-2001-060402 auto run delta no errors
If you're running a 'delta' or a 'restore' in AUTORUN mode, and an ERROR
happens, the default action is to stay at the menu. By using the 'exit' option,
it will always exit from the restore menu/program. Such as:
# sh 1001-dft01.vmdk.delta-2006.04.19-2001-060402 auto run delta no errors exit
Automatically RESTORE this Backup File, with all default options. The default
is to name the file the same as the backup and create it in the current
directory. The Default Path can be set with an Environment variable. Our
DEFAULT name would be 'dft01.vmdk' in our example.
# sh 1001-dft01.vmdk.delta-2006.04.19-2001-060402 auto run restore
After a RESTORE is complete, the default action is NOT TO VERIFY when running
in 'autorun' mode. If you want to VERIFY, add the 'verify' command.
# sh 1001-dft01.vmdk.delta-2006.04.19-2001-060402 autorun restore verify
24
Page 29
Quantum esXpress Restoration and Disaster Recovery Guide
WHY
You might be asking yourself, 'Why do I care about this?'
By knowing a little about shell scripting you can easily restore all your
backup files. You can have your backup server verify your backups every day.
You can setup a cronjob that restores the VMDK files from your Exchange or
other servers that you frequently need. When the Boss deletes the wrong email
you can literally have it 'ex-merged' within minutes.
A simple script to verify all the delta's would be (cut & paste):
export USE_SPASS="system.password"
export USE_RPATH=/u
for i in `ls [0-9]*`
do
sh $i autorun delta noerrors
done
Replace the word 'delta' with 'restore' and you sit back and watch all your
VMDKs restore. You should not restore more than one at a time on a particular
File system (unless you’re just testing) because with multiple restores running
will cause the VMDKs to be heavily fragmented.
Better AUTORUN options coming.
25
Page 30
Quantum esXpress Restoration and Disaster Recovery Guide
Restoration of a Virtual Machine to a Non-ESX Host Via the Command Line
Delta Files –
There is a tutorial on the esXpress web site documentation page explaining
how to restore a Delta backup to a Windows Host.
http://www.esxpress.com/cygwin/index.php
26
Page 31
Quantum esXpress Restoration and Disaster Recovery Guide
Restoration of a Virtual Machine via the esXpress Console Menu
To start the esXpress software, type phd at the command prompt. The
esXpress Backup Menu will open.
Figure 2, esXpress Main Menu
The first three selections displayed are also status indicators. In the above
example, the first selection, Current Running Status, will show you if backups
are currently running, and note if you had any errors. The second selection,
Daemon Status, also displays the current status of the background daemon.
The third selection, Lock Status, also displays the current lock status.
Restorations respect lock status.
NOTE: If accessing esXpress via SSH using putty.exe, make sure to enlarge the window beyond the default size, or open
and run in full screen. The default window size is not always sufficient to display all backup archives when accessing
the Restoration Menu.
Select the Restoration Menu by using the mouse, arrow keys or pressing the
“E” hot key.
This menu allows you to initiate restores of entire virtual machines, single
VMDK files, and individual VMX files as well.
27
Page 32
Quantum esXpress Restoration and Disaster Recovery Guide
The following table (Table 1) describes the available menu options and their
function.
Table 2, esXpress Restore Menu Options
Option Description Function
C
X
B
R
L
T
I
S
H
Restore through the ESX
Console
VMX Restore through the ESX
Console
Background Restore Status
Replication Actions Displays the Replication Menu
BG Lock Status Work with restore locks
Tail esXpress Log Views the esXpress backup Log
Re-Index NET and VMFS
Backup Targets
Create a STUB File for an
existing VMDK
Help on Restores (Updated) esXpress Restore ReadMe
Initiates the esXpress Restore process
Restore a VMX file only
Shows the current and completion
status of all back grounded
restorations for that host
Re-Index your backup targets.
Creates vmdk Stub File if necessary
A
Q
Abort to the PHD Main Menu Abort the Restoration Menu
Quit to the PHD Main Menu Quit the PHD esXpress application.
Figure 3, esXpress Restore Menu
28
Page 33
Quantum esXpress Restoration and Disaster Recovery Guide
Upon selection the Restore option, if the local database has not been updated
within the previous 15 minutes, esXpress will re-index your backup targets to
ensure the restore process has the most current backup information.
Figure 3, esXpress Re-Index Backup Target
Figure 4, esXpress Restore Menu – Search By
This menu allows you to restore virtual machines, individual VMDK files, or just
the virtual machine configuration file (.vmx).
The following table (Table 1) describes the available menu options and their
function.
Table 2, esXpress Restore Menu – Search By Options
Option Description Function
29
Page 34
Quantum esXpress Restoration and Disaster Recovery Guide
V
F
X
S
Select by Virtual Machine
Name
Select by VMDK File Name Restore just the select VMDK file
VMX Restore through the ESX
Console
Create a STUB File for an
existing VMDK
Restore the selected virtual machine
and optionally register it to the host
Restore a VMX file only
Creates vmdk Stub File if necessary
Shows the current and completion
B
Background Restore Status
status of all back grounded
restorations for that host
A
Q
Abort to the PHD Main Menu Abort the Restoration Menu
Quit to the PHD Main Menu Quit the PHD esXpress application.
At this point you are ready to choose your backup archive to restore. There are
2 methods of restoration, the entire virtual machine, or just a single VMDK file.
The following menus are of the Select by Virtual Machine Name choice. The
Select by VMDK file name menu option works identically to Select by Virtual
Machine Name, except the system will not prompt you to register the virtual
machine.
Figure 5, esXpress Restore Menu – Search by Virtual Machine Name
This menu displays a list of virtual machines available for restoration and is
listed using their VMware display name.
30
Page 35
Quantum esXpress Restoration and Disaster Recovery Guide
Depending upon the size of your farm, this list can be very large. You can use
the Filter option to locate a particular virtual machine by entering part of its
name into the search box.
Move the cursor to select the desired virtual machine and press
Enter
. The
system will display a listing of the dates for which archives are available for the
selected virtual machine (figure 6). Or you can select Show All Backups to have
the system display all archives available for this virtual machine.
Table 3, esXpress Restore Menu – Search By Options
Option Description Function
S
1 – N
B
Show All Backups Display all archives for all dates
Date
Display archives for only the selected
date
Back Return to the previous menu
A
Q
Abort to the PHD Main Menu Abort the Restoration Menu
Quit to the PHD Main Menu Quit the PHD esXpress application.
Select the desired archive date or simply pres Enter to see all available archives
for the selected virtual machine.
31
Page 36
Quantum esXpress Restoration and Disaster Recovery Guide
Figure 6, esXpress Restore Menu – Select “virtual machine name” by Which Date
Whether you selected a particular date or all dates, the system will next display
all archives available from all defined restoration locations, including fail over
hosts. If esXpress is configured to backup to multiple locations, you may have
duplicate entries for the selected virtual machine.
Figure 7, esXpress Restore Menu – Choose Backup File to Restore
32
Page 37
Quantum esXpress Restoration and Disaster Recovery Guide
At this point you are ready to select the archive file you which to restore, either
a Delta or Full archive. When restoring a Delta archive the restore process will
also automatically located the associated Full archive. The second column
describes where the particular archive is located. If an archive is stored in two
or more locations, make sure to select the location that is nearest you (local FTP
vs. FTP over the WAN, or VMFS vs. remote SSH) as this can have a dramatic
effect on the restoration speed.
In the following example (figure 8), we have selected to restore a Delta Archive
from a network backup target which is a DXi. A confirmation screen is
presented. Make sure to review your selection carefully before Selecting Yes to
continue.
Figure 8, esXpress Restore Menu – Restoration of a Delta archive
33
Page 38
Quantum esXpress Restoration and Disaster Recovery Guide
Once you have confirmed this is the correct virtual machine and archive date,
the system will ask you to select which FULL archive to restore from, if there are
multiple archives available. Again, remember to select the archive that is most
local to your location.
Figure 9, esXpress Restore Menu – Confirm Full for Delta Restore
In this example the Full exists on two network backup targets, ftp1 and ftp5
(
Figure 9
). The system will ask you to confirm the Full archive for this Delta
Restore. Virtual machine names predicated with an asterisk means the system
has validated this is the correct FULL archive for the selected DELTA.
For restorations where you are restoring to the virtual machine’s original
location (or current location if over writing), and restoring with the original
VMDK file names, you need to do nothing more then select Original Location
(
figure 10
), the select OK or Yes through the remaining restoration menus,
accepting the defaults.
Table 4, esXpress Restore Menu – Select Location to Restore To
Option Description Function
1
L
O
B
A
Q
Virtual Machine Path Display/Select original location
Original Location Restore to Original location
Other Location Restore to a different location
Back Return to the previous menu
Abort to the PHD Main Menu Abort the Restoration Menu
Quit to the PHD Main Menu Quit the PHD esXpress application.
34
Page 39
Quantum esXpress Restoration and Disaster Recovery Guide
Figure 10, esXpress Restore Menu – Select Location to Restore To
By selecting Other Location the system will prompt you to select a VMFS
volume and directory, or optionally create a new directory. The menu in
11
allows you to select from all VMFS volumes presented to that particular
Figure
host.
Figure 11, esXpress Restore Menu – Select Other Location
Once you have selected the volume to restore to, you are prompted with a list
of available subdirectories. If you are not restoring to an existing directory,
select Make New Folder (
figure 12
).
35
Page 40
Quantum esXpress Restoration and Disaster Recovery Guide
Figure 12, esXpress Restore Menu – Select Which VMFS Subfolder to Restore to
You are now prompted to enter a name for this new subdirectory.
Figure 13, esXpress Restore Menu – Enter New Folder Name
You are then asked to confirm the folder creation. Select Yes to continue.
Figure 14, esXpress Restore Menu – Confirm Creation of New Folder
36
Page 41
Quantum esXpress Restoration and Disaster Recovery Guide
The system will confirm whether or not the folder was created successfully.
Figure 15, esXpress Restore Menu – New Folder Successfully Created
The system will return you to the Select Which VMFS Subfolder to Restore to
menu, with the newly created directory listed and highlighted.
Figure 16, esXpress Restore Menu – Select Which VMFS Subfolder to Restore to
37
Page 42
Quantum esXpress Restoration and Disaster Recovery Guide
Select the newly created directory.
After you have either created a new location, or selected an existing location,
the system will ask you to confirm the VMDK name. You can change the name
of the VMDK file here if you like (figure 17).
Figure 17, esXpress Restore Menu – Confirm Restore to VMDK name
38
Page 43
Quantum esXpress Restoration and Disaster Recovery Guide
Note – if you are restoring to the current location, make sure that the virtual
machine is powered off. As a safety precaution, esXpress will not restore over
a running virtual machine.
Figure 18, esXpress Restore Menu – Final Restore Confirmation
Review the restore information to ensure everything is correct. (
Figure 18
). To
continue with restoration of the selected virtual machine, select Yes.
esXpress restore engine has the ability to submit the restore job to the
background. With this ability you can process multiple restores in the
background, releasing your main session. The status of the background restore
processes can be checked on the Main Restore Menu, under the Background
Restore Status Option.
Select Restore in background or continue the restore in the foreground (
19
).
Figure
Figure 19, esXpress Restore Menu – Submit Restore to Background
39
Page 44
Quantum esXpress Restoration and Disaster Recovery Guide
If running the restoration job in the foreground, the process will display the
progress (
figure 20
).
Figure 20, esXpress Restore Menu – Sample Restore Screen in Foreground
Upon completion of a successfully restore, you should see the following (
21
).
Figure 21, esXpress Restore Menu – Restore Completion in Foreground
Figure
40
Page 45
Quantum esXpress Restoration and Disaster Recovery Guide
Press Enter to return to the restoration menu.
** Important Note –
If you restore to a different location then you must modify the Virtual
Machine from the VI3 client to point the vmx to the new location. esXpress
does not modify the vmx in this situation. Without doing this step the VM will
not power on with a ‘File Not Found Error’
Text Menu Restores – Searching using Filter
When searching for esXpress backup archives in environments with a large
number of VMs it can become very tedious to page up and down through the
Virtual Machine or VMDK search options to find your archive. The Filter feature
makes this process much easier and can help you refine your search criteria to
find the correct archive quickly.
Figure 18, esXpress Restore Menu – Filter
41
Page 46
Quantum esXpress Restoration and Disaster Recovery Guide
Figure 18
VMDK works exactly the same way to filter your search. To setup a search filter
highlight “Filter” and hit Enter.
In
Figure 19
your filter options highlight Back and hit Enter or use the hot key “B”. Your
options will now apply to the search criteria for your restore process. Initially
the settings are all set to the default, which is searching everything.
Filter Options :
shows the Search by Virtual Machine name menu but the search by
you have the ability to setup your filter options. Once you have set
Note – to change an option highlight that Option and then highlight OK, then
hit Enter. Highlighting each option and also OK and then hitting Enter will
cycle through the available choices
.
- Filter by Any Part Name : This will open up a box (
can enter any partial name to limit your search. For example if you are
looking for all Virtual Machines that have dev in its name then you would
enter “dev” here.
- Filter by Transport Type : limits by the network target transport types (
ALL/SSH/FTP).
- Filter by VMFS Backup : There are 3 vmfs filter options available. The
options are Show which keeps your vmfs target in the search, Only which
will only show vmfs archives and Hide which will exclude vmfs archives
from the search.
- Filter by Target Number : this lets you filter by a backup target number (1
through 9) or set it to “%” which will search across all targets.
Figure 20
) where you
42
Page 47
Quantum esXpress Restoration and Disaster Recovery Guide
- Filter by Backup Mode : you can search for All backups, limit to just Fulls
or just Delta archives.
- Reset Filter : This will erase any filter option values you have defined and
reset them back to the esXpress defaults which show all backup archives.
- Back : saves your filter options
- Quit to Last Menu : quits and returns to the prior menu
Figure 19, esXpress Restore Menu – Filter
Figure 20, esXpress Restore Menu – Filter by Any Part Name
43
Page 48
Quantum esXpress Restoration and Disaster Recovery Guide
Checking the Status of Background Restore Jobs
Note – Background restores is a licensed feature
From the Restore Menu select the Background Restore Status Option (
22
).
Figure 22, esXpress Restore Menu
.
figure
Figure 23, esXpress Restore Menu – Background Job Status Main Screen
44
Page 49
Quantum esXpress Restoration and Disaster Recovery Guide
Figure 25
Figure 24, esXpress Restore Menu – Background Job Status
shows the current status of a Background Restore job.
You can also view the running background restore log by highlighting < Log >
and hitting enter. To exit the log hit
CTL-C
.
L – BG Lock Status –
This option works similar to the normal esXpress locks. You set and clear locks
for background restores (
Figure 24.1
Figure 24.1, esXpress Background Restore Menu – Lock Status
).
45
Page 50
Quantum esXpress Restoration and Disaster Recovery Guide
D – Delete all Completed/Error Jobs –
This option can be used to clear out old background restore logs that you no
longer need. It will remove all logs that completed, either restoring successfully
or with an error.
Figure 24.2, esXpress Background Restore Menu – Confirm Delete Completed/Error Jobs
R – Remove Pending (Waiting Jobs ) –
If you need to cancel a pending background restore job you can use this option
to remove that job.
Figure 24.3, esXpress Background Restore Menu – Confirm Remove Pending Jobs
A – Abort to PHD Main Menu – Selecting this option will bring you back to the
Main esXpress Text menu.
46
Page 51
Quantum esXpress Restoration and Disaster Recovery Guide
Restore Queue Readme
When esXpress does a restore, it can do it a number of ways. When you use the
esXpress Text menu to restore a VMDK, you have the option to submit it to the
restore
queue. When you choose a Delta backup to restore, at the end you have the
option
of submitting it to the restore queue. Or you can run it in the foreground like
esXpress has done until now (version 3.1).
(Always restore Delta backups over a Full backup. Restoring a Full might be
faster, but when you restore a Delta backup it has many advantages. The
VMDK
will be pre-allocated and the backup imported into the VMDK. Each block of
the Full and the Delta are compared against the index map to validate the
checksum of all data. esXpress knows the proper name of the VMDK to
restore.)
In the esxpress Text menu, option (C) Replication / Restore Options menu under
the (C) Configuration menu has the options for the restore queue.
--It needs to be enabled, it is by default.
--You can configure how many concurrent restore jobs to run at once.
The default is 1, with a max of 4. Do not run 4 unless you increase the
MHZ reserved for the console.
The restore queue is /etc/phd/restore
When backups are submitted for restore, a control file is created in this
folder, then the phd_daemon will pick it up, and run the restore. If you
define 2 restores to run, then 2 will run at a time. The log for each
restore is also kept in this folder. From the Replication menu you have
then option to clean-up and delete the restore jobs. (auto purging coming)
(Running 2 in a normal console is OK, but increasing console CPU Mhz will
help keep the console from bogging down. Do not run more then 2 unless you
increase the CPU allocation. But do experiment and test.)
From the (B) Backup Restore Status menu you can see the restore queue. It
shows all the restore jobs in the /etc/phd/restore folder. The restore
status for each is shown if it is complete, or Waiting to run. For each
restore you can see who submitted the restore job, along with the log
for each. If the restore is currently running you can watch the log as it
runs. Remember to hit ^C (Control C) to exit the live viewer. When a
backup is complete you can also view the log, this will use nano or vi.
The restore queue is only for Delta archive restores. It will not work
47
Page 52
Quantum esXpress Restoration and Disaster Recovery Guide
for a Full backup. Even in Free mode, when a Full backup is made, an
empty Delta backup is also made. The restore queue is also a licensed
feature of esXpress. This means to auto replicate or mass restore, or
for even background restore, you need a licensed copy of esXpress.
48
Page 53
Quantum esXpress Restoration and Disaster Recovery Guide
Automatic Mass Restorations of Virtual Machines
The Mass/Auto Restore or Simple Replication feature allows you to restore
VMDKs very easily, which is very helpful when doing recovery. By only restoring
a few VMDKs and doing it automatically, esXpress turns mass restores into
simple replication. As of now, the mass restore will only restore delta backups.
The basic premise is that you have a list of VMDK names that you want to
restore, and the name of the VMFS you want to restore them to. From either
the EsXpress Text menu (manually) or automatically these VMDK files will be
checked against the FTP servers, and the backups that meet the correct criteria
will then be restored. Backups are matched by name, and you might get
multiple matches, but only the first one will be used. Make sure to adequately
test to ensure you are restoring the correct VMDK backups.
To restore a delta backup (which the mass restore uses) it must be downloaded
to the local host first. Then the delta backup is run like a program. The delta
backup will then pull the Full backup from the FTP on the fly and make a new
VMDK file. The backup will be imported into the VMDK correctly (by creating or
reusing the VMDK file). Afterwards the delta backup file will be deleted. Then
the next VMDK file is processed and restored. This entire process is
automatic. After you have defined the VMDK backups to restore, you initiate
the process and watch.
Restoring Your Datacenter in 10 Easy Steps.
1. Start by restoring your backups from tape to a server.
2. On this server, enable the FTP service and configure.
3. Install the esXpress rpm on the ESX hosts and configure.
4. Choose option E for Restoration from the PHD Main Menu and then
Option R for Replication Actions.
5. Choose option F ‘Load vmdks.auto file’ to import the VMDK list from FTP
server or other backup targets defined.
6. Choose option E to edit the VMDK list, selecting the VM you want to
restore and to which VMFS.
7. Choose option R to run mass restore.
8. Choose option B to check the status of the restoration of your VMDK
files.
9. Choose option T to view the esXpress log file to check the status of the
restoration.
49
Page 54
Quantum esXpress Restoration and Disaster Recovery Guide
10. Repeat steps 3-9 on the other ESX hosts.
50
Page 55
Quantum esXpress Restoration and Disaster Recovery Guide
Simple Replication/Mass Restore
esXpress defines ‘Simple Replication’ as
to a host or hosts automatically
day and restore them on another host automatically, you have achieved simple
replication. Replication can be as often as hourly, or as little as once a day.
This feature is available in all licensed versions of esXpress. For hosts that only
need to restore backups, and not create them, they can use esXpress LE to
replicate their environment.
. When you backup your virtual machines every
restoring of complete VMDK backups
Default Options
• The original VMDK filename will be used.
• If the VMDK already exists, it will be overwritten.
• It will not be verified afterwards.
Note : The replication host level configuration options are discussed in detail in
the esXpress User’s Manual.
Replication Actions Menu
Quantum does not support host based replication.
Figure 24, esXpress Restore Menu – Replication Actions Menu
51
Page 56
Quantum esXpress Restoration and Disaster Recovery Guide
Table 10, esXpress Replication Actions Menu
Option Description Function
Shows current status of Replication
L
B
R
Replication is – Disabled
BG Lock Status
Background Restore Status
Run Replication Manually Now Initiate replication Now
DISABLE. Replication is performed on
the target DXi appliance
View statuses and logs of background
restore jobs
T
E
F
V
A
Q
Tail esXpress Log View current esXpress main log
Edit ‘vmdks.auto’ Restore file
Load ‘vmdks.auto’ file VMDK
names
View ‘/etc/phd/restored.log’ View the listing of restored vmdks
Abort to PHD Main Menu Returns to the PHD Main Menu
Quit to the PHD Main Menu Quit the PHD esXpress application.
Edit the esXpress replication
instructions file
Load the replication file with distinct
vmdk names from host
L – BG Lock Status - <Current Status>
Just like the backups, the mass restoration uses lock files. You can set or clear
the mass restore locks here. These locks do not affect backups, only mass
restores.
From the Mass Restore menu, the line for L – BG Lock Status, is also an
indicator for the current lock status. In the example below the status is ‘Clear’.
Figure 25, Lock Status
The mass restore locks are currently checked before a restore is started. It will
not cancel a currently running restore. Use the Kill option in conjunction with
Locks to cancel out of mass restores.
52
Page 57
Quantum esXpress Restoration and Disaster Recovery Guide
Figure 26, Replication/ BG Restore Lock Menu
Table 3, Replication / BG Restore Locks
Option Description Function
C
P
L
T
Q
Clear Locks
Stop Restores
Clear Quorum
Locks
Stop Restores Restores after the current VMDK is finished
Quit
This will clear any Mass Restore Locks and enable
restorations.
The mass restore will be Stopped after the current
VMDK is finished restoring.
Clear locks for normal operation – run backups
Quit this menu and return to the Replication
menu.
R - Run Replication Manually Now
Now that you have configured the ‘vmdks’ files and ran the test restore a few
times, its time to run the mass restore/replication for real. It’s a good idea to
run it from the menu here first before enabling auto-restorations/replication.
53
Page 58
Quantum esXpress Restoration and Disaster Recovery Guide
Figure 27, Run Manual Replication
54
Page 59
Quantum esXpress Restoration and Disaster Recovery Guide
E - Edit ‘vmdks’ Restore file
This file /etc/phd/vmdks is the instructions for the esXpress Mass
Restore/Replication engine. It contains a list of VMDK files to restore and to
which VMFS to restore them to. It also contains variables that control the
behavior of the restorations.
Set variables USE_SPASS and USE_MPASS for the System and Master password
you want to use when restoring the backups. You do not have to set both, but
you can. Setting a password here will override the password in the Encryption
Configuration (Page Error! Bookmark not defined.) menu. These variables
must have the ‘#’ in front of them. They are comments in this file. But two ‘#’
as in ‘##’ means the variable will be ignored, commented out.
The variable USE_DAYS sets which day of backups you want to use. The default
is zero or commented out (##). When set like this, the most recent backup on
the FTP server will be used. Otherwise you want to set it for negative values.
Minus One (-1) will restore yesterday’s backups. Minus Seven (-7) will only
restore backups at least 7 days old.
Option ARGS is for passing extra arguments to the Delta backup. The option in
the example above is ‘noerrors’. By default the restore program in the Delta
backup will accept some checksum errors (10) before failing out of a restore. By
enabling #ARGS=noerrors, you are telling the restorer to accept no errors.
Any error will cause it to fail.
55
Page 60
Quantum esXpress Restoration and Disaster Recovery Guide
Figure 28, Editing the ‘vmdks’ File
When entering the VMDK filename to restore, it can be in the VMHBAFilename.vmdk or just Filename.vmdk notation. Use the VMHBA number to
distinguish between different backup files. Hopefully your VMDKs are named in
a way that they all have distinct names.
If the VMDK name has a ‘#’ in front of it, then it is a comment and will be
ignored. Remove the leading ‘#’ to select a VMDK for restoration. After the
VMDK name there is a pipe ‘|’ followed by the name of the VMFS. Currently
there cannot be any spaces used in the VMDK name or the VMFS name.
In the following example, linux_imap.vmdk will be restored to the /vmfs/LOCAL
filesystem. Each VMDK file can be restored to a different VMFS.
56
Page 61
Quantum esXpress Restoration and Disaster Recovery Guide
Figure 29, VMDK Filenames to Restore
By editing this list of VMDK filenames, and making a list that you want to
restore and where to restore them to, you can easily restore large numbers of
VMDK backups with little effort.
F - Load ‘vmdks’ file with distinct VMDK filenames
When doing Mass Restores or Replication, the ‘/etc/phd/vmdks’ file controls
which VMDK files to restore and to what VMFS. By choosing this option ‘F’, a
list of all distinct VMDK filenames will be loaded into the ‘vmdks’ file for you.
Then you just need to edit it for the VMDKs you want restore.
Figure 30, Import VMDK Names
To get the distinct names, the following SQL is run against the FTP server
database.
select distinct vmdk_name from ftp_database;
57
Page 62
Quantum esXpress Restoration and Disaster Recovery Guide
At the end of the ‘vmdks’ file, a list of VMDK filenames are appended. The
name is based upon the Delta backup files on the FTP servers. The appended
lines are commented ‘#’ out. Remove the ‘#’ to use that VMDK. The VMFS
name is defaulted only as /vmfs/, you must add the rest of the VMFS name.
# VMDK Name Import
# 0005-esxpressmui.vmdk|/vmfs/
# 0005-VisualStudio.vmdk|/vmfs/
# 1001-dft01.dsk|/vmfs/
# 1001-dft02.dsk|/vmfs/
# 1101-oraforms00.vmdk|/vmfs/
# 1101-Genrad.vmdk|/vmfs/
# 0006-snapshots.vmdk|/vmfs/
V – View ‘/etc/phd/restored.log’
This option will show you the ‘restored.log’ which is a log of all the vmdks that
have been restored through esXpress replication.
of this file.
Figure 31, View ‘/etc/phd/restored.log’
Figure 31
shows an example
58
Page 63
Quantum esXpress Restoration and Disaster Recovery Guide
When viewing the restored.log, the only way to exit is to enter ‘^C’ Control-C.
D – Delete ‘/etc/phd/restored.log’
This option will clear the restored.log file which is the log of all replicated
vmdk’s that have been restored. Clearing it will start esXpress replication in a
fresh state with no vmdk’s being marked as restores.
a Delta backup through replication that has already been restores, delete this
file and you can restore that Delta backup again.
Figure 32, Delete‘/etc/phd/restored.log’
If you are trying to restore
A – Abort to PHD Main Menu
Exits the Replication Actions menu and goes back to the Main esXpress Text
menu.
59
Page 64
Quantum esXpress Restoration and Disaster Recovery Guide
Appendix A
Simple Replication/Mass Restores Readme
PHD esXpress backups, Patent Pending
esXpress Restores release README
copyright 2009, PHD Technologies, Inc., www.p2v.net, www.esxpress.com
This file is /home/phd/bin/RESTORES.txt
###################################################
#############################
www.esxpress.com or backup.p2v.net
BASIC ESXPRESS TERMINOLOGY
--------------------------------------------------------------------------------
-IN THE CONSOLEA console restore is simply the ability to restore a VMDK file or a VMX to the
VMFS from within the VI3 console of a host. In esXpress v3 this function is
the standard restore procedure. Currently to restore an esXpress v3 backup, you
have to do it in the console, and not in a VBA.
We consider the ability to restores virtual machines in the console an important
feature. If restores were limited to a VBA only, then you would not be able do
any restores until your complete virtual framework was up and running. By
enabling restores in the console, you are always provided the ability to
restore your backups. We consider this a crucial DR feature.
A console restore is a menu driven text mode UI. You are walked through the
complete restoration process of a VMDK file or a VMX file from beginning to end.
Nothing more than hitting the 'Enter' key is usually required.
-SIMPLE REPLICATIONIf you were to take a backup, and restore it on another host, that is basically
a restoration. If this happened automatically, then we could call it a
replication of a Virtual Machine. This is how we define 'Simple Replication'.
-IN THE GUINow that the restore queue has been implemented, GUI restores are coming.
-MANUAL RESTORESAbove all, you should always be able to restore your backups. This is a simple
statement, that we at PHD stand behind. With esXpress backups, you can always
restore your backups, whether you have access to our software or not. With our
product, only backups are licensed. There is no licensing involved with doing
restorations. You own your data, not us.
-FULL BACKUPSWhen a FULL backup is made, it is just a gzip'd copy of the file-flat.vmdk
60
Page 65
Quantum esXpress Restoration and Disaster Recovery Guide
File. This file can easily be restored to any version of VMware or even other virtual
platforms. With VMware, you just need to make a STUB file, and you can use your
FULL backup after
unzipping it. Restoring a FULL is as simple as:
zcat fullbackup.gz.phd > newfile-flat.vmdk
-DELTA BACKUPSesXpress Delta backups are more then just backup files. The delta backup file
itself it an executable program that will allow you to find and locate and
combine the FULL and DELTA archives together to create the VMDK file.
Restoring a DELTA backup manually is fairly simple. If the DELTA and FULL backup
are together on a share, you can just shell the archive.
Example: sh the_delta_file.phd
and a restore menu will appear. If the FULL is on the local share with the
DELTA, then it should automatically find it. If not, you can tell it where it
is, or even grab it on the fly from your FTP server.
Once in the DELTA Restore menu, you can perform many tasks including: restore a
VMDK, verify the FULL archive, and a few others. This delta backup is
scriptable. See the help in the restore menu itself.
THE ESXPRESS RESTORATION PROCESS
-------------------------------------------------------------------------------The way restores work in esXpress is different from most backup solutions. The
backup and restore programs in esXpress are essentially 2 completely different
applications. One does not have anything to do with the other. In fact the
licensing for esXpress is only in the backup engine, not the restore programs.
The Full backup is always restorable because it is nothing more than a
copy of the -flat.vmdk file. The Delta backup is actually a self restorable
archive. It can restore itself without the need of the esXpress software.
You own your backup archives, not us. You know with esXpress your backup
archive are yours, and that you can always restore them, forever.
esXpress does not keep a grid or database of backups it has made like other
products, but will connect and index the backup targets when needed. This way
you can restore a FTP server, restore your backup tapes, and then have your VI3
hosts go out and index it. Then you can start doing restores. With the mass/auto
restore you can setup numerous restore jobs at once, submit them to restore,
and then watch it all restore automatically.
When choosing to restore a backup, the backup targets will probably need to be
indexed unless they were recently indexed. If it's been more than 5 hours, then
the targets will be re-indexed automatically for you.
Because of this indexing ability of esXpress, you can safely remove, move around
61
Page 66
Quantum esXpress Restoration and Disaster Recovery Guide
or restore backups to a backup share. Then just re-index the share, and you can
now restore your backups. It does not matter to esXpress if the delta is on the
local drive and the Full is somewhere else. As long as both the Delta and Full
are included in the index of the backup targets.
As of 3.0-9 you can easily restore backups from version 2 of esXpress also. Just
note, for the v2 backups, they will show up in the restore menu with their
hostname as the VM Name. This is because the naming conventions used in version
2 of esXpress were different then those used in esXpress v3.
DELTA vs FULL RESTORE
-------------------------------------------------------------------------------When restoring backups, it is always better to restore a Delta backup over a
Full backup, but a Full is faster. When a Delta backup is restored, esXpress
knows the size of the VMDK. On restoration esXpress will create the VMDK file on
the VMFS first, then import the backup into that pre-allocated VMDK file. This
is the proper way to write to the VMFS. Because of this import, you can restore
multiple VMDK files to the same VMFS at the same time.
If you are restoring a Full backup, it is restored as a plain file. It is no
different then using the copy command or tar to write out the VMDK file. These
methods do not pre-allocate the VMDK first, but write out the new file in chunks.
If you were to copy multiple files to one VMFS at the same time, these files
would effectively be interleaved. They would be severely fragmented.
If you still need to restore a Full backup, only do one at a time per VMFS, and
you will be OK. If you do more, then they will be fragmented and performance can
suffer in the running VMs.
When esXpress makes a Full backup, it always makes an empty delta too, even in
the free version. The Delta backup contains the metadata information about the
backup, along with VMX file and the index maps.
When esXpress Delta backups are restored, the restored file is checked, block by
block on restoration against the index map. If there is a problem, then the
restore will be aborted. If you were to lose a Full backup, and try to rename
another one to replace it, it will not work. The checksums will not be the same
and the restore process will be aborted.
ESXPRESS BACKGROUND RESTORE QUEUE
-------------------------------------------------------------------------------When esXpress does a restore, it can do it a number of ways. When you use the
esXpress Text menu to restore a VMDK, you have the option to submit it to the restore
queue. When you choose a Delta backup to restore, at the end you have the option
of submitting it to the restore queue. Or you can run it in the foreground like
esXpress has done until now (version 3.1). Note - Background Restores are a licensed
feature of esXpress.
62
Page 67
Quantum esXpress Restoration and Disaster Recovery Guide
(Always restore Delta backups over a Full backup. Restoring a Full might be
faster, but when you restore a Delta backup it has many advantages. The VMDK
will be pre-allocated and the backup imported into the VMDK. Each block of
the Full and the Delta are compared against the index map to validate the
checksum of all data. esXpress knows the proper name of the VMDK to restore.)
In the esXpress Text menu, option (C) Replication / Restore Options menu under
the (C) Configuration menu has the options for the restore queue.
--It needs to be enabled, it is by default.
--You can configure how many concurrent restore jobs to run at once.
The default is 1, with a max of 4. Do not run 4 unless you increase the
MHZ reserved for the console.
The restore queue is /etc/phd/restore
When backups are submitted for restore, a control file is created in this
folder, then the phd_daemon will pick it up, and run the restore. If you
define 2 restores to run, then 2 will run at a time. The log for each
restore is also kept in this folder. From the Replication menu you have
then option to clean-up and delete the restore jobs. (auto purging coming)
(Running 2 in a normal console is OK, but increasing console CPU Mhz will
help keep the console from bogging down. Do not run more than 2 unless you
increase the CPU allocation. But do experiment and test.)
From the (B) Backup Restore Status menu you can see the restore queue. It
shows all the restore jobs in the /etc/phd/restore folder. The restore
status for each is shown if it is complete, or Waiting to run. For each
restore you can see who submitted the restore job, along with the log
for each. If the restore is currently running you can watch the log as it
runs. Remember to hit ^C (Control C) to exit the live viewer. When a
backup is complete you can also view the log, this will use nano or vi.
The restore queue is only for Delta archive restores. It will not work
for a Full backup. Even in Free mode, when a Full backup is made, an
empty Delta backup is also made. The restore queue is also a licensed
feature of esXpress. This means to auto replicate or mass restore, or
for even background restore, you need a licensed copy of esXpress.
If you choose to restore a Full backup, it cannot be run by the restore
queue in the background. You must run the restore in the foreground, on
the console or through putty (ssh). This is how all esXpress restores
were run until now (version 3.1, with the restore queue)
SIMPLE REPLICATION / MASS RESTORE
-------------------------------------------------------------------------------Simple replication is defined as having one Host backup to your Backup Target,
then have others Hosts check the Backup Target and look for new Delta backups.
When new backups are found they are automatically restored. This is what we
call simple replication, even one to many replication.
63
Page 68
Quantum esXpress Restoration and Disaster Recovery Guide
Unlike other replication products, simple replication is included with esXpress.
It is included at no additional charge, but does require a licensed copy.
esXpress can only replicate from the available backups. This means you always
have a copy on the backup server, in addition to the replicated copies. Other
products copy the data directly from one VM to another VM, with no backup.
HOW IT WORKS:
The default action is to look for the newest backups and restore them. But you
can also define a VMDK as -3, this means restore one at least 3 days old instead.
This way you can keep a copy of the Exchange server VMDK auto restored on another
LUN, and a second copy from last week.
The Replication Actions are defined from the Restore Menu, which is item (E) from the
phd Main Menu in the console. From the (R) 'Replication Action' menu you define the
VMDKs you want to replicate, and manage the restore jobs. When the replication runs
and chooses Delta backups to restore, they are submitted to the esXpress Restore
Queue.
Choose option (B) to manage the Backup Restore Queue.
All the options, the queue and log for restores are in the /etc/phd folder.
/etc/phd/restore is the restore queue itself. Restores are submitted to this folder then
processed by the background phd_daemon. This is enabled by default and requires the
phd_daemon to be running. The restore queue is a licensed feature of esXpress.
/etc/phd/restored.log is a list of previously restored VMDKs. This file is checked after
then Backup Targets have been indexed and compared against VMDKs defined in the
/etc/phd/vmdks.auto file. If you delete this file or edit it, then you can restore a
previously replicated restored backup.
/etc/phd/vmdks.auto is where the replication is configured. This is a simple text file
describing with VMDKs to restore and where to restore them to.
You can edit the /etc/phd/vmdks.auto manually or from the esXpress Text menu.
Option (E) in the
Replication Menu. This file contains some example on how to configure the
replication.
You also pre-load this vmdks.auto file with a list of all known VMDKs on the backup
targets. This is option (F) on the Replication Menu. Afterwards the Backup Targets will
be
indexed, and all unique VMDKs will be appended to the vmdks.auto file. Then you can
easily
edit this file, to include the VMDKs you want to restore. When each VMDK is loaded, it
is
defaulted to restore to the VMFS as defined in the (C) Replication / Restore Options in
the (C) Configuration Menu.
The /etc/phd/vmdks.auto file looks like this:
## /etc/phd/vmdks
64
Page 69
Quantum esXpress Restoration and Disaster Recovery Guide
# Copyright PHD Technologies Inc, 2007
# Part of PHD esXpress backups, www.esxpress.com
#
# This file defines the VMDKs to restore and
# which VMFS to restore them on. This is meant for mass restores.
#
# By default, the most recent backup will be restored.
# USE_DAYS=-1 will not restore a backup newer then yesterday.
# USE_DAYS=-3 will not restore a backup newer then 3 days ago.
#
# USE_DAYS=-3
#
# You can also set the USE_DAYS value on a per VMDK basis instead of globally.
#
# UUID can be the correct UUID, or *, or % to accept
# any UUID where the VMDK name and the SCSI_ID match
#
# When the default restore path is created, it will use the VMFS defined in
# the Mass restore menu, off the Restore Menu.
#
#
RESTORE:VM_NAME|UUID_OF_VM|SCSI_ID|VMDK_NAME|Complete_Restore_Path|Us
e_Days
#
RESTORE:CRM|564d122a-0ed7-3b92-2f46aee9456b2074|00|CRM.vmdk|/vmfs/volumes/ISCSI02/ronzo/CRM.vmdk
RESTORE:CRM|564d122a-0ed7-3b92-2f46aee9456b2074|00|CRM.vmdk|/vmfs/volumes/ISCSI02/caleb/CRM.vmdk|-3
The concept is that you define a list of VMs and VMDKs you want to mass restore
or replicate. Every hour (or as defined) the backup targets will be re-indexed
and matching backup archives will be matched against the /etc/phd/vmdks.auto file.
When defining which VMDKs for replication you need to know:
0. Line must start with RESTORE:
1. Which VM to restore. This is the folder where the VMX file lives.
2. Each VM is unique by the UUID, enter the UUID or * (For any). Be careful of DUPES.
3. The SCSI ID (or * For any) of the VMDK you want to restore. The backup archives
are name 00- which mean SCSI ID 0:0
4. The Name of the VMDK you want to restore.
5. And the Full path to restore the VMDK to.
This is usually /vmfs/volumes/some_vmfs/some_folder/machine.vmdk
6. Optionally, how many days behind the restore should be.
If you make this -7, then only a backup at least 7 days old will be restored.
The example above showing CRM.vmdk is configured to restore the most current
VMDK
and a copy from 3 days ago. This allows you to backup one VM, and have multiple
copies of it restored, for multiple purposes.
65
Page 70
Quantum esXpress Restoration and Disaster Recovery Guide
ON TO RESTORES FROM THE esXpress TEXT MENU
-------------------------------------------------------------------------------From the (E) Restore Menu you can restore VMDK files and VMX files from within
the console, and create STUB files if needed. Creation of STUB is not required
but you may at time need to create a new STUB file.
Console restores are VMDK restores. You select the VMDKs you want to restore.
If a VM as 4 VMDKs then you need to select and restore all 4 VMDKs for this VM.
To restore a VMDK you need to choose:
(C) Restore through the ESX Console
Then on the next menu you can search by VM Name or VMDK name. Or restore a VMX
file. After a Delta backup is restored, you are asked to restore the VMX file at
that time. The restore VMX option is here incase you just want to restore a VMX
only.
Choose: (V) Select by Virtual Machine name
Now you should get a list of all known VMs. The VM name here is from the folder
on the backup target. The folder name on the backup target is based on the date
of the backup and the original folder name of the VM from the ESX host where
the backup came from. Version 2 backups will show up here by host name, not VM
name.
If you choose (F) Select by VMDK name, then a list of all unique VMDK name are
shown.
Sometimes these menus can have a lot of choices, and moving around the menu can
be tiresome. Some quick keys to remember. If you hit (B) or (Q) you will jump to
the bottom, and pressing (1) will bring you back to the top.
SELECT A BACKUP TO RESTORE
-------------------------------------------------------------------------------Once you select a VM and press enter, you are shown a list of all dates for this
VM. This is all the backup archives found on the backup targets. Choose a date
or All Dates. Then the backups that match this VM name are shown. This will
include all VMDKs that match this VM name. Be warned, if you have the same VM
name on different hosts, they will show up here together.
For each backup listed, you are told from which target it was found, the size,
the type of backup (Full/Delta), the date, the SCSI ID and the VMDK name. You
can choose the Filter option and filter the output by various ways.
Once you select a VMDK to restore, you are then shown a summary of what you want
to do. Here you are shown the full path to the backup file and the backup target
information. Once you select to continue, the connection is checked to make sure
66
Page 71
Quantum esXpress Restoration and Disaster Recovery Guide
the backup can be successfully accessed. This can a few seconds as the header
for each backup is actually downloaded and verified.
If this is a Delta restore, then you are asked to choose which Full backup to
use. If the Delta backup file you choose has a UUID then esXpress will show with
an asterisk '*' next to the Full backup it thinks is correct by a matching UUID.
If you choose the wrong Full backup because you have Dupe VM names, then the
restore will be aborted because esXpress will know it is not correct when it
validates the blocks on restore.
You could be shown multiple Full backups that match the Delta you are restoring.
If you were backing up to VMFS and NET, you have two Fulls, and both will be
shown on the restore menu.
SELECT WHERE TO RESTORE
-------------------------------------------------------------------------------After the connection to the Full and Delta have been verified, you are now asked
where to restore this backup to. You are shown a list of where this VMDK file
currently exists and the option to restore elsewhere.
Warning, if your VMFS Nice names have funny characters in them, then esXpress is
not happy. Instead of ISO's, use ISOs. The single quote is a problem at this
time. Other characters may cause issues too.
You can either choose to over-write an existing VMDK file or choose a new path
location. If you choose to over-write, then that VM has to be powered off. If
the VM is running, and then selecting it will not hurt it, as esXpress will tell
you that it can not select that location because it cannot get a write lock.
Once you choose a location, you are asked to name the VMDK. esXpress will
suggest a name. If you are restoring a Delta, then esXpress knows the name. If
it is a Full backup then esXpress can only guess at what the name should be.
AND THE RESTORE IS STARTED
-------------------------------------------------------------------------------You are asked for a final verify before starting.
Once you select to continue you are asked if you want to submit this restore job
to the background restore queue. Saying 'Yes' will add this restore job to the
queue. You can keep submitting jobs to the restore queue and they will be run
in order. For this example we selected 'No' to submit to the restore queue.
The restore is started in the foreground. Do not close a putty window if you
are running a restore in the foreground. Your restore will probably die and
the esXpress process might now properly clean-up leftover files.
If you are doing a Full, then the restore starts and you are shown a status of
how much and how fast. When restoring the Full, esXpress is literally grabbing
the file from FTP/SSH and passing through gzip and writing the file out.
67
Page 72
Quantum esXpress Restoration and Disaster Recovery Guide
(Make sure you read Delta Restores vs Full Restores above)
A Full restore.
-------------------------------------------------------------esXpress, Doing FULL Backup Restoration, www.esxpress.com
Restore FULL Backup from
Location: 192.168.201.2:21/local/esxpress/backups
Folder : 2007.09.20-CRM.564d122a-0ed7-3b92-2f46-aee9456b2074
File : 00-CRM.vmdk.gz-070920-1601.phd
Restore to:
VMFS: /ISCSI02/ronzo
VMDK: CRM.vmdk
FLAT: CRM-flat.vmdk
Restoring FULL Backup from NET FTP Target
405 MB processed Avg(15.0 MB/s) Cur(13.7 MB/s)
If you are restoring a Delta backup, the process is a little more involved
internally. The Delta backup needs to be downloaded to the local VMFS first.
Then the Delta backup is executed like a program, which then pulls the Full and
makes the new VMDK file on the fly. When the delta is being restored you are
shown real stats on how much, what percent, how fast and how much time
remaining.
Delta: 67% Full: 32%, 1300 mb of 4096 mb, at 11 meg/sec, Elapsed: 01:50s Remain:
04:14s
When restoring a Delta archive, the VMDK is pre-created on the VMFS first using
vmkfstools. Then the restore is imported into the existing VMDK. Because the
Delta is importing the backup into the VMFS, and it is validating the data of
the Full and the Delta blocks against the index maps, it is not as fast as a
Full restore. When the data blocks are verified on restore, any bad data will
cause the restore to abort. Even one flipped bit will cause it to abort.
After the Delta restore is complete, you are asked if you want to restore the
VMX file also. When you restore the VMX file it is the copy form when the
backup was made. If you restored to a different path and setting, you must
check the configuration of this restore VMX in the VI3 client. esXpress will
not edit or update the VMX to reflect changes you made on restore. You must
do this yourself from the VI3 client
Afterwards the Delta backup file is removed, but only if it was downloaded.
Delta backups on the VMFS as a backup target will not be deleted.
If you want to test the DR ability of esXpress you can simply run the delta
backup file on your backup host and restore a backup there. You do this by
68
Page 73
Quantum esXpress Restoration and Disaster Recovery Guide
'sh the_delta_archive.phd' and a restore menu will be shown. If your backup
host is a Windows server, you just need install Cygwin first and you can run
the self restoring Delta archive. (Install Perl and Lynx too with Cygwin).
The Delta backup is a run-able Linux program that will use the Full backup and
recreate the VMDK file. No additional software is required. To restore an
esXpress delta backup on any Linux or ESX host, without the esXpress software,
you need to only: sh (the delta file)
Example: sh 00-Win2K3.vmdk.delta-2007.01.29-0220-070124-2111.phd
Then the delta restore menu will be shown. From this menu you can restore the
backup or just validate it. This is how Delta backups are restored. Once the
Delta has been downloaded, the delta file is then executed by the esXpress Text menu.
The restoration of the Delta files can also be scripted easily!
69
Page 74
Quantum esXpress Restoration and Disaster Recovery Guide
Appendix B
Known Issues
Opteron CPUs and Checksum Errors
Occasionally checksum errors may occur while doing a restore. When verifying
Delta blocks or performing a restoration, which also verifies the delta blocks,
esXpress may detect that a checksum does not match. The Delta blocks are
checked against the Delta Index that was made during the first stage of the
Delta Backup.
If backups are being made and restored on Intel based hosts, a checksum error
indicates the archive is invalid. Otherwise, if an AMD Opteron platform, the
error is most likely due to the “Opteron Bug”, when in a tight loop performing
repetitious mathematical functions, sometimes the CPU will “flip a bit”.
This is a very tight loop that computes the checksums. For a 10 GB file it would
loop 40,000 times doing the same checksum calc routine, over and over again.
The “Opteron Bug”.
In proving this, we start with a 10GB VMDK file. Then on an Intel based host
we compute the md5 checksum for each 256k block of data, or 40,000 total.
Then on an Opteron based host, using the same VMDK file, we perform the
same task, comparing the checksum to the control list. Very often, the system
would report a checksum that did not match, even though the file is verified
valid. This test is repeatable with predictable results.
Doesn’t this checksum affect my backup? No. The problem esXpress detects is
that a Delta Block does not match its checksum value. The value of the
checksum is incorrect, but the data in the Delta block is correct.
70
Page 75
Quantum esXpress Restoration and Disaster Recovery Guide
Appendix C
Installation of CYGWIN
To start you will need the CYGWIN installation package which can be
downloaded at http://www.cygwin.com
Create a folder named
CYGWIN installer into that folder.
Next, run the installer by double clicking the setup.exe that you just
downloaded. The installation menu will open, select
cygwin
in the root of your C: drive and download the
.
Next
to continue.
Figure 4, Cygwin Main Menu
Next you are prompted to select a download source. This manual assumes the
server you are installing to has access to the Internet.
71
Page 76
Quantum esXpress Restoration and Disaster Recovery Guide
When prompted to “
Figure 2, Choose A Download Source
Choose A Download source
”, select “
Install from Internet
”.
ext, select the directory to install Cygwin, or accept the default selection.
N
nsure that the “
E
Default Text File Type Unix / binary
“
Install For All Users
” radio button is selected as well as the
” is selected as well.
ig
ure 3, Cygwin root directory F
72
Page 77
Quantum esXpress Restoration and Disaster Recovery Guide
Next you are prompted for the “
Local Package Directory
”. Select the same
directory as you did for the Cygwin installation.
Figure 4, Local Package Directory
he next prompt requires you to specify how the application should connect to
T
e Internet. Select the option that applies to your environment.
th
Figure 5, Internet Connection Method
73
Page 78
Quantum esXpress Restoration and Disaster Recovery Guide
At this point you are prompted to
download speeds, try to select a site that is located within your country o
Choose A Download Site
. For the best
r
region.
Figure 6, C
hoose A Download Site
he installation now requires you to select the desired packages. In addition to
T
e default selections we will need to select additional packages for use with
th
sXpress.
e
Figure 6, Select Packages
74
Page 79
Quantum esXpress Restoration and Disaster Recovery Guide
Scroll the window until you see the
Interpreters
section. Click on its preceding
plus sign to expand the installation options for this section.
Here we can select the additional Interpreters. Make sure to select the
following; expat: XML parser library, gawk: GNU awk, and perl: Practical
Extracting and Reporting Language.
Figure 7, Interpreters
75
Page 80
Quantum esXpress Restoration and Disaster Recovery Guide
Now, scroll the window until you see the
openssh
. This will automatically select
Net
openssl
section. From this section, select
as well.
Figure 8, Nets
Scroll the window again until you see the
bzip2, cygutils,
Figure 9, Utils
and
gnupg
.
Utils
section. From this section, select
76
Page 81
Quantum esXpress Restoration and Disaster Recovery Guide
And finally, scroll to and select the
mode WWW browser
.
Web
section. Make sure to select
lynx: Text-
Figure 10, Web
And last, select whether or not you want the
Cygwin
icon created on the
desktop and start menu.
77
Page 82
Quantum esXpress Restoration and Disaster Recovery Guide
Figure 11, Create Icons
Installation is complete. Press OK.
Figure 12, Installation Complete
78
Page 83
Quantum esXpress Restoration and Disaster Recovery Guide
Appendix D
FAQ
Q: Can I restore my esXpress backup on my Linux backup server?
A: Yes, the delta backup file is actually a Linux program, you can run it on any
Linux host. Because of this, you do not need our esXpress software to do
recovery. Not only can you rebuild the VMDK file on your Linux machine, but
you can use the VMDK directly with VMware Server (or GSX) on that Linux
machine. With a helper machine you can mount backup copies of VMDKs and
easily recover files or data.
Q: Can I restore my esXpress backups in Windows?
A: Yes, (See Previous Question) With the CYGWIN environment installed on your
Windows server you can execute the backup file just like it was Linux and
restore your backups. This includes encryption. This way on your Windows FTP
server you can restore a VMDK backup, and use it directly in Windows with
VMware Server
Q: When I try to run the 'phd' menu or go to a different menu item,
nothing happens, the screen just flashes.
A: You probably are using a terminal program (such as Putty) and have your
window too small. The menu requires a minimum screen size equal to the
console (80 x 24 characters).
Q: My NIC is only 100MB, would using Gigabit be better?
A: Yes, you will get faster backup speeds on a gigabit NIC
Technical Support
with
For customers and partners
http:/Quantum.com/support
and support programs.
an active support agreement, go to
for information about software patches, technical documentation,
79
Page 84
Quantum esXpress Restoration and Disaster Recovery Guide
Contacts
Quantum company contacts are listed below.
To order documentation for esXpress or other products contact:
Quantum Corporation
P.O. Box 57100
Irvine, CA 92619-7100
(949) 856-7800
(800) 284-5101
Quantum Home Page
Visit the Quantum home page at:
http://www.quantum.com
esXpress Product Information
You can register your esXpress software at:
http://www.quantum/express/activation
Request More Information:
http://quantum.mv.treehousei.com/Surveys/06/1BE8BC76ECC42185/RequestMo
reInfo.aspx
Getting More Information or Help
StorageCare™, Quantum’s comprehensive service approach, leverages
advanced data access and diagnostics technologies with cross-environment,
multi-vendor expertise to resolve backup issues faster and at lower cost.
Accelerate service issue resolution with these exclusive Quantum StorageCare
services:
Service and Support Web site - Register products, license software, browse
Quantum Learning courses, check backup software and operating system
support, and locate manuals, FAQs, firmware downloads, product updates and
more in one convenient location.
Benefit today at:
http://www.quantum.com/ServiceandSupport/Index.aspx
eSupport - Submit online service requests, update contact information, add
attachments, and receive status updates via e-mail. Online Service accounts are
free from Quantum. That account can also be used to access Quantum’s
Knowledge, a comprehensive repository of product support information.
.
80
Page 85
Quantum esXpress Restoration and Disaster Recovery Guide
Sign up today at:
http://www.quantum.com/ServiceandSupport/eSupport/Index.aspx.
StorageCare Guardian - Securely links Quantum hardware and the diagnostic
data from the surrounding storage ecosystem to Quantum's Global Services
Team for faster, more precise root cause diagnosis. StorageCare Guardian is
simple to set up through the internet and provides secure, two-way
communications with Quantum’s Secure Service Center.
More StorageCare Guardian information can be found at:
http://www.quantum.com/ServiceandSupport/Services/GuardianInformation/Ind
ex.aspx.
For further assistance, or if training is desired, contact the Global Call Center:
1-800-284-5101
Worldwide support:
http://www.quantum.com/ServiceandSupport/Contacts/Worldwide/Index.aspx
For the most up to date information on Quantum Global Services, please visit:
http://www.quantum.com/ServiceandSupport/Contacts/Worldwide/Index.aspx.
81
Loading...