Document Title, 6-66957-01 Rev A, June 2010, Product of USA.
Quantum Corporation provides this publication “as is” without warranty of any kind, either express or
implied, including but not limited to the implied warranties of merchantability or fitness for a particular
purpose. Quantum Corporation may revise this publication from time to time without notice.
Your right to copy this manual is limited by copyright law. Making copies or adaptations without prior
written authorization of Quantum Corporation is prohibited by law and constitutes a punishable
violation of the law.
TRADEMARK STATEMENT
Quantum, the Quantum logo, DLT, DLTtape, the DLTtape logo, Scalar, and StorNext are registered
trademarks of Quantum Corporation, registered in the U.S. and other countries.
Backup. Recovery. Archive. It’s What We Do., the DLT logo, DLTSage, DXi, DXi-Series, Dynamic
Powerdown, FastSense, FlexLink, GoVault, MediaShield, Optyon, Pocket-sized. Well-armored, SDLT,
SiteCare, SmartVerify, StorageCare, Super DLTtape, SuperLoader, and Vision are trademarks of Quantum.
LTO and Ultrium are trademarks of HP, IBM, and Quantum in the U.S. and other countries. All other
trademarks are the property of their respective companies.
Specifications are subject to change without notice.
Quantum Partial File Retrieval allows you to quickly retrieve and utilize
segments of large media files—rather than the entire file— based on
timecode parameters.
Segments of large media files that have been archived to tape using
Quantum StorNext Storage Manager can be requested using a simple
XML request file. Quantum Partial File Retrieval instructs StorNext to
retrieve only the necessary portion from tape, and then presents that
segment as a complete MXF or QuickTime file.
If the file has not yet been truncated by StorNext Storage Manager, the
result will still be the same. However, in this case the Partial File will be
generated directly from the complete media file on disk.
The Quantum Partial File Retrieval feature requires a dedicated
Windows-based server on which to run. For larger scale installations, the
Quantum Partial File Retrieval functionality can be distributed across
multiple Windows server machines.
Quantum Partial File Retrieval consists of two primary components: an
indexer which indexes media files before they are archived to tape and
truncated, and a Partial File Retrieval service that performs the
requested retrievals.
Partial File Retrieval Indexer
Chapter 2
Retrieval Works
The Partial File Retrieval Indexer looks for new media to appear in
StorNext Storage Manager. When it finds a new media file it generates
an index for it, in effect mapping timecode values in the original media
file to byte offsets. This allows subsequent timecode- based requests to
retrieve only the necessary portions of a large media file from tape.
The index files, which are written as QuickTime reference files, are
generated in a parallel folder structure on the managed storage. The
two file locations below represent a MXF based media file and its
associated index file.
Chapter 2: How Quantum Partial File Retrieval Works
Partial File Retrieval Service
Note: The index files do not contain any content themselves, and are
therefore very small in comparison with the files they index.
It is important that the indexed files themselves are never
truncated.
The Indexer does not attempt to index files that it thinks might
be currently in-record. Therefore, the first time the indexer
encounters the file, there may be some delay before a
particular file is indexed while the indexer is trying to establish
if the file is currently being written to.
Partial File Retrieval Service
Folder StructureWithin the Quantum Partial File Retrieval Configuration Tool, it is
possible to set up multiple sets of folders for different request types. For
example, a set of folders for news requests might look like this:
Chapter 2: How Quantum Partial File Retrieval Works
Partial File Retrieval Service
4 If an error occurred in retrieving the partial file, the XML request file
will be moved to the
fail
folder.
Issuing a RequestA Partial File Retrieval request is issued in the form of a small XML file,
the format of which can be seen in the example later in this User Guide.
The XML file simply describes a movie (via a full UNC path) with an in
and out timecode value.
Quantum PFR will handle a number of the requests simultaneously with
the overall retrieval performance dependant upon the quantity of tape
drives available.
The number of tape drives limits how many requests can be processed
simultaneously. Once all tape drives are busy, additional requests will be
queued until a tape drive becomes available.
Monitoring RequestsRequests will be processed automatically once the request file is
detected. The Partial File Retrieval service automatically detects the XML
request file, appends the processed request to the queue of pending
requests, and processes them in the order received. XML request files
are moved to the
status of the request.
success
or
fail
folder as appropriate based on the
Use the PFR status GUI to monitor requests, cancel pending or active
requests or change the priority of the pending requests.
Quantum Partial File Retrieval consists of the following software
components:
• PFR Control Server and Configuration GUI: This is the core of the
system and the tool that configures it
• PFR Transfer Engine: This is the engine that retrieves and creates
the partial file
• PFR Status GUI: This is the user application which allows
monitoring and modifying the queue of pending and in-progress
retrievals
Chapter 3
• Index Engine and Configuration / Status GUI: This is the service
that indexes media files before they are truncated
• Virtual Machine (VM) with SNAPI and PFR Linux Component:
This is the interface to the StorNext API (SNAPI)
A typical installation of Quantum Partial File Retrieval installs all of the
above components on a single Windows-based server machine. This is
fine for most installations, but for larger installations the components
can be distributed as follows:
• PFR Server: Includes "PFR Control Server and Configuration GUI" and
• PFR Client (aka Status GUI): Allows the PFR queues to be remotely
managed
• PFR Indexer: Includes "Index Engine and Configuration / Status
GUI". Would most likely be installed on its own on a separate
machine when additional performance is required.
• PFR Transfer Engine: Again, would most likely be installed on its
own on a separate machine when additional performance is
required.
Prerequisites
Operating Systems• When installing on a single machine or for any of the individual
components: Windows Server 2003 or Windows Server 2008, 32-bit
or 64-bit.
• When installing the PFR Client (Status GUI) alone, this could also be
installed on Windows XP or Windows Vista
StorNextThe StorNext metadata controller must have Storage Manager and
StorNext API (SNAPI) installed.
VMWareThe "VM w/ SNAPI and PFR Linux Component" is actually a small piece of
Linux software that provides the communications link between the PFR
Server and the StorNext API. This is hosted in VMWare Server on the PFR
Server machine.
VMWare Server should be installed and running prior to installing the
Quantum PFR Components. You can find VMWare Server here:
http://www.vmware.com/products/server/
Note: When installing separate components on separate machines
for performance reasons, VMWare Server is required only on
the Server machine onto which the PFR Server is installed.
Note: The VMware server configuration may have to be changed
after installation to make sure the "standard" virtual machine
store is pointing to "C:\Virtual Machines." By default, the
VMWare installer chooses the biggest drive for the "standard"
virtual machine store, which is often the StorNext system. If
this is not changed, the PFR Installer cannot automate
registering and starting the "VM w/ SNAPI and PFR Linux
Component".
Note: On some systems that already have an instance of Apache web
server already installed, the VMware web service may fail to
start. For this reason we recommend installing on a machine
that has no web server software already running.
BonjourBonjour, also known as zero-configuration networking, enables
automatic discovery of computers, devices, and services on IP networks.
Bonjour uses industry-standard IP protocols to allow devices to
automatically discover each other without needing to enter IP addresses
or configure DNS servers.
Quantum Partial File Retrieval uses Bonjour in order to discover the "VM
w/ SNAPI and PFR Linux Component". Bonjour should be installed prior
to installing Quantum Partial File Retrieval components. You can find
Bonjour here:
Note that when installing separate components on separate machines
for performance reasons, Bonjour is required only on the machine(s)
onto which the PFR Server and PFR Transfer Engine are installed.
User AccountsThe PFR Control Server and the PFR Transfer Engine both require a
specific user to be specified to run the Services. This user must have full
read and write access to the StorNext storage in order to perform partial
file retrievals. The same user is used for both components.
If you are performing an installation where some components are
installed onto separate server machines, the user must be available on
The StorNext file system should be shared via SMB (standard Windows
share). The security attributes of the Windows share on the file system
must be set to “Full Control.”
Installation
The installation starts with an options screen that allows either a "Full
Install" or any of the individual components to be installed:
Select any of the options and press the Install button, and then follow
the installation instructions to complete the installation. The "normal"
installation will be completely performed by selecting "Full Install".
Note: During the installation process you will be asked for a PFR User
Name and password: once for the PFR Server installation and
once for the PFR Engine installation. Enter the User Account
information that was created during the prerequisites stage.
During the PFR Server installation the installer copies, registers and
starts the "VM w/ SNAPI and PFR Linux Component". Since this Virtual
Machine image is very large, this can take quite a long time to complete.
This is completely normal.
Most of the Quantum Partial File Retrieval configuration is handled via a
single configuration tool (the Primary Configuration). However, an initial
manual configuration step is required before this is run.
In addition, since the PFR Indexer can be installed on a separate
machine, this element is also configured separately (although it can be
called from the main configuration application if both are installed on
the same machine).
SNAPI / PFR Linux Component Configuration
Chapter 4
The connection between the PFR Server and the StorNext API (SNAPI) is
accomplished via a small Linux application that resides in a Virtual
Machine, hosted on the same machine that runs the PFR Server. This
requires configuration before it can be used for the first time.
The Virtual Machine will be installed and launched by the PFR Server
installer. To launch the Virtual Machine Manager and to ensure that the
VM is running, launch the VMWare Server Home Page. There should be
a shortcut to this on the desktop (created when VMWare Server was
installed). Alternatively, point your web browser at the following URL:
Chapter 4: Configuration
SNAPI / PFR Linux Component Configuration
https://localhost:8333
This will request the following login screen from within your web
browser.
Note: You may be warned by your web browser that this site does
Figure 5 VMWare Login
not have a valid certificate. You should ignore this warning and
continue to the site.
Type the username and password for an account on the current
machine which has Administrative privileges.
It is necessary to manually configure VMWare Server to automatically
start/stop the virtual machines when the Windows host OS starts/shuts
down.
To d o t h is :
1 Click the computer name in the Inventory tab (not the name of the
virtual machine), and select the Summary tab.
2 Click Edit Virtual Machine Startup/Shutdown Settings (on the far
3 Check the box beside Allow virtual machines to start and stop
automatically with the system, and in the Default Shutdown
Delay box, select Suspend.
4 Highlight the virtual machine in the Startup Order list and click
Move Up to put it in "Any Order".
5 Click OK to close the window.
In the Inventory that appears on the left, there should be an item called
"Linux_SNAPI". This is the name of the Virtual Machine that runs the "VM
w/ SNAPI and PFR Linux Component". It will appear as follows:
Chapter 4: Configuration
SNAPI / PFR Linux Component Configuration
Figure 12 Edit Network
Settings
2 Set up the network settings according to the requirements of your
network configuration. If you are not using DHCP, the specific
settings are outside of the scope of this User Guide. Please consult
your Network Administrator.
3 Click Next to configure. A configuration overview appears.
ArchiveThe Archive pane allows you to enter the machine on which the storage
resides (the archive host) and the SNFS share on this storage that will be
used for access to this storage. These can either be typed in manually or,
better, looked up using the "..." buttons.
The SNAPI Host should be populated automatically. If it does not
populate automatically, ensure that the SNAPI application is running in
the Virtual Machine and that Bonjour is installed and running (see
Chapter 3, Installation).
The path where the file system has been mounted on the StorNext
system (e.g. stornext/snfs1).
Press Next to continue.
Job Results FolderIt is possible to create up to twenty Job Results Folders for the
application, each for different roles within your organization (for
example, News, Sports and Current Affairs). This page allows you to set
up the first of these folders.
The Name is simply a friendly Name that is displayed in the Status GUI,
while the Location is the UNC path to the parent location of the folder
set (see
Folder Structure on page 4).
Finally, select the desired File Type for this Job Result: either MXF (the
default,) or QuickTime. Note that this setting will generate partial files
of the specified type, irrespective of the source file format.
Press Next to continue.
Work FoldersAt this stage most of the configuration has been completed. The Work
Folders page provides a summary of the Archive and Job Results folders
that have been set up so far. From this point you can do the following:
•Use the Add Directory button to add an additional Job Results
folder (up to a maximum of 20), or Edit or Remove to change or
delete previously configured folders
•Use Import License to import a permanent license (which replaces
the 30-day temporary license provided upon initial installation)
Transfer EnginesIn a standard installation where everything is installed onto a single
machine, there will be only a single Transfer Engine, located on the
current machine (localhost). This will be configured automatically and
no further configuration is necessary.
However, in more extensive installations where Transfer Engines have
been installed on separate Windows server machines, this is the place to
specify the location of the separate machines.
This window also allows you to specify the number of simultaneous
transfers that should be attempted at once. The value entered here
should not exceed the number of tape drives available to perform
restores.
Press the Next to continue.
IndexerConfiguring the Indexer is accomplished using a separate application
since the indexer might be physically present on a different Windows
server machine. However, in case it is present on this machine it can be
launched here for convenience. Please see the next section (Indexer
Configuration and Monitoring) for more details.
Note that the Indexer configuration can also be launched separately.
Chapter 4: Configuration
Indexer Configuration and Monitoring
Figure 22 PFR Index
Configuration
On this screen you can do the following:
• Set one or more Archive Locations for which you would like to
create indexes. To make changes, you can use the Add and Remove
buttons on the right, or right-click with the mouse on the Archive Locations list. Ideally, the locations entered here will only contain
media files.
• Set the Log (logging) level.
•Set the Scan Interval (in minutes). 60 minutes is the recommended
interval, but the figure is a balance between speed of indexing and
the load the indexer puts onto the storage. there is no benefit in
creating indexes as soon as possible as long as they are created
before the files are truncated by StorNext Storage Manager.
•The Listen Port is the TCP/IP port that this application and the
Indexer Service use to communicate on. The default TCP/IP port is
515, and this should not be changed.
After any changes have been made, click Apply to save and commit
your changes.
MonitoringThe Indexer tool also allows monitoring of the indexing process. As the
Indexer Service scans the configured Archive Locations, log messages
are written to the Log Window:
By default, the PFR Services should handle all retrieval requests
automatically. However, should you wish to monitor the process,
manipulate the pending requests or look at the log files, this can all be
done using the Status GUI application.
Anatomy of Status GUI
Chapter 5
Using the Status GUI
Application
The left side of the Status GUI screen lists the PFR source requests, which
are the XML request files that have not yet been processed, categorized
according to the folder sets defined in the configuration.
On the right are the destinations: one tab (along the bottom) for each
configured folder set. The entries on the right represent pending, inprogress and complete (or failed) retrievals.
Chapter 5: Using the Status GUI Application
Manipulating the Retrievals
Figure 24 Status GUI Screen
Manipulating the Retrievals
The following actions can be performed on retrievals using the Status
GUI. In the following descriptions, any action will reflect the selected
transfer.
• Prioritization. Using the set of five buttons below the Destination Servers, a pending transfer can be given high or lower priority by
moving the pending transfer to the top, up, down or bottom of the
list. All high priority transfers will take place before all standard
priority transfers.
• Delete. Using the trash icon in the bottom right, pending transfers
can be removed. That also cancels active retrievals and deletes any
complete transfers (it will remove the partial file).
• View Log File. From the Help menu it is possible to view the current
log file. If the Status GUI is run on the same machine which runs the
main PFR services, this will be the log file which also contains all of
the retrieval information.
ClipThere can only be one clip provided
UIDA unique ID for the source clip, but otherwise not used
TitleName of the (partial) output file
FileThe full UNC path to the source file on the StorNext managed
storage
StartStart timecode of partial clip to be retrieved
EndEnd timecode of partial clip (inclusive - e.g. timecode of last
frame of partial clip)
NumberVideoTracksNumber of video tracks in source clip. Should always be "1"
NumberAudioTracksNumber of audio tracks in source clip