Dell OneFS 8.x User Manual

H17701.2
Best Practices
Dell EMC Isilon: OneFS 8.x Best Practices for Collaborative Workflows on DaVinci Resolve 15–16.x
This document describes best practice optimizations for Black Magic Designs® DaVinci Resolve® 15–16.x with the Dell EMC™ Isilon™ OneFS operating system version 8.x, and provides guidance for designing, configuring, and maintaining systems.
March 2021
Revisions
2 Dell EMC Isilon: OneFS 8.x Best Practices for Collaborative Workflows on DaVinci Resolve 15–16.x | H17701.2
Revisions
Date
Description
June 2019
Initial release
March 2021
Updated inclusive language notification
Acknowledgements
Author: Gregory Shiff, Greg.Shiff@Dell.com
This document may contain certain words that are not consistent with Dell's current language guidelines. Dell plans to update the document over subsequent future releases to revise these words accordingly.
This document may contain language from third party content that is not under Dell's control and is not consistent with Dell's current guidelines for Dell's own content. When such third party content is updated by the relevant third parties, this document will be revised accordingly.
The information in this publication is provided “as is.” Dell Inc. makes no representations or warranties of any kind with respect to the information in this publication, and specifically disclaims implied warranties of merchantability or fitness for a particular purpose.
Use, copying, and distribution of any software described in this publication requires an applicable software license.
© 2021 Dell Inc. or its subsidiaries. All Rights Reserved. Dell, EMC, Dell EMC and other trademarks are trademarks of Dell Inc. or its subsidiaries. Other trademarks may be trademarks of their respective owners. Published in the USA. [3/26/2021] [Best Practices] [H17701]
Dell believes the information in this document is accurate as of its publication date. The information is subject to change without notice.
Table of contents
3 Dell EMC Isilon: OneFS 8.x Best Practices for Collaborative Workflows on DaVinci Resolve 15–16.x | H17701.2
Table of contents
Revisions............................................................................................................................................................................. 2
Acknowledgements ............................................................................................................................................................. 2
Table of contents ................................................................................................................................................................ 3
Document scope and audience .......................................................................................................................................... 4
About DaVinci Resolve ....................................................................................................................................................... 4
DaVinci Resolve and Dell EMC Isilon ................................................................................................................................. 4
1 Before you begin .......................................................................................................................................................... 5
1.1 Multi-platform support ......................................................................................................................................... 5
1.2 Environment design and optimization ................................................................................................................ 5
1.3 Video and audio codec choice ............................................................................................................................ 5
1.4 Isilon sizing guidelines ........................................................................................................................................ 6
2 DaVinci Resolve file system access profile .................................................................................................................. 7
2.1 File types ............................................................................................................................................................ 7
2.2 Access profiles .................................................................................................................................................... 7
2.3 Workflow phases ................................................................................................................................................ 7
2.4 Optimizing for collaboration ................................................................................................................................ 8
2.5 Representative project and sequence ................................................................................................................ 9
2.6 Dropped frames test ........................................................................................................................................... 9
3 Isilon OneFS configuration guide ............................................................................................................................... 11
3.1 Quick checklist .................................................................................................................................................. 11
3.2 Unique environments ........................................................................................................................................ 11
3.2.1 Security and access control .............................................................................................................................. 11
3.2.2 Connection protocol .......................................................................................................................................... 11
3.2.3 OneFS optimizations ........................................................................................................................................ 12
4 Resolve configuration guide ....................................................................................................................................... 15
4.1 Disk and PostGreSQL project databases ......................................................................................................... 15
4.2 Resolve settings ............................................................................................................................................... 16
4.2.1 Application settings ........................................................................................................................................... 16
4.2.2 Project settings ................................................................................................................................................. 22
5 Collaborative projects ................................................................................................................................................. 24
5.1 Mapped storage mounts and permissions ....................................................................................................... 25
6 Media management in Resolve .................................................................................................................................. 26
7 Project archive and restore ........................................................................................................................................ 28
8 Conclusion .................................................................................................................................................................. 29
A Technical support and resources ............................................................................................................................... 30
Document scope and audience
4 Dell EMC Isilon: OneFS 8.x Best Practices for Collaborative Workflows on DaVinci Resolve 15–16.x | H17701.2
Document scope and audience
This document provides guidance to system administrators and architects designing and configuring Black Magic Designs® DaVinci Resolve® to be used with Dell EMC Isilon™ storage. The document covers configurations and optimizations for both the DaVinci Resolve application and Isilon systems.
This document touches on such topics as network and workstation design, however the main focus is on Isilon storage and is not intended as a configuration guide for all infrastructure required to support DaVinci Resolve. Blackmagic Design (makers of DaVinci Resolve) have produced a hardware guide that details the supporting infrastructure.
Official hardware configuration guide can be found at in the DaVinci Resolve Configuration Guide.
About DaVinci Resolve
The following information is provided in DaVinci Resolve documentation: DaVinci Resolve integrates editing, compositing, and motion graphics, color correction, audio recording and
mixing, and finishing within a single, easy to learn application. The editing, compositing, grading, and audio
tools found in DaVinci Resolve should be immediately familiar to experienced artists who’ve used other
applications, but they’re also approachable to folks who are new to post-production.
And you can go further, using the collaborative features of DaVinci Resolve to enable multiple artists, for example, an editor, a colorist, and assistants to work together on the same timeline simultaneously, for the ultimate integrated workflow.
DaVinci Resolve and Dell EMC Isilon
DaVinci Resolve is a powerful application that brings together many workflow elements from post-production through finishing into a single place with robust collaboration tools. Performant shared storage is an essential foundation of any collaborative workflow, and Dell EMC Isilon may be deployed with DaVinci Resolve for streamlined and efficient operations:
DaVinci Resolve supports collaborative production workflows using shared project files and connected user-designated storage locations for assets.
DaVinci Resolve allows users to edit native files directly from their storage for greater speed and efficiency and a simplified infrastructure. Working in this way demands the high performance delivered by Dell EMC Isilon.
DaVinci Resolve supports native workflows where source material can be incorporated and manipulated in its native form, without the need for transcoding. Working with native and high­resolution material can make heavy demands on the supporting infrastructure, and requires the high performance, capacity and scalability of Dell EMC Isilon.
Before you begin
5 Dell EMC Isilon: OneFS 8.x Best Practices for Collaborative Workflows on DaVinci Resolve 15–16.x | H17701.2
1 Before you begin
This section describes a few important points to review before you begin optimizing your solution.
1.1 Multi-platform support
DaVinci Resolve is a cross-platform application, and versions are available to support Linux®, macOS, and Microsoft® Windows®. Before making any application optimizations, generic platform optimizations should be addressed, details of which may be available in other guides (refer to your Dell EMC support representative). It should be noted however that some application configurations contained in this guide may vary according to platform.
1.2 Environment design and optimization
Before making any application optimizations, the specific requirements for operating environments should be addressed. For example:
Networking optimization
Network Interface device configuration and optimization
Workstation RAM, CPU, and GPU
Consideration of the full workload
DaVinci Resolve and Dell EMC Isilon are capable of supporting very high throughput workflows - including multiple streams of uncompressed 4K DPX. When designing and configuring a solution to work with high throughput, it is important to pay close attention to the solution as a whole.
It must be recognized that the overhead of supporting any remote storage is different to the overhead of supporting local or direct-attached storage. When iterating through optimizations, there is only minimal value in comparing the performance of any application as it interacts with different types of storage. Instead, it there is more value in measuring improvements.
Similarly, whilst speed-test utilities may help with initial “ball park” optimizations, care should be taken to optimize with the target application. This is because not all applications and test utilities access the storage in the same way – and may (for example) use a different number of concurrent threads, or different protocol calls. Instead, there is more value in measuring improvements when using the target application. Blackmagic Design make just a such a tool, the Blackmagic Design Disk Speed Test for Windows and macOS as well as a Disk IO binary for Linux. This tool too needs to be used with care as it’s read/write caching can give misleading results.
1.3 Video and audio codec choice
The choice of video and audio codec can have an impact on the performance of the application. Different codecs have different benefits, and present different challenges to each part of a system. Fundamentally, there are three key parameters - and a system can be optimized for any two, at the expense of the third.
Data rate
Encode and decode overhead
Image quality
Before you begin
6 Dell EMC Isilon: OneFS 8.x Best Practices for Collaborative Workflows on DaVinci Resolve 15–16.x | H17701.2
For example, an uncompressed DPX sequence is very high quality and has a very low decode overhead, but it has a relatively high data rate. A high quality XAVC clip will have a relatively low data rate, but a much higher decode overhead.
Quality, data rate, and decode overhead comparison
It is important for systems architects to understand where the limits are in their systems. For instance, a workstation’s resources may be overwhelmed playing back multiple streams of ProRes 422HQ 4K 60fps content long before Isilon’s maximum throughput has been achieved.
1.4 Isilon sizing guidelines
This document does not provide detailed guidance regarding the sizing or node-type specification of Dell EMC Isilon that should be most appropriately used for any throughput or workflow.
Other documents are available that detail the performance requirements for certain workflows (for example, uncompressed 4K/UHD, or compressed HD editing). Care should be taken to size the storage correctly for performance and capacity - which may be calculated based on the requirements of the installation.
DaVinci Resolve file system access profile
7 Dell EMC Isilon: OneFS 8.x Best Practices for Collaborative Workflows on DaVinci Resolve 15–16.x | H17701.2
2 DaVinci Resolve file system access profile
This section includes a brief background on designing storage systems for media applications.
2.1 File types
For the most part, DaVinci Resolve reads and writes media files that tend to be large (often multiple GB in size). These files need to be located on a storage system with consistent, low latency because media playback is a temporal operation. The performance of any application that uses media files tends to benefit from configurations that are optimized for streaming filesystem access.
All filesystem interaction from applications that is rapid and frequent benefits from the optimizations that are brought about by filesystem metadata acceleration (an option within Isilon OneFS). Rapid and frequent access is characteristic of real-time growing media files, transcoding of media files, and frequent updating of filesystem metadata.
DaVinci Resolve includes options to read and write different type files to different directories on Isilon storage, and this feature may be used in conjunction with Isilon configuration options to make optimal choices for the different access profiles of the application. Guidance is provided later in this document.
2.2 Access profiles
DaVinci Resolve (in common with many applications of the same type) accesses files on storage in a specific manner that is very different to typical enterprise applications.
Streaming video and audio - with latency critical read profiles.
Video frame-based file access
Read-ahead application caching
Video and audio files represent moving images and audio. A key characteristic of a video or audio file is that when the file is being consumed, the contents must be temporally consistent with expectation. That is, the frames of video or audio must be displayed or presented at the right time, in sequence.
For example, a moving video is typically described as having a number of still image frames per second ­which might be fixed at 25, 30, 50 or 60 (or more), or might be variable. In all cases, the metadata information embedded in the file will contain information about the exact timing required for each frame. When the application reads those files, it must access the data for each frame in time for it to be displayed.
The requirement for this behavior demands that files must be read with a consistent latency, and with a latency that permits real-time playback. This is in contrast to a standard enterprise requirement where reading files in a less consistent way may be tolerated. For more background information, please review:
https://www.emc.com/collateral/whitepaper/h16818-wp-uncompressed-high-resolution-workflows-isilon.pdf
2.3 Workflow phases
There are four key workflow phases when interacting with DaVinci Resolve:
Import Media – Introduce media to the application
Work with Media – The Creative Process
Deliver – Exporting the competed product to the next stage in the workflow
DaVinci Resolve file system access profile
8 Dell EMC Isilon: OneFS 8.x Best Practices for Collaborative Workflows on DaVinci Resolve 15–16.x | H17701.2
Media lifecycle management – Archive and delete
Each phase of the workflow makes different demands on the storage, and requires different optimisation. Typically, a solution will be designed to optimise for the most important or demanding phase - the creative process.
At the import phase, the application may just be pointed at existing media files - but low latency filesystem access can be important when the application is required to index a high number of files. This is particularly important when working with image sequences (such as DPX), where there are a very high number of files.
Where the import phase consists of using the application to acquire linear media (SDI or IP Video), then performance and latency again become critical.
During the creative process, it is important to maintain a high-quality user experience. When designing a system, architects will need to consider both quantitative and subjective requirements.
Throughput
Latency
Demands of different codecs
Scrubbing
Multi-track
Subjective “feel”
Though hardest to quantify and measure, the subjective “feel” of an application as it is used by a Creative Artist is arguably the most important measure of a successful system design. An Artist may describe a system
that responds well and is easy to use as “snappy”, whilst a system that does not respond well and is difficult to use may be described as “sluggish”.
For a system to be “snappy”, the storage component must deliver data to the application in a way that doesn’t
interrupt the creative process. When fulfilling the delivery phase, the immediate user experience of the application is less apparent.
However, DaVinci Resolve makes high demands as it writes completed files. Depending on the nature of the file format being created, the demands may be significant - particularly for high resolution images, image sequences, and codecs that demand many small write operations.
The archive and deletion phase demands a high performance and responsiveness, as the application moves and deletes files.
2.4 Optimizing for collaboration
DaVinci Resolve simplifies collaborative workflows by seperating management of project files from the media files that are referenced within those projects (as explained in section 2 above). Nonetheless, care should be taken to avoid unexpected behavior when multiple users access the same media files simultaneously. This is especially true when the same media file is referenced in several different projects.
In large, busy facilities, filesystem permissions and other forms of access control afforded by Isilon storage should be implemented to prevent users from accidently moving or deleting data that should onot be otherwise left alone. Planning is key here: logical folder structures on the Isilon storage should be implemented and enforced by media managers and storage administrators. This is an area where 3rd party Media Asset Management software can be of particularly use in automating some of this process of organizing media files on storage.
DaVinci Resolve file system access profile
9 Dell EMC Isilon: OneFS 8.x Best Practices for Collaborative Workflows on DaVinci Resolve 15–16.x | H17701.2
2.5 Representative project and sequence
The first step in optimizing an environment is to build a project and sequence that represents the use case. This could include:
Test media clips of the correct format and encoding.
- The clips should be of sufficient duration such that their size they should exceed the available
memory capacity of the workstation. The media should be placed on the Isilon storage cluster, in a directory that has the desired settings applied (e.g. Streaming profile and Metadata acceleration, etc).
Sequences (sometimes also known as a timeline) with an appropriate number of video and audio tracks.
- The tracks should be layered, and have some effect applied such that all the video tracks are
“visible”. The sequence should match the format of the source material.
- It is important to validate that the workstation is able to support the application, and has sufficient
CPU, GPU and RAM to work with the material being tested.
Simple and complex sequences should be prepared. A simple sequence has one or more contiguous portions of media, with few (if any) edits. A complex sequence will have a number of layers, and complex edits. The purpose of a complex sequence is to challenge the application and its demands on storage.
The above example pertains to the “Edit” capabilities of DaVinci Resolve. Edit is only 1 available module in the DaVinci Resolve, but it is a good starting point. Similar representative examples can and should be created in other DaVinci Resolve modules depending on required uses, for instance the “Color” module.
2.6 Dropped frames test
The ability to play a video track (or multiple video tracks) without dropping frames is a fundamental requirement.
A perfect scenario is one where there are zero dropped frames at any point in playback. However, it can be considered acceptable to drop just a few frames during the course of a playback – particularly in the case of a long or complex sequence.
If the number of dropped frames briefly grows around edit point transitions, but then stabilizes, then that may be an indication that the application is finding it difficult to open a new file and start streaming in a timely manner. Consider checking the entire system for latency, and that the OneFS optimizations (as detailed below) have been applied.
If the number of dropped frames grows only at the start of playback, then that may be an indication that the system is unable to deal with the playback throughput surge.
If the number of dropped frames on one workstation can be negatively impacted by throughput from a different workstation, then that may indicate either a system bottleneck (for example, network) or that the applications are reaching the maximum available throughput of the storage cluster, as the workstations contend for available throughput.
DaVinci Resolve gives users several ways to detect and understand dropped frames:
DaVinci Resolve file system access profile
10 Dell EMC Isilon: OneFS 8.x Best Practices for Collaborative Workflows on DaVinci Resolve 15–16.x | H17701.2
There is a User Preference setting that can be applied which stops playback if a dropped frame is detected. This can be useful during initial system optimizations but probably should be disabled during regular use, since the application may occasionally drop a frame for any number of reasons (such as maintaining audio/video sync) and this may not be indicative of a problem.
The other indicator is at the top of the left of the Viewer. It features a colored indicator along with the currently playing frames per second. A green indicator shows that the video is playing at the specific frames per second, a red indicator shows that frames are being dropped.
Isilon OneFS configuration guide
11 Dell EMC Isilon: OneFS 8.x Best Practices for Collaborative Workflows on DaVinci Resolve 15–16.x | H17701.2
3 Isilon OneFS configuration guide
There are five categories of configuration actions to consider when configuring and optimizing an environment to support DaVinci Resolve working with Dell EMC Isilon.
Security and Access Control
Environment optimizations – e.g. networking
Platform optimizations (workstation operating system)
Storage configuration and optimizations
Application configuration and optimizations
As noted earlier, this document does not cover platform or environment optimizations since these topics are covered elsewhere (application optimizations that are specific to one platform are covered in this document). Similarly, security and access control considerations are covered elsewhere – and this document is limited to defining requirements rather than making configuration recommendations.
3.1 Quick checklist
Create Local Users or bind to Active Directory in accordance with the access policy.
Configure firewalls and internet access of client machines in accordance with security policies, and
DaVinci Resolve licensing requirements.
Create SMB or NFS Share.
SmartConnect - configure as appropriate to the environment.
Configure data layout and access patterns to Streaming.
Configure node SSDs to support Metadata Read/Write Acceleration.
Configure application for each workstation.
3.2 Unique environments
It should be noted that every environment is different, and that no one set of guidelines can be applied universally. System administrators and architects are advised to consider the implications of different configuration options, and make a selection according the to the local environment and requirements.
3.2.1 Security and access control
Media stored by an organization may be that business’ most valuable asset. In the modern age it is no longer a sensible option to simply have “read-write access for everyone, everywhere”. Proper attention should be paid to security and access control. In particular, consideration should be given to:
Access to the SMB or NFS share.
Permission to read, write, change or delete files.
OneFS supports a rich set of authentication options, access control and directory binding capabilities, and these may be used in conjunction to manage different users as they access the shared storage. Alternatively, it may be possible to use the local user provider of OneFS to manage a small number of connecting clients.
3.2.2 Connection protocol
DaVinci Resolve supports Windows, macOS and Linux and as such both SMB and NFS connection protocols need to be considered:
Isilon OneFS configuration guide
12 Dell EMC Isilon: OneFS 8.x Best Practices for Collaborative Workflows on DaVinci Resolve 15–16.x | H17701.2
For Linux workstations the protocol of choice is NFS (no surprise there).
For Windows, SMB is the way to go (also unsurprising).
- The nice benefit with SMB3 on Windows is that it Isilon storage supports SMB3 multichannel
which can deliver higher single stream throughput capability on that platform.
For macOS 10.13+ SMB is also the recommended protocol.
- This may come as surprise to experienced macOS administrators, since historically NFS was
highest performing connection protocol. However, in the latest versions of macOS, Apple’s SMBX software has seen significant improvements and now outperforms NFS for multi-threaded operations on that platform.
Configuration and optimization of the client SMB and NFS stack is outside of the scope of this document and is covered elsewhere. Care should be taken to review the connection, configuration and optimization guidelines that are appropriate for the platform being used. For environments utilizing both SMB and NFS, OneFS does a very good job of creating coherent permissions between for clients of both protocols. Some additional information about permissions and identity mapping can be found here:
https://www.dellemc.com/resources/en-us/asset/white-papers/products/storage/h13115-wp-emc-isilon-onefs­multiprotocol-security-untangled.pdf
https://community.emc.com/community/products/isilon/blog/2015/10/08/emc-isilon-multiprotocol-concepts­series
It should be noted that regardless of connection protocol, for high throughput workflows, and especially for network connections of 10GbE and greater, use of Jumbo frames (9000mtu) can have a significant positive impact. However, care must be taken that the entire connection from Isilon storage to network switch to workstation are properly configured for the benefit of jumbo frames to be recognized.
3.2.3 OneFS optimizations
Architects should consider three key optimizations for OneFS when deploying in a DaVinci Resolve environment.
Metadata Read/Write Acceleration
Streaming Mode
Filename Pre-Fetch
3.2.3.1 Metadata read/write acceleration
File access performance can in some workflows be optimised by enabling Metadata read/write acceleration. When enabled, this optimization sees OneFS use SSDs for reading and writing filesystem metadata - which can improve access time and reduce latency. Where all underlying storage units are SSD - as in the case of Isilon F800 All Flash, then this is not a meaningful or required optimization.
3.2.3.2 Enabling
Metadata read/write optimisation can be enabled as a File Pool Policy at the OneFS Web User Interface. More information and guidance is available in published documentation:
http://doc.isilon.com/onefs/8.1.0/help/en-us/index.html#ifs_c_ssd_pools.html
Isilon OneFS configuration guide
13 Dell EMC Isilon: OneFS 8.x Best Practices for Collaborative Workflows on DaVinci Resolve 15–16.x | H17701.2
3.2.3.3 Streaming mode
Streaming performance of OneFS can be optimized by enabling Streaming Mode at the file pool, or at the directory level. Streaming performance is important when playing media linearly - for example when playing a video sequence in a time line.
Consideration should be given to other workflows hosted in the same pool or directory before enabling Streaming mode. Best performance for all workflows on a single cluster might be achieved by selectively enabling Streaming Mode on a limited number of directories.
3.2.3.4 Operation
Streaming Mode optimizes two behaviors of OneFS to deliver increased streaming performance:
Data is striped across more underlying storage units (disc drives or SSDs).
Data is prefetched aggressively.
3.2.3.5 Enabling
Streaming mode can be enabled as a File Pool Policy at the OneFS Web User Interface. More information and guidance is available in published documentation:
http://doc.isilon.com/onefs/8.1.0/help/en-us/index.html#ifs_t_configure_default_io_optimization_settings.html http://doc.isilon.com/onefs/8.1.0/help/en-us/index.html#ifs_t_modify_file_and_directory_properties.html
It should be noted that when Filename Pre-Fetch (outlined below) is also required and enabled, there is an interaction between the two optimisations - which must be considered when configuring.
3.2.3.6 Filename Pre-Fetch
Overview
Filename Pre-Fetch can be enabled to optimize the streaming performance of OneFS when working with image sequences. It is critical that this optimization only be applied to directories containing image sequences of successively named files. Applying it to directories that do not contain image sequences may result in false fetches (where resources are used pre-fetching files that are not ever requested by the client).
If DaVinci Resolve is being used to work with image sequences, then improved performance may be achieved by enabling Filename based Pre-Fetch. Where image sequences are not used in the environment, Filename based Pre-Fetch should not be enabled.
Operation
Filename Pre-Fetch enables a performance optimization by detecting when image sequences are requested by a client. When a sequence is detected, OneFS will pre-fetch files from the underlying storage units (disc drives or SSDs) before they are requested.
Isilon OneFS configuration guide
14 Dell EMC Isilon: OneFS 8.x Best Practices for Collaborative Workflows on DaVinci Resolve 15–16.x | H17701.2
The FileName PreFetch algorithm is optimized to detect image sequences with either decimal or hexadecimal filename numerical increments.
Enabling
Filename Pre-Fetch can be enabled on individual directories at the command line of OneFS. A detailed overview and configuration guide is available at:
https://dellemcevents.com/uploads/Dell-ECM-Isilon-Filename-Based-Prefetch.pdf
The Filename Based Pre-Fetch guide also provides good explanations of setting and examining OneFS file layout and access strategies using the command line interface and is recommended reading even if Filename Pre-Fetch is not to being used.
3.2.3.7 Further information
For detailed guidance on File Pool Policies and data layout and access strategies, check:
https://www.emc.com/collateral/hardware/white-papers/h10719-isilon-onefs-technical-overview-wp.pdf
Guidance for using the command line to make administrative changes is available at:
https://www.emc.com/collateral/TechnicalDocument/docu84280.pdf
Guidance for using the web interface to make administrative changes is available at:
https://www.emc.com/collateral/TechnicalDocument/docu84277.pdf
Detect Sequence Pre-Fetch Files Continuing Sequence
Resolve configuration guide
15 Dell EMC Isilon: OneFS 8.x Best Practices for Collaborative Workflows on DaVinci Resolve 15–16.x | H17701.2
4 Resolve configuration guide
The focus of this section is on DaVinci Resolve settings that directly affect storage performance and interaction. The Resolve Manual (made available when installing DaVinci Resolve) goes into considerable detail on other aspects application performance tuning.
4.1 Disk and PostGreSQL project databases
DaVinci Resolve features two types of project file storage locations, Disk (local) and PostGreSQL (remote, collaborative). Resolve displays a project manager upon launch which allows for working with these storage locations and managing the projects contained within them.
In general terms, if a single user will be accessing a particular project, using the Disk database to store project files is the simplest option. For collaborative workflows and for central management of projects, use of the PostGreSQL option is typically the best choice. The DaVinci Resolve Manual (made available when installing Resolve) goes into significant detail about the various options for managing and storing Resolve project files.
PostGreSQL is required for the advanced features made possible by Dell EMC Isilon storage, for instance allowing multiple users access to the same project simultaneously. The DaVinci Resolve installer can install and configure the database. This database can either be hosted on a user workstation or on a dedicated machine acting as a project server. Use of a dedicated project sever is the recommended practice for facilities
Resolve configuration guide
16 Dell EMC Isilon: OneFS 8.x Best Practices for Collaborative Workflows on DaVinci Resolve 15–16.x | H17701.2
that have multiple rooms and shared storage (such as Dell EMC Isilon). The primary reasons for having a dedicated project server:
Project access not impacted by user workstation operations
Security, update, backup, sleep, and power cycle schedules can be more tightly managed
Significantly, the machine acting as a project server does not need access to the Dell EMC Isilon storage that contains the actual media files in use in the projects themselves. The project server is simply hosting a PostGreSQL database that contains project data. See the simplified diagram below for an example of this configuration.
It is not recommended to store the PostGreSQL database containing Resolve project data on the Isilon shared storage at all. This database should be stored on storage optimized for hosting structured data, though in most cases the local storage of the Project Server itself will suffice. A facility may choose to host backups of the PostGreSQL database on Isilon for disaster recovery and rollback purposes.
4.2 Resolve settings
Resolve has two sets of preferences that need to be considered: Application Preferences and Project Settings. As the names imply, Application settings carry between projects, while Project Settings are specific to the particular project that is open.
4.2.1 Application settings
Application Preferences are accessed via the DaVinci Resolve application menu in the top left of the main DaVinci Resolve window. Within this window there are two tabs, System and User.
Application Settings, System, Memory and GPU
Resolve configuration guide
17 Dell EMC Isilon: OneFS 8.x Best Practices for Collaborative Workflows on DaVinci Resolve 15–16.x | H17701.2
The options in the Memory and GPU sub-menu sets the maximum amount of available RAM that Resolve and Fusion will be permitted to consume. The user is invited to reserve an amount of available RAM for use by all other applications that are running on the workstation.
Where the workstation is used only to support DaVinci Resolve, it may make sense to reserve only a very small amount of RAM for other applications. If other applications are required to be active at the same time, then it may make sense to experiment with different settings.
Where DaVinci Resolve is permitted to consume more of the available RAM, the size of the application pre­fetch buffer may be permitted to grow - which in turn may deliver a more robust playback performance.
Application Settings, System, Media Storage
Resolve configuration guide
18 Dell EMC Isilon: OneFS 8.x Best Practices for Collaborative Workflows on DaVinci Resolve 15–16.x | H17701.2
These settings are critical when working in collaborative environments, especially with workstations running multiple operating systems, for instance macOS and Windows.
Obviously, the Isilon storage that holds the media files that Resolve will use should be added as a destination.
But order matters here. The first storage location listed will be the default location for Resolve’s Cache and
Gallery locations (though these defaults can be overridden by the Project Settings, described later.) The Cache location in DaVinci Resolve is used to store media clips with resource intensive processes
destructively applied to them. This allows for playback of material that otherwise would not be possible on that particular workstation or environment. The DaVinci Resolve manual goes into detail about how and when files gets created in the cache location.
In a collaborative environment, DaVinci Resolve 15 cannot share Cache for a particular project between multiple workstations. This has changed in Resolve 16, where multiple systems can share the same cache files. As such, using Isilon as the Cache location is typically going to be the best choice. Doing so centralizes the location of the cached media and avoids filling the workstation’s system drive with large cached files and in Resolve 16 negates the need for redundant copies of cache.
On the other hand, in an environment where the Isilon storage is being heavily taxed and users are making regular use of cached/optimized media to achieve consistent playback it may make sense to use a different storage location for the Cache files.
The other critical setting in this window is the Mapped Mount column. In environments with multiple operating systems, Resolve is able to map between several different locations. In the screen capture above (taken from a Windows workstation) the Windows Z:\ drive is mapped to the macOS /Volumes/f800/ location. For collaboration to work properly, it is critical that these locations are mapped coherently between workstations of differing operating system type and that each workstation has read/write access to the Gallery and Cache locations (which as mentioned above default to the first entry in the Media Storage list.
Resolve configuration guide
19 Dell EMC Isilon: OneFS 8.x Best Practices for Collaborative Workflows on DaVinci Resolve 15–16.x | H17701.2
Finally, in Windows and Linux implementations of Resolve, there is a check box for Direct I/O. This option disables some of the OS level caching of video data. In general, Direct I/O should be disabled when using Isilon storage with Resolve. That said, test results with and without Direct I/O enabled where inconsistent. For some workloads with heavy simultaneous read and write operations, enabling Direct I/O resulting in marginally better performance. As always, testing is critical in terms of real-world performance differences between platforms.
At the time of this writing, the Direct I/O option is unavailable on the macOS variant of Resolve. Blackmagic Design development has confirmed that Direct I/O is enabled at all times on macOS. However, excellent performance was still achieved with Isilon and Resolve on modern Apple hardware.
Application Settings, System, General
Audio Processing Block Size is the setting to pay attention to in this window. This setting controls how the
Fairlight module within Resolve caches and plays back audio files. The Auto setting is suitable for most environments, however for audio mixes with extremely high track counts where playback is an issue,
Resolve configuration guide
20 Dell EMC Isilon: OneFS 8.x Best Practices for Collaborative Workflows on DaVinci Resolve 15–16.x | H17701.2
increasing this value can allow for glitch-free playback. Keep in mind that higher values for the Audio Processing Block Size add latency to which may be a problem when syncing audio to picture.
It is important to understand if audio problems are being caused because of constrained Workstation processing power vs Isilon storage bandwidth before adjusting this setting. Tools such as Activity Monitor on macOS or Task Manager in Windows can help with this determination. As always testing is key. The DaVinci Resolve user manual also provides some good guidance on this setting.
Application Settings, User, UI Settings
Resolve configuration guide
21 Dell EMC Isilon: OneFS 8.x Best Practices for Collaborative Workflows on DaVinci Resolve 15–16.x | H17701.2
The check box Stop playback when a dropped frame is detected can be useful while tuning the overall environment. It can be helpful to enable this feature, make other changes to other aspects of the environment and see how it changes when playback may (or may not stop) due to dropped frames. During general use, it is may be advisable to disable this feature as the red indicator and frames per second indicator in the viewer will show if playback is consistently below requested frames per second (as described in section 2.7 above). On the other hand, when doing operations such as layout to tape (yes that is still a thing), it may be necessary to enable this feature to ensure every frame is displayed.
Application Settings, User, Project Save and Load
Under Save Settings there is an option to enable automatic project backups and an option to select the location of these project backups. Accessing and restoring these project backups is done via the Project Manager and is covered in the Resolve manual.
Resolve configuration guide
22 Dell EMC Isilon: OneFS 8.x Best Practices for Collaborative Workflows on DaVinci Resolve 15–16.x | H17701.2
There are performance implications if project backups are stored on the Isilon storage. The first time a project backup is created Resolve has to create the project database file and will copy gallery stills, in cases where the project is very complex this could take some time. The Resolve application will pause during the process backup. Subsequent project backups should be fast as Resolve only has to update this database and new gallery stills.
As of this writing, if a project has Collaboration Enabled (see section 5 below), project backups are disabled, meaning these settings are ignored for those projects. If Resolve made backups for a project prior to enabling collaboration, those backups will still be available to the individual workstation that made the backups. Similarly, if Collaboration is disabled on a project, Resolve will resume making backups.
Project backups are only available to the individual workstation that made those backups. Therefore, storing project backups on the Isilon storage is of limited utility and given the possible performance implications of backing up complex projects, keeping project backups pointed at local workstation storage is going to be the best choice in most circumstances. The good news is that when an individual workstation restores a project backup back to the Project Manager, all workstations can access the restored project.
4.2.2 Project settings
Project Settings are accessed via the File menu in DaVinci Resolve.
Project Settings, Master Settings, Optimized Media and Render Cache
There is an entire chapter in the DaVinci Resolve manual dedicated to improving playback performance. As
of this writing it’s in Part 1, Chapter 6 of the DaVinci Resolve manual. Clearly, this is a deep topic that is
largely out of scope of this guide. However, using Render Cache can allow for playback of material that otherwise might be beyond the capabilities of a particular system, either due to workstation CPU/GPU or bandwidth constraints.
As discussed earlier in this guide, typically the Render Cache is going to be located on the Isilon shared storage. Given that is the case, it is important to understand what kind of media is being written to this cache and at what resolution, as it will add extra load to the Isilon as it being created.
Resolve configuration guide
23 Dell EMC Isilon: OneFS 8.x Best Practices for Collaborative Workflows on DaVinci Resolve 15–16.x | H17701.2
This setting controls just what resolution and codec of video is being created during the Render Cache process. More complex and higher resolution codecs will cause more load.
Project Settings, Master Settings, Working Folders
The default location for Cache files and Gallery stills is the first storage location listed in Application Settings, System, Media Storage. The working folder section overrides these default locations for the Render Cache and Gallery Stills.
Cache location is described in detail in section 4.2.1 above under Application Settings, System, Media Storage.
The Gallery Stills setting is where still images used in the color grading process are stored. DaVinci Resolve supports shared Gallery Stills in collaborative projects and as such these still should be stored on the Isilon storage. DaVinci Resolve stores these images in a hidden .gallery directory in this location.
Project Settings, Capture and Playback, Capture
The settings here determine DaVinci Resolve’s behavior when capturing clips from tape or via film scanner into the media pool. Typically, these clips will be saved onto a location on the Isilon shared storage. It is important to verify that the Isilon storage being used has enough capacity and bandwidth to capture the requested material and format.
Collaborative projects
24 Dell EMC Isilon: OneFS 8.x Best Practices for Collaborative Workflows on DaVinci Resolve 15–16.x | H17701.2
5 Collaborative projects
Using a Project Server to hold project data allows for multiple users to access a single DaVinci Resolve project simultaneously. Information on configuring a project server is discussed in section 4.1 above.
Collaboration must be enabled on each individual project. Collaboration can be enabled in the Project Manager by right clicking on the project name and selecting Enable Collaboration
Once enabled, individual users can lock bins within the project and even chat with one another from within the application. For collaborative projects to work properly, it is critical that Application Settings, System, Media Storage be configured appropriately on each workstation accessing the shared projects. This is described in Section 4.2.1 above.
It is also critical that Project Settings, Master Settings, Working Folders, described in section 4.2.2 be set to directories that are mapped on each workstation. This is straight forward if these working folders are located on the Isilon storage. If the working folders are local folders to each workstation, then each workstation must have equivalent paths available to avoid errors.
Collaborative projects
25 Dell EMC Isilon: OneFS 8.x Best Practices for Collaborative Workflows on DaVinci Resolve 15–16.x | H17701.2
5.1 Mapped storage mounts and permissions
Storage Mapping (as discussed in section 4.2.1 above) is probably the most important setting to have configured when using project collaboration. Given the breadth of functions available in DaVinci Resolve, facilities are likely to have workstations with a mix of OS types on them. Mapped Mount settings need to be configured for each workstation.
File system permissions also come into play here. Each workstation/user is going to need read+write access to the Gallery and Cache locations, which are defaulted to the first storage mapping (but may be overridden on a project by project basis). At least read access to the location of the media files for the project will need to be provided. Assuming all workstations are either macOS or Windows, ACL permissions in OneFS should be sufficient. However, if a Linux workstation is in the mix, POSIX permissions become relevant. OneFS supports both types of file system permissions and has robust mechanisms for creating coherency between them. Links to information about cross protocol permissions mapping can be found in section 3.2.2. above.
Media management in Resolve
26 Dell EMC Isilon: OneFS 8.x Best Practices for Collaborative Workflows on DaVinci Resolve 15–16.x | H17701.2
6 Media management in Resolve
Resolve offers the ability to consolidate and clean up media used in a project. This option is accessed via the File > Media Management menu option. Resolve projects tend to involve very large files that can quickly consume available storage space. As such, implementing effective media management is important. Having this step baked into the workflow is essential to maintaining free space on the Isilon storage.
If in the course of completing a project, the media needs to be moved from its original storage location to a new location, Media Management is going to be the tool to use in order to keep the links in project file current and aware of the location of all the referenced media. This is also true when restoring archived project. For instance, if an archived project is restored to nearline storage and and needs to be moved to a production storage, using Media Management to move the data is the proper technique.
In workflows and facilities where the same media is re-used in multiple projects, it may be best to use the copy function to bundle up all the files for a project in preparation for an archive step and manually remove data that is specific to a particular project. Of note, when using the Movefeature of the Media Management function in Resolve, there is an option to delete unused media. Great caution should be applied when using this function as it does destructively delete those files from disk. The Media Management function will delete media even if that media is in use in another project under management by the same Project Server.
Media management in Resolve
27 Dell EMC Isilon: OneFS 8.x Best Practices for Collaborative Workflows on DaVinci Resolve 15–16.x | H17701.2
Isilon storage features a variety of node types with differing performance characteristics and the ability to tier between these nodes seamlessly. Facilities could reserve their highest performing nodes for production work and use lower performing nodes as nearline or archive storage for older projects, all within a single Isilon OneFS file system for easy immediate access.
Project archive and restore
28 Dell EMC Isilon: OneFS 8.x Best Practices for Collaborative Workflows on DaVinci Resolve 15–16.x | H17701.2
7 Project archive and restore
DaVinci Resolve projects can be archived and restored using the Project Manager. Project archiving is accessed by right clicking on the project in the Resolve Project Manager and selecting Export Project Archive… The DaVinci Resolve manual goes into some detail about how to use this function. It goes hand in hand with the media management function above.
Keeping a shared storage environment tidy is often a challenge. Using the built-in Resolve tools for archiving projects can simplify this process considerably. When archiving the project, Resolve will copy all media in use in that project and gives the option for copying Cached and Optimized media as well. There are no particular implications here for Isilon storage users.
Restoring archived projects does have implications in Isilon environments. As mentioned in section 3.2.3 above, there are folder specific settings in OneFS that can and should be implemented for streaming media and for file-name prefetching of image sequences. If media is stored in directories with these particular settings applied, when the archived project is restored, those file system settings are not carried along with them. Therefore, it is recommended that administrators re-apply streaming or file-name prefetch settings within OneFS as required in the media storage locations for projects restored from archive.
Conclusion
29 Dell EMC Isilon: OneFS 8.x Best Practices for Collaborative Workflows on DaVinci Resolve 15–16.x | H17701.2
8 Conclusion
DaVinci Resolve and Dell EMC Isilon are a powerful combination for post-production workflows. While Resolve users have traditionally relied upon SAN storage, Isilon NAS provides performance with simplicity that complex SAN infrastructures cannot. With an understanding of a few OneFS optimizations and Resolve preferences, administrators can empower creatives to focus on making great content.
Technical support and resources
30 Dell EMC Isilon: OneFS 8.x Best Practices for Collaborative Workflows on DaVinci Resolve 15–16.x | H17701.2
A Technical support and resources
Dell.com/support is focused on meeting customer needs with proven services and support. Storage technical documents and videos provide expertise that helps to ensure customer success on Dell
EMC storage platforms.
Loading...