The information in this document is subject to change without notice.
Hewlett-Packard makes no warranty of any kind with regard to this manual, including, but
not limited to, the implied warranties of merchantability and fitness for a particular purpose.
Hewlett-Packard shall not be held liable for errors contained herein or direct, indirect,
special, incidental or consequential damages in connection with the furnishing, performance,
or use of this material.
Warranty. A copy of the specific warranty terms applicable to your Hewlett-Packard product
and replacement parts can be obtained from your local Sales and Service Office.
Restricted Rights Legend. Use, duplication or disclosure by the U.S. Government is
subject to restrictions as set forth in subparagraph (c) (1) (ii) of the Rights in Technical Data
and Computer Software clause at DFARS 252.227-7013 for DOD agencies, and
subparagraphs (c) (1) and (c) (2) of the Commercial Computer Software Restricted Rights
clause at FAR 52.227-19 for other agencies.
Hewlett-Packard Company
19420 Homestead Road
Cupertino, California 95014 U.S.A.
Use of this manual and flexible disk(s) or tape cartridge(s) supplied for this pack is restricted
to this product only. Additional copies of the programs may be made for security and back-up
purposes only. Resale of the programs in their present form or with alterations, is expressly
prohibited.
B.5 Competitor’s Claims for NFS Non-Locking Protection ..........................................48
3
Chapter 1 Introduction
A key feature of HP’s CIFS/9000 product is file server consolidation on a UNIX
platform with transparent shared file access between Windows and UNIX clients. A major
challenge of shared file access is providing secure, fast access for Windows, UNIX, and mixed
client environments. File access security is implemented using file locking schemes
typically executed at the file system level and/or application level, but while Windows
provides a comprehensive locking model, UNIX and NFS only provide an advisory locking
model. The significant differences of the platform locking models results in integration
opportunities and issues for CIFS/9000, and these must be clearly identified and addressed
for CIFS/9000 potential installations.
4
Chapter 2 CIFS/9000 Product Overview
CIFS is a file system specification called Common Internet File System, which evolved from
Microsoft’s SMB protocol for Windows operating systems. CIFS/9000 is an HP-UX product
(based upon Samba version 2.0.6, 2.0.7 in 3/2001, or 2.0.9 in 9/2001) that utilizes the
Common Internet File System to provide two major benefits for two platforms. First, the
CIFS/9000 Server delivers smooth file and print sharing access to both Windows and UNIX
platform clients, but with all of the benefits of a HP-UX operating system. Second, the
CIFS/9000 Client delivers HP-UX client access to both Windows and UNIX file servers, and
integrated user authentication between HP-UX and Windows domains using NTLM.
CIFS/9000 allows management consolidation of many Windows servers onto a reliable,
highly available HP-UX operating system, while still delivering file access to Windows
clients utilizing their native CIFS (SMB) protocol. CIFS/9000 also provides a common
platform for efficient file sharing from varying client access methods: Windows, UNIX, NFS,
or PC/NFS. The versatility of CIFS/9000 allows Windows-only file serving, UNIX-only file
serving, or mixed file serving.
To ensure integrity and security of user data files despite the variable client access methods,
file locking can be enabled at the file system and/or application level.
5
Chapter 3 File Locking Overview
File locking is typically initiated by the file system – either by configuration parameters or
API and function calls – to prevent data corruption by more than one process accessing a file
while it is open in write mode (either dual writes occurring, or reading a file that has since
been changed due to a write – “stale data”). There are two important factors to consider
when enabling file locking: first – when to use file locking; second – how to lock the file.
File locking provides a security and/or integrity benefit, but the benefit is not free. Locking
uses system resources, therefore it can affect performance. In addition, some kinds of locking
can actually enhance performance, but expose data to corruption under certain types of
access. Also, using file-system -provided file locking can be redundant if data is already
locked or managed at the application level, thus using system resources affecting
performance for no benefit. Finally, how data is used should partially dictate how files are
locked: if a file is to be opened read-only, then locking the file may not make much sense.
HP Sales Force personnel often encounter RFPs that include CIFS/NFS cross-platform file
locking as a requirement. It is essential that the TC or SR understand what security and
data integrity issues are driving the file locking requirement. Customers often include such
requirements without understanding their own environment.
HP-UX 11 CIFS/9000 / NFS File Server & Storage
Files Accessed by
Windows Clients
Shared CIFS/NFS
File Access
Files Accessed by
NFS Clients
Locking Needed?Data MgtACLrwxFilename
No data managementNo ACLRead onlyFilename1
No data managementACE on ACLRead/WriteFilename2
PDM, Clearcase, etc…No ACLRead/WriteFilename3
No data managementNo ACLRead/WriteFilename4
YES – Locking Needed
NO
NO
NO
1. Are there files or directories that will be accessed by both CIFS and NFS?
2. How many files or directories, and what kind of data?
3. Will clients have write access to the files or directories?
4. Are there ACLs on the file or directories that manage client access?
5. Is there a data management application that administers access (Clearcase or PDM server)?
6
Educating customers about when file locking is required will reduce the
possibility of this feature being a lockout for bidding a deal2.
How a file is locked by a file system depends upon the locking features that are invoked
(Windows has multiple locking modes), or what platform the file resides upon (Windows or
UNIX), or what access method the client is using to open the file. With CIFS/9000 providing
file-sharing access to multiple client types, the shared (heterogeneous) access combinations
can become confusing. We will start to clarify locking behavior in these combinations by
explaining how each access method locks a file on its own file system (homogenous).
2
See Appendix B
7
Chapter 4 File Locking Implementations
File systems and pseudo file systems (like PC-NFS) implement autonomous file locking
models. These file-locking models are invoked by the particular file system, and do not
necessarily interoperate, or even know each other exist. Thus, when multiple client access
methods open the same file – even with valid locking procedures for the opening file system –
the locks can be ignored, resulting in concurrent file access and likely data corruption. To
understand how the different file locking mechanisms interoperate (or do not interoperate),
we must first understand how file locking is implemented for homogeneous access on the
following file systems:
Windows (CIFS),
UNIX (JFS, HFS, etc)
NFS,
PC-NFS.
4.1 WINDOWS (CIFS)
The Windows file system (using CIFS) utilizes three different file-locking mechanisms:
MANDATORY LOCKING is invoked at file open by the Windows Createfile API. Locking
parameters supplied to Createfile include access mode and share mode.
Access mode defines how an application (caller of Createfile) wants to access the file:
Read: Read access only
Write: Write access only
Read-Write: Read and Write access
Share mode defines how an application wants to limit or grant concurrent access while it
has the file open, essentially comprising the locking scenario for MANDATORY LOCKS:
Deny-None: Concurrent read and write access allowed
Deny-All: No concurrent read or write access allowed
Deny-Read: No concurrent read access allowed
Deny-Write: No concurrent write access allowed
These are Windows default file locking mechanisms. There are obscure ways to disable
mandatory locks, but they should never be disabled (which is why they are obscure).
Windows documentation uses the terms “mandatory” and “share mode”. However, it is
helpful to think of mandatory share mode locking as “OPEN MODE LOCKING”, because it is
initiated during a file open, so subsequent references will use “Mandatory Share Mode (Open
Mode) Locking”.
8
BYTE RANGE LOCKING is invoked programmatically – on a file that is already open - by
the Windows LockFile and LockFileEx APIs when a client application wants to lock a region
of a file for exclusive access.
LockFile parameters only specify the file name and byte ranges to be locked. Therefore the
lock must be exclusive – all other processes are denied access to the specified region of the
file. If the region cannot be locked, the API returns immediately – it does not block or wait.
LockFileEx includes a flags parameter that allows the process to wait for a locked region of a
file to unlock. It also includes a sharing option that allows another process to read the locked
region concurrently but denying write access concurrently.
Byte Range locks are honored unilaterally by Windows and Windows applications that call
LockFile and LockFileEx, so a process is guaranteed to have the particular access granted by
the API when the call succeeds.
OPPORTUNISTIC LOCKING (Oplocks) is invoked by the Windows file system (as opposed
to an API) via registry entries (on the server AND client) for the purpose of enhancing
network performance when accessing a file residing on a server. Performance is enhanced by
caching the file locally on the client which allows:
Read-ahead: The client reads the local copy of the file, eliminating network latency
Write caching: The client writes to the local copy of the file, eliminating network
latency
Lock caching: The client caches application locks locally, eliminating network
latency
The performance enhancement of oplocks is due to the opportunity of exclusive access to the
file, even if it is opened with deny-none, because Windows monitors the file’s status for
concurrent access from other processes.
Windows defines 4 kinds of Oplocks:
Level1 Oplock - The redirector sees that the file was opened with deny none (allowing
concurrent access), verifies that no other process is accessing the file, checks that oplocks are
enabled, then grants deny-all read-write exclusive access to the file. The client now performs
operations on the cached local file.
If a second process attempts to open the file, the open is deferred while the redirector
“breaks” the original Oplock. The Oplock break signals the caching client to write the local
file back to the server, flush the local locks, and discard read-ahead data. The break is then
complete, the deferred open is granted, and the multiple processes can enjoy concurrent file
access as dictated by mandatory or byte-range locking options. However, if the original
opening process opened the file with a share mode other than deny-none, then the second
process is granted limited or no access, despite the Oplock break.
Level2 Oplock – Performs like a level1 oplock, except caching is only operative for reads. All
other operations are performed on the server disk copy of the file.
Filter Oplock – Does not allow write or delete file access.
Batch Oplock – Manipulates file openings and closings.
9
An important detail is that Oplocks are invoked by the file system, not an application API.
Therefore, an application can close an oplocked file, but the file system does not relinquish
the Oplock. When the Oplock break is issued, the file system then simply closes the file in
preparation for the subsequent open by the second process.
4.2 UNIX
UNIX utilizes 2 kinds of file locking:
Advisory locking
Mandatory locking
Both lock types are implemented with the same function calls: fcntl and lockf. fcntl is the
POSIX interface (introduced at SVR4), and lockf is provided for backward compatibility. The
calling semantics are slightly different, but the functionality is basically the same.
Both functions specify read and/or write locking, and allow for byte-range locks or locking of
the entire file. A read lock allows concurrent access for reading, but not writing. A write
lock is exclusive.
ADVISORY LOCKING assumes that all processes attempting to access a file participate in
the locking protocol by calling the locking functions correctly and honoring the lock status of
the file. There is no requirement that a process must adhere to the advisory locking protocol.
A process that does not participate in the protocol can obliviously write or read to a file that
is locked with locking functions. Therefore there is a significant assumption of risk of data
corruption with advisory locking.
MANDATORY LOCKING is implemented with function calls identical to advisory locking.
Mandatory locking is then enabled per-file by manipulating the file’s permissions with
chmod. To enable a file for mandatory locking, the group-id bit must be set, and the execute
bit must be cleared (implying that executable files cannot be locked). Once set, any calls to
open, read, or write to a previously locked file are failed by the kernel, regardless of whether
the process participates in the file locking protocol. Most UNIX locking scenarios refer to advisory locking protocol. The UNIX mandatory option seems to be rarely utilized, so further
references to mandatory locking refer to Windows only.
Note that UNIX does not employ the concept of Mandatory Share Mode (Open Mode) locking
like Windows does. UNIX locking consists of byte range locking and is strictly programmatic
on a previously opened file. Also, UNIX is incapable of sending an oplock break to a client,
and therefore cannot interoperate with Windows clients using Opportunistic Locking.
4.3 NFS
File locking for NFS is provided by two cooperating protocols:
Network Lock Manager
Network Status Monitor
10
NETWORK LOCK MANAGER - like UNIX - uses an advisory locking protocol: an NFS
server can obliviously grant a client read and/or write access to a file that has already been
locked by another client. The NLM assumes that all processes attempting to access a file
will follow locking protocol by calling the locking functions correctly and honoring the lock
status of the file. The NLM honors read and/or write locking, and allows for byte-range locks
or locking of the entire file. A read lock allows nonexclusive concurrent access for reading,
but not writing. A write lock is exclusive. If a process is denied access due to a conflict in
lock type, it can optionally block until the lock is freed or it can return an error condition.
UNIX advisory locks are propagated to the NFS Network Lock Manager. The UNIX locking
process will interoperate with the NLM to implement the fcntl file locking calls via the NFS
mount to the actual remote disk file. Thus the file is protected from other UNIX processes
that access the file – either locally on the NFS server or via multiple NFS mounts of the
same file system – and adhere to the advisory locking protocol.
Note that UNIX/NFS does not employ the concept of Mandatory Share Mode (Open Mode)
locking like Windows does. UNIX/NFS locking consists of byte range locking and is strictly
programmatic on a previously opened file. Also, UNIX/NFS is incapable of sending an
oplock break to a client, and therefore cannot interoperate with Windows clients using
Opportunistic Locking.
NETWORK STATUS MONITOR provides the lock manager with information on host status.
It monitors the system to ensure that locks will be handled properly if a system crashes
while a file is locked. This is referred to as monitored locking. If the server crashes with
monitored locks in place, the locks will be reinstated when the server recovers. Conversely, if
the client crashes with monitored locks in place, the locks will be automatically freed by the
server and must be reinstated by the client when it recovers.
Clients have the option of using lock manager without the status monitor in which case the
locking is non-monitored. Non-monitor ed locking is provided for client systems that do not
support multi-tasking and cannot run both lock manager and status monitor at the same
time. In all other cases, monitored locking is preferred.
4.4 PC NFS
PC-NFS implements Byte Range programmatic file locking by interoperating with the server
NFS Network Lock Manager. The Windows client will propagate application byte range
locks to the UNIX server, where the NLM will interoperate with the UNIX Kernel Lock
Manager to actually place the lock on the file. PC-NFS file locking participates in the UNIX
advisory locking protocol.
PC-NFS also implements Mandatory Share Mode (Open Mode) locking between PC-NFS
clients. However, PC-NFS uses NLM to implement the share mode locks, and a Windows
client does not. Therefore, PC-NFS does not recognize Windows client Mandatory Share
Mode (Open Mode) locks, and vice versa.
Also, PC-NFS is incapable of sending an oplock break to a client, and therefore cannot
interoperate with Windows clients using Opportunistic Locking.
PC-NFS testing for this paper was done with WRQ Reflections NFS Connection Software
version 7.1.
11
Chapter 5 CIFS/9000 File Locking
Implementation
CIFS/9000 mirrors the Windows file locking mechanisms, using the CIFS (SMB) protocol to
provide Windows-equivalen t file locking on a UNIX system for Windows clients (homogenous
client access). For Windows client access to files residing on the local UNIX file system,
CIFS/9000 utilizes:
This locking scheme is executed by the CIFS/9000 smbd UNIX process. Every Windows
client connection to the CIFS/9000 server starts an individual smbd daemon that
administers the client access. The smbd daemon implements the Windows file locking
scheme, and interoperates with every other smbd daemon that is running on the local system
to coordinate the file locks that each Windows client requests. The CIFS/9000 smbd
daemons provide the common interface that ensures the validity of each file lock. However,
since the smbd is not executing on the Windows native file server, some client applications
using very strict adherence to programmatic native Windows locking protocol could exhibit
unexpected behavior.
CIFS/9000 configures file locking in the /etc/opt/samba/smb.conf file. Default values do not
need to be explicitly configured in the smb.conf file.
5.1. MANDATORY SHARE MODE LOCKING (OPEN MODE)
– as implemented by CIFS/9000 - uses Windows access modes and share modes.
Access mode:
Read: Read access only
Write: Write access only
Read-Write: Read and Write access
Share mode:
Deny-None: Concurrent read and write access allowed
Deny-All: No concurrent read or write access allowed
Deny-Read: No concurrent read access allowed
Deny-Write: No concurrent write access allowed
The Windows client application determines the access mode when calling Createfile.
CIFS/9000 disables share mode in the smb.conf file on a per -share basis:
[share_name]
share modes = no
The default is “yes”, so the parameter is usually not explicitly set unless it is set to “no”.
However, most applications expect share mode to be set to yes – don’t change it.
12
Mandatory Share Mode(Open Mode) locking applies to Windows clients only. File sharing
issues arise when concurrent file access occurs between Windows clients and UNIX clients,
PC-NFS clients, or on files that have been NFS-mounted. UNIX processes have no concept of
Mandatory Share Mode locks. A key issue with sharing files between UNIX and
Windows is that a Windows client can hold a Mandatory Share Mode lock on a file,
but a UNIX process can open the same file concurrently for write access, resulting
in unacceptable risk of data corruption. CIFS/9000 has been enhanced to translate
Windows Mandatory Share Mode locks into byte range locks, therefore providing
locking interoperation with the UNIX advisory locking protocol and reducing the
risk of data corruption.
Providing protection for Windows clients from UNIX/NFS concurrent file access is an
important added-value feature for Windows-UNIX integration, but is not a comprehensive
solution. There are Windows-UNIX integration products that can provide Windows client
Mandatory Share Mode (open mode) locking protection from UNIX/NFS processes, even if
they do not participate in the UNIX advisory locking protocol. However, if a UNIX/NFS
application is not properly coded to lock files, then there is no way to protect other
UNIX/NFS processes from concurrent file access data corruption. If data can be corrupted by
UNIX/NFS concurrent file access, then providing Windows Share Mode (open mode) locking
protection from those same UNIX/NFS processes (mixed-mode access) does not remove the
element of risk from data corruption because the locking interoperability is not
comprehensive system-wide3.
A future enhancement will protect Windows Clients with Mandatory Share Mode (open
mode) locks from UNIX/NFS concurrent file access even if the UNIX/NFS process does not
participate in the advisory locking protocol. This file locking interoperability issue is the
focus of many claims by HP’s competitors. See section 8 and Appendix B.5 for more
details.
5.2. BYTE RANGE LOCKING (programmatic) is implemented by the
Windows client calls to the LockFile and LockFileEx APIs. The smbd process honors the byte
range lock by calling the UNIX fcntl function and invoking the UNIX byte range advisory
locks on the file (see the UNIX topic above). Thus, if the Windows application sets byte range
locks, CIFS/9000 can interoperate with UNIX processes utilizing advisory locking4. Windows
client access (homogenous) will honor the CIFS/9000 byte range locks.
CIFS/9000 disables byte range locking with the “locking” parameter in the smb.conf file:
[share_name]
locking = no
The default is “yes”. If locking is set to “no”, the byte range locks will appear to succeed on
the Windows client, but the smbd will not actually apply the advisory locks to the file. When
set to yes, the second open process blocks waiting for the release of the lock. Most
applications expect the locking to be enabled (byte range locking) – don’t change it.
Strict locking changes the way byte range locking is enforced. By default, byte range locking
is implemented when a Windows client application calls the LockFile or LockFileEx APIs.
Byte range lock status is only checked when the application calls these APIs. Strict locking
enables the smbd to check for byte range locks on every read and write access to a file. The
3
See Appendix B.5
4
See Appendix B.2
13
benefit is improved security, but in most cases the additional checks will be redundant to the
application checks, and the cost will be reduced performance due to the extra overhead of
checking for locks on every read and write.
CIFS/9000 enables strict locking on a per-share basis in the smb.conf file:
[share_name]
strict locking = yes
The default is “no”.
Blocking locks enables the smbd to recognize a timeout period specified on a LockFileEx call.
If a previous lock is encountered by a client attempting a byte range lock, the smbd will wait
(block) for the timeout period to expire before failing the lock. If the previous lock is released
before the timeout, the smbd will then grant the pending lock. When disabled, the lock
request is failed immediately.
CIFS/9000 disables blocking locks on a per -share basis in the smb.conf file:
[share_name]
blocking locks = no
The default is “yes”.
5.3.OPPORTUNISTIC LOCKING (Oplocks) is implemented by the
CIFS/9000 server on a per-share basis in the smb.conf file. CIFS/9000 Oplock functionality
operates just like Windows. Oplocks are enabled by default for each share, which allows the
Windows client to cache a local copy of a file for:
Read-ahead
Write-caching
Lock caching
CIFS/9000 disables Oplocks on a per -share basis in the smb.conf file:
[share_name]
oplocks = no
The default is “yes”. The default oplock type is Level1.
CIFS/9000 enables Level2 Oplocks on a per -share basis in the smb.conf file:
[share_name]
level2 oplocks = yes
The default is “no”. Oplocks must also be set to “yes” for the Level2 oplock parameter to
function.
Oplocks apply to Windows clients only. File sharing issues arise when concurrent file access
occurs between Windows clients and UNIX clients, PC-NFS clients, or on files that have been
NFS-mounted. A key issue with sharing files between UNIX/NFS, PC-NFS, and
Windows clients is that a Windows client can request an Oplock from the
CIFS/9000 server and be granted the Oplock (thus caching the file locally), but a
14
UNIX/NFS process or PC-NFS application can open the same file for write access
without an Oplock Break being sent to the client, resulting in unacceptable risk of
data corruption. Even a UNIX/NFS process that adheres to the advisory locking protocol
will not send an oplock break. Disabling Oplocks removes the risk of this particular data
corruption, but also removes the performance benefit that Oplock file caching provides.
Currently, CIFS/9000 does not provide interoperability between UNIX/NFS, PC-NFS and
Windows clients for oplocks. However, CIFS/9000 has a planned enhancement to provide
this interoperability. This enhancement will allow CIFS/9000 to send an oplock break
to a Windows client when a UNIX/NFS process or PC-NFS application attempts to
access a file, even if the process or application does not participate in the advisory
locking protocol. This file locking interoperability issue is the focus of many claims by
HP’s competitors. See section 8 for more details.
Kernel oplocks is a Samba smb.conf parameter that notifies Samba if the UNIX kernel has
the capability to send a Windows client an Oplock Break if a UNIX process is attempting to
open the file that is cached. This parameter addresses sharing files between UNIX and
Windows with Oplocks enabled on the a Samba server: the UNIX process can open the file
that is Oplocked (cached) by the Windows client and the smbd process will not send an
Oplock break, which exposes the file to the risk of data corruption. If the UNIX kernel has
the ability to send an Oplock break, then the kernel oplocks parameter enables Samba to
send the Oplock break. Kernel oplocks are enabled on a per -server basis in the smb.conf file.
However, CIFS/9000 currently does not support kernel oplocks, so the parameter has no
effect:
[global]
kernel oplocks = yes
The default is “no”. The planned enhancement for CIFS/9000 Oplocks will likely utilize this
parameter.
Veto oplocks is a smb.conf parameter that identifies specific files for which Oplocks are
disabled. When a Windows client opens a file that has been configured for veto oplocks, the
client will not be granted the oplock, and all operations will be executed on the original file
on disk instead of a client-cached file copy. By explicitly identifying files that are shared
with UNIX processes, and disabling Oplocks for those files, the server-wide Oplock
configuration can be enabled to allow Windows clients to utilize the performance benefit of
file caching without the risk of data corruption. Veto Oplocks can be enabled on a per -share
basis, or globally for the entire server, in the smb.conf file: