Paragon NTFS, HFS+ Instruction Manual

0 (0)

Paragon Technologie GmbH

Leo-Wohleb-Straße 8 79098 Freiburg, Germany

Tel. +49-761-59018-201 Fax +49-761-59018-130

Website: www.paragon-software.com

E-mail: sales@paragon.software.com

Paragon NTFS&HFS+ for Linux 9.5

User manual

Abstract

This document covers implementation of NTFS and HFS+ ﬁle systems support in Linux operating systems using Paragon NTFS&HFS+ ﬁle system driver. Basic installation procedures are described. Detailed mount options description is given. File system creation (formatting) and checking utilities are described. List of supported NTFS/HFS+ features is given with limitations imposed by Linux. There are also advanced troubleshooting section.

Information

All rights reserved. No parts of this work may be reproduced in any form or by any means - graphic, electronic, or mechanical, including photocopying, recording, taping, or information storage and retrieval systems - without the written permission of the publisher.

Products that are referred to in this document may be either trademarks and/or registered trademarks of the respective owners. The publisher and the author make no claim to these trademarks.

While every precaution has been taken in the preparation of this document, the publisher and the author assume no responsibility for errors or omissions, or for damages resulting from the use of information contained in this document or from the use of programs and source code that may accompany it. In no event shall the publisher and the author be liable for any loss of proﬁt or any other commercial damage caused or alleged to have been caused directly or indirectly by this document.

Printed: September 2017 in Freiburg, Germany.

Special thanks to:

All the people who contributed to this document, either by writing text, developing solutions to various issues, performing tests, collecting information or by requested support from our team. To our customers who continue to support us and help us to improve the product by constantly demanding more.

We welcome your feedback

Please send your feedback to your Paragon contact or to sales@paragon-software.com.

Paragon Technologie GmbH

Leo-Wohleb-Straße 8, 79098 Freiburg, Germany

3 Paragon NTFS&HFS+ for Linux 9.5

Contents

1 Introduction		4
1.1	Historical review . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .	4
1.2	Paragon UFSD technology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .	4
1.3	How UFSD works on Linux . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .	5
1.4	Key features . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .	5

2	System requirements			7
	2.1	Hardware requirements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .		7
	2.2	Software requirements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .		7
3	Installation			9
	3.1	Shipment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .		9
	3.2	Components . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .		9
	3.3	Installing driver . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .		9
	3.4	Uninstalling driver . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .		11
4	Using The Driver			12
	4.1	Getting started . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .		12
	4.2	Mounting and unmounting partitions . . . . . . . . . . . . . . . . . . . . . . . . . . . . .		12
	4.3	Dirty ﬂag issues . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .		13
	4.4	GPT issues . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .		13
	4.5	Issues with large HDDs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .		14
5	Mount options			15
	5.1	Mount options . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .		15
6	Additional Utilities			19
	6.1	ufsd utility . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .		19
	6.2	chkufsd utility . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .		19
	6.3	NTFS utilities . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .		20
		6.3.1	mkntfs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .	20
		6.3.2	chkntfs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .	22
	6.4	HFS+ utilities . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .		23
		6.4.1	mkhfs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .	23
		6.4.2	chkhfs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .	24
7	Troubleshooting			26
	7.1	Troubleshooting processes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .		26
	7.2	Mount troubleshooting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .		27
	7.3	The install.sh script can’t ﬁnd kernel sources . . . . . . . . . . . . . . . . . . . . . . . .		27
	7.4	Can’t compile the NTFS/HFS+ for Linux driver . . . . . . . . . . . . . . . . . . . . . . . .		28
	7.5	“Can’t load module” message at the end of installation . . . . . . . . . . . . . . . . . . .		28
	7.6	ufsd module: kernel-module version mismatch . . . . . . . . . . . . . . . . . . . . . . .		28
	7.7	ufsd module: create_module: operation is not permitted . . . . . . . . . . . . . . . . . .		29
	7.8	insmod: a module named as ufsd already exists . . . . . . . . . . . . . . . . . . . . . .		29
	7.9	insmod: Unknown symbol jnl_op (err0) . . . . . . . . . . . . . . . . . . . . . . . . . . . .		29
	7.10	Can’t mount NTFS/HFS+ volume . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .		29
	7.11	Hardware issues . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .		30
8	Sysdump utility			31
	8.1	Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .		31

User manual			4
	8.2	Main functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .	31
	8.3	Using the sysdump utility to collect both platform system information and metadata . . .	31
	8.4	Changing the name of output archive . . . . . . . . . . . . . . . . . . . . . . . . . . . . .	32
	8.5	Output ﬁle description . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .	32
	8.6	Delivering collected data to Paragon . . . . . . . . . . . . . . . . . . . . . . . . . . . . .	32
	8.7	Privacy policy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .	32
9	UFSD driver compatibility		34
	9.1	NTFS features . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .	34
	9.2	HFS+ features . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .	34
10	Frequently Asked Questions		35
	10.1	What are ’minor errors’ reported by chkntfs utility? . . . . . . . . . . . . . . . . . . . . .	35
	10.2	Warnings on Windows7/Vista when NTFS HDD is reconnected from Linux . . . . . . . .	36
	10.3	Recently changed ﬁle has its modiﬁcation time a few hours ahead of or behind the
		current system time. Why? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .	38
	10.4	Why does mount option A make driver ignore mount option B? . . . . . . . . . . . . . .	39
	10.5	Does the driver have an optimization for avoiding data fragmentation? . . . . . . . . . .	39
	10.6	Why a lot of memory is used for volume mounting? . . . . . . . . . . . . . . . . . . . . .	40
	10.7	Why the disk can’t be dismounted? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .	40

Paragon Technologie GmbH

Leo-Wohleb-Straße 8, 79098 Freiburg, Germany

5	Paragon NTFS&HFS+ for Linux 9.5

1Introduction

1.1Historical review

Historically, different operating systems supported different ﬁle systems. Sharing ﬁles among different platforms was not an easy task. For instance, documents that were created in Windows and are stored on NTFS partitions may be inaccessible under Linux, because Linux does not include full support for NTFS. For example, open-source NTFS-3G NTFS driver does not support random write access to compressed ﬁles.

Paragon NTFS&HFS+ for Linux 9.5 solves these problems — now everyone can access NTFS and HFS+ partitions from Linux in a usual manner with maximum performance and reliability. The driver allows mounting NTFS and HFS+ partitions, so that programs may work transparently with these mounted partitions — browse contents, open documents, run applications, work with existing ﬁles (delete/copy/modify) and create new ones.

Paragon NTFS&HFS+ for Linux 9.5 is a commercial Linux driver for local access to NTFS and HFS+ volumes. It supports full read/write access. The driver is a Kernel module, which guarantees rapid and transparent access to supported ﬁle systems. Mount volumes manually or insert into fstab, and NTFS/HFS+ partitions will be available like any other directory tree.

Paragon NTFS&HFS+ for Linux 9.5 Professional also includes useful additional utilities that provide the ability to check integrity and create NTFS/HFS+ volumes.

1.2Paragon UFSD technology

UFSD (Universal File System Driver) is an unique technology developed by Paragon Software to provide full access (read/write, format, etc.) to volumes of the popular ﬁle systems: NTFS, exFAT, HFS+ under various platforms, including Windows, Linux, Mac OS X, etc. in case these ﬁle systems are not otherwise supported.

UFSD technology provides access directly to the physical devices that is why it can process partitions regardless of their support by the current Operating System (OS). With UFSD it is possible to mount NTFS, HFS+ and exFAT partitions under Linux, thus getting access to its contents, just the way it is implemented in the NTFS&HFS+ for Linux driver, and the technology also allows direct access via physical device addressing, the way it is implemented in the driver too.

Paragon UFSDs are designed to be readily integrated into any solution using our UFSD Software Development Kit (UFSD SDK), which includes all of the necessary tools to develop applications with the following main features:

•Access to un-mounted partitions (i.e. drive letter not assigned);

•Access to other ﬁle systems that normally would not be supported by the operating system;

•Platform-independent UFSD API.

This software product contains components, which are partly subject to the license terms of the GNU General Public License or GNU Lesser General Public License ("LGPL"). You can request the modiﬁed source code of this software via a contact request: https://www.paragonsoftware.com/contact.html. The offer is valid for at least 3 years from the date of the publication of the corresponding software product. We deliver the software on CD/DVD or USB stick, whose production costs we claim in return.

User manual

Note: NTFS, HFS+ and exFAT drivers for Linux as well as utilities were written using UFSD SDK.

1.3How UFSD works on Linux

Modern operating systems are based on the concept of Installable File System drivers (IFS). User simply needs to provide an operating system with the proper ﬁle system driver to work with the ﬁle system in usual manner. Paragon NTFS&HFS+ for Linux 9.5 includes NTFS and HFS+ drivers for Linux environment. Once appropriate components of Paragon NTFS&HFS+ for Linux 9.5 are installed, the operating system can mount these ﬁle systems and work with directories/ﬁles stored on the ﬁle systems.

1.4Key features

Paragon NTFS&HFS+ for Linux 9.5 is released in the Express and Professional Editions. All of the products share the following features:

•Transparent read-write access to NTFS and HFS+ volumes — single Kernel module provides support both NTFS and HFS+ ﬁle systems;

•High performance (in some cases even better than Ext4 FS);

•Easy installation and uninstallation (assistant scripts);

•Support for the latest Linux Kernels and distributions;

•Support for SMP kernels;

•Support for iSCSI and SSD storages;

•File sharing over network via SAMBA;

•Low CPU load during data transfers;

•Unlimited ﬁle and volume size (within NTFS/HFS+ and Kernel limitations).

What’s new in Paragon NTFS&HFS+ for Linux 9.5

•Support for Kernel versions from 2.6.36 up to 4.12.x;

•Added read support of new system compression methods on NTFS from Windows 10/Server 2016;

•Improved xattr and symlink support;

•Overall performance enhancements;

•Stability improvements in unsafe removal scenarios;

•Improved automounting script (Professional version).

NTFS speciﬁc features:

•NTFS versions 1.2, 3.0 and 3.1 (Windows NT 4.0, 2000, XP, 2003, Vista, 7, 8.1, 10, Server 2016);

•Support for compressed ﬁles (random access for reading and writing with no limitations);

Paragon Technologie GmbH

Leo-Wohleb-Straße 8, 79098 Freiburg, Germany

7	Paragon NTFS&HFS+ for Linux 9.5

• Sparse ﬁles support.

HFS+ speciﬁc features:

•Both case sensitive and case instensitive types of HFS+ ﬁle system are supported;

•During ﬁle copy operation (using cp command) on Linux only ‘data’ fork is copied;

•Support replay native HFS+ journal.

NTFS compatibility information:

File system version	Comments
NTFS version 1.2	Originates from Microsoft Windows NT 4.0

NTFS version 3.0	Originates from Microsoft Windows 2000

NTFS version 3.1	Originates from Microsoft Windows XP/2003/ Vista/7/8.1/10 and
	Server 2016

Additional features of the Professional Edition:

•Full support of the native HFS+ journal;

•Automatic driver rebuilt for newer supported Kernels (support for the DKMS library);

•Automatic NTFS/HFS+ volume mounting;

•Additional utilities:

–ufsd utility - single binary for all ﬁle system utilities;

–chkufsd utility - single binary for cheking every supported ﬁle system;

–mkntfs - format any partition as NTFS under Linux;

–chkntfs - check NTFS partition integrity and ﬁx errors;

–mkhfs- format any partition as HFS+ under Linux;

–chkhfs - check HFS+ partition for integrity and ﬁx errors;

–Sysdump utility - debug utility for collecting volume metadata image and debug system information.

User manual

2System requirements

This topic highlights requirements to hardware and software that may be used to run Paragon NTFS&HFS+ for Linux 9.5.

2.1Hardware requirements

Minimum hardware requirements:

•Processor: Intel Pentium 300 MHz and higher, or compatible;

•both 32and 64-bit CPUs are supported;

•16MB of RAM.

Due to unique technology NTFS&HFS+ for Linux drivers have low system requirements. For example, it is enough for our driver to have 650KB of free RAM to work with NTFS partitions larger than 250 GB. NTFS&HFS+ Kernel modules occupy around 800 Kb of RAM.

Real-life values:

•200 Kb while executing 5 commands like ’dd if=/dev/zero of=/mnt/sda1/test bs=1M count=1000&’ in background.

• 516 Kb while executing ’rsync -r /home /mnt/sda1’ command.

•17 Mb while compiling bench test on Desktop Linux system in virtual environment using NTFS ﬁle system: a ﬁle-tree with a size about 220 Mb was created and patched, simulating Linux Kernel installation process.

RAM consumption depends ﬁrst of all on whole amount of memory available in the system. If it is low then the driver wouldn’t keep a lot of descriptors opened to keep the memory usage at minimum.

2.2Software requirements

Supported Linux Kernel versions

•Linux with kernel versions 2.6.36 and newer;

•Linux with kernel versions up to 4.12.x (NTFS&HFS+ driver was tested with Kernels up to 4.12.5).

Linux distributions the products were tested with:

•Ubuntu 17.10;

•Debian 9.1;

•Fedora 26;

•OpenSuse 42.3;

•CentOS 7.

Paragon Technologie GmbH

Leo-Wohleb-Straße 8, 79098 Freiburg, Germany

9	Paragon NTFS&HFS+ for Linux 9.5

Development Environment

A development environment is required to compile Linux drivers and utilities. Please verify that these tools are all functional. The easiest way is to choose the developer toolkit when installing Linux. What must be installed:

• Kernel source code (recommended) or Kernel header ﬁles (doesn’t always work);

#rpm -qa|grep kernel-source (for RPM based kernel-sources)

• GNU C compiler;

#gcc --version

• GNU C++ compiler (for Professional version only);

#g++ --version

•GNU glibc-static (recommended for Professional version only);

•GNU Make;

#make --version

• GNU ld (binutils);

#ld --version

• Modutils (module-init tools);

#insmod -V

• DKMS library (for Professional version only).

#dkms --version

Limitations

•GNU C compiler (gcc) version 4.9 or higher is required.

•The user should login as root to install the drivers and utilities.

•Correct operation is not guaranteed for customized Linux kernels. Commercial porting service to customized Linux kernels is available from Paragon Software Group — for more information send e-mail to sales@paragon-software.com).

User manual

3Installation

This section describes workﬂows related to installing and using Paragon NTFS&HFS+ for Linux driver.

3.1Shipment

The setup ﬁles for each product of the family are provided as the downloadable TGZ archives, which can be downloaded from the company site.

3.2Components

The package includes the following components:

•Source ﬁles for the NTFS&HFS+ for Linux driver;

•Assistant script ﬁles, which are purposed to simplify the installation and uninstallation routines;

•Source ﬁles for additional utilities (for Professional edition only);

•Source ﬁles for DKMS library support (for Professional edition only);

•Source ﬁles for automatic mounting integration (for Professional edition only).

Paragon NTFS&HFS+ Linux driver and utilities must be compiled on the end user’s system for correct conﬁguration. By installing the software you accept the terms of End User License Agreement listed in License ﬁle.

3.3Installing driver

First, NTFS&HFS+ driver must be built and installed.

Steps to install the NTFS&HFS+ for Linux driver are as follows:

1.Log in as root. This step is obligatory;

2.Build and install the Paragon NTFS&HFS+ for Linux 9.5 using install.sh script. Alternatively, driver binary module may be built manually using ’configure’ ’make driver’ commands.

3.Install the NTFS & HFS+ driver (this step will make the modules available for use);

4.Activating (loading) the driver. After building and installing, the NTFS & HFS+ driver can be referenced as “universal ﬁle system driver” (ufsd) when mounting NTFS and HFS+ partitions.

The steps 1-3 should be made only once while the step 4 is the standard way of using ﬁle system drivers in Linux environment.

NTFS & HFS+ for Linux include a set of assistant script ﬁles for the simpliﬁcation of building, installing and uninstalling procedures. Note that these assistant scripts may fail to work in customized Linux conﬁgurations or unsupported Linux distributions. Use install.sh and uninstall.sh script ﬁles to install and uninstall (correspondingly) NTFS & HFS+ driver and utilities. The sections below describe the installation procedure in details.

Paragon Technologie GmbH

Leo-Wohleb-Straße 8, 79098 Freiburg, Germany

11	Paragon NTFS&HFS+ for Linux 9.5

Unpacking Setup Files

The setup ﬁles of the Linux-based version of the NTFS & HFS+ for Linux are provided in the form of a gzip archive. The archive should be copied to the hard disk and decompressed. For example: For the NTFS & HFS+ for Linux driver and utilities:

• create separate folder:

$ mkdir /usr/tmp/ufsd

• change the current directory to the new one $ cd /usr/tmp/ufsd

• use tar utility to unpack initial archive

$ tar -xf /path/to/the/initial/archive/ufsd\_*.tar.gz

Next step is to build and install the NTFS & HFS+ for Linux driver.

Using the INSTALL.SH Assistant Script

The assistant script "install.sh" provides the extremely easy and ﬂexible way to make the NTFS & HFS+ for Linux and install driver module in the system. Additionally, the script can reconﬁgure OS so that driver module is automatically rebuilt for another supported Kernel version (Professional edition only) and NTFS/HFS+ volumes are automatically mounted with the UFSD driver (Professional edition only).

Please note that development tools and kernel sources are required to present on the system and stay in the default locations to build and install the drivers.

Installation

Just run the install.sh script with root privileges:

# ./install.sh or \$ sudo ./install.sh

The assistant script will automatically perform the following actions:

1.Detect the Linux Kernel version;

2.Find kernel header ﬁles and libraries needed for building the drivers;

3.Add service for rebuilding driver module for supported Kernels via the DKMS library (Professional edition only);

4.Build driver binary modules (jnl.ko and ufsd.ko);

5.Install the driver;

6.Add automatic mounting settings for NTFS/HFS+ volumes (Professional edition only);

7.Build and install additional utilities (Professional edition only);

INSTALL.SH default mode for the NTFS&HFS+ for Linux driver

The assistant script install.sh always names the NTFS&HFS+ for Linux driver module as ufsd (it is the abbreviation of the project name Universal File System Driver).

Now you can mount any NTFS/HFS+ partition:

User manual

$ sudo mount -t ufsd <device> <mount_point>.

3.4Uninstalling driver

To completely remove the drivers and the utilities from the current Kernel, one should dismount all NTFS/HFS+ partitions mounted with the driver, uninstall the drivers and unload binary modules from the Kernel.

NTFS&HFS+ for Linux provides tools for the drivers/utilities uninstall automation.

The assistant script uninstall.sh completely removes the drivers/utilities from the system.

Using the UNINSTALL.SH Assistant Script

The assistant script uninstall.sh provides the extremely easy and ﬂexible way to deactivate and remove the drivers and utilities from the system. The script performs the correct deactivation, uninstallation and the complete removing of the driver’s and utilities’ ﬁles.

Uninstalling

Unmount all currently mounted NTFS and/or HFS+ partitions and then run the ’uninstall.sh’ script:

$ sudo ./uninstall.sh

The assistant script will automatically perform the following actions:

1.Deactivate the driver modules. If the drivers is still in use, the further script execution is aborted;

2.Uninstall the drivers;

3.Remove all binary and source ﬁles of the driver

4.Uninstall utilities (for Professional version only).

Paragon Technologie GmbH

Leo-Wohleb-Straße 8, 79098 Freiburg, Germany

13	Paragon NTFS&HFS+ for Linux 9.5

4Using The Driver

After building and installing Paragon NTFS&HFS+ for Linux 9.5 driver, it can be automatically loaded at the system startup. The driver allows to mount supported partitions and provides access to their whole contents.

4.1Getting started

The goal of this section is to help system engineers to quickly ﬁnd out how to use the product. It describes general approach to mounting partitions using UFSD ﬁle system driver and helps to avoid common issues. We strongly recommend reading this section before starting using our driver.

To mount volume using UFSD driver, standard mount command is used, with FS type set to ufsd, e.g.:

# mount -t ufsd /dev/sda1 /mnt/sda1

After this command is executed there can be several mount scenarios for a disk (for more information see Mount troubleshooting subsection):

•The disk is “clean” (without any errors), mounted by the driver and ready to use.

•Disk can’t be mounted. In this case can be several scenarios:

1.Disk has “dirty” ﬂag set (for more information see Dirty ﬂag issues subsection):

–Use chkntfs/chkhfs utilities with -a -f options to check the volume for errors and inconsistencies and ﬁx them (if any). This is recommended approach (see ?? or ?? subsections);

–Use ‘force’ mount option (see Dirty ﬂag issues subsection).

2.The disk is a GPT-partitioned disk — check GPT issues subsection for more information.

3.Follow other steps on the Mount troubleshooting diagram to ﬁnd the cause of the issue. Update/- ﬁx Kernel source code (if necessary) and request new driver for the new Kernel from Paragon Software.

Analyze returned status and check output of (dmesg | tail). In case of failure, follow the ?? to ﬁnd possible causes and and try to mount the partition again using the same or different mount options, if needed (see Mount options subsection).

If there is still a problem mounting the partition ﬁll out Paragon’s online request form from your user account so we could help you with the issue.

4.2Mounting and unmounting partitions

Mounting NTFS and HFS+ volumes

To mount volume using UFSD driver, use mount command with FS type set to ufsd, e.g.:

# mount -t ufsd /dev/sda1 /mnt/sda1

Sometimes volumes cannot be mounted using ’mount’ command, for example:

# mount -t ufsd	/dev/sdd1	/mnt/sdd1
# mount -t ufsd	/dev/sdd1	/mnt/sdd1

mount: wrong fs	type, bad	option,	bad superblock on	/dev/sdd1,
missing	codepage or helper		program, or other	error
In some	cases useful info is found in syslog			- try

+ 28 hidden pages