Sun Microsystems 5800 User Manual

Sun StorageTek 5800 System Client API Reference Manual

Sun Microsystems, Inc. 4150 Network Circle Santa Clara, CA 95054 U.S.A.

Part No: 820–4796 June 2008

Sun Microsystems, Inc. has intellectual property rights relating to technology embodied in the product that is described in this document. In particular, and without limitation, these intellectual property rights may include one or more U.S. patents or pending patent applications in the U.S. and in other countries.

U.S. Government Rights – Commercial software. Government users are subject to the Sun Microsystems, Inc. standard license agreement and applicable provisions of the FAR and its supplements.

This distribution may include materials developed by third parties.

Parts of the product may be derived from Berkeley BSD systems, licensed from the University of California. UNIX is a registered trademark in the U.S. and other countries, exclusively licensed through X/Open Company, Ltd.

Sun, Sun Microsystems, the Sun logo, the Solaris logo, the Java Coee Cup logo, docs.sun.com, Java, and Solaris are trademarks or registered trademarks of Sun Microsystems, Inc. in the U.S. and other countries. All SPARC trademarks are used under license and are trademarks or registered trademarks of SPARC International, Inc. in the U.S. and other countries. Products bearing SPARC trademarks are based upon an architecture developed by Sun Microsystems, Inc.

The OPEN LOOK and Sun of Xerox in researching and developing the concept of visual or graphical user interfaces for the computer industry. Sun holds a non-exclusive license from Xerox to the Xerox Graphical User Interface, which license also covers Sun's licensees who implement OPEN LOOK GUIs and otherwise comply with Sun's written license agreements.

Products covered by and information contained in this publication are controlled by U.S. Export Control laws and may be subject to the export or import laws in other countries. Nuclear, missile, chemical or biological weapons or nuclear maritime end uses or end users, whether direct or indirect, are strictly prohibited. Export or reexport to countries subject to U.S. embargo or to entities identied on U.S. export exclusion lists, including, but not limited to, the denied persons and specially designated nationals lists is strictly prohibited.

DOCUMENTATION IS PROVIDED “AS IS” AND ALL EXPRESS OR IMPLIED CONDITIONS, REPRESENTATIONS AND WARRANTIES, INCLUDINGANY IMPLIED WARRANTY OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE OR NON-INFRINGEMENT, ARE DISCLAIMED, EXCEPT TO THE EXTENT THATSUCH DISCLAIMERS ARE HELD TO BE LEGALLY INVALID.

Graphical User Interface was developed by Sun Microsystems, Inc. for its users and licensees. Sun acknowledges the pioneering eorts

Sun Microsystems, Inc. détient les droits de propriété intellectuelle relatifs à la technologie incorporée dans le produit qui est décrit dans ce document. En particulier, et ce sans limitation, ces droits de propriété intellectuelle peuvent inclure un ou plusieurs brevets américains ou des applications de brevet en attente aux Etats-Unis et dans d'autres pays.

Cette distribution peut comprendre des composants développés par des tierces personnes.

Certaines composants de ce produit peuvent être dérivées du logiciel Berkeley BSD,licenciés par l'Université de Californie. UNIX est une marque déposée aux Etats-Unis et dans d'autres pays; elle est licenciée exclusivement par X/Open Company, Ltd.

Sun, Sun Microsystems, le logo Sun, le logo Solaris, le logo Java Coee Cup, docs.sun.com, Java et Solaris sont des marques de fabrique ou des marques déposées de Sun Microsystems, Inc. aux Etats-Unis et dans d'autres pays. Toutes les marques SPARC sont utilisées sous licence et sont des marques de fabrique ou des marques déposées de SPARC International, Inc. aux Etats-Unis et dans d'autres pays. Les produits portant les marques SPARC sont basés sur une architecture développée par Sun Microsystems, Inc.

L'interface d'utilisation graphique OPEN LOOK et Sun a été développée par Sun Microsystems, Inc. pour ses utilisateurs et licenciés. Sun reconnaît les eorts de pionniers de Xerox pour la recherche et le développement du concept des interfaces d'utilisation visuelle ou graphique pour l'industrie de l'informatique. Sun détient une licence non exclusive de Xerox sur l'interface d'utilisation graphique Xerox, cette licence couvrant également les licenciés de Sun qui mettent en place l'interface d'utilisation graphique OPEN LOOK et qui, en outre, se conforment aux licences écrites de Sun.

Les produits qui font l'objet de cette publication et les informations qu'il contient sont régis par la legislation américaine en matière de contrôle des exportations et peuvent être soumis au droit d'autres pays dans le domaine des exportations et importations. Les utilisations nales, ou utilisateurs naux, pour des armes nucléaires, des missiles, des armes chimiques ou biologiques ou pour le nucléaire maritime, directement ou indirectement, sont strictement interdites. Les exportations ou réexportations vers des pays sous embargo des Etats-Unis, ou vers des entités gurant sur les listes d'exclusion d'exportation américaines, y compris, mais de manière non exclusive, la liste de personnes qui font objet d'un ordre de ne pas participer, d'une façon directe ou indirecte, aux exportations des produits ou des services qui sont régis par la legislation américaine en matière de contrôle des exportations et la liste de ressortissants spéciquement designés, sont rigoureusement interdites.

LA DOCUMENTATIONEST FOURNIE "EN L'ETAT"ET TOUTES AUTRESCONDITIONS, DECLARATIONS ET GARANTIES EXPRESSES OU TACITES SONT FORMELLEMENT EXCLUES, DANS LA MESURE AUTORISEE PAR LA LOI APPLICABLE, Y COMPRIS NOTAMMENT TOUTE GARANTIE IMPLICITE RELATIVE A LA QUALITE MARCHANDE, A L'APTITUDE A UNE UTILISATIONPARTICULIERE OU A L'ABSENCE DE CONTREFACON.

080616@20490

Contents

Preface ...................................................................................................................................................11

1 Sun StorageTek 5800 System Client API .......................................................................................... 15

Changes in Version 1.1 ....................................................................................................................... 15

5800 System Overview ........................................................................................................................ 16

5800 System Summary ................................................................................................................ 16

The 5800 System and Honeycomb ............................................................................................. 17

The 5800 System Data Model ..................................................................................................... 17

The 5800 System Metadata Model ............................................................................................. 19

The 5800 System Query Model ................................................................................................... 20

The 5800 System Query Integrity Model .................................................................................. 21

Deleting Objects from the 5800 System .................................................................................... 22

2 Sun StorageTek 5800 System Java Client API ................................................................................. 25

Overview of the 5800 System Java Client API .................................................................................. 25

Client Library ............................................................................................................................... 25

Interfaces ....................................................................................................................................... 26

Retrying Operations .................................................................................................................... 26

Performance and Scalability .......................................................................................................26

Updating Client View of the Schema ......................................................................................... 27

Java Client Application Deployment ................................................................................................ 27

Java API ................................................................................................................................................ 27

Java API Packages ........................................................................................................................ 27

Java API Documentation ............................................................................................................27

Basic Concepts ............................................................................................................................. 28

Key Classes .................................................................................................................................... 28

NameValueObjectArchive Application Access ........................................................................ 30

Contents

3 Sun StorageTek 5800 System C Client API ....................................................................................... 39

Overview of the 5800 System C Client API ...................................................................................... 39

Architecture .................................................................................................................................. 40

Interfaces ....................................................................................................................................... 40

Retrying Operations .................................................................................................................... 40

Multithreaded Access .................................................................................................................. 40

Performance and Scalability .......................................................................................................40

Memory Usage ............................................................................................................................. 41

Updating Schema Denitions .................................................................................................... 41

Session Management ................................................................................................................... 41

C Client Application Deployment ..................................................................................................... 43

Nonblocking C API ............................................................................................................................. 43

Synchronous C API ............................................................................................................................. 44

Changes for the 1.1 Release .........................................................................................................44

Limitations .................................................................................................................................... 45

Synchronous C Data Types ................................................................................................................ 46

hc_string_t ................................................................................................................................. 46

hc_long_t ..................................................................................................................................... 46

hc_double_t ................................................................................................................................. 46

hc_type_t ..................................................................................................................................... 47

hc_value_t ................................................................................................................................... 47

hc_schema_t ................................................................................................................................. 48

hc_nvr_t ....................................................................................................................................... 48

hc_session_t ............................................................................................................................... 48

hc_pstmt_t ................................................................................................................................... 49

hc_query_result_set_t ............................................................................................................ 49

read_from_data_source ............................................................................................................ 49

write_to_data_destination ................................................................................................... 50

hcerr_t ......................................................................................................................................... 51

Synchronous C API Functions .......................................................................................................... 53

Managing 5800 System Sessions ................................................................................................ 53

hc_session_create_ez .............................................................................................................. 53

hc_session_free ........................................................................................................................ 55

hc_session_get_status ............................................................................................................ 55

hc_session_get_schema ............................................................................................................ 56

hc_session_get_host ................................................................................................................ 57

Sun StorageTek5800 SystemClient API Reference Manual • June 20084

Contents

hc_session_get_platform_result ......................................................................................... 58

hc_session_get_archive .......................................................................................................... 59

Managing a Schema ..................................................................................................................... 59

hc_schema_get_type .................................................................................................................. 60

hc_schema_get_length .............................................................................................................. 61

hc_schema_get_count ................................................................................................................ 61

hc_schema_get_type_at_index ............................................................................................... 62

Manipulating Name-Value Records .......................................................................................... 63

Using the API for Storing Name-Value Records ...................................................................... 63

Using Returned Name-Value Records ...................................................................................... 64

Creating and Freeing Name-Value Records ............................................................................. 65

hc_nvr_create ............................................................................................................................. 65

hc_nvr_free ................................................................................................................................. 66

Building Name-Value Records ................................................................................................... 66

hc_nvr_add_value ...................................................................................................................... 67

hc_nvr_add_long ........................................................................................................................ 68

hc_nvr_add_double .................................................................................................................... 69

hc_nvr_add_string .................................................................................................................... 70

hc_nvr_add_binary .................................................................................................................... 71

hc_nvr_add_date ........................................................................................................................ 72

hc_nvr_add_time ........................................................................................................................ 73

hc_nvr_add_timestamp .............................................................................................................. 74

hc_nvr_add_from_string .......................................................................................................... 75

Retrieving Name-Value Records ................................................................................................ 76

hc_nvr_get_count ...................................................................................................................... 76

hc_nvr_get_value_at_index ................................................................................................... 77

hc_nvr_get_long ........................................................................................................................ 78

hc_nvr_get_double .................................................................................................................... 79

hc_nvr_get_string .................................................................................................................... 80

hc_nvr_get_binary .................................................................................................................... 81

hc_nvr_get_date ........................................................................................................................ 82

hc_nvr_get_time ........................................................................................................................ 82

hc_nvr_get_timestamp .............................................................................................................. 83

Creating and Converting Name-Value Records From and To String Arrays ...................... 84

hc_nvr_create_from_string_arrays .................................................................................... 84

hc_nvr_convert_to_string_arrays ....................................................................................... 86

Contents

Storing Data and Metadata ......................................................................................................... 87

hc_store_both_ez ...................................................................................................................... 87

hc_store_metadata_ez .............................................................................................................. 88

hc_check_indexed_ez ................................................................................................................ 89

Retrieving Data and Metadata .................................................................................................... 91

hc_retrieve_ez .......................................................................................................................... 91

hc_retrieve_metadata_ez ....................................................................................................... 92

hc_range_retrieve_ez .............................................................................................................. 93

Querying Metadata ...................................................................................................................... 94

hc_query_ez ................................................................................................................................. 94

hc_qrs_next_ez .......................................................................................................................... 96

hc_qrs_is_query_complete ..................................................................................................... 97

hc_qrs_get_query_integrity_time ....................................................................................... 98

hc_qrs_free ................................................................................................................................. 99

hc_pstmt_create ...................................................................................................................... 100

hc_pstmt_free ........................................................................................................................... 101

hc_pstmt_set_string .............................................................................................................. 101

hc_pstmt_set_char .................................................................................................................. 102

hc_pstmt_set_double .............................................................................................................. 103

hc_pstmt_set_long .................................................................................................................. 104

hc_pstmt_set_date .................................................................................................................. 105

hc_pstmt_set_time .................................................................................................................. 106

hc_pstmt_set_timestamp ........................................................................................................ 107

hc_pstmt_set_binary .............................................................................................................. 108

hc_pstmt_query_ez .................................................................................................................. 109

Querying With a Prepared Statement ............................................................................................. 110

Deleting Records ........................................................................................................................ 111

hc_delete_ez ............................................................................................................................. 111

Translating Error and Type Codes .......................................................................................... 112

hc_decode_hcerr ...................................................................................................................... 112

hc_decode_hc_type .................................................................................................................. 113

4 Sun StorageTek 5800 System Query Language ............................................................................ 115

Interfaces ............................................................................................................................................ 115

Operation ........................................................................................................................................... 116

Sun StorageTek5800 SystemClient API Reference Manual • June 20086

Contents

Supported Data Types ....................................................................................................................... 116

Queries ................................................................................................................................................ 117

Translating a Query to the Underlying Database ................................................................... 117

Attribute Format in Queries ..................................................................................................... 117

SQL Syntax in 5800 System Queries ........................................................................................ 118

Literals In Queries ............................................................................................................................. 118

Dynamic Parameters ................................................................................................................. 118

String Literals ............................................................................................................................. 118

Numeric Literals ......................................................................................................................... 118

Literals for 5800 System Data Types ........................................................................................ 119

Canonical String Format .................................................................................................................. 119

The Canonical String Decode Operation ................................................................................ 120

JDBC and HADB Date and Time Operations ................................................................................ 120

Reserved Words ................................................................................................................................. 121

Supported Expression Types ............................................................................................................ 121

Examples of Supported Query Expressions ................................................................................... 123

Queries Not Supported in Version 1.1 ............................................................................................ 123

SQL Words That Are Allowed in Queries ...................................................................................... 124

SQL Words That Are Not Allowed in Queries ............................................................................... 124

5 Programming Considerations and Best Practices .......................................................................127

Retries and Timeouts ........................................................................................................................ 127

Query Size Limit ................................................................................................................................ 127

Limit the Size of Schema Query Parameters and Literals ............................................................. 128

Limit Results Per Fetch ..................................................................................................................... 128

Index ................................................................................................................................................... 129

Tables

TABLE 4–1 Canonical String Representation of Data Types .................................................. 119

Preface

The Sun StorageTek 5800 System Client API Reference Manual is written for programmers and application developers who develop custom applications for the Sun StorageTek This document, along with the Sun StorageTek 5800 SystemSDK Reference Manual, provides the information that you need to develop custom applications for the 5800 system.

HowThis Book Is Organized

■

Chapter 1, “Sun StorageTek 5800 System Client API,” provides a summary of the changes

for the Sun StorageTek 5800 System 1.1 release, and overviews of the client APIs and query language.

■

Chapter 2, “Sun StorageTek 5800 System Java Client API,” provides detailed information on

the Sun StorageTek 5800 System Java client API.

■

Chapter 3, “Sun StorageTek 5800 System C Client API,” provides detailed information on

the Sun StorageTek 5800 System C client API.

■

Chapter 4, “Sun StorageTek 5800 System Query Language,” provides detailed information

on the Sun StorageTek 5800 System query language.

■

Chapter 5, “Programming Considerations and Best Practices,” provides programming

considerations and best practices that can help you create ecient 5800 system applications.

5800 System.

Related Books

■

Sun StorageTek 5800 System Regulatory and Safety Compliance Manual, part number 819–3809

■

Sun StorageTek 5800 System Site Preparation Guide, part number 820–1635

■

Sun StorageTek 5800 System Administration Guide, part number 820–4118

■

Sun StorageTek 5800 System SDK Reference Manual, part number 820–4797

■

Sun StorageTek 5800 System 1.1.1 Release Notes, part number 820–4120

Preface

RelatedThird-Party Web Site References

Third-party URLs are referenced in this document and provide additional, related information.

Note – Sun is not responsible for the availability of third-party web sites mentioned in this

document. Sun does not endorse and is not responsible or liable for any content, advertising, products, or other materials that are available on or through such sites or resources. Sun will not be responsible or liable for any actual or alleged damage or loss caused or alleged to be caused by or in connection with use of or reliance on any such content, goods, or services that are available on or through such sites or resources.

Documentation, Support, and Training

The Sun web site provides information about the following additional resources:

■

Documentation (http://www.sun.com/documentation/)

■

Support (http://www.sun.com/support/)

■

Training (http://www.sun.com/training/)

Typographic Conventions

The following table describes the typographic conventions that are used in this book.

TABLE P–1 Typographic Conventions

Typeface Meaning Example

AaBbCc123 The names of commands, les, and directories,

and onscreen computer output

AaBbCc123 What you type, contrasted with onscreen

computer output

aabbcc123 Placeholder: replace with a real name or value The command to remove a le is rm

Sun StorageTek5800 SystemClient API Reference Manual • June 200812

Edit your .login le.

Use ls -a to list all les.

machine_name% you have mail.

machine_name% su

Password:

lename.

TABLE P–1 Typographic Conventions (Continued)

Typeface Meaning Example

Preface

AaBbCc123 Book titles, new terms, and terms to be

emphasized

Shell Prompts in Command Examples

The following table shows the default UNIX® system prompt and superuser prompt for the C shell, Bourne shell, and Korn shell.

TABLE P–2 Shell Prompts

Shell Prompt

C shell machine_name%

C shell for superuser machine_name#

Bourne shell and Korn shell $

Bourne shell and Korn shell for superuser #

Read Chapter 6 in the User's Guide.

A cache is a copy that is stored locally.

Do not save the le.

Note: Some emphasized items appear bold online.

Sun WelcomesYour Comments

Sun is interested in improving its documentation and welcomes your comments and suggestions. You can submit your comments by clicking the Feedback link on the http://docs.sun.com web site.

Please include the title and part number of your document with your feedback:

Sun StorageTek 5800 System Client API Reference Manual, part number 820-4796

CHAPTER 1

Sun StorageTek 5800 System Client API

The SunTMStorageTekTM5800 system client API provides programmatic access to a 5800 system server to store, retrieve, query, and delete object data and metadata. Synchronous versions are provided in C and Java use with POSIX operations.

This chapter provides a summary of the changes for the Sun StorageTek 5800 System 1.1 release, and overviews of the client APIs and query language.

The following topics are discussed:

■

“Changes in Version 1.1” on page 15

■

“5800 System Overview” on page 16

languages. A future release will implement a non-blocking C API for

Changes in Version 1.1

The following general changes have been made in Version 1.1.

■

Handling is added for storing, retrieving, and querying the following metadata types:

– char — for Latin 1 character set – string — for Unicode character set – binary – date – time – timestamp

■

Query and queryplus are merged.

■

Prepared statements (pstmts) are introduced to handle the values of queries that cannot be placed inline, and a new query is introduced to handle them.

■

The handling of strings that are longer than the string length of the associated eld has changed.

5800 System Overview

In 5800 system version 1.1, an attempt to store a value that is longer than the associated eld generates an immediate error.

5800 System Overview

This section provides an overviews of the 5800 system, the 5800 system history, and a summaries of the key points of the 5800 system usage model.

The following topics are discussed:

■

“5800 System Summary” on page 16

■

“The 5800 System and Honeycomb” on page 17

■

“The 5800 System Data Model” on page 17

■

“The 5800 System Metadata Model” on page 19

■

“The 5800 System Query Model” on page 20

■

“The 5800 System Query Integrity Model” on page 21

■

“Deleting Objects from the 5800 System” on page 22

5800 SystemSummary

The 5800 system is an object-based storage archive appliance for xed-content data and metadata. The 5800 system is designed from the ground up to be reliable, aordable, and scalable, and to integrate data storage with intelligent data retrieval. It is designed to store huge amounts of data for decades at a time. At that scale, issues of how and where the data is stored — and how that changes over time — can be quite cumbersome. The 5800 system usage model is designed to manage those issues for you, so that your application can deal with just the data.

A custom Application Programming Interface (the 5800 Client API) is provided so that your applications can take advantage of all the features in the 5800 system usage model. The API provides the following capabilities:

■

Store a new object into the archive (storeObject)

■

Associate a new metadata record with stored object data (storeMetadata)

■

Retrieve the data from an object that was previously stored (retrieveData)

■

Retrieve the metadata from an object that was previously stored (retrieveMetadata)

■

Delete an object (delete)

■

Query for matching objects given a query expression of specic object characteristics (query)

The 5800 system API Release 1.1 provides two APIs:

■

The Java API is described in Chapter 2, “Sun StorageTek 5800 System Java Client API”

■

The C API is described in Chapter 3, “Sun StorageTek 5800 System C Client API”

Sun StorageTek5800 SystemClient API Reference Manual • June 200816

5800 System Overview

This chapter provides a summary of key points of the 5800 system usage model that are useful for understanding either API.

In the following sections, the terms from the Java API are used as an aid to exposition. In all cases, a simple equivalent using the C API is available.

■

Chapter 4, “Sun StorageTek 5800 System Query Language,” provides a detailed description

of query capabilties and query syntax.

■

Chapter 5, “Programming Considerations and Best Practices,” provides programming

considerations and best practices that can help you create ecient 5800 system applications.

The 5800 System and Honeycomb

The original code name for the project that grew into the 5800 system was Project Honeycomb. The Honeycomb name lives on as the name of an Open Solaris community that is bringing the Honeycomb software stack into the world of Open Source. The rst realization of the Honeycomb storage model as a real product is the 5800 system as described in this guide and related guides.

As a model for programmable storage systems, however, the Honeycomb API has a much broader reach than just the 5800 system. The programming model is designed to scale both up and down to any storage archive system that needs to abstract and separate the issues of how data is stored from how it is used. In recognition of both the past and the future, the string “honeycomb” and the initials “hc“ still live on in certain aspects of the API described in this guide. When the 5800 system API is used in contexts outside of the 5800 system, the API is referred to as the Project Honeycomb API.

The 5800 System Data Model

The 5800 system stores two types of data: arbitrary object data and structured metadata records. Every metadata record is associated with exactly one data object. Every data object has at least one metadata record. A unique object identier (OID) is returned when a metadata record is stored. This OID can later be used to retrieve the metadata record or its data object. In addition, metadata records can be retrieved by a query:

OID ↔ Metadata Record → Underlying Object Data

There are two types of metadata, system metadata and user metadata. You cannot override the names and types of system metadata.

Each object in the 5800 system archive consists of some arbitrary bytes of data together with associated metadata that describes the data. Once an object is stored, it is immutable. The 5800 system programming model does not allow the data or the metadata associated with an object to be changed once the object has been stored, in other words the system is a Write-Once

Chapter 1 • Sun StorageTek 5800 System Client API 17

5800 System Overview

Read-Multiple (WORM) archive. Each object corresponds to a single stream of data and a single set of metadata; there are no “grouped objects” or “compound objects” other than by application convention.

Each object corresponds to a single stream of data and a single set of metadata. There are no “grouped objects” or “compound objects” other than by application convention. Similarly, there are no “links” or “associations“ from one object to another. The customer application is shielded from all details of how or where the object is stored. Internally, the actual location of an object might change over time, or several objects might even share the same underlying storage. The customer application can retrieve the object without knowing these details.

A stream of data is stored in the object archive using storeObject. Once stored, each such object is associated with an object identier or objectid (OID). The storeObject operation takes both a stream of data and an optional set of user metadata information and returns an OID. The OID can be remembered outside of the 5800 system and may later be used to retrieve the data associated with that object using the retrieveObject operation.

Every object has metadata whether or not user metadata was supplied at the time of the store. For example each object has system metadata that is system assigned and can never be modied by the user. The OID is associated with the metadata record that represents this object as a whole; the metadata record is then associated with the underlying data:

OID ↔ Metadata Record → Underlying Object Data

The retrieveObject operation takes an OID as input and returns a stream of bytes as output that are identical to the bytes stored during the storeObject operation. Both the storeObject and retrieveObject operations handle the data in a streaming manner. Not all of the data need be present in client memory or in server memory at the same time, which is a crucial point for working with large objects.

For the 5800 system Release 1.1, data sizes up to 400 GBytes are tested and supported. Using sizes even smaller than this may be appropriate as a best practice. For more information, see

Chapter 5, “Programming Considerations and Best Practices.”

From within a customer application, the storing of an object into the archive is an all-or-nothing event. Either the object is stored or it is not; there are no partial stores. If a store operation is interrupted, the entire storeObject call fails. Once an OID is returned to the customer application, the object is known to be durable. In the event of an outage that causes some data loss, the system should be no more likely to lose a newly stored object than any other object. There is no way to tie together two dierent store operations so that both either succeed together or fail together.

Note – A stored object may or may not immediately be queryable. For more information, see

“The 5800 System Query Integrity Model” on page 21.

Sun StorageTek5800 SystemClient API Reference Manual • June 200818

5800 System Overview

The 5800 System Metadata Model

Metadata means “data about the data”; it describes the data and helps to determine how the data should be interpreted. In addition, metadata can be used to facilitate querying the 5800 system for objects that match a particular set of search criteria.

For the 5800 system, the supported metadata option is in the form of name-value elds stored with each object. The set of possible elds is dened in the metadata schema. Setting up a metadata schema is an important system administration task that is described in the 5800 System Administration Guide, and is analogous to the process of database design that goes into creating a data management application. The metadata schema determines what eld names, types, and lengths may be used with the metadata stored with each object. In addition, the layout of elds into tables within the schema, together with the denition of views that speed certain searches, determine which kinds of queries about that metadata will be both possible and eective. As such, the metadata schema should match the characteristics of the expected range of applications that will deal with the stored data. The underlying software is designed to support multiple dierent kinds of metadata to aid in searching. For example, eventually there might be a specialized index to facilitate full-text search within the data objects. This document describes only the API for dealing with the name-value metadata type.

Fields in the schema can be either queryable or non-queryable. The values for non-queryable elds may be retrieved later but may not be used in queries. The 5800 system supports only single-valued elds. Each object can have only a single name-value pair of a given name. There is no built-in support for multiple-valued elds, such as a list of authors of a book in the form of multiple elds named 'author'.

Each data object is associated with a set of name-value pairs at the time the object is stored. Some metadata (system metadata) is assigned by the5800 system as each object is stored. For example, each object contains an “object creation time” (system.object_ctime) and an OID (system.object_id), both of which are assigned by the system at the time an object is created. Some metadata (the computed metadata) is implicit in the stored data, and is made explicit at the time of the object store. For example, the system exposes the object data length as a metadata eld (system.object_size). In addition, the 5800 system computes a Secure Hash Algorithm (SHA1) hash of the stored data as the data is stored and stores the hash as a metadata eld (system.object_hash). There is also an associated eld (system.object_hash_alg)to specify which hash algorithm was used in computing the system.object_hash. It is currently always set to “sha1.”

Finally, some metadata (the user metadata) is supplied by the customer application in the API call at the time an object is stored. Each store operation is allowed to include a NameValueRecord that indicates a set of name-value pairs to be associated with the data object as metadata. Each name in the name-value record must match a eld name from the metadata schema; in addition, the data value supplied for each eld must match the type and length for the eld as specied in the schema. If the names or values supplied for the user metadata do not match the active schema, then an exception is generated and the object is not stored.

Chapter 1 • Sun StorageTek 5800 System Client API 19

5800 System Overview

The metadata associated with an object is immutable. There is no operation to modify the metadata associated with an object after the object has been stored. Instead, the storeMetadata operation can be used to create a completely new object by associating new user metadata with the underlying data and system-metadata of an existing object. The storeMetadata operation does not merge the new metadata in with the metadata from the original OID; instead, the storeMetadata operation creates a new metadata record pointing to the same data object. To accomplish a merge of new eld values into existing metadata, the customer application must manually retrieve the existing metadata from the original object, perform the merge into a single NameValueRecord on the client side, and then call storeMetadata to create a new object with the merged metadata.

When creating a new object using storeMetadata, a new system.object_id and new system.object_ctime are generated, to indicate that a new object has been created. The metadata computed from the object data itself (system.object_length, system.object_hash_alg, and system.object_hash) does not change. Both the storeObject and the storeMetadata operations return a SystemRecord value that includes all of the system-assigned elds.

While retrieving the OID is the most common use of the SystemRecord, the other system elds can also be helpful. For example, the customer application might use the system.object_length, the system.object_hash_alg and the system.object_hash elds to verify that the data as stored matches the data as present in the customer application. If a hash independently computed on the client matches the hash stored on the 5800 system, then the data store has been validated.

The metadata values associated with an object can be retrieved using the retrieveMetadata operation. The retrieveMetadata operation takes an OID as input, and returns the entire set of user, system, and system-computed metadata. The retrieved metadata is in the form of a NameValueRecord that contains the value of each eld as originally stored. The system elds occur using their eld names, for example. the eld system.object_ctime contains the object creation time. There is no operation to retrieve just a single eld or a subset of elds by supplying a list of eld names. The retrieveMetadata operation retrieves the values of both queryable and non-queryable elds.

The 5800 System Query Model

One of the primary methods for retrieving data is to specify the characteristics of the desired data and then let the system nd it for you. In the 5800 system, a query expression species a set of conditions on metadata eld values. The system then returns a list of all the objects whose metadata values match the query conditions. Each object is considered individually without reference to any other objects. There are no queries that compare elds in one object with elds in a dierent object.

Sun StorageTek5800 SystemClient API Reference Manual • June 200820

5800 System Overview

Query expressions can use much of the power of Structured Query Language (SQL). Each query expression combines SQL functions and operators, eld names from the metadata schema, and literal values. There are no query expressions that select objects based on the data stored in the object itself; all queries apply only to the metadata elds associated with the object. Only queryable elds can be used in query expressions. For an object to show up in a query result set, the object must have a value for each of the elds mentioned in the query; in other words, there is an implicit INNER JOIN between the elds in the query.

A query may optionally specify that the result set should include not just the OID of each matching object, but also the values from a set of selected elds of each matched object . The value retrieved by Query With Select for some eld may be a canonical equivalent of the value originally stored in that eld. For example, values in numeric elds may have been converted to standard numeric format. Trailing spaces at the end of string elds will have been truncated (The value that is returned will be some value that would match the original data as stored, in the SQL sense.) To be included in the result set, an object must include values for all queried elds and all selected elds. In other words, there is an implicit INNER JOIN between all the elds in the query and in the select list.

There are signicant limitations on which queries may be executed eciently, or at all. See

Chapter 4, “Sun StorageTek 5800 System Query Language,” and Chapter 5, “Programming Considerations and Best Practices”

s for details of these limitations.

There are no ordering guarantees between queries and store operations that are proceeding at the same time. If an object is added to the 5800 system while a query is being performed, and the object matches the query, then the object may or may not show up in the query result set.

For a detailed description of query syntax and query semantics, including a description of exactly what it means for an object to match a query, see

System Query Language.”

Chapter 4, “Sun StorageTek 5800

The 5800 System Query Integrity Model

The result set of any query will only return results that match the query. But will it return ALL the matching results? That is the concept of query completeness, referred to here as query integrity. 100% query integrity for a result set is dened as a state in which the result set contains all the objects in the 5800 system that match that particular query. The 5800 system is not always in a state of 100% query integrity. Various system events can induce a state in which the set of objects that are available for query is smaller than the total set of objects stored in the archive. Each query result set supports an operation (isQueryComplete) whereby the customer application can ask, once all the results from the query result set have been processed, whether that set of results constitutes a complete set.

Chapter 1 • Sun StorageTek 5800 System Client API 21

5800 System Overview

Note – The format of records as stored in the reliable and scalable object archive is not suitable

for fast query. To enable searching, the queryable elds from the metadata are indexed in a query engine that can provide fast and exible query services. The query engine is basically an SQL database. This is why the 5800 system's query language can borrow so heavily from SQL. At various times, the data as indexed in the query engine can get out of date compared to what is stored in the archive. When this happens, query result sets are not known to be complete until the contents of the query engine can be brought back up to date with the actual contents of the archive again.

The 5800 system concept of query integrity as actually implemented is somewhat looser than that of 100% query integrity. Even if a query result set indicates the result set is complete, the 5800 system allows certain objects, known as store index exceptions, to be missing from the query result set, as long as those exceptions were communicated to the customer application at the time the object was stored.

A store index exception is an object for which the original store of the object into the archive succeeded, but at least some part of the insert into the query engine (database) did not succeed. The object may or may not show up in all of the queries that it matches. A store index exception is communicated to the customer application at the time of store by means of a method SystemRecord.isIndexed. A value of false from isIndexed means that the object is not immediately available for query.

A store index exception is said to be resolved when the object becomes available for query. The checkIndexed method can be used to attempt to resolve a store index exception under program control. The checkIndexed operation checks if the object has been added to the query engine, and attempts to insert it if the object has not been added. If the insert into the query engine succeeds, the object is thereby restored to full queryability.

All store index exceptions will also eventually be resolved automatically by ongoing system healing. Each query result set also exports a method getQueryIntegrityTime that can be used to get detailed status on which store index exceptions might still be unresolved. The query integrity time is a time such that all store index exceptions from before that time have been resolved. There is an “ideal” query integrity time, which is the time of the oldest still-unresolved store index exception: an ideal implementation when asked for the query integrity time would always report this ideal value. In actual implementation, the reported query integrity time might be hours or even days earlier than the ideal query integrity time, depending on how far the ongoing system healing has progressed.

Deleting Objects from the 5800 System

The 5800 system client API exports an operation to delete a specic object as specied by its OID. Once a delete operation completes normally, subsequent attempts to retrieve that object will fail with an exception. In addition, the object will stop showing up in query result sets that

Sun StorageTek5800 SystemClient API Reference Manual • June 200822

5800 System Overview

match the original object metadata. There are no transactional guarantees regarding ordering of queries and delete operations that are occurring at the same time. If an object is being deleted at the same time that a query that matches that object is being performed, then that object may or may not show up in the query result set, with no guarantee either way.

Note – When all objects that share an underlying block of data storage been deleted, the

underlying block of data storage will itself be scavenged and returned to the supply of free disk space. But all details of how objects are stored, and how and whether they ever share data — or ever are scavenged — are outside of the scope of this API.

Delete operations are all-or-nothing,with some caveats. Specically, if a delete operation fails with an error, it is possible that the object is not fully deleted but is temporarily not queryable. Such an object is in an analogous state to a store index exception (see

“The 5800 System Query Integrity Model” on page 21). The queryability of such an object will eventually be resolved by

automatic system healing. In addition, the queryability of such an object can be resolved under program control by using the checkIndexed method. Alternatively, the customer application may choose to re-execute the delete operation until it succeeds, or until it fails with an error that indicates the object is already deleted.

Chapter 1 • Sun StorageTek 5800 System Client API 23

CHAPTER 2

Sun StorageTek 5800 System Java Client API

This chapter provides information on the 5800 system Java client API.

The following topics are discussed:

■

“Overview of the 5800 System Java Client API” on page 25

■

“Java Client Application Deployment” on page 27

■

“Java API” on page 27

Note – You can nd detailed information on the 5800 system Java client API in the Javadocs,

which are located in the java/doc/htdocs directory.

Overview of the 5800 System Java Client API

This section provides an overview of the 5800 system Java client API. The following topics are discussed:

■

“Client Library” on page 25

■

“Interfaces” on page 26

■

“Retrying Operations” on page 26

■

“Performance and Scalability” on page 26

■

“Updating Client View of the Schema” on page 27

Client Library

The 5800 system Java client library provides a simple way to communicate with 5800 system clusters. It provides programmatic access to the 5800 system network protocol, which operates over HTTP, enabling you to store, retrieve, query, and delete object data and metadata.

Overview of the 5800 System Java Client API

The 5800 system Java client library provides a platform-independent mechanism to upload data and metadata to a 5800 system, and to retrieve and query the data and metadata. The Java client library works with any implementation of J2SE to the 5800 system cluster. Access is designed to be high-level and easy to use. Most operations are accomplished in a single (synchronous) function call.

Interfaces

The Java client API interacts with the 5800 system server entirely through an HTTP protocol. The HTTP communication layer uses the Apache Commons HTTP client.

Object data is streamed through the Java client library opaquely and a well-dened data hash is returned for verication purposes. Metadata is added or retrieved with typed accessors. The stored representation of metadata on the 5800 system server is not exposed to the user, and no hash is returned when metadata is stored.

The 5800 system Java client library provides the NameValueObjectArchive class as an application access layer, which should be appropriate for most applications. In addition, an advanced interface provides a mechanism to customize the 5800 system and to serve as a toolkit to build new applications.

Note – The advanced toolkit is not described in this document. If you are interested in pursuing

advanced applications, contact your 5800 system Sales Representative.

platform 4.0 or later with HTTP connectivity

Retrying Operations

Calls to the Java API should be wrapped with retry logic so that their applications are resilient to transient failures that may be experienced when a node or switch fails while servicing an operation.

Requests that fail on recoverable HTTP errors are automatically retried once. A typical recoverable error occurs when the 5800 system HTTP server times out a connection that the client then tries to reuse (the client maintains a collection pool). This results in a connection failure at request time. Because this is a recoverable error, it is retried and the retry typically succeeds.

Performance and Scalability

Starting the Java Virtual Machine (JVM) incurs a performance penalty, but once the JVM is running, you can use the client object archive repeatedly and from multiple threads. I/O is synchronous (blocking). HTTP connections are pooled for performance. You should instantiate one instance of the NameValueObjectArchive per 5800 system server and use it for all access to that server until exit.

Sun StorageTek5800 SystemClient API Reference Manual • June 200826

Updating ClientView of the Schema

In the Java client API, the schema is fetched when the NameValueObjectArchive class is instantiated. If the schema has changed, the client application needs to create a new NameValueArchive. A local copy of the schema is used for some metadata operations.

Java Client Application Deployment

Java applications using the 5800 system Java API reference the honeycomb-client.jar library. You must include this library in your classpath when running your application. The 5800 system Java API was designed to run on Java v1.4, so you need to run your client applications with a Java environment of v1.4 or greater.

Java API

The 5800 system Java client library provides a simple way of communicating with 5800 system clusters. It provides programatic access to the 5800 system network protocol, which operates over HTTP. You can implement most applications using a handful of these classes, but access to “expert” features is also included.

This section discusses the following topics:

■

“Java API Packages” on page 27

■

“Java API Documentation” on page 27

■

“Basic Concepts” on page 28

■

“Key Classes” on page 28

■

“NameValueObjectArchive Application Access” on page 30

Java API

Java API Packages

The Java API is implemented in two packages:

■

com.sun.honeycomb.client

Provides the base classes required to interact with a 5800 system cluster.

■

com.sun.honeycomb.common

Contains classes for server-side exceptions.

Java API Documentation

The Java API documentation (Javadoc) is located in the SDK java/doc/htdocs directory, and can be accessed using a browser.

Chapter 2 • Sun StorageTek 5800 System Java Client API 27

Java API

Basic Concepts

The root of the 5800 system Java client API is the NameValueObjectArchive class, which represents a connection to a single 5800 system server. All operations are initiated by invoking methods on a NameValueObjectArchive instance after initializing it with the address of a cluster. The fact that a cluster of machines, rather than a single server, is handling the requests is transparent to the application programmer.

A NameValueObjectArchive uses instances of the ObjectIdentifier class to uniquely identify stored data objects. That is, there is a one-to-one correspondence between instances of ObjectIdentifer and 5800 system metadata objects.

Note – There is potentially a many-to-one relationship between metadata and data objects.

When using NameValueObjectArchive, all metadata queries are executed against a 5800 system server’s user-congurable index of name-value pair lists. This class also ensures that a metadata entry is created for every data object stored, even if no metadata is provided at store time.

An instance of the NameValueObjectArchive class functions as a proxy for the 5800 system server. Instantiation incurs some overhead in establishing communication, so reusing a single instance is the recommended practice. Multithreading is supported with the same instance.

NameValueObjectArchive also allows all metadata operations to be performed in terms of two classes that represent metadata records: SystemRecord and NameValueRecord. These classes represent 5800 system metadata entries. When using NameValueObjectArchive, every stored data object has a corresponding NameValueRecord that contains the extended attributes stored with that data object, and each NameValueRecord has a reference to its SystemRecord, which contains built-in system attributes such as data object size and creation time. In this model, all instances of ObjectIdentifer returned from store operations and metadata queries correspond directly to instances of NameValueRecord.

The results of a 5800 system metadata query are returned using instances of the QueryResultSet class, which the application can step through to retrieve metadata or identiers. This class manages the details of fetching one batch of results after another.

Key Classes

This section provides an overview of the following key classes in the 5800 system Java client API. For more information on using the following classes, see Also see the Javadoc provided with the 5800 system SDK.

■

“NameValueObjectArchive” on page 29

■

“NameValueSchema” on page 29

Sun StorageTek5800 SystemClient API Reference Manual • June 200828

“Basic Concepts” on page 28.

Java API

■

“ObjectIdentifier” on page 29

■

“QueryResultSet” on page 30

■

“SystemRecord” on page 30

■

“NameValueRecord” on page 30

For more information on using these classes, see “Basic Concepts” on page 28.

NameValueObjectArchive

The NameValueObjectArchive class is the main entry point into the 5800 system. Each instance of NameValueObjectArchive provides access to a specic 5800 system server, functioning as a proxy object on which operations can be performed. Multiple simultaneous operations can be accomplished in separate threads on the same NameValueObjectArchive instance. Communication with the 5800 system server is entirely by means of HTTP requests. A pool of HTTP connections is maintained for eciency.

A NameValueObjectArchive instance enables you to store, retrieve, query and delete object data and associated metadata records. Metadata is associated with an object in a set of name-value pairs (see associate application-specic information with the raw data, such as name, mime type, or purge date. Metadata records consist of structured data that can be queried. Object data is opaque to the 5800 system.

“NameValueRecord” on page 30). Metadata records can be used to

A NameValueObjectArchive instance always ensures that a metadata record is created on the 5800 system server for each newly stored object, even if no metadata is provided with the store. This enables a model of programming where every stored data object is accessed by name-value metadata records (for example, for examining results from queries or performing delete operations). Object data is never deleted directly; it is deleted when its last referencing metadata record is deleted.

For additional information, see

“NameValueObjectArchive Application Access” on page 30.

NameValueSchema

An instance of NameValueSchema represents information about the name-value metadata that the 5800 system system uses to index data. This instance can be used to enumerate the elds available in the schema as attributes. Each attribute has a name and a type.

See the Sun StorageTek 5800 System Administrator’s Guide for information on how to dene attributes.

ObjectIdentifier

Instances of ObjectIdentifier uniquely represent objects in a 5800 system store. The 5800 system creates these instances when objects are stored and are returned to the client as part of the store result.ObjectIdentifier instances can be stored outside of the 5800 system and used

Chapter 2 • Sun StorageTek 5800 System Java Client API 29

Java API

later for retrieving objects. External storage can be accomplished using an identier's string representation by invoking the toString method. An instance of ObjectIdentifier can be reconstituted using the constructor that takes String as an argument.

QueryResultSet

Instances of QueryResultSet provide access to the objects and metadata matching a query. The query results can be stepped through using the next method. The individual results are identiers representing objects that match the query.

If selectKeys was specied in the original query, these metadata elds can be accessed using the typed getter methods with each eld’s name.

SystemRecord

Instances of SystemRecord represent the system metadata for an object, including OID, object size, SHA1 hash, and creation time. They are returned by storeObject and storeMetadata.

NameValueRecord

Instances of NameValueRecord represent metadata used by the 5800 system to store and index user-extensible lists of name-value pairs. For convenience, instances of NameValueRecord also contain references to the SystemRecord instances of the objects they represent.

NameValueObjectArchive Application Access

Most applications make use of the NameValueObjectArchive class. This class ensures that a default metadata entry is created for every data object stored, even if no metadata is explicitly provided at store time.

The NameValueObjectArchive object functions as a proxy for the 5800 system server. All access is enabled by invoking methods on this object.

The following key methods and classes are used with the NameValueObjectArchive class:

■

“NameValueObjectArchive” on page 31

■

“delete” on page 31

■

“storeObject” on page 31

■

“storeMetadata” on page 32

■

“checkIndexed” on page 32

■

“retrieveObject” on page 33

■

“retrieveMetadata” on page 33

■

“getSchema” on page 33

■

“query” on page 34

■

“query (with selectKeys)” on page 34

Sun StorageTek5800 SystemClient API Reference Manual • June 200830

■

“query (with PreparedStatement)” on page 35

■

“query (with PreparedStatement and selectKeys)” on page 35

■

“PreparedStatement” on page 36

■

“QueryResultSet” on page 37

■

“getObjectIdentifier” on page 37

■

“isQueryComplete” on page 37

■

“getQueryIntegrityTime” on page 38

■

“QueryIntegrityTime” on page 38

NameValueObjectArchive

Sun StorageTek5800 SystemClient API Reference Manual • June 200832

Java API

checkIndexed returns an int value that indicates if the metadata for this object has been inserted into the query engine. The value is -1 if the metadata was already inserted before this operation was called, 0 if the metadata has still not been inserted, or 1 if the metadata was just now inserted.

retrieveObject

Writes all of the data for the specied object into the provided channel, returning the amount of data actually retrieved.

Synopsis

public long retrieveObject(ObjectIdentifier oid,

WritableByteChannel dataChannel) throws ArchiveException, java.io.IOException

public long retrieveObject(ObjectIdentifier oid,

java.nio.channels.WritableByteChannel dataChannel,long firstByte, long lastByte) throws ArchiveException, java.io.IOException

Description

Takes an OID and downloads the data object into a supplied WritableByteChannel.

retrieveMetadata

Returns a NameValueRecord instance containing the system and name-value metadata for the metadata record identied by the OID.

Synopsis

public NameValueRecord

retrieveMetadata(ObjectIdentifier oid) throws ArchiveException, java.io.IOException

Description

Returns a NameValueRecord instance containing the system and name-value metadata for the metadata record identied by the OID.

getSchema

Returns the runtime conguration of the name-value object archive as a NameValueSchema instance.

Chapter 2 • Sun StorageTek 5800 System Java Client API 33

Java API

Synopsis

public NameValueSchema getSchema()

throws ArchiveException, java.io.IOException

Description

Returns the runtime conguration of the name-value object archive as a NameValueSchema instance.

query

Returns a ResultSet of SystemRecord instances containing MetadataRecord OIDs.

Synopsis

public QueryResultSet

query(java.lang.String query,int resultsPerFetch) throws ArchiveException, java.io.IOException

Description

Takes a where clause and returns a QueryResultSet of SystemRecord instances containing MetadataRecord OIDs.

The query parameter is a where clause in the 5800 system query syntax, which is a subset of SQL.

Returns a QueryResultSet. The results are stepped through by calling the next method and using the typed getXXX accessor methods.

Note – For more information on the 5800 system query language, refer to Chapter 4, “Sun

StorageTek 5800 System Query Language.”

query (with selectKeys)

Returns a ResultSet of NameValueRecord instances containing the selected values.

Synopsis

public QueryResultSet

query(java.lang.String query, java.lang.String[] selectKeys,int maxResults) throws ArchiveException, java.io.IOException

Sun StorageTek5800 SystemClient API Reference Manual • June 200834

Java API

Description

Takes a where clause and a select clause and returns a QueryResultSet of NameValueRecord instances containing the selected values.

selectKeys identies the values to be returned, functioning as an SQL select clause.

The query parameter is a where clause in the 5800 system query syntax, which is a subset of SQL.

Returns a QueryResultSet. The results are stepped through by calling the next method and using the getObjectIdentifier accessor.

Note – For more information on the 5800 system query language, refer to Chapter 4, “Sun

StorageTek 5800 System Query Language.”

query (with PreparedStatement)

Returns the OIDs of metadata records matching the query as a QueryResultSet instance.

Synopsis

public QueryResultSet query(PreparedStatement query,

int resultsPerFetch)

Description

Takes a PreparedStatement and returns a QueryResultSet of SystemRecord instances containing MetadataRecord OIDs.

The PreparedStatement parameter enables queries with dynamic parameters to pass typed data items to the query.

Returns a QueryResultSet. The results are stepped through by calling the next method and using the typed getXXX accessor methods.

Note – For more information on the 5800 system query language, refer to Chapter 4, “Sun

StorageTek 5800 System Query Language.”

query (with PreparedStatement and selectKeys)

Returns specied elds from metadata records matching the query as a QueryResultSet instance.

Chapter 2 • Sun StorageTek 5800 System Java Client API 35

Java API

Synopsis

public QueryResultSet

query(PreparedStatement query, java.lang.String[] selectKeys, int resultsPerFetch)

Description

Takes a where clause and a select clause and returns a QueryResultSet of NameValueRecord instances containing the selected values.

selectKeys identies the values to be returned, functioning as an SQL select clause.

The PreparedStatement parameter enables queries with dynamic parameters to pass typed data items to the query.

Returns a QueryResultSet. The results are stepped through by calling the next method and using the getObjectIdentifier accessor.

Note – For more information on the 5800 system query language, refer to Chapter 4, “Sun

StorageTek 5800 System Query Language.”

PreparedStatement

Extends com.sun.honeycomb.common.Encoding

Synopsis

public PreparedStatement(java.lang.String sql);

Description

Used to implement queries with Dynamic Parameters, which is the preferred way to pass typed data items to a StorageTek 5800 query.

The number of bindParameter calls should match the number of question marks (?) in the query string in the prepared statement. Parameters are specied positionally. For example, a bindParameter call with index = 1 supplies a value for the rst ? in the supplied query string. Once a value has been supplied for each of the dynamic parameters, then the PreparedStatement may be passed to the NameValueObjectArchive.query method to be executed, for example:

NameValueObjectArchive archive = new NameValueObjectArchive(hostname); Date date_value= new java.sql.Date(); PreparedStatement stmt = new PreparedStatement("date_field<?”);

Sun StorageTek5800 SystemClient API Reference Manual • June 200836

Java API

The query integrity time is a time such that all store index exceptions from before that time have been resolved. There is an ideal query integrity time, which is the time of the oldest still-unresolved store index exception: an ideal implementation when asked for the query integrity time would always report this ideal value. In actual implementation, the reported query integrity time might be hours or even days earlier than the ideal query integrity time, depending on how far the ongoing system healing has progressed.

QueryIntegrityTime

Get detailed status on which store index exceptions might still be unresolved

Synopsis

QueryResultSet.isQueryComplete(), QueryResultSet.getQueryIntegrityTime();

Sun StorageTek5800 SystemClient API Reference Manual • June 200838

CHAPTER 3

Sun StorageTek 5800 System C Client API

This chapter provides detailed information on the 5800 system C client API.

The following topics are discussed:

■

“Overview of the 5800 System C Client API” on page 39

■

“C Client Application Deployment” on page 43

■

“Nonblocking C API” on page 43

■

“Synchronous C API” on page 44

■

“Synchronous C Data Types” on page 46

■

“Synchronous C API Functions” on page 53

■

“Querying With a Prepared Statement” on page 110

Overview of the 5800 System C Client API

This section provides an overview of the 5800 system C client API. The following topics are discussed:

■

“Architecture” on page 40

■

“Interfaces” on page 40

■

“Retrying Operations” on page 40

■

“Multithreaded Access” on page 40

■

“Performance and Scalability” on page 40

■

“Memory Usage” on page 41

■

“Updating Schema Denitions” on page 41

■

“Session Management” on page 41

Overview of the 5800 System C Client API

Architecture

The 5800 system C API client supports two dierent access patterns: a synchronous “EZ” access very similar to the current Java implementation, and a more exible, nonblocking access based on the POSIX model.

Note – For this release, the nonblocking C API client is not implemented.

Interfaces

The C client library interacts with the 5800 system server entirely through an HTTP protocol.

Retrying Operations

Calls to the C API should be wrapped with retry logic so that their applications are resilient to transient failures that may be experienced when a node or switch fails while servicing an operation.

Multithreaded Access

Both the synchronous and the nonsynchronous C APIs are fully thread-safe and can be used simultaneously in multiple threads from the same process. Each thread must call

“hc_session_create_ez” on page 53 to create its own session. Sessions must not be shared

between threads.

Caution – Name resolution must be done in a single thread with the subsequent IP address

passed to hc_session_create_ez, otherwise core dumps will occur if multiple name resolution threads call getaddrinfo at the same time.

Performance and Scalability

The 5800 system C client library provides high performance and is highly scalable.

The synchronous C API performs its own calls to select() internally.

For the nonblocking C API (not yet implemented), access is provided to the underlying fd_set so that all pending I/O operations can be serviced by a single thread on the basis of status returned by the POSIX select() function, possibly after merging the 5800 system fd_set with some external, application-specic fd_set.

Sun StorageTek5800 SystemClient API Reference Manual • June 200840

Overview of the 5800 System C Client API

Memory Usage

The 5800 system C client library generally follows the model of populating externally allocated data structures such as handles, buers, and result arrays.

Some internal data structures are generated during XML document construction. These data structures are allocated and freed using the function pointers supplied to hc_init when initializing the library (see

Other data structures are allocated and returned to the user; these have corresponding functions to free them. For example, hc_session_create_ez and hc_session_free.

“Initializing a Global Session” on page 41).

Updating Schema Denitions

The C client library does not automatically refresh its in-memory schema denitions. If the schema is changed through the command-line interface (CLI), a new session must be created with a new call to

“hc_session_create_ez” on page 53 to access the schema changes.

Session Management

A global session must be explicitly initialized with a call to hc_init and released with a call to hc_cleanup. Memory allocators and deallocators are supplied in the initialization function to

control how memory allocation occurs. You will normally supply the standard malloc, free, and realloc functions for this functionality.

Heap Memory Allocator

The heap memory allocator is dened as follows:

typedef void* (*allocator_t) (size_t size);

Heap Memory Deallocator

The heap memory deallocator is dened as follows:

typedef void (*deallocator_t) (void *p);

Heap Memory Reallocator

The heap memory reallocator is dened as follows:

typedef void (*reallocator_t) (void *p,size_t size);

Initializing a Global Session

The following function initializes a global session:

Chapter 3 • Sun StorageTek 5800 System C Client API 41

Overview of the 5800 System C Client API

hcerr_t hc_init(allocator_t,

This function must be called once per process to initialize the memory functions used in the 5800 system C API. Italso initializes global session properties.

A global session is initialized once per process, regardless of how many threads in that process are using the C API.

Note – hc_init should be called once per process before any thread calls

“hc_session_create_ez” on page 53.If“hc_session_create_ez” on page 53 is called before

hc_init, an implicit call is made to hc_init from that thread. But that call to hc_init is not interlocked with other threads, and it uses the C API shared library’s version of malloc and free, which might be dierent than the application’sversion of malloc and free. It is strongly recommended that all applications call hc_init once per process with their own allocator and deallocator.

Note – For more information on “hc_session_create_ez” on page 53 , see

“hc_session_create_ez” on page 53.

deallocator_t, reallocator_t);

Terminating a Global Session

The following function terminates a global session:

void hc_cleanup();

System Record

All 5800 system store operations return a system record, which encapsulates information about the stored object. In particular, the system record contains the OID, which can be used to retrieve the stored object data or metadata.

typedef struct hc_system_record_ {

char is_indexed; hc_oid oid; hc_digest_algo digest_algo; hc_digest data_digest; hc_long_t size; hc_long_t creation_time; hc_long_t deleted_time; char shredMode;

} hc_system_record_t;

About the elds:

Sun StorageTek5800 SystemClient API Reference Manual • June 200842

Nonblocking C API

■

oid — The objectid for this object, equivalent to the system.object_id eld.

■

digest_algo — Always set to "sha1" for this release. Equivalent to the

system.object_hash_alg eld.

■

data_digest — An array of bytes that represent the content digest of this object's data. Equivalent to the system.object_hash eld.

■

size — The size of the data in this object, in bytes. Equivalent to the system.object_size eld.

■

creation_time — The object creation time, expressed as number of milliseconds since the epoch. Equivalent to the system.object_ctime eld.

■

deleted_time — The deletion time of this record, if any, as the number of milliseconds since the epoch.

■

shredMode — Not used in this release.

■

is_indexed — indicates, after a store_data or store_metadata operation, whether the metadata for the object was successfully inserted into the query engine, and the object is hence available for query. 0 if false, 1 if true.

Failureand Recovery

Every function in the 5800 system C client library returns a result code of type hcerr_t.Any value other than HCERR_OK indicates a nonrecoverable error. See the hc.h le for specic error codes.

C Client Application Deployment

C applications using the 5800 system C API use both the 5800 system libraries and the curl libraries. These libraries are dierent for each supported platform (Windows, Linux, Solaris (x86), Solaris (SPARC)) and are located in the c/<OS>/lib directory in the SDK.

Note – The environment variable http_proxy should not be set for processes using the C API,

since the HTTP client library ( curl) makes use of it.

Nonblocking C API

The nonblocking C API is not implemented for this release of the 5800 system. If you are interested in working with the nonblocking C API, contact your 5800 system Sales Representative.

Chapter 3 • Sun StorageTek 5800 System C Client API 43

Synchronous C API

A multiplatform synchronous C API in which operations are accomplished in a few simple function calls is provided for the 5800 system. The API calls include operations for storing, retrieving, deleting, and querying of data and metadata records. Multiple threads are supported, and operations block until they complete.

You must call hc_init (once per process) and

“hc_session_create_ez” on page 53 (once per

thread) prior to making any other API calls.

All functions in the 5800 system C API return an hc_err. Any value other than HCERR_OK indicates failure.

This section discusses the following topics for the 5800 system synchronous C API.

■

“Changes for the 1.1 Release” on page 44

■

“Limitations” on page 45

Changes for the 1.1 Release

This release of the synchronous C API contains the following changes:

■

Handling is added for storing, retrieving and querying the following new metadata types:

– char — for Latin 1 character set – unicode – binary – date – time – timestamp

Query and queryplus are merged

■

Prepared statements (pstmts) are introduced to handle values of queries that cannot be placed inline, and a new query is introduced to handle them.

■

The following new functions have been added to the API:

–

“hc_check_indexed_ez” on page 89

– “hc_decode_hcerr” on page 112 – “hc_decode_hc_type” on page 113 – “hc_schema_get_length” on page 61 – “hc_nvr_add_binary” on page 71 – “hc_nvr_add_date” on page 72 – “hc_nvr_add_time” on page 73 – “hc_nvr_add_timestamp” on page 74 – “hc_nvr_get_binary” on page 81 – “hc_nvr_get_date” on page 82

Sun StorageTek5800 SystemClient API Reference Manual • June 200844

– “hc_nvr_get_time” on page 82 – “hc_nvr_get_timestamp” on page 83 – “hc_pstmt_create” on page 100 – “hc_pstmt_free” on page 101 – “hc_pstmt_set_string” on page 101 – “hc_pstmt_set_char” on page 102 – “hc_pstmt_set_double” on page 103 – “hc_pstmt_set_long” on page 104 – “hc_pstmt_set_date” on page 105 – “hc_pstmt_set_time” on page 106 – “hc_pstmt_set_timestamp” on page 107 – “hc_pstmt_set_binary” on page 108 – “hc_pstmt_query_ez” on page 109 – “hc_qrs_is_query_complete” on page 97 – “hc_qrs_get_query_integrity_time” on page 98

The following functions have changed in the API:

■

“hc_query_ez” on page 94

■

“hc_qrs_next_ez” on page 96

The following functions and types have been removed from the API:

■

hc_query_plus_result_set_t

■

hc_query_plus_ez

■

hc_qprs_next_ez

■

hc_qprs_free

Synchronous C API

Limitations

This release of the synchronous C API is subject to the following limitations:

■

Changes to the metadata schema at the server are only detected at the client at the next call to

“hc_session_create_ez” on page 53.

■

The values returned by “hc_session_get_platform_result” on page 58 will not be updated properly after calls to the functions

“hc_delete_ez” on page 111.

■

When using the C API, the maximum metadata size of a data item stored using either

“hc_store_both_ez” on page 87 or “hc_store_metadata_ez” on page 88 is limited to

approximately 76300 bytes. The exact maximum metadata size depends on many factors and should not be relied on. This limitation does not apply to metadata stored using the Java API.

Chapter 3 • Sun StorageTek 5800 System C Client API 45

“hc_retrieve_ez” on page 91 and

Synchronous C DataTypes

The following data types are dened for the C API:

■

“hc_string_t” on page 46

■

“hc_long_t” on page 46

■

“hc_double_t” on page 46

■

“hc_type_t” on page 47

■

“hc_value_t” on page 47

■

“hc_schema_t” on page 48

■

“hc_nvr_t” on page 48

■

“hc_session_t” on page 48

■

“hc_pstmt_t” on page 49

■

“read_from_data_source” on page 49

■

“write_to_data_destination” on page 50

■

“hcerr_t” on page 51

hc_string_t

Type for holding Unicode (UTF-8) and Latin-1 null-terminated strings.

Synopsis

5800 system name-value metadata type specier.

Synopsis

typedef enum hc_types_{

HC_UNKNOWN_TYPE = -1, HC_BOGUS_TYPE = 0, HC_STRING_TYPE = 1, HC_LONG_TYPE = 2, HC_DOUBLE_TYPE = 3, HC_BYTE_TYPE = 4, HC_CHAR_TYPE = 5, HC_BINARY_TYPE = 6, HC_DATE_TYPE = 7, HC_TIME_TYPE = 8, HC_TIMESTAMP_TYPE = 9, HC_OBJECTID_TYPE = 10,

} hc_type_t;

Synchronous C DataTypes

Description

Species one of the 5800 system metadata types that can go in the archive.

hc_value_t

5800 system name-value metadata data value.

Synopsis

typedef struct hc_value_ {

hc_type_t hcv_type; union {

hc_string_t hcv_string; hc_long_t hcv_long; hc_double_t hcv_double;

Chapter 3 • Sun StorageTek 5800 System C Client API 47

Synchronous C DataTypes

Structure describing the connection from one thread to one 5800 system server.

Synopsis

typedef void hc_session_t;

Sun StorageTek5800 SystemClient API Reference Manual • June 200848

Synchronous C DataTypes

Description

An opaque structure to represent the session from one thread to one 5800 system server. It contains the schema used to interpret metadata store and retrieve operations to this 5800 system server.

hc_pstmt_t

Structure for holding a prepared statement.

Synopsis

typedef void hc_pstmt_t;

Description

An opaque structure representing a query, including the query text and bound elds.

hc_query_result_set_t

Structure used to hold the results of a query.

Synopsis

typedef void hc_query_result_set_t;

Description

This opaque structure is used to hold the results of a query. For more information on the functions that use this structure, see

“Querying Metadata” on page 94.

read_from_data_source

Data source template used to upload object data to the cluster.

Synopsis

typedef long (*read_from_data_source) (void *cookie, char *buf, long buf_size);

Chapter 3 • Sun StorageTek 5800 System C Client API 49

Synchronous C DataTypes

Description

Function pointers of read_from_data_source type are used to upload object data. The function pointer and opaque cookie reference are supplied as arguments to

page 87

called repeatedly, with the supplied cookie as an argument, to gather the object data to upload into storage.

A read_from_data_source function should read up to buf_size bytes from the data source indicated by cookie into the buer at location buff and return the actual number of bytes read as the return value from the function.

There are two special return codes:

■

and other functions that store object data. The data source reader function will be

A return value of 0 indicates the end-of-le condition. The data should be committed to the data store.

A return value of -1 indicates a request to cancel the store. The store operation should be aborted with an error code of HCERR_ABORTED_BY_CALLBACK.

“hc_store_both_ez”on

Parameters

An opaque data structure to identify this data cookie. The cookie is likely to be an open le descriptor.

buf

Where to store the data.

buf_size

The number of available bytes of space in buf.

write_to_data_destination

Data destination template used to download object data to the cluster.

Synopsis

typedef long (*write_to_data_destination) (void *cookie, char *buff, long buff_len);

Sun StorageTek5800 SystemClient API Reference Manual • June 200850

Synchronous C DataTypes

Description

Function pointers of write_to_data_destination type are used to download object data to a network or other destination from the 5800 system server using The function pointer and opaque cookie reference are supplied as arguments to

“hc_retrieve_ez” on page 91, and the function will be called with the supplied cookie

argument to deliver the downloaded data to a local data storage function.

A write_to_data_destination function should write exactly buff_len bytes to the data destination indicated by cookie, reading the bytes from the buer at location buff. It should return a long value indicating the number of bytes actually processed. A return code that diers from buff_len indicates that the transfer should be terminated.

“hc_retrieve_ez” on page 91.

Parameters

cookie An opaque data structure to identify this data cookie. The cookie is likely to be

an open le descriptor.

buff Where to copy the data from.

buff_len The number of bytes of space in buff.

hcerr_t

5800 system C client API error codes.

To decode hcerr_t values into strings, see

Synopsis

typedef enum hcerr {

HCERR_OK = 0, HCERR_NOT_INITED, HCERR_ALREADY_INITED, HCERR_INIT_FAILED, HCERR_OOM, HCERR_NOT_YET_IMPLEMENTED, HCERR_SESSION_CREATE_FAILED, HCERR_ADD_HEADER_FAILED,HCERR_IO_ERR, HCERR_FAILOVER_OCCURRED, HCERR_CAN_CALL_AGAIN, HCERR_GET_RESPONSE_CODE_FAILED, HCERR_CONNECTION_FAILED,

Chapter 3 • Sun StorageTek 5800 System C Client API 51

“hc_decode_hcerr” on page 112

Synchronous C DataTypes

HCERR_BAD_REQUEST, HCERR_NO_SUCH_OBJECT, HCERR_INTERNAL_SERVER_ERROR, HCERR_FAILED_GETTING_FDSET, HCERR_FAILED_CHECKING_FDSET, HCERR_MISSING_SELECT_CLAUSE, HCERR_URL_TOO_LONG, HCERR_COULD_NOT_OPEN_FILE, HCERR_FAILED_TO_WRITE_TO_FILE, HCERR_NULL_SESSION, HCERR_INVALID_SESSION, HCERR_INVALID_OID, HCERR_NULL_HANDLE, HCERR_INVALID_HANDLE, HCERR_INVALID_SCHEMA, HCERR_INVALID_RESULT_SET, HCERR_INVALID_NVR, HCERR_WRONG_HANDLE_FOR_OPERATION, HCERR_HANDLE_IN_WRONG_STATE_FOR_OPERATION, HCERR_READ_PAST_LAST_RESULT, HCERR_XML_PARSE_ERROR, HCERR_XML_MALFORMED_XML, HCERR_XML_EXPECTED_LT, HCERR_XML_INVALID_ELEMENT_TAG, HCERR_XML_MALFORMED_START_ELEMENT, HCERR_XML_MALFORMED_END_ELEMENT, HCERR_XML_BAD_ATTRIBUTE_NAME, HCERR_XML_BUFFER_OVERFLOW, HCERR_BUFFER_OVERFLOW, HCERR_NO_SUCH_TYPE, HCERR_ILLEGAL_VALUE_FOR_METADATA, HCERR_NO_SUCH_ATTRIBUTE, HCERR_NO_MORE_ATTRIBUTES, HCERR_EOF,HCERR_FAILED_GETTING_SILO_DATA, HCERR_PLATFORM_NOT_INITED, HCERR_PLATFORM_ALREADY_INITED, HCERR_PLATFORM_INIT_FAILED, HCERR_PLATFORM_HEADER_TOO_LONG, HCERR_PLATFORM_TOO_LATE_FOR_HEADERS, HCERR_PLATFORM_NOT_ALLOWED_FOR_GET, HCERR_FAILED_TO_GET_SYSTEM_RECORD, HCERR_PARTIAL_FILE, HCERR_ABORTED_BY_CALLBACK, HCERR_PLATFORM_GENERAL_ERROR, HCERR_ILLEGAL_ARGUMENT

} hcerr_t;

Sun StorageTek5800 SystemClient API Reference Manual • June 200852

Description

This structure denes the 5800 system C client API error codes.

Synchronous C API Functions

The 5800 system synchronous C API functions are dened to perform the following tasks:

■

“Managing 5800 System Sessions” on page 53

■

“Managing a Schema” on page 59

■

“Manipulating Name-Value Records” on page 63

■

“Storing Data and Metadata” on page 87

■

“Retrieving Data and Metadata” on page 91

■

“Querying Metadata” on page 94

■

“Deleting Records” on page 111

■

“Translating Error and Type Codes” on page 112

Managing 5800 System Sessions

The following functions are used to manage 5800 system sessions:

■

“hc_session_create_ez” on page 53

■

“hc_session_free” on page 55

■

“hc_session_get_status” on page 55

■

“hc_session_get_schema” on page 56

■

“hc_session_get_host” on page 57

■

“hc_session_get_platform_result” on page 58

■

“hc_session_get_archive” on page 59

Synchronous C API Functions

hc_session_create_ez

Creates a session.

Synopsis

hcerr_t hc_session_create_ez(char *host,

int port, hc_session_t **sessionp);

Chapter 3 • Sun StorageTek 5800 System C Client API 53

Synchronous C API Functions

Description

This function initializes the 5800 system API and must be called before calling any of the other functions in this API. Itdownloads a copy of the schema for a particular host or port. The schema is used to validate the name-value-type tuples that are added to metadata records.

Both the synchronous and the nonsynchronous C APIs are fully thread-safe and can be used simultaneously in multiple threads from the same process. Each thread must call

“hc_session_create_ez” on page 53 to create its own session. Sessions must not be shared

between threads.

Note – hc_init should be called once per process before any thread calls

hc_session_create_ez.Ifhc_session_create_ez is called before hc_init, an implicit call is made to hc_init from that thread. But that call to hc_init is not interlocked with other threads, and it uses the C API shared library’s version of malloc and free, which might be dierent than the application’s version of malloc and free. Itis strongly recommended that all applications call hc_init once per process with their own allocator and deallocator.

For more information on hc_init, see

“Initializing a Global Session” on page 41

Parameters

host

IN: The name or IP address of a 5800 system server.

port

IN: The port number of the 5800 system server (normally 8080).

sessionp

OUT: Updated to point to a session object.

Return Codes

HCERR_OK HCERR_BAD_REQUEST HCERR_OOM HCERR_ILLEGAL_ARGUMENT

Sun StorageTek5800 SystemClient API Reference Manual • June 200854

Synchronous C API Functions

hc_session_free

Releases the session object.

Synopsis

hcerr_t hc_session_free (hc_session_t *session);

Description

This function releases the session object and recovers handles and memory for one connection.

Parameters

session

IN: The session object to free.

Return Codes

HCERR_OK HCERR_BAD_REQUEST HCERR_OOM HCERR_NULL_SESSION HCERR_INVALID_SESSION

hc_session_get_status

Gets the session status.

Synopsis

hcerr_t hc_session_get_status(hc_session_t *session,

int32_t *response_codep,char **errstrp);

Description

This function returns the HTTP response code and the error message string associated with the last request on this session.

Parameters

session

IN: The session object.

Chapter 3 • Sun StorageTek 5800 System C Client API 55

Synchronous C API Functions

response_codep

OUT: Updated to be the HTTP response code.

errstr

IN: Updated to be the error returned in the response body if the response code is not 200 (OK). errstr should not be written to by the application (that is, it is read only), and will persist until the next request to the 5800 system server or until whichever comes rst.

Return Codes

hc_session_get_schema

Gets schema object associated with this session.

“hc_session_free” on page 55 is called,

HCERR_OK HCERR_BAD_REQUEST HCERR_OOM HCERR_NULL_SESSION HCERR_INVALID_SESSION

Synopsis

hcerr_t hc_session_get_schema (hc_session_t *session,

hc_schema_t **schemap);

Description

This function returns the current schema object associated with this session.

Parameters

session

IN: The session object.

schemap

OUT: Updated to be the schema associated with the current session. schemap should not be modied by the application and will persist until the next call to

page 55, at which time it will be implicitly released.

Sun StorageTek5800 SystemClient API Reference Manual • June 200856

“hc_session_free”on

Synchronous C API Functions

Return Codes

HCERR_OK HCERR_BAD_REQUEST HCERR_OOM HCERR_NULL_SESSION HCERR_INVALID_SESSION

hc_session_get_host

Returns the host name and port number associated with the session.

Synopsis

hc_session_get_host(hc_session_t *session,

char **hostp,int *portp);

Description

This function returns the host name and port number associated with the session.

Parameters

session

IN: The session object.

hostp

OUT: Updated to point to host name (read-only string). hostp should not be modied by the application and will persist until the next call to

portp

OUT: Updated to be the port number.

“hc_session_free” on page 55.

Return Codes

HCERR_OK HCERR_BAD_REQUEST HCERR_OOM HCERR_NULL_SESSION HCERR_INVALID_SESSION

Chapter 3 • Sun StorageTek 5800 System C Client API 57

Synchronous C API Functions

hc_session_get_platform_result

Returns low-level error codes associated with the current session.

Synopsis

Description

This function returns low-level error codes associated with the current session.

Note – The values returned by hc_session_get_platform_result will not be updated properly

after calls to the functions hc_retrieve_ez and hc_delete_ez.

Parameters

session

IN: The session object.

hcerr_t hc_session_get_platform_result(hc_session_t *session,

int32_t *connect_errnop, int32_t *platform_resultp);

connect_errnop

OUT: Updated to be the operating system’s errno value associated with the last connect operation on the current session.

platform_resultp

OUT: Updated to be the error code reported by the underlying HTTP library (for example, the Curl library).

Return Codes

HCERR_OK HCERR_BAD_REQUEST HCERR_OOM HCERR_NULL_SESSION HCERR_INVALID_SESSION

Sun StorageTek5800 SystemClient API Reference Manual • June 200858

Synchronous C API Functions

hc_session_get_archive

Returns the current archive object associated with this session.

Synopsis

hcerr_t hc_session_get_archive(hc_session_t *session,

hc_archive_t **archivep);

Description

This function returns the current archive object associated with this session.

Note – The archive object is not needed for the synchronous C API, but is made available for

interfacing with the lower-level (nondocumented) API.

Parameters

session

IN: The session object.

archivep

OUT: Updated to be the archive object associated with the current session.

Return Codes

HCERR_OK HCERR_BAD_REQUEST HCERR_OOM HCERR_NULL_SESSION HCERR_INVALID_SESSION

Managing a Schema

When a session commences, the C client API downloads information about the metadata schema that is in use on the 5800 system server.

The following functions are used to manage a schema:

■

“hc_schema_get_type” on page 60

■

“hc_schema_get_length” on page 61

■

“hc_schema_get_count” on page 61

Chapter 3 • Sun StorageTek 5800 System C Client API 59

Synchronous C API Functions

■

hc_schema_get_type

Looks up type in schema.

Synopsis

Description

This function looks up the type associated with a given name in the current metadata schema, or returns an error if the name is not known.

Parameters

schema

IN: The schema to interrogate.

“hc_schema_get_type_at_index” on page 62

hcerr_t hc_schema_get_type(hc_schema_t *schema,

char *name, hc_type_t *typep);

name

IN: The attribute name to look up in the schema.

typep

OUT: Updated to be the type associated with that name in the schema.

Return Codes

HCERR_OK HCERR_BAD_REQUEST HCERR_OOM HCERR_INVALID_SCHEMA HCERR_ILLEGAL_ARGUMENT

Sun StorageTek5800 SystemClient API Reference Manual • June 200860

Synchronous C API Functions

hc_schema_get_length

Looks up length of char and string attribute elds.

Synopsis

hcerr_t hc_schema_get_length(hc_schema_t *schema,

char *name, int *length);

Description

This function looks up the length of a char or string eld associated with a given attribute name in the current metadata schema, or returns an error if the name is not known.

Parameters

schema

IN: The schema to interrogate.

name

IN: The attribute name to look up in the schema.

length

OUT: Updated to be the length of the eld associated with that name in the schema.

Return Codes

HCERR_OK HCERR_BAD_REQUEST HCERR_INVALID_SCHEMA HCERR_ILLEGAL_ARGUMENT

hc_schema_get_count

Returns the number of name-value pairs in the metadata schema.

Synopsis

hcerr_t hc_schema_get_count(hc_schema_t *hsp,

hc_long_t *countp);

Chapter 3 • Sun StorageTek 5800 System C Client API 61

Synchronous C API Functions

Description

This function returns the number of name-value pairs in the metadata schema.

Parameters

hsp

IN: The schema to interrogate.

countp

OUT: Updated with the number of name-value pairs in the schema.

Return Codes

hc_schema_get_type_at_index

Iterates through the name-value pairs in a schema.

Synopsis

hcerr_t hc_schema_get_type_at_index (hc_schema_t *hsp,

hc_long_t index,char **namep, hc_type_t *typep);

Description

This function provides a simple way to iterate through the name-value pairs in a schema.

Parameters

hsp

IN: The schema to query.

index

Sun StorageTek5800 SystemClient API Reference Manual • June 200862

Synchronous C API Functions

IN: Should range from 0 up to the count-1 returned in “hc_schema_get_count” on page 61.

namep

OUT: Updated to point to a string that is an attribute name of one attribute in the schema.

typep

OUT: Updated to be the type associated with that name in the schema. If the server schema references a type that the client library does not support, then the type is returned as HC_UNKNOWN_TYPE.

Return Codes

HCERR_OK HCERR_BAD_REQUEST HCERR_OOM HCERR_INVALID_SCHEMA HCERR_ILLEGAL_ARGUMENT

Manipulating Name-Value Records

5800 system synchronous C API functions are dened to perform the following name-value record manipulation tasks:

■

“Using the API for Storing Name-Value Records” on page 63

■

“Using Returned Name-Value Records” on page 64

■

“Creating and Freeing Name-Value Records” on page 65

■

“Building Name-Value Records” on page 66

■

“Retrieving Name-Value Records” on page 76

■

“Creating and Converting Name-Value Records From and To String Arrays” on page 84

Using the API for Storing Name-Value Records

A common way of storing metadata in the synchronous C API for the 5800 system is to use the name-value record API.

Chapter 3 • Sun StorageTek 5800 System C Client API 63

Synchronous C API Functions

▼

To Use the API for Storing Name-ValueRecords

Call hc_init once per process.

Call

“hc_session_create_ez”on page 53 to initialize the session and download the schema.

Create the metadata record with

Fill the new metadata piece by piece with hc_nvr_add_metadata_* functions (see

Name-Value Records”on page 66

Call either

“hc_store_metadata_ez”on page 88 or “hc_store_both_ez”on page 87 to store

“hc_nvr_create”on page 65.

“Building

) for each 5800 system type.

the new metadata record.

When you are done, free the metadata record by calling

When the session is nished, call

“hc_session_free”on page 55 to free the session data

“hc_nvr_free”on page 66.

structures.

When all threads are completed, call hc_cleanup to release the global session.

Using Returned Name-Value Records

Name-value records are also returned as the result of queries that return metadata information,

“hc_retrieve_metadata_ez” on page 92.

such as

▼

To Use Returned Name-Value Records

Run the query to create an “hc_nvr_t”on page 48 record or a table of “hc_nvr_t”on page 48

structures.

Use either name-based access (for example, hc_nvr_get_*) or index-based access (for example,

“hc_nvr_get_count” on page 76 and “hc_nvr_get_value_at_index” on page 77).

To free the

Note – Structures created by hc_nvr_create can also be freed by calling hc_nvr_free.

Sun StorageTek5800 SystemClient API Reference Manual • June 200864

“hc_nvr_t”on page 48 structure, call “hc_nvr_free” on page 66.

Synchronous C API Functions

Creating and Freeing Name-Value Records

The following functions are dened to create and free name-value records:

■

“hc_nvr_create” on page 65

■

“hc_nvr_free” on page 66

hc_nvr_create

Creates a name-value record.

Synopsis

hcerr_t hc_nvr_create(hc_session_t *session,

hc_long_t nslots, hc_nvr_t **nvrp);

Description

This function creates a name-value record with a designated initial size that is associated with a particular session. Metadata that is added to this name-value record must match the schema associated with the session.

Parameters

session

IN: The session with which this name-value record is associated.

nslots

IN: The number of slots for name-value-type tuples.

nvrp

OUT: Updated with a pointer to a new name-value record of the designated size.

Return Codes

HCERR_OK HCERR_ILLEGAL_ARGUMENT HCERR_OOM

Chapter 3 • Sun StorageTek 5800 System C Client API 65

Synchronous C API Functions

hc_nvr_free

Frees a name-value record.

Synopsis

Description

This function frees a name-value record that was created by “hc_nvr_create” on page 65.

Parameter

nvr

IN: Points to the name-value-record to be freed.

hcerr_t hc_nvr_free(hc_nvr_t *nvr);

Return Codes

HCERR_OK HCERR_BAD_REQUEST HCERR_OOM HCERR_INVALID_NVR

Building Name-Value Records

The following functions are dened to build name-value records:

■

“hc_nvr_add_value” on page 67

■

“hc_nvr_add_long” on page 68

■

“hc_nvr_add_double” on page 69

■

“hc_nvr_add_string” on page 70

■

“hc_nvr_add_binary” on page 71

■

“hc_nvr_add_date” on page 72

■

“hc_nvr_add_time” on page 73

Sun StorageTek5800 SystemClient API Reference Manual • June 200866

Synchronous C API Functions

■

“hc_nvr_add_timestamp” on page 74

■

“hc_nvr_add_from_string” on page 75

hc_nvr_add_value

Adds a new metadata value.

Synopsis

hcerr_t hc_nvr_add_value(hc_nvr_t *nvr,

char *name, hc_value_t value);

Description

This function adds a new metadata name-value-type tuple to a designated name-value record. The name-value record will automatically expand as needed.

Parameters

nvr

Points to a name-value-record.

name

IN: Name for the tuple.

value

IN: Value for the tuple, in the type-tagged

“hc_value_t” on page 47 format.

Return Codes

HCERR_OK HCERR_BAD_REQUEST HCERR_OOM HCERR_INVALID_NVR HCERR_ILLEGAL_ARGUMENT HCERR_ILLEGAL_VALUE_FOR_METADATA HCERR_NO_SUCH_ATTRIBUTE

Chapter 3 • Sun StorageTek 5800 System C Client API 67

Synchronous C API Functions

hc_nvr_add_long

Adds a new metadata value of type hc_long_t.

Synopsis

Description

This function adds a new metadata name-value-type tuple to a designated name-value record, where type is known to be hc_long_t (see will automatically expand as needed.

Parameters

nvr

Points to a name-value-record.

name

hcerr_t hc_nvr_add_long(hc_nvr_t *nvr,

char *name, hc_long_t value)

“hc_type_t” on page 47). The name-value record

IN: Name for the tuple.

value

IN: The hc_long_t value.

Return Codes

HCERR_OK HCERR_BAD_REQUEST HCERR_OOM HCERR_INVALID_NVR HCERR_ILLEGAL_ARGUMENT HCERR_ILLEGAL_VALUE_FOR_METADATA HCERR_NO_SUCH_ATTRIBUTE

Sun StorageTek5800 SystemClient API Reference Manual • June 200868

Synchronous C API Functions

hc_nvr_add_double

Adds a new metadata value of type hc_double_t.

Synopsis

hcerr_t hc_nvr_add_double(hc_nvr_t *nvr,

char *name, hc_double_t value);

Description

This function adds a new metadata name-value-type tuple to a designated name-value record, where type is known to be hc_double_t (see “hc_type_t” on page 47). The name-value record will automatically expand as needed.

Parameters

nvr

Points to a name-value-record.

name

IN: Name for the tuple.

value

IN: The hc_double_t value.

Return Codes

HCERR_OK HCERR_BAD_REQUEST HCERR_OOM HCERR_INVALID_NVR HCERR_ILLEGAL_ARGUMENT HCERR_ILLEGAL_VALUE_FOR_METADATA HCERR_NO_SUCH_ATTRIBUTE

Chapter 3 • Sun StorageTek 5800 System C Client API 69

Synchronous C API Functions

hc_nvr_add_string

Adds a new metadata value of type Unicode UTF-8 string.

Synopsis

Description

This function adds a new metadata name-value-type tuple to a designated name-value record, where type is a Unicode UTF-8 string. The name-value record automatically expands as needed. The string is copied into the structure.

Parameters

nvr

Points to a name-value-record.

name

hcerr_t hc_nvr_add_string(hc_nvr_t *nvr,

char *name, hc_string_t value);

IN: Name for the tuple.

value

IN: The hc_string_t value.

Return Codes

HCERR_OK HCERR_BAD_REQUEST HCERR_OOM HCERR_INVALID_NVR HCERR_ILLEGAL_ARGUMENT HCERR_ILLEGAL_VALUE_FOR_METADATA HCERR_NO_SUCH_ATTRIBUTE

Sun StorageTek5800 SystemClient API Reference Manual • June 200870

Synchronous C API Functions

hc_nvr_add_binary

Adds new metadata value of type binary.

Synopsis

hcerr_t hc_nvr_add_binary(hc_nvr_t *nvr,

hc_string_t name, int size, unsigned char *bytes);

Description

This function adds a new metadata name-value-type tuple to a designated name-value record, where type is binary data. The name-value record automatically expands as needed. The name and data are copied into the structure.

Parameters

nvr

Points to a name-value-record.

name

IN: Name for the tuple.

size

IN: The size of the data.

bytes

IN: The binary data.

Return Codes

HCERR_OK HCERR_BAD_REQUEST HCERR_OOM HCERR_INVALID_NVR HCERR_ILLEGAL_ARGUMENT HCERR_ILLEGAL_VALUE_FOR_METADATA HCERR_NO_SUCH_ATTRIBUTE

Chapter 3 • Sun StorageTek 5800 System C Client API 71

Synchronous C API Functions

hc_nvr_add_date

Adds new metadata value of type date.

Synopsis

Description

This function adds a new metadata name-value-type tuple to a designated name-value record.

The struct tm elds are as dened in the POSIX standard and interpreted by mktime(3C).All elds are ignored except:

The name-value record automatically expands as needed. The name and value are copied into the structure.

#include <time.h> hcerr_t hc_nvr_add_date(hc_nvr_t *nvr,

hc_string_t name,struct tm *value);

int tm_mday; /* day of the month - [1, 31] */ int tm_mon; /* months since January - [0, 11] */ int tm_year; /* years since 1900 */

Parameters

nvr

Points to a name-value-record.

name

IN: Name for the tuple.

value

IN: The struct tm (time.h) value.

Return Codes

HCERR_OK HCERR_BAD_REQUEST HCERR_OOM HCERR_INVALID_NVR HCERR_ILLEGAL_ARGUMENT HCERR_ILLEGAL_VALUE_FOR_METADATA HCERR_NO_SUCH_ATTRIBUTE

Sun StorageTek5800 SystemClient API Reference Manual • June 200872

Synchronous C API Functions

hc_nvr_add_time

Adds new metadata value of type time.

Synopsis

#include <time.h> hcerr_t hc_nvr_add_time(hc_nvr_t *nvr,

hc_string_t name, time_t *value);

Description

This function adds a new metadata name-value-type tuple to a designated name-value record. The value represents seconds since midnight.

The name-value record automatically expands as needed. The name and value are copied into the structure.

Parameters

nvr

Points to a name-value-record.

name

IN: Name for the tuple.

value

IN: The time_t (time.h) value.

Return Codes

HCERR_OK HCERR_BAD_REQUEST HCERR_OOM HCERR_INVALID_NVR HCERR_ILLEGAL_ARGUMENT HCERR_ILLEGAL_VALUE_FOR_METADATA HCERR_NO_SUCH_ATTRIBUTE

Chapter 3 • Sun StorageTek 5800 System C Client API 73

Synchronous C API Functions

hc_nvr_add_timestamp

Adds new metadata value of type timestamp.

Synopsis

Description

This function adds a new metadata name-value-type tuple to a designated name-value record, where type is hc_timestamp_t. The struct timespec is dened in the POSIX standard:

where tv_sec is measured since the UNIX epoch (00:00:00 UTC on January 1, 1970). The maximum value of tv_sec is truncated by three decimal digits owing to database limitations and tv_nsec is truncated to milliseconds. The name-value record automatically expands as needed. The name and value are copied into the structure.

#include <time.h> hcerr_t hc_nvr_add_timestamp(hc_nvr_t *nvr,

hc_string_t name, struct timespec *value);

time_t tv_sec; /* seconds */ long tv_nsec; /* and nanoseconds */

Parameters

nvr

Points to a name-value-record.

name

IN: Name for the tuple.

value

IN: The ’struct timespec’ (time.h) value.

Return Codes

HCERR_OK HCERR_BAD_REQUEST HCERR_OOM HCERR_INVALID_NVR HCERR_ILLEGAL_ARGUMENT HCERR_ILLEGAL_VALUE_FOR_METADATA HCERR_NO_SUCH_ATTRIBUTE

Sun StorageTek5800 SystemClient API Reference Manual • June 200874

Synchronous C API Functions

hc_nvr_add_from_string

Adds a new metadata value where the value always starts out as a string.

Synopsis

hcerr_t hc_nvr_add_from_string(hc_nvr_t *nvr,

char *name, char *value);

Description

This is a convenient function for adding a new metadata name-value-type tuple to a designated name-value, where the value always starts out as a string. The correct metadata type for name must be looked up from the schema. The name-value record will automatically expand as needed. The string is copied into the structure.

Parameters

nvr

Points to a name-value-record.

name

IN: Name for the tuple.

value

IN: The string value to be added.

Return Codes

HCERR_OK HCERR_BAD_REQUEST HCERR_OOM HCERR_INVALID_NVR HCERR_ILLEGAL_ARGUMENT

Chapter 3 • Sun StorageTek 5800 System C Client API 75

Synchronous C API Functions

Retrieving Name-ValueRecords

The following functions are dened to retrieve name-value records:

■

hc_nvr_get_count

Retrieves the number of metadata name and value tuples in this name-value record.

Synopsis

“hc_nvr_get_count” on page 76 “hc_nvr_get_value_at_index” on page 77 “hc_nvr_get_long” on page 78 “hc_nvr_get_double” on page 79 “hc_nvr_get_string” on page 80 “hc_nvr_get_binary” on page 81 “hc_nvr_get_date” on page 82 “hc_nvr_get_time” on page 82 “hc_nvr_get_timestamp” on page 83

hcerr_t hc_nvr_get_count(hc_nvr_t *nvr,

hc_long_t *retcount);

Description

This function retrieves the number of metadata name and value tuples in the specied name-value record.

Parameters

nvr

IN: Points to a name-value-record.

retcount

OUT: Updated to contain the count of name-value pairs.

Return Codes

HCERR_OK HCERR_BAD_REQUEST HCERR_OOM HCERR_INVALID_NVR

Sun StorageTek5800 SystemClient API Reference Manual • June 200876

Synchronous C API Functions

hc_nvr_get_value_at_index

Iterates through the names and values in a name-value record.

Synopsis

hc_nvr_get_value_at_index(hc_nvr_t *nvr,

hc_long_t index, char **namep, hc_value_t *valuep);

Description

This function iterates through the names and values in a name-value record. The returned names are read-only. Unpredictable results will occur if either the name or the value is referenced after either

on page 84 is called on this name-value record.

Parameters

nvr

Points to a name-value-record.

“hc_nvr_free” on page 66 or “hc_nvr_create_from_string_arrays”

index

IN: The index to examine.

namep

OUT: Updated to point to the attribute name at the specied index.

valuep

OUT: Updated with the

“hc_value_t” on page 47 type-tagged value at the specied index.

Return Codes

HCERR_OK HCERR_INVALID_NVR HCERR_ILLEGAL_ARGUMENT HCERR_NO_MORE_ARGUMENTS HCERR_OOM

Chapter 3 • Sun StorageTek 5800 System C Client API 77

Synchronous C API Functions

hc_nvr_get_long

Retrieves a value of type hc_long_t.

Synopsis

Description

This function retrieves the value of type hc_long_t (see “hc_type_t” on page 47) associated with an indicated attribute name in a name-value record.

Parameters

nvr

Points to a name-value-record.

name

IN: Attribute name to look for.

hcerr_t hc_nvr_get_long(hc_nvr_t *nvr,

char *name, hc_long_t *retlong);

retlong

OUT: Updated to contain the hc_long_t value.

Return Codes

HCERR_OK HCERR_BAD_REQUEST HCERR_OOM HCERR_INVALID_NVR HCERR_ILLEGAL_VALUE_FOR_METADATA HCERR_ILLEGAL_ARGUMENT

Sun StorageTek5800 SystemClient API Reference Manual • June 200878

Synchronous C API Functions

hc_nvr_get_double

Retrieves a value of type hc_double_t.

Synopsis

hcerr_t hc_nvr_get_double(hc_nvr_t *nvr,

char *name, hc_double_t *retdouble);

Description

This function retrieves the value of type hc_double_t (see “hc_type_t” on page 47) associated with an indicated attribute name in a name-value record.

Parameters

nvr

Points to a name-value-record.

name

IN: Attribute name to look for.

retdouble

OUT: Updated to contain the hc_double_t value.

Return Codes

HCERR_OK HCERR_BAD_REQUEST HCERR_OOM HCERR_INVALID_NVR HCERR_ILLEGAL_VALUE_FOR_METADATA HCERR_ILLEGAL_ARGUMENT

Chapter 3 • Sun StorageTek 5800 System C Client API 79

Synchronous C API Functions

hc_nvr_get_string

Retrieves a value of a Unicode UTF-8 string.

Synopsis

Description

This function retrieves the value of a Unicode UTF-8 string associated with an indicated attribute name in a name-value record. Note that the memory pointed to will be freed when the record is freed.

Parameters

nvr

Points to a name-value-record.

name

hcerr_t hc_nvr_get_string(hc_nvr_t *nvr,

char *name, hc_string_t *retstring);

IN: Attribute name to look for.

retstring

OUT: Updated to contain the hc_string_t value.

Return Codes

HCERR_OK HCERR_BAD_REQUEST HCERR_OOM HCERR_INVALID_NVR HCERR_ILLEGAL_VALUE_FOR_METADATA HCERR_ILLEGAL_ARGUMENT

Sun StorageTek5800 SystemClient API Reference Manual • June 200880

Synchronous C API Functions

hc_nvr_get_binary

Retrieves a metadata value of type binary.

Synopsis

hcerr_t hc_nvr_get_binary(hc_nvr_t *nvr,

hc_string_t name, int *size, unsigned char **bytes);

This function retrieves the value of type binary associated with an indicated attribute name in a name-value record. The binary data is not copied and is freed when the name-value record is freed.

Parameters

nvr

Points to a name-value-record.

name

IN: Name for the tuple.

size

OUT: Updated with the size of the data.

bytes

OUT: Updated to point to the binary data.

Return Codes

HCERR_OK HCERR_BAD_REQUEST HCERR_OOM HCERR_INVALID_NVR HCERR_ILLEGAL_ARGUMENT HCERR_ILLEGAL_VALUE_FOR_METADATA HCERR_NO_SUCH_ATTRIBUTE

Chapter 3 • Sun StorageTek 5800 System C Client API 81

Synchronous C API Functions

hc_nvr_get_date

Retrieves metadata value of type date.

Synopsis

Description

This function retrieves the value of type struct tm associated with an indicated attribute name in a name-value record.

Parameters

nvr

Points to a name-value-record.

name

#include <time.h> hcerr_t hc_nvr_get_date(hc_nvr_t *nvr,

hc_string_t name, struct tm *value);

IN: Name for the tuple.

value

OUT: Updated with the struct tm (time.h) value.

Return Codes

HCERR_OK HCERR_BAD_REQUEST HCERR_OOM HCERR_INVALID_NVR HCERR_ILLEGAL_ARGUMENT HCERR_ILLEGAL_VALUE_FOR_METADATA HCERR_NO_SUCH_ATTRIBUTE

hc_nvr_get_time

Retrieves metadata value of type time.

Sun StorageTek5800 SystemClient API Reference Manual • June 200882

Synchronous C API Functions

Synopsis

#include <time.h> hcerr_t hc_nvr_get_time(hc_nvr_t *nvr,

hc_string_t name, time_t *value);

This function retrieves the value of type time_t (seconds since midnight) associated with an indicated attribute name in a name-value record.

Parameters

nvr

Points to a name-value-record.

name

IN: Name for the tuple.

value

OUT: Updated with the time_t (time.h) value.

Return Codes

HCERR_OK HCERR_BAD_REQUEST HCERR_OOM HCERR_INVALID_NVR HCERR_ILLEGAL_ARGUMENT HCERR_ILLEGAL_VALUE_FOR_METADATA HCERR_NO_SUCH_ATTRIBUTE

hc_nvr_get_timestamp

Retrieves metadata value of type timestamp.

Synopsis

#include <time.h> hcerr_t hc_nvr_get_timestamp(hc_nvr_t *nvr,

hc_string_t name, struct timespec *value);

Chapter 3 • Sun StorageTek 5800 System C Client API 83

Synchronous C API Functions

This function retrieves the value of type struct timespec associated with an indicated attribute name in a name-value record.

Parameters

nvr

Points to a name-value-record.

name

IN: Name for the tuple.

value

OUT: Updated with the struct timespec (time.h) value.

Return Codes

HCERR_OK HCERR_BAD_REQUEST HCERR_OOM HCERR_INVALID_NVR HCERR_ILLEGAL_ARGUMENT HCERR_ILLEGAL_VALUE_FOR_METADATA HCERR_NO_SUCH_ATTRIBUTE

Creating and Converting Name-Value Records From and To String Arrays

The following functions are dened to create name-value records from string arrays and convert name-value records to string arrays:

■

“hc_nvr_create_from_string_arrays” on page 84

■

“hc_nvr_convert_to_string_arrays” on page 86

hc_nvr_create_from_string_arrays

Creates name-value-record from string names and string values.

Synopsis

hcerr_t hc_nvr_create_from_string_arrays(hc_session_t *session,

hc_nvr_t **nvrp,

Sun StorageTek5800 SystemClient API Reference Manual • June 200884

Synchronous C API Functions

char **names, char **values, hc_long_t nitems);

Description

This function creates a name-value-record from parallel tables of string names and string values. The correct metadata type for each name must be looked up from the schema associated with this session. The name-value record will automatically expand as needed. The names and data values are copied into the

“hc_nvr_t” on page 48 structure, so the original names table and

values table are left unchanged.

Note – Any time there is a conversion from a double type to or from a string type, there might

be a loss of precision.

Parameters

session

IN: The session with which this name-value record is associated.

nvrp

OUT: Updated to point to a name-value-record.

names

IN: Points to an array of string names.

values

IN: Points to an array of string values.

nitems

IN: Number of active elements in the paired arrays.

Return Codes

HCERR_OK HCERR_BAD_REQUEST HCERR_OOM HCERR_NULL_SESSION HCERR_INVALID_SESSION HCERR_ILLEGAL_ARGUMENT

Chapter 3 • Sun StorageTek 5800 System C Client API 85

Synchronous C API Functions

hc_nvr_convert_to_string_arrays

Converts name-value-record to string names and string values.

Synopsis

Description

This function converts a name-value-record into parallel tables of string names and string values. This destructively modies the name-value record and frees it, so do not call

“hc_nvr_free” on page 66 after calling this function.

When the conversion is nished, each string in the names and values tables should be freed with the designated deallocator (for example, free), as well as the names and values tables themselves.

Note – Any time there is a conversion from a double type to or from a string type, there might

be a loss of precision.

hcerr_t hc_nvr_convert_to_string_arrays(hc_nvr_t *nvr,

char ***namesp, char ***valuesp, int *nitemsp);

Parameters

nvr

IN: The name-value-record.

namesp

OUT: Updated to point to an array of string names.

valuesp

OUT: Updated to point to an array of string values.

nitemsp

OUT: Updated to the number of active elements in the paired arrays.

Return Codes

HCERR_OK HCERR_BAD_REQUEST

Sun StorageTek5800 SystemClient API Reference Manual • June 200886

Synchronous C API Functions

HCERR_OOM HCERR_INVALID_NVR HCERR_ILLEGAL_ARGUMENT

Storing Data and Metadata

The following functions are dened to store data and metadata and to enforce indexing of metadata where necessary:

■

“hc_store_both_ez” on page 87

■

“hc_store_metadata_ez” on page 88

■

“hc_check_indexed_ez” on page 89

Note – The is_indexed value in the returned system record (hc_system_record_t) indicates

whether this record was successfully inserted in the query engine at the time of store, and is hence available for query. Objects that were not inserted into the query engine at time of store are called store index exceptions. Until they are resolved, store index exceptions may or may not show up in the result sets of queries that match the store. A store index exception is resolved once the metadata for that object has been successfully inserted into the query engine, after which the object will denitely show up in the result sets of queries that match the store. The

“hc_check_indexed_ez” on page 89 method may be used to attempt to resolve a store index

exception under program control. Store index exceptions will also be resolved automatically (eventually) by ongoing system healing.

hc_store_both_ez

Stores a metadata record and associated data.

Synopsis

hcerr_t hc_store_both_ez (hc_session_t *session,

*data_source_reader, void *cookie, hc_nvr_t *nvr, hc_system_record_t *system_record);

Description

This function stores both object data and metadata and returns a system_record descriptor. The status from this operation can be reclaimed using “hc_session_get_status” on page 55.

Chapter 3 • Sun StorageTek 5800 System C Client API 87

Synchronous C API Functions

Parameters

session

IN: The session for the host and port to talk to.

data_source_reader

IN: The source of data to be stored. See

“read_from_data_source” on page 49.

IN: An opaque data structure (cookie) to be provided to data_source_reader. For example, this could be a le descriptor.

nvr

IN: Pointer to a name-value record with the metadata.

system_record

OUT: Returned descriptor of a stored metadata record.

Return Codes

HCERR_OK HCERR_BAD_REQUEST HCERR_OOM HCERR_NULL_SESSION HCERR_INVALID_SESSION HCERR_INVALID_NVR HCERR_ILLEGAL_ARGUMENT HCERR_NO_SUCH_TYPE HCERR_XML_BUFFER_OVERFLOW

hc_store_metadata_ez

Adds a metadata record for the specied OID.

Synopsis

hcerr_t hc_store_metadata_ez(hc_session_t *session,

hc_oid *oid, hc_nvr_t *nvr, hc_system_record_t *system_record);

Sun StorageTek5800 SystemClient API Reference Manual • June 200888

Synchronous C API Functions

Description

This function adds a metadata record for the specied OID and returns a system_record descriptor.

Parameters

session

IN: The session for the host and port to talk to.

oid

IN: An identier of object data to which the metadata record is attached.

nvr

IN: Pointer to a name-value record with the metadata.

system_record

OUT: Returned descriptor of a stored metadata record.

Return Codes

HCERR_OK HCERR_BAD_REQUEST HCERR_OOM HCERR_NULL_SESSION HCERR_INVALID_SESSION HCERR_INVALID_NVR HCERR_INVALID_OID HCERR_ILLEGAL_ARGUMENT HCERR_XML_BUFFER_OVERFLOW

hc_check_indexed_ez

Checks if the metadata for an object is present in the query engine, and inserts it if not.

Synopsis

hcerr_t hc_check_indexed_ez(hc_session_t *session,

hc_oid *oid, int *resultp);

Chapter 3 • Sun StorageTek 5800 System C Client API 89

Synchronous C API Functions

Description

checkIndexed is intended as way to resolve a store index exception under program control (see

“The 5800 System Query Integrity Model” on page 21). Once a store index exception occurs (as

indicated by a non-zero value of the is_indexed eld in the hc_system_record_t returned from a store operation) then hc_check_indexed_ez can be called repeatedly until it returns with *resultp set to any non-zero value. This will ensure that the metadata for the object has been inserted into the query engine; the object should then start to show up in matching queries.

Parameters

session

IN: The session for the host and port to talk to.

oid

IN: An identier of object data to which the metadata record is attached.

resultp

OUT: Points to an int that is updated to a value that indicates if the metadata for this object has been inserted into the query engine. The returned value of *resultp is set to -1 if the object was already present in the query engine, and is set to 0 if the object was not already in the query engine and could not be added, and to 1 if the object was just now added to the query engine. In other words, a non-zero value of resultp indicates that the store index exception has been resolved.

Return Codes

HCERR_OK HCERR_BAD_REQUEST HCERR_OOM HCERR_NULL_SESSION HCERR_INVALID_SESSION HCERR_INVALID_OID

Sun StorageTek5800 SystemClient API Reference Manual • June 200890

Retrieving Data and Metadata

The following functions are dened to retrieve data and metadata:

■

“hc_retrieve_ez” on page 91

■

“hc_retrieve_metadata_ez” on page 92

■

“hc_range_retrieve_ez” on page 93

hc_retrieve_ez

Retrieves data for the specied OID.

Synopsis

hcerr_t hc_retrieve_ez(hc_session_t *session,

*data_writer, void *cookie, hc_oid *oid);

Description

This function retrieves data for the specied OID.

Synchronous C API Functions

Parameters

session

IN: The session for the host and port to talk to.

data_writer

IN: A function callback to store the retrieved data locally. See

on page 50.

IN: The opaque data delivered to the data_writer callback to identify this data stream.

oid

IN: Identier for the metadata record to retrieve.

“write_to_data_destination”

Return Codes

HCERR_OK HCERR_BAD_REQUEST

Chapter 3 • Sun StorageTek 5800 System C Client API 91

Synchronous C API Functions

hc_retrieve_metadata_ez

Retrieves a metadata record for the specied OID.

Synopsis

Description

This function retrieves a metadata record for the specied OID. When it has nished, you should call

Parameters

session

HCERR_OOM HCERR_NULL_SESSION HCERR_INVALID_SESSION HCERR_INVALID_OID

hcerr_t hc_retrieve_metadata_ez (hc_session_t *session,

hc_oid *oid, hv_nvr_t **nvrp);

“hc_nvr_free” on page 66 to free the name-value-record.

IN: The session for the host and port to talk to.

oid

IN: An identier of the metadata record to retrieve.

nvrp

OUT: Updated with a pointer to a dynamically allocated name-value record with the metadata.

Return Codes

HCERR_OK HCERR_BAD_REQUEST HCERR_OOM HCERR_NULL_SESSION HCERR_INVALID_SESSION HCERR_INVALID_OID

Sun StorageTek5800 SystemClient API Reference Manual • June 200892

hc_range_retrieve_ez

Retrieves a specied range of data for a specied OID.

Synopsis

hc_range_retrieve_ez(hc_session_t *session,

write_to_data_destination data_writer, void *cookie, hc_oid *oid, hc_long_t; firstbyte, hc_long_t lastbyte);

Description

This function retrieves a specied range of data for a specied OID.

Parameters

session

IN: The session for the host and port to talk to.

data_writer

Synchronous C API Functions

IN: Function callback to store the retrieved data locally.

IN: Opaque data delivered to the data_writer callback to identify this data cookie.

oid

IN: An identier of the data record to retrieve.

firstbyte

IN: First byte of data range to retrieve.

lastbyte

IN: Last byte of data range to retrieve, or -1 for the end of the record.

Return Codes

HCERR_OK HCERR_BAD_REQUEST HCERR_OOM

Chapter 3 • Sun StorageTek 5800 System C Client API 93

Synchronous C API Functions

Querying Metadata

The following functions are dened for simple queries:

■

The following functions are dened for prepared statement queries:

■

HCERR_NULL_SESSION HCERR_INVALID_SESSION HCERR_INVALID_OID HCERR_ILLEGAL_ARGUMENT

“hc_query_ez” on page 94 “hc_qrs_next_ez” on page 96 “hc_qrs_is_query_complete” on page 97 “hc_qrs_get_query_integrity_time” on page 98 “hc_qrs_free” on page 99

“hc_pstmt_create” on page 100 “hc_pstmt_free” on page 101 “hc_pstmt_set_string” on page 101 “hc_pstmt_set_char” on page 102 “hc_pstmt_set_double” on page 103 “hc_pstmt_set_long” on page 104 “hc_pstmt_set_date” on page 105 “hc_pstmt_set_time” on page 106 “hc_pstmt_set_timestamp” on page 107 “hc_pstmt_set_binary” on page 108 “hc_pstmt_query_ez” on page 109

Prepared statement example:

“Querying With a Prepared Statement” on page 110

hc_query_ez

Retrieves OIDs and optionally name-value records matching a query.

Synopsis

hcerr_t hc_query_ez(hc_session_t *session,

hc_string_t query, hc_string_t selects[], int n_selects,

Sun StorageTek5800 SystemClient API Reference Manual • June 200894

Synchronous C API Functions

int results_per_fetch, hc_query_result_set_t **rsetp);

Description

This function retrieves OIDs and optionally name-value records matching a query. If the selects list is NULL, only OIDs are retrieved. If selects is not NULL, name-value records are also retrieved and should each be freed using set should be freed using

Note – When a query is incorrect and elicits an error from the server, the error is often reported

“hc_qrs_free” on page 99and not from hc_query_ez. Your application should be

after

“hc_qrs_free” on page 99.

“hc_nvr_free” on page 66. In both cases the result

prepared to receive and report an error from either place.

Parameters

session

IN: The session for the host and port to talk to.

query

IN: Query (where clause with names in single quotes).

selects

IN: Points to an array of

“hc_string_t” on page 46, each member of which is the name of a eld

to retrieve from the metadata (select clause). Set to NULL to only retrieve OIDs matching the query.

n_selects

IN: The number of items in the select clause.

results_per_fetch

IN: The number of results to return on each fetch from the server. results_per_fetch must be greater than 0.

rsetp

OUT: Updated to point to the new result set. See

Chapter 3 • Sun StorageTek 5800 System C Client API 95

“hc_query_result_set_t” on page 49.

Synchronous C API Functions

Return Codes

hc_qrs_next_ez

Fetches the next OID and optionally name-value record from the QueryResultSet.

Synopsis

HCERR_OK HCERR_OOM HCERR_BAD_REQUEST HCERR_NULL_SESSION HCERR_INVALID_SESSION HCERR_ILLEGAL_ARGUMENT

hcerr_t hc_qrs_next_ez(**rset,

hc_oid *oid, hc_nvr_t **nvrp, int *finishedp);

Description

This function fetches an OID and optionally name-value record from the query result set. Once the last result is fetched, in subsequent calls the int pointed to by finishedp is set to 1.

Parameters

rset

IN: Current query result set. See “hc_query_result_set_t” on page 49.

oid

OUT: Points to an OID that is updated to the OID of a record matching the query, assuming

finishedp is 0.

nvrp

OUT: Updated to point to a name-value record with the metadata from the OID matching the query, assuming the query specied selects and assuming finishedp is 0. Note that you must free the name-value record using

Sun StorageTek5800 SystemClient API Reference Manual • June 200896

“hc_nvr_free” on page 66.

Synchronous C API Functions

finishedp

OUT: Points to an int that is updated to 0 if query data has been returned and to 1 if the result set is empty.

Return Codes

HCERR_OK HCERR_OOM HCERR_BAD_REQUEST HCERR_INVALID_RESULT_SET HCERR_ILLEGAL_ARGUMENT

hc_qrs_is_query_complete

Indicates whether results of this query are complete in the sense that all objects that match the query, aside from possible store index exceptions, are included in the result set,

Synopsis

hcerr_t hc_qrs_is_query_complete(hc_query_result_set_t *rset,

int *completep);

Description

Indicates whether results of this query are complete in the sense that all objects that match the query, aside from possible store index exceptions, are included in the result set. Applications that depend on completeness of query results can interrogate hc_qrs_is_query_complete after retrieving all the query results that match a particular query. When completep is set to 1, the only items that should be missing from the result set are store index exceptions that were indicated to the application by a value of 0 in the is_indexed eld of the hc_system_record_t structure returned from the store.

Parameters

rset

IN: Current query result set. See

completep

OUT: Points to an int that is updated to 1 if all objects that match the query (other than potential store index exceptions) should be present in the result set

Chapter 3 • Sun StorageTek 5800 System C Client API 97

“hc_query_result_set_t” on page 49.

Synchronous C API Functions

Return Codes

hc_qrs_get_query_integrity_time

Returns a time that helps get more detail on which store index exceptions might still be unresolved.

Synopsis

Description

If the query integrity time is non-zero, then all store index exceptions whose object creation time falls before the query integrity time have been resolved. Stored objects from before that time should show up in all matching query result sets. Store index exceptions that occurred after that time may not yet have been resolved, and hence might still be missing from a matching query result set. If the Query Integrity Time is zero, then the set of results in this ResultSet is not known to be complete. Note that hc_is_query_complete will return a non-zero completep value if and only if hc_get_query_integrity_time would set

query_timep to non-zero query integrity time.

HCERR_OK HCERR_BAD_REQUEST HCERR_OOM HCERR_INVALID_RESULT_SET

hcerr_t hc_qrs_get_query_integrity_time(hc_query_result_set_t *rset,

hc_long_t *query_timep);

Time values from getQueryIntegrityTime can be compared to object creation time values returned in the creation_time eld of the hc_system_record_t structure to determine if a particular store operation has been resolved. Note: the query integrity time as reported may well be earlier than the actual oldest time of a still-unresolved store index exception. The query integrity time can even go backwards, in other words, a later query can report an earlier query integrity time.

Parameter

rset

Updated to point to the new query result set. See

“hc_query_result_set_t” on page 49.

Return Codes

HCERR_OK HCERR_BAD_REQUEST

Sun StorageTek5800 SystemClient API Reference Manual • June 200898

Synchronous C API Functions

HCERR_OOM HCERR_INVALID_RESULT_SET

hc_qrs_free

Releases the resources associated with this QueryResultSet.

Synopsis

hcerr_t hc_qrs_free (**rsetp);

Description

This function releases the resources associated with this QueryResultSet.

Note – When a query is incorrect and elicits an error from the server, the error is often reported

after hc_qrs_free and not from “hc_query_ez” on page 94. Your application should be prepared to receive and report an error from either place.

Parameter

rset

Return Codes

HCERR_OK HCERR_BAD_REQUEST HCERR_OOM HCERR_INVALID_RESULT_SET

hc_pstmt_create

Creates an “hc_pstmt_t” on page 49 for use with the “hc_pstmt_query_ez” on page 109 function.

Synopsis

Description

This function creates a prepared statement for use with the “hc_pstmt_query_ez” on page 109 function.

Parameters

session

IN: session that this query will be used with.

query

hcerr_t hc_pstmt_create(hc_session_t *session,

hc_string_t query, hc_pstmt_t **ptr);

IN: Query (where clause with ”?’ for values).

ptr

OUT: Updated to point to opaque

“hc_pstmt_t” on page 49.

Return Codes

HCERR_OK HCERR_OOM

Sun Microsystems 5800 User Manual

Specifications and Main Features

Frequently Asked Questions

User Manual

Sun StorageTek 5800 System Client API Reference Manual

Preface

HowThis Book Is Organized

Related Books

RelatedThird-Party Web Site References

Documentation, Support, and Training

Typographic Conventions

Shell Prompts in Command Examples

Sun WelcomesYour Comments

Sun StorageTek 5800 System Client API

Changes in Version 1.1

5800 System Overview

5800 SystemSummary

The 5800 System and Honeycomb

The 5800 System Data Model

The 5800 System Metadata Model

The 5800 System Query Model

The 5800 System Query Integrity Model

Deleting Objects from the 5800 System

Sun StorageTek 5800 System Java Client API

Overview of the 5800 System Java Client API

Client Library

Interfaces

Retrying Operations

Performance and Scalability

Updating ClientView of the Schema

Java Client Application Deployment

Java API

Java API Packages

Java API Documentation

Basic Concepts

Key Classes

NameValueObjectArchive

NameValueSchema

ObjectIdentifier

QueryResultSet

SystemRecord

NameValueRecord

NameValueObjectArchive Application Access

NameValueObjectArchive

Synopsis

Description

delete

Synopsis

Description

storeObject

Synopsis

Description

storeMetadata

Synopsis

Description

checkIndexed

Description

retrieveObject

Synopsis

Description

retrieveMetadata

Synopsis

Description

getSchema

Synopsis

Description

query

Synopsis

Description

query (with selectKeys)

Synopsis

Description

query (with PreparedStatement)

Synopsis

Description

query (with PreparedStatement and selectKeys)

Synopsis

Description

PreparedStatement

Synopsis