Pinnacle Systems Media Index - 2.6 User Manual

Avid Media | Index

Configuration Guide

Version 2.6

Legal Notices

Product specifications are subject to change without notice and do not represent a commitment on the part of Avid Technology, Inc.

This product is subject to the terms and conditions of a software license agreement provided with the software. The product may only be used in accordance with the license agreement.

This product may be protected by one or more U.S. and non-U.S patents. Details are available at www.avid.com/patents.

This document is protected under copyright law. An authorized licensee of [product name] may reproduce this publication for the licensee’s own use in learning how to use the software. This document may not be reproduced or distributed, in whole or in part, for commercial purposes, such as selling copies of this document or providing support or educational services to others. This document is supplied as a guide for Media | Index. Reasonable care has been taken in preparing the information it contains. However, this document may contain omissions, technical inaccuracies, or typographical errors. Avid Technology, Inc. does not accept responsibility of any kind for customers’ losses due to the use of this document. Product specifications are subject to change without notice.

The following disclaimer is required by Apple Computer, Inc.:

APPLE COMPUTER, INC. MAKES NO WARRANTIES WHATSOEVER, EITHER EXPRESS OR IMPLIED, REGARDING THIS PRODUCT, INCLUDING WARRANTIES WITH RESPECT TO ITS MERCHANTABILITY OR ITS FITNESS FOR ANY PARTICULAR PURPOSE. THE EXCLUSION OF IMPLIED WARRANTIES IS NOT PERMITTED BY SOME STATES. THE ABOVE EXCLUSION MAY NOT APPLY TO YOU. THIS WARRANTY PROVIDES YOU WITH SPECIFIC LEGAL RIGHTS. THERE MAY BE OTHER RIGHTS THAT YOU MAY HAVE WHICH VARY FROM STATE TO STATE.

The following disclaimer is required by Sam Leffler and Silicon Graphics, Inc. for the use of their TIFF library:

Permission to use, copy, modify, distribute, and sell this software [i.e., the TIFF library] and its documentation for any purpose is hereby granted without fee, provided that (i) the above copyright notices and this permission notice appear in all copies of the software and related documentation, and (ii) the names of Sam Leffler and Silicon Graphics may not be used in any advertising or publicity relating to the software without the specific, prior written permission of Sam Leffler and Silicon Graphics.

THE SOFTWARE IS PROVIDED “AS-IS” AND WITHOUT WARRANTY OF ANY KIND, EXPRESS, IMPLIED OR OTHERWISE, INCLUDING WITHOUT LIMITATION, ANY WARRANTY OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE.

IN NO EVENT SHALL SAM LEFFLER OR SILICON GRAPHICS BE LIABLE FOR ANY SPECIAL, INCIDENTAL, INDIRECT OR CONSEQUENTIAL DAMAGES OF ANY KIND, OR ANY DAMAGES WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER OR NOT ADVISED OF THE POSSIBILITY OF DAMAGE, AND ON ANY THEORY OF LIABILITY, ARISING OUT OF OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.

The following disclaimer is required by the Independent JPEG Group:

This software is based in part on the work of the Independent JPEG Group.

This Software may contain components licensed under the following conditions:

Redistribution and use in source and binary forms are permitted provided that the above copyright notice and this paragraph are duplicated in all such forms and that any documentation, advertising materials, and other materials related to such distribution and use acknowledge that the software was developed by the University of California, Berkeley. The name of the University may not be used to endorse or promote products derived from this software without specific prior written permission. THIS SOFTWARE IS PROVIDED ``AS IS'' AND WITHOUT ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, WITHOUT LIMITATION, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE.

Permission to use, copy, modify, and distribute this software and its documentation for any purpose and without fee is hereby granted, provided that the above copyright notice appear in all copies and that both that copyright notice and this permission notice appear in supporting documentation. This software is provided "as is" without express or implied warranty.

Permission to use, copy, modify, distribute, and sell this software for any purpose is hereby granted without fee, provided that the above copyright notice appear in all copies and that both that copyright notice and this permission notice appear in supporting documentation, and that the name of Daniel Dardailler not be used in advertising or publicity pertaining to distribution of the software without specific, written prior permission. Daniel Dardailler makes no representations about the suitability of this software for any purpose. It is provided "as is" without express or implied warranty.

Permission to use, copy, modify, and distribute this software for any purpose without fee is hereby granted, provided that this entire notice is included in all copies of any software which is or includes a copy or modification of this software and in all copies of the supporting documentation for such software.

THIS SOFTWARE IS BEING PROVIDED "AS IS", WITHOUT ANY EXPRESS OR IMPLIED WARRANTY. IN PARTICULAR, NEITHER THE AUTHOR NOR AT&T MAKES ANY REPRESENTATION OR WARRANTY OF ANY KIND CONCERNING THE MERCHANTABILITY OF THIS SOFTWARE OR ITS FITNESS FOR ANY PARTICULAR PURPOSE.

This product includes software developed by the University of California, Berkeley and its contributors.

The following disclaimer is required by Paradigm Matrix:

Portions of this software licensed from Paradigm Matrix.

The following disclaimer is required by Ray Sauers Associates, Inc.:

“Install-It” is licensed from Ray Sauers Associates, Inc. End-User is prohibited from taking any action to derive a source code equivalent of “Install-It,” including by reverse assembly or reverse compilation, Ray Sauers Associates, Inc. shall in no event be liable for any damages resulting from reseller’s failure to perform reseller’s obligation; or any damages arising from use or operation of reseller’s products or the software; or any other damages, including but not limited to, incidental, direct, indirect, special or consequential Damages including lost profits, or damages resulting from loss of use or inability to use reseller’s products or the software for any reason including copyright or patent infringement, or lost data, even if Ray Sauers Associates has been advised, knew or should have known of the possibility of such damages.

The following disclaimer is required by Videomedia, Inc.:

“Videomedia, Inc. makes no warranties whatsoever, either express or implied, regarding this product, including warranties with respect to its merchantability or its fitness for any particular purpose.”

“This software contains V-LAN ver. 3.0 Command Protocols which communicate with V-LAN ver. 3.0 products developed by Videomedia, Inc. and V-LAN ver. 3.0 compatible products developed by third parties under license from Videomedia, Inc. Use of this software will allow “frame accurate” editing control of applicable videotape recorder decks, videodisc recorders/players and the like.”

The following disclaimer is required by Altura Software, Inc. for the use of its Mac2Win software and Sample Source Code:

The following disclaimer is required by 3Prong.com Inc.:

Certain waveform and vector monitoring capabilities are provided under a license from 3Prong.com Inc.

The following disclaimer is required by Interplay Entertainment Corp.:

The “Interplay” name is used with the permission of Interplay Entertainment Corp., which bears no responsibility for Avid products.

This product includes portions of the Alloy Look & Feel software from Incors GmbH.

This product includes software developed by the Apache Software Foundation (http://www.apache.org/).

This product may include the JCifs library, for which the following notice applies:

Avid Interplay contains components licensed from LavanTech. These components may only be used as part of and in connection with Avid Interplay.

This product includes FFmpeg, which is covered by the GNU Lesser General Public License.

This product includes software that is based in part of the work of the FreeType Team.

This software is based in part on the work of the Independent JPEG Group.

This product includes libjpeg-turbo, which is covered by the wxWindows Library License, Version 3.1.

P41-RR02188 by the National Institutes of Health.

Portions relating to JPEG and to color quantization copyright 2000, 2001, 2002, Doug Becker and copyright (C) 1994, 1995, 1996, 1997, 1998, 1999, 2000, 2001, 2002, Thomas G. Lane. This software is based in part on the work of the Independent JPEG Group. See the file README-JPEG.TXT for more information. Portions relating to WBMP copyright 2000, 2001, 2002 Maurice Szmurlo and Johan Van den Brande.

Permission has been granted to copy, distribute and modify gd in any context without fee, including a commercial application, provided that this notice is present in user-accessible supporting documentation.

This does not affect your ownership of the derived work itself, and the intent is to assure proper credit for the authors of gd, not to interfere with your productive use of gd. If you have questions, ask. "Derived works" includes all programs that utilize the library. Credit must be given in user-accessible documentation.

This software is provided "AS IS." The copyright holders disclaim all warranties, either express or implied, including but not limited to implied warranties of merchantability and fitness for a particular purpose, with respect to this code and accompanying documentation.

Although their code does not appear in gd, the authors wish to thank David Koblas, David Rowley, and Hutchison Avenue Software Corporation for their prior contributions.

This product includes software developed by the OpenSSL Project for use in the OpenSSL Toolkit (http://www.openssl.org/)

Avid Interplay Pulse enables its users to access certain YouTube functionality, as a result of Avid's licensed use of YouTube's API. The charges levied by Avid for use of Avid Interplay Pulse are imposed by Avid, not YouTube. YouTube does not charge users for accessing YouTube site functionality through the YouTube APIs.

Avid Interplay Pulse uses the bitly API, but is neither developed nor endorsed by bitly.

Attn. Government User(s). Restricted Rights Legend

U.S. GOVERNMENT RESTRICTED RIGHTS. This Software and its documentation are “commercial computer software” or “commercial computer software documentation.” In the event that such Software or documentation is acquired by or on behalf of a unit or agency of the U.S. Government, all rights with respect to this Software and documentation are subject to the terms of the License Agreement, pursuant to FAR §12.212(a) and/or DFARS §227.7202-1(a), as applicable.

Trademarks

003, 192 Digital I/O, 192 I/O, 96 I/O, 96i I/O, Adrenaline, AirSpeed, ALEX, Alienbrain, AME, AniMatte, Archive, Archive II, Assistant Station, AudioPages, AudioStation, AutoLoop, AutoSync, Avid, Avid Active, Avid Advanced Response, Avid DNA, Avid DNxcel, Avid DNxHD, Avid DS Assist Station, Avid Ignite, Avid Liquid, Avid Media Engine, Avid Media Processor, Avid MEDIArray, Avid Mojo, Avid Remote Response, Avid Unity, Avid Unity ISIS, Avid VideoRAID, AvidRAID, AvidShare, AVIDstripe, AVX, Beat Detective, Beauty Without The Bandwidth, Beyond Reality, BF Essentials, Bomb Factory, Bruno, C|24, CaptureManager, ChromaCurve, ChromaWheel, Cineractive Engine, Cineractive Player, Cineractive Viewer, Color Conductor, Command|24, Command|8, Control|24, Cosmonaut Voice, CountDown, d2, d3, DAE, D-Command, D-Control, Deko, DekoCast, D-Fi, D-fx, Digi 002, Digi 003, DigiBase, Digidesign, Digidesign Audio Engine, Digidesign Development Partners, Digidesign Intelligent Noise Reduction, Digidesign TDM Bus, DigiLink, DigiMeter, DigiPanner, DigiProNet, DigiRack, DigiSerial, DigiSnake, DigiSystem, Digital Choreography, Digital Nonlinear Accelerator, DigiTest, DigiTranslator, DigiWear, DINR, DNxchange, Do More, DPP-1, D-Show, DSP Manager, DS-StorageCalc, DV Toolkit, DVD Complete, D-Verb, Eleven, EM, Euphonix, EUCON, EveryPhase, Expander, ExpertRender, Fader Pack, Fairchild, FastBreak, Fast Track, Film Cutter, FilmScribe, Flexevent, FluidMotion, Frame Chase, FXDeko, HD Core, HD Process, HDpack, Home-to-Hollywood, HYBRID, HyperSPACE, HyperSPACE HDCAM, iKnowledge, Image Independence, Impact, Improv, iNEWS, iNEWS Assign, iNEWS ControlAir, InGame, Instantwrite, Instinct, Intelligent Content Management, Intelligent Digital Actor Technology, IntelliRender, Intelli-Sat, Intelli-sat Broadcasting Recording Manager, InterFX, Interplay, inTONE, Intraframe, iS Expander, iS9, iS18, iS23, iS36, ISIS, IsoSync, LaunchPad, LeaderPlus, LFX, Lightning, Link & Sync, ListSync, LKT-200, Lo-Fi, MachineControl, Magic Mask, Make Anything Hollywood, make manage move | media, Marquee, MassivePack, Massive Pack Pro, Maxim, Mbox, Media Composer, MediaFlow, MediaLog, MediaMix, Media Reader, Media Recorder, MEDIArray, MediaServer, MediaShare, MetaFuze, MetaSync, MIDI I/O, Mix Rack, Moviestar, MultiShell, NaturalMatch, NewsCutter, NewsView, NewsVision, Nitris, NL3D, NLP, NSDOS, NSWIN, OMF, OMF Interchange, OMM, OnDVD, Open Media Framework, Open Media Management, Painterly Effects, Palladium, Personal Q, PET, Podcast Factory, PowerSwap, PRE, ProControl, ProEncode, Profiler, Pro Tools, Pro Tools|HD, Pro Tools LE, Pro Tools M-Powered, Pro Transfer, QuickPunch, QuietDrive, Realtime Motion Synthesis, Recti-Fi, Reel Tape Delay, Reel Tape Flanger, Reel Tape Saturation, Reprise, Res Rocket Surfer, Reso, RetroLoop, Reverb One, ReVibe, Revolution, rS9, rS18, RTAS, Salesview, Sci-Fi, Scorch, ScriptSync, SecureProductionEnvironment, Serv|GT, Serv|LT, Shape-to-Shape, ShuttleCase, Sibelius, SimulPlay, SimulRecord, Slightly Rude Compressor, Smack!, Soft SampleCell, Soft-Clip Limiter, SoundReplacer, SPACE, SPACEShift, SpectraGraph, SpectraMatte, SteadyGlide, Streamfactory, Streamgenie, StreamRAID, SubCap, Sundance, Sundance Digital, SurroundScope, Symphony, SYNC HD, SYNC I/O, Synchronic, SynchroScope, Syntax, TDM FlexCable, TechFlix, Tel-Ray, Thunder, TimeLiner, Titansync, Titan, TL Aggro, TL AutoPan, TL Drum Rehab, TL Everyphase, TL Fauxlder, TL In Tune, TL MasterMeter, TL Metro, TL Space, TL Utilities, tools for storytellers, Transit, TransJammer, Trillium Lane Labs, TruTouch, UnityRAID, Vari-Fi, Video the Web Way, VideoRAID, VideoSPACE, VTEM, Work-N-Play, Xdeck, X-Form, Xmon and XPAND! are either registered trademarks or trademarks of Avid Technology, Inc. in the United States and/or other countries.

Adobe and Photoshop are either registered trademarks or trademarks of Adobe Systems Incorporated in the United States and/or other countries. Apple and Macintosh are trademarks of Apple Computer, Inc., registered in the U.S. and other countries. Windows is either a registered trademark or trademark of Microsoft Corporation in the United States and/or other countries. All other trademarks contained herein are the property of their respective owners.

Avid Media | Index Configuration Guide • Created March 31, 2016 • This document is distributed by Avid in online (electronic) form only, and is not available for purchase in printed form.

Using This Guide . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7

Symbols and Conventions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7

If You Need Help. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8

Avid Training Services . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8

Chapter 1 Media | Index Configuration Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9

Prerequisites for Media | Index Configuration. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9

Checklist for Configuring Media | Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10

Chapter 2 Upgrading Media | Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11

Upgrading Media Index from v2.0.x or v2.1.x . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11

Upgrading Media Index from v2.2.x to v2.3 or Later. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14

Reactivating the Cluster After Upgrading . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15

Media | Index Backup Locations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16

Chapter 3 Configuring Media | Index for a Cluster . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17

Preparing for Media | Index Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17

Shard Count and Usage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18

Configuring Media | Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19

Validating the Media | Index Configuration on a Cluster. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22

Next Steps for Media | Index Installation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24

Chapter 4 Setting Up Multi-Zone Search . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25

Setting the Zone Bindings. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25

Validating the Elasticsearch Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27

Removing Zones from a Multi-Zone Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30

Chapter 5 Configuring Media | Index for a Single Node. . . . . . . . . . . . . . . . . . . . . . . . . . . . 31

Preparing for Media | Index Configuration for a Single Node . . . . . . . . . . . . . . . . . . . . . . . . . 31

Configuring Media | Index for a Single Node . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32

Validating the Media | Index Configuration on a Single Node. . . . . . . . . . . . . . . . . . . . . . . . . 35

Next Steps for Media | Index Installation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35

Chapter 6 Production Engine Bus Connector Installation . . . . . . . . . . . . . . . . . . . . . . . . . 36

Creating a PEBCo Instance on a Single Node . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38

Creating a PEBCo Instance in a Cluster Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38

Configuring the ICS Bus Service in Interplay Administrator . . . . . . . . . . . . . . . . . . . . . . . . . . 39

Configuring and Using the Production Engine Bus Connector Service . . . . . . . . . . . . . . . . . 40

Chapter 7 Installing and Configuring Production Services for

MediaCentral Delivery . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44

Understanding Production Services for MediaCentral Delivery . . . . . . . . . . . . . . . . . . . . . . . 44

Check List for Installing and Configuring Automation and Consolidate Services . . . . . . . . . . 45

Prerequisites for Installing and Configuring Automation and Consolidate Services . . . . . . . . 46

Installing the Automation and Consolidate Services. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47

Registering the Automation and Consolidate Services. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48

Registering a Provider with the Production Services Engine . . . . . . . . . . . . . . . . . . . . . . . . . 49

Creating a Consolidate Profile . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53

Chapter 8 Setting Up Interplay | MAM Sync Service . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57

About Interplay | MAM Sync Service Administrator . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57

Opening Sync Service Administrator . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60

Understanding the Sync Service Administrator Layout. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61

Quitting Sync Service Administrator . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66

Chapter 9 Setting Up iNEWS for Media | Index Integration . . . . . . . . . . . . . . . . . . . . . . . . . 67

Manually Installing the log4cpp File . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69

Editing iNEWS Site Files. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69

(Optional) Enabling Logging . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72

Assigning the Media | Index Attribute . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72

Activating the Mediaindex Program . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72

Chapter A Using the Media | Index Command Line Tool . . . . . . . . . . . . . . . . . . . . . . . . . . . 74

Updating Localization Strings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 74

Displaying Information about Initialized Systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 74

Changing Replica Counts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 75

Changing the Index Refresh Interval . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 75

Appendix B Media | Index Custom Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 77

Using the Media | Index System Settings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 77

Setting Elasticsearch Memory Usage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 82

Configuring the PEBCo for Custom Configurations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83

Changing the Default Search Type. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83

Changing the Maximum Number of Markers Returned in a Search . . . . . . . . . . . . . . . . . . . . 84

Using the Media | Index Restart Command Line Tool . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 85

Restarting Media Index Services . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 86

Scalability of Media | Index Configuration. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 86

Adjusting Relevance for Search Results. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 87

Disabling Search Suggestions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 90

Configuring Search Criteria Behavior . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 92

Restarting the Search and PEBCo Services. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 93

Appendix C Updating Media | Index Property Schemas . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 96

Property Changes in Production and Schema Updates . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 96

Monitor the Re-index Process. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 99

Using This Guide

This guide is intended for users of Avid MediaCentral Platform Services (MCS) with the Media | Index feature. This guide describes the configuration procedures for Media Index, such as configuring Media Index for cluster and single-node installations..

For information on installing MCS, see the MediaCentral Platform Services Installation and

Configuration Guide. For administrative information for MediaCentral | UX, see the Avid MediaCentral | UX Administration Guide.

Symbols and Conventions

Avid documentation uses the following symbols and conventions:

Symbol or Convention Meaning or Action

A note provides important related information, reminders,

recommendations, and strong suggestions.

A caution means that a specific action you take could cause harm to your computer or cause you to lose data.

A warning describes an action that could cause you physical harm.

> This symbol indicates menu commands (and subcommands) in the

(Windows), (Windows only), (Macintosh), or (Macintosh only)

Bold font Bold font is primarily used in task instructions to identify user interface

Italic font Italic font is used to emphasize certain words and to indicate variables.

Courier Bold font

Ctrl+key or mouse action Press and hold the first key while you press the last key or perform the

Follow the guidelines in this document or on the unit itself when handling electrical equipment.

order you select them. For example, File > Import means to open the File menu and then select the Import command.

This symbol indicates a single-step procedure. Multiple arrows in a list indicate that you perform one of the actions listed.

This text indicates that the information applies only to the specified operating system, either Windows or Macintosh OS X.

items and keyboard sequences.

Courier Bold font identifies text that you type.

mouse action. For example, Command+Option+C or Ctrl+drag.

If You Need Help

If you are having trouble using your Avid product:

1. Retry the action, carefully following the instructions given for that task in this guide. It is especially important to check each step of your workflow.

2. Check the latest information that might have become available after the documentation was published. You should always check online for the most up-to-date release notes or ReadMe because the online version is updated whenever new information becomes available. To view these online versions, select ReadMe from the Help menu, or visit the Knowledge Base at

www.avid.com/support.

3. Check the documentation that came with your Avid application or your hardware for maintenance or hardware-related issues.

4. Visit the online Knowledge Base at www.avid.com/support. Online services are available 24 hours per day, 7 days per week. Search this online Knowledge Base to find answers, to view error messages, to access troubleshooting tips, to download updates, and to read or join online message-board discussions.

If You Need Help

Avid Training Services

Avid makes lifelong learning, career advancement, and personal development easy and convenient. Avid understands that the knowledge you need to differentiate yourself is always changing, and Avid continually updates course content and offers new training delivery methods that accommodate your pressured and competitive work environment.

For information on courses/schedules, training centers, certifications, courseware, and books, please visit www.avid.com/support and follow the Training links, or call Avid Sales at 800-949-AVID (800-949-2843).

1 Media | Index Configuration Overview

This document describes the tasks required to configure a Media Index system.

Media Index comprises the following components:

• Elasticsearch

• Elasticsearch tribe service, the federated client that enables multi-zone search functionality.

• Data Import API, a bus-based service for metadata indexing, search, configuration, status, permissions, and updates for Media Index.

• Search API, a bus-based service that provides the search API

• Media Index Configuration API, a bus-based service that provides the API for the Media Index configuration

• Media Index Status API, a bus-based service that provides the API for the Media Index

• Media Index Permissions API, a bus-based service that provides per asset permissions for the search service

• Media Index Feed API, a bus-based service that provides an updated assets feed for external services

• Media Index Thesaurus API, a bus-based service that provides API for importing structured metadata (dictionaries)

service, which manages data storage and runs the search engine.

Elasticsearch is an open source, distributed, real-time search and analytics engine. Elasticsearch stores information about assets and makes it accessible across multiple sites.

Prerequisites for Media | Index Configuration

Before you configure Media Index, you need to install and configure the following MediaCentral components:

• MediaCentral Platform Services (MCS) — Media Index is configured after you have installed and set up your MCS system, either for a single node or for a cluster configuration. For more information about installing and configuring MCS, see the MediaCentral Platform Services Installation and Configuration Guide.

• Multi-zone components — If you use MCS in a multi-zone environment, you must first configure the multi-zone components. For information on installing MCS and configuring a multi-zone environment, see the MediaCentral Platform Services Installation and Configuration Guide.

Media Index is installed with MediaCentral Platform Services (MCS) by default, but the Media Index services are not started. You can install Media Index in both a single- and a multi-zone configuration. Each zone used for a Media Index installation includes a full MCS configuration with an Interplay Production database. The zone can include a single node or a cluster setup. Multi-zone configurations federate separate nodes, which can be separated geographically — for example, one node could be located in Toronto and another in Munich.

Checklist for Configuring Media | Index

All required Media Index components install when you install MCS. However, the services needed to run Media Index by default are not set to start automatically since you must configure them before you can use Media Index. Once you complete the configuration of the services, you can enable them to start automatically.

Checklist for Configuring Media | Index

The following table provides a checklist of steps for configuring Media Index components. The checklist also provides references where to find more information about each step.

Task Section Reference

Check the prerequisites. “Prerequisites for Media | Index Configuration”

on page 9.

If you are upgrading Media Index, reset your

existing Media Index configuration to prepare for the upgrade installation. If you have a multi-zone configuration, you need to upgrade any slave zones before your master zone. This is important for upgrading Media Index because as soon as you upgrade one of your zones, indexed search no longer works reliably in the old zones.

Configure the Media Index and the Elasticsearch

service.

Validate your configuration. “Validating the Media | Index Configuration on a

If you have a multi-zone setup, configure

Media Index for multiple zones.

Create and configure the Production Engine Bus

Connector.

Install and configure the Media Services Engine

and the three Interplay Media Services needed for delivering media and assets between Interplay Production workgroups or MediaCentral zones.

“Upgrading Media | Index” on page 11.

“Configuring Media | Index for a Cluster” on page 17 and “Configuring Media | Index for a Single Node” on page 31.

Single Node” on page 35 and “Validating the Media | Index Configuration on a Cluster” on page 22.

“Setting Up Multi-Zone Search” on page 25.

“Production Engine Bus Connector Installation” on page 36.

“Installing and Configuring Production Services for MediaCentral Delivery” on page 44.

Configure iNEWS for integration with Media

Index. This includes editing host and site files, assigning the Media Index attribute to queues or folders, and starting the ftsindex program.

Configure Interplay MAM for integration with

Media Index. This includes using the Sync Service Administrator to configure the SyncCentralIndex Service.

“Setting Up iNEWS for Media | Index Integration” on page 67.

“Setting Up Interplay | MAM Sync Service” on page 57.

For information on installing and configuring Interplay MAM, see the Interplay | MAM

Installation Manual and the Interplay | MAM Sync Service Administrator User’s Guide.

2 Upgrading Media | Index

The procedures for upgrading your Media Index configuration depend on which version of MediaCentral Platform Services (MCS) you want to upgrade. If you upgrade from a version earlier than MCS v2.3 — for example, if you upgrade from v2.0 to v2.1 or from v2.1 to v2.3 — you need to reset your Media Index before the upgrade can start. If you upgrade from v2.2.x to v2.3, you only need to migrate any index that you use in your Media Index configuration.

For upgrading earlier versions, the procedures for resetting Media Index must be completed prior to upgrading your MCS environment. For more information on upgrading MCS, see the Avid MediaCentral Platform Services Upgrading Guide.

Upgrading Media Index from v2.0.x or v2.1.x

Before upgrading your Media Index deployment, you must reset Media Index. In a multi-zone environment, you must reset the Media Index in each zone. During the Media Index reset procedure, all indexed data is deleted. After upgrade and reconfiguration, you must use the Production Engine Bus Connector (PEBCo) service to resync and restore the Media Index database. For information on using PEBCo, see “Production Engine Bus Connector Installation” on page 36.

If you are upgrading a multi-zone configuration, you need to upgrade any slave zones before your master zone. When you upgrade a zone, it cannot fully access the database in older zones. The zone can read from the user database so you can still log in as a user in the older configuration, but access to the database remains in read-only mode so that you cannot create new users in the new zone until you complete your upgrade.

In multi-zone configurations, you also must edit the ElasticSearch tribe configuration file to remove information about the zones that you are upgrading. For example, in a two-zone configuration where you start by upgrading the Zone1, you must remove the information that exposes the zone’s tribe node to Zone2, and you must remove the Zone2 configuration that exposes the node to Zone1. If there are other zones that you are not upgrading, you still need to edit the configuration file. However, any zones remaining at the older version can continue to have configuration information that exposes the zone to other old zones.

To reset Media Index to prepare for an upgrade:

1. On each node in your configuration, find the PEBCo instance by typing following command:

/opt/avid/bin/pam-agent-ctrl list

The server returns the name of your PEBCo instance.

2. Stop the PEBCo service by typing the following command:

/opt/avid/bin/pam-agent-ctrl stop <instance>

Upgrading Media Index from v2.0.x or v2.1.x

3. On the server hosting a zone within your configuration, list the information about systems within all configured Elasticsearch clusters by typing the following command:

/opt/avid/bin/avid-acs-elastic-index-setup -m info

The terminal window displays the system information for your Media Index configuration.

The following is a sample output:

Systems found: 1 ************************* System id : BC2F27F5-DBC5-4611-869A-F7A43690EE57 System type : interplay System display name : IP-VM-WG3 Number of assets : 18880 *************************

4. Type the following command for each system you want to delete from the index:

/opt/avid/bin/avid-acs-elastic-index-setup -m deleteSystem --systemId

<id_of_the_system>

To remove more systems from the index, type this command for each system.

5. Type the following command to remove the System Data index:

/opt/avid/bin/avid-acs-elastic-index-setup -m removeSystemData

6. Stop the search API by typing the following commands:

t For single-node installations:

service elasticsearch stop service elasticsearch-tribe stop service avid-acs-search stop service avid-acs-search-import stop service avid-acs-media-index-status-provider stop service avid-acs-media-index-permission stop service avid-acs-media-index-feed stop

t For cluster installations:

service elasticsearch stop service elasticsearch-tribe stop crm resource stop AvidSearchEverywhere crm resource stop AvidSearchImport service avid-acs-media-index-status-provider stop service avid-acs-media-index-permission stop service avid-acs-media-index-feed stop

To configure the Elasticsearch tribe configuration file before upgrading your multi-zone setup:

1. On each node that you want to upgrade, navigate to the directory containing the Elasticsearch tribe configuration file:

cd /etc/elasticsearch-tribe

Upgrading Media Index from v2.0.x or v2.1.x

2. Using the Linux text editor, vi, open the elasticsearch.yml configuration file for editing:

vi elasticsearch.yml

3. In the configuration file, delete the zone bindings information that expose that node to other nodes in the configuration. For example, if you have a two-zone setup — with Zone1 and Zone2 — and you want to upgrade Zone2 first, delete the sections of the configuration file highlighted below:

4. Press the Escape key to return to command mode, then type quit the text editor.

5. Restart Elasticsearch tribe service by typing the following command:

service elasticsearch-tribe restart

Validating the Media Index Changes

You can change the number of replicas for all indexes, for indexes at the specified alias, or for a specific index.

To validate your changes:

1. Open a browser and navigate to http://<server_name>:9200/_plugin/head, where sever_name is the hostname for the server in one zone in your configuration — for example, the local zone.

The Elasticsearch head plug-in Web page opens in your browser.

:wq

and press Return to write and

Upgrading Media Index from v2.2.x to v2.3 or Later

2. Check to make sure the system_data index and index information are not listed.

When Media Index is reset for your upgrade process, you can install the latest version of Media Index. For more information, see one of the following topics:

- For installing Media Index in a cluster configuration, see “Configuring Media | Index for a

Cluster” on page 17.

- For installing Media Index in a single-node configuration, see “Configuring Media | Index

for a Single Node” on page 31.

Upgrading Media Index from v2.2.x to v2.3 or Later

Before you upgrade your MCS environment to v2.3 or later, you need to ensure that there is no active schema migration currently affecting Media Index. When you change a field set of the system (Interplay Production, iNEWS, or Interplay Archive) and the changes propagate to Media Index, any existing data is automatically migrated to reflect the changes. For large indexes, schema migration can take a while. To avoid an inconsistent Media Index state, you need to make sure that there is no schema migration actively affecting your indexes.

Once any current migration completes, you can continue with the MCS upgrade procedures. For information on upgrading MCS, see the Avid MediaCentral Platform Services Upgrading Guide.

When you complete the MCS upgrade, you must migrate your indexes so that all new changes take effect. Media Index continues to function during migration, with the exception for marker searches, but it uses the older version of the index do that the migration does not require any downtime for your MCS system. Once the migration completes, all functionality of the new index is available and the old index is deleted.

If you upgrade Media Index from v2.4 or v2.5 to v2.6 with an Interplay MAM system, you must re-sync your legal list using the MAM CentralSyncAgent. Use the CentralSyncAgent Re-sync Legal List option to resend dictionary data to Media Index. For more information on using the Interplay MAM Sync Service Administrator, see “Setting Up Interplay | MAM Sync Service” on page 57 and the Interplay | MAM Sync Service Administrator User’s Guide.

To check the migration status prior to upgrading:

1. Sign in to MediaCentral UX as an administrator.

2. Select System Settings from the Layout selector.

The System Settings layout opens.

3. In the Settings pane select Media Index > Cluster State.

The Media Index Cluster State displays in the Details pane.

Reactivating the Cluster After Upgrading

4. Check the cluster state:

- If there is no active migration, all index labels appear in black.

- If the index has an active migration, its label appears in red with an indication of the

migration progress.

5. If there is active migration is in progress, wait for the process to finish before proceeding with the MCS upgrade procedures.

To migrate the Media Index data after upgrading MCS:

t Type the following command to run media index migration tool:

/opt/avid/media-index/avid-acs-media-index-migration/avid-acs-mediaindex-indices-migration

Media Index migration tool detects any existing indexes and automatically migrates the data. A progress indicator displays during the migration.

Reactivating the Cluster After Upgrading

When you upgrade a cluster configuration, you must reset the cluster by running the reactivate-cluster script on all nodes that have been upgraded before you set up Media Index in your upgraded environment. This is due to the cluster still having the search resources configured after the upgrade, and when the system tries to start the resources it fails because no system data has been created.

If you have more than two nodes in a zone, you also must put the load-balancing node into standby after running the cluster reactivation script and before beginning your Media Index configuration.

To reactivate the cluster and set the load balancing node into standby:

1. Log in to the master node of your configuration as the root user, and type the following command:

/opt/avid/cluster/bin/reactivate-cluster

The cluster resources are reset.

2. To put the load-balancing nodes into standby mode, type the following command:

crm node standby <node_name>

Media | Index Backup Locations

When you re-install the MediaCentral Platform Services software, a backup of all data is created before a complete re-installation using the USB key. However, indexes are not backed up by the system backup script. This typically does not cause an issue since users generally do not need to back up their indexes. For cluster configurations, the following notes also apply:

• In a cluster, the index should be configured with data redundancy to avoid data loss.

• In the case of a complete rebuild or of complete data loss within a cluster, Avid recommends that you do not use a backup but re-index the data instead. This is because the backup will no longer be in sync with what the index agents are expecting.

With the release of Elasticsearch v1.6, backups are only allowed on shared file locations, the backup in Media Index v2.4 requires manual configuration of the backup locations. Setting this up correctly requires a file server to be mounted on all machines. Customers should contact Avid Customer Care or Avid Professional Services for instructions and support from Avid Engineering in setting this up.

Media | Index Backup Locations

3 Configuring Media | Index for a Cluster

When you set up the Elasticsearch service, you create a node that runs the service on a cluster. Every node in an MCS cluster runs an Elasticsearch instance as well as an Elasticsearch-tribe instance. The Elasticsearch instance runs the indexing process while the Elasticsearch-tribe instance is used for searching. Tribe instances allow you to perform read operations against the nodes in multiple clusters.

The MCS cluster and the Elasticsearch cluster do not share configuration details and use different mechanisms for the clustering, even though the MCS cluster contains the Elasticsearch cluster.

You can configure a tribe instance in a federated way so that a tribe instance of one Elasticsearch cluster can be connected to other Elasticsearch clusters (one or more Elasticsearch data instances). For multi-zone configurations, this functionality is required where each zone (a single node or an MCS cluster) runs its own Elasticsearch cluster. The tribe node configuration allows these two clusters to be connected to each other so you can find assets from remote Elasticsearch clusters.

When you create an index, Elasticsearch creates a number of “shards” and “replicas,” each shard containing indexed data and each replica containing a copy of that data. You use shards and replicas to spread the search load and provide redundancy. The default number of shards is 1 and the default number of replicas is 0, which is appropriate for single node configurations. For cluster configurations, Avid recommends setting the replica count to one fewer than the number of cluster nodes to accommodate fail-over safety and load balancing. For example, if your configuration includes a cluster with five nodes, set the replica count to four. This allows for each node in the cluster to contain a replica of all your indexed data, with the other node in the cluster containing the shard. For a cluster configuration, you do not need to reconfigure shards, but you should set the replica count. If needed, you can change this during your Media Index configuration, but you should not change the shard count once configuration is complete. If you modify a shard that has been configured, you must re-index your system. For information on setting replica counts, see

“Configuring Media | Index” on page 19.

Preparing for Media | Index Configuration

When you install MCS, Media Index services are stopped. Before you begin configuring Media Index, you must make sure all Media Index services are added to the cluster.

To ensure that the Media Index services are added to the cluster, do the following:

t On one of the nodes of your cluster, type the following command:

/opt/avid/cluster/bin/search-cluster setup --cluster-ip=<CLUSTER-IP>

The necessary resources are added to the cluster configuration, and the resources needed for the configuration start.

Shard Count and Usage

When you configure Media Index, you can set the number of primary shards (see “Configuring

Media | Index” on page 19). The number of shards in your configuration affects the amount of data

that can be stored in your index, although the type of data and metadata affects the size of the index as well. For single node configurations, Avid recommends that you set the default number of shards as 1 and the default number of replicas as 0. For a cluster setup, the actual number of shards and replicas needed depends on your configuration and use cases.

A single shard can contain your entire index, and it can use all of the resources of a single node. In cluster configurations, you can increase the shard count to decrease the amount of data per shard. This spreads the load across multiple nodes. Coupled with replicas, this can provide redundancy and flexibility for your index. For example, if you start with two nodes but know that at some point you might scale your configuration up to six nodes, you might want to configure Media Index initially for six shards.

Using multiple shards and replicas also gives your index more stability in case of a failure on one of your nodes or if you need to take one or more nodes offline for maintenance. For example, the following table illustrates a configuration with three nodes, three shards distributed on the nodes, and one replica (for each shard).

Shard Count and Usage

Node Shard Replica

112

223

331

If Node 1 goes offline, Replica 1 on Node 3 becomes a primary shard and the index remains fully functional. However, there are now no replicas for Shards 1 or 2. Elasticsearch then re-creates the missing replicas, as illustrated in the following table.

Node Shard Replica

1 [offline] [offline]

2 2 3, 1

33, 12

If Node 2 goes offline, the index remains functional because Node 3 contains Shards 1 and 3, as well as a replica of Shard 3. However, if Node 2 goes offline before Elasticsearch re-creates the replicas, then the index is incomplete and cannot work.

Configuring Media | Index

If you need to take a node offline, you can do any of the following, depending on your needs and your setup:

• Increase the replica count before removing a node to allow for more nodes to be taken offline. This causes Elasticsearch to copy data on startup, which can consume processor resources. This can be a good solution if you remove multiple nodes. However, increasing the number of replicas puts more load on the existing nodes, both in terms of memory needed for the index and for storage space.

• Decrease number of shards. This is not a good solution for configurations with large indexes because you cannot dynamically change the shard count changes on existing indexes. You can change the replica count at any time, but if you want to change the shard count you must reset the index, reconfigure it with the new shards count, and then re-index the data.

• Stop Media Index services completely before taking a node offline. You can start the services again after the node is brought back online and the cluster is restored. This is the best solution when you need to remove a node temporarily from the cluster.

You can change the number of replicas in a cluster, which allows you to scale your index up or down as demand requires. However, once you configure your shard count for an index, you cannot modify this without a full re-index operation. Setting a higher shard count during Media Index configuration does not impact performance for the index, but there are few advantages. The replica count is more important in determining the functionality of the index when a node goes offline.

Configuring Media | Index

You configure Media Index in the Media Central | UX System Settings. You must provide the following properties in your settings:

• Cluster Name — Avid recommends that you use the zone name or location. The cluster name identifies the Elasticsearch node as part of a cluster. Nodes with the same cluster name can be joined in a cluster. You must use a different cluster name for each zone in a multi-zone setup, which requires modifying the cluster name for remote zones.

When you configure Elasticsearch in a multi-zone environment, Avid recommends that you use the zone name used during the bus and UMS multi-zone setup step as the Elasticsearch cluster name.

If you change the cluster name, during a restart Elasticsearch creates a new data folder. If you had any indexes before you changed the cluster name, the indexes fail to appear in your configuration and the information in them is not accessible. While the index data is not lost, Avid recommends that you not change the cluster name after your initial configuration. When you work in a multi-zone set-up, you should consider which cluster names you plan to use in all zones and not change them after the indexing process has started.

• Elasticsearch Cluster Hosts — Specify the server or servers where Elasticsearch service is running. Avid recommends you use the hostname of the system to easily identify the different systems within the Elasticsearch configuration. In a standalone deployment, the host is a hostname or IP address of the Media Central node. In a clustered environment, the host is a series of hostnames or IP addresses of all nodes in a cluster. Multiple cluster host values must be separated with commas.

• Elasticsearch TCP Port — Specify the TCP port of the Elasticsearch data node, used for binary connection protocol and for inter-node communications. The default value is 9300. In most cases you, you do not need to modify this property.

Configuring Media | Index

• Elasticsearch HTTP Port — Specify the port of the Elasticsearch data node HTTP API. The default value is 9200. In most cases you, you do not need to modify this property.

• Elasticsearch Tribe TCP Port — Specify the TCP port of the Elasticsearch tribe node, used for inter-node communications. The default value is 9305. In most cases you, you do not need to modify this property.

• Elasticsearch Tribe HTTP Port — Specify the port of the Elasticsearch tribe node HTTP API. The default value is 9201. In most cases you, you do not need to modify this property

• Default Shards — Specify default shards count. Avid recommends using a default value of 1 shard.

• Default Replicas — Specify default replica count. The default number of replicas is 0, which is the required replica count for a single-node configuration. In an MCS cluster installation, you must set the default number of replicas to a number greater than zero.

A replica count greater than zero is necessary in a cluster configuration in order to ensure that your data gets distributed properly and remains safe. For maximum security, you should set the replica count to a value of <number of MCS cluster nodes> – 1. Do not set the replica count to a value greater than this. For example, if your configuration uses two nodes, then set the replica count to 1.

• Supported languages — Lists the languages supported by Media Index.

Media Index supports multiple languages for your search. However, enabling all languages can slow down searches. You should enable only those languages your users require. If you work with a multi-zone configuration, you can enable different languages for each zone, and these languages are then available for all indexes within the specified zone. If possible, your language selection should be the same across all zones, because a search using one language returns results only from indexes that you have configured for this language. The following table provides an example of which languages are used in searches within a multi-zone configuration with different languages enabled for different zones.

Enabled Languages Language Selected for Search Languages in Results

Zone 1: English, French

Zone 2: English, Korean

Zone 1: English, French

Zone 2: English, Korean

English Zone 1: English

Zone 2: English

French Zone 1: French

Zone 2: [no results]

Zone 1: English, French, Korean

Zone 2: English, French, Korean

If you change your language settings after you have configured Media Index, you must reset Media Index. This deletes all indexes, which you then need to re-index.

When you set up a tribe node, you need to make sure the Elasticsearch tribe node configuration file contains the appropriate properties. For information on configuring tribe nodes in a multi-zone environment, see “Setting Up Multi-Zone Search” on page 25.

When you set these values and apply your changes, Media Index writes to several configuration files and restarts cluster services and resources. It also creates the system_data index. This process might take some time, depending on your cluster configuration.

French Zone 1: French

Zone 2: French

To configure Media Index:

1. Sign in to MediaCentral UX as an administrator.

2. Select System Settings from the Layout selector.

The System Settings layout opens.

3. In the Settings pane select Media Index > Settings.

The Media Index Settings display in the Details pane.

Configuring Media | Index

4. In the Supported Languages section, select the languages you want to make available for searching. You can select as many languages as needed, but Avid recommends you select only those languages your users are likely to use in a search.

5. Click the Default Language menu, and select the default language for indexed searches.

6. Specify the required settings for the following properties:

-Cluster Name

- Elasticsearch Cluster Hosts

- Elasticsearch TCP Port

- Elasticsearch HTTP Port

- Elasticsearch Tribe TCP Port

- Elasticsearch Tribe HTTP Port

Validating the Media | Index Configuration on a Cluster

- Default Shards

- Default Replicas

7. Click Apply. It might take a couple of minutes for configuration to finish.

A message displays when your configuration completes. This might take some time as Media Index updates configuration files.

8. Click Close.

Validating the Media | Index Configuration on a Cluster

You can validate your setup by checking the status of the Media Index services and ensuring that they are running. Using the Cluster State pane in MediaCentral UX, you can view all nodes with core services running and see that the system data index is available. You can also use the cluster resource manager utility, crm_mon, to monitor the status of MediaCentral services.

For information on using the cluster resource manager, see “Monitoring Services and Resources” in

the MediaCentral Platform Services Installation and Configuration Guide.

You should also check the cluster for any errors and validate the system_data index. You can use the cluster resource monitor utility to monitor fail counts, and you can check the Indexes pane in MediaCentral UX to view the status of the indexes in your configuration.

In the MediaCentral UX System Settings for Media Index, the Indexes Details pane displays only the

indexes in the local zone. To display indexes of other zones, connect directly to the MediaCentral UX of that zone.

If you are configuring your installation for a multi-zone environment, see “Setting Up Multi-Zone

Search” on page 25 and the MediaCentral Platform Services Installation and Configuration Guide

for more information.

To check the current status of Media Index:

1. Sign in to MediaCentral UX as an administrator.

2. Select System Settings from the Layout selector.

The System Settings layout opens.

Validating the Media | Index Configuration on a Cluster

3. In the Settings pane select Media Index > Cluster State.

The Media Index Cluster State displays in the Details pane.

Media Index Cluster State pane, with the system_data index and two indexes available

The core services for your configuration must display a status of “Running,” and the indexes must display the shard count in green.

To monitor the fail count within the cluster:

1. Log in to any node in the cluster as root and verify the cluster fail counts by typing the following command:

crm_mon -f

2. Use the cleanup command to reset any observed failure counts:

crm resource cleanup <resource>

<resource> is the name of the resource you want to monitor — for example, avid-acs-search.

For more information on using the cluster resource monitor utility to check fail counts, see

“Observing Failover in the Cluster” in the MediaCentral Platform Services Installation and Configuration Guide.

To check the current status of your indexes:

1. Sign in to MediaCentral UX as an administrator.

2. Select System Settings from the Layout selector.

The System Settings layout opens.

3. In the Settings pane select Media Index > Indexes.

The Media Index Indexes status displays in the Details pane.

Next Steps for Media | Index Installation

All indexes for your configuration display with their status and details listed in the pane.

Next Steps for Media | Index Installation

If you are installing Media Index in a multi-zone configuration, see “Setting Up Multi-Zone Search”

on page 25.

When you have finished setting up Media Index, you need to configure and start the Production Engine Bus Connector (PEBCo) service to resync and restore the Media Index database. For information on PEBCo, see “Production Engine Bus Connector Installation” on page 36.

4 Setting Up Multi-Zone Search

In a multi-zone MCS configuration, each zone runs its own Elasticsearch cluster with its own Elasticsearch cluster name. These different Elasticsearch clusters are then connected to each other using the Elasticsearch tribe service. For example, in a two zone configuration, if you call one cluster “Zone1” and another cluster “Zone2,” you need to set the cluster name to Zone1 on all nodes in one MCS cluster and to Zone2 on all nodes in the second MCS cluster.

Avid recommends that you name the clusters according to the zone names you used for the multi-zone

configuration on the UMS and bus level.

You can configure the MCS multi-zones before or after starting the Media Index multi-zone configuration. However, if you complete your Media Index multi-zone configuration without having previously configured your MCS multi-zones, you cannot play back remote assets.

If you need to change the cluster name, you must make the change before proceeding with your multi-zone configuration. For information on changing the cluster name, see “Using the

Media | Index System Settings” on page 77.

If you change the cluster name, Elasticsearch creates a new data folder during a restart. If you had any indexes before you changed the cluster name, the indexes fail to appear in your configuration and the information in them is not accessible. While the index data is not lost, Avid recommends that you not change the cluster name after your initial configuration. When you work in a multi-zone set-up, you should consider which cluster names you plan to use in all zones and not change them after the indexing process has started.

For information for configuring MCS for multi-zones, see the MediaCentral Platform Services Installation and Configuration Guide. For information on user management and multi-zones in MediaCentral UX, see “Managing Multi-Zone Environments” in the Avid MediaCentral | UX Administration Guide.

If you are upgrading your multi-zone configuration, you must reset your database for multi-zone access before you begin your multi-zone configuration.

If you want to remove a zone from your multi-zone configuration, you must first remove the bindings for each zone cluster.

Setting the Zone Bindings

To join two individual zones in a multi-zone setup, you must update the Elasticsearch tribe configuration on all of your tribe nodes and set bindings for all Elasticsearch clusters. If you set up both zones with the same cluster name, you also have to reconfigure the cluster name in one of the zones to make it unique. This requires changes in the Elasticsearch and the Elasticsearch tribe.

You must update the Elasticsearch configuration file — /etc/elasticsearch-tribe/elasticsearch.yml —

on all cluster nodes in each of the zones.

Setting the Zone Bindings

To set a binding to the zone cluster and to update the Elasticsearch tribe configuration:

1. Navigate to the directory containing the Elasticsearch tribe configuration file:

cd /etc/elasticsearch-tribe

2. Using the Linux text editor, vi, open the elasticsearch.yml tribe configuration file for editing:

vi elasticsearch.yml

3. Add the following lines in the configuration file to add the binding that allows the tribe to connect to another cluster:

tribe.<myRemoteClusterName>.cluster.name: <myRemoteClusterName> tribe.<myRemoteClusterName>.transport.tcp.port:

You must use a space following the colon (:) before you add your value to the line in the

configuration file.

The port number must be unique. If you add another remote cluster to your configuration, then you

must use the next port (9314).

4. Add the following line to configure Unicast for tribe node binding:

tribe.<myRemoteClusterName>.discovery.zen.ping.unicast.hosts:

- <"remoteMCSNode1:9300">

- <"remoteMCSNode2: 9300">

...

- <"remoteMCSNodeN:9300">

You must use a space following the colon (:) before you add your values to the lines in the

configuration file.

You can use either the hostname or the IP address of the nodes.

5. Restart all Elasticsearch tribe nodes affected by your configuration changes:

t For single node configurations, type the following command:

service elasticsearch-tribe restart

9313

t For cluster configurations, type the following command on one of the nodes in your cluster:

crm resource restart elasticsearchTribeEverywhere

You do not use this command when you have already configured the MCS cluster with the search

resources.

6. If you have a cluster configuration, repeat these steps for each node within the cluster.

For examples of the Elasticseaarch configuration files, see “Example elasticsearch.yml on

Master node in Master Zone” on page 28.

Validating the Elasticsearch Configuration

When you finish updating the Elasticsearch configuration file, you can validate your setup by checking the Elasticsearch information for each server.

To validate the Elasticsearch configuration:

1. Open a browser and navigate to http://<server_name>:9200/_plugin/head, where sever_name is the hostname for the server in one zone in your configuration — for example, the local zone.

The Elasticsearch head plug-in Web page opens in your browser.

2. Verify that the local zone and the bindings between zones are listed.

1 The remote zone tribe is bound to the local data node.

2 The local zone data node is listed.

3 The local zone tribe is bound to the local data node.

3. Open a browser and navigate to http://<server_name>:9200/_plugin/head, where sever_name is the hostname for the server in another zone in your configuration — for example, the remote zone.

The Elasticsearch head plug-in Web page opens in your browser.

4. Verify that this local zone (the second zone in your multi-zone configuration) and the bindings between zones are listed.

Validating the Elasticsearch Configuration

1 The local zone data node is listed.

2 The local zone tribe is bound to the local data node.

3 The remote zone tribe is bound to the local data node.

5. Open a browser and navigate to http://<server_name>:9201/_plugin/head, where sever_name is the hostname for the server in a zone in your configuration — for example, the local zone — and the port number is configured for the tribe node of the zone.

The Elasticsearch head plug-in Web page for the tribe node opens in your browser.

6. Verify that the tribe node lists the all indexes in your configuration as well as the local and remote data nodes.

1 Make sure you are connected to the tribe node.

2 Indexes from all nodes are listed.

3 The remote zone data node is listed.

4 The local zone data node is listed.

5 The local zone tribe node is listed.

Example elasticsearch.yml on Master node in Master Zone

discovery.zen.ping.multicast.enabled: false tribe.local.cluster.name: MCSclusterHostname-Master-Zone tribe.local.transport.tcp.port: 9312 tribe.local.discovery.zen.ping.unicast.hosts:

- "MCS-Master-Node1:9300"

- "MCS-Master-Node2:9300"

Validating the Elasticsearch Configuration

- "MCS-Master-Node3:9300"

- "localhost:9300" http.port: 9201 transport.tcp.port: 9305 node.name: MCS-Master-Node1-tribe tribe.remote.cluster.name: MCSclusterHostname-Slave-Zone tribe.remote.transport.tcp.port: 9313 tribe.remote.discovery.zen.ping.unicast.hosts:

- "MCS-Slave-Node1:9300"

- "MCS-Slave-Node2:9300"

- "MCS-Slave-Node3:9300"

"The elasticsearch.yml on the Slave node in the Master Zone is the same as the Master in the Master

zone, except for the node.name value — for example:

node.name: MCS-Master-Node2-tribe

Example elasticsearch.yml on Master node in Slave Zone

discovery.zen.ping.multicast.enabled: false tribe.local.cluster.name: MCSclusterHostname-Slave-Zone tribe.local.transport.tcp.port: 9312 tribe.local.discovery.zen.ping.unicast.hosts:

- "MCS-Slave-Node1:9300"

- "MCS-Slave-Node2:9300"

- "MCS-Slave-Node3:9300"

- "localhost:9300" http.port: 9201 transport.tcp.port: 9305 node.name: MCS-Slave-Node1-tribe tribe.remote.cluster.name: MCSclusterHostname-Master-Zone tribe.remote.transport.tcp.port: 9313 tribe.remote.discovery.zen.ping.unicast.hosts:

- "MCS-Master-Node1:9300"

- "MCS-Master-Node2:9300"

- "MCS-Master-Node3:9300"

"The elasticsearch.yml on the Slave node in the Slave Zone is the same as the Master in the Slave

zone, except for the node.name value — for example:

node.name: MCS-Slave-Node2-tribe

Removing Zones from a Multi-Zone Configuration

If you want to remove a zone from your multi-zone configuration, you must first remove the bindings for each zone cluster.

To remove the zone information in the Elasticsearch tribe configuration file before removing a zone in your multi-zone setup:

1. On each node that you want to upgrade, navigate to the directory containing the Elasticsearch tribe configuration file:

cd /etc/elasticsearch-tribe

2. Using the Linux text editor, vi, open the elasticsearch.yml configuration file for editing:

vi elasticsearch.yml

3. In the configuration file, delete the zone bindings information that expose that node to other nodes in the configuration. For example, if you have a two-zone setup — with Zone1 and Zone2 — and you want to remove Zone2, delete the sections of the configuration file highlighted below:

:wq

4. Press the Escape key to return to command mode, then type

and press Return to write and

quit the text editor.

5. Restart Elasticsearch tribe service by typing the following command:

service elasticsearch-tribe restart

+ 72 hidden pages

Pinnacle Systems Media Index - 2.6 User Manual

Contents

Using This Guide

Symbols and Conventions

If You Need Help

Avid Training Services

1 Media | Index Configuration Overview

Prerequisites for Media | Index Configuration

Checklist for Configuring Media | Index

2 Upgrading Media | Index

Upgrading Media Index from v2.0.x or v2.1.x

Validating the Media Index Changes

Upgrading Media Index from v2.2.x to v2.3 or Later

Reactivating the Cluster After Upgrading

Media | Index Backup Locations

3 Configuring Media | Index for a Cluster

Preparing for Media | Index Configuration

Shard Count and Usage

Configuring Media | Index

Validating the Media | Index Configuration on a Cluster

Next Steps for Media | Index Installation

4 Setting Up Multi-Zone Search

Setting the Zone Bindings

Validating the Elasticsearch Configuration

Removing Zones from a Multi-Zone Configuration