Veryx PktBlaster User Manual

SDN Controller Test

Performance Benchmarking

Version 1.4

User Guide

Part Number: T/UG–PKTBLASTER-SDN-CTR-PERF-1.4–1015/1.1

Veryx™ and PktBlaster™ are trademarks of Veryx Technologies Pvt. Ltd. All other trademarks of respective owners are acknowledged.

This document may not be copied either in whole or in part by any means or media without the permission of Veryx Technologies Pvt. Ltd.

THE PROGRAMS FURNISHED ARE PROVIDED ON AN "AS IS" BASIS, WITHOUT ANY WARRANTIES OR REPRESENTATIONS EXPRESS, IMPLIED, OR STATUTORY; INCLUDING, WITHOUT LIMITATION, WARRANTIES OF QUALITY, PERFORMANCE, NONINFRINGEMENT, MERCHANTABILITY, OR FITNESS FOR A PARTICULAR PURPOSE. NOR ARE THERE ANY WARRANTIES CREATED BY A COURSE OF DEALING, COURSE OF PERFORMANCE, OR TRADE USAGE. LICENSOR DOES NOT WARRANT THAT THE PRODUCTS WILL MEET LICENSEE'S NEEDS OR BE FREE FROM ERRORS, OR THAT THE OPERATION OF THE PRODUCTS WILL BE UNINTERRUPTED.

Table of Contents

1 About Pkt Blaster SDN Controller Test ..................................................... 1

Key Features .......................................................................................................................... 2

Working with PktBlaster SDN Controller Test ........................................................................ 2

User Interface ................................................................................................................. 2

Troubleshooting .............................................................................................................. 2

Reporting ........................................................................................................................ 3

SDN Controller Test Process Flow......................................................................................... 4

Intended Audience for this Guide ........................................................................................... 5

2 Getting Started ............................................................................................ 7

Log in to PktBlaster SDN Controller Test ............................................................................... 7

About the Home Screen ......................................................................................................... 8

Log out of PktBlaster SDN Controller Test ............................................................................. 9

3 Configuring an SDN Controller ................................................................ 11

Add SDN Controllers ............................................................................................................ 12

View Available SDN Controllers ........................................................................................... 14

Key features to Customize Controller View .................................................................. 15

Edit SDN Controller Details .................................................................................................. 17

Delete SDN Controller .......................................................................................................... 19

4 Configuring a Traffic Profile .................................................................... 21

Add a New Traffic Profile ...................................................................................................... 22

Type–IPV4 .................................................................................................................... 28

Type-Custom ................................................................................................................ 30

View Traffic Profiles .............................................................................................................. 31

Modify a Traffic Profile .......................................................................................................... 32

Delete a Traffic Profile .......................................................................................................... 33

5 Performance Tests .................................................................................... 35

Types of Tests ...................................................................................................................... 35

Performance Throughput ..................................................................................................... 35

Performance Latency ........................................................................................................... 35

Performance Robustness ..................................................................................................... 35

Testing Modes ...................................................................................................................... 36

Throughput Test in Normal Mode......................................................................................... 37

Throughput Test in Incremental Switch Mode ...................................................................... 37

Latency Test in Normal Mode .............................................................................................. 37

Latency Test in Incremental Switch Mode ........................................................................... 38

Configure Test Parameters .................................................................................................. 38

Step 1: Select Controller ............................................................................................... 39

Step 2: Network Configuration ...................................................................................... 40

Show Advanced Configuration ..................................................................................... 41

Step 3: Test Configuration ............................................................................................ 43

6 Test Execution Details .............................................................................. 47

View a Test in Progress ....................................................................................................... 47

View Test Results ................................................................................................................. 48

Re-execute a Test ................................................................................................................ 50

Test Fails to Complete ......................................................................................................... 51

7 Executing Throughput Tests ................................................................... 53

Throughput Test in Normal Mode......................................................................................... 53

View Test Result ........................................................................................................... 56

Throughput Test in Incremental Switch Mode ...................................................................... 59

View Test Result ........................................................................................................... 61

8 Executing Latency Tests ......................................................................... 65

Latency Test in Normal Mode ............................................................................................... 65

View Test Result ........................................................................................................... 68

Latency Test in Incremental Switch Mode ............................................................................ 69

View Test Result ........................................................................................................... 72

9 Executing Robustness Tests ................................................................... 75

View Test Result ........................................................................................................... 78

10 Viewing and Generating Reports ............................................................ 81

View the Dashboard ............................................................................................................. 81

View Bookmarks ................................................................................................................... 82

View Throughput Report ....................................................................................................... 83

View Latency Report ............................................................................................................ 85

View Robustness Report ...................................................................................................... 87

View Individual Test Execution Report ......................................................................... 88

Search for a Report .............................................................................................................. 91

Download Report in PDF Format ......................................................................................... 93

Glossary ......................................................................................................... 95

Index ............................................................................................................... 97

Table of Figures

Figure 1: PktBlaster SDN Controller Test-Performance Benchmarking Deployment .................. 3

Figure 2: PktBlaster SDN Controller Test Login Screen .............................................................. 7

Figure 3: Home Screen ................................................................................................................ 7

Figure 4: PktBlaster SDN Controller Test Logout Screen ............................................................ 9

Figure 5: Logout Confirmation Dialog Box ................................................................................... 9

Figure 6: Controller Screen ........................................................................................................ 11

Figure 7: Add a new Controller ................................................................................................... 12

Figure 8: New Controller Added ................................................................................................. 14

Figure 9: Available Controllers ................................................................................................... 15

Figure 10: Controller Configuration Screen ................................................................................ 16

Figure 11: Controller Configuration-Customize Display ............................................................. 17

Figure 12: Controller Details ....................................................................................................... 18

Figure 13: Modify Controller Details ........................................................................................... 18

Figure 14: Delete Confirmation Dialog Box ................................................................................ 19

Figure 15: Traffic Profile Screen ................................................................................................. 21

Figure 16: Add a New Traffic Profile........................................................................................... 22

Figure 17: Traffic Profile-Type IPV4 ........................................................................................... 28

Figure 18: Traffic Profile-TCP/UDP ............................................................................................ 29

Figure 19: Traffic Profile-L3 Custom........................................................................................... 30

Figure 20: Traffic Profile-Ether Type Custom ............................................................................. 31

Figure 21: View Traffic Profile .................................................................................................... 32

Figure 22: Edit a Traffic Profile ................................................................................................... 33

Figure 23: Traffic Profile Details ................................................................................................. 34

Figure 24: Delete Confirmation Dialog Box ................................................................................ 34

Figure 25: Performance Screen ................................................................................................. 39

Figure 26: Select Controller Pane .............................................................................................. 39

Figure 27: Network Configuration Pane ..................................................................................... 40

Figure 28: Advanced Configuration ............................................................................................ 42

Figure 29: Switch Capabilities OF Version 1.0 and 1.3 ............................................................. 42

Figure 30: Test Configuration Pane ........................................................................................... 43

Figure 31: Test Execution in Progress (Upper) .......................................................................... 47

Figure 32: Test Execution in Progress (Lower) .......................................................................... 48

Figure 33: Test Results-Configuration Details ........................................................................... 49

Figure 34: Test Results-Graph ................................................................................................... 49

Figure 35: Test Results-Table .................................................................................................... 50

Figure 36: Re-execute Test ........................................................................................................ 51

Figure 37: Test Configuration-Throughput Test Mode ............................................................... 54

Figure 38: Throughput Normal Mode (Upper) ............................................................................ 55

Figure 39: Throughput Normal Mode (Lower) ............................................................................ 56

Figure 40: Throughput Normal Result-Graph ............................................................................. 57

Figure 41: Throughput Normal Result-Table .............................................................................. 58

Figure 42: Test Configuration- Throughput Incremental Switch Mode ...................................... 59

Figure 43: Throughput Incremental Switch Mode (Upper) ......................................................... 60

Figure 44: Throughput Incremental Switch Mode (Lower) ......................................................... 61

Figure 45: Throughput Incremental Switch Result-Graph .......................................................... 62

Figure 46: Throughput Incremental Switch Result-Table ........................................................... 63

Figure 47: Test Configuration-Latency Normal Mode ................................................................ 66

Figure 48: Latency Normal Mode (Upper) .................................................................................. 67

Figure 49: Latency Normal Mode (Lower) .................................................................................. 67

Figure 50: Latency Normal Result .............................................................................................. 68

Figure 51: Latency Normal Table Report ................................................................................... 69

Figure 52: Test Configuration-Latency Incremental Switch Mode ............................................. 70

Figure 53: Latency Incremental Switch Mode (Upper) ............................................................... 71

Figure 54: Latency Incremental Switch Mode (Lower) ............................................................... 72

Figure 55: Latency Incremental Switch Result ........................................................................... 72

Figure 56: Latency Incremental Switch Table Result ................................................................. 73

Figure 57: Robustness Flow Mod Test Configuration ................................................................ 76

Figure 58: Robustness Flow Mod Result Report (Upper) .......................................................... 77

Figure 59: Robustness Flow Mod Result Report (Lower) .......................................................... 78

Figure 60: Robustness Flow Mod Graph Result ........................................................................ 78

Figure 61: Result Table Report .................................................................................................. 79

Figure 62: Dashboard ................................................................................................................. 81

Figure 63: Bookmarks Pane ....................................................................................................... 82

Figure 64: Execution History Pane ............................................................................................. 83

Figure 65: Throughput Summary Report (Upper) ...................................................................... 84

Figure 66: Throughput Summary Report (Lower) ...................................................................... 84

Figure 67: Execution History Pane ............................................................................................. 85

Figure 68: Latency Summary Report (Upper) ............................................................................ 86

Figure 69: Latency Summary Report (Lower) ............................................................................ 86

Figure 70: Execution History Pane ............................................................................................. 87

Figure 71: Robustness Summary Report (Upper) ..................................................................... 88

Figure 72: Robustness Summary Report (Lower) ..................................................................... 88

Figure 73: Execution History-Individual Test Results ................................................................ 89

Figure 74: Test Execution Results (Upper) ................................................................................ 90

Figure 75: Test Execution Results (Lower) ................................................................................ 90

Figure 76: Reports Screen ......................................................................................................... 91

Figure 77: Search Options ......................................................................................................... 92

Figure 78: Report-Specific Period .............................................................................................. 92

Figure 79: Download Report ...................................................................................................... 93

Figure 80: PDF Report ............................................................................................................... 94

1 ABOUT PKT BLASTER SDN

CONTROLLER TEST

PktBlaster SDN Controller Test is the industry’s first integrated test solution for benchmarking the performance of an SDN controller as well as emulating SDN networks in a single box. It supports a number of real-world application traffic for performance testing and emulation. It runs either on a Veryx hardware platform or on suitable commodity servers.

PktBlaster SDN Controller Test solution consists of two product components that are licensed separately:

 Network Emulator and  Performance Benchmarking

PktBlaster SDN Controller Test-Performance Benchmarking simulates up-to one thousand switches and millions of nodes and thereby significantly reduces dependency on large-scale physical network infrastructure for testing and benchmarking SDN controllers.

Some of the performance benchmarking capabilities include:

 Throughput and Robustness measurement – per second  Latency measurement – in milliseconds  Detailed Tx/Rx statistics of switch group network per iteration  Comprehensive reporting and analysis

PktBlaster SDN Controller Test – Network Emulator is available separately and it supports real-time emulation of large-scale SDN networks. Some of the emulation capabilities supported include:

 Emulation of upto 100 SDN devices and upto 100 endpoints with millions of

flows

 Generation of multiple traffic flows from a single simulated port.  Supports network flow path tracing and network failure simulation.  Detailed traffic and flow statistics.

P a g e | 11

PktBlaster SDN Controller Test-Performance Benchmarking User Guide

For more details, please refer to the PktBlaster SDN Controller Test Network Emulator User Guide.

Key Features

The following are some of the key features of PktBlaster SDN Controller Test for performance benchmarking:

 Measurement of Performance Throughput, Latency and Robustness:

 For OpenFlow versions 1.0, 1.3 or both

 Uses TCP connection mode

 Programmatic Support (REST API)  Comprehensive Reporting and Analysis

Working with PktBlaster SDN Controller Test

PktBlaster SDN Controller Test runs either on Veryx Hardware or on standard commodity servers. The platform allows easy configuration of controllers and supports easy web-based access.

User Interface

PktBlaster SDN Controller Test supports user friendly GUI for configuration and testing. Also, PktBlaster SDN Controller GUI supports creation of various real-world application data for testing the performance and functionality.

Troubleshooting

PktBlaster SDN Controller Test provides extensive debugging options at multiple levels. It provides detailed statistics of the requests and responses for a test run for quick identification of problem root cause.

About Pkt Blaster SDN Controller Test

Reporting

PktBlaster SDN Controller Test report provides comprehensive insight on controller metrics such as flow set-up rate and flow set-up time in the form of graphs.

The individual test report provides OF connection time and detailed statistics of switch group network.

Figure 1: PktBlaster SDN Controller Test-Performance Benchmarking Deployment

PktBlaster SDN Controller Test-Performance Benchmarking User Guide

SDN Controller Test Process Flow

This user guide enables users to:

 Configure controllers and test parameters in PktBlaster SDN Controller Test  Execute tests to measure the performance of controllers

The process flow to configure and execute performance tests is depicted in the following diagrams.

Configure Controller

It is possible to add, modify, and delete a controller when needed.

Configure Traffic Profile

It is possible to create, modify, and delete a traffic profile when needed.

About Pkt Blaster SDN Controller Test

Execute Test

The procedure to execute a test involves the following phases.

Intended Audience for this Guide

This guide is intended to help the following categories of users who need to perform testing of SDN Controllers.

 Equipment Vendors

 Development Testing

 QA Testing

 Service Providers

 Proof of Concept

 Acceptance Testing

 Enterprise IT

 Piloting

 Benchmarking

A user can configure multiple controllers. However, the current version only allows one

controller to be tested at a time.

PktBlaster SDN Controller Test-Performance Benchmarking User Guide

This page has been intentionally left blank.

2 GETTING STARTED

This chapter describes how to log in to PktBlaster SDN Controller Test and access the features.

To log in to PktBlaster SDN Controller Test,

1. Type the PktBlaster SDN Controller Test IP address in the address bar of the

browser. The PktBlaster SDN Controller Test Login screen is displayed.

Figure 2: PktBlaster SDN Controller Test Login Screen

2. Enter the user name in the User Name text box.

3. Enter the password in the Password text box.

4. Click . On successful authentication, the Home screen is displayed.

In a new installation, the default User Name is admin and the Password is admin.

P a g e | 77

Figure 3: Home Screen

PktBlaster SDN Controller Test-Performance Benchmarking User Guide

MODULE

TAB

THIS WHERE YOU CAN

Dashboard

View the status of controller tests that were performed in the previous week.

Controllers

Add new controller information. Modify and delete existing controller information.

Traffic Profile

Add a new traffic profile.

Modify and delete an existing traffic profile.

Emulation

Execute Emulation tests.

Performance

Configure and execute Performance tests.

In a new installation, no data is displayed in the Dashboard. After the tool has been used to test controllers, a dashboard showing the results of testing is displayed (as shown in the View the Dashboard section).

About the Home Screen

The tabs in the PktBlaster SDN Controller Test allow testers to configure and perform tasks related to setting up and executing tests. The following table shows the tabs that you will see in the Home screen (Figure 3). To learn more about a module, click on the corresponding link.

Table 1: Home Screen Tabs

Getting Started

MODULE

TAB

THIS WHERE YOU CAN

Reports

View reports of selected tests and generate reports for a specific period.

The Emulation tab is disabled if the Emulation Test feature has not been installed.

Log out of PktBlaster SDN Controller Test

To exit from PktBlaster SDN Controller Test:

1. Click the Logout icon that is displayed above the user information as shown

in the PktBlaster SDN Controller Test-Logout screen.

Figure 4: PktBlaster SDN Controller Test Logout Screen

The Logout Confirmation dialog box is now displayed.

Figure 5: Logout Confirmation Dialog Box

2. Click Yes to log out of PktBlaster SDN Controller Test.

PktBlaster SDN Controller Test-Performance Benchmarking User Guide

This page has been intentionally left blank.

3 CONFIGURING AN SDN CONTROLLER

This chapter describes how you can configure a SDN controller in PktBlaster SDN Controller Test.

An SDN controller can be tested for performance throughput, latency and robustness. To perform these tests, you need to add the controller to the tool and the controller parameters need to be configured.

To test the performance of a controller,

 The controller needs to be connected to the network  The controller should be reachable through the port of the tool

To configure the parameters of a controller, click the Controllers tab. The Controllers screen is now displayed.

In the Controllers screen, you can:

 View the details of the controllers that have been added  Add the profile of a new controller

P a g e | 1111

Figure 6: Controller Screen

PktBlaster SDN Controller Test-Performance Benchmarking User Guide

 Modify the parameters of an existing controller  Remove a controller

Add SDN Controllers

To add a new test controller,

1. In the Controllers screen, click . The Add a New Controller screen

(Figure 7) is now displayed.

Figure 7: Add a new Controller

The fields in the Add a New Controller screen are described in the following table.

Configuring an SDN Controller

FIELD NAME

DESCRIPTION

Name

Controller name

Software Build Version

The version number that relates to the date and time of the build (in this case the build version of the controller that is tested).

IP Address

The IP Address of the server where the controller is installed (the controller runs its applications on this server).

Controller Port

The port number at which the controller listens. The default port number is 6633. This can be set to a maximum value of 65535.

Mode of Connection

The mode of connectivity between the switches and the controller. The TCP connection mode is supported.

Open Flow Version

The version number of the OpenFlow protocol (protocol used by the SDN controller to communicate with the switches) that is supported by the controller. The protocol versions supported are 1.0, 1.3 and both versions (1.0, 1.3).

Table 2: Add a New Controller Fields

All fields that have an beside them are mandatory.

2. After you have entered the relevant information, click to save the

controller details

The new controller is now added to the tool as shown in Figure 8.

PktBlaster SDN Controller Test-Performance Benchmarking User Guide

Figure 8: New Controller Added

View Available SDN Controllers

To view a list of available controllers, in the Home screen, click the Controllers tab. The Available Controllers screen is now displayed.

Configuring an SDN Controller

Figure 9: Available Controllers

To view the details of a controller, point to the controller with your cursor. The details of the controller are now displayed in a pop-up as shown in the Available Controllers screen (Figure 9).

Key features to Customize Controller View

The tabular information relating to controller configuration is displayed as shown in the Controller Configuration Screen.

PktBlaster SDN Controller Test-Performance Benchmarking User Guide

Figure 10: Controller Configuration Screen

In the Controller Configuration screen, you can customize the information that is displayed, using the features shown in the Controller Configuration-Customize Display screen.

Configuring an SDN Controller

Figure 11: Controller Configuration-Customize Display

The information can be customized based on the following:

 Grouping: Enables you to group the information based on a specific field.  Customizable Columns: Enables you to select the fields that you want, to be

displayed.

Edit SDN Controller Details

To modify the parameters of a controller that has been added,

1. Click the Controllers tab to display the Available Controllers screen (Figure

9).

2. From the list of controllers that is displayed, select the controller that you

want to modify. The details of the selected controller are displayed in the relevant fields as shown in the Controller Details screen (Figure 12).

PktBlaster SDN Controller Test-Performance Benchmarking User Guide

Figure 12: Controller Details

3. Click to display the fields in editable mode as shown in the Modify

Controller Details screen.

Figure 13: Modify Controller Details

4. Make the necessary changes and click to save the changes.

Click to discard the changes made to the controller details.

Configuring an SDN Controller

Delete SDN Controller

You can only delete a controller that has not been tested.

To delete a controller,

1. Click the Controllers tab to display the Controllers screen (Figure 6).

2. From the list of controllers that is displayed, select the controller that you

want to delete. The details of the selected controller are displayed in the relevant fields as shown in the Controller Details screen (Figure 12).

3. Click . The Delete Confirmation dialog box is now displayed.

Figure 14: Delete Confirmation Dialog Box

4. In the Delete Confirmation dialog box, click Yes to delete the controller.

Click No to stop deleting the controller.

PktBlaster SDN Controller Test-Performance Benchmarking User Guide

This page has been intentionally left blank.

4 CONFIGURING A TRAFFIC PROFILE

This chapter describes how you can configure the type of traffic that needs to be sent to the controller. Predefined traffic types such as ARP_Reply, ARP_Request, TCP, and UDP are available in Traffic profile module. It is possible to select or modify the existing traffic types and create new traffic types.

These traffic types are used for the packet-in messages that are sent to the controller. After a traffic profile is created, it can be used to test the performance of a controller.

To configure the traffic that needs to be generated, click the Traffic Profile tab. The Traffic Profile Screen displays the predefined traffic types ARP_Reply, ARP_Request, TCP and UDP.

Figure 15: Traffic Profile Screen

P a g e | 2121

PktBlaster SDN Controller Test-Performance Benchmarking User Guide

In the Traffic Profile screen, you can view the existing profiles that have been defined. When you click on a traffic profile, you can view the field details and the modifier details for that profile.

In the Traffic Profile screen, you can:

 Add a new traffic profile  View traffic profiles  Modify an existing traffic profile  Delete a traffic profile

Add a New Traffic Profile

To define a new traffic profile:

1. In the Traffic Profile screen, click . The Add a New Traffic Profile screen

is now displayed.

Figure 16: Add a New Traffic Profile

Configuring a Traffic Profile

PARAMETER

DESCRIPTION

Offset

The starting position in the field from which the mask is applied. This indicates the position (starting from 1) from which the modifier is applied.

Mask

The value corresponding to the number of byte(s) that are modified and the offset at which the modifier is applied. For example, when 1 byte of a field (of length 4 bytes) is modified, the mask value that is applied from an offset position 3 is 00 00 ff 00.

Count

The number of combinations that are generated with a specified mask value. For example, when 1 byte of a field is modified, the maximum number of unique values that will be generated is 255.

In the Add a New Traffic Profile screen, you can view the following:

 The field details for the packets.  The modifier information for the relevant fields.

In the Add a New Traffic Profile, screen (Figure 16):

1. In the Profile Name text box, enter a suitable name that uniquely identifies

the traffic type associated with the profile. For example, to define a Layer 3 custom traffic, you could use a profile name such as L3_CUSTOM_1.

2. In the Packet Length box, type or select the length of each packet in the

traffic profile. The length of a packet (includes the header and data parts) can vary from a minimum of 64 bytes to a maximum of 512 bytes.

In the Field Details section:

The values in the fields can be modified by means of the parameters shown in the following table.

Table 3: Fields and Associated Modifiers

+ 76 hidden pages

Veryx PktBlaster User Manual

Specifications and Main Features

Frequently Asked Questions

User Manual