Juniper Multichassis Link Aggregation User Manual

Junos® OS

c ss s Link r n User Guide for EX Series, MX Series, and QFX Series Devices

Published

2021-04-23

ii

Juniper Networks, Inc. 1133 nn v n Way Sunnyvale, California 94089 USA

408-745-2000 www.juniper.net

Juniper Networks, the Juniper Networks logo, Juniper, and Junos are registered trademarks of Juniper Networks, Inc. in the United States and other countries. All other trademarks, service marks, registered marks, or registered service marks are the property of their r s c v owners.

Juniper Networks assumes no responsibility for any inaccuracies in this document. Juniper Networks reserves the right

to change, modify, transfer, or otherwise revise this b c				n without n c
Junos® OS M	c ss s Link	r	n User Guide for EX Series, MX Series, and QFX Series Devices
Copyright © 2021 Juniper Networks, Inc. All rights reserved.
The n rm	n in this document is current as of the date on the				page.

YEAR 2000 NOTICE

Juniper Networks hardware and s w r products are Year 2000 compliant. Junos OS has no known m r

m ns through the year 2038. However, the NTP c n is known to have some c y in the year 2036.

END USER LICENSE AGREEMENT

The Juniper Networks product that is the subject of this technical						c m n	n consists of (or is intended for use
with) Juniper Networks s w r	Use of such s		w r	is subject to the terms and c n				ns of the End User License
Agreement ("EULA") posted at	s s	r	n r n	s	r	. By downloading, installing or using such
s w r you agree to the terms and c n		ns of that EULA.

iii

Table of Contents

About This Guide | xi

1Understanding MC-LAG

Understanding M		c	ss s Link		r	n Groups | 2
MC-LAG Concepts | 4
Advanced MC-LAG Concepts | 7
	Understanding C	n	r	n Sync	r n z	n | 8
	Understanding M	c	ss s Link		r	n Group C n r n Consistency Check | 13
	Unknown Unicast and IGMP Snooping | 28
	Layer 3 Unicast Feature Support | 29
	MAC Address Management | 29
	MAC Aging | 30
	Address R s	n Protocol c v			c v	MC-LAG Support Methodology | 30
	DHCP Relay with		n 82 | 30
	Layer 2 Unicast Features Supported | 32
	Protocol Independent M			c s | 32
	IGMP Report Sync r n z			n | 34
	IGMP Snooping in MC-LAG			c v	c v	Mode | 34
	Understanding MC-LAGs on an FCoE Transit Switch | 41
	Understanding EVPN-MPLS Interworking with Junos Fusion Enterprise and MC-LAG | 45
	Understanding the Incremented Values of S s c Counters for Loop-Free MC-LAG
	Networks | 52
	Enhanced Convergence | 56
	IPv6 Neighbor Discovery Protocol | 57
	Anycast Gateways | 58

iv

2m m n n MC-LAG

G n Started with MC-LAG | 65

C n		r n	M	c	ss s Link	r	n on MX Series Routers | 65
C	n	r n	M	c	ss s Link	r	n on EX Series Switches | 72
C	n	r n	ICCP for MC-LAG | 78

MC-LAG Examples | 80

Example: C			n	r n	M	c	ss s Link	r
	Requirements | 81
	Overview | 81
	C n		r	n | 82
	r	c	n | 101
	r	b	s	n	| 104
Example: C			n	r n	M	c	ss s Link	r
	Requirements | 105
	Overview | 106
	C n		r n	the PE Routers | 107
	C n		r n	the CE Device | 117
	C n		r n	the Provider Router | 121
	r	c	n | 125
Example: C			n	r n	M	c	ss s Link	r
	Series Routers | 125
	Requirements | 126
	Overview | 126
	C n		r the Devices | 128
	r	c	n | 146

n on the QFX Series | 80

n on the MX Series | 105

n Between QFX Series Switches and MX

Example: C			n	r n	M	c ss s Link	r	n on EX9200 Switches in the Core for
	Campus Networks | 150
	Requirements | 151
	Overview | 151
	C n		r	n | 154
	(	n	) C n		r n	RSTP | 180
	(	n	) C n		r n	IGMP Snooping | 182

v

(

n

) C n

r n

VRRP | 184

(

n

) C n

r n

MAC Address Sync r n z

n | 187

(

n

) C n

r n

OSPF | 188

(

n

) C

n

r n

PIM | 190

(

n

) C n

r n

DHCP Relay | 193

r

c

n | 196

Example: C n

r

n

Features For M

c

ss s Link

r

n | 209

Requirements | 210

Overview | 210

(

n

) C n

r n

RSTP | 211

(

n

) C n

r n

IGMP Snooping | 214

(

n

) C n

r n

VRRP | 215

(

n

) C n

r n

MAC Address Sync r n z

n | 218

(

n

) C n

r n

OSPF | 219

(

n

) C

n

r n

PIM | 220

(

n

) C n

r n

DHCP Relay | 224

r

c

n | 227

r

b s

n

| 228

Example: Simplifying M

c

ss s Link

r

n on EX9200 Switches in the Core for

Campus Networks | 229

Requirements | 229

Overview | 230

C n

r

n | 233

r

c

n | 263

Example: C n

r n

CoS for FCoE Transit Switch

r

c Across an MC-LAG | 291

Requirements | 292

Overview | 292

C n

r

n | 299

r

c

n | 312

Example: EVPN-MPLS Interworking With an MC-LAG Topology | 325

Requirements | 326

Overview and Topology | 326

PE1 and PE2 C n

r

n | 329

PE3 C n

r

n | 346

vi

3

Other MC-LAG C n

r

ns

Other MC-LAG C n

r

ns | 353

C n

r n

c

v c

v

Bridging and VRRP over IRB in M c ss s Link r

n on MX

Series Routers | 353

C n

r n

MC-LAG | 354

C

n

r n

the Interchassis

n

r

c

n Link | 355

C n

r n

M

Chassis | 355

C

n

r n

the Service ID | 356

C n

r n

IGMP Snooping for c

v

c

v MC-LAG | 358

C	n	r n	IGMP Snooping in MC-LAG c v c v Mode | 359
C	n	r n	Manual and	m c Link Switchover for MC-LAG Interfaces on MX Series
	Routers | 360

Forcing MC-LAG Links or Interfaces with Limited LACP Capability to Be Up | 362

Increasing ARP and Network Discovery Protocol Entries for Enhanced MC-LAG and Layer 3

VXLAN Topologies | 363

Understanding the Need for an Increase in ARP and Network Discovery Protocol (NDP)

Entries | 364

Increasing ARP and Network Discovery Protocol Entries for Enhanced MC-LAG Using IPv4

Transport | 364

Increasing ARP and Network Discovery Protocol Entries for Enhanced MC-LAG Using IPv6

Transport | 366

Increasing ARP for EVPN-VXLAN Gateway for Border-Leaf in Edge Routed Bridge (ERB) or

Spine in Centrally Routed Bridge (CRB) for IPv4 Tenant r c | 368

Increasing ARP and Network Discovery Protocol Entries for EVPN-VXLAN gateway for

Border-Leaf in Edge Routed Bridge (ERB) or Spine in Centrally Routed Bridge (CRB) for

IPv6 Tenant r c | 370

Synchronizing and C mm				n	C n	r	ns | 373
	C n	r	Devices for C n		r	n Sync		r n z n | 373
	Create a Global C n			r	n Group | 376
	Create a Local C n			r	n Group | 379
	Create a Remote C n			r	n Group | 382
	Create Apply Groups for the Local, Remote, and Global C n r ns | 384
	Synchronizing and C mm				n	C n	r	ns | 385
	r	b s	n Remote Device C nn c					ns | 385

4

5

6

vii

MC-LAG Upgrade Guidelines

MC-LAG Upgrade Guidelines | 390

Best r c c s and Usage Notes

Best r c c s and Usage Notes | 393

r	b	s	n	M	c ss s Link		r	n
r	b	s	n M	c	ss s Link	r	n | 405
	MAC Addresses Learned on M c					ss s Aggregated Ethernet Interfaces Are Not Removed
		from the MAC Address Table | 406
	MC-LAG Peer Does Not Go into Standby Mode | 407
	Secondary MC-LAG Peer with Status Control Set to Standby Becomes n c v | 408
	Redirect Filters Take Priority over					s r	n Filters	| 408

rn Command Output Is Wrong | 409

ICCP C nn c n Might Take Up to 60 Seconds to Become						c v	| 410
MAC Address Age Learned on a M				c ss s Aggregated Ethernet Interface Is Reset to Zero | 411
MAC Address Is Not Learned Remotely in a Default VLAN | 412
Snooping Entries Learned on M c				ss s Aggregated Ethernet Interfaces Are Not Removed | 412
ICCP Does Not Come Up			r You Add or Delete an			n c	n Key | 413
Local Status Is Standby When It Should Be c v					| 414
Packets Loop on the Server When ICCP Fails | 414
Both MC-LAG Peers Use the Default System ID					r a Reboot or an ICCP C n r n
Change | 415
No Commit Checks Are Done for ICL-PL Interfaces | 416
Double Failover Scenario | 416
M c s	r	c Floods the VLAN When the ICL-PL Interface Goes Down and Up | 417
Layer 3	r	c Sent to the Standby MC-LAG Peer Is Not Redirected to c v MC-LAG Peer | 418
Aggregated Ethernet Interfaces Go Down | 418
Flooding of Upstream r			c | 419

viii

ARP and MAC Table Entries Become Out of Sync in an MC-LAG C n r n | 420

	C n		r n	Interface			n s		cs Tools to Test the Physical Layer C nn c ns | 421
		C n		r n	Loopback		s	n	| 421
		C n		r n	BERT s		n	| 424
		S r	n	and Stopping a BERT Test | 428
7	C n		r		n Statements
	apply-groups | 432
	arp-enhanced-scale | 433
	arp-l2-validate | 435
		n	c	n		y (ICCP) | 436
	b c		v n ss			c	n | 438
	backup-peer-ip | 440
	bgp-peer | 441
	chassis-id | 442
		c	n	m	(Liveness				c n) | 444
	enhanced-convergence | 445
	groups | 447
	iccp | 451
	iccp | 453
	interface (M				c	ss s	r	c	n) | 455

local-ip-addr (ICCP) | 457 mc-ae | 458

mc-ae-id | 464 mclag | 466

mclag-arp-nd-sync | 467

ix

mclag-arpreq-sync | 469
minimum-interval (Liveness						c	n) | 470
minimum-receive-interval (Liveness							c n) | 472
mode (QFX Series) | 473
m		r (Liveness			c	n) | 475
m	c	ss s | 476
m	c	ss s	r	c	n | 478
m	c	ss s	r	c	n | 479

n		n (Liveness		c	n) | 481
peer (ICCP) | 482
peer (M	c	ss s) | 484
peer | 485
peers (Commit) | 487
peers-synchronize | 489
status-control | 490
s ss n	s	b s m n		m	| 492
threshold (		c	n Time) | 493
transmit-interval (Liveness				c	n) | 495
version (Liveness			c n) | 496

8	r	n Commands
	request interface mc-ae switchover (M c ss s Link		r	n) | 500
	request interface (revert | switchover) (Aggregated Ethernet Link			r c n) | 502
	request lacp link-switchover | 504
	show iccp | 506
	show interfaces mc-ae | 509

x

show l2-learning redundancy-groups | 513
show m	c	ss s mc-lag c n	r	n c	ns s	ncy list-of-parameters | 520
show m	c	ss s mc-lag c n	r	n c ns s		ncy | 535
show m	c	ss s mc-lag c n	r	n c ns s		ncy	b	c n	| 542
show m	c	ss s mc-lag c n	r	n c	ns s	ncy c	c n	| 545
show m	c	ss s mc-lag c n	r	n c ns s		ncy mc	c n		| 548
show m	c	ss s mc-lag c n	r	n c ns s		ncy v n c n			| 553
show m	c	ss s mc-lag c n	r	n c	ns s	ncy vrr	c	n	| 557

xi

About This Guide

Use this guide to c n r and monitor m c ss s link

r

n groups (MC-LAGs).

1

CHAPTER

Understanding MC-LAG

Understanding M c ss s Link

r

n Groups | 2

MC-LAG Concepts | 4

Advanced MC-LAG Concepts | 7

2

Understanding M c ss s Link	r	n
Groups

IN THIS SECTION

B n s of MC-LAGs | 3

NOTE: S r n in Junos OS Release 17.4R1, MC-LAG is not supported on EX switches except for EX4600, EX4650-48Y, and EX9200 switches. Use the Virtual Chassis feature instead to provide equivalent nc n y

Layer 2 networks are increasing in scale mainly because of technologies such as v r			z	n. Protocol
and control mechanisms that limit the disastrous	c s of a topology loop in the network are
necessary. The Spanning Tree Protocol (STP) is the primary s		n to this problem because it provides

a loop-free Layer 2 environment. STP has gone through a number of enhancements and extensions, and

even though it scales to very large network environments, it s only provides one c v								path from one
device to another, regardless of how many actual c						nn c	ns might exist in the network. Although STP
is a robust and scalable s				n to redundancy in a Layer 2 network, the single logical link creates two
problems: At least half of the available system bandwidth is							m s to data r c and network
topology changes occur. The Rapid Spanning Tree Protocol (RSTP) reduces the overhead of the
rediscovery process and allows a Layer 2 network to reconverge faster, but the delay is s								high.
Link	r	n (IEEE 802.3ad) solves some of these problems by enabling users to use more than one
link c	nn c	n between switches. All physical c nn				c ns are considered one logical c		nn c n The
problem with standard link				r	n is that the c	nn c	ns are point to point.
M	c ss s link		r	n groups (MC-LAGs) enable a client device to form a logical LAG interface

between two MC-LAG peers. An MC-LAG provides redundancy and load balancing between the two MC-LAG peers, m m n support, and a loop-free Layer 2 network without running STP.

On one end of an MC-LAG, there is an MC-LAG client device, such as a server, that has one or more physical links in a link r n group (LAG). This client device uses the link as a LAG. On the other side of the MC-LAG, there can be a maximum of two MC-LAG peers. Each of the MC-LAG peers has one or more physical links connected to a single client device.

The MC-LAG peers use the Inter-Chassis Control Protocol (ICCP) to exchange control n rm	n and
coordinate with each other to ensure that data r c is forwarded properly.

3

The Link	r	n Control Protocol (LACP) is a subcomponent of the IEEE 802.3ad standard. LACP is
used to discover m			links from a client device connected to an MC-LAG peer. LACP must be
c n r	on both MC-LAG peers for an MC-LAG to work correctly.
NOTE: You must specify a service n r (service-id) at the global level; otherwise,
m	c ss s link	r	n will not work.

Figure 1: Basic MC-LAG Topology

The following s		c	ns provide n rm	n regarding the nc n behavior of m c ss s link
r	n c	n	r n guidelines, and best r c c s

B n s of MC-LAGs

• Reduces	r	n expenses by providing c v c v links within a Link	r	n Group
(LAG).

•Provides faster layer 2 convergence upon link and device failures.

•Adds node-level redundancy to the normal link-level redundancy that a LAG provides.

• Improves network resiliency, which reduces network down m as well as expenses.

4

MC-LAG Concepts

IN THIS SECTION
	ICCP and ICL | 4	r c n |	4
	M c ss s Link
	Service ID | 5
	Failure Handling |	5
	Load Balancing |	6
	MC-LAG Packet Forwarding |		6
	Virtual Router Redundancy Protocol (VRRP) over IRB and MAC Address Sync r n z n | 6

ICCP and ICL

The MC-LAG peers use the Inter-Chassis Control Protocol (ICCP) to exchange control n rm n and coordinate with each other to ensure that data r c is forwarded properly. ICCP replicates control

r c and forwarding states across the MC-LAG peers and communicates the r n state of the MC-LAG members. Because ICCP uses TCP/IP to communicate between the peers, the two peers must

be connected to each other. ICCP messages exchange MC-LAG c n	r	n parameters and ensure
that both peers use the correct LACP parameters.
The interchassis link (ICL), also known as the interchassis n r c	n link (ICL-PL), is used to forward

data r c across the MC-LAG peers. This link provides redundancy when a link failure (for example, an MC-LAG trunk failure) occurs on one of the c v links. The ICL can be a single physical Ethernet interface or an aggregated Ethernet interface.

You can c n r m addresses. You can c n

ICLs between MC-LAG peers. Each ICL can learn up to 512K MAC

rn ICLs for virtual switch instances.

M c ss s Link r c n

M c ss s link r	c
LAG. If the ICCP c	nn c

n provides link r c n between the two MC-LAG peers that host an MC- n is up and the ICL comes up, the peer c n r as standby brings up the

5

m	c	ss s aggregated Ethernet interfaces shared with the peer. M c ss s r c n must be
c n	r	on each MC-LAG peer that is s n an MC-LAG.

Service ID

You must c n	r the same service ID on each MC-LAG peer when the MC-LAG logical interfaces are
part of a bridge domain. The service ID, which is c n		r	under the sw c	ns hierarchy, is used
to synchronize	c ns such as IGMP, ARP, and MAC learning across MC-LAG members. If you are
c n r n virtual switch instances, c n r a		r n	service ID for each virtual switch instance.

Failure Handling

C n r n ICCP adjacency over an aggregated interface with child links on m FPCs m s the possibility of a split-brain state. A split-brain occurs when ICCP adjacency is lost between the MC-LAG peers. To work around this problem, enable backup liveness c n With backup liveness c n enabled, the MC-LAG peers establish an out-of-band channel through the management network in

n to the ICCP channel.

During a split-brain state, both c v and standby peers change LACP system IDs. Because both MCLAG peers change the LACP system ID, the customer edge (CE) device accepts the LACP system ID of

the rs link that comes up and brings down other links carrying	r n LACP system IDs. When the
ICCP c nn c n is c v both of the MC-LAG peers use the c n	r LACP system ID. If the LACP

system ID is changed during failures, the server that is connected over the MC-LAG removes these links from the aggregated Ethernet bundle.

When the ICL is	r n y down and the ICCP c nn c n is c v the LACP state of the links with
status control c n	r as standby is set to the standby state. When the LACP state of the links is

changed to standby, the server that is connected over the MC-LAG makes these links n c v and does not use them for sending data.

Recovery from the split-brain state occurs	m c y when the ICCP adjacency comes up between
MC-LAG peers.

If only one physical link is available for ICCP, then ICCP might go down due to link failure or FPC failure,

while the peer is s up. This results in a split-brain state. If you do not set a special c n		r	n to
avoid this s	n the MC-LAG interfaces change the LACP sytem ID to their local defaults, thus

ensuring that only one link (the rs ) comes up from the downstream device. A convergence delay results from the LACP state changes on both c v and standby peers.

6

Load Balancing

Load balancing of network r network r c between m LAG hashing algorithm.

c between MC-LAG peers is 100 percent local bias. Load balancing of LAG members in a local MC-LAG node is achieved through a standard

MC-LAG Packet Forwarding

To prevent the server from receiving m		copies from both of the MC-LAG peers, a block mask is
used to prevent forwarding of r	c received on the ICL toward the m c ss s aggregated Ethernet
interface. r v n n forwarding of	r	c received on the ICL interface toward the m c ss s

aggregated Ethernet interface ensures that r c received on MC-LAG links is not forwarded back to the same link on the other peer. The forwarding block mask for a given MC-LAG link is cleared if all of the local members of the MC-LAG link go down on the peer. To achieve faster convergence, if all local members of the MC-LAG link are down, outbound r c on the MC-LAG is redirected to the ICL interface on the data plane.

Virtual Router Redundancy Protocol (VRRP) over IRB and MAC Address

Sync r n z	n
There are two methods for enabling Layer 3 r			n	nc	n y across a m c ss s link	r	n
group (MC-LAG). You can choose either to c		n	r the Virtual Router Redundancy Protocol (VRRP)
over the integrated r	n and bridging (IRB) interface or to synchronize the MAC addresses for the
Layer 3 interfaces of the switches r c		n	in the MC-LAG.
VRRP over IRB or RVI requires that you c n			r	r n	IP addresses on IRB or RVI interfaces, and

run VRRP over the IRB or RVI interfaces. The virtual IP address is the gateway IP address for the MCLAG clients.

If you are using the VRRP over IRB method to enable Layer 3 nc n	y you must c n r s c
ARP entries for the IRB interface of the remote MC-LAG peer to allow r	n protocols to run over the

IRB interfaces. This step is required so you can issue the ping command to reach both the physical IP addresses and virtual IP addresses of the MC-LAG peers.

For example, you can issue the set interfaces irb unit 18 family inet address 10.181.18.3/8 arp 10.181.18.2 mac 00:00:5E:00:2f:f0 command.

7

When you issue the show interfaces irb see that the s c ARP entries are n

command r you have c n r VRRP over IRB, you will n to the IRB MAC addresses of the remote MC-LAG peer:

user@switch> show interfaces irb

Physical interface: irb, Enabled, Physical link is Up

Interface index: 180, SNMP ifIndex: 532

Type: Ethernet, Link-level type: Ethernet, MTU: 1514

Device flags	: Present Running
Interface flags: SNMP-Traps
Link type	: Full-Duplex
Link flags	: None
Current address: 00:00:5E:00:2f:f0, Hardware address: 00:00:5E:00:2f:f0
Last flapped	: Never
Input packets : 0
Output packets: 0
MAC address sync	r n z	n enables MC-LAG peers to forward Layer 3 packets arriving on

m c ss s aggregated Ethernet interfaces with either their own IRB or RVI MAC address or their peer’s IRB or RVI MAC address. Each MC-LAG peer installs its own IRB or RVI MAC address as well as the peer’s IRB or RVI MAC address in the hardware. Each MC-LAG peer treats the packet as if it were its own packet. If MAC address sync r n z n is not enabled, the IRB or RVI MAC address is installed on the MC-LAG peer as if it were learned on the ICL.

MAC address sync r n z n requires you to c n r the same IP address on the IRB interface in the VLAN on both MC-LAG peers. To enable the MAC address sync r n z n feature using the standard CLI, issue the set vlan vlan-name mcae-mac-synchronize command on each MC-LAG peer. If you are using the Enhanced Layer 2 CLI, issue the set bridge-domains name mcae-mac-synchronize command on each MC-LAG peer. C n r the same IP address on both MC-LAG peers. This IP address is used as the default gateway for the MC-LAG servers or hosts.

Advanced MC-LAG Concepts

IN THIS SECTION
	Understanding C n	r	n Sync	r n z	n | 8
	Understanding M	c	ss s Link	r	n Group C n r n Consistency Check | 13

Unknown Unicast and IGMP Snooping | 28

8

Layer 3 Unicast Feature Support | 29
MAC Address Management | 29
MAC Aging | 30
Address R s	n Protocol		c v	c v	MC-LAG Support Methodology | 30
DHCP Relay with	n 82		| 30
Layer 2 Unicast Features Supported | 32
Protocol Independent M		c s | 32
IGMP Report Sync r n z			n | 34
IGMP Snooping in MC-LAG			c v	c v	Mode | 34

Understanding MC-LAGs on an FCoE Transit Switch | 41

Understanding EVPN-MPLS Interworking with Junos Fusion Enterprise and MC-LAG | 45

Understanding the Incremented Values of S s c Counters for Loop-Free MC-LAG Networks | 52

Enhanced Convergence | 56

IPv6 Neighbor Discovery Protocol | 57

Anycast Gateways | 58

Understanding C n r

n Sync r n z

n

IN THIS SECTION
	B n	s of C n		r	n Sync r n z		n |	9
	How C n		r	n Sync r n z		n Works | 9
	How to Enable C n				r n Sync r n z			n | 9
	How C n		r	n Sync r n z		n is Supported | 10				ns | 10
	C n	r	n Groups for Local, Remote and Global C						n r
	Cr	n C	n	n	Groups for Certain Devices | 10
	Applying C n			r	n Groups | 10					n | 11
	Device C n		r	n Details for C		n	r	n Sync r	n z
	How C n		r	ns and Commits Are Synchronized Between Devices | 12

9

C n r n sync r n z	n works on QFX Series switches, Junos Fusion Provider Edge, Junos Fusion
Enterprise, EX Series switches, and MX Series routers.
This topic describes:

B n	s of C n	r	n Sync r n z	n
C n r	n sync	r n z	n enables you to propagate, synchronize, and commit c n r ns from

one device to another. You can log into any one of those devices to manage all devices, thus having a single point of management.

How C n				r	n Sync r n z			n Works
Use c		n		r n groups to simplify the c				n r n process. For example, you can create one
c	n	r		n group for the local device, one or more for the remote devices, and one for the global
c	n	r		n which is ss n		y a c	n	r n that is common to all devices.
In			n you can create c n			n	groups to specify when a c n r n is synchronized with

another device. You can enable the peers-synchronize statement at the [edit system commit] hierarchy

to synchronize the c	n r	ns and commits across the devices by default. NETCONF over SSH
provides a secure c	nn c	n between the devices, and Secure Copy Protocol (SCP) copies the
c n r ns securely between them.

How to Enable C n					r	n Sync r n z	n
To enable c		n	r	n sync r		n z n perform the following steps:
1.	S c y map the local device to the remote devices.
2.	Create c	n	r	n groups for local, remote, and global c n r ns
3.	Create c	n	n	groups.

4.Create apply groups.

5.Enable NETCONF over SSH.

6.	C n r the device details and user	n c	n details for c n r n sync r n z	n
7.	Enable the peers-synchronize statement or issue the commit peers-synchronize command to
	synchronize and commit the c n r	ns between local and remote devices.

10

How C n r n Sync r n z n is Supported

On MX Series routers and Junos Fusion, support for c n	r
OS Release 14.2R6. On QFX Series switches, support for c	n
Junos OS Release 15.1X53-D60.

n sync r n z	n started with Junos
r n sync	r n z	n started with

C n

r

n Groups for Local, Remote and Global C

n

r

ns

You can create c

n

r

n groups for local, remote and global c

n

r

ns A local c n r n

group is used by the local device, a remote c

n

r n group is used by the remote device, and a

global c

n

r n group is shared between the local and remote devices.

For example, you could create a local c

n

r

n group called Group A, which would include the

c n

r

n used by the local device (Switch A), a remote c n

r

n group called Group B, which

would include the c

n

r n used by remote devices (Switch B, Switch C, and Switch D), and a global

c n

r

n group called Group C, which would include the c n

r

n that is common to all

devices.

Create c

n

r n groups at the [edit groups] hierarchy level.

NOTE: C n

r

n sync r n z

n does not support nested groups.

Cr	n C n	n Groups for Certain Devices
You can create c n		n groups to specify when a r c r c n r n should be applied to a

device. If you want to apply the global c n r n to all devices in a four-device c n r n for example, enable the when peers [<name of local peer> <name of remote peer> <name of remote peer> <name of remote peer>] statement at the [edit groups] hierarchy level. If, for example, you want to apply the global c n r n (Group C) to the local and remote devices (Switch A, Switch B, Switch C, and Switch D), you could issue the set groups Group C when peers [Switch A Switch B Switch C Switch D] command.

Applying C n	r	n Groups
To apply c n r	n groups, enable the apply-groups statement at the [edit] hierarchy level. For
example, to apply the local c n r			n group (Group A, for example), remote c n r n group
(Group B, for example), and global c n			r n group (Group C, for example), issue the set apply-

groups [ GroupA GroupB GroupC ] command.

11

Device C n	r	n Details for C n	r	n Sync	r n z	n
To synchronize c	n r	ns between devices, you need to c n			r the hostname or IP address,

username, and password for the remote devices. To do this, issue the set peers <hostname-of-remote-

peer> user <name-of-user>				n	c	n <plain-text-password-string> command at the [edit system
commit] hierarchy on the local device.
For example, to synchronize a c				n	r	n from Switch A to Switch B, issue the set peers SwitchB user
administrator		n c	n test123 command on Switch A.
You also need to s		c y map the local device to the remote devices. To this, issue the set system
commit peers
For example, to synchronize a c				n	r	n from Switch A to Switch B, Switch C, and Switch D,
c n	r the following on Switch A:
Switch A
[edit system commit]
peers {
	switchB {
	user admin-swB;
	authentication			"$ABC123";
	}
	switchC {
	user admin-swC;
	authentication			""$ABC123";
	}
	switchD {
	user admin-swD;
	authentication			"$ABC123";
}	}

[edit system] static-host-mapping [

SwitchA{

inet [ 10.92.76.2 ];

}

SwitchB{

inet [ 10.92.76.4 ];

}

SwitchC{

inet [ 10.92.76.6 ];

}

12

SwitchD{

inet [ 10.92.76.8 ];

}

}

}

If you only want to synchronize c n

r ns from Switch A to Switch B, Switch C, and Switch D, you

do not need to c

n

r the peers statement on Switch B, Switch C, and Switch D.

The c

n

r

n details from the peers statements are also used to establish a NETCONF over SSH

c nn c

n between the devices. To enable NETCONF over SSH, issue the set system services netconf

ssh command on all devices.

How C n

r

ns and Commits Are Synchronized Between Devices

The local (or r q

s

n ) device on which you enable the peers-synchronize statement or issue the

commit peers-synchronize command copies and loads its c

n

r

n to the remote (or responding)

device. Each device then performs a syntax check on the c

n

r

n

being c

mm

If no errors

are found, the c

n

r

n is

c v

and becomes the current

r

n c n

r

n on all

devices. The commits are propagated using a remote procedural call (RPC).

The following events occur during c n

r

n sync r

n z

n

1. The local device sends the sync-peers.conf

(the c

n

r

n that will be shared with the devices

s

c

in the c

n

n

group) to the remote devices.

2. The remote devices load the c n

r n send the results of the load to the local device, export

their c

n

r

n to the local device, and reply that the commit is complete.

3. The local device reads the replies from the remote devices.

4.	If successful, the c		n r	n is c mm
C	n r	n sync r	n z	n is not successful if either a) the remote device is unavailable or b) the
remote device is reachable, but there are failures due to the following reasons:
•	SSH c	nn c n fails because of user and			n c	n issues.

• Junos OS RPC fails because a lock cannot be obtained on the remote database.

•	Loading the c n r n fails because of syntax problems.
•	Commit check fails.

The peers-synchronize statement uses the hostname or IP address, username, and password for the
devices you c n	r in the peers statement. With the peers-synchronize statement enabled, you can

simply issue the commit command to synchronize the c n r n from one device to another. For

13

example, if you c n r the peers statement on the local device, and want to synchronize the

c n r n with the remote device, you can simply issue the commit command on the local device. However, if you issue the commit command on the local device and the remote device is not reachable, you will receive a warning message saying that the remote device is not reachable and only the

	c n	r	n on the local device is c		mm
	Here is an example warning message:
	error: netconf: could not read hello
	error: did not receive hello packet from server
	error: Setting up sessions for peer: 'peer1' failed
	warning: Cannot connect to remote peers, ignoring it
	commit complete
	If you do not have the peers statement c n					r	with the remote device n rm			n and you issue the
	commit command, only the c n			r	n on the local device is c			mm	If the remote device is
	unreachable and there are other failures, the commit is unsuccessful on both the local and remote
	devices.
	NOTE: When you enable the peers-synchronize statement and issue the commit command, the
	commit might take longer than a normal commit. Even if the c							n r	n is the same across the
	devices and does not require sync				r n z	n the system s		m	s to synchronize the
	c	n	r ns
	The commit peers-synchronize command also uses the hostname or IP address, username, and
	password for the devices c n			r in the peers statement. If you issue the commit peers-synchronize
	command on the local device to synchronize the c						n r n with the remote device and the remote

device is reachable but there are other failures, the commit fails on both the local and remote devices.

Understanding M c ss s Link	r	n Group C n r	n
Consistency Check

IN THIS SECTION

B n s of Using MC-LAG Consistency Check | 14

How MC-LAG Consistency Checks Work | 14

14

C n r n Consistency Requirements | 15

When Remote Peers are Not Reachable | 15

Enabling MC-LAG C n	r	n Consistency Checking | 16
Learning the Status of a C	n	r n Consistency Check | 27
Support for MC-LAG C n	r	n Consistency Checking | 28

When there is a M	c ss s Link	r	n Group (MC-LAG) inconsistency, you are n			and can
take c n to resolve it. An example of an inconsistency is c n r n				n c	chassis IDs on both
peers instead of c n	r n unique chassis IDs on both peers. Only c			mm	MC-LAG parameters are
checked for consistency.

B n s of Using MC-LAG Consistency Check

• This feature helps you n c n r n r m r inconsistencies between m c ss s link

rn group (MC-LAG) peers.

How MC-LAG Consistency Checks Work

The following events take place during c				n	r	n consistency check				r you issue a commit on the
local MC-LAG peer:
1.	Commit an MC-LAG c n	r	n on the local MC-LAG peer.
2.	ICCP parses the MC-LAG c n		r	n and then sends the c n					r	n to the remote MC-LAG
	peer.
3.	The remote MC-LAG peer receives the MC-LAG c n						r	n from the local MC-LAG peer and
	compares it with its own MC-LAG c n				r	n
	If the there is a severe inconsistency between the two MC-LAG c								n	r ns the MC-LAG
	interface is brought down, and syslog messages are issued.
	If there is a moderate inconsistency between the two c						n	r	ns syslog messages are issued.
The following events take place during c				n	r	n consistency check				r you issue a commit on the
remote MC-LAG peer:
•	Commit an MC-LAG c n	r	n on the remote MC-LAG peer.
•	ICCP parses the MC-LAG c n		r	n and then sends the c n					r	n to the local MC-LAG peer.

15

• The local MC-LAG peer receives the c n r n from the remote MC-LAG peer and compares it
with its own c n r n
If the there is a severe inconsistency between the two c	n	r	ns the MC-LAG interface is
brought down, and syslog messages are issued.
If there is a moderate inconsistency between the two c	n	r	ns syslog messages are issued.

C n

r

n Consistency Requirements

There are

r n

c n r n consistency requirements depending on the MC-LAG parameters. The

consistency requirements are either

n

c

or unique, which means that some parameters must be

c n

r

n c

y and some must be c n

r

uniquely on the MC-LAG peers. For example, the

chassis ID must be unique on both peers, whereas the LACP mode must be

n

c

on both peers.

The enforcement level of the consistency requirements (

n

c

or unique) is either mandatory or

desired. When the enforcement level is mandatory, and you c

n

r

the MC-LAG parameter

incorrectly, the system brings down the interface and issues a syslog message.

For example, you receive a syslog message that says, “Some of the M

c

ss s Link

r

n (MC-

LAG) c

n r

n parameters between the peer devices are not consistent. The concerned MC-LAG

interfaces were explictly brought down to prevent unwanted behavior.” When you correct the

inconsistency, and issue a successful commit, the system will bring up the interface. When the

enforcement is desired, and you c n

r the MC-LAG parameter incorrectly, you receive a syslog

message that says, "Some of the M

c

ss s Link

r

n(MC

G) c n

r

n parameters

between the peer devices are not consistent. This may lead to s

b

m

performance of the

feature." As noted in the syslog message, performance will be s

b

m

in this s

n You can also

issue the "show interfaces mc-ae" on page 509 command to display the c

n

r

n consistency check

status of the m

c ss s aggregated Ethernet interface.

If there are m

inconsistencies, only the

rs inconsistency is shown. If the enforcement level for an

MC-LAG parameter is mandatory, and you did not c

n

r that parameter correctly, the command

shows that the MC-LAG interface is down.

When Remote Peers are Not Reachable

When you issue a commit on the local peer, and the remote peer is not reachable, c

n

r

n

consistency check will pass so that the local peer can come up in standalone mode. When the remote

peer comes up, ICCP exchanges the c n r	ns between the peers. If the consistency check fails, the
MC-LAG interface goes down, and the system n	s you of the parameter that caused the

inconsistency. When you correct the inconsistency, and issue a successful commit, the system brings up the interface.

16

Enabling MC-LAG C n	r	n Consistency Checking
Consistency check is not enabled by default. To enable consistency check, issue the set m				c ss s
mc-lag consistency-check command.
Table 1 on page 16 provides a sample list of c mm			MC-LAG parameters that are checked for
consistency, along with their consistency requirements ( n c or unique), hierarchies in which the

parameters are c n r			and the consistency check enforcement levels (mandatory or desired).
Table 1: MC-LAG Parameters Checked for C n						r	n Consistency
C n	r	n Knob			Hierarchy				Consistency		Enforcement
									Requirement
s ss	n	s b s m n	m		Global, ICCP Peer				n c		Mandatory

Specify the m during which an Inter-Chassis Control Protocol (ICCP) c nn c n must be established between peers.

interface-mac-limit	Global	n	c	Desired
Specify the maximum number of
MAC addresses to be associated
with an interface.
MC-AE interface mac limit	MCAE	n	c	Desired
Specify the maximum number of
MAC addresses to be associated
with the MC-AE interface.
mac-limit	Global	n	c	Desired
Specify the maximum number of
MAC addresses to be associated
with a VLAN—the default is
unlimited, which can leave the
network vulnerable to	n

17

Table 1: MC-LAG Parameters Checked for C n r					n Consistency (C n n			)
C n	r	n Knob		Hierarchy			Consistency			Enforcement
							Requirement
m c	n	m r		Global			n c			Desired

Specify how long MAC addresses remain in the Ethernet switching table.

r	n	m r					Global	n	c	Desired
Specify the ARP aging						m r in
minutes for a logical interface of
inet.
rs	sys	m		n	r		Global	n	c	Desired
Specify			r n	bridge		n	rs
for	r n		RSTP r			n
instances.
ms	sys	m		n	r		Global	n	c	Desired
Specify			r n	bridge		n	rs
for	r n		MSTP r			n
instances.
rstp-bridge-priority							Global	n	c	Desired
Determine which bridge is elected
as the root bridge for RSTP. If two
bridges have the same path cost
to the root bridge, the bridge
priority determines which bridge
becomes the designated bridge for
a LAN segment.

18

Table 1: MC-LAG Parameters Checked for C n

r

n Consistency (C n n

)

C n

r

n Knob

Hierarchy

Consistency

Enforcement

Requirement

mstp-bridge-priority

Global

n

c

Desired

Determine which bridge is elected

as the root bridge for MSTP. If two

bridges have the same path cost

to the root bridge, the bridge

priority determines which bridge

becomes the designated bridge for

a LAN segment.

rstp-bpdu-block-on-edge

Global

n

c

Desired

C n

r

bridge protocol data

unit (BPDU)

r

c

n on all edge

ports of a switch for RSTP.

vstp-bpdu-block-on-edge

Global

n

c

Desired

C n

r

bridge protocol data

unit (BPDU)

r

c

n on all edge

ports of a switch for VSTP.

mstp-bpdu-block-on-edge

Global

n

c

Desired

C n

r

bridge protocol data

unit (BPDU)

r

c

n on all edge

ports of a switch for MSTP.

service-id

Global

n

c

Mandatory

Specify a service

n

r for

each m

c

ss s aggregated

Ethernet interface that belongs to

a link

r

n group (LAG).

19

Table 1: MC-LAG Parameters Checked for C n

r

n Consistency (C n n

)

C

n

r

n Knob

Hierarchy

Consistency

Enforcement

Requirement

bfd minimum-interval

ICCP Peer

n

c

Mandatory

C

n

r

the minimum interval

r which the local r

n

device transmits hello packets and

then expects to receive a reply

from a neighbor with which it has

established a BFD session.

iccp/minimum-transmit-interval

ICCP Peer

n

c

Mandatory

Specify the minimum interval at

which the local r

n

device

transmits hello packets to a

neighbor with which it has

established a BFD session.

iccp/minimum-receive-interval

ICCP Peer

n

c

Mandatory

Specify the minimum interval

r

which the local r

n

device

must receive a reply from a

neighbor with which it has

established a BFD session.

iccp/bfd m

r

ICCP Peer

n

c

Mandatory

C

n

r the number of hello

packets not received by a

neighbor that causes the

r

n

n

interface to be

declared down.