Friday, February 5, 2016

Configuring ERSPAN on Cisco Routers and Switches

In two recent posts, I covered SPAN, for mirroring traffic to a port on a local switch, and RSPAN, for mirroring traffic across a VLAN to a port on a remote switch.  What if we want to mirror traffic traffic to a destination across a L3 link?  Cisco provides the ability to do this natively with a feature called ERSPAN, or encapsulated RSPAN.  However, this feature is only available on higher end platforms such as Catalyst 6500 and 6800 series switches, 7600 series routers, ASR1000, and CSR1000v (this is not a complete list).

ERSPAN

Like SPAN and RSPAN, configuring ERSPAN is pretty straightforward.  ERSPAN simply requires L3 connectivity between source and destination devices.  The ERSPAN monitor session then builds a GRE tunnel that transports mirrored frames from the source port to the destination port.

Basic ERSPAN configuration is as follows:
! Source switch
monitor session SESSION-NUMBER type erspan-source 
 source-interface INTERFACE(S)|VLAN(S) {TX|RX|BOTH}
 no shutdown
 destination
  erspan-id ERSPAN-ID
  ip address DESTINATION-IP
  origin ip address ORIGIN-IP

! Destination switch
monitor session SESSION-NUMBER type erspan-destination
 destination-interface INTERFACE(S)
 no shutdown
 source
  erspan-id ERSPAN-ID
  ip address SOURCE-IP
    It is important to note that when configuring the destination switch "source IP," you should select the source IP on the destination switch itself - the GRE tunnel endpoint.  Source IP does not refer to the GRE tunnel origin IP address.  Therefore, the "ip address" command should match on the source and destination.

    Below is a basic ERSPAN config to mirror data from R1 interface g3 to R3 interface g3.  I created this topology using VIRL using CSR1000V routers for R1 and R3.



    Mirror ingress and egress traffic from g3 and send through GRE tunnel from 10.1.1.1 to 10.2.2.1
    R1(config)#monitor session 1 type erspan-source 
    R1(config-mon-erspan-src)#source interface g3 both
    R1(config-mon-erspan-src)#no shutdown
    R1(config-mon-erspan-src)#destination 
    R1(config-mon-erspan-src-dst)#erspan-id 100
    R1(config-mon-erspan-src-dst)#ip address 10.2.2.1
    R1(config-mon-erspan-src-dst)#origin ip address 10.1.1.1

    Mirror traffic from GRE tunnel to g3.
    R3(config)#monitor session 1 type erspan-destination 
    R3(config-mon-erspan-dst)#destination interface g3
    R3(config-mon-erspan-dst)#no shutdown
    R3(config-mon-erspan-dst)#source 
    R3(config-mon-erspan-dst-src)#erspan-id 100
    R3(config-mon-erspan-dst-src)#ip address 10.2.2.1

    R1 verification:
    R1#show monitor session 1
    Session 1
    ---------
    Type                   : ERSPAN Source Session
    Status                 : Admin Enabled
    Source Ports           : 
        Both               : Gi3
    Destination IP Address : 10.2.2.1
    Destination ERSPAN ID  : 100
    Origin IP Address      : 10.1.1.1
    
    
    R1#show platform hardware qfp active feature erspan state
    ERSPAN State:
      Status    : Active
      Complexes : 1
      CPPs      : 1
    Capabilites:
      Max sessions : 1024
      Max outputs  : 128
      Encaps type  : ERSPAN type-II
      GRE protocol : 0x88BE
      MTU          : 1464
      IP TOS       : 0
      IP TTL       : 255
      COS          : 0
    System Statistics:
      DROP src session replica  :                  0 /                  0
      DROP term session replica :                  0 /                  0
      DROP receive malformed    :                  0 /                  0
      DROP receive invalid ID   :                  0 /                  0
      DROP recycle queue full   :                  0 /                  0
      DROP no GPM memory        :                  0 /                  0
      DROP no channel memory    :                  0 /                  0
    Client Debug Config:
      Enabled: Info, Warn
    Data Path Debug Config:
      0x00000000
    
    
    R1#
    


    R3 verification:
    R3#show monitor session 1
    Session 1
    ---------
    Type                   : ERSPAN Destination Session
    Status                 : Admin Enabled
    Destination Ports      : Gi3
    Source IP Address      : 10.2.2.1
    Source ERSPAN ID       : 100
    
    
    R3#show platform hardware qfp active feature erspan state
    ERSPAN State:
      Status    : Active
      Complexes : 1
      CPPs      : 1
    Capabilites:
      Max sessions : 1024
      Max outputs  : 128
      Encaps type  : ERSPAN type-II
      GRE protocol : 0x88BE
      MTU          : 1464
      IP TOS       : 0
      IP TTL       : 255
      COS          : 0
    System Statistics:
      DROP src session replica  :                  0 /                  0
      DROP term session replica :                  0 /                  0
      DROP receive malformed    :                  0 /                  0
      DROP receive invalid ID   :                  0 /                  0
      DROP recycle queue full   :                  0 /                  0
      DROP no GPM memory        :                  0 /                  0
      DROP no channel memory    :                  0 /                  0
    Client Debug Config:
      Enabled: Info, Warn
    Data Path Debug Config:
      0x00000000
    
    
    R3#
    

    If your ERSPAN session is functioning properly, the drop counters shown in the above show output should not increment.  If the "DROP receive invalid ID" counter increments, there is most likely an ERSPAN session ID mismatch or misconfigured source IP on the destination router.

    To show ERSPAN functionality and the GRE encapsulation and decapsulation process, I sent five ICMP echo requests from the Source router interface g0/1 (172.16.10.1) to R1 interface g3 (172.16.10.2) and took packet captures at three points in the network. 

    Here is a capture taken from interface g0/1 on the Source router.  This traffic is tagged VLAN 10.  Without mirroring this traffic with ERSPAN, it would stay local to the link between Source and R1.

    (full capture)

    Here is a capture taken from interface g3 on R1.  You can see the original source and destination frames have been encapsulated using GRE.  The new source address is 10.1.1.1 and new destination address is 10.2.2.1.

    (full capture)

    Here is a capture taken from interface g3 on R3.  You can the GRE encapsulation has been stripped, and we are left with the frames from the original capture, including VLAN tagging.

    (full capture)

    And, that's it!  We have L2 frames encapsulated and sent across a L3 link for analysis.  As always, it is best to check platform specific documentation in order to make sure this will work as intended for your specific use case.

     CCIE R&S v5.0/5.1 exam topic: 2.1.g [i] SPAN, RSPAN, ERSPAN

    3 comments:

    1. you have stated that "It is important to note that when configuring the destination switch "source IP," you should select the source IP on the destination switch itself - the GRE tunnel endpoint."

      In other words
      Source switch.DESTINATION-IP == Destination switch.SOURCE-IP

      But in the example you have not configured it that way.. Can you please clarify ?

      ReplyDelete
      Replies
      1. Good catch. I was testing several scenarios in my lab and pasted the wrong example. The config snippet I originally posted would not have worked.

        When configuring R3 with the source ip address of 10.1.1.1, ERSPAN will not work, and you will see the "DROP receive invalid ID" counter increment as I mentioned in the original post. The source ip on R3 should have been 10.2.2.1. I just corrected this. Thanks for the heads up!

        Here's the show output with the misconfigured source ip of 10.1.1.1 on R3:

        R3#show platform hardware qfp active feature erspan state
        ERSPAN State:
        Status : Active
        Complexes : 1
        CPPs : 1
        Capabilites:
        Max sessions : 1024
        Max outputs : 128
        Encaps type : ERSPAN type-II
        GRE protocol : 0x88BE
        MTU : 1464
        IP TOS : 0
        IP TTL : 255
        COS : 0
        System Statistics:
        DROP src session replica : 0 / 0
        DROP term session replica : 0 / 0
        DROP receive malformed : 0 / 0
        DROP receive invalid ID : 10 / 1640
        DROP recycle queue full : 0 / 0
        DROP no GPM memory : 0 / 0
        DROP no channel memory : 0 / 0
        Client Debug Config:
        Enabled: Info, Warn
        Data Path Debug Config:
        0x00000000


        R3#

        Delete
    2. "Here is a capture taken from interface g3 on R1"

      This might be a typo here, shouldn't this be Interface g2 on R1?

      Regards,
      Jason.

      ReplyDelete