Ethernet

  1. Twisted pair
    1. IOS CLI
    2. NX-OS CLI
    3. Auto-MDIX
  2. Coaxial
  3. Topologies
  4. Carrier sense multiple access / collision detection (CSMA/CD)
  5. Switching
  6. Fast Link Pulses (FLP)
    1. FLP configuration register
    2. FLP link code word
    3. FLP message page
  7. Ethernet
    1. Frame format
    2. Ethertype
    3. Link-layer control (LLC)
      1. Subnetwork access point (SNAP)
    4. Media access control (MAC)
    5. Virtual LAN (VLAN)
    6. QinQ
    7. Gigabit Ethernet
      1. 1000Base config register
    8. 10 Gigabit Ethernet

Twisted pair

  • shielding types:
    • unshielded twisted pair (UTP)
    • shielded twisted pair (STP)
    • foiled twisted pair (FTP)
  • 2/4 pairs of wire
  • 8P8C (8 positions, 8 contacts) ≠ RJ-45 (connector)
  • wires are twisted to reduce crosstalk between pairs: different pairs – different length of twist
  • 4th and 5th contacts – for phone (4P2C, RJ-11)
  • cable types
    • straight-through: switch – PC
    • crossover: Rx ←→ Tx
    • rollover: console
  • cable core
    • 24/26 AWG (how many passes through drawing die)
  • categories
    • 5, 5e: 100Mbps
    • 6, 6a: 1Gbps
    • 7: 10 Gbps, every pair is foiled separately

IOS CLI

; 2s default, delay between physical loss of carrier and interface going down
(config-if)# carrier-delay [msec] <sec>

NX-OS CLI

; ≡ carrier-delay
(config-if)# link debounce time <ms>

Auto-MDIX

  • medium dependent interface crossover
  • speed = auto && duplex = auto
(config-if)# mdix auto
# show interface <INTF> transceiver properties

Coaxial

  • waveguide
  • pros:
    • resistant to EMI
    • long length
  • cons:
    • expensive
    • requires terminators

Topologies

  • bus
    • class 1 hubs cannot be interconnected
    • no more than two class 2 hubs can be interconnected
    • if above not met – CSMA/CD timers are broken
  • star
  • ring, double ring
  • full mush
  • spine & leaf
  • microsegmentation: two devices in collision domain

Carrier sense multiple access / collision detection (CSMA/CD)

  • frame is considered bad if collision is detected
  • steps
    1. listen to channel
    2. transmit
    3. detect collision: voltage level exceeded on the line
    4. start timer
    5. retransmit
  • 16 errors in sequence – abort + error to ULP
  • 5-4-3 rule:
    • not more than 5 segments
    • not more than 4 repeaters
    • not more than 3 segments with users
    • rule + min frame length = min time for transmission, enough to detect collision

Switching

  • modes:
    • store & forward
      • receive whole frame, verify CRC – transmit if successful
      • buffering
      • ACL processing
      • 1GE → 10GE, 1GE → 1GE
    • cut through
      • receive DA to detect egress port – transmit
      • low latency
      • propagates CRC error, no collision detection
      • only if egress port is not congested
      • 10GE → 10GE, 10GE → 1GE
    • fragment free
      • receive 64 bytes, transmit if no collision is detected
  • order
    • verify CRC
    • process source MAC
      • learn MAC address
    • process destination MAC
      • if ingress port = egress port → drop (segment with hub)
      • send to egress port or flood
; 5 mins default
(config)# mac address-table aging-time <sec> vlan <VLAN>

(config)# mac address-table static <MAC> vlan <VLAN> interface <INTF>

; disable conversational learning for VLAN
(config)# no mac address-table learning-mode conversational vlan <N>

; NX-OS
(config)# hardware ethernet store-and-fwd-switching
; do not flood unknown unicast or unknown multicast
(config-if)# switchport block multicast|unicast
# show mac address-table unicast-flood

Fast Link Pulses (FLP)

  • L1 protocol
  • exchange information between NLP (normal link pulses)
  • autosense: speed only, not duplex

FLP configuration register

  0                                       1
  0   1   2   3   4   5   6   7   8   9   0   1   2   3   4   5
+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+
|  Reserved |100|100|10T|10T| Rsvd  |100| PAUSE |   RF  |ACK|NP |
+           |HD |FD |HD |FD |       |T4 |       |       |   |   |
+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+

FD: full-duplex
HD: half-duplex
NP: next page
ACK: 1 ≡ same config_reg received 3 times in sequence
RF: remote fault

  • 00 ≡ no error
  • 01 ≡ offline
  • 10 ≡ link failure
  • 11 ≡ link error
  0                                       1
  0   1   2   3   4   5   6   7   8   9   0   1   2   3   4   5
+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+
|      Selector     |10T|10T|TX |TX |T4 |PS |ASM|Rsv|RF |ACK|NP |
+                   |   |FD |   |FD |   |   |DIR|   |   |   |   |
+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+

Selector:

  • 16 ≡ Ethernet
  • 8 ≡ IEEE 802.9
  • 24 ≡ Token Ring

ACK: 1 ≡ received three equal FLP in sequence
NP: 0 ≡ extra pages, required for 1 GE

FLP message page

  0                                       1
  0   1   2   3   4   5   6   7   8   9   0   1   2   3   4   5
+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+
|               Message code                | T |AC2|MP |ACK|NP |
+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+

Message code:

  • 0..0: Null – last page
  • 64: 1000Base-T in message page, data in unformatted message page

T: toogle, flips between 0 and 1 between pages, acknowledges order
AC2: whether peer can acknowledge message, sent by its peer
MP: 0 ≡ unformatted page, 1 ≡ message page

Ethernet

  • frames
    • jumbo: 1500 < payload ≤ 9000
    • baby giant: 1500 < payload < 2000
    • runt: < 64 bytes + bad CRC
    • undersize: < 64 bytes + valid CRC
  • MTU
    • 1518 bytes
    • 1522 bytes: IEEE 802.3ac (to account for 802.1q)
  • interframe spacing = 96 bit
  • MAC flap
    • C4k: MAC changes port 2 times in 15s – drop frames with this src MAC for 15s
  • characters:
    • control: K.x.y
      • start of control frame ≡ ordered set, because character order is fixed
    • data: D.x.y
      • do not contain five 0 or 1 in sequence
    • x = 5 LSB, y = 3 MSB

Frame format

 0                   1                   2                   3
 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|                      Destination address                      |
+                               +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|                               |                               |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+                               +
|                        Source address                         |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|           Type/Length         |                               \
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+                               /
\                              Data                             \
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|                              FCS                              |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

Type/Length:

  • > 1536 – Type, Ethernet II (DIX: Digital, Intel, Xerox)
  • ≤ 1500 – Length, 802.1 LLC

Ethertype

  • 0x010b: PVST+
  • 0x0200: CDP
  • 0x0800: IPv4
  • 0x0806: ARP
  • 0x8035: RARP
  • 0x8100: 802.1q VLAN
  • 0x86dd: IPv6
  • 0x8808: MAC control
  • 0x8809: LACP
  • 0x8847: MPLS unicast
  • 0x8848: MPLS mcast
  • 0x8863: PPPoE discovery
  • 0x8864: PPPoE session
  • 0x888e: 802.1x
  • 0x8892: PROFINET
  • 0x88a8: QinQ
  • 0x88cc: LLDP
  • 0x88e7: PBB
  • 0x88f7: PTP
  • 0x8902: Ethernet OAM
  • 0x8903: FabricPath
  • 0x8906: FCoE
  • 0x8914: FCoE initialization protocol
  • 0x8915: RoCE
  • used when Ethernet T/L – length
  • destination service access point (DSAP): destination ULP
  • source service access point (SSAP): source ULP
  • SAP
    • 0x06: IPv4
    • 0x2a: 802.1D
 0                   1                   2                   3
 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|      DSAP     |      SSAP     |           Control             |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

I: individual/group
C: command/response

Subnetwork access point (SNAP)

  • used when no necessary DSAP or SSAP is available
  • DSAP = SSAP = 0xAA|0xAB
 0                   1                   2                   3
 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|                      OUI                      |           Ethertype           |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

Media access control (MAC)

  • vendor ID + device ID
  • 7th bit:
    • 0 ≡ universal (globally administered)
    • 1 ≡ local (locally administered)
  • 8th bit
    • 0 ≡ individual
    • 1 ≡ group
  • MAC SA is always individual

Virtual LAN (VLAN)

  • reserved:
    • 1002-1005: FDDI, Token Ring
    • 1006-1024: system use
  • if VLAN is deleted, member ports → inactive ≡ not forwarding
  • Cisco HW support
    • 2950: 250 VLANs
    • 2960: 255 VLANs
    • 3550, 3560, 3750: 1005 VLANs
    • 4000, 4500, 6500: 4094 VLANs
  • internal VLANs are created for L3 ports
 0                   1                   2                   3
 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|           Ethertype           |Prio |D|        VLAN ID        |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

D: Drop eligible indicator (CFI)

  • 0 ≡ Ethernet
  • 1 ≡ Token Ring
(config)# vlan dot1q tag native

; ascending default
(config)# vlan internal allocation policy ascending|descending
; global setting, distributed by VTP
(config-vlan)# state active|suspend

; disable locally, not distributed by VTP
(config-vlan)# shutdown
# show vlan dot1q tag native# show interface status inactive

QinQ

  • Ethertype = 0x88a8
  • does not tunnel L2 management protocols by default
; 100 Mbps interfaces
(config)# system mtu 1504

; 1+ Gbps interfaces
(config)# system mtu jumbo 1504
(config-if)# switchport mode dot1q-tunnel

; S-tag
(config-if)# switchport access vlan <VLAN>

(config-if)# switchport l2protocol-tunnel cdp|lldp|stp|vtp

Gigabit Ethernet

  • 8b/10b encoding for fiber, 8b1q4 encoding for copper
  • symbols:
    • K.28.5:
      • first symbol for all ordered sets
      • synchronizes Tx and Rx
    • K.23.7 (EOP2)
      • end of packet 2: if there is next frame after this one
    • K.27.7 (SOP):
      • start of packet
    • K.29.7 (EOP1):
      • end of packet 1
    • K.30.7 (EOP Invalid):
      • propagation error
    • K.28.5 + D.21.5 + config_reg (C1):
      • link configuration 1
      • flip running disparity (RD)
      • passes information
      • bit-level synchronization
    • K.28.5 + D.2.2 + config_reg (C2):
      • link configuration 2
      • no flip running disparity (RD)
      • passes information
      • bit-level synchronization
    • K.28.5 + D.5.6 (I1)
      • idle/flip disparity
      • holds carrier if data is absent
      • changes disparity from + to –
    • K.28.5 + D.16.2 (I2)
      • idle/disparity OK
    • K.28.5 + D.21.5
      • link not available
      • transmitted by Tx until Rx is synched
  • Ethernet uses negative running disparity (RD); if becomes positive – I1 symbol required
  • initialization:
    1. power on → /C1/ and /C2/, config_reg = 0 ⇒ bit-level sync
    2. search Rx for K.28.5, send /C1/ and /C2/ with info in config_reg ⇒ receiving sequentially 3×K.28.5 with /Cx/ ≡ word alignment
    3. search for config_reg ⇒ receive 3 equal config_reg sequentially → ACK = 1 in Tx
    4. LinkTime (10ms) expires ⇒ use parameters
  • autonegotiation
    • fiber:
      • requires bit-level sync before negotiation ⇒ cannot negotiate speed beforehand (chicken-egg problem)
    • copper:
      • FLP does not use encoding ⇒ not bit-level sync required ⇒ can negotiate speed
      • FLP has necessary information, so init ≡ transmit idle for bit-level sync

1000Base config register

  0                                       1
  0   1   2   3   4   5   6   7   8   9   0   1   2   3   4   5
+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+
|      Reserved     |FD |HD | PAUSE |  Reserved |  RF   |ACK|NP |
+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+

FD: full-duplex
HD: half-duplex
RF: remote fault
NP: next page
ACK: 1 ≡ same config_reg received 3 times in sequence

10 Gigabit Ethernet

  • 64b/66b encoding
  • symbols:
    • 0x06 (LI):
      • low power idle
    • 0x07 (I)
      • idle
      • sync, interframe spacing
    • 0x5c (Fsig)
      • signal ordered set
      • used by FC
    • 0x9c (Q)
      • sequence ordered set
      • control info after 3 data characters, adapt clock rate
    • 0xfb (S):
      • start of frame
    • 0xfd (T):
      • terminate
      • end of frame, then /S/ or /I/
    • 0xfe (E)
      • error propagation