Routers, Switching Fabrics and the Origins of SDN

Created

Routers

Routers run routing protocols such as OSPF (shortest paths via Dijkstra).

Networking layers:

  • layer one, get the bits.
  • layer two, like ethernet
  • layer three, IP layer

Interface card

  • has cubes / boxes because packets are not processed immediately
  • over the year, they've gotten more sophisticated
  • entire routing table is on these
  • where IP forwarding is done
    • datapath depends on the needs

Your laptop can be a router but commercial routers have more sophisticated interconnection network and interface cards.

  • large number of internet protocols
    • IETF is the team that manages
    • RFC are what these protocols are called

Switch fabric:

  • crossbar. each input line can be connected to the output line
  • if all packets are being sent to one destination, there will be a queue

Slotted chassis:

  • large routers are built as a slotted chassis where the interface cards and route processor are inserted in the slot
  • this simplifies repairs and upgrades

Router architecture evolution:

  • early routers were essentially general purpose computers. Everything was done in software
  • now, high-performance routers resemble super computers
    • parallelism
    • special hardware components
    • longest prefix matches in nanoseconds.
  • timeline:
    • 1980s (1st generation): standard computer. Low end routers are still of this design today. Your laptop can be turned into this router
      • arriving packets are copied into the main memory via DMA (direct memory access).
        • if the main processor needs to copy from the interface card to the main memory it would take a lot of cycles, so DMA offers the ability to copy without using the main processor
        • All IP forwarding functions are done in the central processor (like longest prefix matching)
        • routing cache at processor can accelerate the routing table lookup
    • Early 1990s (2nd generations): delegate to interfaces. main logic was still in control processor. some caching
      • how it works:
        • keeps shared bus architecture but IP forwarding is offloaded to interface card. routing table is not in interface card
        • (slow path) processor processes the routing table the first time, but is then cached locally in the interface card so that in the future (fast path) then the packet is forwarded directly to the outgoing interface
      • drawback:
        • shared interface
      • ip forwarding is done by forwarding engine (separate components)
      • is not used today
    • late 1990s (3rd generation): distributed
      • how it works:
        • the interconnection network is a switch fabric (like a crossbar switch)
        • distributed architecture
          • interface card operate independent of each other
          • no centralized processing for ip forwarding. this is done in the interface card. there is a routing table stored in each one.
    • today: distributed over multiple racks, and they are connected together.

Switching Fabric

Ports: fiber optic receivers and laser transmitters + buffers

Crossbar switch:

  • each input is connected with every output
  • complexity = n2n^2 (n inputs, n outputs)
  • each output port has to deal with contention when multiple packets are sent to the same output simultaneously, causing a collision
    • has to be able to handle this situation
  • first invented in telephone networks

Shared-memory switch:

  • one big buffer built out of off-the-shelf memory chips
  • can be made to act like a crossbar
  • busses connecting the multiplexer to/from memory must have a bandwidth n times the speed of a single input.

Self-routing fabrics

  • made up of a bunch of small switches that self routes
  • when a packet comes in, it is broken into smaller segments

Banyan self-routing switch

  • made up of 2x2 switched element
  • each element switches on basis of just 1 bit in the self-routing header
  • if the bit is zero, it sends it up, and if the bit is 1 it sends it down.
  • hardware complexity: n2log⁡2n\frac{n}{2}\log_2 n

Router Capacity

  • from single rack (<1TB/s) to multi-rack (>1 TB/s)
  • internet traffic doubles every 12 months and router capacity doubles every 18 months. Routers will reach 100 Tb/s in 201x

Why do current packet routers not scale?

  • when the router can't keep up then there is a lot of heat and data is lost.
  • so the limitation is power and heat dissipation limit density.
  • to decrease density, we require multiple shelves and racks of line cards. This means an increase in distance between line cards and switch fabrics.

Bell lab solutions (theoretical):

  • optical switch fabric
  • distributed scheduler

Softrouter

At bell labs (theoretical):

  • when an RFC is established, every router needs to implement it like multicasting, IPSEC, Mobile IP, etc.
    • multicasting: from one source to multiple receivers simultaneously. i.e., send to selective receiver. in the network you can form a multicasting tree.
    • packets are sent via unicast, broadcast, or multicast

Origins of SDN

  • telecom network wanted to add network feature, it would take too long, so then software defined networks were formed.

Each router is a control processor. In SDN, you take this processor out.

Complexity is an IP middle-age problem

  • IP Hour glass.
  • emerging applications are driving more functions to IP

Industry

The same way we have applications on top of OS, we want something similar for networking.

  • Cisco, Juniper, and Nokia are the industry leaders right now.
  • Google, Amazon, Microsoft all have their own SDN network for their datacenters.
  • 5G/6G networks have a separate control plane

SDN Architecture

Between the {single packet, forwarding, and hardware} and {network operating system} there is an open interface to the hardware.

there are many network operating systems they need to be extensible and possibly open source. Then above the network OS, you need a well-defined open API between it and the {Apps}

System view:

  • application tier
  • controller tier
  • dataplane tier

Decouples data, control, and application planes, creating a programmable network.

Phone network used to be 3 min of downtime a year. Insane reliability

OpenFlow

  • northbound API describes the api between applications to OpenFlow-based SDN controller
  • Southbound API (openflow) is between the controller and the OF switches

Usage:

  • when the switch gets a packet, it uses the openflow protocol to communicate with the controller, which then decides what each router should do.

Example:

  • when the packet comes in
  • then you mask to get all the information you need
    • MAC src/dst
    • IP src/dst
    • TCP sport/dport
    • Action
  • If there's a match, you execute the action
  • star is don't care

MAC addresses are 48 bits, IPv4 addresses are 32 bits.

Flow table entries:

  • rule
  • action
  • stats (what information you want to keep)

OpenFlow: is more like a payload protocol (sending data down)

switching at the mac layer is called "switching"

flow switching is setting up a flow

firewall: only look at the TCP dport and then you drop

Routing: only set IP dst

VLAN switching: MAC dst and VLAN ID

Flow-based:

  • if the number of flows are limited

Aggregated:

  • you would want to group flows (baltimore to NYC)

Network Slicing and Virtualization

Network slicing: different protocol stats for different users. important part of 5g

  • the same switch can have separate networks running
  • modularization
  • VMware: layer on top of OS. take hardware and run other OS on top.

Virtualization: run many OSes on top of the hardware.

  • run multiple networking OS on the slicing / virtualization layer
  • follows computing industry