ovn-northd: Add logical flows to support native DHCPv4

[cascardo/ovs.git] / ovn / northd / ovn-northd.8.xml

<?xml version="1.0" encoding="utf-8"?>
<manpage program="ovn-northd" section="8" title="ovn-northd">
    <h1>Name</h1>
    <p>ovn-northd -- Open Virtual Network central control daemon</p>

    <h1>Synopsis</h1>
    <p><code>ovn-northd</code> [<var>options</var>]</p>

    <h1>Description</h1>
    <p>
      <code>ovn-northd</code> is a centralized daemon responsible for
      translating the high-level OVN configuration into logical
      configuration consumable by daemons such as
      <code>ovn-controller</code>.  It translates the logical network
      configuration in terms of conventional network concepts, taken
      from the OVN Northbound Database (see <code>ovn-nb</code>(5)),
      into logical datapath flows in the OVN Southbound Database (see
      <code>ovn-sb</code>(5)) below it.
    </p>

    <h1>Configuration</h1>
    <p>
      <code>ovn-northd</code> requires a connection to the Northbound
      and Southbound databases.  The defaults are <code>ovnnb_db.sock</code>
      and <code>ovnsb_db.sock</code> respectively
      in the local Open vSwitch's "run" directory.  This may be
      overridden with the following commands:
    </p>
    <ul>
      <li>
        <p>
          <code>--ovnnb-db=<var>database</var></code>
        </p>
        <p>
          The database containing the OVN Northbound Database.
        </p>
      </li>
      <li>
        <p>
          <code>--ovnsb-db=<var>database</var></code>
        </p>
        <p>
          The database containing the OVN Southbound Database.
        </p>
      </li>
    </ul>
    <p>
      The <var>database</var> argument must take one of the following forms:
    </p>
    <ul>
      <li>
        <p>
          <code>ssl:<var>ip</var>:<var>port</var></code>
        </p>
        <p>
          The specified SSL <var>port</var> on the host at the given
          <var>ip</var>, which must be expressed as an IP address (not a DNS
          name) in IPv4 or IPv6 address format.  If <var>ip</var> is an IPv6
          address, then wrap <var>ip</var> with square brackets, e.g.:
          <code>ssl:[::1]:6640</code>.  The <code>--private-key</code>,
          <code>--certificate</code>, and <code>--ca-cert</code> options are
          mandatory when this form is used.
        </p>
      </li>
      <li>
        <p>
          <code>tcp:<var>ip</var>:<var>port</var></code>
        </p>
        <p>
          Connect to the given TCP <var>port</var> on <var>ip</var>, where
          <var>ip</var> can be IPv4 or IPv6 address. If <var>ip</var> is an
          IPv6 address, then wrap <var>ip</var> with square brackets, e.g.:
          <code>tcp:[::1]:6640</code>.
        </p>
      </li>
      <li>
        <p>
          <code>unix:<var>file</var></code>
        </p>
        <p>
          On POSIX, connect to the Unix domain server socket named
          <var>file</var>.
        </p>
        <p>
          On Windows, connect to a localhost TCP port whose value is written
          in <var>file</var>.
        </p>
      </li>
    </ul>

    <h1>Runtime Management Commands</h1>
    <p>
      <code>ovs-appctl</code> can send commands to a running
      <code>ovn-northd</code> process.  The currently supported commands
      are described below.
      <dl>
      <dt><code>exit</code></dt>
      <dd>
        Causes <code>ovn-northd</code> to gracefully terminate.
      </dd>
      </dl>
    </p>

    <h1>Logical Flow Table Structure</h1>

    <p>
      One of the main purposes of <code>ovn-northd</code> is to populate the
      <code>Logical_Flow</code> table in the <code>OVN_Southbound</code>
      database.  This section describes how <code>ovn-northd</code> does this
      for switch and router logical datapaths.
    </p>

    <h2>Logical Switch Datapaths</h2>

    <h3>Ingress Table 0: Admission Control and Ingress Port Security - L2</h3>

    <p>
      Ingress table 0 contains these logical flows:
    </p>

    <ul>
      <li>
        Priority 100 flows to drop packets with VLAN tags or multicast Ethernet
        source addresses.
      </li>

      <li>
        Priority 50 flows that implement ingress port security for each enabled
        logical port.  For logical ports on which port security is enabled,
        these match the <code>inport</code> and the valid <code>eth.src</code>
        address(es) and advance only those packets to the next flow table.  For
        logical ports on which port security is not enabled, these advance all
        packets that match the <code>inport</code>.
      </li>
    </ul>

    <p>
      There are no flows for disabled logical ports because the default-drop
      behavior of logical flow tables causes packets that ingress from them to
      be dropped.
    </p>

    <h3>Ingress Table 1: Ingress Port Security - IP</h3>

    <p>
      Ingress table 1 contains these logical flows:
    </p>

    <ul>
      <li>
        <p>
          For each element in the port security set having one or more IPv4 or
          IPv6 addresses (or both),
        </p>

        <ul>
          <li>
            Priority 90 flow to allow IPv4 traffic if it has IPv4 addresses
            which match the <code>inport</code>, valid <code>eth.src</code>
            and valid <code>ip4.src</code> address(es).
          </li>

          <li>
            Priority 90 flow to allow IPv4 DHCP discovery traffic if it has a
            valid <code>eth.src</code>. This is necessary since DHCP discovery
            messages are sent from the unspecified IPv4 address (0.0.0.0) since
            the IPv4 address has not yet been assigned.
          </li>

          <li>
            Priority 90 flow to allow IPv6 traffic if it has IPv6 addresses
            which match the <code>inport</code>, valid <code>eth.src</code> and
            valid <code>ip6.src</code> address(es).
          </li>

          <li>
            Priority 90 flow to allow IPv6 DAD (Duplicate Address Detection)
            traffic if it has a valid <code>eth.src</code>. This is is
            necessary since DAD include requires joining an multicast group and
            sending neighbor solicitations for the newly assigned address. Since
            no address is yet assigned, these are sent from the unspecified
            IPv6 address (::).
          </li>

          <li>
            Priority 80 flow to drop IP (both IPv4 and IPv6) traffic which
            match the <code>inport</code> and valid <code>eth.src</code>.
          </li>
        </ul>
      </li>

      <li>
        One priority-0 fallback flow that matches all packets and advances to
        the next table.
      </li>
    </ul>

    <h3>Ingress Table 2: Ingress Port Security - Neighbor discovery</h3>

    <p>
      Ingress table 2 contains these logical flows:
    </p>

    <ul>
      <li>
        <p>
          For each element in the port security set,
        </p>

        <ul>
          <li>
            Priority 90 flow to allow ARP traffic which match the
            <code>inport</code> and valid <code>eth.src</code> and
            <code>arp.sha</code>. If the element has one or more
            IPv4 addresses, then it also matches the valid
            <code>arp.spa</code>.
          </li>

          <li>
            Priority 90 flow to allow IPv6 Neighbor Solicitation and
            Advertisement traffic which match the <code>inport</code>,
            valid <code>eth.src</code> and
            <code>nd.sll</code>/<code>nd.tll</code>.
            If the element has one or more IPv6 addresses, then it also
            matches the valid <code>nd.target</code> address(es) for Neighbor
            Advertisement traffic.
          </li>

          <li>
            Priority 80 flow to drop ARP and IPv6 Neighbor Solicitation and
            Advertisement traffic which match the <code>inport</code> and
            valid <code>eth.src</code>.
          </li>
        </ul>
      </li>

      <li>
        One priority-0 fallback flow that matches all packets and advances to
        the next table.
      </li>
    </ul>

    <h3>Ingress Table 3: <code>from-lport</code> Pre-ACLs</h3>

    <p>
      This table prepares flows for possible stateful ACL processing in
      ingress table <code>ACLs</code>.  It contains a priority-0 flow that
      simply moves traffic to the next table.  If stateful ACLs are used in the
      logical datapath, a priority-100 flow is added that sets a hint
      (with <code>reg0[0] = 1; next;</code>) for table
      <code>Pre-stateful</code> to send IP packets to the connection tracker
      before eventually advancing to ingress table <code>ACLs</code>.
    </p>

    <h3>Ingress Table 4: Pre-LB</h3>

    <p>
      This table prepares flows for possible stateful load balancing processing
      in ingress table <code>LB</code> and <code>Stateful</code>.  It contains
      a priority-0 flow that simply moves traffic to the next table.  If load
      balancing rules with virtual IP addresses (and ports) are configured in
      <code>OVN_Northbound</code> database for a logical datapath, a
      priority-100 flow is added for each configured virtual IP address
      <var>VIP</var> with a match <code>ip &amp;&amp; ip4.dst == <var>VIP</var>
      </code> that sets an action <code>reg0[0] = 1; next;</code> to act as a
      hint for table <code>Pre-stateful</code> to send IP packets to the
      connection tracker for packet de-fragmentation before eventually
      advancing to ingress table <code>LB</code>.
    </p>

    <h3>Ingress Table 5: Pre-stateful</h3>

    <p>
      This table prepares flows for all possible stateful processing
      in next tables.  It contains a priority-0 flow that simply moves
      traffic to the next table.  A priority-100 flow sends the packets to
      connection tracker based on a hint provided by the previous tables
      (with a match for <code>reg0[0] == 1</code>) by using the
      <code>ct_next;</code> action.
    </p>

    <h3>Ingress table 6: <code>from-lport</code> ACLs</h3>

    <p>
      Logical flows in this table closely reproduce those in the
      <code>ACL</code> table in the <code>OVN_Northbound</code> database
      for the <code>from-lport</code> direction. The <code>priority</code>
      values from the <code>ACL</code> table have a limited range and have
      1000 added to them to leave room for OVN default flows at both
      higher and lower priorities.
    </p>
    <ul>
      <li>
        <code>allow</code> ACLs translate into logical flows with
        the <code>next;</code> action.  If there are any stateful ACLs
        on this datapath, then <code>allow</code> ACLs translate to
        <code>ct_commit; next;</code> (which acts as a hint for the next tables
        to commit the connection to conntrack),
      </li>
      <li>
        <code>allow-related</code> ACLs translate into logical
        flows with the <code>ct_commit(ct_label=0/1); next;</code> actions
        for new connections and <code>reg0[1] = 1; next;</code> for existing
        connections.
      </li>
      <li>
        Other ACLs translate to <code>drop;</code> for new or untracked
        connections and <code>ct_commit(ct_label=1/1);</code> for known
        connections.  Setting <code>ct_label</code> marks a connection
        as one that was previously allowed, but should no longer be
        allowed due to a policy change.
      </li>
    </ul>

    <p>
      This table also contains a priority 0 flow with action
      <code>next;</code>, so that ACLs allow packets by default.  If the
      logical datapath has a statetful ACL, the following flows will
      also be added:
    </p>

    <ul>
      <li>
        A priority-1 flow that sets the hint to commit IP traffic to the
        connection tracker (with action <code>reg0[1] = 1; next;</code>).  This
        is needed for the default allow policy because, while the initiator's
        direction may not have any stateful rules, the server's may and then
        its return traffic would not be known and marked as invalid.
      </li>

      <li>
        A priority-65535 flow that allows any traffic in the reply
        direction for a connection that has been committed to the
        connection tracker (i.e., established flows), as long as
        the committed flow does not have <code>ct_label[0]</code> set.
        We only handle traffic in the reply direction here because
        we want all packets going in the request direction to still
        go through the flows that implement the currently defined
        policy based on ACLs.  If a connection is no longer allowed by
        policy, <code>ct_label[0]</code> will get set and packets in the
        reply direction will no longer be allowed, either.
      </li>

      <li>
        A priority-65535 flow that allows any traffic that is considered
        related to a committed flow in the connection tracker (e.g., an
        ICMP Port Unreachable from a non-listening UDP port), as long
        as the committed flow does not have <code>ct_label[0]</code> set.
      </li>

      <li>
        A priority-65535 flow that drops all traffic marked by the
        connection tracker as invalid.
      </li>

      <li>
        A priority-65535 flow that drops all trafic in the reply direction
        with <code>ct_label[0]</code> set meaning that the connection
        should no longer be allowed due to a policy change.  Packets
        in the request direction are skipped here to let a newly created
        ACL re-allow this connection.
      </li>
    </ul>

    <h3>Ingress Table 7: LB</h3>

    <p>
      It contains a priority-0 flow that simply moves traffic to the next
      table.  For established connections a priority 100 flow matches on
      <code>ct.est &amp;&amp; !ct.rel &amp;&amp; !ct.new &amp;&amp;
      !ct.inv</code> and sets an action <code>reg0[2] = 1; next;</code> to act
      as a hint for table <code>Stateful</code> to send packets through
      connection tracker to NAT the packets.  (The packet will automatically
      get DNATed to the same IP address as the first packet in that
      connection.)
    </p>

    <h3>Ingress Table 8: Stateful</h3>

    <ul>
      <li>
        For all the configured load balancing rules in
        <code>OVN_Northbound</code> database that includes a L4 port
        <var>PORT</var> of protocol <var>P</var> and IPv4 address
        <var>VIP</var>, a priority-120 flow that matches on
        <code>ct.new &amp;&amp; ip &amp;&amp; ip4.dst == <var>VIP
        </var>&amp;&amp; <var>P</var> &amp;&amp; <var>P</var>.dst == <var>PORT
        </var></code> with an action of <code>ct_lb(<var>args</var>)</code>,
        where <var>args</var> contains comma separated IPv4 addresses (and
        optional port numbers) to load balance to.
      </li>
      <li>
        For all the configured load balancing rules in
        <code>OVN_Northbound</code> database that includes just an IP address
        <var>VIP</var> to match on, a priority-110 flow that matches on
        <code>ct.new &amp;&amp; ip &amp;&amp; ip4.dst == <var>VIP</var></code>
        with an action of <code>ct_lb(<var>args</var>)</code>, where
        <var>args</var> contains comma separated IPv4 addresses.
      </li>
      <li>
        A priority-100 flow commits packets to connection tracker using
        <code>ct_commit; next;</code> action based on a hint provided by
        the previous tables (with a match for <code>reg0[1] == 1</code>).
      </li>
      <li>
        A priority-100 flow sends the packets to connection tracker using
        <code>ct_lb;</code> as the action based on a hint provided by the
        previous tables (with a match for <code>reg0[2] == 1</code>).
      </li>
      <li>
        A priority-0 flow that simply moves traffic to the next table.
      </li>
    </ul>

    <h3>Ingress Table 9: ARP responder</h3>

    <p>
      This table implements ARP responder for known IPs.  It contains these
      logical flows:
    </p>

    <ul>
      <li>
        Priority-100 flows to skip ARP responder if inport is of type
        <code>localnet</code>, and advances directly to the next table.
      </li>

      <li>
        <p>
          Priority-50 flows that matches ARP requests to each known IP address
          <var>A</var> of logical port <var>P</var>, and respond with ARP
          replies directly with corresponding Ethernet address <var>E</var>:
        </p>

        <pre>
eth.dst = eth.src;
eth.src = <var>E</var>;
arp.op = 2; /* ARP reply. */
arp.tha = arp.sha;
arp.sha = <var>E</var>;
arp.tpa = arp.spa;
arp.spa = <var>A</var>;
outport = <var>P</var>;
inport = ""; /* Allow sending out inport. */
output;
        </pre>

        <p>
          These flows are omitted for logical ports (other than router ports)
          that are down.
        </p>
      </li>

      <li>
        One priority-0 fallback flow that matches all packets and advances to
        the next table.
      </li>
    </ul>

    <h3>Ingress Table 10: DHCP option processing</h3>

    <p>
      This table adds the DHCPv4 options to a DHCPv4 packet from the
      logical ports configured with IPv4 address(es) and DHCPv4 options.
    </p>

    <ul>
      <li>
        <p>
          A priority-100 logical flow is added for these logical ports
          which matches the IPv4 packet with <code>udp.src</code> = 68 and
          <code>udp.dst</code> = 67 and applies the action
          <code>put_dhcp_opts</code> and advances the packet to the next table.
        </p>

        <pre>
reg0[3] = put_dhcp_opts(offer_ip = <var>O</var>, <i>options</i>...);
next;
        </pre>

        <p>
          For DHCPDISCOVER and DHCPREQUEST, this transforms the packet into a
          DHCP reply, adds the DHCP offer IP <var>O</var> and options to the
          packet, and stores 1 into reg0[3].  For other kinds of packets, it
          just stores 0 into reg0[3].  Either way, it continues to the next
          table.
        </p>

      </li>

      <li>
        A priority-0 flow that matches all packets to advances to table 11.
      </li>
    </ul>

    <h3>Ingress Table 11: DHCP responses</h3>

    <p>
      This table implements DHCP responder for the DHCP replies generated by
      the previous table.
    </p>

    <ul>
      <li>
        <p>
          A priority 100 logical flow is added for the logical ports configured
          with DHCPv4 options which matches IPv4 packets with <code>udp.src == 68
          &amp;&amp; udp.dst == 67 &amp;&amp; reg0[3] == 1</code> and
          responds back to the <code>inport</code> after applying these
          actions.  If <code>reg0[3]</code> is set to 1, it means that the
          action <code>put_dhcp_opts</code> was successful.
        </p>

        <pre>
eth.dst = eth.src;
eth.src = <var>E</var>;
ip4.dst = <var>O</var>;
ip4.src = <var>S</var>;
udp.src = 67;
udp.dst = 68;
outport = <var>P</var>;
inport = ""; /* Allow sending out inport. */
output;
        </pre>

        <p>
          where <var>E</var> is the server MAC address and <var>S</var> is the
          server IPv4 address defined in the DHCPv4 options and <var>O</var> is
          the IPv4 address defined in the logical port's addresses column.
        </p>

        <p>
          (This terminates ingress packet processing; the packet does not go
           to the next ingress table.)
        </p>
      </li>

      <li>
        A priority-0 flow that matches all packets to advances to table 12.
      </li>
    </ul>

    <h3>Ingress Table 12: Destination Lookup</h3>

    <p>
      This table implements switching behavior.  It contains these logical
      flows:
    </p>

    <ul>
      <li>
        A priority-100 flow that outputs all packets with an Ethernet broadcast
        or multicast <code>eth.dst</code> to the <code>MC_FLOOD</code>
        multicast group, which <code>ovn-northd</code> populates with all
        enabled logical ports.
      </li>

      <li>
        One priority-50 flow that matches each known Ethernet address against
        <code>eth.dst</code> and outputs the packet to the single associated
        output port.
      </li>

      <li>
        One priority-0 fallback flow that matches all packets and outputs them
        to the <code>MC_UNKNOWN</code> multicast group, which
        <code>ovn-northd</code> populates with all enabled logical ports that
        accept unknown destination packets.  As a small optimization, if no
        logical ports accept unknown destination packets,
        <code>ovn-northd</code> omits this multicast group and logical flow.
      </li>
    </ul>

    <h3>Egress Table 0: Pre-LB</h3>

    <p>
      This table is similar to ingress table <code>Pre-LB</code>.  It
      contains a priority-0 flow that simply moves traffic to the next table.
      If any load balancing rules exist for the datapath, a priority-100 flow
      is added with a match of <code>ip</code> and action of <code>reg0[0] = 1;
       next;</code> to act as a hint for table <code>Pre-stateful</code> to
      send IP packets to the connection tracker for packet de-fragmentation.
    </p>

    <h3>Egress Table 1: <code>to-lport</code> Pre-ACLs</h3>

    <p>
      This is similar to ingress table <code>Pre-ACLs</code> except for
     <code>to-lport</code> traffic.
    </p>

    <h3>Egress Table 2: Pre-stateful</h3>

    <p>
      This is similar to ingress table <code>Pre-stateful</code>.
    </p>

    <h3>Egress Table 3: LB</h3>
    <p>
      This is similar to ingress table <code>LB</code>.
    </p>

    <h3>Egress Table 4: <code>to-lport</code> ACLs</h3>

    <p>
      This is similar to ingress table <code>ACLs</code> except for
      <code>to-lport</code> ACLs.
    </p>

    <h3>Egress Table 5: Stateful</h3>

    <p>
      This is similar to ingress table <code>Stateful</code> except that
      there are no rules added for load balancing new connections.
    </p>

    <p>
      Also a priority 34000 logical flow is added for each logical port which
      has DHCPv4 options defined to allow the DHCPv4 reply packet from the
      <code>Ingress Table 11: DHCP responses</code>.
    </p>

    <h3>Egress Table 6: Egress Port Security - IP</h3>

    <p>
      This is similar to the port security logic in table
      <code>Ingress Port Security - IP</code> except that <code>outport</code>,
      <code>eth.dst</code>, <code>ip4.dst</code> and <code>ip6.dst</code>
      are checked instead of <code>inport</code>, <code>eth.src</code>,
      <code>ip4.src</code> and <code>ip6.src</code>
    </p>

    <h3>Egress Table 7: Egress Port Security - L2</h3>

    <p>
      This is similar to the ingress port security logic in ingress table
      <code>Admission Control and Ingress Port Security - L2</code>,
      but with important differences.  Most obviously, <code>outport</code> and
      <code>eth.dst</code> are checked instead of <code>inport</code> and
      <code>eth.src</code>.  Second, packets directed to broadcast or multicast
      <code>eth.dst</code> are always accepted instead of being subject to the
      port security rules; this is implemented through a priority-100 flow that
      matches on <code>eth.mcast</code> with action <code>output;</code>.
      Finally, to ensure that even broadcast and multicast packets are not
      delivered to disabled logical ports, a priority-150 flow for each
      disabled logical <code>outport</code> overrides the priority-100 flow
      with a <code>drop;</code> action.
    </p>

    <h2>Logical Router Datapaths</h2>

    <p>
      Logical router datapaths will only exist for <ref table="Logical_Router"
      db="OVN_Northbound"/> rows in the <ref db="OVN_Northbound"/> database
      that do not have <ref column="enabled" table="Logical_Router"
      db="OVN_Northbound"/> set to <code>false</code>
    </p>

    <h3>Ingress Table 0: L2 Admission Control</h3>

    <p>
      This table drops packets that the router shouldn't see at all based on
      their Ethernet headers.  It contains the following flows:
    </p>

    <ul>
      <li>
        Priority-100 flows to drop packets with VLAN tags or multicast Ethernet
        source addresses.
      </li>

      <li>
        For each enabled router port <var>P</var> with Ethernet address
        <var>E</var>, a priority-50 flow that matches <code>inport ==
        <var>P</var> &amp;&amp; (eth.mcast || eth.dst ==
        <var>E</var></code>), with action <code>next;</code>.
      </li>
    </ul>

    <p>
      Other packets are implicitly dropped.
    </p>

    <h3>Ingress Table 1: IP Input</h3>

    <p>
      This table is the core of the logical router datapath functionality.  It
      contains the following flows to implement very basic IP host
      functionality.
    </p>

    <ul>
      <li>
        <p>
          L3 admission control: A priority-100 flow drops packets that match
          any of the following:
        </p>

        <ul>
          <li>
            <code>ip4.src[28..31] == 0xe</code> (multicast source)
          </li>
          <li>
            <code>ip4.src == 255.255.255.255</code> (broadcast source)
          </li>
          <li>
            <code>ip4.src == 127.0.0.0/8 || ip4.dst == 127.0.0.0/8</code>
            (localhost source or destination)
          </li>
          <li>
            <code>ip4.src == 0.0.0.0/8 || ip4.dst == 0.0.0.0/8</code> (zero
            network source or destination)
          </li>
          <li>
            <code>ip4.src</code> is any IP address owned by the router.
          </li>
          <li>
            <code>ip4.src</code> is the broadcast address of any IP network
            known to the router.
          </li>
        </ul>
      </li>

      <li>
        <p>
          ICMP echo reply.  These flows reply to ICMP echo requests received
          for the router's IP address.  Let <var>A</var> be an IP address
          owned by a router port.  Then, for each <var>A</var>, a priority-90
          flow matches on <code>ip4.dst == <var>A</var></code> and <code>
          icmp4.type == 8 &amp;&amp; icmp4.code == 0</code> (ICMP echo
          request).  The port of the router that receives the echo request
          does not matter. Also, the ip.ttl of the echo request packet is not
          checked, so it complies with RFC 1812, section 4.2.2.9. These flows
          use the following actions:
        </p>

        <pre>
ip4.dst &lt;-&gt; ip4.src;
ip.ttl = 255;
icmp4.type = 0;
inport = ""; /* Allow sending out inport. */
next;
        </pre>
      </li>

      <li>
        <p>
          Reply to ARP requests.
        </p>

        <p>
          These flows reply to ARP requests for the router's own IP address.
          For each router port <var>P</var> that owns IP address <var>A</var>
          and Ethernet address <var>E</var>, a priority-90 flow matches
          <code>inport == <var>P</var> &amp;&amp; arp.op == 1 &amp;&amp;
          arp.tpa == <var>A</var></code> (ARP request) with the following
          actions:
        </p>

        <pre>
eth.dst = eth.src;
eth.src = <var>E</var>;
arp.op = 2; /* ARP reply. */
arp.tha = arp.sha;
arp.sha = <var>E</var>;
arp.tpa = arp.spa;
arp.spa = <var>A</var>;
outport = <var>P</var>;
inport = ""; /* Allow sending out inport. */
output;
        </pre>
      </li>

      <li>
        <p>
          These flows reply to ARP requests for the virtual IP addresses
          configured in the router for DNAT. For a configured DNAT IP address
          <var>A</var>, for each router port <var>P</var> with Ethernet
          address <var>E</var>, a priority-90 flow matches
          <code>inport == <var>P</var> &amp;&amp; arp.op == 1 &amp;&amp;
          arp.tpa == <var>A</var></code> (ARP request)
          with the following actions:
        </p>

        <pre>
eth.dst = eth.src;
eth.src = <var>E</var>;
arp.op = 2; /* ARP reply. */
arp.tha = arp.sha;
arp.sha = <var>E</var>;
arp.tpa = arp.spa;
arp.spa = <var>A</var>;
outport = <var>P</var>;
inport = ""; /* Allow sending out inport. */
output;
        </pre>
      </li>

      <li>
        ARP reply handling.  These flows use ARP replies to populate the
        logical router's ARP table.  A priority-90 flow with match <code>arp.op
        == 2</code> has actions <code>put_arp(inport, arp.spa,
        arp.sha);</code>.
      </li>

      <li>
        <p>
          UDP port unreachable.  Priority-80 flows generate ICMP port
          unreachable messages in reply to UDP datagrams directed to the
          router's IP address.  The logical router doesn't accept any UDP
          traffic so it always generates such a reply.
        </p>

        <p>
          These flows should not match IP fragments with nonzero offset.
        </p>

        <p>
          Details TBD.  Not yet implemented.
        </p>
      </li>

      <li>
        <p>
          TCP reset.  Priority-80 flows generate TCP reset messages in reply to
          TCP datagrams directed to the router's IP address.  The logical
          router doesn't accept any TCP traffic so it always generates such a
          reply.
        </p>

        <p>
          These flows should not match IP fragments with nonzero offset.
        </p>

        <p>
          Details TBD.  Not yet implemented.
        </p>
      </li>

      <li>
        <p>
          Protocol unreachable.  Priority-70 flows generate ICMP protocol
          unreachable messages in reply to packets directed to the router's IP
          address on IP protocols other than UDP, TCP, and ICMP.
        </p>

        <p>
          These flows should not match IP fragments with nonzero offset.
        </p>

        <p>
          Details TBD.  Not yet implemented.
        </p>
      </li>

      <li>
        Drop other IP traffic to this router.  These flows drop any other
        traffic destined to an IP address of this router that is not already
        handled by one of the flows above, which amounts to ICMP (other than
        echo requests) and fragments with nonzero offsets.  For each IP address
        <var>A</var> owned by the router, a priority-60 flow matches
        <code>ip4.dst == <var>A</var></code> and drops the traffic.  An
        exception is made and the above flow is not added if the router
        port's own IP address is used to SNAT packets passing through that
        router.
      </li>
    </ul>

    <p>
      The flows above handle all of the traffic that might be directed to the
      router itself.  The following flows (with lower priorities) handle the
      remaining traffic, potentially for forwarding:
    </p>

    <ul>
      <li>
        Drop Ethernet local broadcast.  A priority-50 flow with match
        <code>eth.bcast</code> drops traffic destined to the local Ethernet
        broadcast address.  By definition this traffic should not be forwarded.
      </li>

      <li>
        <p>
          ICMP time exceeded.  For each router port <var>P</var>, whose IP
          address is <var>A</var>, a priority-40 flow with match <code>inport
          == <var>P</var> &amp;&amp; ip.ttl == {0, 1} &amp;&amp;
          !ip.later_frag</code> matches packets whose TTL has expired, with the
          following actions to send an ICMP time exceeded reply:
        </p>

        <pre>
icmp4 {
    icmp4.type = 11; /* Time exceeded. */
    icmp4.code = 0;  /* TTL exceeded in transit. */
    ip4.dst = ip4.src;
    ip4.src = <var>A</var>;
    ip.ttl = 255;
    next;
};
        </pre>

        <p>
          Not yet implemented.
        </p>
      </li>

      <li>
        TTL discard.  A priority-30 flow with match <code>ip.ttl == {0,
        1}</code> and actions <code>drop;</code> drops other packets whose TTL
        has expired, that should not receive a ICMP error reply (i.e. fragments
        with nonzero offset).
      </li>

      <li>
        Next table.  A priority-0 flows match all packets that aren't already
        handled and uses actions <code>next;</code> to feed them to the ingress
        table for routing.
      </li>
    </ul>

    <h3>Ingress Table 2: UNSNAT</h3>

    <p>
      This is for already established connections' reverse traffic.
      i.e., SNAT has already been done in egress pipeline and now the
      packet has entered the ingress pipeline as part of a reply.  It is
      unSNATted here.
    </p>

    <ul>
      <li>
        <p>
          For each configuration in the OVN Northbound database, that asks
          to change the source IP address of a packet from <var>A</var> to
          <var>B</var>, a priority-100 flow matches <code>ip &amp;&amp;
          ip4.dst == <var>B</var></code> with an action
          <code>ct_snat; next;</code>.
        </p>

        <p>
          A priority-0 logical flow with match <code>1</code> has actions
          <code>next;</code>.
        </p>
      </li>
    </ul>

    <h3>Ingress Table 3: DNAT</h3>

    <p>
      Packets enter the pipeline with destination IP address that needs to
      be DNATted from a virtual IP address to a real IP address.  Packets
      in the reverse direction needs to be unDNATed.
    </p>
    <ul>
      <li>
        <p>
          For each configuration in the OVN Northbound database, that asks
          to change the destination IP address of a packet from <var>A</var> to
          <var>B</var>, a priority-100 flow matches <code>ip &amp;&amp;
          ip4.dst == <var>A</var></code> with an action <code>inport = "";
          ct_dnat(<var>B</var>);</code>.
        </p>

        <p>
          For all IP packets of a Gateway router, a priority-50 flow with an
          action <code>inport = ""; ct_dnat;</code>.
        </p>

        <p>
          A priority-0 logical flow with match <code>1</code> has actions
          <code>next;</code>.
        </p>
      </li>
    </ul>

    <h3>Ingress Table 4: IP Routing</h3>

    <p>
      A packet that arrives at this table is an IP packet that should be routed
      to the address in <code>ip4.dst</code>.  This table implements IP
      routing, setting <code>reg0</code> to the next-hop IP address (leaving
      <code>ip4.dst</code>, the packet's final destination, unchanged) and
      advances to the next table for ARP resolution.  It also sets
      <code>reg1</code> to the IP address owned by the selected router port
      (which is used later in table 6 as the IP source address for an ARP
      request, if needed).
    </p>

    <p>
      This table contains the following logical flows:
    </p>

    <ul>
      <li>
        <p>
          Routing table.  For each route to IPv4 network <var>N</var> with
          netmask <var>M</var>, on router port <var>P</var> with IP address
          <var>A</var> and Ethernet
          address <var>E</var>, a logical flow with match <code>ip4.dst ==
          <var>N</var>/<var>M</var></code>, whose priority is the number of
          1-bits in <var>M</var>, has the following actions:
        </p>

        <pre>
ip.ttl--;
reg0 = <var>G</var>;
reg1 = <var>A</var>;
eth.src = <var>E</var>;
outport = <var>P</var>;
inport = ""; /* Allow sending out inport. */
next;
        </pre>

        <p>
          (Ingress table 1 already verified that <code>ip.ttl--;</code> will
          not yield a TTL exceeded error.)
        </p>

        <p>
          If the route has a gateway, <var>G</var> is the gateway IP address.
          Instead, if the route is from a configured static route, <var>G</var>
          is the next hop IP address.  Else it is <code>ip4.dst</code>.
        </p>
      </li>

      <li>
        <p>
          Destination unreachable.  For each router port <var>P</var>, which
          owns IP address <var>A</var>, a priority-0 logical flow with match
          <code>in_port == <var>P</var> &amp;&amp; !ip.later_frag &amp;&amp;
          !icmp</code> has the following actions:
        </p>

        <pre>
icmp4 {
    icmp4.type = 3; /* Destination unreachable. */
    icmp4.code = 0; /* Network unreachable. */
    ip4.dst = ip4.src;
    ip4.src = <var>A</var>;
    ip.ttl = 255;
    next(2);
};
        </pre>

        <p>
          (The <code>!icmp</code> check prevents recursion if the destination
          unreachable message itself cannot be routed.)
        </p>

        <p>
          These flows are omitted if the logical router has a default route,
          that is, a route with netmask 0.0.0.0.
        </p>
      </li>
    </ul>

    <h3>Ingress Table 5: ARP Resolution</h3>

    <p>
      Any packet that reaches this table is an IP packet whose next-hop IP
      address is in <code>reg0</code>.  (<code>ip4.dst</code> is the final
      destination.)  This table resolves the IP address in <code>reg0</code>
      into an output port in <code>outport</code> and an Ethernet address in
      <code>eth.dst</code>, using the following flows:
    </p>

    <ul>
      <li>
        <p>
          Static MAC bindings.  MAC bindings can be known statically based on
          data in the <code>OVN_Northbound</code> database.  For router ports
          connected to logical switches, MAC bindings can be known statically
          from the <code>addresses</code> column in the
          <code>Logical_Switch_Port</code> table.  For router ports
          connected to other logical routers, MAC bindings can be known
          statically from the <code>mac</code> and <code>networks</code>
          column in the <code>Logical_Router_Port</code> table.
        </p>

        <p>
          For each IP address <var>A</var> whose host is known to have Ethernet
          address <var>E</var> on router port <var>P</var>, a priority-100 flow
          with match <code>outport === <var>P</var> &amp;&amp; reg0 ==
          <var>A</var></code> has actions <code>eth.dst = <var>E</var>;
          next;</code>.
        </p>

        <p>
          For each logical router port with an IP address <var>A</var> and
          a mac address of <var>E</var> that is reachable via a different
          logical router port <var>P</var>, a priority-100 flow with
          match <code>outport === <var>P</var> &amp;&amp; reg0 ==
          <var>A</var></code> has actions <code>eth.dst = <var>E</var>;
          next;</code>.
        </p>
      </li>

      <li>
        <p>
          Dynamic MAC bindings.  This flows resolves MAC-to-IP bindings that
          have become known dynamically through ARP.  (The next table will
          issue an ARP request for cases where the binding is not yet known.)
        </p>

        <p>
          A priority-0 logical flow with match <code>1</code> has actions
          <code>get_arp(outport, reg0); next;</code>.
        </p>
      </li>
    </ul>

    <h3>Ingress Table 6: ARP Request</h3>

    <p>
      In the common case where the Ethernet destination has been resolved, this
      table outputs the packet.  Otherwise, it composes and sends an ARP
      request.  It holds the following flows:
    </p>

    <ul>
      <li>
        <p>
          Unknown MAC address.  A priority-100 flow with match <code>eth.dst ==
          00:00:00:00:00:00</code> has the following actions:
        </p>

        <pre>
arp {
    eth.dst = ff:ff:ff:ff:ff:ff;
    arp.spa = reg1;
    arp.op = 1;  /* ARP request. */
    output;
};
        </pre>

        <p>
          (Ingress table 4 initialized <code>reg1</code> with the IP address
          owned by <code>outport</code>.)
        </p>

        <p>
          The IP packet that triggers the ARP request is dropped.
        </p>
      </li>

      <li>
        Known MAC address.  A priority-0 flow with match <code>1</code> has
        actions <code>output;</code>.
      </li>
    </ul>

    <h3>Egress Table 0: SNAT</h3>

    <p>
      Packets that are configured to be SNATed get their source IP address
      changed based on the configuration in the OVN Northbound database.
    </p>
    <ul>
      <li>
        <p>
          For each configuration in the OVN Northbound database, that asks
          to change the source IP address of a packet from an IP address of
          <var>A</var> or to change the source IP address of a packet that
          belongs to network <var>A</var> to <var>B</var>, a flow matches
          <code>ip &amp;&amp; ip4.src == <var>A</var></code> with an action
          <code>ct_snat(<var>B</var>);</code>.  The priority of the flow
          is calculated based on the mask of <var>A</var>, with matches
          having larger masks getting higher priorities.
        </p>
        <p>
          A priority-0 logical flow with match <code>1</code> has actions
          <code>next;</code>.
        </p>
      </li>
    </ul>

    <h3>Egress Table 1: Delivery</h3>

    <p>
      Packets that reach this table are ready for delivery.  It contains
      priority-100 logical flows that match packets on each enabled logical
      router port, with action <code>output;</code>.
    </p>

</manpage>