ovn: Apply ACL changes to existing connections.

ovn: Apply ACL changes to existing connections.

Prior to this commit, once a connection had been committed to the
connection tracker, the connection would continue to be allowed, even
if the policy defined in the ACL table changed.  This patch changes
the implementation so that existing connections are affected by policy
changes.

The implementation is based on the suggested approach in this mailing
list thread:

    http://openvswitch.org/pipermail/dev/2016-February/065716.html

Instead of always allowing packets associated with an established
connection, we now put all packets in the request direction through
the flows generated based on OVN ACLs.  If a packet associated with an
established connection hits a "drop" ACL, that means we have
encountered a policy change and should drop packets associated with
this connection from now on.  We handle this by setting "ct_label" on
the associated connection tracking entry.

These changes also account for re-allowing a known connection after
ct_label had been set on it. This can happen if you delete an ACL and
then re-create it while connection state is still known.

The proposal on the mailing list also discussed the idea that
ovn-controller could periodically sweep the connection tracker and
delete entries with ct_label set.  That is not implemented in this
patch.  Instead, we rely on connections dying since we're dropping
its packets and then allowing the connection tracking entry to
eventually time out.  More proactively clearing them out could be a
future enhancement.

As a realistic example of how this works, consider this security policy
from an OpenStack+OVN development environment.

    +---------+-----------------------+
    | name    | security_group_rules  |
    +---------+-----------------------+
    | default | egress, IPv4          |
    |         | egress, IPv6          |
    |         | ingress, IPv4, 22/tcp |
    |         | ingress, IPv4, icmp   |
    +---------+-----------------------+

The OpenStack Neutron plugin creates ACLs that drop traffic by default
and higher priority ACLs for each type of traffic that is allowed.  In
this case, the ACLs for a port using the "default" security group are:

  from-lport  1002 (inport == "23706cbe-98b6-4a8b-b78b-a8e12e6d773f" && ip4) allow-related
  from-lport  1002 (inport == "23706cbe-98b6-4a8b-b78b-a8e12e6d773f" && ip6) allow-related
  from-lport  1001 (inport == "23706cbe-98b6-4a8b-b78b-a8e12e6d773f" && ip) drop
    to-lport  1002 (outport == "23706cbe-98b6-4a8b-b78b-a8e12e6d773f" && ip4 && icmp4) allow-related
    to-lport  1002 (outport == "23706cbe-98b6-4a8b-b78b-a8e12e6d773f" && ip4 && tcp && tcp.dst == 22) allow-related
    to-lport  1001 (outport == "23706cbe-98b6-4a8b-b78b-a8e12e6d773f" && ip) drop

which results in the following logical flows:

  table=3 (ls_in_pre_acl      ), priority=100  , match=(ip), action=(reg0[0] = 1; next;)
  table=3 (ls_in_pre_acl      ), priority=0    , match=(1), action=(next;)
  table=4 (ls_in_pre_lb       ), priority=0    , match=(1), action=(next;)
  table=5 (ls_in_pre_stateful ), priority=100  , match=(reg0[0] == 1), action=(ct_next;)
  table=5 (ls_in_pre_stateful ), priority=0    , match=(1), action=(next;)
  table=6 (ls_in_acl          ), priority=65535, match=(!ct.est && ct.rel && !ct.new && !ct.inv && ct_label[0] == 0), action=(next;)
  table=6 (ls_in_acl          ), priority=65535, match=(ct.est && !ct.rel && !ct.new && !ct.inv && ct.rpl && ct_label[0] == 0), action=(next;)
  table=6 (ls_in_acl          ), priority=65535, match=(ct.inv || (ct.est && ct.rpl && ct_label[0] == 1)), action=(drop;)
  table=6 (ls_in_acl          ), priority=65535, match=(nd), action=(next;)
  table=6 (ls_in_acl          ), priority=2002 , match=(!ct.new && ct.est && !ct.rpl && ct_label[0] == 0 && (inport == "23706cbe-98b6-4a8b-b78b-a8e12e6d773f" && ip4 && (ip4.dst == 255.255.255.255 || ip4.dst == 10.0.0.0/24) && udp && udp.src == 68 && udp.dst == 67)), action=(next;)
  table=6 (ls_in_acl          ), priority=2002 , match=(!ct.new && ct.est && !ct.rpl && ct_label[0] == 0 && (inport == "23706cbe-98b6-4a8b-b78b-a8e12e6d773f" && ip4)), action=(next;)
  table=6 (ls_in_acl          ), priority=2002 , match=(!ct.new && ct.est && !ct.rpl && ct_label[0] == 0 && (inport == "23706cbe-98b6-4a8b-b78b-a8e12e6d773f" && ip6)), action=(next;)
  table=6 (ls_in_acl          ), priority=2002 , match=(((ct.new && !ct.est) || (!ct.new && ct.est && !ct.rpl && ct_label[0] == 1)) && (inport == "23706cbe-98b6-4a8b-b78b-a8e12e6d773f" && ip4 && (ip4.dst == 255.255.255.255 || ip4.dst == 10.0.0.0/24) && udp && udp.src == 68 && udp.dst == 67)), action=(reg0[1] = 1; next;)
  table=6 (ls_in_acl          ), priority=2002 , match=(((ct.new && !ct.est) || (!ct.new && ct.est && !ct.rpl && ct_label[0] == 1)) && (inport == "23706cbe-98b6-4a8b-b78b-a8e12e6d773f" && ip4)), action=(reg0[1] = 1; next;)
  table=6 (ls_in_acl          ), priority=2002 , match=(((ct.new && !ct.est) || (!ct.new && ct.est && !ct.rpl && ct_label[0] == 1)) && (inport == "23706cbe-98b6-4a8b-b78b-a8e12e6d773f" && ip6)), action=(reg0[1] = 1; next;)
  table=6 (ls_in_acl          ), priority=2001 , match=((!ct.est || (ct.est && ct_label[0] == 1)) && (inport == "23706cbe-98b6-4a8b-b78b-a8e12e6d773f" && ip)), action=(drop;)
  table=6 (ls_in_acl          ), priority=2001 , match=(ct.est && ct_label[0] == 0 && (inport == "23706cbe-98b6-4a8b-b78b-a8e12e6d773f" && ip)), action=(ct_commit(ct_label=1/1);)
  table=6 (ls_in_acl          ), priority=1    , match=(ip && (!ct.est || (ct.est && ct_label[0] == 1))), action=(reg0[1] = 1; next;)
  table=6 (ls_in_acl          ), priority=0    , match=(1), action=(next;)
  table=7 (ls_in_lb           ), priority=0    , match=(1), action=(next;)
  table=8 (ls_in_stateful     ), priority=100  , match=(reg0[1] == 1), action=(ct_commit(ct_label=0/1); next;)
  table=8 (ls_in_stateful     ), priority=100  , match=(reg0[2] == 1), action=(ct_lb;)
  table=8 (ls_in_stateful     ), priority=0    , match=(1), action=(next;)

  table=0 (ls_out_pre_lb      ), priority=0    , match=(1), action=(next;)
  table=1 (ls_out_pre_acl     ), priority=110  , match=(ip && outport == "351f0012-0c13-4330-b471-b0d4719c5031"), action=(next;)
  table=1 (ls_out_pre_acl     ), priority=110  , match=(ip && outport == "4e0e294d-e54a-400c-a240-f121175904c2"), action=(next;)
  table=1 (ls_out_pre_acl     ), priority=110  , match=(nd), action=(next;)
  table=1 (ls_out_pre_acl     ), priority=100  , match=(ip), action=(reg0[0] = 1; next;)
  table=1 (ls_out_pre_acl     ), priority=0    , match=(1), action=(next;)
  table=2 (ls_out_pre_stateful), priority=100  , match=(reg0[0] == 1), action=(ct_next;)
  table=2 (ls_out_pre_stateful), priority=0    , match=(1), action=(next;)
  table=3 (ls_out_lb          ), priority=0    , match=(1), action=(next;)
  table=4 (ls_out_acl         ), priority=65535, match=(!ct.est && ct.rel && !ct.new && !ct.inv && ct_label[0] == 0), action=(next;)
  table=4 (ls_out_acl         ), priority=65535, match=(ct.est && !ct.rel && !ct.new && !ct.inv && ct.rpl && ct_label[0] == 0), action=(next;)
  table=4 (ls_out_acl         ), priority=65535, match=(ct.inv || (ct.est && ct.rpl && ct_label[0] == 1)), action=(drop;)
  table=4 (ls_out_acl         ), priority=65535, match=(nd), action=(next;)
  table=4 (ls_out_acl         ), priority=2002 , match=(!ct.new && ct.est && !ct.rpl && ct_label[0] == 0 && (outport == "23706cbe-98b6-4a8b-b78b-a8e12e6d773f" && ip4 && icmp4)), action=(next;)
  table=4 (ls_out_acl         ), priority=2002 , match=(!ct.new && ct.est && !ct.rpl && ct_label[0] == 0 && (outport == "23706cbe-98b6-4a8b-b78b-a8e12e6d773f" && ip4 && ip4.src == $as_ip4_85300131_274c_492c_a000_b1782315196d)), action=(next;)
  table=4 (ls_out_acl         ), priority=2002 , match=(!ct.new && ct.est && !ct.rpl && ct_label[0] == 0 && (outport == "23706cbe-98b6-4a8b-b78b-a8e12e6d773f" && ip4 && ip4.src == 10.0.0.0/24 && udp && udp.src == 67 && udp.dst == 68)), action=(next;)
  table=4 (ls_out_acl         ), priority=2002 , match=(!ct.new && ct.est && !ct.rpl && ct_label[0] == 0 && (outport == "23706cbe-98b6-4a8b-b78b-a8e12e6d773f" && ip6 && ip6.src == $as_ip6_85300131_274c_492c_a000_b1782315196d)), action=(next;)
  table=4 (ls_out_acl         ), priority=2002 , match=(((ct.new && !ct.est) || (!ct.new && ct.est && !ct.rpl && ct_label[0] == 1)) && (outport == "23706cbe-98b6-4a8b-b78b-a8e12e6d773f" && ip4 && icmp4)), action=(reg0[1] = 1; next;)
  table=4 (ls_out_acl         ), priority=2002 , match=(((ct.new && !ct.est) || (!ct.new && ct.est && !ct.rpl && ct_label[0] == 1)) && (outport == "23706cbe-98b6-4a8b-b78b-a8e12e6d773f" && ip4 && ip4.src == $as_ip4_85300131_274c_492c_a000_b1782315196d)), action=(reg0[1] = 1; next;)
  table=4 (ls_out_acl         ), priority=2002 , match=(((ct.new && !ct.est) || (!ct.new && ct.est && !ct.rpl && ct_label[0] == 1)) && (outport == "23706cbe-98b6-4a8b-b78b-a8e12e6d773f" && ip4 && ip4.src == 10.0.0.0/24 && udp && udp.src == 67 && udp.dst == 68)), action=(reg0[1] = 1; next;)
  table=4 (ls_out_acl         ), priority=2002 , match=(((ct.new && !ct.est) || (!ct.new && ct.est && !ct.rpl && ct_label[0] == 1)) && (outport == "23706cbe-98b6-4a8b-b78b-a8e12e6d773f" && ip6 && ip6.src == $as_ip6_85300131_274c_492c_a000_b1782315196d)), action=(reg0[1] = 1; next;)
  table=4 (ls_out_acl         ), priority=2001 , match=((!ct.est || (ct.est && ct_label[0] == 1)) && (outport == "23706cbe-98b6-4a8b-b78b-a8e12e6d773f" && ip)), action=(drop;)
  table=4 (ls_out_acl         ), priority=2001 , match=(ct.est && ct_label[0] == 0 && (outport == "23706cbe-98b6-4a8b-b78b-a8e12e6d773f" && ip)), action=(ct_commit(ct_label=1/1);)
  table=4 (ls_out_acl         ), priority=1    , match=(ip && (!ct.est || (ct.est && ct_label[0] == 1))), action=(reg0[1] = 1; next;)
  table=4 (ls_out_acl         ), priority=0    , match=(1), action=(next;)
  table=5 (ls_out_stateful    ), priority=100  , match=(reg0[1] == 1), action=(ct_commit(ct_label=0/1); next;)
  table=5 (ls_out_stateful    ), priority=100  , match=(reg0[2] == 1), action=(ct_lb;)
  table=5 (ls_out_stateful    ), priority=0    , match=(1), action=(next;)

One way I tested this by leaving ping running, ensuring that it was
blocked when the rule for ICMP was deleted, and then re-allowed when
the rule allowing ICMP was restored.  In this case, the ICMP
connection is still known by the connection tracker, but the flows
ensure that ct_label gets reset back to 0.

Reported-by: Xiao Li Xu <xiaolixu@cn.ibm.com>
Reported-at: https://bugs.launchpad.net/networking-ovn/+bug/1536080
Suggested-by: Justin Pettit <jpettit@ovn.org>
Signed-off-by: Russell Bryant <russell@ovn.org>
Acked-by: Han Zhou <zhouhan@gmail.com>
Acked-by: Ben Pfaff <blp@ovn.org>
Acked-by: Justin Pettit <jpettit@ovn.org>
Tested-by: Babu Shanmugam <bschanmu@redhat.com>