ipfix: Support tunnel information for Flow IPFIX.
authorBenli Ye <daniely@vmware.com>
Tue, 14 Jun 2016 08:53:34 +0000 (16:53 +0800)
committerBen Pfaff <blp@ovn.org>
Fri, 17 Jun 2016 17:21:11 +0000 (10:21 -0700)
Add support to export tunnel information for flow-based IPFIX.
The original steps to configure flow level IPFIX:
    1) Create a new record in Flow_Sample_Collector_Set table:
       'ovs-vsctl -- create Flow_Sample_Collector_Set id=1 bridge="Bridge UUID"'
    2) Add IPFIX configuration which is referred by corresponding
       row in Flow_Sample_Collector_Set table:
       'ovs-vsctl -- set Flow_Sample_Collector_Set
       "Flow_Sample_Collector_Set UUID" ipfix=@i -- --id=@i create IPFIX
       targets=\"IP:4739\" obs_domain_id=123 obs_point_id=456
       cache_active_timeout=60 cache_max_flows=13'
    3) Add sample action to the flows:
       'ovs-ofctl add-flow mybridge in_port=1,
       actions=sample'('probability=65535,collector_set_id=1,
       obs_domain_id=123,obs_point_id=456')',output:3'
NXAST_SAMPLE action was used in step 3. In order to support exporting tunnel
information, the NXAST_SAMPLE2 action was added and with NXAST_SAMPLE2 action
in this patch, the step 3 should be configured like below:
       'ovs-ofctl add-flow mybridge in_port=1,
       actions=sample'('probability=65535,collector_set_id=1,obs_domain_id=123,
       obs_point_id=456,sampling_port=3')',output:3'
'sampling_port' can be equal to ingress port or one of egress ports. If sampling
port is equal to output port and the output port is a tunnel port,
OVS_USERSPACE_ATTR_EGRESS_TUN_PORT will be set in the datapath flow sample action.
When flow sample action upcall happens, tunnel information will be retrieved from
the datapath and then IPFIX can export egress tunnel port information. If
samping_port=65535 (OFPP_NONE), flow-based IPFIX will keep the same behavior
as before.

This patch mainly do three tasks:
    1) Add a new flow sample action NXAST_SAMPLE2 to support exporting
       tunnel information. NXAST_SAMPLE2 action has a new added field
       'sampling_port'.
    2) Use 'other_configure: enable-tunnel-sampling' to enable or disable
       exporting tunnel information.
    3) If 'sampling_port' is equal to output port and output port is a tunnel
       port, the translation of OpenFlow "sample" action should first emit
       set(tunnel(...)), then the sample action itself. It makes sure the
       egress tunnel information can be sampled.
    4) Add a test of flow-based IPFIX for tunnel set.

How to test flow-based IPFIX:
    1) Setup a test environment with two Linux host with Docker supported
    2) Create a Docker container and a GRE tunnel port on each host
    3) Use ovs-docker to add the container on the bridge
    4) Listen on port 4739 on the collector machine and use wireshark to filter
       'cflow' packets.
    5) Configure flow-based IPFIX:
       - 'ovs-vsctl -- create Flow_Sample_Collector_Set id=1 bridge="Bridge UUID"'
       - 'ovs-vsctl -- set Flow_Sample_Collector_Set
          "Flow_Sample_Collector_Set UUID" ipfix=@i -- --id=@i create IPFIX \
          targets=\"IP:4739\" cache_active_timeout=60 cache_max_flows=13 \
          other_config:enable-tunnel-sampling=true'
       - 'ovs-ofctl add-flow mybridge in_port=1,
          actions=sample'('probability=65535,collector_set_id=1,obs_domain_id=123,
          obs_point_id=456,sampling_port=3')',output:3'
       Note: The in-port is container port. The output port and sampling_port
             are both open flow port and the output port is a GRE tunnel port.
    6) Ping from the container whose host enabled flow-based IPFIX.
    7) Get the IPFIX template pakcets and IPFIX information packets.

Signed-off-by: Benli Ye <daniely@vmware.com>
Signed-off-by: Ben Pfaff <blp@ovn.org>
17 files changed:
NEWS
include/openvswitch/ofp-actions.h
lib/odp-util.c
lib/odp-util.h
lib/ofp-actions.c
ofproto/ofproto-dpif-ipfix.c
ofproto/ofproto-dpif-ipfix.h
ofproto/ofproto-dpif-upcall.c
ofproto/ofproto-dpif-xlate.c
ofproto/ofproto.h
tests/odp.at
tests/ofp-actions.at
tests/ofproto-dpif.at
tests/ovs-ofctl.at
utilities/ovs-ofctl.8.in
vswitchd/bridge.c
vswitchd/vswitch.xml

diff --git a/NEWS b/NEWS
index 08094c5..7689183 100644 (file)
--- a/NEWS
+++ b/NEWS
@@ -15,6 +15,8 @@ Post-v2.5.0
        now implemented.  Only flow mod and port mod messages are supported
        in bundles.
      * New OpenFlow extension NXM_NX_MPLS_TTL to provide access to MPLS TTL.
+     * New "sampling_port" option for "sample" action to allow sampling
+       ingress and egress tunnel metadata.
    - ovs-ofctl:
      * queue-get-config command now allows a queue ID to be specified.
      * '--bundle' option can now be used with OpenFlow 1.3.
index 038ef87..91c7ee5 100644 (file)
@@ -775,13 +775,14 @@ struct ofpact_note {
 
 /* OFPACT_SAMPLE.
  *
- * Used for NXAST_SAMPLE. */
+ * Used for NXAST_SAMPLE and NXAST_SAMPLE2. */
 struct ofpact_sample {
     struct ofpact ofpact;
-    uint16_t probability;  // Always >0.
+    uint16_t probability;  /* Always positive. */
     uint32_t collector_set_id;
     uint32_t obs_domain_id;
     uint32_t obs_point_id;
+    ofp_port_t sampling_port;
 };
 
 /* OFPACT_DEC_TTL.
index 289b969..48c05f5 100644 (file)
@@ -311,11 +311,13 @@ format_odp_userspace_action(struct ds *ds, const struct nlattr *attr)
                 ds_put_format(ds, ",flow_sample(probability=%"PRIu16
                               ",collector_set_id=%"PRIu32
                               ",obs_domain_id=%"PRIu32
-                              ",obs_point_id=%"PRIu32")",
+                              ",obs_point_id=%"PRIu32
+                              ",output_port=%"PRIu32")",
                               cookie.flow_sample.probability,
                               cookie.flow_sample.collector_set_id,
                               cookie.flow_sample.obs_domain_id,
-                              cookie.flow_sample.obs_point_id);
+                              cookie.flow_sample.obs_point_id,
+                              cookie.flow_sample.output_odp_port);
             } else if (userdata_len >= sizeof cookie.ipfix
                        && cookie.type == USER_ACTION_COOKIE_IPFIX) {
                 ds_put_format(ds, ",ipfix(output_port=%"PRIu32")",
@@ -951,9 +953,11 @@ parse_odp_userspace_action(const char *s, struct ofpbuf *actions)
         } else if (ovs_scan(&s[n], ",flow_sample(probability=%"SCNi32","
                             "collector_set_id=%"SCNi32","
                             "obs_domain_id=%"SCNi32","
-                            "obs_point_id=%"SCNi32")%n",
+                            "obs_point_id=%"SCNi32","
+                            "output_port=%"SCNi32")%n",
                             &probability, &collector_set_id,
-                            &obs_domain_id, &obs_point_id, &n1)) {
+                            &obs_domain_id, &obs_point_id,
+                            &output, &n1)) {
             n += n1;
 
             cookie.type = USER_ACTION_COOKIE_FLOW_SAMPLE;
@@ -961,6 +965,7 @@ parse_odp_userspace_action(const char *s, struct ofpbuf *actions)
             cookie.flow_sample.collector_set_id = collector_set_id;
             cookie.flow_sample.obs_domain_id = obs_domain_id;
             cookie.flow_sample.obs_point_id = obs_point_id;
+            cookie.flow_sample.output_odp_port = u32_to_odp(output);
             user_data = &cookie;
             user_data_size = sizeof cookie.flow_sample;
         } else if (ovs_scan(&s[n], ",ipfix(output_port=%"SCNi32")%n",
index 2c3cc8c..53ee661 100644 (file)
@@ -297,6 +297,7 @@ union user_action_cookie {
         uint32_t collector_set_id; /* ID of IPFIX collector set. */
         uint32_t obs_domain_id; /* Observation Domain ID. */
         uint32_t obs_point_id;  /* Observation Point ID. */
+        odp_port_t output_odp_port; /* The output odp port. */
     } flow_sample;
 
     struct {
@@ -304,7 +305,7 @@ union user_action_cookie {
         odp_port_t output_odp_port; /* The output odp port. */
     } ipfix;
 };
-BUILD_ASSERT_DECL(sizeof(union user_action_cookie) == 16);
+BUILD_ASSERT_DECL(sizeof(union user_action_cookie) == 20);
 
 size_t odp_put_userspace_action(uint32_t pid,
                                 const void *userdata, size_t userdata_size,
index 7ddadb8..ea55896 100644 (file)
@@ -289,6 +289,8 @@ enum ofp_raw_action_type {
 
     /* NX1.0+(29): struct nx_action_sample. */
     NXAST_RAW_SAMPLE,
+    /* NX1.0+(38): struct nx_action_sample2. */
+    NXAST_RAW_SAMPLE2,
 
     /* NX1.0+(34): struct nx_action_conjunction. */
     NXAST_RAW_CONJUNCTION,
@@ -4697,6 +4699,24 @@ struct nx_action_sample {
 };
 OFP_ASSERT(sizeof(struct nx_action_sample) == 24);
 
+/* Action structure for NXAST_SAMPLE2.
+ *
+ * This replacement for NXAST_SAMPLE makes it support exporting
+ * egress tunnel information. */
+struct nx_action_sample2 {
+    ovs_be16 type;                  /* OFPAT_VENDOR. */
+    ovs_be16 len;                   /* Length is 32. */
+    ovs_be32 vendor;                /* NX_VENDOR_ID. */
+    ovs_be16 subtype;               /* NXAST_SAMPLE. */
+    ovs_be16 probability;           /* Fraction of packets to sample. */
+    ovs_be32 collector_set_id;      /* ID of collector set in OVSDB. */
+    ovs_be32 obs_domain_id;         /* ID of sampling observation domain. */
+    ovs_be32 obs_point_id;          /* ID of sampling observation point. */
+    ovs_be16 sampling_port;         /* Sampling port. */
+    uint8_t  pad[6];                /* Pad to a multiple of 8 bytes */
+ };
+ OFP_ASSERT(sizeof(struct nx_action_sample2) == 32);
+
 static enum ofperr
 decode_NXAST_RAW_SAMPLE(const struct nx_action_sample *nas,
                         enum ofp_version ofp_version OVS_UNUSED,
@@ -4705,10 +4725,35 @@ decode_NXAST_RAW_SAMPLE(const struct nx_action_sample *nas,
     struct ofpact_sample *sample;
 
     sample = ofpact_put_SAMPLE(out);
+    sample->ofpact.raw = NXAST_RAW_SAMPLE;
     sample->probability = ntohs(nas->probability);
     sample->collector_set_id = ntohl(nas->collector_set_id);
     sample->obs_domain_id = ntohl(nas->obs_domain_id);
     sample->obs_point_id = ntohl(nas->obs_point_id);
+    /* Default value for sampling port is OFPP_NONE */
+    sample->sampling_port = OFPP_NONE;
+
+    if (sample->probability == 0) {
+        return OFPERR_OFPBAC_BAD_ARGUMENT;
+    }
+
+    return 0;
+}
+
+static enum ofperr
+decode_NXAST_RAW_SAMPLE2(const struct nx_action_sample2 *nas,
+                         enum ofp_version ofp_version OVS_UNUSED,
+                         struct ofpbuf *out)
+{
+    struct ofpact_sample *sample;
+
+    sample = ofpact_put_SAMPLE(out);
+    sample->ofpact.raw = NXAST_RAW_SAMPLE2;
+    sample->probability = ntohs(nas->probability);
+    sample->collector_set_id = ntohl(nas->collector_set_id);
+    sample->obs_domain_id = ntohl(nas->obs_domain_id);
+    sample->obs_point_id = ntohl(nas->obs_point_id);
+    sample->sampling_port = u16_to_ofp(ntohs(nas->sampling_port));
 
     if (sample->probability == 0) {
         return OFPERR_OFPBAC_BAD_ARGUMENT;
@@ -4721,13 +4766,21 @@ static void
 encode_SAMPLE(const struct ofpact_sample *sample,
               enum ofp_version ofp_version OVS_UNUSED, struct ofpbuf *out)
 {
-    struct nx_action_sample *nas;
-
-    nas = put_NXAST_SAMPLE(out);
-    nas->probability = htons(sample->probability);
-    nas->collector_set_id = htonl(sample->collector_set_id);
-    nas->obs_domain_id = htonl(sample->obs_domain_id);
-    nas->obs_point_id = htonl(sample->obs_point_id);
+    if (sample->ofpact.raw == NXAST_RAW_SAMPLE2
+        || sample->sampling_port != OFPP_NONE) {
+        struct nx_action_sample2 *nas = put_NXAST_SAMPLE2(out);
+        nas->probability = htons(sample->probability);
+        nas->collector_set_id = htonl(sample->collector_set_id);
+        nas->obs_domain_id = htonl(sample->obs_domain_id);
+        nas->obs_point_id = htonl(sample->obs_point_id);
+        nas->sampling_port = htons(ofp_to_u16(sample->sampling_port));
+    } else {
+        struct nx_action_sample *nas = put_NXAST_SAMPLE(out);
+        nas->probability = htons(sample->probability);
+        nas->collector_set_id = htonl(sample->collector_set_id);
+        nas->obs_domain_id = htonl(sample->obs_domain_id);
+        nas->obs_point_id = htonl(sample->obs_point_id);
+    }
 }
 
 /* Parses 'arg' as the argument to a "sample" action, and appends such an
@@ -4740,8 +4793,9 @@ parse_SAMPLE(char *arg, struct ofpbuf *ofpacts,
              enum ofputil_protocol *usable_protocols OVS_UNUSED)
 {
     struct ofpact_sample *os = ofpact_put_SAMPLE(ofpacts);
-    char *key, *value;
+    os->sampling_port = OFPP_NONE;
 
+    char *key, *value;
     while (ofputil_parse_key_value(&arg, &key, &value)) {
         char *error = NULL;
 
@@ -4756,6 +4810,10 @@ parse_SAMPLE(char *arg, struct ofpbuf *ofpacts,
             error = str_to_u32(value, &os->obs_domain_id);
         } else if (!strcmp(key, "obs_point_id")) {
             error = str_to_u32(value, &os->obs_point_id);
+        } else if (!strcmp(key, "sampling_port")) {
+            if (!ofputil_port_from_string(value, &os->sampling_port)) {
+                error = xasprintf("%s: unknown port", value);
+            }
         } else {
             error = xasprintf("invalid key \"%s\" in \"sample\" argument",
                               key);
@@ -4767,6 +4825,7 @@ parse_SAMPLE(char *arg, struct ofpbuf *ofpacts,
     if (os->probability == 0) {
         return xstrdup("non-zero \"probability\" must be specified on sample");
     }
+
     return NULL;
 }
 
@@ -4776,13 +4835,17 @@ format_SAMPLE(const struct ofpact_sample *a, struct ds *s)
     ds_put_format(s, "%ssample(%s%sprobability=%s%"PRIu16
                   ",%scollector_set_id=%s%"PRIu32
                   ",%sobs_domain_id=%s%"PRIu32
-                  ",%sobs_point_id=%s%"PRIu32"%s)%s",
+                  ",%sobs_point_id=%s%"PRIu32,
                   colors.paren, colors.end,
                   colors.param, colors.end, a->probability,
                   colors.param, colors.end, a->collector_set_id,
                   colors.param, colors.end, a->obs_domain_id,
-                  colors.param, colors.end, a->obs_point_id,
-                  colors.paren, colors.end);
+                  colors.param, colors.end, a->obs_point_id);
+    if (a->sampling_port != OFPP_NONE) {
+        ds_put_format(s, ",%ssampling_port=%s%"PRIu16,
+                      colors.param, colors.end, a->sampling_port);
+    }
+    ds_put_format(s, "%s)%s", colors.paren, colors.end);
 }
 \f
 /* debug_recirc instruction. */
index b692026..f166014 100644 (file)
@@ -1,5 +1,5 @@
 /*
- * Copyright (c) 2012, 2013, 2014, 2015 Nicira, Inc.
+ * Copyright (c) 2012, 2013, 2014, 2015, 2016 Nicira, Inc.
  *
  * Licensed under the Apache License, Version 2.0 (the "License");
  * you may not use this file except in compliance with the License.
@@ -479,6 +479,7 @@ ofproto_ipfix_flow_exporter_options_equal(
     return (a->collector_set_id == b->collector_set_id
             && a->cache_active_timeout == b->cache_active_timeout
             && a->cache_max_flows == b->cache_max_flows
+            && a->enable_tunnel_sampling == b->enable_tunnel_sampling
             && sset_equals(&a->targets, &b->targets));
 }
 
@@ -950,6 +951,22 @@ dpif_ipfix_get_bridge_exporter_tunnel_sampling(const struct dpif_ipfix *di)
     return ret;
 }
 
+bool
+dpif_ipfix_get_flow_exporter_tunnel_sampling(const struct dpif_ipfix *di,
+                                             const uint32_t collector_set_id)
+    OVS_EXCLUDED(mutex)
+{
+    ovs_mutex_lock(&mutex);
+    struct dpif_ipfix_flow_exporter_map_node *node
+        = dpif_ipfix_find_flow_exporter_map_node(di, collector_set_id);
+    bool ret = (node
+                && node->exporter.options
+                && node->exporter.options->enable_tunnel_sampling);
+    ovs_mutex_unlock(&mutex);
+
+    return ret;
+}
+
 static void
 dpif_ipfix_clear(struct dpif_ipfix *di) OVS_REQUIRES(mutex)
 {
@@ -1895,11 +1912,19 @@ dpif_ipfix_bridge_sample(struct dpif_ipfix *di, const struct dp_packet *packet,
 
 void
 dpif_ipfix_flow_sample(struct dpif_ipfix *di, const struct dp_packet *packet,
-                       const struct flow *flow, uint32_t collector_set_id,
-                       uint16_t probability, uint32_t obs_domain_id,
-                       uint32_t obs_point_id) OVS_EXCLUDED(mutex)
+                       const struct flow *flow,
+                       const union user_action_cookie *cookie,
+                       odp_port_t input_odp_port,
+                       const struct flow_tnl *output_tunnel_key)
+    OVS_EXCLUDED(mutex)
 {
     struct dpif_ipfix_flow_exporter_map_node *node;
+    const struct flow_tnl *tunnel_key = NULL;
+    struct dpif_ipfix_port * tunnel_port = NULL;
+    odp_port_t output_odp_port = cookie->flow_sample.output_odp_port;
+    uint32_t collector_set_id = cookie->flow_sample.collector_set_id;
+    uint16_t probability = cookie->flow_sample.probability;
+
     /* Use the sampling probability as an approximation of the number
      * of matched packets. */
     uint64_t packet_delta_count = USHRT_MAX / probability;
@@ -1907,9 +1932,24 @@ dpif_ipfix_flow_sample(struct dpif_ipfix *di, const struct dp_packet *packet,
     ovs_mutex_lock(&mutex);
     node = dpif_ipfix_find_flow_exporter_map_node(di, collector_set_id);
     if (node) {
+        if (node->exporter.options->enable_tunnel_sampling) {
+            if (output_odp_port == ODPP_NONE && flow->tunnel.ip_dst) {
+                /* Input tunnel. */
+                tunnel_key = &flow->tunnel;
+                tunnel_port = dpif_ipfix_find_port(di, input_odp_port);
+            }
+            if (output_odp_port != ODPP_NONE && output_tunnel_key) {
+                /* Output tunnel, output_tunnel_key must be valid. */
+                tunnel_key = output_tunnel_key;
+                tunnel_port = dpif_ipfix_find_port(di, output_odp_port);
+            }
+        }
+
         dpif_ipfix_sample(&node->exporter.exporter, packet, flow,
-                          packet_delta_count, obs_domain_id, obs_point_id,
-                          ODPP_NONE, NULL, NULL);
+                          packet_delta_count,
+                          cookie->flow_sample.obs_domain_id,
+                          cookie->flow_sample.obs_point_id,
+                          output_odp_port, tunnel_port, tunnel_key);
     }
     ovs_mutex_unlock(&mutex);
 }
index c8e4593..0808ede 100644 (file)
@@ -40,6 +40,8 @@ uint32_t dpif_ipfix_get_bridge_exporter_probability(const struct dpif_ipfix *);
 bool dpif_ipfix_get_bridge_exporter_tunnel_sampling(const struct dpif_ipfix *);
 bool dpif_ipfix_get_bridge_exporter_input_sampling(const struct dpif_ipfix *);
 bool dpif_ipfix_get_bridge_exporter_output_sampling(const struct dpif_ipfix *);
+bool dpif_ipfix_get_flow_exporter_tunnel_sampling(const struct dpif_ipfix *,
+                                                  const uint32_t);
 bool dpif_ipfix_get_tunnel_port(const struct dpif_ipfix *, odp_port_t);
 void dpif_ipfix_set_options(
     struct dpif_ipfix *,
@@ -52,8 +54,8 @@ void dpif_ipfix_bridge_sample(struct dpif_ipfix *, const struct dp_packet *,
                               const struct flow *,
                               odp_port_t, odp_port_t, const struct flow_tnl *);
 void dpif_ipfix_flow_sample(struct dpif_ipfix *, const struct dp_packet *,
-                            const struct flow *, uint32_t, uint16_t, uint32_t,
-                            uint32_t);
+                            const struct flow *, const union user_action_cookie *,
+                            odp_port_t, const struct flow_tnl *);
 
 void dpif_ipfix_run(struct dpif_ipfix *);
 void dpif_ipfix_wait(struct dpif_ipfix *);
index b947da9..c83df9e 100644 (file)
@@ -1265,17 +1265,22 @@ process_upcall(struct udpif *udpif, struct upcall *upcall,
     case FLOW_SAMPLE_UPCALL:
         if (upcall->ipfix) {
             union user_action_cookie cookie;
+            struct flow_tnl output_tunnel_key;
 
             memset(&cookie, 0, sizeof cookie);
             memcpy(&cookie, nl_attr_get(userdata), sizeof cookie.flow_sample);
 
+            if (upcall->out_tun_key) {
+                odp_tun_key_from_attr(upcall->out_tun_key, false,
+                                      &output_tunnel_key);
+            }
+
             /* The flow reflects exactly the contents of the packet.
              * Sample the packet using it. */
             dpif_ipfix_flow_sample(upcall->ipfix, packet, flow,
-                                   cookie.flow_sample.collector_set_id,
-                                   cookie.flow_sample.probability,
-                                   cookie.flow_sample.obs_domain_id,
-                                   cookie.flow_sample.obs_point_id);
+                                   &cookie, flow->in_port.odp_port,
+                                   upcall->out_tun_key ?
+                                       &output_tunnel_key : NULL);
         }
         break;
 
index cca5c5c..6013831 100644 (file)
@@ -2601,7 +2601,10 @@ xlate_normal(struct xlate_ctx *ctx)
  * 'cookie' (of length 'cookie_size' bytes) is passed back in the callback for
  * each sampled packet.  'tunnel_out_port', if not ODPP_NONE, is added as the
  * OVS_USERSPACE_ATTR_EGRESS_TUN_PORT attribute.  If 'include_actions', an
- * OVS_USERSPACE_ATTR_ACTIONS attribute is added.
+ * OVS_USERSPACE_ATTR_ACTIONS attribute is added.  If 'emit_set_tunnel',
+ * sample(sampling_port=1) would translate into datapath sample action
+ * set(tunnel(...)), sample(...) and it is used for sampling egress tunnel
+ * information.
  */
 static size_t
 compose_sample_action(struct xlate_ctx *ctx,
@@ -2655,8 +2658,10 @@ compose_sflow_action(struct xlate_ctx *ctx)
                                  true);
 }
 
-/* If IPFIX is enabled, this appends a "sample" action to implement IPFIX to
- * 'ctx->odp_actions'. */
+/* If flow IPFIX is enabled, make sure IPFIX flow sample action
+ * at egress point of tunnel port is just in front of corresponding
+ * output action. If bridge IPFIX is enabled, this appends an IPFIX
+ * sample action to 'ctx->odp_actions'. */
 static void
 compose_ipfix_action(struct xlate_ctx *ctx, odp_port_t output_odp_port)
 {
@@ -2674,7 +2679,7 @@ compose_ipfix_action(struct xlate_ctx *ctx, odp_port_t output_odp_port)
         return;
     }
 
-    /* For output case, output_odp_port is valid*/
+    /* For output case, output_odp_port is valid*/
     if (output_odp_port != ODPP_NONE) {
         if (!dpif_ipfix_get_bridge_exporter_output_sampling(ipfix)) {
             return;
@@ -4145,6 +4150,15 @@ static void
 xlate_sample_action(struct xlate_ctx *ctx,
                     const struct ofpact_sample *os)
 {
+    odp_port_t output_odp_port = ODPP_NONE;
+    odp_port_t tunnel_out_port = ODPP_NONE;
+    struct dpif_ipfix *ipfix = ctx->xbridge->ipfix;
+    bool emit_set_tunnel = false;
+
+    if (!ipfix || ctx->xin->flow.in_port.ofp_port == OFPP_NONE) {
+        return;
+    }
+
     /* Scale the probability from 16-bit to 32-bit while representing
      * the same percentage. */
     uint32_t probability = (os->probability << 16) | os->probability;
@@ -4158,7 +4172,51 @@ xlate_sample_action(struct xlate_ctx *ctx,
         return;
     }
 
-    xlate_commit_actions(ctx);
+    /* If ofp_port in flow sample action is equel to ofp_port,
+     * this sample action is a input port action. */
+    if (os->sampling_port != OFPP_NONE &&
+        os->sampling_port != ctx->xin->flow.in_port.ofp_port) {
+        output_odp_port = ofp_port_to_odp_port(ctx->xbridge,
+                                               os->sampling_port);
+        if (output_odp_port == ODPP_NONE) {
+            static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(1, 1);
+            VLOG_WARN_RL(&rl, "can't use unknown port %d in flow sample "
+                         "action", os->sampling_port);
+            return;
+        }
+
+        if (dpif_ipfix_get_flow_exporter_tunnel_sampling(ipfix,
+                                                         os->collector_set_id)
+            && dpif_ipfix_get_tunnel_port(ipfix, output_odp_port)) {
+            tunnel_out_port = output_odp_port;
+            emit_set_tunnel = true;
+        }
+    }
+
+     xlate_commit_actions(ctx);
+    /* If 'emit_set_tunnel', sample(sampling_port=1) would translate
+     * into datapath sample action set(tunnel(...)), sample(...) and
+     * it is used for sampling egress tunnel information. */
+    if (emit_set_tunnel) {
+        const struct xport *xport = get_ofp_port(ctx->xbridge,
+                                                 os->sampling_port);
+
+        if (xport && xport->is_tunnel) {
+            struct flow *flow = &ctx->xin->flow;
+            tnl_port_send(xport->ofport, flow, ctx->wc);
+            if (!ovs_native_tunneling_is_on(ctx->xbridge->ofproto)) {
+                struct flow_tnl flow_tnl = flow->tunnel;
+
+                commit_odp_tunnel_action(flow, &ctx->base_flow,
+                                         ctx->odp_actions);
+                flow->tunnel = flow_tnl;
+            }
+        } else {
+            static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(1, 1);
+            VLOG_WARN_RL(&rl, "sampling_port:%d should be a tunnel port.",
+                         os->sampling_port);
+        }
+    }
 
     union user_action_cookie cookie = {
         .flow_sample = {
@@ -4167,10 +4225,11 @@ xlate_sample_action(struct xlate_ctx *ctx,
             .collector_set_id = os->collector_set_id,
             .obs_domain_id = os->obs_domain_id,
             .obs_point_id = os->obs_point_id,
+            .output_odp_port = output_odp_port,
         }
     };
     compose_sample_action(ctx, probability, &cookie, sizeof cookie.flow_sample,
-                          ODPP_NONE, false);
+                          tunnel_out_port, false);
 }
 
 static bool
index 8588872..774b5b6 100644 (file)
@@ -89,6 +89,7 @@ struct ofproto_ipfix_flow_exporter_options {
     struct sset targets;
     uint32_t cache_active_timeout;
     uint32_t cache_max_flows;
+    bool enable_tunnel_sampling;
 };
 
 struct ofproto_rstp_status {
index 808a83b..7b94c92 100644 (file)
@@ -255,8 +255,8 @@ userspace(pid=9765,slow_path(cfm))
 userspace(pid=9765,slow_path(cfm),tunnel_out_port=10)
 userspace(pid=1234567,userdata(0102030405060708090a0b0c0d0e0f),actions)
 userspace(pid=1234567,userdata(0102030405060708090a0b0c0d0e0f),tunnel_out_port=10)
-userspace(pid=6633,flow_sample(probability=123,collector_set_id=1234,obs_domain_id=2345,obs_point_id=3456))
-userspace(pid=6633,flow_sample(probability=123,collector_set_id=1234,obs_domain_id=2345,obs_point_id=3456),tunnel_out_port=10)
+userspace(pid=6633,flow_sample(probability=123,collector_set_id=1234,obs_domain_id=2345,obs_point_id=3456,output_port=10))
+userspace(pid=6633,flow_sample(probability=123,collector_set_id=1234,obs_domain_id=2345,obs_point_id=3456,output_port=10),tunnel_out_port=10)
 userspace(pid=6633,ipfix(output_port=10))
 userspace(pid=6633,ipfix(output_port=10),tunnel_out_port=10)
 set(in_port(2))
index 83a2301..50f74e9 100644 (file)
@@ -127,6 +127,9 @@ ffff 0020 00002320 0015 000500000000 80003039005A02fd 0400000000000000
 # actions=sample(probability=12345,collector_set_id=23456,obs_domain_id=34567,obs_point_id=45678)
 ffff 0018 00002320 001d 3039 00005BA0 00008707 0000B26E
 
+# actions=sample(probability=12345,collector_set_id=23456,obs_domain_id=34567,obs_point_id=45678,sampling_port=56789)
+ffff 0020 00002320 0026 3039 00005BA0 00008707 0000B26E DDD50000 00000000
+
 # bad OpenFlow10 actions: OFPBAC_BAD_LEN
 & ofp_actions|WARN|OpenFlow action OFPAT_OUTPUT length 240 exceeds action buffer length 8
 & ofp_actions|WARN|bad action at offset 0 (OFPBAC_BAD_LEN):
index 406bde1..9b94f3d 100644 (file)
@@ -5892,7 +5892,7 @@ for i in `seq 1 3`; do
 done
 AT_CHECK([ovs-appctl dpctl/dump-flows | sed 's/.*\(packets:\)/\1/' | sed 's/used:[[0-9]].[[0-9]]*s/used:0.001s/'], [0], [dnl
 flow-dump from non-dpdk interfaces:
-packets:2, bytes:120, used:0.001s, actions:sample(sample=100.0%,actions(userspace(pid=0,flow_sample(probability=65535,collector_set_id=1,obs_domain_id=0,obs_point_id=0)))),2
+packets:2, bytes:120, used:0.001s, actions:sample(sample=100.0%,actions(userspace(pid=0,flow_sample(probability=65535,collector_set_id=1,obs_domain_id=0,obs_point_id=0,output_port=4294967295)))),2
 ])
 
 dnl Remove the flow which contains sample action.
@@ -5911,6 +5911,80 @@ packets:2, bytes:120, used:0.001s, actions:drop
 OVS_VSWITCHD_STOP(["/sending to collector failed/d"])
 AT_CLEANUP
 
+dnl Flow IPFIX sanity check for tunnel set
+AT_SETUP([ofproto-dpif - Flow IPFIX sanity check - tunnel set])
+AT_XFAIL_IF([test "$IS_WIN32" = "yes"])
+OVS_VSWITCHD_START([add-port br0 p1 -- set Interface p1 type=gre \
+                    options:remote_ip=1.1.1.1 options:local_ip=2.2.2.2 \
+                    options:key=5 ofport_request=1\
+                    -- add-port br0 p2 -- set Interface p2 type=stt \
+                    options:remote_ip=1.1.1.2 options:local_ip=2.2.2.3 \
+                    options:key=6 ofport_request=2\
+                    -- add-port br0 p3 -- set Interface p3 type=dummy \
+                    ofport_request=3 \
+                    -- --id=@br0 get Bridge br0 \
+                    -- --id=@ipfix create IPFIX targets=\"127.0.0.1:4739\" \
+                    -- --id=@cs create Flow_Sample_Collector_Set id=1 bridge=@br0 ipfix=@ipfix],
+                   [<0>
+<1>
+])
+
+OVS_VSWITCHD_DISABLE_TUNNEL_PUSH_POP
+
+dnl Add openflow sample action without sampling_port.
+AT_DATA([flows.txt], [dnl
+in_port=3, actions=sample(probability=65535,collector_set_id=1),output:1
+])
+AT_CHECK([ovs-ofctl add-flows br0 flows.txt], [0], [ignore])
+
+AT_CHECK([ovs-appctl ofproto/trace ovs-dummy 'in_port(3),eth(src=50:54:00:00:00:09,dst=50:54:00:00:00:0a),eth_type(0x0800),ipv4(src=10.10.10.2,dst=10.10.10.1,proto=1,tos=1,ttl=128,frag=no),icmp(type=8,code=0)'], [0], [stdout])
+AT_CHECK([tail -1 stdout], [0], [dnl
+Datapath actions: sample(sample=100.0%,actions(userspace(pid=0,flow_sample(probability=65535,collector_set_id=1,obs_domain_id=0,obs_point_id=0,output_port=4294967295)))),set(tunnel(tun_id=0x5,src=2.2.2.2,dst=1.1.1.1,tos=0x1,ttl=64,flags(df|key))),1
+])
+
+dnl Remove the flow which contains sample action.
+AT_CHECK([ovs-ofctl del-flows br0 in_port=3], [0], [ignore])
+AT_CHECK([ovs-appctl revalidator/purge])
+
+dnl Add openflow sample action with sampling_port which is
+dnl equal to output port.
+AT_DATA([flows2.txt], [dnl
+in_port=3, actions=sample(probability=65535,collector_set_id=1,sampling_port=1),output:1
+])
+AT_CHECK([ovs-ofctl add-flows br0 flows2.txt], [0], [ignore])
+
+AT_CHECK([ovs-appctl ofproto/trace ovs-dummy 'in_port(3),eth(src=50:54:00:00:00:09,dst=50:54:00:00:00:0a),eth_type(0x0800),ipv4(src=10.10.10.2,dst=10.10.10.1,proto=1,tos=1,ttl=128,frag=no),icmp(type=8,code=0)'], [0], [stdout])
+dnl Make sure flow sample action in datapath is behind set tunnel
+dnl action at egress point of tunnel port.
+AT_CHECK([tail -1 stdout], [0], [dnl
+Datapath actions: set(tunnel(tun_id=0x5,src=2.2.2.2,dst=1.1.1.1,tos=0x1,ttl=64,flags(df|key))),sample(sample=100.0%,actions(userspace(pid=0,flow_sample(probability=65535,collector_set_id=1,obs_domain_id=0,obs_point_id=0,output_port=1),tunnel_out_port=1))),1
+])
+
+dnl Remove the flow which contains sample action.
+AT_CHECK([ovs-ofctl del-flows br0 in_port=3], [0], [ignore])
+AT_CHECK([ovs-appctl revalidator/purge])
+
+dnl Add a rule with two sample actions and each sample action
+dnl has a sampling_port
+AT_DATA([flows3.txt], [dnl
+in_port=3, actions=sample(probability=65535,collector_set_id=1,sampling_port=1),output:1,sample(probability=65535,collector_set_id=1,sampling_port=2),output:2
+])
+AT_CHECK([ovs-ofctl add-flows br0 flows3.txt], [0], [ignore])
+
+AT_CHECK([ovs-appctl ofproto/trace ovs-dummy 'in_port(3),eth(src=50:54:00:00:00:09,dst=50:54:00:00:00:0a),eth_type(0x0800),ipv4(src=10.10.10.2,dst=10.10.10.1,proto=1,tos=1,ttl=128,frag=no),icmp(type=8,code=0)'], [0], [stdout])
+dnl Make sure flow sample action in datapath is behind set tunnel
+dnl action at egress point of tunnel port.
+AT_CHECK([tail -1 stdout], [0], [dnl
+Datapath actions: set(tunnel(tun_id=0x5,src=2.2.2.2,dst=1.1.1.1,tos=0x1,ttl=64,flags(df|key))),sample(sample=100.0%,actions(userspace(pid=0,flow_sample(probability=65535,collector_set_id=1,obs_domain_id=0,obs_point_id=0,output_port=1),tunnel_out_port=1))),1,set(tunnel(tun_id=0x6,src=2.2.2.3,dst=1.1.1.2,tos=0x1,ttl=64,flags(df|key))),sample(sample=100.0%,actions(userspace(pid=0,flow_sample(probability=65535,collector_set_id=1,obs_domain_id=0,obs_point_id=0,output_port=7471),tunnel_out_port=7471))),7471
+])
+
+dnl Remove the flow which contains sample action.
+AT_CHECK([ovs-ofctl del-flows br0 in_port=3], [0], [ignore])
+AT_CHECK([ovs-vsctl destroy Flow_Sample_Collector_Set 1], [0], [ignore])
+
+OVS_VSWITCHD_STOP
+AT_CLEANUP
+
 dnl Flow based IPFIX statistics check
 AT_SETUP([ofproto-dpif - Flow IPFIX statistics check])
 OVS_VSWITCHD_START
index 8287cd2..613d9ce 100644 (file)
@@ -160,6 +160,7 @@ sctp actions=drop
 sctp actions=drop
 in_port=0 actions=resubmit:0
 actions=sample(probability=12345,collector_set_id=23456,obs_domain_id=34567,obs_point_id=45678)
+actions=sample(probability=12345,collector_set_id=23456,obs_domain_id=34567,obs_point_id=45678,sampling_port=56789)
 actions=ct(nat)
 actions=ct(commit,nat(dst))
 actions=ct(commit,nat(src))
@@ -190,6 +191,7 @@ OFPT_FLOW_MOD: ADD sctp actions=drop
 OFPT_FLOW_MOD: ADD sctp actions=drop
 OFPT_FLOW_MOD: ADD in_port=0 actions=resubmit:0
 OFPT_FLOW_MOD: ADD actions=sample(probability=12345,collector_set_id=23456,obs_domain_id=34567,obs_point_id=45678)
+OFPT_FLOW_MOD: ADD actions=sample(probability=12345,collector_set_id=23456,obs_domain_id=34567,obs_point_id=45678,sampling_port=56789)
 OFPT_FLOW_MOD: ADD actions=ct(nat)
 OFPT_FLOW_MOD: ADD actions=ct(commit,nat(dst))
 OFPT_FLOW_MOD: ADD actions=ct(commit,nat(src))
@@ -219,6 +221,7 @@ sctp actions=drop
 sctp actions=drop
 in_port=0 actions=resubmit:0
 actions=sample(probability=12345,collector_set_id=23456,obs_domain_id=34567,obs_point_id=45678)
+actions=sample(probability=12345,collector_set_id=23456,obs_domain_id=34567,obs_point_id=45678,sampling_port=56789)
 ]])
 
 AT_CHECK([ovs-ofctl --protocols OpenFlow11 parse-flows flows.txt
@@ -238,6 +241,7 @@ OFPT_FLOW_MOD (OF1.1): ADD sctp actions=drop
 OFPT_FLOW_MOD (OF1.1): ADD sctp actions=drop
 OFPT_FLOW_MOD (OF1.1): ADD in_port=0 actions=resubmit:0
 OFPT_FLOW_MOD (OF1.1): ADD actions=sample(probability=12345,collector_set_id=23456,obs_domain_id=34567,obs_point_id=45678)
+OFPT_FLOW_MOD (OF1.1): ADD actions=sample(probability=12345,collector_set_id=23456,obs_domain_id=34567,obs_point_id=45678,sampling_port=56789)
 ]])
 AT_CLEANUP
 
@@ -260,6 +264,7 @@ ip actions=mod_nw_src:10.1.1.2,mod_nw_dst:192.168.10.1,mod_nw_ttl:1,mod_nw_tos:1
 in_port=0 actions=mod_dl_src:11:22:33:44:55:66,mod_dl_dst:10:20:30:40:50:60
 in_port=0 actions=resubmit:0
 actions=sample(probability=12345,collector_set_id=23456,obs_domain_id=34567,obs_point_id=45678)
+actions=sample(probability=12345,collector_set_id=23456,obs_domain_id=34567,obs_point_id=45678,sampling_port=56789)
 ]])
 
 AT_CHECK([ovs-ofctl --protocols OpenFlow12 parse-flows flows.txt
@@ -283,6 +288,7 @@ OFPT_FLOW_MOD (OF1.2): ADD ip actions=set_field:10.1.1.2->ip_src,set_field:192.1
 OFPT_FLOW_MOD (OF1.2): ADD in_port=0 actions=set_field:11:22:33:44:55:66->eth_src,set_field:10:20:30:40:50:60->eth_dst
 OFPT_FLOW_MOD (OF1.2): ADD in_port=0 actions=resubmit:0
 OFPT_FLOW_MOD (OF1.2): ADD actions=sample(probability=12345,collector_set_id=23456,obs_domain_id=34567,obs_point_id=45678)
+OFPT_FLOW_MOD (OF1.2): ADD actions=sample(probability=12345,collector_set_id=23456,obs_domain_id=34567,obs_point_id=45678,sampling_port=56789)
 ]])
 AT_CLEANUP
 
@@ -375,6 +381,7 @@ check_overlap,actions=output:1,exit,output:2
 tcp,actions=fin_timeout(idle_timeout=5,hard_timeout=15)
 actions=controller(max_len=123,reason=invalid_ttl,id=555)
 actions=sample(probability=12345,collector_set_id=23456,obs_domain_id=34567,obs_point_id=45678)
+actions=sample(probability=12345,collector_set_id=23456,obs_domain_id=34567,obs_point_id=45678,sampling_port=56789)
 ip,actions=ct(commit,zone=5)
 ip,actions=ct(commit,exec(load(1->NXM_NX_CT_MARK[])))
 ip,actions=ct(commit,exec(load(0x1->NXM_NX_CT_LABEL[])))
@@ -417,6 +424,7 @@ NXT_FLOW_MOD: ADD table:255 check_overlap actions=output:1,exit,output:2
 NXT_FLOW_MOD: ADD table:255 tcp actions=fin_timeout(idle_timeout=5,hard_timeout=15)
 NXT_FLOW_MOD: ADD table:255 actions=controller(reason=invalid_ttl,max_len=123,id=555)
 NXT_FLOW_MOD: ADD table:255 actions=sample(probability=12345,collector_set_id=23456,obs_domain_id=34567,obs_point_id=45678)
+NXT_FLOW_MOD: ADD table:255 actions=sample(probability=12345,collector_set_id=23456,obs_domain_id=34567,obs_point_id=45678,sampling_port=56789)
 NXT_FLOW_MOD: ADD table:255 ip actions=ct(commit,zone=5)
 NXT_FLOW_MOD: ADD table:255 ip actions=ct(commit,exec(load:0x1->NXM_NX_CT_MARK[]))
 NXT_FLOW_MOD: ADD table:255 ip actions=ct(commit,exec(load:0x1->NXM_NX_CT_LABEL[0..63],load:0->NXM_NX_CT_LABEL[64..127]))
@@ -457,6 +465,7 @@ dl_dst=00:00:00:00:00:00/01:00:00:00:00:00,actions=drop
 dl_dst=aa:bb:cc:dd:ee:ff/fe:ff:ff:ff:ff:ff,actions=drop
 dl_dst=aa:bb:cc:dd:ee:ff/00:00:00:00:00:00,actions=drop
 actions=sample(probability=12345,collector_set_id=23456,obs_domain_id=34567,obs_point_id=45678)
+actions=sample(probability=12345,collector_set_id=23456,obs_domain_id=34567,obs_point_id=45678,sampling_port=56789)
 ip,actions=ct(commit,zone=5)
 ip,actions=ct(commit,exec(load(1->NXM_NX_CT_MARK[[]])))
 ip,actions=ct(commit,exec(load(0x1->NXM_NX_CT_LABEL[[]])))
@@ -493,6 +502,7 @@ NXT_FLOW_MOD: ADD dl_dst=00:00:00:00:00:00/01:00:00:00:00:00 actions=drop
 NXT_FLOW_MOD: ADD dl_dst=aa:bb:cc:dd:ee:ff/fe:ff:ff:ff:ff:ff actions=drop
 NXT_FLOW_MOD: ADD actions=drop
 NXT_FLOW_MOD: ADD actions=sample(probability=12345,collector_set_id=23456,obs_domain_id=34567,obs_point_id=45678)
+NXT_FLOW_MOD: ADD actions=sample(probability=12345,collector_set_id=23456,obs_domain_id=34567,obs_point_id=45678,sampling_port=56789)
 NXT_FLOW_MOD: ADD ip actions=ct(commit,zone=5)
 NXT_FLOW_MOD: ADD ip actions=ct(commit,exec(load:0x1->NXM_NX_CT_MARK[[]]))
 NXT_FLOW_MOD: ADD ip actions=ct(commit,exec(load:0x1->NXM_NX_CT_LABEL[[0..63]],load:0->NXM_NX_CT_LABEL[[64..127]]))
@@ -529,6 +539,7 @@ actions=move:OXM_OF_ETH_DST[]->OXM_OF_ETH_SRC[]
 actions=push:NXM_NX_REG0[0..31],pop:NXM_NX_REG0[]
 vlan_tci=0x1123/0x1fff,actions=drop
 actions=sample(probability=12345,collector_set_id=23456,obs_domain_id=34567,obs_point_id=45678)
+actions=sample(probability=12345,collector_set_id=23456,obs_domain_id=34567,obs_point_id=45678,sampling_port=56789)
 ip,actions=ct(commit,zone=5)
 ip,actions=ct(commit,exec(load(1->NXM_NX_CT_MARK[])))
 ip,actions=ct(commit,exec(load(1->NXM_NX_CT_LABEL[])))
@@ -565,6 +576,7 @@ NXT_FLOW_MOD: ADD <any> actions=move:NXM_OF_ETH_DST[]->NXM_OF_ETH_SRC[]
 NXT_FLOW_MOD: ADD <any> actions=push:NXM_NX_REG0[],pop:NXM_NX_REG0[]
 NXT_FLOW_MOD: ADD NXM_OF_VLAN_TCI_W(1123/1fff) actions=drop
 NXT_FLOW_MOD: ADD <any> actions=sample(probability=12345,collector_set_id=23456,obs_domain_id=34567,obs_point_id=45678)
+NXT_FLOW_MOD: ADD <any> actions=sample(probability=12345,collector_set_id=23456,obs_domain_id=34567,obs_point_id=45678,sampling_port=56789)
 NXT_FLOW_MOD: ADD NXM_OF_ETH_TYPE(0800) actions=ct(commit,zone=5)
 NXT_FLOW_MOD: ADD NXM_OF_ETH_TYPE(0800) actions=ct(commit,exec(load:0x1->NXM_NX_CT_MARK[]))
 NXT_FLOW_MOD: ADD NXM_OF_ETH_TYPE(0800) actions=ct(commit,exec(load:0x1->NXM_NX_CT_LABEL[0..63],load:0->NXM_NX_CT_LABEL[64..127]))
index 604107c..fad2709 100644 (file)
@@ -2320,6 +2320,12 @@ Observation Domain ID sent in every IPFIX flow record.  Defaults to 0.
 .IP "\fBobs_point_id=\fIid\fR"
 When sending samples to IPFIX collectors, the unsigned 32-bit integer
 Observation Point ID sent in every IPFIX flow record.  Defaults to 0.
+.IP "\fBsampling_port=\fIport\fR"
+Sample packets on the port.  It can be set as input port or output
+port.  When this option is omitted, or specified as \fBNONE\fB, IPFIX
+does not differentiate between ingress packets and egress packets and
+does not export egress tunnel information.  This option was added in
+Open vSwitch 2.5.90.
 .RE
 .IP
 Refer to \fBovs\-vswitchd.conf.db\fR(5) for more details on
index 41ec4ba..4273552 100644 (file)
@@ -1225,6 +1225,9 @@ bridge_configure_ipfix(struct bridge *br)
                     ? *fe_cfg->ipfix->cache_active_timeout : 0;
                 opts->cache_max_flows = fe_cfg->ipfix->cache_max_flows
                     ? *fe_cfg->ipfix->cache_max_flows : 0;
+                opts->enable_tunnel_sampling = smap_get_bool(
+                                                   &fe_cfg->ipfix->other_config,
+                                                  "enable-tunnel-sampling", true);
                 opts++;
             }
         }
index 5cf1ee1..6e0ee80 100644 (file)
       disabled.
     </column>
 
+    <column name="other_config" key="enable-tunnel-sampling"
+            type='{"type": "boolean"}'>
+      <p>
+        Set to <code>true</code> to enable sampling and reporting tunnel
+        header 7-tuples in IPFIX flow records.  Tunnel sampling is enabled
+        by default.
+      </p>
+
+      <p>
+        The following enterprise entities report the sampled tunnel info:
+      </p>
+
+      <dl>
+        <dt>tunnelType:</dt>
+        <dd>
+          <p>ID: 891, and enterprise ID 6876 (VMware).</p>
+          <p>type: unsigned 8-bit integer.</p>
+          <p>data type semantics: identifier.</p>
+          <p>description: Identifier of the layer 2 network overlay network
+          encapsulation type: 0x01 VxLAN, 0x02 GRE, 0x03 LISP, 0x05 IPsec+GRE,
+          0x07 GENEVE.</p>
+        </dd>
+        <dt>tunnelKey:</dt>
+        <dd>
+          <p>ID: 892, and enterprise ID 6876 (VMware).</p>
+          <p>type: variable-length octetarray.</p>
+          <p>data type semantics: identifier.</p>
+          <p>description: Key which is used for identifying an individual
+          traffic flow within a VxLAN (24-bit VNI), GENEVE (24-bit VNI),
+          GRE (32-bit key), or LISP (24-bit instance ID) tunnel. The
+          key is encoded in this octetarray as a 3-, 4-, or 8-byte integer
+          ID in network byte order.</p>
+        </dd>
+        <dt>tunnelSourceIPv4Address:</dt>
+        <dd>
+          <p>ID: 893, and enterprise ID 6876 (VMware).</p>
+          <p>type: unsigned 32-bit integer.</p>
+          <p>data type semantics: identifier.</p>
+          <p>description: The IPv4 source address in the tunnel IP packet
+          header.</p>
+        </dd>
+        <dt>tunnelDestinationIPv4Address:</dt>
+        <dd>
+          <p>ID: 894, and enterprise ID 6876 (VMware).</p>
+          <p>type: unsigned 32-bit integer.</p>
+          <p>data type semantics: identifier.</p>
+          <p>description: The IPv4 destination address in the tunnel IP
+          packet header.</p>
+        </dd>
+        <dt>tunnelProtocolIdentifier:</dt>
+        <dd>
+          <p>ID: 895, and enterprise ID 6876 (VMware).</p>
+          <p>type: unsigned 8-bit integer.</p>
+          <p>data type semantics: identifier.</p>
+          <p>description: The value of the protocol number in the tunnel
+          IP packet header. The protocol number identifies the tunnel IP
+          packet payload type.</p>
+        </dd>
+        <dt>tunnelSourceTransportPort:</dt>
+        <dd>
+          <p>ID: 896, and enterprise ID 6876 (VMware).</p>
+          <p>type: unsigned 16-bit integer.</p>
+          <p>data type semantics: identifier.</p>
+          <p>description: The source port identifier in the tunnel transport
+          header. For the transport protocols UDP, TCP, and SCTP, this is
+          the source port number given in the respective header.</p>
+        </dd>
+        <dt>tunnelDestinationTransportPort:</dt>
+        <dd>
+          <p>ID: 897, and enterprise ID 6876 (VMware).</p>
+          <p>type: unsigned 16-bit integer.</p>
+          <p>data type semantics: identifier.</p>
+          <p>description: The destination port identifier in the tunnel
+          transport header. For the transport protocols UDP, TCP, and SCTP,
+          this is the destination port number given in the respective header.
+          </p>
+        </dd>
+      </dl>
+
+      <p>
+        Before Open vSwitch 2.5.90, <ref column="other_config"
+        key="enable-tunnel-sampling"/> was only supported with per-bridge
+        sampling, and ignored otherwise.  Open vSwitch 2.5.90 and later support
+        <ref column="other_config" key="enable-tunnel-sampling"/> for
+        per-bridge and per-flow sampling.
+      </p>
+    </column>
+
     <group title="Per-Bridge Sampling">
       <p>
         These values affect only per-bridge sampling.  See above for a
         specified, defaults to 0.
       </column>
 
-      <column name="other_config" key="enable-tunnel-sampling"
-              type='{"type": "boolean"}'>
-        <p>
-          Set to <code>true</code> to enable sampling and reporting tunnel
-          header 7-tuples in IPFIX flow records.  Tunnel sampling is disabled
-          by default.
-        </p>
-
-        <p>
-          The following enterprise entities report the sampled tunnel info:
-        </p>
-
-        <dl>
-          <dt>tunnelType:</dt>
-          <dd>
-            <p>ID: 891, and enterprise ID 6876 (VMware).</p>
-            <p>type: unsigned 8-bit integer.</p>
-            <p>data type semantics: identifier.</p>
-            <p>description: Identifier of the layer 2 network overlay network
-            encapsulation type: 0x01 VxLAN, 0x02 GRE, 0x03 LISP, 0x05 IPsec+GRE,
-            0x07 GENEVE.</p>
-          </dd>
-          <dt>tunnelKey:</dt>
-          <dd>
-            <p>ID: 892, and enterprise ID 6876 (VMware).</p>
-            <p>type: variable-length octetarray.</p>
-            <p>data type semantics: identifier.</p>
-            <p>description: Key which is used for identifying an individual
-            traffic flow within a VxLAN (24-bit VNI), GENEVE (24-bit VNI),
-            GRE (32-bit key), or LISP (24-bit instance ID) tunnel. The
-            key is encoded in this octetarray as a 3-, 4-, or 8-byte integer
-            ID in network byte order.</p>
-          </dd>
-          <dt>tunnelSourceIPv4Address:</dt>
-          <dd>
-            <p>ID: 893, and enterprise ID 6876 (VMware).</p>
-            <p>type: unsigned 32-bit integer.</p>
-            <p>data type semantics: identifier.</p>
-            <p>description: The IPv4 source address in the tunnel IP packet
-            header.</p>
-          </dd>
-          <dt>tunnelDestinationIPv4Address:</dt>
-          <dd>
-            <p>ID: 894, and enterprise ID 6876 (VMware).</p>
-            <p>type: unsigned 32-bit integer.</p>
-            <p>data type semantics: identifier.</p>
-            <p>description: The IPv4 destination address in the tunnel IP
-            packet header.</p>
-          </dd>
-          <dt>tunnelProtocolIdentifier:</dt>
-          <dd>
-            <p>ID: 895, and enterprise ID 6876 (VMware).</p>
-            <p>type: unsigned 8-bit integer.</p>
-            <p>data type semantics: identifier.</p>
-            <p>description: The value of the protocol number in the tunnel
-            IP packet header. The protocol number identifies the tunnel IP
-            packet payload type.</p>
-          </dd>
-          <dt>tunnelSourceTransportPort:</dt>
-          <dd>
-            <p>ID: 896, and enterprise ID 6876 (VMware).</p>
-            <p>type: unsigned 16-bit integer.</p>
-            <p>data type semantics: identifier.</p>
-            <p>description: The source port identifier in the tunnel transport
-            header. For the transport protocols UDP, TCP, and SCTP, this is
-            the source port number given in the respective header.</p>
-          </dd>
-          <dt>tunnelDestinationTransportPort:</dt>
-          <dd>
-            <p>ID: 897, and enterprise ID 6876 (VMware).</p>
-            <p>type: unsigned 16-bit integer.</p>
-            <p>data type semantics: identifier.</p>
-            <p>description: The destination port identifier in the tunnel
-            transport header. For the transport protocols UDP, TCP, and SCTP,
-            this is the destination port number given in the respective header.
-            </p>
-          </dd>
-        </dl>
-      </column>
-
       <column name="other_config" key="enable-input-sampling"
               type='{"type": "boolean"}'>
         By default, Open vSwitch samples and reports flows at bridge port input