1 * Flow match expression handling library.
3 ovn-controller is the primary user of flow match expressions, but
4 the same syntax and I imagine the same code ought to be useful in
5 ovn-nbd for ACL match expressions.
7 ** Definition of data structures to represent a match expression as a
10 ** Definition of data structures to represent variables (fields).
12 Fields need names and prerequisites. Most fields are numeric and
13 thus need widths. We need also need a way to represent nominal
14 fields (currently just logical port names). It might be
15 appropriate to associate fields directly with OXM/NXM code points;
16 we have to decide whether we want OVN to use the OVS flow structure
17 or work with OXM more directly.
19 Probably should be defined so that the data structure is also
20 useful for references to fields in action parsing.
24 Probably should be defined so that the lexer can be reused for
27 ** Parsing into syntax tree.
29 ** Semantic checking against variable definitions.
31 ** Applying prerequisites.
33 ** Simplification into conjunction-of-disjunctions (CoD) form.
35 ** Transformation from CoD form into OXM matches.
39 ** Flow table handling in ovn-controller.
41 ovn-controller has to transform logical datapath flows from the
42 database into OpenFlow flows.
44 *** Definition (or choice) of data structure for flows and flow table.
46 It would be natural enough to use "struct flow" and "struct
47 classifier" for this. Maybe that is what we should do. However,
48 "struct classifier" is optimized for searches based on packet
49 headers, whereas all we care about here can be implemented with a
50 hash table. Also, we may want to make it easy to add and remove
51 support for fields without recompiling, which is not possible with
52 "struct flow" or "struct classifier".
54 On the other hand, we may find that it is difficult to decide that
55 two OXM flow matches are identical (to normalize them) without a
56 lot of domain-specific knowledge that is already embedded in struct
57 flow. It's also going to be a pain to come up with a way to make
58 anything other than "struct flow" work with the ofputil_*()
59 functions for encoding and decoding OpenFlow.
61 It's also possible we could use struct flow without struct
64 *** Assembling conjunctive flows from flow match expressions.
66 This transformation explodes logical datapath flows into multiple
67 OpenFlow flow table entries, since a flow match expression in CoD
68 form requires several OpenFlow flow table entries. It also
69 requires merging together OpenFlow flow tables entries that contain
70 "conjunction" actions (really just concatenating their actions).
72 *** Translating logical datapath port names into port numbers.
74 Logical ports are specified by name in logical datapath flows, but
75 OpenFlow only works in terms of numbers.
77 *** Translating logical datapath actions into OpenFlow actions.
79 Some of the logical datapath actions do not have natural
80 representations as OpenFlow actions: they require
81 packet-in/packet-out round trips through ovn-controller. The
82 trickiest part of that is going to be making sure that the
83 packet-out resumes the control flow that was broken off by the
84 packet-in. That's tricky; we'll probably have to restrict control
85 flow or add OVS features to make resuming in general possible. Not
86 sure which is better at this point.
88 *** OpenFlow flow table synchronization.
90 The internal representation of the OpenFlow flow table has to be
91 synced across the controller connection to OVS. This probably
92 boils down to the "flow monitoring" feature of OF1.4 which was then
93 made available as a "standard extension" to OF1.3. (OVS hasn't
94 implemented this for OF1.4 yet, but the feature is based on a OVS
95 extension to OF1.0, so it should be straightforward to add it.)
97 We probably need some way to catch cases where OVS and OVN don't
98 see eye-to-eye on what exactly constitutes a flow, so that OVN
99 doesn't waste a lot of CPU time hammering at OVS trying to install
100 something that it's not going to do.
102 *** Logical/physical translation.
104 When a packet comes into the integration bridge, the first stage of
105 processing needs to translate it from a physical to a logical
106 context. When a packet leaves the integration bridge, the final
107 stage of processing needs to translate it back into a physical
108 context. ovn-controller needs to populate the OpenFlow flows
109 tables to do these translations.
111 *** Determine how to split logical pipeline across physical nodes.
113 From the original OVN architecture document:
115 The pipeline processing is split between the ingress and egress
116 transport nodes. In particular, the logical egress processing may
117 occur at either hypervisor. Processing the logical egress on the
118 ingress hypervisor requires more state about the egress vif's
119 policies, but reduces traffic on the wire that would eventually be
120 dropped. Whereas, processing on the egress hypervisor can reduce
121 broadcast traffic on the wire by doing local replication. We
122 initially plan to process logical egress on the egress hypervisor
123 so that less state needs to be replicated. However, we may change
124 this behavior once we gain some experience writing the logical
127 The split pipeline processing split will influence how tunnel keys
130 ** Interaction with Open_vSwitch and OVN databases:
132 *** Monitor Chassis table in OVN.
134 Populate Port records for tunnels to other chassis into
135 Open_vSwitch database. As a scale optimization later on, one can
136 populate only records for tunnels to other chassis that have
137 logical networks in common with this one.
139 *** Monitor Pipeline table in OVN, trigger flow table recomputation on change.
141 ** ovn-controller parameters and configuration.
143 *** Tunnel encapsulation to publish.
145 Default: VXLAN? Geneve?
147 *** SSL configuration.
149 Can probably get this from Open_vSwitch database.
153 ** Monitor OVN_Northbound database, trigger Pipeline recomputation on change.
155 ** Translate each OVN_Northbound entity into Pipeline logical datapath flows.
157 We have to first sit down and figure out what the general
158 translation of each entity is. The original OVN architecture
160 http://openvswitch.org/pipermail/dev/2015-January/050380.html had
161 some sketches of these, but they need to be completed and
164 Initially, the simplest way to do this is probably to write
165 straight C code to do a full translation of the entire
166 OVN_Northbound database into the format for the Pipeline table in
167 the OVN Southbound database. As scale increases, this will probably
168 be too inefficient since a small change in OVN_Northbound requires a
169 full recomputation. At that point, we probably want to adopt a more
170 systematic approach, such as something akin to the "nlog" system used
171 in NVP (see Koponen et al. "Network Virtualization in Multi-tenant
172 Datacenters", NSDI 2014).
174 ** Push logical datapath flows to Pipeline table.
176 ** Monitor OVN Southbound database Bindings table.
178 Sync rows in the OVN Bindings table to the "up" column in the
179 OVN_Northbound database.
183 ovsdb-server should have adequate features for OVN but it probably
184 needs work for scale and possibly for availability as deployments
185 grow. Here are some thoughts.
187 Andy Zhou is looking at these issues.
189 ** Scaling number of connections.
191 In typical use today a given ovsdb-server has only a single-digit
192 number of simultaneous connections. The OVN Southbound database will
193 have a connection from every hypervisor. This use case needs testing
194 and probably coding work. Here are some possible improvements.
196 *** Reducing amount of data sent to clients.
198 Currently, whenever a row monitored by a client changes,
199 ovsdb-server sends the client every monitored column in the row,
200 even if only one column changes. It might be valuable to reduce
201 this only to the columns that changes.
203 Also, whenever a column changes, ovsdb-server sends the entire
204 contents of the column. It might be valuable, for columns that
205 are sets or maps, to send only added or removed values or
208 Currently, clients monitor the entire contents of a table. It
209 might make sense to allow clients to monitor only rows that
210 satisfy specific criteria, e.g. to allow an ovn-controller to
211 receive only Pipeline rows for logical networks on its hypervisor.
213 *** Reducing redundant data and code within ovsdb-server.
215 Currently, ovsdb-server separately composes database update
216 information to send to each of its clients. This is fine for a
217 small number of clients, but it wastes time and memory when
218 hundreds of clients all want the same updates (as will be in the
221 (This is somewhat opposed to the idea of letting a client monitor
222 only some rows in a table, since that would increase the diversity
227 If it turns out that other changes don't let ovsdb-server scale
228 adequately, we can multithread ovsdb-server. Initially one might
229 only break protocol handling into separate threads, leaving the
230 actual database work serialized through a lock.
232 ** Increasing availability.
234 Database availability might become an issue. The OVN system
235 shouldn't grind to a halt if the database becomes unavailable, but
236 it would become impossible to bring VIFs up or down, etc.
238 My current thought on how to increase availability is to add
239 clustering to ovsdb-server, probably via the Raft consensus
240 algorithm. As an experiment, I wrote an implementation of Raft
241 for Open vSwitch that you can clone from:
243 https://github.com/blp/ovs-reviews.git raft
245 ** Reducing startup time.
247 As-is, if ovsdb-server restarts, every client will fetch a fresh
248 copy of the part of the database that it cares about. With
249 hundreds of clients, this could cause heavy CPU load on
250 ovsdb-server and use excessive network bandwidth. It would be
251 better to allow incremental updates even across connection loss.
252 One way might be to use "Difference Digests" as described in
253 Epstein et al., "What's the Difference? Efficient Set
254 Reconciliation Without Prior Context". (I'm not yet aware of
255 previous non-academic use of this technique.)
259 ** Write ovn-nbctl utility.
261 The idea here is that we need a utility to act on the OVN_Northbound
262 database in a way similar to a CMS, so that we can do some testing
263 without an actual CMS in the picture.
267 ** Init scripts for ovn-controller (on HVs), ovn-nbd, OVN DB server.
269 ** Distribution packaging.
275 This is being developed on OpenStack's development infrastructure
276 to be along side most of the other Neutron plugins.
278 http://git.openstack.org/cgit/stackforge/networking-ovn
280 http://git.openstack.org/cgit/stackforge/networking-ovn/tree/doc/source/todo.rst