1 OVSDB replication implementation
2 --------------------------------
7 Given two Open vSwitch databases with the same schema, OVSDB
8 replication keeps these databases in the same state, i.e. each of the
9 databases have the same contents at any given time even if they are
10 not running in the same host. This document elaborates on the
11 implementation details to provide this functionality.
16 - Source of truth database: database whose content will be replicated
19 - Active server: ovsdb-server providing RPC interface to the source of
22 - Standby server: ovsdb-server providing RPC interface to the database
23 that is not the source of truth.
28 The overall design of replication consists of one ovsdb-server (active server)
29 communicating the state of its databases to another ovsdb-server
30 (standby server) so that the latter keep its own databases in that same state.
31 To achieve this, the standby server acts as a client of the active
32 server, in the sense that it sends a monitor request to keep up to date with
33 the changes in the active server databases. When a notification from the
34 active server arrives, the standby server executes the necessary set of
35 operations so its databases reach the same state as the the active server
36 databases. Below is the design represented as a diagram.
38 +--------------+ replication +--------------+
39 | Active |<-------------------| Standby |
40 | OVSDB-server | | OVSDB-server |
41 +--------------+ +--------------+
49 Setting up the replication
50 ==========================
52 To initiate the replication process, the standby server must be executed
53 indicating the location of the active server via the command line option
54 "--sync-from=server", where server can take any form described in the
55 ovsdb-client manpage and it must specify an active connection type (tcp, unix,
56 ssl). This option will cause the standby server to attempt to send a monitor
57 request to the active server in every main loop iteration, until the active
60 When sending a monitor request the standby server is doing the following:
62 1. Erase the content of the databases for which it is providing a RPC
65 2. Open the jsonrpc channel to communicate with the active server.
67 3. Fetch all the databases located in the active server.
69 4. For each database with the same schema in both the active and
70 standby servers: construct and send a monitor request message
71 specifying the tables that will be monitored (i.e all the tables on
72 the database except the ones blacklisted*).
74 5. Set the standby database to the current state of the active
77 Once the monitor request message is sent, the standby server will continuously
78 receive notifications of changes occurring to the tables specified in the
79 request. The process of handling this notifications is detailed in the next
82 *A set of tables that will be excluded from replication can be
83 configure as a blacklist of tables via the command line option
84 "--sync-exclude-tables=db:table[,db:table]...", where db corresponds
85 to the database where the table resides.
90 The replication process consists on handling the update notifications received
91 in the standby server caused by the monitor request that was previously sent to
92 the active server. In every loop iteration, the standby server attempts to
93 receive a message from the active server which can be an error, an echo
94 message (used to keep the connection alive) or an update notification. In case
95 the message is a fatal error, the standby server will disconnect from the
96 active without dropping the replicated data. If it is an echo message, the
97 standby server will reply with an echo message as well. If the message is an
98 update notification, the following process occurs:
100 1. Create a new transaction.
102 2. Get the \<table-updates\> object from the "params" member of the
105 3. For each \<table-update\> in the \<table-updates\> object do:
107 1. For each \<row-update\> in \<table-update\> check what kind of
108 operation should be executed according to the following criteria about
109 the presence of the object members:
111 - If "old" member is not present, execute an insert operation
112 using \<row\> from the "new" member.
114 - If "old" member is present and "new" member is not present,
115 execute a delete operation using \<row\> from the "old"
118 - If both "old" and "new" members are present, execute an
119 update operation using \<row\> from the "new" member.
121 4. Commit the transaction.
123 If an error occurs during the replication process, all replication is
124 restarted by resending a new monitor request as described in the section
125 "Setting up the replication".
128 Runtime management commands
129 ===========================
131 Runtime management commands can be sent to a running standby server via
132 ovs-appctl in order to configure the replication functionality. The available
133 commands are the following.
135 - ovsdb-server/set-remote-ovsdb-server {server}: sets the name of the active
138 - ovsdb-server/get-remote-ovsdb-server: gets the name of the active server
140 - ovsdb-server/connect-remote-ovsdb-server: causes the server to attempt to
141 send a monitor request every main loop iteration.
143 - ovsdb-server/disconnect-remote-ovsdb-server: closes the jsonrpc channel
144 between the active server and frees the memory used for the replication
147 - ovsdb-server/set-sync-excluded-tables {db:table,...}: sets the tables list
148 that will be excluded from being replicated.
150 - ovsdb-server/get-sync-excluded-tables: gets the tables list that is
151 currently excluded from replication.