Documentation: DPDK IVSHMEM VM Communications

Adds documentation on how to run IVSHMEM communication through VM.

Signed-off-by: Mike A. Polehn <mike.a.polehn@intel.com>
Acked-by: Pravin B Shelar <pshelar@nicira.com>

diff --git a/INSTALL.DPDK b/INSTALL.DPDK
index ff729e3..bba7d29 100644 (file)
--- a/INSTALL.DPDK
+++ b/INSTALL.DPDK
@@ -20,9 +20,13 @@ DPDK:
 Set dir i.g.:   export DPDK_DIR=/usr/src/dpdk-1.7.0
 cd $DPDK_DIR
 update config/common_linuxapp so that dpdk generate single lib file.
+(modification also required for IVSHMEM build)
 CONFIG_RTE_BUILD_COMBINE_LIBS=y
 
+For default install without IVSHMEM:
 make install T=x86_64-native-linuxapp-gcc
+To include IVSHMEM (shared memory):
+make install T=x86_64-ivshmem-linuxapp-gcc
 For details refer to  http://dpdk.org/
 
 Linux kernel:
@@ -30,9 +34,13 @@ Refer to intel-dpdk-getting-started-guide.pdf for understanding
 DPDK kernel requirement.
 
 OVS:
+Non IVSHMEM:
+export DPDK_BUILD=$DPDK_DIR/x86_64-native-linuxapp-gcc/
+IVSHMEM:
+export DPDK_BUILD=$DPDK_DIR/x86_64-ivshmem-linuxapp-gcc/
+
 cd $(OVS_DIR)/openvswitch
 ./boot.sh
-export DPDK_BUILD=$DPDK_DIR/x86_64-native-linuxapp-gcc/
 ./configure --with-dpdk=$DPDK_BUILD
 make
 
@@ -73,7 +81,7 @@ First setup DPDK devices:
 
 Prepare system:
   - mount hugetlbfs
-    e.g. mount -t hugetlbfs -o pagesize=1G none /mnt/huge/
+    e.g. mount -t hugetlbfs -o pagesize=1G none /dev/hugepages
 
 Ref to http://www.dpdk.org/doc/quick-start for verifying DPDK setup.
 
@@ -91,7 +99,7 @@ Start ovsdb-server as discussed in INSTALL doc:
       ./ovsdb/ovsdb-server --remote=punix:/usr/local/var/run/openvswitch/db.sock \
           --remote=db:Open_vSwitch,Open_vSwitch,manager_options \
           --private-key=db:Open_vSwitch,SSL,private_key \
-          --certificate=dbitch,SSL,certificate \
+          --certificate=Open_vSwitch,SSL,certificate \
           --bootstrap-ca-cert=db:Open_vSwitch,SSL,ca_cert --pidfile --detach
     First time after db creation, initialize:
       cd $OVS_DIR
@@ -105,12 +113,13 @@ for dpdk initialization.
 
    e.g.
    export DB_SOCK=/usr/local/var/run/openvswitch/db.sock
-   ./vswitchd/ovs-vswitchd --dpdk -c 0x1 -n 4 -- unix:$DB_SOCK  --pidfile --detach
+   ./vswitchd/ovs-vswitchd --dpdk -c 0x1 -n 4 -- unix:$DB_SOCK --pidfile --detach
 
-If allocated more than 1 GB huge pages, set amount and use NUMA node 0 memory:
+If allocated more than one GB hugepage (as for IVSHMEM), set amount and use NUMA
+node 0 memory:
 
    ./vswitchd/ovs-vswitchd --dpdk -c 0x1 -n 4 --socket-mem 1024,0 \
-      -- unix:$DB_SOCK  --pidfile --detach
+      -- unix:$DB_SOCK --pidfile --detach
 
 To use ovs-vswitchd with DPDK, create a bridge with datapath_type
 "netdev" in the configuration database.  For example:
@@ -136,9 +145,7 @@ Test flow script across NICs (assuming ovs in /usr/src/ovs):
 ############################# Script:
 
 #! /bin/sh
-
 # Move to command directory
-
 cd /usr/src/ovs/utilities/
 
 # Clear current flows
@@ -158,7 +165,8 @@ help.
 
 At this time all ovs-vswitchd tasks end up being affinitized to cpu core 0
 but this may change. Lets pick a target core for 100% task to run on, i.e. core 7.
-Also assume a dual 8 core sandy bridge system with hyperthreading enabled.
+Also assume a dual 8 core sandy bridge system with hyperthreading enabled
+where CPU1 has cores 0,...,7 and 16,...,23 & CPU2 cores 8,...,15 & 24,...,31.
 (A different cpu configuration will have different core mask requirements).
 
 To give better ownership of 100%, isolation maybe useful.
@@ -178,11 +186,11 @@ taskset -p 080 1762
   pid 1762's new affinity mask: 80
 
 Assume that all other ovs-vswitchd threads to be on other socket 0 cores.
-Affinitize the rest of the ovs-vswitchd thread ids to 0x0FF007F
+Affinitize the rest of the ovs-vswitchd thread ids to 0x07F007F
 
-taskset -p 0x0FF007F {thread pid, e.g 1738}
+taskset -p 0x07F007F {thread pid, e.g 1738}
   pid 1738's current affinity mask: 1
-  pid 1738's new affinity mask: ff007f
+  pid 1738's new affinity mask: 7f007f
 . . .
 
 The core 23 is left idle, which allows core 7 to run at full rate.
@@ -207,8 +215,8 @@ with the ring naming used within ovs.
 location tests/ovs_client
 
 To run the client :
-
-    ovsclient -c 1 -n 4 --proc-type=secondary -- -n "port id you gave dpdkr"
+  cd /usr/src/ovs/tests/
+  ovsclient -c 1 -n 4 --proc-type=secondary -- -n "port id you gave dpdkr"
 
 In the case of the dpdkr example above the "port id you gave dpdkr" is 0.
 
@@ -218,6 +226,9 @@ The application simply receives an mbuf on the receive queue of the
 ethernet ring and then places that same mbuf on the transmit ring of
 the ethernet ring.  It is a trivial loopback application.
 
+DPDK rings in VM (IVSHMEM shared memory communications)
+-------------------------------------------------------
+
 In addition to executing the client in the host, you can execute it within
 a guest VM. To do so you will need a patched qemu.  You can download the
 patch and getting started guide at :