1 Star 0 Fork 2

Ken / k8s-ovs

forked from joliny / k8s-ovs 
加入 Gitee
与超过 1200万 开发者一起发现、参与优秀开源项目,私有仓库也完全免费 :)
免费加入
克隆/下载
贡献代码
同步代码
取消
提示: 由于 Git 不支持空文件夾,创建文件夹后会生成空的 .keep 文件
Loading...
README
Apache-2.0

k8s-ovs

==============================

k8s-ovs是一个使用openvswitchK8S提供SDN功能的项目。该项目基于openshift SDN的原理进行开发。由于openshift的SDN网络方案和openshift自身的代码耦合在一起,无法像flannelcalico等网络方案以插件的方式独立的为K8S提供服务,所以我(隶属于万达网络科技集团云平台部)开发了k8s-ovs,它拥有openshift优秀的SDN功能,又可以独立为K8S提供服务。

该项目中有一部分基础代码库是从openshift的pkg/sdn/plugin直接拷贝或进行了一些修改的。如果有License方面的问题请随时联系我进行修正:tangle3@wanda.cn

如果对该项目有任何疑问,欢迎加入k8s-ovs-sdn的QQ交流群477023854进行讨论。

下面将对k8s-ovs的功能和安装进行详细介绍。如果你想了解不同功能的配置方法,可以跳转到admin.md进行阅读。

k8s-ovs的功能


k8s-ovs支持单租户模式和多租户模式。

  • 单租户模式直接使用openvswitch+vxlan将K8S的POD网络组成一个大二层,所有POD可以互通。
  • 多租户模式也使用openvswitch+vxlan来组建K8S的POD网络,但是它可以基于K8S中的NAMESPACE来分配虚拟网络从而形成一个网络独立的租户,一个NAMESPACE中的POD无法访问其他NAMESPACE中的PODS和SERVICES
  • 多租户模式下可以对一些NAMESPACE进行设置,使这些NAMESPACE中的POD可以和其他所有NAMESPACE中的PODS和SERVICES进行互访。
  • 多租户模式下可以合并某两个NAMESPACE的虚拟网络,让他们的PODS和SERVICES可以互访。
  • 多租户模式下也可以将上面合并的NAMESPACE虚拟网络进行分离。
  • 单租户和多租户模式下都支持POD的流量限制功能,这样可以保证同一台主机上的POD相对公平的分享网卡带宽,而不会出现一个POD因为流量过大占满了网卡导致其他POD无法正常工作的情况。
  • 单租户和多租户模式下都支持外联负载均衡。

安装


安装部署,需要准备至少3台服务器,其中一台作为K8S的master,另外两台作为node节点。我的测试环境为Centos7.2,docker(1.12.6)版本以及golang(1.7.1)版本。每台node节点都需要安装openvswitch-2.5.0或以上版本,并且每台node节点都需要将ovsdb-serverovs-vswitchd运行起来。

K8S集群安装

请参考K8S安装手册,推荐安装v1.6.0以后的版本,因为之前版本的kubelet在使用CNI的情况下存在IP地址泄漏问题

1,K8S集群安装过程中应该跳过网络部署这一步,网络部署将由下面的k8s-ovs部署完成。

2,安装过程中需要设置kubelet使用cni,也就是kubelet启动参数需要设置为--network-plugin=cni --cni-conf-dir=/etc/cni/net.d --cni-bin-dir=/opt/cni/bin,如果kubelet是使用容器的方式启动的需要将/etc/cni/net.d/opt/cni/bin/var/run/挂在到kubelet内部。

3,安装完成后K8S的node节点将会呈现出下面的状态。NotReady是因为还没有部署网络,kubelet在/etc/cni/net.d/目录下面没有发现cni配置文件导致,这会随着后面网络的部署会得到解决。

$ kubectl get node
NAME        STATUS     AGE       VERSION
sdn-test1   NotReady   10s       v1.6.4
sdn-test2   NotReady   4m        v1.6.4
sdn-test3   NotReady   6s        v1.6.4

安装k8s-ovs

下面我们将会分两种情况进行安装,用户可以选择其中适合自己的一种。 1,使用yaml直接一键部署k8s-ovs到k8s集群中,并使其作为daemonset运行起来。 2,详细介绍k8s-ovs的每一个组件的安装步骤,以便用户对k8s-ovs的各个组件依赖关系有一个深入了解。

开始下列安装操作的前提是你已经按照上面步骤安装好了K8S集群。并且在每一台node节点上将ovsdb-serverovs-vswitchd运行起来了。

快速安装

快速安装需要你部署K8S 1.6以上版本的集群,如果是1.5或者1.4的集群请下载yaml文件做相应修改。

$ kubectl apply -f https://github.com/tangle329/k8s-ovs/tree/master/rootfs/k8s-ovs.yaml

上面命令成功返回之后,你可以通过下列查询命令获取pod和node的运行状态来确认是否安装成功:

$ kubectl get pod --namespace=kube-system | grep k8s-ovs
k8s-ovs-etcd-h0fsc                                   1/1       Running   0          2h
k8s-ovs-node-c27jr                                   1/1       Running   0          2h
k8s-ovs-node-fxwwl                                   1/1       Running   0          2h
k8s-ovs-node-p09jd                                   1/1       Running   0          2h
$ kubectl get node
NAME        STATUS    AGE       VERSION
sdn-test1   Ready     11m       v1.6.4
sdn-test2   Ready     15m       v1.6.4
sdn-test3   Ready     11m       v1.6.4

至此,k8s-ovs部署完成,用户可以跳转到admin.md进行功能配置了。

详细安装

详细安装需要你部署K8S v1.4版本以上的集群。 下列命令需要到每台K8S的node节点运行,也可以在一台服务器上将对应文件编译好,然后使用批量部署工具将对应文件谁送到所有node节点上。 你也可以使用k8s-ovs-rpm项目中的RPM SPEC来制作本项目的RPM包,然后直接安装RPM包来完成下列命令的操作。

$ cd $GOPATH/src/
$ git clone https://github.com/tangle329/k8s-ovs.git
$ cd k8s-ovs
$ go build -o rootfs/opt/cni/bin/k8s-ovs k8s-ovs/cniclient
$ cp rootfs/opt/cni/bin/k8s-ovs /opt/cni/bin/
$ cp rootfs/opt/cni/bin/host-local /opt/cni/bin/
$ cp rootfs/opt/cni/bin/loopback /opt/cni/bin/
$ cp rootfs/etc/cni/net.d/80-k8s-ovs.conf /etc/cni/net.d/
$ go build -o rootfs/usr/sbin/k8s-ovs  k8s-ovs
$ cp rootfs/usr/sbin/k8s-ovs /usr/sbin/
$ cp rootfs/usr/sbin/k8s-sdn-ovs /usr/sbin/

其中第一个go build -o rootfs/opt/cni/bin/k8s-ovs k8s-ovs/cniclient生成的k8s-ovs是cni客户端,kubelet在创建和删除POD的时候会调用它来对POD的网络部分进行配置。第二个go build -o rootfs/usr/sbin/k8s-ovs k8s-ovs生成的k8s-ovs是我们的整个k8s-ovs的核心,前面提到的所有功能都由它来实现,它也是cni的服务端,接受并处理前面cni客户端的请求。注意请不要把/opt/cni/bin/目录设置到PATH环境变量中。

通常在kubelet使用了cni的情况下要执行了cp rootfs/etc/cni/net.d/80-k8s-ovs.conf /etc/cni/net.d/命令之后k8s的node节点才会进行ready状态,另外请确保在/etc/cni/net.d/中只有80-k8s-ovs.conf这个文件,执行完上面的命令后K8S的node节点状态为:

$ kubectl get node
NAME        STATUS    AGE       VERSION
sdn-test1   Ready     11m       v1.6.4
sdn-test2   Ready     15m       v1.6.4
sdn-test3   Ready     11m       v1.6.4

设置k8s-ovs的网络参数

设置网络参数之前,你需要搭建一个etcd服务,或者和K8S的apiserver共用一个etcd服务,所有K8S节点都需要能访问到该etcd服务。

搭建好etcd服务器之后,使用下列命令设置k8s-ovs的网络参数:

$ etcdctl set /k8s.ovs.com/ovs/network/config '{"Name":"k8ssdn", "Network":"172.11.0.0/16", "HostSubnetLength":10, "ServiceNetwork":"10.96.0.0/12", "PluginName":"k8s-ovs-multitenant"}'

其中,Network用于设置整个K8S集群POD网络的网段;HostSubnetLength用于设置每个node节点的子网长度;ServiceNetwork用于设置K8S中service的网段,这个需要和K8S apiserver的参数--service-cluster-ip-range指定的网络保持一致;PluginName用于设置租户模式,k8s-ovs-multitenant用于设置多租户模式,k8s-ovs-subnet用于设置单租户模式。

启动k8s-ovs

1,启动之前要在每个K8S node节点上设置访问K8S apiserver的环境变量,k8s-ovs就是通过该环境变量和apiserver进行通信的。 如果K8S使用的非加密方式则需要设置KUBERNETES_MASTER,你需要把下面两个变量apiserver_vipapiserver_port替换成你自己的apiserver服务的ip和port:

$ export KUBERNETES_MASTER="${apiserver_vip}:${apiserver_port}"

如果K8S使用加密方式则需要设置KUBECONFIG环境变量。我们使用的是加密方式所以设置的KUBECONFIG环境变量,其中每一台节点上面都需要有/etc/kubernetes/admin.conf这个文件,该文件是在部署加密方式服务的K8S集群时在K8S master上生成的,你需要将它依次拷贝到每一台node节点上:

$ export KUBECONFIG="/etc/kubernetes/admin.conf"

2,设置好环境变量后就可以运行k8s-ovs了。k8s-ovs有几个重要的选项--etcd-endpoints用于指定etcd服务的访问ip+port列表;如果是加密的etcd服务可以通过--etcd-cafile--etcd-certfile--etcd-keyfile来指定CA,证书,秘钥;--etcd-prefix用于指定k8s-ovs网络配置存放的目录,需要和前面网络配置小节中etcdctl set命令指定的目录一样;--hostname用于指定k8s-ovs所运行的node节点的名字,该名字需要和前面kubectl get node输出的名字一致,通常--hostname不需要指定,但有时候一些K8S集群的部署脚本会通过给kubelet传递--hostname-override选项来覆盖默认node节点名,这时就需要设置k8s-ovs的--hostname以便能够保持一致。 由于我们的环境没有覆盖node节点名,etcd也没有使用加密方式,所以运行命令如下:

$ /usr/sbin/k8s-ovs --etcd-endpoints=http://${etcd_ip}:2379 --etcd-prefix=/k8s.ovs.com/ovs/network --alsologtostderr --v=5

至此,k8s-ovs部署完成,用户可以跳转到admin.md进行功能配置了。

Apache License Version 2.0, January 2004 http://www.apache.org/licenses/ TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION 1. Definitions. "License" shall mean the terms and conditions for use, reproduction, and distribution as defined by Sections 1 through 9 of this document. "Licensor" shall mean the copyright owner or entity authorized by the copyright owner that is granting the License. "Legal Entity" shall mean the union of the acting entity and all other entities that control, are controlled by, or are under common control with that entity. For the purposes of this definition, "control" means (i) the power, direct or indirect, to cause the direction or management of such entity, whether by contract or otherwise, or (ii) ownership of fifty percent (50%) or more of the outstanding shares, or (iii) beneficial ownership of such entity. "You" (or "Your") shall mean an individual or Legal Entity exercising permissions granted by this License. "Source" form shall mean the preferred form for making modifications, including but not limited to software source code, documentation source, and configuration files. "Object" form shall mean any form resulting from mechanical transformation or translation of a Source form, including but not limited to compiled object code, generated documentation, and conversions to other media types. "Work" shall mean the work of authorship, whether in Source or Object form, made available under the License, as indicated by a copyright notice that is included in or attached to the work (an example is provided in the Appendix below). "Derivative Works" shall mean any work, whether in Source or Object form, that is based on (or derived from) the Work and for which the editorial revisions, annotations, elaborations, or other modifications represent, as a whole, an original work of authorship. For the purposes of this License, Derivative Works shall not include works that remain separable from, or merely link (or bind by name) to the interfaces of, the Work and Derivative Works thereof. "Contribution" shall mean any work of authorship, including the original version of the Work and any modifications or additions to that Work or Derivative Works thereof, that is intentionally submitted to Licensor for inclusion in the Work by the copyright owner or by an individual or Legal Entity authorized to submit on behalf of the copyright owner. For the purposes of this definition, "submitted" means any form of electronic, verbal, or written communication sent to the Licensor or its representatives, including but not limited to communication on electronic mailing lists, source code control systems, and issue tracking systems that are managed by, or on behalf of, the Licensor for the purpose of discussing and improving the Work, but excluding communication that is conspicuously marked or otherwise designated in writing by the copyright owner as "Not a Contribution." "Contributor" shall mean Licensor and any individual or Legal Entity on behalf of whom a Contribution has been received by Licensor and subsequently incorporated within the Work. 2. Grant of Copyright License. Subject to the terms and conditions of this License, each Contributor hereby grants to You a perpetual, worldwide, non-exclusive, no-charge, royalty-free, irrevocable copyright license to reproduce, prepare Derivative Works of, publicly display, publicly perform, sublicense, and distribute the Work and such Derivative Works in Source or Object form. 3. Grant of Patent License. Subject to the terms and conditions of this License, each Contributor hereby grants to You a perpetual, worldwide, non-exclusive, no-charge, royalty-free, irrevocable (except as stated in this section) patent license to make, have made, use, offer to sell, sell, import, and otherwise transfer the Work, where such license applies only to those patent claims licensable by such Contributor that are necessarily infringed by their Contribution(s) alone or by combination of their Contribution(s) with the Work to which such Contribution(s) was submitted. If You institute patent litigation against any entity (including a cross-claim or counterclaim in a lawsuit) alleging that the Work or a Contribution incorporated within the Work constitutes direct or contributory patent infringement, then any patent licenses granted to You under this License for that Work shall terminate as of the date such litigation is filed. 4. Redistribution. You may reproduce and distribute copies of the Work or Derivative Works thereof in any medium, with or without modifications, and in Source or Object form, provided that You meet the following conditions: (a) You must give any other recipients of the Work or Derivative Works a copy of this License; and (b) You must cause any modified files to carry prominent notices stating that You changed the files; and (c) You must retain, in the Source form of any Derivative Works that You distribute, all copyright, patent, trademark, and attribution notices from the Source form of the Work, excluding those notices that do not pertain to any part of the Derivative Works; and (d) If the Work includes a "NOTICE" text file as part of its distribution, then any Derivative Works that You distribute must include a readable copy of the attribution notices contained within such NOTICE file, excluding those notices that do not pertain to any part of the Derivative Works, in at least one of the following places: within a NOTICE text file distributed as part of the Derivative Works; within the Source form or documentation, if provided along with the Derivative Works; or, within a display generated by the Derivative Works, if and wherever such third-party notices normally appear. The contents of the NOTICE file are for informational purposes only and do not modify the License. You may add Your own attribution notices within Derivative Works that You distribute, alongside or as an addendum to the NOTICE text from the Work, provided that such additional attribution notices cannot be construed as modifying the License. You may add Your own copyright statement to Your modifications and may provide additional or different license terms and conditions for use, reproduction, or distribution of Your modifications, or for any such Derivative Works as a whole, provided Your use, reproduction, and distribution of the Work otherwise complies with the conditions stated in this License. 5. Submission of Contributions. Unless You explicitly state otherwise, any Contribution intentionally submitted for inclusion in the Work by You to the Licensor shall be under the terms and conditions of this License, without any additional terms or conditions. Notwithstanding the above, nothing herein shall supersede or modify the terms of any separate license agreement you may have executed with Licensor regarding such Contributions. 6. Trademarks. This License does not grant permission to use the trade names, trademarks, service marks, or product names of the Licensor, except as required for reasonable and customary use in describing the origin of the Work and reproducing the content of the NOTICE file. 7. Disclaimer of Warranty. Unless required by applicable law or agreed to in writing, Licensor provides the Work (and each Contributor provides its Contributions) on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied, including, without limitation, any warranties or conditions of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A PARTICULAR PURPOSE. You are solely responsible for determining the appropriateness of using or redistributing the Work and assume any risks associated with Your exercise of permissions under this License. 8. Limitation of Liability. In no event and under no legal theory, whether in tort (including negligence), contract, or otherwise, unless required by applicable law (such as deliberate and grossly negligent acts) or agreed to in writing, shall any Contributor be liable to You for damages, including any direct, indirect, special, incidental, or consequential damages of any character arising as a result of this License or out of the use or inability to use the Work (including but not limited to damages for loss of goodwill, work stoppage, computer failure or malfunction, or any and all other commercial damages or losses), even if such Contributor has been advised of the possibility of such damages. 9. Accepting Warranty or Additional Liability. While redistributing the Work or Derivative Works thereof, You may choose to offer, and charge a fee for, acceptance of support, warranty, indemnity, or other liability obligations and/or rights consistent with this License. However, in accepting such obligations, You may act only on Your own behalf and on Your sole responsibility, not on behalf of any other Contributor, and only if You agree to indemnify, defend, and hold each Contributor harmless for any liability incurred by, or claims asserted against, such Contributor by reason of your accepting any such warranty or additional liability. END OF TERMS AND CONDITIONS APPENDIX: How to apply the Apache License to your work. To apply the Apache License to your work, attach the following boilerplate notice, with the fields enclosed by brackets "{}" replaced with your own identifying information. (Don't include the brackets!) The text should be enclosed in the appropriate comment syntax for the file format. We also recommend that a file or class name and description of purpose be included on the same "printed page" as the copyright notice for easier identification within third-party archives. Copyright {yyyy} {name of copyright owner} Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.

简介

k8s-ovs是一个使用openvswitch为K8S提供SDN功能的项目。该项目基于openshift SDN的原理进行开发。由于openshift的SDN网络方案和openshift自身的代码耦合在一起,无法像flannel和calico等网络方案以插件的方式独立的为K8S提供服务,所以我(隶属于万达网络科技集团云平台部)开发了k8s-ovs,它拥有openshift优秀的SDN功能,又可以独立为K8S提供服务。 k8s-ovs支持单租户模式和多租户模式。 单租户模式直接使用openvswitch+vxlan将K8S的POD网络组成一个大二层,所有POD可以互通。 多租户模式也使用openvswitch+vxlan来组建K8S的POD网络,但是它可以基于K8S中的NAMESPACE... 展开 收起
Apache-2.0
取消

发行版

暂无发行版

贡献者

全部

近期动态

加载更多
不能加载更多了
1
https://gitee.com/kenstime_admin/k8s-ovs.git
git@gitee.com:kenstime_admin/k8s-ovs.git
kenstime_admin
k8s-ovs
k8s-ovs
master

搜索帮助

14c37bed 8189591 565d56ea 8189591