行至水穷处 坐看“云”起时

Cloud Native ADN -> CNadn.Net

How to build Nginx Plus as k8s Ingress controller and run with F5 CIS together

  1. GET nginx-repo.crt and nginx-repo.key, put them into the root of kubernetets-ingress/
  2. Docker login your docker hub with: docker login . or using private repo.
  3. follow https://github.com/nginxinc/kubernetes-ingress/blob/master/build/README.md, and build the docker image. In my lab, it is:
    make PREFIX=myf5/nginx-plus-ingress-opentracing DOCKERFILE=DockerfileWithOpentracingForPlus GENERATE_DEFAULT_CERT_AND_KEY=1

  4. Follow https://github.com/nginxinc/kubernetes-ingress/blob/master/docs/installation.md to Create namespace, serviceacount, and defualt ssl cert/key for ingress
    [root@k8s-master-v1-16 common]# pwd
    /root/selab/kubernetes-ingress/deployments/common

  5. Create a default blank configmap for nginx configurations. This means there is no customization for nginx now.

  1. Create CRD resources for nginx extend usage. Note, to use NGINX CRD, you must have option “-enable-custom-resources” to be enabled when you deploy nginx controller. We will highlight this in afterward step
    NOTE: NGINX CRD only support k8s v1.11+ version.

  1. Create RBAC for ingress controller

  1. Now, we start deploy nginx controller pods. You can deploy as daemonset, this means there is nginx controller pod on each work node and each node can be ingress traffic entry.
    If you select deployment, you can decide to deploy nginx controller numbers and into which nodes, for example, you can deploy into specific node that only work as traffic entry. T
    These dedicate nodes will be as ingress EDGE.

  2. I will deploy Ingress controller on node 1, so Label node 1 as ingress edge first.
    [root@k8s-master-v1-16 deployment]# kubectl label node k8s-node1-v1-16.lab.f5se.io Ingressedege-
    node/k8s-node1-v1-16.lab.f5se.io labeled

  1. Edit deployment yaml. I enable custom-resource option and enable prometheus related setting, also I set node affinity to make sure the nginx controller only being deployed on node1.

Now you can access the node1 hostIP:8888, you can see the NGINX Plus’s dashboard page.

  1. Deploy the classic cafe application, follow https://github.com/nginxinc/kubernetes-ingress/blob/master/examples/complete-example/README.md
    Access the application http://cafe.example.com/coffee, you will get similar output

  1. check Ingress status:

At this time, I have F5 CIS in the cluster. I did not add any F5-CIS-Ingress annotations to the cafe-ingress resource, the CIS creates pool/ltm policy without vs, this is just Unattached pools.

  1. Add F5 CIS Ingress annotations to the Cafe-ingress, we try use a shared-vs-ip for all Ingress resources.
    To use shared-vs-ip for ingress on BIGIP, first need to change f5-bigip-ctlr (cc)’s deployment setting, add below:

Then, add annotations to the cafe-ingress resource. In the health check part, must make sure the path same to the Ingress rule’s path, otherwise the f5-bigip-ctlr will not create the monitor. In my test, the f5-bigip-ctlr can not create ssl profile base on the tls.secret setting, dont know why.

Since we made a shared-vs setting, if we create a new ingress resource, the new ingress rule will be a new ltm policy on the same vs.
Create a new ingress:

Check ingress resource:

  1. Next, pls follow https://github.com/nginxinc/kubernetes-ingress/tree/master/examples to understand all examples and advance CRD examples.
点赞

发表评论

电子邮件地址不会被公开。 必填项已用*标注

此站点使用Akismet来减少垃圾评论。了解我们如何处理您的评论数据