TELEPRESENCE

Docker for Mac with Kubernetes (and some tips on using Ingress)

Richard Li

January 10, 2018 | 3 min read

Table of contents

Docker released a beta version of Docker that includes Kubernetes support. I was excited to try it out on my Mac. Here are my notes and observations from experimenting with Docker for Mac with Kubernetes.

Installation

The Docker folks usually do a great job with a simple user experience, and installation was no exception. I downloaded the edge installer for Docker, which uninstalled my stable version of Docker. In the preferences pane, I enabled Kubernetes, and shortly thereafter, I had a working Kubernetes cluster.

Installing Docker for Mac with Kubernetes

I was also able to use the preexisting

kubectl

on my laptop. The installer assumes your Kubernetes configuration is stored in

$HOME/.kube/config

. If you have set KUBECONFIG to a different file, you’ll want to point it back to

config

Docker for Mac and Ingress

I decided to try installing Edge Stack, our Kubernetes-Native API Gateway built on the Envoy Proxy. Ambassador strives to be as idiomatic to Kubernetes as possible (e.g., it’s configured via annotations), so it’s a good real-world test for a Kubernetes implementation.

Docker for Mac is based on Kubernetes 1.8.2, so I installed Ambassador with RBAC:

---
apiVersion: v1
kind: Service
metadata:
  labels:
    service: ambassador-admin
  name: ambassador-admin
spec:
  type: NodePort
  ports:
  - name: ambassador-admin
    port: 8877
    targetPort: 8877
  selector:
    service: ambassador
---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: ClusterRole
metadata:
  name: ambassador
rules:
- apiGroups: [""]
  resources:
  - services
  verbs: ["get", "list", "watch"]
- apiGroups: [""]
  resources:
  - configmaps
  verbs: ["create", "update", "patch", "get", "list", "watch"]
- apiGroups: [""]
  resources:
  - secrets
  verbs: ["get", "list", "watch"]
---
apiVersion: v1
kind: ServiceAccount
metadata:
  name: ambassador
---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: ClusterRoleBinding
metadata:
  name: ambassador
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: ambassador
subjects:
- kind: ServiceAccount
  name: ambassador
  namespace: default
---
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  name: ambassador
spec:
  replicas: 1
  template:
    metadata:
      labels:
        service: ambassador
    spec:
      serviceAccountName: ambassador
      containers:
      - name: ambassador
        image: datawire/ambassador:0.21.0
        imagePullPolicy: Always
        resources:
          limits:
            cpu: 1
            memory: 400Mi
          requests:
            cpu: 200m
            memory: 100Mi
        env:
        - name: AMBASSADOR_NAMESPACE
          valueFrom:
            fieldRef:
              fieldPath: metadata.namespace
        livenessProbe:
          httpGet:
            path: /ambassador/v0/check_alive
            port: 8877
          initialDelaySeconds: 3
          periodSeconds: 3
        readinessProbe:
          httpGet:
            path: /ambassador/v0/check_ready
            port: 8877
          initialDelaySeconds: 3
          periodSeconds: 3
      - name: statsd-sink
        image: datawire/prom-statsd-exporter:0.6.0
      restartPolicy: Always

I then deployed an Ambassador

LoadBalancer

service:

---
apiVersion: v1
kind: Service
metadata:
  labels:
    service: ambassador
  name: ambassador
spec:
  type: LoadBalancer
  ports:
  - name: ambassador
    port: 80
    targetPort: 80
  selector:
    service: ambassador

I wanted to try to connect to Ambassador, but this is what I saw:

$ kubectl get svc NAME CLUSTER-IP EXTERNAL-IP PORT(S) AGE ambassador 10.106.236.196 <pending> 80:30612/TCP 45m ambassador-admin 10.102.220.182 <nodes> 8877:31079/TCP 4h kubernetes 10.96.0.1 <none> 443/TCP 4h

Note that the

ambassador

service has been running for 45 minutes, and it was still for an external IP. Hopping on the Docker Slack channel, I found out that the service controller doesn’t update the service object (yet). It turns out that the service is actually exposed locally. So the following worked:

$ curl localhost:80

I added a mapping for Ambassador to route

/httpbin/

to the

httpbin.org

service with the following configuration YAML:

apiVersion: v1
kind: Service
metadata:
  name: httpbin
  annotations:
    getambassador.io/config: |
      ---
      apiVersion: ambassador/v0
      kind:  Mapping
      name:  httpbin_mapping
      prefix: /httpbin/
      service: httpbin.org:80
      host_rewrite: httpbin.org
spec:
  ports:
  - port: 80

And it worked perfectly:

$ curl localhost:80/httpbin/ip { "origin": "65.217.185.138" }

Some other notes on Ingress

In some conversations on the Slack channel, I learned a few other quirks:

To get a list of open ports, you can compile this binary. I haven’t tried this.
The service controller does not yet handle collisions between competing services. So the last service will win.

Docker Stacks and CRDs

Docker includes a native integration between Docker Swarm and Kubernetes with a

stack

custom resource definition. I’ve heard from many users how they love the simplicity of Docker Compose , and the

stack

CRD seems like a compelling approach.

Conclusion

Docker for Mac with Kubernetes has a lot of promise. While there are the rough edges you’d expect with any beta software, the Docker team has done an amazing job of building a useful alternative to Minikube. In addition, I’m excited to see how they’ve thought through how to make the experience idiomatic for Kubernetes users. I’m looking forward to updates!