Blogs

Etcdctl v2 and v3 Aliases for Peer Authenticated Commands

February 8, 2019

Getting all the arguments to etcdctl right can be a bit of a pain. Here are a couple of aliases which take advantage of the values in the etcd.conf file. alias etcd2='. /etc/etcd/etcd.conf && \ ETCDCTL_API=2 etcdctl \ --cert-file ${ETCD_PEER_CERT_FILE} \ --key-file ${ETCD_PEER_KEY_FILE} \ --ca-file ${ETCD_PEER_TRUSTED_CA_FILE:-$ETCD_PEER_CA_FILE} \ --endpoints "${ETCD_ADVERTISE_CLIENT_URLS}"' alias etcd3='. /etc/etcd/etcd.conf && \ ETCDCTL_API=3 etcdctl \ --cert ${ETCD_PEER_CERT_FILE} \ --key ${ETCD_PEER_KEY_FILE} \ --cacert ${ETCD_PEER_TRUSTED_CA_FILE:-$ETCD_PEER_CA_FILE} \ --endpoints "${ETCD_ADVERTISE_CLIENT_URLS}"' If you are using OpenShift, you may also find that you already have some bash functions enabled by the etcd role in /etc/profile.

Migration of Etcd to Masters for OpenShift 3.9 to 3.10 Upgrade

February 8, 2019

As of OpenShift Container Platform 3.10 etcd is expected to run in static pods on the master nodes in the control plane. You may have a deployed an HA cluster with dedicated etcd nodes managed with systemd. How do you migrate the this new architecture? Assumptions: You are running OCP 3.9 You have multiple Master nodes You have dedicated Etcd nodes You are running RHEL, not Atomic nodes Outline: Backup etcd Scale up Etcd cluster to include Master nodes Configure Openshift Masters to ignore the old Etcd nodes Scale down etcd cluster to remove old Etcd nodes Detailed Steps Follow along in this document https://docs.

How to Create and Use OpenStack Heat Orchestration Templates Part 1

November 21, 2018

OpenStack enables automated creation of resources such as networks, routers, and servers using Heat Orchestration Templates. If you are new to OpenStack and are using a TripleO based distribution you may have seen them up close and personal without knowing it. What follows is a very basic exploration of Heat. Heat templates are written in YAML format, and you can quickly see from the documentation that a basic template will likely have 4 sections:

Creating OpenStack Provider Network for Use by a Single Project

October 30, 2018

OpenStack supports “provider” networks, which are networks that pre-exist in your physical infrastructure and are “provided” to the cloud users rather than created by the user. Only an admin is permitted to create a provider network. A prequisite is the provider network must be plumbed to the external bridge on your controller and nova nodes. Here is an Ansible playbook to create a project, place a unshared provider network and subnet in that project.

Load balancing of OpenShift HA Routers Mind the GARP

February 16, 2018

OpenShift HA Routing uses haproxy application routers to get traffic into the cluster. These application routers are made redundant by running ipfailover (keepalived) pods to maintain a set of Virtual IPs on each infrastructure node where the application routers run. These VIPs are then referenced by round robin DNS records to enable a measure of load balancing. OK, so now you are load balancing at the network layer, but what about the link layer?

OpenShift 3.6 Upgrade Metrics Fails Missing heapster-certs Secret

October 13, 2017

After your upgrade to OpenShift v3.6 did the deployment of cluster metrics wind up with empty graphs? Check if the heapster pod failed to start due to a missing secret called heapster-certs in the openshift-infra namespace. Problem Heapster pod is failing to start $ oc get pods NAME READY STATUS RESTARTS AGE hawkular-cassandra-1-l1f3s 1/1 Running 0 9m hawkular-metrics-rdl07 1/1 Running 0 9m heapster-cfpcj 0/1 ContainerCreating 0 3m Check what volumes it is attempting to mount

Installing OpenShift on OpenStack

August 20, 2017

This is a work in progress The OpenShift Container Platform (OCP) can run on many types of infrastructure; from a Docker contrainer, to a single VM, to a fleet of baremetal or VMs on an infrastructure provider such as RHV, VMware, Amazon EC2, Google Compute Engine, or OpenStack Platform (OSP). This post is to document my experimentation with setting up OCP on OSP. Doc Overview So where are the docs?

OpenStack Network Diagram

August 14, 2017

What does the networking for OpenStack look like? Maybe something like this. # Network VLAN IP CIDR N1 Provisioning (PXE) V:310 172.23.32.0/20 N2 Internal API V:311 172.23.21.0/24 N3 Storage Network (Front) V:312 172.23.22.0/24 N4 Storage Mgmt (Back) V:313 172.23.23.0/24 N5 External Floating IPs V:179 192.0.179.0/24 N6 Public API V:177 192.0.177.0/24 N7 Overcloud Provisioning (Tenant PXE) V:314 172.23.48.0/20 N8 Provider Network (Tenant VM with physical router) V:175 192.0.175.0/24 N9 Tenant Network (tunnels) V:317 172.

How to push an image to an unexposed OpenShift Docker registry

August 9, 2017

How do I push an image to the OpenShift Docker registry if it is not exposed outside the cluster? Login to a member node Get on a machine that has docker and participates in the cluster SDN or can somehow access that network. (eg. 172.30.0.0/16) Get the IP of the registry oc get svc docker-registry -n default --template "{{ .spec.clusterIP }}" SVC_REGISTRY=$(oc get svc docker-registry -n default --template "{{ .spec.clusterIP }}") Get a token for your session

Automated Pruning of OpenShift Artifacts; Builds, Deploys, Images

March 22, 2017

After running openshift for a while I discovered that letting builds pile up to around to around 1,200 led to what was essentially a deadlock in the scheduling of new builds. New builds were stuck in a New, waiting state indefinitely. This was fixed as of OCP 3.4.1, but it caused me to get more pro-active in the pruning of artifacts within OpenShift. I threw together a script and a playbook to deploy it.