If you are a developer, the chances are you have heard about Kubernetes. You heard that it is an amazing tool to help you scale your applications and manage your micro-services. But, you probably also heard that it is VERY complex. It is so complex that you were probably scared off. And I don’t blame you; that is the first reaction I got as well.

If you search the top posts with the Kubernetes tags on this website, you will find a myriad of tutorials and people explaining Kubernetes. These posts are the most trending because people WANT to understand Kubernetes because we feel like, in today's software development world, Kubernetes is unavoidable. And this is true, to an extent…

Software developers are often required to understand and work with Kubernetes; if you have ever looked for jobs in this sector, you know this already. But what if there was a tool to minimize your touching points with Kubernetes? A tool that simplifies the process and gives you guidance when trying to deploy applications into Kubernetes clusters. A tool that is highly customizable and lets someone in your organization (who understands Kubernetes, commonly known as a DevOps) create a user interface for you!

Yep, you guessed it, it’s Cyclops! 😄

And just to clarify, Cyclops is not used to create and manage Kubernetes clusters and other infrastructure; rather, Cyclops is used for deploying and managing applications INSIDE the cluster.

Show us your support 🙏🏻​

We are building Cyclops to be open-source, and your support would mean the world to us. Consider giving us a star on GitHub and following us on ProductHunt, where we scheduled our very first release!

Before we start​

In order to test out Cyclops, you are going to need a few things. If this is not your first time using Kubernetes, the chances are you already have everything ready, but we will still describe each of the components for the newcomers to the Kubernetes space. These tools are not only used for Cyclops, and you can use them for anything Kubernetes-related.

The main thing you are going to need to test out Cyclops is a Kubernetes cluster. If you have one that you can use to play with, great; if not, we will show you how to spin up a cluster on your own computer. So, the three prerequisites for doing this are:

1. 1. Docker
2. 2. Minikube
3. 3. kubectl

Docker is the most popular containerization tool, and we will use it to download and spin up a Minikube image. Downloading Docker is straightforward: go to their webpage and download the Docker Desktop application.

Minikube plays the role of a Kubernetes cluster on your local machine. It is a great tool for developing and testing out your Kubernetes applications, and it is perfect for this scenario. You can find a guide on how to install it here.

The final thing missing is a way of communicating with your Kubernetes cluster, and this is done through the Kubernetes command line tool called kubectl. It can be used to deploy applications, inspect and manage cluster resources, and view logs. In this tutorial, we will use it to install Cyclops into our cluster on Minikube and expose its functionality outside the cluster.

Installing Cyclops​

Once you have your Kubernetes cluster ready (check the Before We Start section), installing Cyclops is a straightforward process. Using kubectl, run the following command in your terminal:

kubectl apply -f https://raw.githubusercontent.com/cyclops-ui/cyclops/v0.2.0/install/cyclops-install.yaml

It will create a new namespace called cyclops and deploy everything you need for your Cyclops instance to run.

Now, all that is left is to expose the Cyclops server outside the cluster. You will need to expose both the backend and frontend with the commands below.

Expose frontend through:

kubectl port-forward svc/cyclops-ui 3000:3000 -n cyclops

And the backend through:

kubectl port-forward svc/cyclops-ctrl 8080:8080 -n cyclops

And that's it! You can now access Cyclops in your browser at http://localhost:3000. If you are having trouble with the port-forward commands, you probably just need to wait a few of seconds after installing Cyclops into your cluster, it can take a while to start all it’s resources.

It’s Demo Time 💥​

Now that you have your Cyclops instance up and running, it’s time to see what it’s capable of.

You should be greeted with an almost empty screen with no deployed modules showing. Module is Cyclops’s slang for application 😎. So, let’s start by creating our first module!

By clicking on the Add module button in the top right corner, you should be taken to a new screen. Here, Cyclops asks us which Helm chart do we want to deploy.

Not to go too deep, but Helm is a very popular open-source package manager for Kubernetes. It helps you create configuration files that are needed for applications running in Kubernetes. These charts let Kubernetes know how to handle your application in the cluster.

Don’t worry; to showcase the basics of Cyclops, we created a simple Helm chart so that anyone can follow along. You can find what it looks like in our GitHub repository, along with a couple of more examples of Helm charts that you can use!

As you can see, once you enter the repository of your chart, Cyclops will render a user interface. If you want to find out the magic behind the rendering, check out our previous blog.

You can fill out the fields as you wish, but be mindful of the Kubernetes naming conventions!

If you want to follow along, my input is as follows:

name: demo
replicas: 1
image: nginx
version: 1.14.2
service: true

We will set the module name to demo as well. Click save, and Cyclops will show you the details of your new module.

This screen shows you all the resources your application is using at the moment. It will list all the deployments, services, pods, or any other resource. Here, we can see that Cyclops deployed one pod into your cluster, as we specified in the replicas field. If you want to make sure that it really is running in your cluster, you can check it out by using the following kubectl command:

kubectl get pods

But what if all of a sudden, there was a need to scale up your application or any other resource? Well, don't worry; with Cyclops, it’s really easy!

By clicking the Edit button, you can change the values of your application’s resources. Let’s try to scale our application up to 3 replicas and see what happens.

You should now see two more pods in the Deployment tab; hurray! 🎉

Of course, this works for any other change you might want to make to your application. Like, the service, perhaps? What if we realized we don't really need it anymore? Well, with Cyclops, it's really easy to shut it down if need be.

Click again on the Edit button, and this time, turn off the service toggle.

Cyclops won't delete it automatically but will warn you (via the warning triangle sign) that you shut it down, and it is not in function anymore. This means you can safely delete it!

And if you are sick and tired of your application, you can delete the whole thing as well 🗑️

Click on the Delete button and fill in the name of the module to safely delete it. You can, again, check if it really was deleted with kubectl:

kubectl get pods

Finish​

And that’s all there really is to it! Cyclops allows people with varying knowledge of Kubernetes to leverage its power. If you followed this tutorial, you should have deployed your very first application using Cyclops; congratz! 🎉 On our webpage, you can find one more tutorial showcasing more features and a more complicated use case, as well as our contact and community info.

If you have any sort of feedback or ideas on how to make Cyclops better, you can fill out our short Google form!

A couple of days ago, I held a talk about Kubernetes and its components at the college I used to go to. My mom said she liked the talk, so I turned it into a blog post.

Many software engineers tend to look away from anything related to Kubernetes, even though they might use it daily. At first glance, it seems complex and like a whole new world to dive into. And yeah, it is, but in this blog post, I will go over all of the main components of a Kubernetes cluster and explain what they do in an example.

By the end of the blog post, you won't be a Kubernetes expert, but you will probably get a good idea of what to look for and how to structure the chaos that Kubernetes seems to be at first.

Show us your support 🙏🏻​

Before we start, we would love it if you starred our repository and helped us get our tool in front of other developers. Our GitHub repo is here: https://github.com/cyclops-ui/cyclops ⭐

Components

First of all, we can divide a Kubernetes cluster into two parts: control plane and worker nodes. The control plane takes care of the whole operation and controls the state of our cluster. We’ll get into what that means shortly. On the other side, our worker nodes are essentially just computers listening to what the control plane tells them to do. They are the computing power of our cluster. Any application we run in the cluster will run on those nodes.

Let’s decompose that further.

Control plane​

As we said, the control plane is making sure our cluster is running as expected. It does that by communicating with the cluster user, scheduling workloads, managing cluster state and so on.

The control plane is made of four crucial components. Simple by themselves, but together, they create a complex system. These components are:

1. 1. API
2. 2. ETCD
3. 3. Scheduler
4. 4. Controller Manager

Control plane components can be run on any machine in the cluster, but are usually run on a separate set of machines, often called master nodes. Those machines are not used to run any other container or application and are reserved for the Kubernetes control plane.

API​

The Kubernetes API acts as the cluster's front-end interface, allowing users to interact with the cluster, define desired states, and perform operations such as creating, updating, and deleting resources.

It is the only point of contact we have with the cluster. Also, no other components are talking directly to each other, but all communication is happening through the API.

ETCD​

ETCD is the API’s database; it's as simple as that. When you tell Kubernetes to create a deployment, it gets stored in the ETCD alongside all the other created resources.

One characteristic of ETCD is that its key-value storage is organized as a filesystem. Another great feature of ETCD is that users can subscribe to events and get notified about changes. For example, let me know when a new pod gets created.

Scheduler​

As the name suggests, the scheduler decides which node a pod will run on. It does that by a set of rules you can read in the Kubernetes documentation. This is what I meant when I said you won't be an expert, but you will know what to google :)

The Scheduler subscribes to all newly created pods saved in ETCD, but it can only talk with the API to get this update.

When it catches that a pod has been created, it calculates which worker node to run it on. Once it's made up its mind, the scheduler doesn't run anything on any machine; it just tells the API to run the pod on a particular node.

Controller Manager​

The last component from the control plane is the controller manager. We can take it as a thermostat for our cluster. Its job is to shift the current state of the cluster to the desired state.

This means that it will create all the needed resources under the hood to satisfy our needs and get our applications up and running.

It runs multiple controller processes subscribed to changes on the ETCD, compiled into the same binary for easier deployment. Controller managers’ roles and what those controllers do will be defined more closely later in the blog.

Worker nodes​

Now that we have concluded what manages the whole cluster, let's dive into where our containers are running and how that is achieved.

There are 3 components running on each node in a Kubernetes cluster. Of course, you can have multiple nodes in a cluster, but each needs these three components to host your applications.

Those being:

1. 1. container runtime
2. 2. kubelet
3. 3. kube proxy

Container runtime​

The component that allows Kubernetes to run containers and manages the lifecycle of a container on a node is the container runtime.

Multiple container runtimes are supported, like conatinerd, cri-o, or other CRI compliant runtimes.

Kubelet​

Another component subscribed to pod events is Kubelet. Each time a pod is scheduled on a node, the Kubelet running on that node will hear that and start all defined containers. On top of that, Kubelet also performs health checks to ensure everything is running as expected.

Kube proxy​

KubeProxy in Kubernetes manages network connectivity between pods across the cluster, handling tasks like load balancing and network routing. It ensures seamless communication among pods by maintaining network rules and translating service abstractions into actionable network policies.

From a deployment to a running container

Now that we have listed all of the components and their role in a Kubernetes cluster, let's tell a story on how a Kubernetes Deployment becomes a set of containers running on various machines across the cluster.

Pods, Replicasets and Deployments​

Just a quick reminder on the relation of these three: Pods, Replicasets, and Deployments.

The smallest unit we can deploy in a Kubernetes cluster is a pod. With it, we are going to define our containers.

Most likely, we will need a couple of instances of the same application, and we can define how to replicate our pods with a Replicaset. It will ensure that we have the desired number of pods running by starting and terminating them.

Cool, now we have our application replicated, but we would like to roll out a new version of our application. We have to tear down existing Pods/Replicaset and create new ones. A Deployment will automate this process, allowing us to roll out our feature safely.

The Prestige​

Now that we have all our terminology and touched on all Kubernetes components and their role, let's see what happens when we “apply” a Deployment to a Kubernetes cluster.

Let's say that we have created a deployment.yaml file defining our application (you can see how to do that here) and ran kubectl apply -f deployment.yaml. kubectl will now submit our deployment definition to our cluster's only point of contact - the Kubernetes API.

Our simple API will store our deployment in the ETCD database. Each time a Deployment object is saved into ETCD, it will let the API know that there was a change on Deployments and that it should let everybody who is subscribed to such an event know about it.

And there is a component in the control plane that would like to know when a new Deployment spawns, and that's the Controller Manager. When it hears about a new Deployment, it will create a new Replicaset based on the Deployment configuration. To make this Replicaset, it will call the API with a create request.

Creating a Replicaset is much like creating a Deployment. API will receive a Replicaset to create and store into ETCD. This will make ETCD tell the API that somebody created a Replicaset and pass that information to all subscribed components, which is again the Controller Manager.

When the Controller Manager hears about the new Replicaset, it will create all the Pods defined with the Replicaset by, you guessed it, calling the API, which will store all those Pods into ETCD.

As we said, a lot of things happened, so we decided to create a GIF that might help you understand the whole process under the hood.

Here, we include the Scheduler, which is subscribed to the Pod creation event. Each time it hears about a new Pod, it decides on which node it should be run. The Scheduler is not running the Pod but only telling the API which node it chose for it. The API will then save that information.

Another component listening to Pod events is the Kubelet, a component running on each worker node in the Kubernetes cluster. Each time the API tells the Kubelet that the Scheduler decided to run the Pod on its node, the Kubelet will start all the containers defined by the Pod.

Finally, we turned our configuration into an application running on a machine! It is a lengthy process with many moving parts, but this may be my favorite part.

Each component takes just a tiny bit of the responsibility of deploying an application, but they solve a pretty complex problem together.

Final thoughts

Hope this article helped you get a grasp on Kubernetes components and helped you demystify the most popular orchestrator out there. We encourage you to dig around yourself because we enjoyed learning about this.

One book we recommend to learn about Kubernetes is “Kubernetes in action” by Marko Lukša. It is pretty popular and gives an excellent overview of what is going on under the hood of Kubernetes and how to use it.

Kubernetes has become the go-to platform for managing containerized applications, offering scalability, flexibility, and robustness. However, the complexity of Kubernetes can be daunting, requiring developers and DevOps teams to navigate through intricate configuration files and command-line interactions.

Several powerful development tools have emerged to simplify the management of Kubernetes clusters and streamline the deployment process. In this article, we will explore five Kubernetes development tools:

1. 1. Prometheus
2. 2. Cyclops
3. 3. Keda
4. 4. Karpenter
5. 5. Velero

These tools offer intuitive user interfaces, automated scaling capabilities, disaster recovery solutions, and improved efficiency in managing Kubernetes clusters.

Show us your support 🙏🏻​

Before we start, we would love it if you starred our repository and helped us get our tool in front of other developers. Our GitHub repo is here: https://github.com/cyclops-ui/cyclops ⭐

1. Prometheus: Monitoring and Alerting for Kubernetes​

Prometheus is an open-source monitoring and alerting toolkit designed specifically for microservices and containers. It offers flexible querying, real-time notifications, and visibility into containerized workloads, APIs, and distributed services.

One of the features of Prometheus is its ability to assist with cloud-native security by detecting irregular traffic or activity that could potentially escalate into an attack.

It uses a pull-based system, sending HTTP requests called "scrapes", to collect metrics from applications and services. These metrics are stored in memory and on local disk, allowing for easy retrieval and analysis.

Prometheus can access data directly from client libraries or through exporters, which are software located adjacent to the application. Exporters accept HTTP requests from Prometheus, ensure the data format compatibility, and serve the requested data to the Prometheus server.

Prometheus provides four main types of metrics: Counter, Gauge, Histogram, and Summary. These metrics offer flexibility in measuring various aspects of applications and services, such as event start counts, memory usage, data aggregation, and quantile ranges.

To discover targets for monitoring, Prometheus utilizes service discovery in Kubernetes clusters. It can access machine-level metrics separately from application information, allowing for comprehensive monitoring.

Once the data collection is complete, Prometheus provides a query language called PromQL, which enables users to access and export monitoring data to graphical interfaces like Grafana or send alerts using Alertmanager.

2. Cyclops: Deploying applications with just a couple of clicks​

Cyclops is a tool that simplifies the management of applications running in Kubernetes clusters. It abstracts complex configuration files into form-based UIs, eliminating the need for manual configuration and command-line interactions. This makes the deployment process more accessible to individuals with varying levels of technical expertise.

With Cyclops, you're not boxed into a one-size-fits-all approach. You can customize modules to suit your unique needs, giving you the freedom to create templates with input validation for seamless collaboration with your team.

This not only speeds up your work but also empowers each team member to work independently, promoting a smoother and more efficient workflow.

In Cyclops, every module lays out a detailed list of resources it uses—deployments, services, pods, and others, all in plain view. You can easily track their status, helping you quickly spot and fix any hiccups in your application. It's like having a clear roadmap to navigate and troubleshoot any issues that pop up.

Within the architecture of Cyclops, a central component is the Helm engine, which allows the dynamic generation of configurations. This engine serves as a key mechanism for efficiently managing settings and parameters in the Cyclops framework.

As Kubernetes-based systems commonly employ Helm as their package manager, seamlessly integrating Cyclops is a straightforward process.

Cyclops promotes consistency and standardization in deployment practices. By providing predefined templates or configuration presets, Cyclops ensures that deployments adhere to established best practices and guidelines. This consistency not only improves the reliability and stability of deployments but also facilitates collaboration.

3. Keda: Event-Driven Autoscaling for Kubernetes Workloads​

Kubernetes Horizontal Pod Autoscaling (HPA) and Vertical Pod Autoscaling (VPA) are widely used for autoscaling Kubernetes clusters based on CPU and memory usage.

However, they have limitations, such as the inability to scale pods to zero or scale based on metrics other than resource utilization. This is where Keda (Kubernetes Event-Driven Autoscaling) comes into play.

Keda is an open-source container autoscaler that extends the capabilities of native Kubernetes autoscaling solutions by scaling pods based on external events or triggers.

Monitoring event sources like AWS SQS, Kafka, and RabbitMQ, Keda efficiently triggers or halts deployments based on predefined rules. This adaptable solution also allows for custom metrics, facilitating effective autoscaling tailored for message-driven microservices, ensuring optimal performance and resource utilization.

The components of Keda include event sources, scalers, metrics adapters, and controllers. Event sources provide the external events that trigger scaling, while scalers monitor these events and fetch metrics. Metrics adapters translate the metrics for the controller, which then scales the deployments accordingly.

By leveraging Keda, DevOps teams can free up resources and reduce cloud costs by scaling down when there are no events to process. Keda also offers interoperability with various DevOps toolchains, supporting both built-in and external scalers.

With Keda, autoscaling becomes more flexible and efficient, empowering teams to optimize resource utilization and adapt to changing workload requirements.

4. Karpenter: Automated Node Provisioning for Kubernetes​

Kubernetes clusters often face the challenge of scheduling pods on available nodes. Karpenter is an open-source cluster auto scaler that automatically provisions new nodes in response to un-schedulable pods. It evaluates the aggregate resource requirements of pending pods and selects the optimal instance type to accommodate them.

Karpenter also supports a consolidation feature, actively moving pods and replacing nodes with cheaper versions to reduce cluster costs.

A standout feature is the introduction of "Node Pools," allowing users to categorize nodes based on various criteria. This customization ensures a tailored approach to resource allocation, with Karpenter dynamically provisioning nodes into the most fitting pools.

At its core, Karpenter is designed to automate the scaling of Kubernetes clusters seamlessly. Leveraging Custom Resource Definitions (CRDs) within Kubernetes, Karpenter integrates seamlessly with existing tools and APIs, providing a familiar experience for users.

The flexibility of Karpenter extends beyond the confines of AWS, making it a versatile solution for both cloud and on-premises environments.

Karpenter's adaptability shines through its support for user-defined strategies and policies through Kubernetes resources. This flexibility enables organizations to align Karpenter with their unique application and workload requirements, enabling better automated and optimized Kubernetes scalability.

5. Velero: Disaster Recovery and Backup for Kubernetes​

Velero is a powerful tool that provides disaster recovery and backup solutions for Kubernetes clusters. It enables users to easily backup, restore, and migrate applications and their persistent volumes.

Velero takes snapshots of cluster resources and data, storing them in object storage providers like AWS S3, Google Cloud Storage, or Azure Blob Storage.

With Velero, users can create backup schedules, ensuring regular snapshots of critical cluster resources. This allows for efficient disaster recovery in case of data loss or cluster failures. Velero also supports cluster migration, simplifying the process of moving applications and data between Kubernetes clusters.

The tool offers resource filtering capabilities, allowing users to selectively backup and restore specific resources.

This flexibility ensures that only relevant data is included in the backup, saving storage space and reducing backup and restore times. Velero integrates with CSI (Container Storage Interface), providing support for backing up volumes and restoring them to their original state.

In addition to disaster recovery and backup, Velero provides features like running in any namespace, extending functionality with hooks, and supporting custom plugins for enhanced customization. It offers troubleshooting guides for diagnosing and resolving common issues, ensuring a smooth experience in managing Kubernetes clusters.

Conclusion​

These five Kubernetes development tools - Prometheus, Cyclops, Keda, Karpenter, and Velero - play pivotal roles in simplifying the complexities of Kubernetes cluster management.

From monitoring and alerting with Prometheus to event-driven autoscaling using Keda, and automated node provisioning through Karpenter, each tool addresses specific challenges, contributing to more efficient and resilient Kubernetes environments.

Cyclops stands out for its user-friendly approach, abstracting complex configurations into intuitive UIs, while Velero provides crucial disaster recovery and backup solutions for safeguarding critical data and applications.

As Kubernetes continues to be a cornerstone in modern application deployment, these tools empower developers and DevOps teams to navigate the intricacies of containerized environments with greater ease.

By integrating these tools into your Kubernetes workflows, you can enhance scalability, streamline deployment processes, and ensure the robustness of your applications in today's dynamic and demanding computing landscape.

Cyclops turns complicated YAML manifests into simple and structured UIs where developers can click away their Kubernetes application configuration. ”Great! But how does it know how to render this UI? Should I implement a UI form each time I need a new set of fields to configure? I don’t know React! I don’t know frontend!“

This blog post should cure your anxiety about implementing a UI for each type of application and explain how Cyclops knows what to render so you can deploy to your K8s cluster carefree.

To better understand how Cyclops renders the UI, we will scratch the surface of Helm, which Cyclops uses as its templating engine.

A bit about Helm​

Helm is a Kubernetes package manager that helps deploy and manage Kubernetes resources by packing them into charts. It also has a templating engine that allows developers to configure their apps depending on the specific values injected into the helm template.

The usual Helm chart structure is as follows:

├── Chart.yaml
├── templates
│   ├── deployment.yaml
│   └── service.yaml
├── values.schema.json
└── values.yaml

A few other Helm chart parts are left out on purpose since they are not tangible to the rest of the blog. You can read more about each of those in Helm’s official documentation

• Chart.yaml - A YAML file containing information about the chart (like name, version…)
• templates - A directory of templates that, when combined with values, will generate valid Kubernetes manifest files
• values.yaml - The default configuration values for this chart
• values.schema.json - A JSON Schema for imposing a structure on the values.yaml file

When using Helm, you can change your values.yaml however you see fit for your application. The problem is that you can change them however you like, which allows you to misconfigure some parts of your application because you misspelled a field or messed up indentation in the values.yaml.

Here is where JSON schema from the values.schema.json comes in. It will define which fields you should set and even to which values (e.g., you can specify that a field called replicas can’t be set to lower than 0). Helm won’t let you render a Kubernetes manifest with values that don’t comply with the schema. There is an example of such schema later in the blog, but you can also check it out on Helms official docs

Helm values schema and Cyclops UI​

Now that the schema's purpose in a Helm chart is explained let’s get into how Cyclops uses it.

Since the primary purpose of the values schema is to describe what the Helm chart needs to render all the Kubernetes resources, we naturally decided to use it for rendering the UI. On the first iterations of Cyclops, we implemented a solution where users can define those fields in the UI, but why reinvent the wheel when Helm already provided a way to specify this?

Cyclops controller reads the Helm chart and values schema. Then, it recursively traverses through all the fields in the schema and renders the field based on the field specification. It knows how to render a field based on the field type (string, boolean, object, array...), description of the field, field rules (e.g., minimum or maximum value), and many more.

Now that the UI is rendered, a user of Cyclops can click through the form and fill in those fields. Thanks to the schema, values entered by a developer will now always conform to the schema since the UI won’t let you specify any fields (e.g., allow you typos in field names) or set the number of replicas to three instead of 3. This is an exaggerated example, but you can probably see the point. The UI will take care of validating your input, and you will have clear guidelines on how to configure your service.

Once values are entered and saved in the UI, they are passed to the Helm templating engine and the templates from the /templates folder. This results in all Kubernetes resources being configured for the needs of each team/developer without getting into specific implementation details of each resource.

Final thoughts​

Hope this blog post helped you understand how the rendering part of Cyclops works and demystified the whole project. We briefly touched on Helm and JSON schema, but both are larger pieces of software that we can't describe in such a short blog post, so we encourage you to check their documentation.

Hi all!

We are launching a blog post series on topics relevant to people following our startup journey. From technical topics like building high availability apps in Kubernetes to nontechnical ones, like our experience in some of the accelerators we have been through.

Overall, we hope you will enjoy the content, and of course, you are more than encouraged to propose some topics you would like to see here on our Discord.

Also, if you are interested in contributing to our project, you can find open issues on our GitHub repository, and while you are there, give it a star ⭐

Blog posts coming soon...