Open in app

Sign up

Sign in

Write

Sign up

Sign in

Abstract Application Models for Kubernetes with the KCL Programming Language and Konfig Library

The KCL Programming Language
Dev Genius
Published in
8 min readJan 29, 2023

--

Introduction

Kusion Configuration Language (KCL) is an open-source constraint-based record and functional language. KCL improves the writing of a large number of complex configurations through mature programming language technology and practice, and is committed to building better modularity, scalability and stability around configuration, simpler logic writing, fast automation and good ecological extensionality.

Overview

In KCL, it is recommended to uniformly manage all configurations and model libraries in the way of configuration library, that is, to store not only KCL definitions of the abstract model itself, but also various types of configurations, such as application operation and maintenance configuration, policy, etc. The configuration is recommended to be hosted in various VCS systems to facilitate configuration rollback and drift check. The best practice code of the configuration repository is Konfig, and the repository is hosted in Github。

⚡ The Konfig repository mainly includes:

  • KCL module declaration file (kcl.mod)
  • KCL domain model libraries (Kubernetes, Prometheus, etc.)
  • Directories of various configurations (application operation and maintenance configuration, etc)
  • Configuration build and test scripts (Makefile, Github CI file, etc.)

The reason for using a unified warehouse to manage all KCL configuration codes is that different code packages have different R&D entities, which will lead to package management and version management problems. When the business configuration code and basic configuration code are stored in a unified warehouse, the version dependency management between codes will be relatively simple. By locating the directory and file of the unique code base, the configuration code can be managed uniformly for easy search, modification and maintenance.

The following is the architecture of Konfig:

Konfig provides users with an out-of-the-box and highly abstract configuration interface. The original simple starting point of the model library is to improve the efficiency and experience of Kubernetes YAML users. We hope to simplify the writing of user-side configuration code by abstracting and encapsulating the model with more complex code into a unified model. Konfig consists of the following parts:

Core model:

  • Front-end model: The front-end model is the “user interface”, which contains all configurable attributes exposed to users on the platform side. Some repetitive and deducible configurations are omitted, and essential attributes are abstracted and exposed to users. It has user-friendly features, such as server.k.
  • Back-end model: The back-end model is “model implementation”, which is the model that makes the properties of the front-end model effective. It mainly contains the rendering logic of the front-end model instance. The back-end model can use KCL to write validation, logic judgment, code fragment reuse and other code to improve the reusability and robustness of the configuration code, and is not sensitive to users, such as server_backend.k.

Domain model: It is a model that does not contain any implementation logic and abstraction. It is often generated by tool transformation and does not need to be modified. It corresponds to the real effective YAML attribute one by one. The domain model needs to be further abstracted and is generally not directly used by users. For example, kusion_kubernetes is the domain model library of Kubernetes scenarios.

In addition, the core model simplifies the configuration code of front-end users through two layers of abstraction: the front-end model and the back-end model. The domain model is automatically generated through the KCL OpenAPI tool. You can find more information at https://medium.com/dev-genius/integrate-kubernetes-models-into-kcl-bb2d562ad81a

Structure of Konfig

Overview​

.
├── .github             # CI Scripts
├── Makefile            # Building and testing scripts
├── README.md           # Documents
├── appops              # Application configuration. This folder is used to place KCL operation and maintenance configuration of all applications
│   ├── clickhouse-operator
│   ├── code-city
│   ├── guestbook
│   ├── http-echo
│   └── nginx-example
├── base                # Models
│   ├── examples        # Examples
│   │   ├── monitoring  # Monitoring example
│   │   ├── native      # Kubernetes resource example
│   │   ├── provider    # Basic resource configuration example such as Terraform resource
│   │   └── server      # Server example.
│   └── pkg
│       ├── kusion_kubernetes   # Kubernetes domain models
│       ├── kusion_models       # Core models
│       ├── kusion_prometheus   # Prometheus domain models
│       └── kusion_provider     # Basic resource models such as Terraform resource
└── kcl.mod             # The KCL module declaration file

Core Model​

The core model library is generally named kusion_models, mainly including front-end model, back-end model, renderer, etc. The directory structure is:

├── commons         # Common models
├── kube            # Cloud-native resource core models
│   ├── backend         # Back-end models
│   ├── frontend        # Front-end models
│   │   ├── common          # Common front-end models
│   │   ├── configmap       # ConfigMap
│   │   ├── container       # Container
│   │   ├── ingress         # Ingress
│   │   ├── resource        # Resource
│   │   ├── secret          # Secret
│   │   ├── service         # Service
│   │   ├── sidecar         # Sidecar
│   │   ├── strategy        # strategy
│   │   ├── volume          # Volume
│   │   └── server.k        # The `Server` model
│   ├── metadata        # Kubernetes metadata
│   ├── mixins          # Mixin
│   ├── render          # Front-to-back-end renderers.
│   ├── templates       # Data template
│   └── utils
└── metadata        # Common metadata

Project and Stack​

Project and Stack are logical isolation concepts used to organize the Konfig.

Project​

Any folder that contains the file project.yaml will be regarded as a Project, and the project.yaml is used to describe the metadata of this Project like name and tenant. Projects must have clear business semantics and must belong to a tenant. Users can map an application or an operation scenario to a Project.

Stack​

Like Project, any folder that contains the file stack.yaml will be regarded as a Stack and stack.yaml is used to describe the metadata of this Stack. Stack is a set of .k files that represents the smallest operation unit that can be configured and deployed individually. It tends to represent different stages in the CI/CD process, such as dev, gray, prod, etc.

Relationship between Project and Stack​

A Project contains one or more Stacks, and a Stack must belong to and can only belong to one Project. Users can interpret the meaning of Project and Stack according to their own needs and flexibly organize the Konfig structure. We provide the following example as a best practice according to our experiences:

appops/nginx-example
├── README.md       # Project readme
├── base            # common configurations for all stacks
│   └── base.k      
├── dev             # dev stack 
│   ├── ci-test     # CI test configs
│   │   ├── settings.yaml       # test data 
│   │   └── stdout.golden.yaml  # expected test result
│   ├── kcl.yaml    # kcl config
│   ├── main.k      
│   └── stack.yaml  # Stack metadata
└── project.yaml    # Project metadata

The Project represents an application and Stack represents different environments of this application, such as dev, pre, prod, etc. Common configurations can be stored in a base directory under this Project.

Quick Start

This guide shows you how to use the KCL language and CLIs to complete the deployment of an application running in Kubernetes. We call the abstraction of application operation and maintenance configuration as Server, and its instance as Application. It is essentially an operation and maintenance model defined by KCL.

In actual production, the application online generally needs to update several k8s resources:

  • Namespace
  • Deployment
  • Service

This guide requires you to have a basic understanding of Kubernetes. If you are not familiar with the relevant concepts, please refer to the links below:

Prerequisites​

Before we start, we need to complete the following steps:

  1. Install KCL — See Download and Install for more details.
  2. Clone the Konfig repo
git clone https://github.com/KusionStack/konfig.git && cd konfig

Compiling​

The programming language of the project is KCL, not JSON/YAML which Kubernetes recognizes, so it needs to be compiled to get the final output.

Enter stack dir appops/nginx-example/dev and compile:

cd appops/nginx-example/dev && kcl -Y kcl.yaml -D __konfig_output_format__=raw

The output YAML is:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx-exampledev
  namespace: nginx-example
spec:
  replicas: 1
  selector:
    matchLabels:
      app.kubernetes.io/name: nginx-example
      app.kubernetes.io/env: dev
      app.kubernetes.io/instance: nginx-example-dev
      app.kubernetes.io/component: nginx-exampledev
  template:
    metadata:
      labels:
        app.kubernetes.io/name: nginx-example
        app.kubernetes.io/env: dev
        app.kubernetes.io/instance: nginx-example-dev
        app.kubernetes.io/component: nginx-exampledev
    spec:
      containers:
      - image: nginx:1.7.8
        name: main
        ports:
        - containerPort: 80
          protocol: TCP
        resources:
          limits:
            cpu: 100m
            memory: 100Mi
            ephemeral-storage: 1Gi
          requests:
            cpu: 100m
            memory: 100Mi
            ephemeral-storage: 1Gi
---
apiVersion: v1
kind: Namespace
metadata:
  name: nginx-example
---
apiVersion: v1
kind: Service
metadata:
  name: nginx-example
  namespace: nginx-example
spec:
  ports:
  - nodePort: 30201
    port: 80
    targetPort: 80
  selector:
    app.kubernetes.io/name: nginx-example
    app.kubernetes.io/env: dev
    app.kubernetes.io/instance: nginx-example-dev
    app.kubernetes.io/component: nginx-exampledev
  type: NodePort

After compiling, we can see three resources:

  • A Deployment with the name nginx-exampledev
  • A Namespace with the name nginx-example
  • A Service with the name nginx-example

The above completes the configuration and takes effect. Later, we can use the command kubectl apply to apply and check the actual status of resources. This guide will not elaborate, you can find more information at https://medium.com/@xpf6677/kcl-make-kubernetes-resource-management-easier-f03ee820c7a4

Modification​

The image attribute in the Server model is used to declare the application's container image. We can modify the image value in base/main.k to modify or upgrade the image:

14c14d
<     image = "nginx:1.7.8"
---
>     image = "nginx:latest"

Recompile the configuration code to obtain the modified YAML output:

kcl -Y kcl.yaml -D __konfig_output_format__=raw
apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx-exampledev
  namespace: nginx-example
spec:
  replicas: 1
  selector:
    matchLabels:
      app.kubernetes.io/name: nginx-example
      app.kubernetes.io/env: dev
      app.kubernetes.io/instance: nginx-example-dev
      app.kubernetes.io/component: nginx-exampledev
  template:
    metadata:
      labels:
        app.kubernetes.io/name: nginx-example
        app.kubernetes.io/env: dev
        app.kubernetes.io/instance: nginx-example-dev
        app.kubernetes.io/component: nginx-exampledev
    spec:
      containers:
      - image: nginx:latest
        name: main
        ports:
        - containerPort: 80
          protocol: TCP
        resources:
          limits:
            cpu: 100m
            memory: 100Mi
            ephemeral-storage: 1Gi
          requests:
            cpu: 100m
            memory: 100Mi
            ephemeral-storage: 1Gi
---
apiVersion: v1
kind: Namespace
metadata:
  name: nginx-example
---
apiVersion: v1
kind: Service
metadata:
  name: nginx-example
  namespace: nginx-example
spec:
  ports:
  - nodePort: 30201
    port: 80
    targetPort: 80
  selector:
    app.kubernetes.io/name: nginx-example
    app.kubernetes.io/env: dev
    app.kubernetes.io/instance: nginx-example-dev
    app.kubernetes.io/component: nginx-exampledev
  type: NodePort

More Documents

See the community for ways to join us.

Give us a star ⭐️ — If you are using KCL or think it is an interesting project, we would love a star on Github

Kubernetes
DevOps
Platform Engineering
Programming
Configuration Management

--

--

The KCL Programming Language
Dev Genius
Follow

Written by The KCL Programming Language

186 Followers

Owner of KCL Projects: https://kcl-lang.io

Follow

Help

Status

About

Careers

Blog

Privacy

Terms

Text to speech

Teams