This is the multi-page printable view of this section. Click here to print.
Platform 101: Conceptual Onboarding
1 - Introduction
Summary
This onboarding section is for you when are new to IPCEI-CIS subproject ‘Edge Developer Framework (EDF)’ and you want to know about
- its context to ‘Platform Engineering’
- and why we think it’s the stuff we need to care about in the EDF
Storyline of our current project plan (2024)
- We have the ‘Edge Developer Framework’
- We think the solution for EDF is in relation to ‘Platforming’ (Digital Platforms)
- The next evolution after DevOps
- Gartner predicts 80% of SWE companies to have platforms in 2026
- Platforms have a history since roundabout 2019
- CNCF has a working group which created capabilities and a maturity model
- Platforms evolve - nowadys there are Platform Orchestrators
- Humanitec set up a Reference Architecture
- There is this ‘Orchestrator’ thing - declaratively describe, customize and change platforms!
- Mapping our assumptions to the CNOE solution
- CNOE is a hot candidate to help and fulfill our platform building
- CNOE aims to embrace change and customization!
- Showtime CNOE
Please challenge this story!
Please do not think this story and the underlying assumptions are carved in stone!
- Don’t miss to further investigate and truely understand EDF specification needs
- Don’t miss to further investigate and truely understand Platform capabilities on top of DevOps
- Don’t miss to further investigate and truely understand Platform orchestration
- Don’t miss to further investigate and truely understand specific orchestratiing solutions like CNOE
Your role as ‘Framework Engineer’ in the Domain Architecture
Pls be aware of the the following domain and task structure of our mission:
2 - Edge Developer Framework
Summary
The ‘Edge Developer Framework’ is both the project and the product we are working for. Out of the leading ‘Portfolio Document’ we derive requirements which are ought to be fulfilled by Platform Engineering.
This is our claim!
What are the specifications we know from the IPCEI-CIS Project Portfolio document
Reference: IPCEI-CIS Project Portfolio Version 5.9, November 17, 2023
DTAG´s IPCEI-CIS Project Portfolio (p.12)
e. Development of DTAG/TSI Edge Developer Framework
- Goal: All developed innovations must be accessible to developer communities in a highly user-friendly and easy way
Development of DTAG/TSI Edge Developer Framework (p.14)
| capability | major novelties | ||
|---|---|---|---|
| e.1. Edge Developer full service framework (SDK + day1 +day2 support for edge installations) | Adaptive CI/CD pipelines for heterogeneous edge environments | Decentralized and self healing deployment and management | edge-driven monitoring and analytics |
| e.2. Built-in sustainability optimization in Edge developer framework | sustainability optimized edge-aware CI/CD tooling | sustainability-optimized configuration management | sustainability-optimized efficient deployment strategies |
| e.3. Sustainable-edge management-optimized user interface for edge developers | sustainability optimized User Interface | Ai-Enabled intelligent experience | AI/ML-based automated user experience testing and optimization |
DTAG objectives & contributions (p.27)
DTAG will also focus on developing an easy-to-use Edge Developer framework for software developers to manage the whole lifecycle of edge applications, i.e. for day-0-, day-1- and up to day-2- operations. With this DTAG will strongly enable the ecosystem building for the entire IPCEI-CIS edge to cloud continuum and ensure openness and accessibility for anyone or any company to make use and further build on the edge to cloud continuum. Providing the use of the tool framework via an open-source approach will further reduce entry barriers and enhance the openness and accessibility for anyone or any organization (see innovations e.).
WP Deliverables (p.170)
e.1 Edge developer full-service framework
This tool set and related best practices and guidelines will adapt, enhance and further innovate DevOps principles and their related, necessary supporting technologies according to the specific requirements and constraints associated with edge or edge cloud development, in order to keep the healthy and balanced innovation path on both sides, the (software) development side and the operations side in the field of DevOps.
What comes next?
Next we’ll see how these requirements seem to be fulfilled by platforms!
3 - Platform Engineering aka Platforming
Summary
Since 2010 we have DevOps. This brings increasing delivery speed and efficiency at scale. But next we got high ‘cognitive loads’ for developers and production congestion due to engineering lifecycle complexity. So we need on top of DevOps an instrumentation to ensure and enforce speed, quality, security in modern, cloud native software development. This instrumentation is called ‘golden paths’ in intenal develoepr platforms (IDP).
History of Platform Engineering
Let’s start with a look into the history of platform engineering. A good starting point is Humanitec, as they nowadays are one of the biggest players (’the market leader in IDPs.’) in platform engineering.
They create lots of beautiful articles and insights, their own platform products and basic concepts for the platform architecture (we’ll meet this later on!).
Further nice reference to the raise of platforms
- What we call a Platform
- Platforms and the cloud native connectionwhat-platform-engineering-and-why-do-we-need-it#why_we_need_platform_engineering
- Platforms and microservices
- Platforms in the product perspective
Benefit of Platform Engineering, Capabilities
In The Evolution of Platform Engineering the interconnection inbetween DevOps, Cloud Native, and the Rise of Platform Engineering is nicely presented and summarizes:
Platform engineering can be broadly defined as the discipline of designing and building toolchains and workflows that enable self-service capabilities for software engineering organizations
When looking at these ‘capabilities’, we have CNCF itself:
CNCF Working group / White paper defines layerwed capabilities
There is a CNCF working group which provides the definition of Capabilities of platforms and shows a first idea of the layered architecture of platforms as service layer for developers (“product and application teams”):
Important: As Platform engineer also notice the platform-eng-maturity-model
Platform Engineering Team
Or, in another illustration for the platform as a developer service interface, which also defines the ‘Platform Engineering Team’ inbetween:
How to set up Platforms
As we now have evidence about the nescessity of platforms, their capabilities and benefit as self service layer for developers, we want to thin about how to build them.
First of all some important wording to motivate the important term ‘internal developer platfoms’ (we will go into this deeper in the next section with the general architecture), which is clear today, but took years to evolve and is still some amount if effort to jump in:
- Platform: As defined above: A product which serves software engineering teams
- Platform Engineering: Creating such a product
- Internal Developer Platforms (IDP): A platform for developers :-)
- Internal Developer Portal: The entry point of a developer to such an IDP
CNCF mapping from capabilities to (CNCF) projects/tools
Ecosystems in InternalDeveloperPlatform
Build or buy - this is also in pltaform engineering a tweaked discussion, which one of the oldest player answers like this with some oppinioated internal capability structuring:
[internaldeveloperplatform.org[(https://internaldeveloperplatform.org/platform-tooling/)
What comes next?
Next we’ll see how these concepts got structured!
Addendum
Digital Platform defintion from What we call a Platform
Words are hard, it seems. ‘Platform’ is just about the most ambiguous term we could use for an approach that is super-important for increasing delivery speed and efficiency at scale. Hence the title of this article, here is what I’ve been talking about most recently.
Definitions for software and hardware platforms abound, generally describing an operating environment upon which applications can execute and which provides reusable capabilities such as file systems and security.
Zooming out, at an organisational level a ‘digital platform’ has similar characteristics - an operating environment which teams can build upon to deliver product features to customers more quickly, supported by reusable capabilities.
A digital platform is a foundation of self-service APIs, tools, services, knowledge and support which are arranged as a compelling internal product. Autonomous delivery teams can make use of the platform to deliver product features at a higher pace, with reduced co-ordination.
Myths :-) - What are platforms not
common-myths-about-platform-engineering
Platform Teams
This is about you :-), the platform engineering team:
how-to-build-your-platform-engineering-team
in comparison: devops vs sre vs platform
https://www.qovery.com/blog/devops-vs-platform-engineering-is-there-a-difference/
4 - Orchestrators
Summary
When defining and setting up platforms next two intrinsic problems arise:
- it is not declarative and automated
- it is not or least not easily changable
Thus the technology of ‘Platform Orchestrating’ emerged recently, in late 2023.
Platform reference architecture
An interesting difference between the CNCF white paper building blocks and them from Internaldeveloperplatforms is the part orchestrators.
This is something extremely new since late 2023 - the rise of the automation of platform engineering!
It was Humanitec creating a definition of platform architecture, as they needed to defien what they are building with their ‘orchestrator’:
Declarative Platform Orchestration
Based on the refence architecture you next can build - or let’s say ‘orchestrate’ - different platform implementations, based on declarative definitions of the platform design.
https://humanitec.com/reference-architectures
Hint: There is a slides tool provided by McKinsey to set up your own platform deign based on the reference architecture
What comes next?
Next we’ll see how we are going to do platform orchestration with CNOE!
Addendum
Building blocks from Humanitecs ‘state-of-platform-engineering-report-volume-2’
You remember the capability mappings from the time before orchestration? Here we have a similar setup based on Humanitecs platform engineering status ewhite paper:
5 - CNOE
Summary
In late 2023 platform orchestration raised - the discipline of declarativley dinfing, building, orchestarting and reconciling building blocks of (digital) platforms.
The famost one ist the platform orchestrator from Humanitec. They provide lots of concepts and access, also open sourced tools and schemas. But they do not have open sourced the ocheastartor itself.
Thus we were looking for open source means for platform orchestrating and found CNOE.
Requirements for an Orchestrator
When we want to set up a complete platform we expect to have
- a schema which defines the platform, its ressources and internal behaviour
- a dynamic configuration or templating mechanism to provide a concrete specification of a platform
- a deployment mechanism to deploy and reconcile the platform
This is what CNOE delivers:
For seamless transition into a CNOE-compliant delivery pipeline, CNOE will aim at delivering “packaging specifications”, “templating mechanisms”, as well as “deployer technologies”, an example of which is enabled via the idpBuilder tool we have released. The combination of templates, specifications, and deployers allow for bundling and then unpacking of CNOE recommended tools into a user’s DevOps environment. This enables teams to share and deliver components that are deemed to be the best tools for the job.
CNOE (capabilities) architecture
Architecture
CNOE architecture looks a bit different than the reference architecture from Humanitec, but this just a matter of details and arrangement:
Hint: This has to be further investigated! This is subject to an Epic.
Capabilities
You have a definition of all the capabilities here:
Hint: This has to be further investigated! This is subject to an Epic.
Stacks
CNOE calls the schema and templating mechnanism ‘stacks’:
Hint: This has to be further investigated! This is subject to an Epic.
There are already some example stacks:
What comes next?
Next we’ll see how a CNOE stacked Internal Developer Platform is deployed on you local laptop!
6 - CNOE Showtime
Summary
CNOE is a ‘Platform Engineering Framework’ (Danger: Our wording!) - it is open source and locally runnable.
It consists of the orchestrator ‘idpbuilder’ and both of some predefined building blocks and also some predefined platform configurations.
Orchestrator ‘idpbuilder’, initial run
The orchestrator in CNOE is called ‘idpbuilder’. It is locally installable binary
A typipcal first setup ist described here: https://cnoe.io/docs/reference-implementation/technology
# this is a local linux shell
# check local installation
type idpbuilder
idpbuilder is /usr/local/bin/idpbuilder
# check version
idpbuilder version
idpbuilder 0.8.0-nightly.20240914 go1.22.7 linux/amd64
# do some completion and aliasing
source <(idpbuilder completion bash)
alias ib=idpbuilder
complete -F __start_idpbuilder ib
# check and remove all existing kind clusters
kind delete clusters --all
kind get clusters
# sth. like 'No kind clusters found.'
# run
$ib create --use-path-routing --log-level debug --package-dir https://github.com/cnoe-io/stacks//ref-implementation
You get output like
stl@ubuntu-vpn:~/git/mms/ipceicis-developerframework$ idpbuilder create
Oct 1 10:09:18 INFO Creating kind cluster logger=setup
Oct 1 10:09:18 INFO Runtime detected logger=setup provider=docker
########################### Our kind config ############################
# Kind kubernetes release images https://github.com/kubernetes-sigs/kind/releases
kind: Cluster
apiVersion: kind.x-k8s.io/v1alpha4
nodes:
- role: control-plane
image: "kindest/node:v1.30.0"
labels:
ingress-ready: "true"
extraPortMappings:
- containerPort: 443
hostPort: 8443
protocol: TCP
containerdConfigPatches:
- |-
[plugins."io.containerd.grpc.v1.cri".registry.mirrors."gitea.cnoe.localtest.me:8443"]
endpoint = ["https://gitea.cnoe.localtest.me"]
[plugins."io.containerd.grpc.v1.cri".registry.configs."gitea.cnoe.localtest.me".tls]
insecure_skip_verify = true
######################### config end ############################
Show time steps
Goto https://cnoe.io/docs/reference-implementation/installations/idpbuilder/usage, and follow the flow
Prepare a k8s cluster with kind
You may have seen: when starting idpbuilder without an existing cluster named localdev it first creates this cluster by calling kindwith an internally defined config.
It’s an important feature of idpbuilder that it will set up on an existing cluster localdev when called several times in a row e.g. to append new packages to the cluster.
That’s why we here first create the kind cluster localdevitself:
cat << EOF | kind create cluster --name localdev --config=-
# Kind kubernetes release images https://github.com/kubernetes-sigs/kind/releases
kind: Cluster
apiVersion: kind.x-k8s.io/v1alpha4
nodes:
- role: control-plane
image: "kindest/node:v1.30.0"
labels:
ingress-ready: "true"
extraPortMappings:
- containerPort: 443
hostPort: 8443
protocol: TCP
containerdConfigPatches:
- |-
[plugins."io.containerd.grpc.v1.cri".registry.mirrors."gitea.cnoe.localtest.me:8443"]
endpoint = ["https://gitea.cnoe.localtest.me"]
[plugins."io.containerd.grpc.v1.cri".registry.configs."gitea.cnoe.localtest.me".tls]
insecure_skip_verify = true
# alternatively, if you have the kind config as file:
kind create cluster --name localdev --config kind-config.yaml
Output
A typical raw kind setup kubernetes cluster would look like this with respect to running pods:
be sure all pods are in status ‘running’
stl@ubuntu-vpn:~/git/mms/idpbuilder$ k get pods -A
NAMESPACE NAME READY STATUS RESTARTS AGE
kube-system coredns-76f75df574-lb7jz 1/1 Running 0 15s
kube-system coredns-76f75df574-zm2wp 1/1 Running 0 15s
kube-system etcd-localdev-control-plane 1/1 Running 0 27s
kube-system kindnet-8qhd5 1/1 Running 0 13s
kube-system kindnet-r4d6n 1/1 Running 0 15s
kube-system kube-apiserver-localdev-control-plane 1/1 Running 0 27s
kube-system kube-controller-manager-localdev-control-plane 1/1 Running 0 27s
kube-system kube-proxy-vrh64 1/1 Running 0 15s
kube-system kube-proxy-w8ks9 1/1 Running 0 13s
kube-system kube-scheduler-localdev-control-plane 1/1 Running 0 27s
local-path-storage local-path-provisioner-6f8956fb48-6fvt2 1/1 Running 0 15s
First run: Start with core applications, ‘core package’
Now we run idpbuilder the first time:
# now idpbuilder reuses the already existing cluster
# pls apply '--use-path-routing' otherwise as we discovered currently the service resolving inside the cluster won't work
ib create --use-path-routing
Output
idpbuilder log
stl@ubuntu-vpn:~/git/mms/idpbuilder$ ib create --use-path-routing
Oct 1 10:32:50 INFO Creating kind cluster logger=setup
Oct 1 10:32:50 INFO Runtime detected logger=setup provider=docker
Oct 1 10:32:50 INFO Cluster already exists logger=setup cluster=localdev
Oct 1 10:32:50 INFO Adding CRDs to the cluster logger=setup
Oct 1 10:32:51 INFO Setting up CoreDNS logger=setup
Oct 1 10:32:51 INFO Setting up TLS certificate logger=setup
Oct 1 10:32:51 INFO Creating localbuild resource logger=setup
Oct 1 10:32:51 INFO Starting EventSource controller=gitrepository controllerGroup=idpbuilder.cnoe.io controllerKind=GitRepository source=kind source: *v1alpha1.GitRepository
Oct 1 10:32:51 INFO Starting Controller controller=gitrepository controllerGroup=idpbuilder.cnoe.io controllerKind=GitRepository
Oct 1 10:32:51 INFO Starting EventSource controller=localbuild controllerGroup=idpbuilder.cnoe.io controllerKind=Localbuild source=kind source: *v1alpha1.Localbuild
Oct 1 10:32:51 INFO Starting Controller controller=localbuild controllerGroup=idpbuilder.cnoe.io controllerKind=Localbuild
Oct 1 10:32:51 INFO Starting EventSource controller=custompackage controllerGroup=idpbuilder.cnoe.io controllerKind=CustomPackage source=kind source: *v1alpha1.CustomPackage
Oct 1 10:32:51 INFO Starting Controller controller=custompackage controllerGroup=idpbuilder.cnoe.io controllerKind=CustomPackage
Oct 1 10:32:51 INFO Starting workers controller=localbuild controllerGroup=idpbuilder.cnoe.io controllerKind=Localbuild worker count=1
Oct 1 10:32:51 INFO Starting workers controller=custompackage controllerGroup=idpbuilder.cnoe.io controllerKind=CustomPackage worker count=1
Oct 1 10:32:51 INFO Starting workers controller=gitrepository controllerGroup=idpbuilder.cnoe.io controllerKind=GitRepository worker count=1
Oct 1 10:32:54 INFO Waiting for Deployment my-gitea to become ready controller=localbuild controllerGroup=idpbuilder.cnoe.io controllerKind=Localbuild name=localdev name=localdev reconcileID=6fc396d4-e0d5-4c80-aaee-20c1bbffea54
Oct 1 10:32:54 INFO Waiting for Deployment ingress-nginx-controller to become ready controller=localbuild controllerGroup=idpbuilder.cnoe.io controllerKind=Localbuild name=localdev name=localdev reconcileID=6fc396d4-e0d5-4c80-aaee-20c1bbffea54
Oct 1 10:33:24 INFO Waiting for Deployment my-gitea to become ready controller=localbuild controllerGroup=idpbuilder.cnoe.io controllerKind=Localbuild name=localdev name=localdev reconcileID=6fc396d4-e0d5-4c80-aaee-20c1bbffea54
Oct 1 10:33:24 INFO Waiting for Deployment ingress-nginx-controller to become ready controller=localbuild controllerGroup=idpbuilder.cnoe.io controllerKind=Localbuild name=localdev name=localdev reconcileID=6fc396d4-e0d5-4c80-aaee-20c1bbffea54
Oct 1 10:33:54 INFO Waiting for Deployment my-gitea to become ready controller=localbuild controllerGroup=idpbuilder.cnoe.io controllerKind=Localbuild name=localdev name=localdev reconcileID=6fc396d4-e0d5-4c80-aaee-20c1bbffea54
Oct 1 10:34:24 INFO installing bootstrap apps to ArgoCD controller=localbuild controllerGroup=idpbuilder.cnoe.io controllerKind=Localbuild name=localdev name=localdev reconcileID=6fc396d4-e0d5-4c80-aaee-20c1bbffea54
Oct 1 10:34:24 INFO expected annotation, cnoe.io/last-observed-cli-start-time, not found controller=localbuild controllerGroup=idpbuilder.cnoe.io controllerKind=Localbuild name=localdev name=localdev reconcileID=6fc396d4-e0d5-4c80-aaee-20c1bbffea54
Oct 1 10:34:24 INFO Checking if we should shutdown controller=localbuild controllerGroup=idpbuilder.cnoe.io controllerKind=Localbuild name=localdev name=localdev reconcileID=6fc396d4-e0d5-4c80-aaee-20c1bbffea54
Oct 1 10:34:25 INFO installing bootstrap apps to ArgoCD controller=localbuild controllerGroup=idpbuilder.cnoe.io controllerKind=Localbuild name=localdev name=localdev reconcileID=0667fa85-af1c-403f-bcd9-16ff8f2fad7e
Oct 1 10:34:25 INFO expected annotation, cnoe.io/last-observed-cli-start-time, not found controller=localbuild controllerGroup=idpbuilder.cnoe.io controllerKind=Localbuild name=localdev name=localdev reconcileID=0667fa85-af1c-403f-bcd9-16ff8f2fad7e
Oct 1 10:34:25 INFO Checking if we should shutdown controller=localbuild controllerGroup=idpbuilder.cnoe.io controllerKind=Localbuild name=localdev name=localdev reconcileID=0667fa85-af1c-403f-bcd9-16ff8f2fad7e
Oct 1 10:34:40 INFO installing bootstrap apps to ArgoCD controller=localbuild controllerGroup=idpbuilder.cnoe.io controllerKind=Localbuild name=localdev name=localdev reconcileID=ec720aeb-02cd-4974-a991-cf2f19ce8536
Oct 1 10:34:40 INFO Checking if we should shutdown controller=localbuild controllerGroup=idpbuilder.cnoe.io controllerKind=Localbuild name=localdev name=localdev reconcileID=ec720aeb-02cd-4974-a991-cf2f19ce8536
Oct 1 10:34:40 INFO Shutting Down controller=localbuild controllerGroup=idpbuilder.cnoe.io controllerKind=Localbuild name=localdev name=localdev reconcileID=ec720aeb-02cd-4974-a991-cf2f19ce8536
Oct 1 10:34:40 INFO Stopping and waiting for non leader election runnables
Oct 1 10:34:40 INFO Stopping and waiting for leader election runnables
Oct 1 10:34:40 INFO Shutdown signal received, waiting for all workers to finish controller=gitrepository controllerGroup=idpbuilder.cnoe.io controllerKind=GitRepository
Oct 1 10:34:40 INFO Shutdown signal received, waiting for all workers to finish controller=custompackage controllerGroup=idpbuilder.cnoe.io controllerKind=CustomPackage
Oct 1 10:34:40 INFO All workers finished controller=custompackage controllerGroup=idpbuilder.cnoe.io controllerKind=CustomPackage
Oct 1 10:34:40 INFO Shutdown signal received, waiting for all workers to finish controller=localbuild controllerGroup=idpbuilder.cnoe.io controllerKind=Localbuild
Oct 1 10:34:40 INFO All workers finished controller=localbuild controllerGroup=idpbuilder.cnoe.io controllerKind=Localbuild
Oct 1 10:34:40 INFO All workers finished controller=gitrepository controllerGroup=idpbuilder.cnoe.io controllerKind=GitRepository
Oct 1 10:34:40 INFO Stopping and waiting for caches
Oct 1 10:34:40 INFO Stopping and waiting for webhooks
Oct 1 10:34:40 INFO Stopping and waiting for HTTP servers
Oct 1 10:34:40 INFO Wait completed, proceeding to shutdown the manager
########################### Finished Creating IDP Successfully! ############################
Can Access ArgoCD at https://cnoe.localtest.me:8443/argocd
Username: admin
Password can be retrieved by running: idpbuilder get secrets -p argocd
ArgoCD applications
When running idpbuilder ‘barely’ (without package option) you get the ‘core applications’ deployed in your cluster:
stl@ubuntu-vpn:~/git/mms/ipceicis-developerframework$ k get applications -A
NAMESPACE NAME SYNC STATUS HEALTH STATUS
argocd argocd Synced Healthy
argocd gitea Synced Healthy
argocd nginx Synced Healthy
ArgoCD UI
Open ArgoCD locally:
https://cnoe.localtest.me:8443/argocd
Next find the provided credentials for ArgoCD (here: argocd-initial-admin-secret):
stl@ubuntu-vpn:~/git/mms/idpbuilder$ ib get secrets
---------------------------
Name: argocd-initial-admin-secret
Namespace: argocd
Data:
password : 2MoMeW30wSC9EraF
username : admin
---------------------------
Name: gitea-credential
Namespace: gitea
Data:
password : LI$T?o>N{-<|{^dm$eTg*gni1(2:Y0@q344yqQIS
username : giteaAdmin
In ArgoCD you will see the deployed three applications of the core package:
Second run: Append ‘package1’ from the CNOE-stacks repo
CNOE provides example packages in https://github.com/cnoe-io/stacks.git. Having cloned this repo you can locally refer to theses packages:
stl@ubuntu-vpn:~/git/mms/cnoe-stacks$ git remote -v
origin https://github.com/cnoe-io/stacks.git (fetch)
origin https://github.com/cnoe-io/stacks.git (push)
stl@ubuntu-vpn:~/git/mms/cnoe-stacks$ ls -al
total 64
drwxr-xr-x 12 stl stl 4096 Sep 28 13:55 .
drwxr-xr-x 26 stl stl 4096 Sep 30 11:53 ..
drwxr-xr-x 8 stl stl 4096 Sep 28 13:56 .git
drwxr-xr-x 4 stl stl 4096 Jul 29 10:57 .github
-rw-r--r-- 1 stl stl 11341 Sep 28 09:12 LICENSE
-rw-r--r-- 1 stl stl 1079 Sep 28 13:55 README.md
drwxr-xr-x 4 stl stl 4096 Jul 29 10:57 basic
drwxr-xr-x 4 stl stl 4096 Sep 14 15:54 crossplane-integrations
drwxr-xr-x 3 stl stl 4096 Aug 13 14:52 dapr-integration
drwxr-xr-x 3 stl stl 4096 Sep 14 15:54 jupyterhub
drwxr-xr-x 6 stl stl 4096 Aug 16 14:36 local-backup
drwxr-xr-x 3 stl stl 4096 Aug 16 14:36 localstack-integration
drwxr-xr-x 8 stl stl 4096 Sep 28 13:02 ref-implementation
drwxr-xr-x 2 stl stl 4096 Aug 16 14:36 terraform-integrations
stl@ubuntu-vpn:~/git/mms/cnoe-stacks$ ls -al basic/
total 20
drwxr-xr-x 4 stl stl 4096 Jul 29 10:57 .
drwxr-xr-x 12 stl stl 4096 Sep 28 13:55 ..
-rw-r--r-- 1 stl stl 632 Jul 29 10:57 README.md
drwxr-xr-x 3 stl stl 4096 Jul 29 10:57 package1
drwxr-xr-x 2 stl stl 4096 Jul 29 10:57 package2
stl@ubuntu-vpn:~/git/mms/cnoe-stacks$ ls -al basic/package1
total 16
drwxr-xr-x 3 stl stl 4096 Jul 29 10:57 .
drwxr-xr-x 4 stl stl 4096 Jul 29 10:57 ..
-rw-r--r-- 1 stl stl 655 Jul 29 10:57 app.yaml
drwxr-xr-x 2 stl stl 4096 Jul 29 10:57 manifests
stl@ubuntu-vpn:~/git/mms/cnoe-stacks$ ls -al basic/package2
total 16
drwxr-xr-x 2 stl stl 4096 Jul 29 10:57 .
drwxr-xr-x 4 stl stl 4096 Jul 29 10:57 ..
-rw-r--r-- 1 stl stl 498 Jul 29 10:57 app.yaml
-rw-r--r-- 1 stl stl 500 Jul 29 10:57 app2.yaml
Output
Now we run idpbuilder the second time with -p basic/package1
idpbuilder log
stl@ubuntu-vpn:~/git/mms/cnoe-stacks$ ib create --use-path-routing -p basic/package1
Oct 1 12:09:27 INFO Creating kind cluster logger=setup
Oct 1 12:09:27 INFO Runtime detected logger=setup provider=docker
Oct 1 12:09:27 INFO Cluster already exists logger=setup cluster=localdev
Oct 1 12:09:28 INFO Adding CRDs to the cluster logger=setup
Oct 1 12:09:28 INFO Setting up CoreDNS logger=setup
Oct 1 12:09:28 INFO Setting up TLS certificate logger=setup
Oct 1 12:09:28 INFO Creating localbuild resource logger=setup
Oct 1 12:09:28 INFO Starting EventSource controller=localbuild controllerGroup=idpbuilder.cnoe.io controllerKind=Localbuild source=kind source: *v1alpha1.Localbuild
Oct 1 12:09:28 INFO Starting Controller controller=localbuild controllerGroup=idpbuilder.cnoe.io controllerKind=Localbuild
Oct 1 12:09:28 INFO Starting EventSource controller=custompackage controllerGroup=idpbuilder.cnoe.io controllerKind=CustomPackage source=kind source: *v1alpha1.CustomPackage
Oct 1 12:09:28 INFO Starting Controller controller=custompackage controllerGroup=idpbuilder.cnoe.io controllerKind=CustomPackage
Oct 1 12:09:28 INFO Starting EventSource controller=gitrepository controllerGroup=idpbuilder.cnoe.io controllerKind=GitRepository source=kind source: *v1alpha1.GitRepository
Oct 1 12:09:28 INFO Starting Controller controller=gitrepository controllerGroup=idpbuilder.cnoe.io controllerKind=GitRepository
Oct 1 12:09:28 INFO Starting workers controller=localbuild controllerGroup=idpbuilder.cnoe.io controllerKind=Localbuild worker count=1
Oct 1 12:09:28 INFO Starting workers controller=gitrepository controllerGroup=idpbuilder.cnoe.io controllerKind=GitRepository worker count=1
Oct 1 12:09:28 INFO Starting workers controller=custompackage controllerGroup=idpbuilder.cnoe.io controllerKind=CustomPackage worker count=1
Oct 1 12:09:29 INFO installing bootstrap apps to ArgoCD controller=localbuild controllerGroup=idpbuilder.cnoe.io controllerKind=Localbuild name=localdev name=localdev reconcileID=0ed7ccc2-dec7-4ab8-909c-791a7d1b67a8
Oct 1 12:09:29 INFO unknown field "status.history[0].initiatedBy" logger=KubeAPIWarningLogger
Oct 1 12:09:29 INFO Checking if we should shutdown controller=localbuild controllerGroup=idpbuilder.cnoe.io controllerKind=Localbuild name=localdev name=localdev reconcileID=0ed7ccc2-dec7-4ab8-909c-791a7d1b67a8
Oct 1 12:09:29 ERROR failed updating repo status controller=custompackage controllerGroup=idpbuilder.cnoe.io controllerKind=CustomPackage name=app-my-app namespace=idpbuilder-localdev namespace=idpbuilder-localdev name=app-my-app reconcileID=f9873560-5dcd-4e59-b6f7-ce5d1029ef3d err=Operation cannot be fulfilled on custompackages.idpbuilder.cnoe.io "app-my-app": the object has been modified; please apply your changes to the latest version and try again
Oct 1 12:09:29 ERROR Reconciler error controller=custompackage controllerGroup=idpbuilder.cnoe.io controllerKind=CustomPackage name=app-my-app namespace=idpbuilder-localdev namespace=idpbuilder-localdev name=app-my-app reconcileID=f9873560-5dcd-4e59-b6f7-ce5d1029ef3d err=updating argocd application object my-app: Operation cannot be fulfilled on applications.argoproj.io "my-app": the object has been modified; please apply your changes to the latest version and try again
Oct 1 12:09:31 INFO installing bootstrap apps to ArgoCD controller=localbuild controllerGroup=idpbuilder.cnoe.io controllerKind=Localbuild name=localdev name=localdev reconcileID=531cc2d4-6250-493a-aca8-fecf048a608d
Oct 1 12:09:31 INFO Checking if we should shutdown controller=localbuild controllerGroup=idpbuilder.cnoe.io controllerKind=Localbuild name=localdev name=localdev reconcileID=531cc2d4-6250-493a-aca8-fecf048a608d
Oct 1 12:09:44 INFO installing bootstrap apps to ArgoCD controller=localbuild controllerGroup=idpbuilder.cnoe.io controllerKind=Localbuild name=localdev name=localdev reconcileID=022b9813-8708-4f4e-90d7-38f1e114c46f
Oct 1 12:09:44 INFO Checking if we should shutdown controller=localbuild controllerGroup=idpbuilder.cnoe.io controllerKind=Localbuild name=localdev name=localdev reconcileID=022b9813-8708-4f4e-90d7-38f1e114c46f
Oct 1 12:10:00 INFO installing bootstrap apps to ArgoCD controller=localbuild controllerGroup=idpbuilder.cnoe.io controllerKind=Localbuild name=localdev name=localdev reconcileID=79a85c21-42c1-41ec-ad03-2bb77aeae027
Oct 1 12:10:00 INFO Checking if we should shutdown controller=localbuild controllerGroup=idpbuilder.cnoe.io controllerKind=Localbuild name=localdev name=localdev reconcileID=79a85c21-42c1-41ec-ad03-2bb77aeae027
Oct 1 12:10:00 INFO Shutting Down controller=localbuild controllerGroup=idpbuilder.cnoe.io controllerKind=Localbuild name=localdev name=localdev reconcileID=79a85c21-42c1-41ec-ad03-2bb77aeae027
Oct 1 12:10:00 INFO Stopping and waiting for non leader election runnables
Oct 1 12:10:00 INFO Stopping and waiting for leader election runnables
Oct 1 12:10:00 INFO Shutdown signal received, waiting for all workers to finish controller=custompackage controllerGroup=idpbuilder.cnoe.io controllerKind=CustomPackage
Oct 1 12:10:00 INFO Shutdown signal received, waiting for all workers to finish controller=gitrepository controllerGroup=idpbuilder.cnoe.io controllerKind=GitRepository
Oct 1 12:10:00 INFO All workers finished controller=custompackage controllerGroup=idpbuilder.cnoe.io controllerKind=CustomPackage
Oct 1 12:10:00 INFO Shutdown signal received, waiting for all workers to finish controller=localbuild controllerGroup=idpbuilder.cnoe.io controllerKind=Localbuild
Oct 1 12:10:00 INFO All workers finished controller=localbuild controllerGroup=idpbuilder.cnoe.io controllerKind=Localbuild
Oct 1 12:10:00 INFO All workers finished controller=gitrepository controllerGroup=idpbuilder.cnoe.io controllerKind=GitRepository
Oct 1 12:10:00 INFO Stopping and waiting for caches
Oct 1 12:10:00 INFO Stopping and waiting for webhooks
Oct 1 12:10:00 INFO Stopping and waiting for HTTP servers
Oct 1 12:10:00 INFO Wait completed, proceeding to shutdown the manager
########################### Finished Creating IDP Successfully! ############################
Can Access ArgoCD at https://cnoe.localtest.me:8443/argocd
Username: admin
Password can be retrieved by running: idpbuilder get secrets -p argocd
ArgoCD applications
Now we have additionally the ‘my-app’ deployed in the cluster:
stl@ubuntu-vpn:~$ k get applications -A
NAMESPACE NAME SYNC STATUS HEALTH STATUS
argocd argocd Synced Healthy
argocd gitea Synced Healthy
argocd my-app Synced Healthy
argocd nginx Synced Healthy
ArgoCD UI
Third run: Finally we append ‘ref-implementation’ from the CNOE-stacks repo
We finally append the so called ‘reference-implementation’, which provides a real basic IDP:
stl@ubuntu-vpn:~/git/mms/cnoe-stacks$ ib create --use-path-routing -p ref-implementation
ArgoCD applications
stl@ubuntu-vpn:~$ k get applications -A
NAMESPACE NAME SYNC STATUS HEALTH STATUS
argocd argo-workflows Synced Healthy
argocd argocd Synced Healthy
argocd backstage Synced Healthy
argocd included-backstage-templates Synced Healthy
argocd external-secrets Synced Healthy
argocd gitea Synced Healthy
argocd keycloak Synced Healthy
argocd metric-server Synced Healthy
argocd my-app Synced Healthy
argocd nginx Synced Healthy
argocd spark-operator Synced Healthy
ArgoCD UI
ArgoCD shows all provissioned applications:
Keycloak UI
In our cluster there is also keycloak as IAM provisioned.
Login into Keycloak with ‘cnoe-admin’ and the KEYCLOAK_ADMIN_PASSWORD.
These credentails are defined in the package:
stl@ubuntu-vpn:~/git/mms/cnoe-stacks$ cat ref-implementation/keycloak/manifests/keycloak-config.yaml | grep -i admin
group-admin-payload.json: |
{"name":"admin"}
"/admin"
ADMIN_PASSWORD=$(cat /var/secrets/KEYCLOAK_ADMIN_PASSWORD)
--data-urlencode "username=cnoe-admin" \
--data-urlencode "password=${ADMIN_PASSWORD}" \
stl@ubuntu-vpn:~/git/mms/cnoe-stacks$ ib get secrets
---------------------------
Name: argocd-initial-admin-secret
Namespace: argocd
Data:
password : 2MoMeW30wSC9EraF
username : admin
---------------------------
Name: gitea-credential
Namespace: gitea
Data:
password : LI$T?o>N{-<|{^dm$eTg*gni1(2:Y0@q344yqQIS
username : giteaAdmin
---------------------------
Name: keycloak-config
Namespace: keycloak
Data:
KC_DB_PASSWORD : k3-1kgxxd/X2Cw//pX-uKMsmgWogEz5YGnb5
KC_DB_USERNAME : keycloak
KEYCLOAK_ADMIN_PASSWORD : zMSjv5eA0l/+0-MDAaaNe+rHRMrB2q0NssP-
POSTGRES_DB : keycloak
POSTGRES_PASSWORD : k3-1kgxxd/X2Cw//pX-uKMsmgWogEz5YGnb5
POSTGRES_USER : keycloak
USER_PASSWORD : Kd+0+/BqPRAvnLPZO-L2o/6DoBrzUeMsr29U
Backstage UI
As Backstage login you either can use the ‘user1’ with USER_PASSWORD : Kd+0+/BqPRAvnLPZO-L2o/6DoBrzUeMsr29U or you create a new user in keycloak
We create user ‘ipcei’ and also set a password (in tab ‘Credentials’):
Now we can log into backstage (rember: you could have already existing usr ‘user1’):
and see the basic setup of the Backstage portal:
Use a Golden Path: ‘Basic Deployment’
Now we want to use the Backstage portal as a developer. We create in Backstage our own platform based activity by using the golden path template ‘Basic Deployment:
When we run it, we see ‘golden path activities’
which finally result in a new catalogue entry:
Software development lifecycle
When we follow the ‘view source’ link we are directly linked to the git repo of our newly created application:
Check it out by cloning into a local git repo (watch the GIT_SSL_NO_VERIFY=true env setting):
stl@ubuntu-vpn:~/git/mms/idp-temporary$ GIT_SSL_NO_VERIFY=true git clone https://cnoe.localtest.me:8443/gitea/giteaAdmin/basicdeployment.git
Cloning into 'basicdeployment'...
remote: Enumerating objects: 10, done.
remote: Counting objects: 100% (10/10), done.
remote: Compressing objects: 100% (8/8), done.
remote: Total 10 (delta 0), reused 0 (delta 0), pack-reused 0 (from 0)
Receiving objects: 100% (10/10), 47.62 KiB | 23.81 MiB/s, done.
stl@ubuntu-vpn:~/git/mms/idp-temporary$ cd basicdeployment/
stl@ubuntu-vpn:~/git/mms/idp-temporary/basicdeployment$ ll
total 24
drwxr-xr-x 5 stl stl 4096 Oct 1 13:00 ./
drwxr-xr-x 4 stl stl 4096 Oct 1 13:00 ../
drwxr-xr-x 8 stl stl 4096 Oct 1 13:00 .git/
-rw-r--r-- 1 stl stl 928 Oct 1 13:00 catalog-info.yaml
drwxr-xr-x 3 stl stl 4096 Oct 1 13:00 docs/
drwxr-xr-x 2 stl stl 4096 Oct 1 13:00 manifests/
Edit and change
Change some things, like the decription and the replicas:
Push
Push your changes, use the giteaAdmin user to authenticate:
stl@ubuntu-vpn:~/git/mms/idp-temporary/basicdeployment$ ib get secrets
---------------------------
Name: argocd-initial-admin-secret
Namespace: argocd
Data:
password : 2MoMeW30wSC9EraF
username : admin
---------------------------
Name: gitea-credential
Namespace: gitea
Data:
password : LI$T?o>N{-<|{^dm$eTg*gni1(2:Y0@q344yqQIS
username : giteaAdmin
---------------------------
Name: keycloak-config
Namespace: keycloak
Data:
KC_DB_PASSWORD : k3-1kgxxd/X2Cw//pX-uKMsmgWogEz5YGnb5
KC_DB_USERNAME : keycloak
KEYCLOAK_ADMIN_PASSWORD : zMSjv5eA0l/+0-MDAaaNe+rHRMrB2q0NssP-
POSTGRES_DB : keycloak
POSTGRES_PASSWORD : k3-1kgxxd/X2Cw//pX-uKMsmgWogEz5YGnb5
POSTGRES_USER : keycloak
USER_PASSWORD : Kd+0+/BqPRAvnLPZO-L2o/6DoBrzUeMsr29U
stl@ubuntu-vpn:~/git/mms/idp-temporary/basicdeployment$ GIT_SSL_NO_VERIFY=true git push
Username for 'https://cnoe.localtest.me:8443': giteaAdmin
Password for 'https://giteaAdmin@cnoe.localtest.me:8443':
Enumerating objects: 5, done.
Counting objects: 100% (5/5), done.
Delta compression using up to 8 threads
Compressing objects: 100% (3/3), done.
Writing objects: 100% (3/3), 382 bytes | 382.00 KiB/s, done.
Total 3 (delta 1), reused 0 (delta 0), pack-reused 0
remote: . Processing 1 references
remote: Processed 1 references in total
To https://cnoe.localtest.me:8443/gitea/giteaAdmin/basicdeployment.git
69244d6..1269617 main -> main
Wait for gitops magic: deployment into the ‘production’ cluster
Next wait a bit until Gitops does its magic and our ‘wanted’ state in the repo gets automatically deployed to the ‘production’ cluster:
What comes next?
The showtime of CNOE high level behaviour and usage scenarios is now finished. We setup an initial IDP and used a backstage golden path to init and deploy a simple application.
Last not least we want to sum up the whole way from Devops to ‘Frameworking’ (is this the correct wording???)
7 - Conclusio
Summary
In the project ‘Edge Developer Framework’ we start with DevOps, set platforms on top to automate golden paths, and finally set ‘frameworks’ (aka Orchestrators’) on top to have declarative,automated and reconcilable platforms.
From Devops over Platform to Framework Engineering
We come along from a quite well known, but already complex discipline called ‘Platform Engineering’, which is the next level devops. On top of these two domains we now have ‘Framework Engineering’, i.e. buildung dynamic, orchestrated and reconciling platforms:
| Classic Platform engineering | New: Framework Orchestration on top of Platforms | Your job: Framework Engineer |
|---|---|---|
 |  |  |
The whole picture of engineering
So always keep in mind that as as ‘Framework Engineer’ you
- include the skills of a platform and a devops engineer,
- you do Framework, Platform and Devops Engineering at the same time
- and your results have impact on Frameworks, Platforms and Devops tools, layers, processes.
The following diamond is illustrating this: on top is you, on the bottom is our baseline ‘DevOps’
7.1 -
// how to create/export c4 images: // see also https://likec4.dev/tooling/cli/
docker run -it –rm –name likec4 –user node -v $PWD:/app node bash npm install likec4 exit
docker commit likec4 likec4 docker run -it –rm –user node -v $PWD:/app -p 5173:5173 likec4 bash
// as root npx playwright install-deps npx playwright install
npm install likec4
// render node@e20899c8046f:/app/content/en/docs/project/onboarding$ ./node_modules/.bin/likec4 export png -o ./images .
8 -
Storyline
- We have the ‘Developer Framework’
- We think the solution for DF is ‘Platforming’ (Digital Platforms)
- The next evolution after DevOps
- Gartner predicts 80% of SWE companies to have platforms in 2026
- Platforms have a history since roundabout 2019
- CNCF has a working group which created capabilities and a maturity model
- Platforms evolve - nowadys there are Platform Orchestrators
- Humanitec set up a Reference Architecture
- There is this ‘Orchestrator’ thing - declaratively describe, customize and change platforms!
- Mapping our assumptions to solutions
- CNOE is a hot candidate to help and fulfill our platform building
- CNOE aims to embrace change and customization!
- Showtime CNOE
Challenges
- Don’t miss to further investigate and truely understand DF needs
- Don’t miss to further investigate and truely understand Platform capabilities
- Don’t miss to further investigate and truely understand Platform orchestration
- Don’t miss to further investigate and truely understand CNOE solution