6GSMART-ICC: A Public-private network integration system approach

A public-private network integration system from a cloud point of view is one of the main drivers to support multi-domain cloud-native orchestration in industrial and manufacturing environments. The system must provide specific functionalities that enable multi-domain orchestration by integrating public-private clouds and with a strategic approach that gives observability and monitoring mechanisms, allowing centralized visibility of the managed infrastructure. In this regard, the solution focuses on providing observability and monitoring mechanisms. It must cover the specific requirements of public-private integration concerning application management, such as allowing the deployment of the different microservices of a distributed application across multiple computing resources, expanding across various domains (public-private), and in turn, being able to monitor metrics and, key performance identifiers (KPIs) related to the deployed applications. At the same time, it must offer the ability to abstract multi-cluster and multi-domain complexity, network and connectivity between internal application components, implementing different observability mechanisms to monitor metrics and logs related to the managed infrastructure’s utilization, performance and status. The Observability Stack A unified platform capable of collecting, processing, and visualizing data from various clusters and domains, whether in public or private clouds, is essential to ensuring effective observability and monitoring in a multi-cloud environment. In this context, lightweight monitoring agents deployed across different clusters collect metrics and send them to the central platform, enabling a unified view regardless of the cluster’s nature. Likewise, data normalization mechanisms, utilizing metrics and log standards, ensure consistent data collection across clusters. For this, a specific taxonomy identifies essential data elements (latency, CPU/memory usage, errors, etc.) and facilitates metadata enrichment, enabling efficient correlation and analysis. Additionally, multi-cluster monitoring dashboards provide real-time visibility into the infrastructure’s status, helping identify potential errors and anomalies. The solution implemented uses OpenTelemetry, a project of the Cloud Native Computing Foundation (CNCF), which defines an industry standard for telemetry data management as the basis for its data collection and processing capabilities. This standardised approach enables the consistent, vendor-independent collection, processing, and export of metrics, logs, and traces. Use case validation To evaluate the solution, we have used the use case developed to validate the Cloud-Continuum ORChestrator (CORC) capabilities, which allows the deployment and management of a distributed application in multiple domains integrating public-private clouds. The following diagram shows three computing domains (extreme edge, edge and public cloud) that will be centrally monitored through the deployment of the observability stack for multi-cluster environments, which will oversee monitoring resources (energy, CPU, memory consumption), covering clusters, nodes, workloads, etcetera. The red arrows indicate the components that make up the observability stack. The use case involves deploying a distributed video streaming application composed of different microservices, each deployed in a specific domain. The three zones or domains (extreme edge, edge and cloud) are independent clusters, each composed of a certain number of nodes. We have a cluster representing the extreme edge deployed on Raspberrys, using K3s as a container orchestrator. A second cluster that represents the edge deployed on physical servers, using K8S as orchestration software. Finally, a third cluster representing the public cloud (simulated) will be deployed in demo time to be part of the managed infrastructure, allowing access to the Graphana dashboard that shows various visualizations of the different metrics that make it possible to monitor the hybrid public-private infrastructure centrally. This allows us to see how new computing resources appear in the public-private infrastructure and where workloads can be deployed. Author: Jesús Benedicto – IT Project Manager & SW Architect at Eviden