OpsCruise Built-In Edge Kubernetes Cluster Observability

By Robert Belson (Developer Relations Lead, Verizon) & Cesar Quintana (Principal Solutions Architect, OpsCruise)

Summary

As Kubernetes continues on the path to greatness to all things cloud architecture, the ability to understand how your workloads are performing becomes an imperative. Verizon’s Wavelength Zones bring compute power closer to end users, meaning network latency is significantly reduced via endpoint optimization.

• Kubernetes workloads are migrating to the edge in Wavelength Zones with the need to deploy applications close to end users
• This melding of network and application aspects leads to a more focused and considered approach by the network providers and the enterprises deploying at the edge

This “compute at the edge” significantly reduces latency for end users, and, in turn, makes the performance of the application itself all the more important. As compute power moves to the edge, the network alone is no longer the sole consideration for performance, but the performance of the application itself becomes a key factor in maintaining happy end users.

Verizon 5G Edge Primer

Verizon 5G Edge brings the best of cloud compute and storage to the edge of the Verizon network – across 4G, 5G, Cat-M IoT and beyond – to deliver more performant user experiences. With AWS Wavelength, developers can leverage the best experiences of their cloud environment across their Amazon Virtual Private Cloud (VPC) environments and Elastic Compute Cloud (EC2) instances through the concept of Wavelength Zones: an extension of the traditional Availability Zone concept in the context of the carrier network. 

From autonomous mobility and predictive maintenance to contactless checkout and multiplayer AR/VR worlds, startups and enterprises alike have begun embracing the edge for new ways to design applications and opportunities to drive performance across existing experiences. 

Observing Kubernetes in Wavelength Zones (and Availability Zones, too!)

The need to observe Kubernetes applications at the edge becomes highly important as businesses deploy more streaming and highly time-critical applications and services. These latency-sensitive applications will not be viable if they  cannot meet their performance targets and ensure that developers can maximize their ROI on edge platforms.

You need insight into all types of Kubernetes cluster deployments:

• Kubernetes Clusters in Wavelength Zones
• Kubernetes Clusters inside Standard Availability Zones 
• Kubernetes Cluster in Hybrid/on-prem data centers

These insights must be present throughout the two phases of a customer’s edge enablement journey:

Phase 1, Deployment: As customers increasingly develop applications at the edge, a number of questions arise:

• Where should logs be collected, aggregated, and analyzed?
• What about Metrics? Traces? Events? Configuration? Changes?
• How will we operationalize it all in a single place?
• How can we get real-time insights on the application performance and availability, preferably automatically?

These decisions require a tactful understanding of Kubernetes networking, application performance monitoring architectures, analytics tools, and a long-term strategy on managing edge applications. For example, wouldn’t it be nice to have full application instrumentation out-of-the-box, regardless of whether you’re deploying to one Wavelength Zone or 13?

Phase 2, Management and Orchestration: After deployment, customers need the confidence that their edge environment is performing as expected across carrier-facing endpoints as well as internal Kuberenetes nodes and pods. With the advent of Verizon’s Edge Discovery Service (EDS), developers can now leverage performance monitoring to maintain an accurate service registry for their edge endpoints.

But what happens if you de-register a carrier-facing endpoint (i.e., NodePort/Load Balancer) or change your deployment manifest altogether? Wouldn't it be nice to have the Edge Discovery Service “automagically” adapt?

These are non-trivial problems to solve, but, here at OpsCruise, we’re teaming up with Verizon to begin to solve them starting with out-of-the-box instrumentation.

OpsCruise Kubernetes Capabilities

OpsCruise provides real-time Application, Kubernetes, and Cloud Infrastructure Insights, helping predict, identify, and in many cases, surface the root cause of the problems across the following areas:

• Kubernetes-based environments and the Infrastructure they’re running on
• Resource Deployment Issues
• Application Performance Degradation
• Long Chain Dependencies

‍

Easily Deploy Kubernetes Clusters with Built-In Observability

Wavelength Kubernetes cluster deployment can be automated, typically with “Infrastructure as Code” tools such as CloudFormation. Not only can the cluster deployment be automated, but the observability portion provided by OpsCruise can be built-in transparently to the cluster right from the get-go.

OpsCruise Observing the Edge - Get Started

‍

‍

1. Go to https://opscruise.com/free-forever and sign up for an OpsCruise account.

2. Once your OpsCruise instance has been created, log in to your OpsCruise instance

3. Click on your Username in the upper right corner, then Deployment Guide

4. Click on Download YAML File
‍

‍

5. In Section 2 of the Deployment guide, select AWS, then Amazon Linux, and follow the instructions to create an IAM role in order to allow OpsCruise to query AWS

‍

‍

6. Upload your opscruise-values.yaml to an S3 bucket, and have the URL handy (you’ll need it in Step 7)

7. Run the following commands (see the GitHub repository for more details on the commands) at https://github.com/Verizon/5GEdgeTutorials/tree/main/opscruise :

‍

git clone https://github.com/Verizon/5GEdgeTutorials.git

cd ./5GEdgeTutorials/opscruise

aws cloudformation deploy \

	--template-file ./awsqseks-helm-typeactivation.packaged.yaml \

	--stack-name AWSEKSTypeActivation \

	--capabilities CAPABILITY_NAMED_IAMEKSwOCStackName=eksCluster-wOpsCruise-$(date +%b%d%Y-%H%M)

aws cloudformation deploy --template-file ./wavelength-eksCluster-withOpsCruise.packaged.yml \

	--stack-name $EKSwOCStackName \

	--capabilities CAPABILITY_NAMED_IAM \

	--parameter-overrides \

		OpsCruiseValuesURL=$S3_URL_TO_OPSCRUISE_VALUES_YAML \

		EKSClusterAdminArn=$ARN_FOR_EKS_CLUSTER_ADMIN_USER \

		EKSClusterAdminName=$CLUSTER_ADMIN_USERNAME \

		OpsCruiseGWVersion=3.1.1 \

		OpsCruiseCollectorsVersion=3.1.1 \

		AWSQSHelmExecutionRole=$HELM_EXECUTION_ROLE

aws eks update-kubeconfig --name ${EKSwOCStackName}-k8s

‍

In about 15 minutes, you’ll have a functional EKS cluster in Wavelength with OpsCruise’s observability built-in!

Here are a couple of screenshots of what you’ll see once you've deployed your EKS Cluster:

Where We’re Heading Together

As we look to further enhance the OpsCruise platform to natively support Verizon 5G Edge with AWS Wavelength capabilities, there are 4 design principles that will guide its evolution.

• No performance impact: Maintain the low latency characteristics of the application itself by ensuring that control plane traffic, especially for observability, stays out of I/O path
• Non-intrusive: Minimize touch of the customer application by avoiding instrumentation of the application code
• Rapidly fault isolate: Leverage ML-driven anomaly detection and AI-driven root-cause analysis (RCA) to determine bottleneck across mobile edge compute (MEC), cloud infrastructure, or application
• Value-add: Proactively manage cloud-native applications 

As one such example, a customer could directly connect to network intelligence via the Edge Discovery Service (EDS) to monitor carrier-facing endpoints. By actively monitoring load balancers and NodePorts exposed via a Carrier IP address, OpsCruise can proactively ingest the information and dynamically populate the edge service registry. In this way, we can reduce the heavy lifting of manually sifting through Carrier IPs across multiple regions and populating the service registry accordingly.

Get started and give us your feedback

We’re excited to see you get started leveraging Verizon 5G Edge along with OpsCruise’s next-gen observability platform. Feedback drives us, and we look forward to hearing about your experience and how we can enhance it!

Join us at the edge.

Email us at info@opscruise.com.

Follow us on LinkedIn at: https://www.linkedin.com/company/opscruise

Follow Verizon Business on Twitter: https://twitter.com/VerizonBusiness

Contact the authors:

Robert Belson: https://twitter.com/robertbelson

Cesar Quintana: cesar@opscruise.com  |  https://www.linkedin.com/in/qcesarjr/

‍