Customization of the MW agent

Beyond basic installation, this guide explains how to further configure and customize the agent.

Switching between Kubernetes contexts

If you are using a third-party resource for managing your Kubernetes Infrastructure (e.g. Lens), your cluster names may be similar. So, it may be hard for you to identify the exact clusters with the Middleware agent.

To fix this, you must force a specific cluster.name. Set the MW_KUBE_CLUSTER_NAME environment variable to the name(s) you are using in your third party resource. This will rename the clusters, automatically reset the agent, and the agent will collect cluster info given the appropriately labeled variables.

Using an existing secret w/ Helm to manage API Keys

You can use an existing secret from your secret manager with the below snippet, assuming that secret has the Middleware API Key:

mw:
  target: your-initial-uid
  apiKeyFromExistingSecret:
    enabled: true
    name: name-of-your-secret
    key: name-of-your-secret-key

clusterMetadata:
  name: my-cluster

Adding Host Tags

Create filterable custom tags by adding the MW_HOST_TAGS environment variable to the installation command as comma-separated key-value pairs. Use the tag name to create an alias for the host (e.g. production). Run this command:

MW_HOST_TAGS=key1:value1,key2:value2,... MW_API_KEY="<MW_API_KEY>"

Namespace Inclusion/Exclusion (for Auto-Instrumentation Agent)

You can configure namespace inclusion/exclusion when installing the Helm chart:

a. Include specific namespaces:

--set webhook.includedNamespaces="{namespace1,namespace2}"

Only the specified namespaces will be monitored for auto-instrumentation.

b. Exclude specific namespaces:

--set webhook.userExcludedNamespaces="{namespace1,namespace2}"

All namespaces except the specified ones will be monitored (system namespaces are automatically excluded).

Manual configuration for auto-instrumentation

To manually instrument your applications, the OTel Kubernetes Operator needs to know which Kubernetes Pods to instrument and which instrumentation config (Instrumentation Custom Resource (CR)) to use for those Pods.

The default Instrumentation CR installed by the mw-agent is mw-autoinstrumentation docked in the mw-agent-ns namespace. To confirm that the Instrumentation CR is installed, run this command:

kubectl get otelinst mw-autoinstrumentation -n mw-agent-ns

If installed, the response will be an Instrumentation CR that looks like this:

apiVersion: opentelemetry.io/v1alpha1
kind: Instrumentation
metadata:
  name: mw-autoinstrumentation
  namespace: mw-agent-ns
spec:
  exporter:
    endpoint: http://mw-service.mw-agent-ns:9319
  propagators:
    - tracecontext

The Instrumentation CR above will be used to annotate Pod specifications. The annotations depend on the programming language used to implement processes running in the Pod.

Below is the list of supported programming languages and their corresponding annotations.

Java

To enable auto-instrumentation for a Java application, add the following annotation to your Pod specification:

instrumentation.opentelemetry.io/inject-java: "mw-agent-ns/mw-autoinstrumentation"

Node.js

To enable auto-instrumentation for a Node.js application, add the following annotation to your Pod specification:

instrumentation.opentelemetry.io/inject-nodejs: "mw-agent-ns/mw-autoinstrumentation"

Python

To enable auto-instrumentation for a Python application, add the following annotation to your Pod specification:

instrumentation.opentelemetry.io/inject-python: "mw-agent-ns/mw-autoinstrumentation"

.NET

To enable auto-instrumentation for a .NET application and specify the runtime identifier (RID), use the following annotations:

For Linux glibc based images (default):

instrumentation.opentelemetry.io/inject-dotnet: "mw-agent-ns/mw-autoinstrumentation"
instrumentation.opentelemetry.io/otel-dotnet-auto-runtime: "linux-x64"  # Optional as it's the default value

For Linux musl based images:

instrumentation.opentelemetry.io/inject-dotnet: "mw-agent-ns/mw-autoinstrumentation"
instrumentation.opentelemetry.io/otel-dotnet-auto-runtime: "linux-musl-x64"

Go

To enable auto-instrumentation for a Go application, you need to set the path to the executable and ensure the container has elevated permissions. Use the following annotations in the Pod specifications:

instrumentation.opentelemetry.io/inject-go: "mw-agent-ns/mw-autoinstrumentation"
instrumentation.opentelemetry.io/otel-go-auto-target-exe: "/path/to/container/executable"

Additionally, set the required security context for the container in the Pod specification (spec.containers) as shown below:

securityContext:
  privileged: true
  runAsUser: 0

Resource Attributes

If you want to add resource attributes to the traces generated, you can add annotations in the following format to your Pod specification.

resource.opentelemetry.io/your-key: "your-value"

your-key and "your-value" are the keys and values for the resource attributes you want to add. You can add multiple resource attributes by having multiple such annotations.

Advanced Configuration

The annotations described above for all the supported programming languages used mw-agent-ns/mw-autoinstrumentation as the value for the annotation. This is because the OTel Kubernetes Operator installed using Middleware installation instructions creates mw-autoinstrumentation Instrumentation CR in mw-agent-ns namespace.

You can also create your own Instrumentation CR in any namespace and use them in the auto-instrumentation annotations.

Below are all the possible values for the auto-instrumentation annotation:

• "true": Inject an Instrumentation CR from the current namespace. It is expected that you will have only one Instrumentation CR defined in the current namespace. The behavior could be unpredictable if you have multiple Instrumentation CRs defined in the current namespace.
• "my-instrumentation": Inject a specific Instrumentation CR with name my-instrumentation from the current namespace.
• "my-other-namespace/my-instrumentation": Inject a specific Instrumentation CR with name my-instrumentation from namespace my-other-namespace. This is the option used in all our examples above with Instrumentation CR name “mw-otel-auto-instrumentation” and namespace mw-agent.
• "false": Do not inject an Instrumentation CR. This is useful for temporarily stopping instrumentation.