Elasticsearch

The Elasticsearch Integration allows you to track your performance and health metrics across all clusters and nodes. This integration is useful for visualizing disk read/write operations on a per node basis, system CPU usage, and total available physical and virtual memory.

Prerequisites

1 MW Host Agent

Middleware Host Agent (MW Agent) must be installed on your local machine. To install the MW Agent, see our Installation Guide.

2 Elastic Superuser

Must be an Elastic superuser as defined here.

Setup

Step 1: Create Database Credentials

Create a .yaml file containing database credentials like the below example. If the database is not password protected, the username and password fields can be removed.

/home/ubuntu/elasticsearch-creds.yaml

elasticsearch:
  endpoint: http://localhost:9200
  username: YOUR_USERNAME
  password: YOUR_PASSWORD

If you are running the Middleware Docker Agent, keep the YAML file within the /var/log directory. Docker Agents might not support dynamic volume binding.

Step 2: Access Integrations

Log in to Middleware, navigate to the Installations Page in the bottom left corner, select All Integration and click MySQL

Step 3: Enable Integration

Add a host machine from the dropdown list, input the credential path from Step 1, and Save

Visualize Your Data

Default Elasticsearch Dashboard

Quickly access your Elasticsearch data with Middleware's default Elasticsearch dashboard. Navigate to the Dashboard Builder and select the Elasticsearch - Metrics (Runtime) Dashboard

Create Elasticsearch Widgets

Create your own widget from scratch. Navigate to the Dashboard Builder and select the dashboard you would like to create a widget in. Select Add New Widget and choose the elasticsearch data source.

Learn more about creating your own widgets and dashboards in the Dashboard Builder section.

Alerts

Alerts can be configured for any Elasticsearch metrics. When creating a new rule select the Database detection method and Elasticsearch database type for available metrics to appear in the Metrics dropdown list. Select the desired metric and continue configuring the alert conditions.

Metrics Collected

Metric	Description
`elasticsearch.breaker.memory.estimated`	Estimated memory used for the operation
`elasticsearch.breaker.memory.limit`	Memory limit for circuit breaker
`elasticsearch.breaker.tripped`	Total number of times the circuit breaker has been triggered and prevented an out of memory error
`elasticsearch.node.cache.memory.usage`	Size in bytes of the cache on a node
`elasticsearch.node.cache.evictions`	Number of evictions from the cache on a node
`elasticsearch.node.cache.count`	Total count of query cache misses across all shards assigned to selected nodes
`elasticsearch.node.fs.disk.available`	Amount of disk space available to the JVM across all file stores for this node
`elasticsearch.node.fs.disk.free`	Amount of unallocated disk space across all file stores for this node
`elasticsearch.node.fs.disk.total`	Amount of disk space across all file stores for this node
`elasticsearch.node.disk.io.read`	Total number of kilobytes read across all file stores for this node
`elasticsearch.node.disk.io.write`	Total number of kilobytes written across all file stores for this node
`elasticsearch.node.cluster.io`	Number of bytes sent and received on the network for internal cluster communication
`elasticsearch.node.cluster.connections`	Number of open tcp connections for internal cluster communication
`elasticsearch.node.http.connections`	Number of HTTP connections to the node
`elasticsearch.node.operations.completed`	Number of operations completed by a node
`elasticsearch.node.operations.time`	Time spent on operations by a node
`elasticsearch.node.shards.size`	Size of shards assigned to this node
`elasticsearch.node.shards.data_set.size`	Total data set size of all shards assigned to the node
`elasticsearch.node.shards.reserved.size`	Prediction of how much larger the shard stores on this node will eventually grow
`elasticsearch.node.translog.operations`	Number of transaction log operations
`elasticsearch.node.translog.size`	Size of the transaction log
`elasticsearch.node.translog.uncommitted.size`	Size of uncommitted transaction log operations
`elasticsearch.node.thread_pool.threads`	Number of threads in the thread pool
`elasticsearch.node.thread_pool.tasks.queued`	Number of queued tasks in the thread pool
`elasticsearch.node.thread_pool.tasks.finished`	Number of tasks finished by the thread pool
`elasticsearch.node.documents`	Number of documents on the node
`elasticsearch.node.open_files`	Number of open file descriptors held by the node
`jvm.classes.loaded`	Number of loaded classes
`jvm.gc.collections.count`	Total number of garbage collections that have occurred
`jvm.gc.collections.elapsed`	Approximate accumulated collection elapsed time
`jvm.memory.heap.max`	Maximum amount of memory can be used for the heap
`jvm.memory.heap.used`	Current heap memory usage
`jvm.memory.heap.committed`	Amount of memory that is guaranteed to be available for the heap
`jvm.memory.nonheap.used`	Current non-heap memory usage
`jvm.memory.nonheap.committed`	Amount of memory that is guaranteed to be available for non-heap purposes
`jvm.memory.pool.max`	Maximum amount of memory can be used for the memory pool
`jvm.memory.pool.used`	Current memory pool memory usage
`jvm.threads.count`	Current number of threads
`elasticsearch.cluster.pending_tasks`	Number of cluster-level changes that have not yet been executed
`elasticsearch.cluster.in_flight_fetch`	Number of unfinished fetches
`elasticsearch.cluster.shards`	Number of shards in the cluster
`elasticsearch.cluster.data_nodes`	Number of data nodes in the cluster
`elasticsearch.cluster.nodes`	Total number of nodes in the cluster
`elasticsearch.cluster.health`	The health status of the cluster
`elasticsearch.os.cpu.usage`	Recent CPU usage for the whole system, or -1 if not supported
`elasticsearch.os.cpu.load_avg.1m`	One-minute load average on the system (field is not present if one-minute load average is not available)
`elasticsearch.os.cpu.load_avg.5m`	Five-minute load average on the system (field is not present if five-minute load average is not available)
`elasticsearch.os.cpu.load_avg.15m`	Fifteen-minute load average on the system (field is not present if fifteen-minute load average is not available)
`elasticsearch.os.memory`	Amount of physical memory
`elasticsearch.memory.indexing_pressure`	Memory consumed, in bytes, by indexing requests in the specified stage
`elasticsearch.indexing_pressure.memory.total.primary_rejections`	Cumulative number of indexing requests rejected in the primary stage
`elasticsearch.indexing_pressure.memory.total.replica_rejections`	Number of indexing requests rejected in the replica stage
`elasticsearch.indexing_pressure.memory.limit`	Configured memory limit, in bytes, for the indexing requests
`elasticsearch.cluster.state_queue`	Number of cluster states in queue
`elasticsearch.cluster.published_states.full`	Number of published cluster states
`elasticsearch.cluster.published_states.differences`	Number of differences between published cluster states
`elasticsearch.cluster.state_update.count`	Number of cluster state update attempts that changed the cluster state since the node started
`elasticsearch.cluster.state_update.time`	Cumulative amount of time updating the cluster state since the node started
`elasticsearch.cluster.indices.cache.evictions`	Number of evictions from the cache for indices in the cluster
`elasticsearch.node.ingest.documents`	Total documents ingested during the lifetime of this node
`elasticsearch.node.ingest.documents.current`	Total documents currently being ingested
`elasticsearch.node.ingest.operations.failed`	Total failed ingest operations during the lifetime of this node
`elasticsearch.node.pipeline.ingest.documents.preprocessed`	Number of documents preprocessed by the ingest pipeline
`elasticsearch.node.pipeline.ingest.operations.failed`	Total failed operations for the ingest pipeline
`elasticsearch.node.pipeline.ingest.documents.current`	Total documents currently being ingested by a pipeline
`elasticsearch.node.script.compilations`	Total inline script compilations performed by the node
`elasticsearch.node.script.cache_evictions`	Total times the script cache has evicted old data
`elasticsearch.node.script.compilation_limit_triggered`	Total times the script compilation circuit breaker has limited inline script compilations
`elasticsearch.node.segments.memory`	Size of memory for segment object of a node
`elasticsearch.index.operations.completed`	Number of operations completed for an index
`elasticsearch.index.operations.time`	Time spent on operations for an index
`elasticsearch.index.shards.size`	Size of the shards assigned to this index
`elasticsearch.process.cpu.usage`	CPU usage in percent
`elasticsearch.node.operations.current`	Number of query operations currently running
`elasticsearch.index.documents`	Number of documents for an index
`elasticsearch.process.memory.virtual`	Size of virtual memory that is guaranteed to be available to the running process
`elasticsearch.process.cpu.time`	CPU time used by the process on which the Java virtual machine is running

Troubleshooting

If you do not see the Integrations Menu in Middlware, that means your account has not been granted Installation permissions. Contact your system administrator to add the Installation permission to your user role in Settings.

Next Steps

Need assistance or want to learn more about Middleware? Contact our support team in Slack.