Splunk Enterprise: Architecture, Features, and Capabilities

Splunk Enterprise In-Depth

Splunk Enterprise Architecture and Process

You install Splunk Enterprise on a host as a process called splunkd. This is a distributed server written in C/C++ that can ingest, process, and index large volumes of data, as well as handle search requests. The splunkd architecture uses several processes called pipelines, each made up of several processes which perform different operations on streaming data.

Key concepts in the architecture:

• Pipelines—threads inside the splunkd process. Each pipeline has its own XML configuration.

• Processors—reusable C/C++ functions that perform an operation on the data stream.

• Queues—used to pass data from one pipeline to another.

• Web server—splunkd runs a web server using SSL on port 8089 by default, and another web server on port 8000 without SSL.

Image Source: Splunk

Splunk Enterprise Metrics

Splunk Enterprise enables users to collect, investigate, monitor, and share metrics in real-time. Service engineers, IT staff, and system administrators can use this functionality to track various aspects of the ecosystem, tracking infrastructure, business applications, and security systems.

You can store metrics data by using metric indexes. In Splunk, each index type is optimized to store and retrieve metric data. This functionality lets you run metrics-specific commands on certain metric data points within the associated metric indexes.

For example, you can run the mstats command to apply aggregate functions like average, count, rate, and sum to these data points. This command can help you isolate and correlate issues from various data sources.

Analytics Workspace

Splunk’s Analytics Workspace provides a user interface (UI) that lets users monitor and analyze metrics and other time series without relying on SPL queries. It facilitates rapid identification and responds to anomalies or issues in your data.

Analytics Workspace lets you choose data sources to create interactive charts in your workspace. You can then apply aggregations and filters to gain insight into your metrics and system performance.

Functions, operations, and actions

Splunk’s Analytics Workspace includes a set of analytic operations and functions to help you gain insights from data. Available operations depend on your data source, and all functions generate SPL in the background. Here are common operations:

• Aggregations—can summarize data points into meaningful values.

• Time shifts—can modify the time range of a series.

• Splits—show results for a specific dimension.

• Filters—either exclude or include specific results.

Once you have sufficiently refined your data, you can use the Analytics Workspace to perform certain actions. For example, you can create a dashboard that monitors or shares your findings. You can also configure an alert to notify you of specific behavior in your data.

Monitoring Splunk Enterprise

Splunk Enterprise is a distributed system which requires monitoring to ensure all components are working properly. You can monitor the system via a monitoring console and a rest-based component monitoring tool.

Monitoring Console

The Splunk Enterprise Monitoring Console is a search-based tool that shows the current topology of the Splunk Enterprise system, and performance of individual components. You can use it to monitor anything from simple single-instance deployments to multi-site indexer clusters.

It provides several dashboards that let you give you visibility into:

• Search and indexing performance

• Resource usage

• License usage

Splunk component monitoring API

Splunk Enterprise provides a REST API, accessible at the /server/health/splunkd endpoint, which lets you access the health of Splunk components programmatically. It uses a tree structure that reports the health of individual features, providing a continuous view of deployment health. The same health report is also provided by the Splunk Web interface.

Securing the Splunk Platform

Splunk Enterprise gives you three ways to protect the platform and data from unauthorized access:

• Role-based access control (RBAC) which can limit who can access what in Splunk Enterprise.

• Defining certificates and SSL encryption for configurations, stored data, and data ingestion.

• Defining SSL encryption for both Splunk Web (external communication) and connections between splunkd instances (internal communication).

• Obfuscating login credentials.

• Hardening splunkd instances by ensuring hosts have a secure configuration, properly managing credentials, and configuring encryption.