Elastic takes the first steps toward building out its SIEM solution
Big Data
The highlight of Elastic NV's latest update to the Elastic Stack is the introduction of a core data model and user interface for Security Information and Event Management (SIEM). As the company has been drawing competition for its core open-source search engine, Elastic has sought to redirect the conversation by developing applications shaped around common use cases for its log, search, and analytics stack. Last fall saw the bulking up of its APM solution with new capabilities for correlating application performance with infrastructure logs, server metrics, and security events.
The 7.2 release is initially only available through the Elastic public cloud-managed service, Elasticsearch Service, and for download with a new dedicated SIEM app (in beta) in Kibana. The SIEM features lay the foundations for a more fleshed-out solution going forward with the new Elastic Common Schema, an open-source specification for field naming conventions and data types; think of the new common schema as a Rosetta Stone for the different types of logs, metrics, and other contextual data that is used for analyzing security events. Additionally, the 7.2 release adds a dedicated user interface for security events, featuring a timeline viewer to store evidence of an attack, pin and annotate relevant events, and provide query filtering capabilities.
The 7.2 release also adds integration with Cisco and Palo Alto firewalls, which come atop existing support for a collection of host-based security data with Auditbeat, which collects data from the Linux OS audit framework, and Winlogbeat, which performs similar tasks with Windows.
Elastic views the primary use cases falling under the buckets of threat detection workflows, where security professionals want to drill down to determine the urgency of different alerts, or for proactive approaches that check potential cyber threat scenarios. While the SIEM features of 7.2 are still skeletal, watch this space. The ink has not yet dried on last month's announced acquisition of Endgame, which provides endpoint security, so it was not part of the 7.2 announcement. The obvious next question is when, not whether, machine learning support will materialize. Today, you can incorporate unsupervised learning into searching that can be managed using the Kibana visualization piece, but nothing that will be specific to SIEM, yet.
The 7.2 release also extends applications search, previously only available from the Elastic cloud, now to on-premise deployments. In effect, it allows you to add search capability within your application using capabilities adapted from Swifttype, a codeless capability that Elastic acquired that enables you to add a search box and search indexes to your website.
On the cloud-native front, they have launched a Kubernetes-based option that allows adventurous customers who want to build their own SaaS-like environments. The Kubernetes operator capitalizes on previous releases of Docker images for Elasticsearch and Kibana, modifying Beats to collect logs from Kubernetes pods, and supporting Helm charts.
Another addition in the 7.2 release is beefing up APM capabilities, such as introducing the .NET agent into beta, adding support for single-page applications with the browser-based Real User Monitoring (RUM) feature, and a new Metrics Explorer to make it easier to visualize key infrastructure metrics. This is part and parcel of Elastic's embrace of "observability" that is supposed to unify disparate elements relevant to APM such as logs, traces, and metrics. The guiding notion is that a scalable search-based approach where everything is indexed should provide intuitive ways to piece together the events that are relevant to what you are seeking to monitor, detect, or troubleshoot, such as security or application performance.
It's not surprising that incumbents like AppDynamics counter that approaches taking log-based views are over-simplistic, as they still require solution providers and developers to respectively sweat the details for designing agents for every language, and then developing the logic to "stitch" together with all relevant entities or events involved with transaction IDs. And we'll expect technology providers specializing in SIEM countering with a similar rationale that context and traceability that search indexes alone won't generate.
As AWS ratchets up competition trying to out-open source Elastic on the core search engine that put it on the map, Elastic's obvious path is doubling down on the use cases where its search-based solutions have drawn the most traction. The draw is an open-source core, built for scale and the flexibility that a search-based approach vs. the deep functionality that incumbents in the APM and SIEM spaces have long competed on. In the long run, we expect a lot of cross-fertilization as Elastic not only creates more agents but also simplifies putting together the tracing, while incumbents borrow on the versatility of search to make their solutions more extensible.