Elasticsearch, Kibana, and Elastic Cloud 7.16: Integrations and more actionable insights

thumbnail.png

With the release of Elastic 7.16, it’s easier than ever to collect, store, search, and analyze data securely and at scale, no matter the use case. Explore third-party solutions like ServiceNow and cloud-native integrations like Amazon Web Services (AWS) FireLens to make your data actionable and streamline workflows across teams and systems.

In addition, 7.16 improves performance and maximizes resource consumption with enhancements to Elasticsearch, gives users the ability to add more context to Kibana visualizations, and more.

Ready to roll up your sleeves and get started? We have the links you need:


New ways to collect and connect your data with integrations

Go from data to insights with ease

Our growing library of integrations — including native integrations for Microsoft Azure, Google Cloud, and AWS — makes it easy to collect and connect data across applications, infrastructure, public content sources, and more.

In 7.16, a new unified Integrations UI in Kibana makes it even faster to find ways to ingest data from a new source — with just a few clicks for Elastic Agent-enabled integrations. The Integrations UI allows you to search for all integrations supported by Elastic Agent, Beats, Logstash, the Elastic App Search web crawler, Workplace Search content source connectors, and Elasticsearch language clients.

videoImage

Developers, practitioners, and analysts alike can get started from one place and use the ingest method that best suits their needs. And, with even more Elastic Agent-enabled integrations entering general availability, users can spend less time wrestling with custom ingest pipelines and editing long YAML files and more time exploring their data using the Elastic Stack.


Streamline ServiceNow workflows

videoImage

New ServiceNow certified applications for existing IT Service Management (ITSM) and Security Incident Response (SIR) integrations and a new integration for IT Operations Management (ITOM) increase automation for generating application, security, or infrastructure ServiceNow incidents and reduce time and resources spent on manually filing incidents and double-documenting information.

These updates help teams ensure that they are always working off the most up-to-date information in ServiceNow when using Elastic source data to generate and update incidents and cases. Teams can customize alerting rules in Kibana for observability and security use cases to automate follow-up and push case observables to ServiceNow to enrich investigations. And there’s more to come for making workflows even more streamlined as Elastic continues to partner with ServiceNow to deepen these integrations.

Reduce operational complexity with the new native integration for AWS FireLens

videoImage

We are excited to announce a way to easily ingest data into Elastic Cloud with the native integration for AWS FireLens, a container log router for Amazon Elastic Container Service (Amazon ECS) launch types: Amazon Elastic Compute Cloud (Amazon EC2) and AWS Fargate. You can now use ECS and Fargate logs to further improve the observability and security of your AWS workloads without installing and managing standalone data shippers. Ingest data into Elastic Cloud faster, with a simplified data architecture, and reduce operational complexity.

Better performance, lower TCO

With 7.16, we’ve introduced several features that enhance performance, improve resilience, and reduce disk and memory usage — ultimately lowering your total cost of ownership.

Scale to new heights

You can now take advantage of lower heap consumption per field on data nodes. Testing shows that prior to 7.16, 1,000 Beats indices on data nodes required 4GB of heap to store data structure; now, this requires only a few hundred MB. Search speed is also significantly improved — it used to fan out requests to every shard matching the index pattern, but now has been reduced to a single request per node. Lowering heap consumption and optimizing shard requests allows you to further grow with your data and scale your Elasticsearch clusters to new heights.

Performance boosts in Elasticsearch

Event query language (EQL), introduced in Elasticsearch 7.9, is a correlation language that matches events across different time spans for event-based time series data, such as logs, metrics, and traces. In 7.16, we improved EQL performance by removing the use of null values as join keys in sequences, resulting in an impressive increase in performance exceeding 830x!

Another notable performance improvement is in search_after on sorted long fields (including timestamp). We developed and contributed this important capability to Lucene and immediately exposed it in Elasticsearch 7.16. With this change, you benefit from a performance improvement of up to 4x for retrieving results from an index of machine-generated data that is sorted by timestamp and for queries that do not require aggregate information. For a deeper and more complete explanation of this change, check out our blog post on optimizing sort queries in Elasticsearch.

Lastly, the vector tiles API, generally available in 7.16, provides a huge performance and scalability improvement when searching geo_points and geo_shapes drawn to a map (through use of vector tiles). This new standard and type of API does not behave like our other APIs and return JSON — instead, it returns MapBox vector tile specification, which can easily be rendered by any map supporting this format.

The benefits are clear when compared side by side. Offloading these calculations to the local GPU significantly improves performance to provide a smooth and scalable zoom.

We're hiring

Work for a global, distributed team where finding someone like you is just a Zoom meeting away. Flexible work with impact? Development opportunities from the start?