<p>If you’re in the technology space, the phrase “server breach” has probably hit your radar at some point. If you’re not in tech, you’re likely still aware of server/data breaches and security lapses that have resulted in private, personal data ending up in the hands of bad actors. This can mean credit information, social security numbers, and other information you’d prefer to keep to yourself&nbsp;— and not on the darkweb.
</p><p>All software has to deal with the possibility of a breach or attack. This includes Elasticsearch, an open source software project developed by Elastic. There have been instances in which data that was not securely stored in Elasticsearch was leaked, lost, or stolen.
</p><h2>How to prevent an Elasticsearch server breach</h2><p>This article will give you an understanding of how breaches come about and how users can best protect against them in the context of Elasticsearch. We’re going to start with a bit of a primer, but if you want, you can jump straight to the section:&nbsp;<a href="#secure-elasticsearch">How do I secure Elasticsearch?</a></p><h2>What is Elasticsearch?</h2><p>Elasticsearch is an open source search and analytics engine, as well as a data store. And with hundreds of millions of downloads, it’s also incredibly popular. We tout its speed, scale, and search relevance, but its wide adoption has also been greatly driven by its ease of use, ability to handle any type of data (text, number, geospatial, etc.), robust set of features, and the fact that anyone can submit a bug fix or suggest an improvement (since it’s <a href="/about/why-open-source">open source</a>).
</p><p>In non-tech speak, imagine having a spreadsheet with billions of rows, but still being able to find or calculate anything in it in a split second. Elasticsearch is like that, and it’s really good at what it does, so <a href="/customers">a lot of organizations use Elasticsearch</a> for all sorts of things — like <a href="/customers/netshoes">powering search on their ecommerce sites</a>, <a href="/elasticon/2015/sf/unlocking-interplanetary-datasets-with-real-time-search">analyzing data sent from Mars rovers</a>, or <a href="/elasticon/conf/2018/sf/lyfts-wild-ride-from-amazon-es-to-self-managed-elasticsearch">making sure rideshares shows up on time</a>.
</p><h2>Who is Elastic? Also, do you have my data?</h2><p>Elastic is the company that develops Elasticsearch, along with the other products of the Elastic Stack (Kibana, Beats, Logstash, etc.). And Elasticsearch is the backbone for a variety of solutions including <a href="/enterprise-search">search</a>, <a href="/observability">observability</a>, <a href="/security">security</a>, and more.
</p><p>And the short answer is, no, we do not have your data. We develop Elasticsearch, which other companies then run locally or within cloud infrastructure.&nbsp;
</p><p>The slightly longer answer is, maybe. Some companies use <a href="/cloud">Elastic Cloud</a>, our managed Elasticsearch offering. We are responsible for maintaining any data our customers put in Elastic Cloud, which includes keeping it secure. This is why all of our <a href="/elasticsearch/service">Elasticsearch Service deployments</a> come <a href="/cloud/security">secure by default</a>.
</p><h2>Where does Elasticsearch hold its data?</h2><p>Since Elasticsearch is open source (meaning anyone can download and install it for free), it can be installed almost anywhere. Some companies download it and install it on their own in-house servers and other companies download and install it in the cloud (on any provider they want). There are also some cloud service companies that offer managed versions of Elasticsearch, which means they host and administer Elasticsearch clusters (that’s what we call them) for customers.&nbsp;
</p><p>In fact, Elastic also offers our own managed <a href="/elasticsearch/service">Elasticsearch Service</a> on Elastic Cloud. With our service, we provide extensive management services (including security by default), and let you choose from a few different cloud providers for the infrastructure.
</p><h2>How do Elasticsearch servers get breached?</h2><p>There are a variety of ways data stores can be breached, everything from stolen passwords, to hackers, to disgruntled employees. In the case of Elasticsearch, the most common type of breach is caused by a cluster to be left unsecured on the internet, meaning anyone can connect without needing a username or password and access the data. These open clusters are often discovered by security researchers who then publish their findings which is how the world finds out about the data being left on the internet.
</p><p>When properly secured, an Elasticsearch cluster is at no more risk of being breached than any other database. And it’s for this reason that we provide the features (free and by default) needed to <a href="/what-is/elastic-stack-security">secure an Elasticsearch cluster</a> by anyone who downloads and uses the Elastic Stack. To that end, let’s look at some things Elasticsearch users can do to make sure they’re using these features and keeping their clusters secure.
</p><h2 id="secure-elasticsearch">How do I secure Elasticsearch?</h2><p>We include <a href="/blog/security-for-elasticsearch-is-now-free">Elasticsearch security features</a> as part of our free, default (<a href="/subscriptions">Basic</a> tier) distribution, meaning anyone can configure security for their Elasticsearch cluster and prevent accidental data leaks. Before looking into using those features, though, let’s first take a look at how to determine if your cluster is already secure or not.
</p><h3>How can I tell if I have an unsecured Elasticsearch cluster?</h3><p>If you are a user or administrator of Elasticsearch, there are a few simple things you can do to check if you have security enabled and working.&nbsp;
</p><p>The best way to determine if you have security enabled and working is to query the settings API. If you issue this query:
</p><pre class="prettyprint">GET _xpack/usage?filter_path=security
</pre><p>You will see results that look something like this if security is disabled:
</p><pre class="prettyprint">{ 
  "security" : { 
    "available" : true, 
    "enabled" : false, // When security is off, this is set to "false" 
    "ssl" : { 
      "http" : { 
        "enabled" : false 
      }, 
      "transport" : { 
        "enabled" : false 
      } 
    } 
  } 
}
</pre><p>Or this if security is working as expected
</p><pre class="prettyprint">{ 
  "security" : { 
    "available" : true, 
    "enabled" : true, // When secure is on, this is set to "true"
    "realms" : { ... }, 
    "roles" : { ... }, 
    "role_mapping" : { ... }, 
    "ssl" : { 
      "http" : { 
        "enabled" : true 
      }, 
      "transport" : { 
        "enabled" : true 
      } 
  }, 
  "token_service" : { ... }, 
  "api_key_service" : { ... }, 
  "audit" : { ... }, 
  "ipfilter" : { ... }, 
  "anonymous" : { ... } 
  } 
}
</pre><p>You can see the results are pretty different. Depending on the version and license level you have deployed you can expect your results to be slightly different as well.
</p><h3>How do I make sure my Elasticsearch cluster is secure?</h3><p>The two easiest things you can to make sure that outsiders don’t have access to your Elasticsearch clusters are:
</p><ol>
	<li><strong>Enable authentication</strong> so no one has access without logging into Elasticsearch.</li>
	<li><strong>Enable TLS</strong> so no one can eavesdrop on Elasticsearch data traveling through your network.</li>
</ol><p>We have a lot of great resources available (listed at the bottom) that explain how to secure Elasticsearch, including how to easily implement the above two recommendations. But the quickest way to get secure is to check out our <a href="/blog/getting-started-with-elasticsearch-security">Getting started with Elasticsearch security (TLS and RBAC)</a> blog and its accompanying video — <em>Securing Elasticsearch in 7 minutes</em>
</p><div class="video embed-container" style="height: 319.725px;">
	<img class="vidyard-player-embed" src="https://play.vidyard.com/aiXTSUt4aR3hfesxsFynaw.jpg" data-uuid="aiXTSUt4aR3hfesxsFynaw" data-v="4" data-type="inline" style="width: 100%; margin: auto; display: block;">
</div><p>We also have an <em>easy button</em> for all of this in our <a href="/elasticsearch/service">Elasticsearch Service</a> on Elastic Cloud. When you use our Elasticsearch Service, you can be certain that it comes with security automatically configured for every cluster.
</p><h2>Security takes teamwork</h2><p>If you have any questions you can always reach out. We have a number of ways to be a part of the <a href="/community/">Elastic Community</a> such as <a href="https://discuss.elastic.co">forums</a>, local events, and <a href="/guide">lots of documentation</a> (so much more than linked above). We’re also always working to make securing Elasticsearch easier than ever before. Be sure to keep an eye on our blogs to know what’s coming and what we’re doing in this space in the future. If you’re a fan of Spiderman, you’re familiar with the quote, “with great power comes great responsibility”, and this is true for Elastic and anyone using Elasticsearch to store data.
</p><h2>Additional resources for securing Elasticsearch</h2><p><strong>Blogs</strong>
</p><ul>
	<li><a href="/blog/getting-started-with-elasticsearch-security">Getting started with Elasticsearch security (TLS and RBAC)</a></li>
	<li><a href="/blog/tips-to-secure-elasticsearch-clusters-for-free-with-encryption-users-and-more">Tips to secure Elasticsearch clusters for free with encryption, users, and more</a></li>
	<li><a href="/blog/configuring-ssl-tls-and-https-to-secure-elasticsearch-kibana-beats-and-logstash">Configuring SSL, TLS, and HTTPS to secure Elasticsearch, Kibana, Beats, and Logstash</a></li>
</ul><p><strong>Videos</strong>
</p><ul>
	<li><a href="/webinars/elasticsearch-security-best-practices-to-keep-your-data-safe">Elasticsearch security: Best practices to keep your data safe</a></li>
	<li><a href="/webinars/kibana-security-access-management-spaces-and-feature-controls">Kibana security: Access management, spaces, and feature controls</a></li>
</ul><p><strong>Documentation</strong>
</p><ul>
	<li><a href="/guide/en/elasticsearch/reference/7.6/elasticsearch-security.html">Security settings in Elasticsearch</a></li>
	<li><a href="/guide/en/elasticsearch/reference/7.6/configuring-security.html">Configuring security in Elasticsearch</a></li>
</ul><p><strong>Training</strong>
</p><ul>
	<li><a href="/training/specializations/elastic-stack-management/fundamentals-of-securing-elasticsearch">Fundamentals of Securing Elasticsearch</a></li>
	<li><a href="/training/specializations/elastic-stack-management/advanced-techniques-for-securing-elasticsearch">Advanced Techniques for Securing Elasticsearch</a></li>
</ul>

Securing Elasticsearch: How to prevent an Elasticsearch server breach

How to prevent an Elasticsearch server breach

Elasticsearch security is now free. This post revisits best practices for securing your clusters, including TLS, native and file realm authentication, and more.

<table style="background: #FFFFD2;">
<tbody>
<tr>
	<td><strong>Editor's Note:</strong> For more information on keeping your data safe, check out our  <a href="/blog/how-to-prevent-elasticsearch-server-breach-securing-elasticsearch">Securing Elasticsearch: How to prevent an Elasticsearch server breach</a> blog post.
	</td>
</tr>
</tbody>
</table><p>With the release of the Elastic Stack 6.8 and 7.1, we’ve made a number of Elasticsearch security features more widely available to everyone by making them <a href="https://www.elastic.co/blog/security-for-elasticsearch-is-now-free">free in the default distribution</a>. We’re really excited to share this, but one of the new challenges now is to update all of our documentation and guidance on how to secure the Elastic Stack. Gone are the days of relying on just a proxy server in front of the Stack to secure it!
</p><p>I’ve gone through most of our old popular security-focused blog posts, and I’m going to update the guidance, dispel some myths, point out some new ideas, and make sure you can secure your Stack better than ever before.
</p><p>I’m not going to be linking to any of the old articles; some give out advice that we now consider dangerous due to how the Elastic Stack has changed over the years. (Be wary of &nbsp;security advice from old blogs, things change pretty quickly.) We will be forwarding all those links to this page, but if you come upon another site serving cached versions of those articles, forward them a link to this one.
</p><h2>TLS encryption
</h2><p>TLS encryption is now part of our free Elasticsearch and Kibana security within the default distribution. The purpose of TLS is to encrypt network communications. We encrypt the communication between the nodes in your cluster, between the clients and the cluster, as well as connections from Kibana to the users.
</p><p>We also use TLS certificates as a way to ensure a node should be allowed to join a cluster. Without TLS, it’s possible for anyone to add a node to a cluster. Although this could be abused by an attacker for a variety of nefarious purposes, the far more common case is when a node accidentally joins the wrong cluster. TLS can help avoid these problems because without the proper certificate a node cannot join the cluster.
</p><p>Prior to 6.8 and 7.1, if you needed TLS for your Elastic Stack while using a <a href="https://elastic.co/subscriptions">Basic</a> license you would have to rely on some sort of proxy, which adds a considerable amount of setup. TLS is already pretty difficult to configure even without having to set up a proxy between the nodes and the clients. By adding a proxy, you would have taken a hard problem and made it substantially harder. We can now leverage the TLS support directly in the Elastic Stack.
</p><p>We have quite a bit of <a href="https://www.elastic.co/guide/en/elastic-stack-overview/7.1/ssl-tls.html">documentation on how to configure TLS</a>. If you’ve worked with TLS, you know it’s never easy to set up certificates, but we have built-in tools to help generate <a href="https://www.elastic.co/guide/en/elasticsearch/reference/7.1/configuring-tls.html#node-certificates">cluster-wide certificates and certificate authorities</a> to make things easier. &nbsp;Watch this space for some future blog posts that go into detail about how to best configure TLS in all of the Elastic Stack components. The two easiest ways to get started today? Check out this <a href="https://www.elastic.co/blog/getting-started-with-elasticsearch-security">Getting started with Elasticsearch security</a> blog post, or take a look at our official <a href="https://www.elastic.co/blog/introducing-elastic-cloud-on-kubernetes-the-elasticsearch-operator-and-beyond">Kubernetes Operator</a>.
</p><h2>Authentication
</h2><p>With versions 6.8 and 7.1, native authentication is available to all. The purpose of authentication is to allow clients to identify themselves to the Elastic Stack. This is often known as the username and password you use to log in to a system. It’s generally a bad idea to leave a cluster full of data open to the whole world. It’s especially dangerous if the cluster is connected directly to the Internet where anyone can connect without using a password.
</p><p>In the past, the advice was to use an Nginx server with basic auth enabled between the client and the cluster. Getting this configuration correct could be incredibly difficult. Since every node can service external requests, if you forgot to firewall off a node, anyone could connect to the cluster without authentication.
</p><p>We allow authentication using the <a href="https://www.elastic.co/guide/en/elastic-stack-overview/7.1/native-realm.html">native</a> and <a href="https://www.elastic.co/guide/en/elastic-stack-overview/7.1/file-realm.html">file</a> realms for free now. The file realm stores user information in a file on every node. You have to keep the files synced on all the nodes. The native realm is a more streamlined experience and does about what one would expect: The user data is stored in an index on the cluster every node has access to. You add usernames and passwords via a REST API, and then those accounts can log in. See the <a href="https://www.elastic.co/blog/getting-started-with-elasticsearch-security">Getting started with Elasticsearch security</a> blog for more details.
</p><p>If you have more complex authentication needs, such as LDAP, Active Directory, SAML, or something else, <a href="https://www.elastic.co/cloud/">we offer a Cloud (SaaS) service</a> with some of these as well as other commercial features baked right in, or you can <a href="https://www.elastic.co/subscriptions">get in touch</a> with us to discuss possible on-prem solutions. To see the different authentication options we offer, be sure to check out the <a href="https://www.elastic.co/guide/en/elastic-stack-overview/7.1/setting-up-authentication.html">authentication docs</a>.
</p><h2>Authorization
</h2><p>Authorization and authentication tend to go hand in hand; the Elastic Stack is no different. We have a very flexible system that can define authorization rules for the cluster and indices, all the way down to <a href="https://www.elastic.co/guide/en/elastic-stack-overview/7.1/document-level-security.html">documents</a> and <a href="https://www.elastic.co/guide/en/elastic-stack-overview/7.1/field-level-security.html">fields</a>. We even have something called an <a href="https://www.elastic.co/guide/en/elastic-stack-overview/7.1/custom-roles-authorization.html">authorization engine</a> where you can build your own authorization plugin.
</p><p>Historically, advice has been pretty rough in this space. There are some ways that have been suggested in the past to write rather complex filtering rules with a proxy like Nginx, and then trying to leverage filtered aliases to achieve something that almost looks like authorization. We strongly suggest everyone avoids using these methods for authorization. They are <strong>not</strong> security features, probably won’t do what you want, and are incredibly fragile to maintain. These solutions won’t do what you really want anymore because over time the number of Elasticsearch APIs has grown substantially. There are many ways to access and query indices and the data they contain. There’s no way to ensure you’ve blocked or restricted every single request. The second reason is related to the first, because there are so many endpoints to protect and we’re always adding new ones. It’s going to be a massive amount of work to keep those filters updated.
</p><p>Luckily, you don’t have to worry about this anymore. In 6.8 and 7.1, you have access to a substantial number of authorization features. There are too many to describe here in this post, but the <a href="https://www.elastic.co/guide/en/elastic-stack-overview/7.1/security-privileges.html">Elastic Stack security docs</a> do a great job of explaining all of the existing privileges. We will spend time in future blog posts to explain some of the solutions that can be built using our permission model.
</p><p>Like authentication, we have more advanced authorization features that can be applied to individual documents and fields. You can build your own authentication and authorization plugins, even log the security events on the cluster. You can make use of these <a href="https://www.elastic.co/cloud/elasticsearch-service/subscriptions">as part of our Cloud service</a> or <a href="https://www.elastic.co/subscriptions">get in touch</a> with us to discuss your on-prem options.
</p><h2>Put your Stack behind a VPN and/or firewall
</h2><p>There has been a fair amount of past advice suggesting you place the Elastic Stack behind a firewall or VPN. The purpose behind much of this advice was because without TLS and authentication it could be difficult to protect a cluster using only a proxy, so just blocking access to everyone was easier than trying to get all the moving parts properly configured.
</p><p>If this is something you still want to do, even with TLS and authentication enabled, it can still be a great idea. Security works best when there are multiple layers. The one nice part of this now is the VPN and firewall aren’t the only layer.
</p><h2>Restricted scripts
</h2><p>We once long ago had a blog post about restricting scripting in Elasticsearch. With the creation of the <a href="https://www.elastic.co/guide/en/elasticsearch/reference/7.1/modules-scripting-painless.html">Painless</a> scripting language it’s much harder to bring down a cluster, even with scripts with ill intent. It’s not impossible, though, and we have some best practices in the <a href="https://www.elastic.co/guide/en/elasticsearch/reference/7.1/modules-scripting-security.html">scripting security docs</a> that discuss our current stance on this topic. Once again though, you should always review your environment and take the steps that make the most sense to you. Security works best in layers, so there isn’t one single best option.
</p><h2>Isolation
</h2><p>There’s been a number of times that isolating your cluster and nodes for security reasons has been mentioned. This can be everything from containers to firewalls to IP filtering.
</p><p>This is still great advice and isolating everything you can is a great security tip. In the last few years, we’ve done a lot to make this easier than ever before. You can use our official <a href="https://www.docker.elastic.co/">container</a> images published on Docker Hub. You can leverage the <a href="https://www.elastic.co/cloud/elasticsearch-service">Elasticsearch Service,</a> and we’ll take care of all the heavy lifting. And we just announced an official <a href="https://www.elastic.co/blog/introducing-elastic-cloud-on-kubernetes-the-elasticsearch-operator-and-beyond">Kubernetes Operator</a> that you can take for a spin.
</p><h2>Back up your indices (especially .kibana because it can be easily changed)
</h2><p>Backing up your data is still just as important as it’s ever been. Security doesn’t change that in any way, but we do get to teach the old dog some new tricks.
</p><p>The biggest advantage is we can use <a href="https://www.elastic.co/guide/en/kibana/7.1/xpack-spaces.html">Kibana Spaces</a> to restrict who can modify certain dashboards. This, too, has been released for free as part of 6.8 and 7.1. In the past, if you could access Kibana you could probably change anything you wanted. It would not be uncommon for someone to mess up the dashboards or in some cases even accidentally delete them all. The advice was to back up your .kibana index on a regular basis so you could recover from mistakes quickly. Restoring an index is a lot faster than rebuilding all of your dashboards. Backing up .kibana is still important, but you shouldn’t need to restore it on a regular basis thanks to spaces and authorization.
</p><p>Another great advantage to security is the ability to control who can take snapshots, create an index, and modify the data. Without security, a user who should only be able to read could make catastrophic changes to the data. With security enabled, we can take some steps to hopefully prevent accidental changes to important data.
</p><h2>What now?
</h2><p>The purpose behind this post was to revisit some of our previous advice on how to secure your Elastic Stack deployments. In the world of security, the only constant is change. Even the advice in this blog will be old and wrong someday. Security in the Elastic Stack can be daunting due to the number of possible options. (That’s a feature, not a bug.) Be sure to <a href="https://discuss.elastic.co">ask questions in the forums</a>, and read our blogs and <a href="https://www.elastic.co/guide">documentation</a>.
</p><p>If you’d prefer an easy button for securing your Elastic Stack, we have a few options. Our <a href="https://www.elastic.co/cloud/elasticsearch-service/subscriptions">Elasticsearch Service</a> comes with TLS configured for all clusters, authentication and authorization enabled, as well as our <a href="https://www.elastic.co/subscriptions">Enterprise</a> security features enabled by default. If cloud isn’t your thing, you can get many of the same features in the Elasticsearch Service in your own datacenter with <a href="https://www.elastic.co/ece">Elastic Cloud Enterprise (ECE)</a>. We also have a <a href="https://www.elastic.co/blog/introducing-elastic-cloud-on-kubernetes-the-elasticsearch-operator-and-beyond">Kubernetes Operator</a> that can take care of some of the heavy lifting, such as enabling TLS and configuring authentication for you by default. And of course if you’re an <a href="https://www.elastic.co/subscriptions">Enterprise</a> customer running your own Stack, you can use our fantastic support team to help you sort out any problems that may arise.
</p><p>Stay tuned for everything we have in store in the future. We have a lot of How Tos planned and even plenty of new security features you’ll see in the future. It’s a very exciting time to be using the Elastic Stack!
</p>

blog-banner-generic-teal.png

blog-thumb-generic-teal.png

Tips to secure Elasticsearch clusters for free with encryption, users, and more

<table style="background-image: initial;background-position-x: initial;background-position-y: initial;background-size: initial;background-repeat-x: initial;background-repeat-y: initial;background-attachment: initial;background-origin: initial;background-clip: initial;background-color: rgb(255, 255, 210);"><tbody><tr><td><strong>UPDATE:</strong> This blog references outdated steps for configuring Elasticsearch security. For the the most current information, refer to our technical documentation for <a href="https://www.elastic.co/guide/en/elasticsearch/reference/current/configuring-stack-security.html" target="_self">configuring security for the Elastic Stack</a>.<br/>	</td></tr></tbody></table><p><a href="https://www.elastic.co/blog/security-for-elasticsearch-is-now-free" target="_self">Starting with Elastic Stack 6.8 and 7.1</a>, security features like TLS encrypted communication, role-based access control (RBAC), and more are available for free within the default distribution. In this blog post, we’re going to cover how to get started with using these features to secure your Elasticsearch clusters.<br/></p><p>As a practical example of securing an Elastic Stack implementation, we’re going to create and secure a 2-node Elasticsearch cluster on a local machine. To do so, we’ll first configure TLS communication between the two nodes. Then we'll enable security for the&nbsp;Kibana instance being used to analyze and visualize the data in the cluster. After that, we’ll configure role-based access control within Kibana to make sure that users can only see data they’re authorized to see. </p><p>While there’s a lot more to how security works, for now, we’re just going to cover what you need to get started. After reading this blog, we’d recommend checking out our <a href="https://www.elastic.co/guide/en/elasticsearch/reference/7.1/configuring-security.html" target="_self">documentation on securing the Elastic Stack</a>, which includes helpful <a href="https://www.elastic.co/guide/en/elastic-stack-overview/7.1/security-getting-started.html" target="_self">practical examples</a>. As a quick note, we’ll be doing this all on a local cluster because, while these features became available by default in 6.8 and 7.1, they have always come standard with our <a href="https://www.elastic.co/cloud/elasticsearch-service/subscriptions" target="_self">Elasticsearch Service</a>. </p><h2>Install Elasticsearch and Kibana</h2><p>For this example, I’ll be installing on a laptop running Linux. Adjust the steps listed below to your specific environment. </p><h3>Step 1: Download Elasticsearch and Kibana</h3><p>The first thing you have to do is <a href="https://www.elastic.co/downloads?elektra=products&storm=main" target="_self">download</a>&nbsp;the default distribution of Elasticsearch and Kibana version 6.8+ or 7.1+. Security was added to the default distribution in 7.1 and 6.8, so if you’re on an older version, you’ll need to upgrade. </p><h3>Step 2: Extract Elasticsearch and Kibana</h3><p>For this example, I’ve downloaded <em>elasticsearch-7.1.0-linux-x86_64.tar.gz</em> and <em>kibana-7.1.0-linux-x86_64.tar.gz</em>. Once downloaded, their contents need to be extracted. </p><h3>Step 3: Create two Elasticsearch nodes</h3><p> To create a 2-node cluster, make two copies of the Elasticsearch directory, one named <em>master</em> one named <em>node</em>. After completing this step, you’ll have the following files and folders: </p><pre class="prettyprint">elasticsearch-7.1.0                      elasticsearch-7.1.0-node<br/>elasticsearch-7.1.0-linux-x86_64.tar.gz  kibana-7.1.0-linux-x86_64<br/>elasticsearch-7.1.0-master               kibana-7.1.0-linux-x86_64.tar.gz<br/></pre><h2>Configure TLS and authentication</h2><p>The first thing we have to do is generate some certificates that allow the nodes to communicate securely. You can do this with an enterprise CA, but for this demo, we have a command called elasticsearch-certutil you can run to do this without any of the usual certificate confusion. </p><h3>Step 1: TLS on Elasticsearch master</h3><p>You can cd into the <em>master</em> directory and then run this command: </p><pre class="prettyprint">bin/elasticsearch-certutil cert -out config/elastic-certificates.p12 -pass ""<br/></pre><p>Next, open up the file config/elasticsearch.yaml in your favorite text editor. Paste the lines below at the end of the file. </p><pre class="prettyprint">xpack.security.enabled: true<br/>xpack.security.transport.ssl.enabled: true<br/>xpack.security.transport.ssl.verification_mode: certificate<br/>xpack.security.transport.ssl.keystore.path: elastic-certificates.p12<br/>xpack.security.transport.ssl.truststore.path: elastic-certificates.p12<br/></pre><p>Save the file, and we’re ready to start the master node. Run the command bin/elasticsearch. This executable has to keep running, put this terminal aside for now. </p><h3>Step 2: Elasticsearch cluster passwords</h3><p>Once the master node is running, it’s time to set the passwords for the cluster. In a new terminal we have to cd into our master node directory. Then we run the command bin/elasticsearch-setup-passwords auto. This will generate random passwords for the various internal stack users. You can alternatively skip the <em>auto</em> parameter to manually define your passwords using the <em>interactive</em> parameter. Keep track of these passwords, we’ll need them again soon. </p><h3>Step 3: TLS on Elasticsearch&nbsp;node</h3><p>Now let’s open another new terminal and cd into our node directory. In my case, this was named <em>elasticsearch-7.1.0-node</em>. </p><p>The easiest thing we can do is to just copy everything from the master’s config directory into the node’s config directory. </p><pre class="prettyprint">cp ../elasticsearch-7.1.0-master/config/* config/<br/></pre><p>You should also add the configuration option node.master: false to the config/elasticsearch.yml file. I won’t get into the details here, you can find additional details in our <a href="https://www.elastic.co/guide/en/elasticsearch/reference/7.1/modules-discovery-bootstrap-cluster.html" target="_self">cluster documentation</a>. Alternative to just copying the config files wholesale, you can simply copy the certificate files and then set the xpack.security.* keys the same as with the master node. </p><p>Then we start the node by running bin/elasticsearch. We’ll see it join the cluster. And if we look at the master node’s terminal window we can see a message there that a node joined the cluster. Now we have a 2-node cluster running. </p><h3>Step 4: Security in Kibana</h3><p>The last thing we need to do is configure Kibana. Once again open another terminal window (this is the last one, I promise). We’ll need to add the password for the Kibana user. We can grab that from the previous output of the setup-passwords command. </p><p>Let’s cd into the Kibana directory and open up the config/kibana.yml file with your text editor. Find the lines that look like this </p><pre class="prettyprint">#elasticsearch.username: "user"<br/>#elasticsearch.password: "pass"<br/></pre><p>Uncomment the username and password fields by removing the # character at the beginning of the line. Change "user" to "kibana" and then change "pass" to whatever the setup-passwords command tells us the Kibana password is. Save the file then we can start Kibana by running bin/kibana. </p><h2>Configure role-based access control (RBAC) within Kibana</h2><p>Once Kibana is running we can switch over to our web browser and open <a href="http://localhost:5601/login?next=%2F#?_g=()" target="_self">http://localhost:5601</a>. It’s going to ask us for a login, we’ll use the <em>elastic</em> superuser with the password from setup-passwords. </p><p><img src="https://images.contentstack.io/v3/assets/bltefdd0b53724fa2ce/blt33ef6ef8666f75bc/5ce2480b32fd88a3767ede65/elasticsearch-security-blog-1.jpg" data-sys-asset-uid="blt33ef6ef8666f75bc" alt="Log into Kibana"/></p><p>Kibana will ask us if we want to load some sample data when it’s a new install such as this. </p><p><img src="https://images.contentstack.io/v3/assets/bltefdd0b53724fa2ce/blt6617bf668ec635cd/5ce2483c7182ba9776722fee/elasticsearch-security-blog-2.jpg" data-sys-asset-uid="blt6617bf668ec635cd" alt="Load sample data into Kibana"/></p><p>Let’s load the flight and web sample data for the sake of this example. Once loaded, click on the configuration <strong>gear icon</strong>. </p><p><img src="https://images.contentstack.io/v3/assets/bltefdd0b53724fa2ce/bltae0b05ad48832121/5ce2486e2f06657f7b6c9907/elasticsearch-security-blog-3.jpg" data-sys-asset-uid="bltae0b05ad48832121" alt="Navigate to management within Kibana"/></p><p>Now let’s create our roles. Find and click on the <strong>roles</strong> option. </p><p><img src="https://images.contentstack.io/v3/assets/bltefdd0b53724fa2ce/blt90c2f3cfca9d44fc/5ce2493908d3fd9e7671c281/elasticsearch-security-blog-4.jpg" data-sys-asset-uid="blt90c2f3cfca9d44fc" alt="Roles within Kibana"/></p><p>Click on <strong>Create role</strong>. </p><p><img src="https://images.contentstack.io/v3/assets/bltefdd0b53724fa2ce/blt3746b50792f5e2c0/5ce2497308d3fd9e7671c287/elasticsearch-security-blog-5.jpg" data-sys-asset-uid="blt3746b50792f5e2c0" alt="Create a new role"/></p><p>We’re going to name the first role <em>read_logs</em>. </p><p><img src="https://images.contentstack.io/v3/assets/bltefdd0b53724fa2ce/blt6fcd624d7201821d/5ce24a7632fd88a3767ede75/elasticsearch-security-blog-6.jpg" data-sys-asset-uid="blt6fcd624d7201821d" alt="Create the read_logs role"/></p><p>Scroll down to the index privileges field and select the log index. Then we give it the <strong>read</strong> privilege. </p><p><img src="https://images.contentstack.io/v3/assets/bltefdd0b53724fa2ce/bltc3c087f105d52fa6/5ce24a87ccfda8847b20bb43/elasticsearch-security-blog-7.jpg" data-sys-asset-uid="bltc3c087f105d52fa6" alt="Configure role privileges"/></p><p>This section about spaces is something you’ll probably want to investigate once you have security working, but for this post we’re going to skip it. To learn more about spaces — which are great for organizing and securing data within Kibana — check out our <a href="https://www.elastic.co/guide/en/kibana/7.1/xpack-spaces.html" target="_self">Kibana spaces documentation</a> or this <a href="https://www.elastic.co/blog/introducing-kibana-spaces-for-organization-and-security" target="_self">introductory Kibana spaces blog</a>. </p><p>Now let’s create another role called <em>read_flight</em>. </p><p><img src="https://images.contentstack.io/v3/assets/bltefdd0b53724fa2ce/bltf26a34f5ff885981/5ce24abc1df9c928761cd92d/elasticsearch-security-blog-8.jpg" data-sys-asset-uid="bltf26a34f5ff885981" alt="Create read_flight role"/></p><p>We assign the flight index the <strong>read</strong> privilege. </p><p><img src="https://images.contentstack.io/v3/assets/bltefdd0b53724fa2ce/blt3e1d9d93aba1294c/5ce24acd2d8f163476ebd44b/elasticsearch-security-blog-9.jpg" data-sys-asset-uid="blt3e1d9d93aba1294c" alt="Configure the role"/></p><p>Now we’re going to create two users and assign them these roles. Select the <strong>Users</strong> link on the left and click the <strong>Create new user</strong> button. </p><p><img src="https://images.contentstack.io/v3/assets/bltefdd0b53724fa2ce/bltc557d504a8ac1f95/5ce24aee7182ba9776722ff9/elasticsearch-security-blog-10.jpg" data-sys-asset-uid="bltc557d504a8ac1f95" alt="Create a new user"/></p><p>We’ll name this user <em>flight_user</em> and set the password. We don’t need to set the full name and email address. We need to assign our <strong>read_flight</strong> role and we also need to assign the <strong>kibana_user</strong> role as this user will be viewing data in Kibana. </p><p><img src="https://images.contentstack.io/v3/assets/bltefdd0b53724fa2ce/blt7adcce531abcc45a/5ce24b00932da9737b2628c6/elasticsearch-security-blog-11.jpg" data-sys-asset-uid="blt7adcce531abcc45a" alt="Create the flight user"/></p><p>Hit create, and let’s do it again for the <em>log_user</em>. This time we add the <strong>read_logs</strong> and <strong>kibana_user</strong> roles. </p><p><img src="https://images.contentstack.io/v3/assets/bltefdd0b53724fa2ce/blt805d2eb40672403d/5ce24b13ccfda8847b20bb4b/elasticsearch-security-blog-12.jpg" data-sys-asset-uid="blt805d2eb40672403d" alt="Create the log user"/></p><p>OK we’re all set with two new users. We can now log out as the <em>elastic</em> user. </p><p><img src="https://images.contentstack.io/v3/assets/bltefdd0b53724fa2ce/bltb07056dabca17b82/5ce24ba52d8f163476ebd451/elasticsearch-security-blog-13.jpg" data-sys-asset-uid="bltb07056dabca17b82" alt="Logout as elastic"/></p><p>Then we can login as <em>log_user</em>. If we click on the <strong>dashboards</strong> icon we can see there are two dashboards, one for flights and one for logs. </p><p><img src="https://images.contentstack.io/v3/assets/bltefdd0b53724fa2ce/bltfb2217281f933c85/5ce24bc53d021e2d76457546/elasticsearch-security-blog-14.jpg" data-sys-asset-uid="bltfb2217281f933c85" alt="Check out your dashboards"/></p><p><strong>If you click on the Logs dashboard you will see the demo log data. If we go back and try to view the flight dashboard we can see the dashboard, but we can’t see any of the data. This is a place Kibana spaces can be used for security purposes to only allow certain users to see certain dashboards. </strong></p><p><strong>Now we can logout as the </strong><em><strong>log_user</strong></em><strong> and log back in as the </strong><em><strong>flight_user</strong></em><strong>. </strong></p><p><strong>If we open the log dashboard, this time we can’t see any data. We even get an error because Kibana can’t access the log index. </strong></p><p><img src="https://images.contentstack.io/v3/assets/bltefdd0b53724fa2ce/blt6f857fe2c548af9e/5ce24be67182ba9776722fff/elasticsearch-security-blog-15.jpg" data-sys-asset-uid="blt6f857fe2c548af9e" alt="Data restricted by RBAC"/></p><p><strong>Now when we check the flight dashboard we can see we have access to the flight data this time. </strong></p><p><img src="https://images.contentstack.io/v3/assets/bltefdd0b53724fa2ce/blt89b48cc8a9e81e31/5ce24bfb3d021e2d7645754e/elasticsearch-security-blog-16.jpg" data-sys-asset-uid="blt89b48cc8a9e81e31" alt="Data access provided by RBAC"/></p><p><strong>Congratulations! We now have a two node cluster configured and two users with restricted access to index data. </strong></p><h2>Wrapping up</h2><p>Security in the Elastic Stack is extremely versatile, there’s so much more that it can do that we didn’t cover in this post. The purpose of this post was to get you started. In the future we’ll have plenty of content covering many of the security features and how you can use them. In the meantime, we have more documentation and blog posts on how Elastic Stack security works and what else you can accomplish. </p><ul><li><a href="https://www.elastic.co/guide/en/elasticsearch/reference/current/configuring-stack-security.html" target="_self">Fundamentals of Securing Elasticsearch training</a></li><li><a href="/guide/en/elasticsearch/reference/current/configuring-security.html" target="_self">Elasticsearch security documentation</a></li><li><a href="/blog/elasticsearch-security-configure-tls-ssl-pki-authentication" target="_self">Configuring TLS blog post</a></li><li><a href="/blog/introducing-kibana-spaces-for-organization-and-security" target="_self">Kibana Spaces blog post</a></li></ul><p>You can also <a href="https://discuss.elastic.co/" target="_self">ask for help on our public forum</a>. </p><p>Happy searching! </p>

Getting started with Elasticsearch security

<html><head></head><body><p>One of the advantages to Elasticsearch and Kibana is the ability to quickly look at data in a new way. Some time ago, I put <a href="https://github.com/joshbressers/blockchain-elasticsearch">all the Bitcoin data</a> into Elasticsearch for a talk I&#x2019;m working on looking for interesting things in the Bitcoin blockchain, but given the <a href="https://www.forbes.com/sites/cbovaird/2018/10/05/bitcoin-price-volatility-falls-to-lowest-of-2018/">current turmoil</a> I see around the price of Bitcoin, I decided to take a look at what&#x2019;s going on. There are a number of interesting data points that tell the story of Bitcoin.
</p><p>Nearly everyone has heard of Bitcoin at this point. Everyone knows it&#x2019;s some sort of internet money that&#x2019;s supposed to be untraceable. The data that makes it all work is fascinating, though. To understand what&#x2019;s happening to Bitcoin right now we need to know a few things about how Bitcoin work. As we have an enormous amount of data available to tell this story it&#x2019;s easier to show you what Bitcoin is rather than just write about it.
</p><p>Bitcoins start with something called a block. There are slightly more than 550,000 blocks as of this writing. Blocks are &#x201C;mined&#x201D; using a complex mathematical equation. Each block has something called &#x201C;difficulty&#x201D; that we&#x2019;ll refer to later. In theory, a new block is generated every ten minutes. If blocks are being mined too fast, we increase difficulty, if they&#x2019;re generated too slow, we decrease difficulty. As we can see from the chart, the generation rate of Bitcoins has been pretty consistent over time.
</p><p><img src="https://images.contentstack.io/v3/assets/bltefdd0b53724fa2ce/blt13dc6bc09a76c5dc/5c5837678d25f6030ca045bd/blockchain_generation_rates.png" data-sys-asset-uid="blt13dc6bc09a76c5dc" alt="" "=""><br></p><figcaption>Block Generation Rates</figcaption><p>By itself, difficulty isn&#x2019;t terribly interesting, but it does show us something really important. It&#x2019;s how much computing power is being dedicated to mining Bitcoins. If you look at the graph below, you can see the difficulty has been growing at an incredible rate since the beginning of bitcoin, probably too fast, and this exponential growth in Bitcoin &#x2014; and processing cycles for mining &#x2014; is why you hear talk about it using as much electricity as some countries. As long as literally anyone is mining Bitcoins, we&#x2019;re going to see a block generated every ten minutes.
</p><p><img src="https://images.contentstack.io/v3/assets/bltefdd0b53724fa2ce/blt15a12c587fb93691/5c583761f967c7fc0b4c6951/block_mining_difficulty.png" data-sys-asset-uid="blt15a12c587fb93691" alt="" "=""><br></p><figcaption>Bitcoin Block Mining Difficulty</figcaption><p>The other important thing we need to understand about Bitcoin is that every block contains one or more transactions. Every time any coins are moved around it has to be part of a transaction. This basically means the number of transactions seen in each block shows how busy the bitcoin network is. The more bitcoins moving around, the more transactions we see.
</p><p><img src="https://images.contentstack.io/v3/assets/bltefdd0b53724fa2ce/blt6f456f895a0c849a/5c58375b9cd9dcf90b745421/bitcoin_transactions.png" data-sys-asset-uid="blt6f456f895a0c849a" alt="" "=""><br></p><figcaption>Bitcoin Transactions</figcaption><p>OK, so at this point we have enough knowledge to look deeper at what&#x2019;s happening.
</p><p>First, we should look at the price of Bitcoin. It&#x2019;s been a pretty wild ride for the past few months. At one point the price was $20,000 which is pretty crazy. It&#x2019;s taken quite the turn down recently. You can see the price drop substantially in the last few weeks.
</p><p><img src="https://images.contentstack.io/v3/assets/bltefdd0b53724fa2ce/blt661f47f48f1e9161/5c5837554c9f4feb0b4c5a67/bitcoin_valuation.png" data-sys-asset-uid="blt661f47f48f1e9161" alt="" "=""><br></p><figcaption>Bitcoin Valuation over Time</figcaption><p>If we compare price vs transactions, it&#x2019;s not surprising to see the transactions going up as the price goes up. A price going up suggests more people are Buying bitcoin. The basics of economics teach us that if demand is greater than supply, the price goes up. This shouldn&#x2019;t surprise anyone as it&#x2019;s basically a sell off. When supply is greater than demand, the price goes down. We also see transactions go up, which would signify a lot of people selling their Bitcoins.
</p><p><img src="https://images.contentstack.io/v3/assets/bltefdd0b53724fa2ce/blt462875741c7f8476/5c58374f65157fdb0b6c6525/price_v_transaction.png" data-sys-asset-uid="blt462875741c7f8476" alt="" "=""><br></p><figcaption>Price vs. Transaction</figcaption><p>There&#x2019;s something really important we can learn if we add in the difficulty graph though.
</p><p><img src="https://images.contentstack.io/v3/assets/bltefdd0b53724fa2ce/blt78a251b16cd0883f/5c58374916a7de000c2b353b/price_v_transaction_v_difficulty.png" data-sys-asset-uid="blt78a251b16cd0883f" alt="" "=""><br></p><figcaption>Price vs. Transaction vs. Difficulty</figcaption><p>The difficulty is going down, by quite a bit. It&#x2019;s never gone down quite like this before as we can see from this graph. Difficulty is an important metric to measure the Bitcoin network. We really don&#x2019;t know how many miners exist in the Bitcoin network. What we do know though is as long as Bitcoin has existed the difficulty has gone up, which suggests an environment where the number of miners is always increasing.
</p><p>The difficulty is dropping because fewer people are mining Bitcoins, meaning there are fewer Bitcoin miners now than there were at the peak of the chart.
</p><p>Now we should discuss some of the economics of Bitcoin. Mining Bitcoin is quite expensive today in terms of cost of gear as well as cost of electricity. Bitcoin is mined using Application Specific Integrated Circuits (or ASIC). Gone are the days when CPUs or GPUs can be used to mine bitcoin. CPUs and GPUs can be used to mine other coins, but not Bitcoin. This means that miners have equipment that can only do one thing, mine Bitcoin. It&#x2019;s also fairly difficult to purchase and costs quite a bit. It&#x2019;s further complicated because you can&#x2019;t repurpose the ASIC gear to do anything else. The ASIC gear can only calculate SHA-256 hashes very quickly, which means it&#x2019;s really only good for calculating Bitcoin hashes.
</p><p>So let&#x2019;s put it all together. Miners have very specific gear that only mines Bitcoins. The cost of the gear and electricity is expensive, very expensive. The amount of gear mining Bitcoins on the network is currently decreasing. So one&#xA0;logical conclusion of this is that some miners have taken their gear offline because the return on their investment is negative.
</p><p>Getting the Bitcoin client running&#xA0;and then getting all the data into Elasticsearch isn&#x2019;t always simple, but if you find this data as fascinating as I do and have any questions, feel free to ping me on twitter <a href="https://twitter.com/joshbressers">@joshbressers</a>. I&#x2019;d be happy to chat about results or help you get started.</p></body></html>

bitcoin_banner.jpg

bitcoin_thumb.jpg

Analyzing Bitcoin Price Behavior with Elasticsearch & Kibana

elasticsearch-feature-security-kerberos-fbi.jpg

elasticsearch-feature-security-kerberos-tbi.jpg

How to Secure your Elasticsearch Clusters using Kerberos

<p>Today we are pleased to announce the release of&nbsp;<strong>Elasticsearch 6.2.0</strong>, based on&nbsp;<strong>Lucene 7.2.1</strong>. This is the latest stable release, and is already available for deployment on&nbsp;<a class="ulink" href="/cloud" target="_top">Elastic Cloud</a>, our Elasticsearch-as-a-service platform.</p><p>Latest stable release in 6.x:</p><div class="itemizedlist"><ul class="itemizedlist" type="disc"><li class="listitem"><a class="ulink" href="/downloads/elasticsearch">Download Elasticsearch 6.2.0</a></li><li class="listitem"><a class="ulink" href="/guide/en/elasticsearch/reference/6.2/release-notes-6.2.0.html">Elasticsearch 6.2.0 release notes</a></li><li class="listitem"><a class="ulink" href="/guide/en/elasticsearch/reference/6.2/breaking-changes.html">Elasticsearch 6.2 breaking changes</a></li><li class="listitem"><a class="ulink" href="/guide/en/x-pack/current/xpack-change-list.html">X-Pack 6.2.0 release notes</a></li></ul></div><p>You can read about all the changes in the release notes linked above, but there are a few changes which are worth highlighting:</p><h2 dir="ltr">SAML Support</h2><p dir="ltr">The ability to login to X-Pack has existed for quite some time. We’re seeing a trend in the industry where every application having its own login screen is not only annoying but less secure than using single sign on for authentication. With the release of 6.2.0 you now have the ability to login to Kibana using a <a href="/guide/en/x-pack/6.2/saml-realm.html">SAML single sign on provider</a>. This should simplify managing users in your elasticsearch cluster and provide a more secure environment. No more forgotten passwords or Kibana login screens. Just login to your SAML provider and when you visit Kibana you have access.</p><h2 dir="ltr">Audit Filtering</h2><p dir="ltr">If you’ve used the <a href="/guide/en/x-pack/6.2/auditing.html">audit logging capabilities of X-Pack</a> in the past you know it has a very all or nothing feel to it. Logging too little may not provide the data you need and too much can fill up storage. As compliance requirements continue to become more strict good audit logs become a necessity. We have added to ability to <a href="/guide/en/x-pack/6.2/auditing.html#audit-log-ignore-policy">filter audit logs in version 6.2.0</a>. Now you don’t have to worry about logging too little data or filling up a disk because of a noisy client. Log what you need and only what you need.</p><h2 dir="ltr">Search Quality and the New Ranking Evaluation API</h2><p dir="ltr">Let's say you've built a local search service, and a user has just searched for "Amsterdam", would he or she prefer a document on "Welcome to Amsterdam!" or "Sights and Sounds of Holland"? How do you ensure a user will always see the best results for the search term "Amsterdam", especially as your search index grows over time, new documents and fields are added, and other factors affect the score of various search results?</p><p dir="ltr">In Elasticsearch 6.2, we are excited to announce a new <a href="/guide/en/elasticsearch/reference/6.2/search-rank-eval.html">ranking evaluation api</a>. The Rank Evaluation API is a new endpoint in Elasticsearch for tracking how the rankings of expected results are quantitatively measured against specific queries. &nbsp;This allows you to create a set of metrics that can be measured programmatically over time. Imagine creating a set of unit tests that ensure "Welcome to Amsterdam!" always ranks highly for our search term above.</p><p dir="ltr">The Ranking Evaluation API can take into account <a href="https://en.wikipedia.org/wiki/Mean_reciprocal_rank">Mean Reciprical Rank (MRR)</a>, <a href="https://en.wikipedia.org/wiki/Information_retrieval#Precision_at_K">Precision-at-k (P@k)</a>, and <a href="https://en.wikipedia.org/wiki/Discounted_cumulative_gain">Discounted Cumulative Gain (DCG)</a>, as ways to measure quality in results. If you've never heard of these, have a look over the linked wiki articles.</p><h2 dir="ltr">Providing greater safeguards to user searches</h2><p dir="ltr">Creating a search query containing 100,000 ids sounds like a bad idea, and any user performing a query like this could negatively impact Elasticsearch. In 6.2, we've provided improved safeguards to limit the number of bad queries that can be executed by Elasticsearch.</p><p dir="ltr">A few of the new safeguards include:</p><ul><li dir="ltr">Default limit for terms in a <a href="/guide/en/elasticsearch/reference/6.2/query-dsl-terms-query.html">terms query</a>: 65,536</li><li dir="ltr">Limit on the number of <a href="/guide/en/elasticsearch/reference/6.2/mapping.html">nested objects</a>: 10,000</li><li dir="ltr">Limit to the number of characters to be analyzed for <a href="/guide/en/elasticsearch/reference/6.2/search-request-highlighting.html">highlighting</a>: 10,000</li><li dir="ltr">Default <a href="/guide/en/elasticsearch/reference/6.2/analysis-shingle-tokenfilter.html">max_shingle_size and min_shingle_size</a>&nbsp;allowed difference: 3</li><li dir="ltr">Default <a href="/guide/en/elasticsearch/reference/6.2/analysis-ngram-tokenizer.html">max_ngram and min_ngram</a>&nbsp;allowed difference: 1</li></ul><p dir="ltr">We’ve created these numbers as sane defaults for most Elasticsearch systems, however, any of the settings above can be modified by an administrator.</p><h2 dir="ltr">Well Known Text Support</h2><p dir="ltr">Several geospatial systems use <a href="https://en.wikipedia.org/wiki/Well-known_text">Well Known Text (WKT)</a> as their preferred/only format for geospatial objects. &nbsp;What if you wanted to use Elasticsearch for your geospatial data though? &nbsp;Until 6.2, Elasticsearch has only provided the option of providing shapes in <a href="http://geojson.org/">GeoJSON</a> format. &nbsp;&nbsp;To get your WKT data into Elasticsearch, you may have to go through a complicated export + conversion process. &nbsp;<a href="/guide/en/elasticsearch/reference/6.2/geo-shape.html#input-structure">No longer</a>! &nbsp;You can now index a shape in a WKT string directly to Elasticsearch.</p><h2 dir="ltr">Better Programmatic Access to the Task Manager</h2><p dir="ltr">In Elasticsearch 2.3, we <a href="/blog/elasticsearch-2-3-0-and-2-2-2-released">introduced a new Task Management API</a>. &nbsp;That API has been responsible for tracking ongoing long-running tasks like <a href="/guide/en/elasticsearch/reference/6.2/docs-reindex.html">reindex operations</a>. &nbsp;However, because Elasticsearch may rewrite parts of tasks once it receives them, actioning ongoing tasks including cancelling them has required manual intervention to try to figure out which of the tasks matched up with a particular action. &nbsp;For many types of tasks, that may be rare and easy, but doing this programmatically would be nearly impossible. &nbsp;Now, in 6.2, we’ve added a new capability to this tasks API: <a href="/guide/en/elasticsearch/reference/6.2/tasks.html#_identifying_running_tasks">you can now assign a task an ID to a new task using the X-Opaque-Id header</a> and &nbsp;then use that to look up and/or cancel tasks programmatically later.</p><h2 dir="ltr">Java 9 Support</h2><p dir="ltr">Plugin developers and users that build at home may have <a href="https://github.com/elastic/elasticsearch/pull/28071">already noticed</a> some changes in Elasticsearch which now requires JDK 9 to build. &nbsp;We’ve outlined many elements of this change in a <a href="/blog/elasticsearch-java-9-and-beyond">blog post</a>. &nbsp;With the release of Elasticsearch 6.2, we’re happy to announce that we now support Java 9! &nbsp;Before you go go through and replace every Java install in your company, you’ll want to be aware of a few things:</p><ol><li dir="ltr">The majority of JDK vendors are treating version 9 and version 10 as <a href="http://www.oracle.com/technetwork/java/eol-135779.html">short term releases</a>. &nbsp;With only a six month total version lifecycle, both JDK 9 and 10 are looking to hit an end of public updates before JDK 8. &nbsp;In fact, at the time of this writing, it looks like JDK 9 will hit end of life as soon as March 2018 and JDK 10 is slated to EOL in September 2018. &nbsp;Practically, all of this means: if you like changing things less often, you may want to stick to Java 8 for now.</li><li dir="ltr">JDK 9 hits end of updates before Elasticsearch 6.2 EOLs. &nbsp;You should be extra aware before using it that JDK vendors&nbsp;will be unable to patch any security issues in JDK 9 after it reaches that stage.</li></ol><p dir="ltr">As a result of these, while we will be supporting Java 9 per our <a href="/support/matrix#matrix_jvm">support matrix</a>, <a href="/guide/en/elasticsearch/reference/6.2/setup.html#jvm-version">we strongly encourage</a> users to stick to Java 8 unless you’re prepared for dealing with this rapidly evolving landscape.</p><div class="titlepage"><h2 class="title">Conclusion</h2></div><p>Please&nbsp;<a class="ulink" href="/downloads/elasticsearch">download Elasticsearch 6.2.0</a>, try it out, and let us know what you think on Twitter (<a class="ulink" href="https://twitter.com/elastic" target="_top">@elastic</a>) or in our&nbsp;<a class="ulink" href="https://discuss.elastic.co/c/elasticsearch" target="_top">forum</a>. You can report any problems on the&nbsp;<a class="ulink" href="https://github.com/elastic/elasticsearch/issues" target="_top">GitHub issues page</a>.</p><div class="titlepage"><h2 class="title">Known Issues</h2></div><p>Elasticsearch 6.2.0 is affected by a <a href="https://github.com/elastic/elasticsearch/issues/28538">bug</a> that impacts the ability to upgrade plugins. A <a href="https://github.com/elastic/elasticsearch/pull/28540">fix</a>&nbsp;will be released shortly. To workaround until a fix is available, remove plugins before upgrading from an older version or downgrade Elasticsearch to your previously-installed&nbsp;version, remove plugins, and then upgrade again.</p><p><br /></p><div><br /></div>

elasticsearch-blog-banner.jpg

elasticsearch-blog-thumb.jpg

Elasticsearch 6.2.0 released

Sense Chrome plugin malware issue

Start free trial

Blog Footer CTA

Sign up for Elastic Cloud free trial

headshot-josh-bressers-300x300.png

Principal Product Security Engineer

Josh Bressers

<p>Надсилаючи цю форму ви погоджуєтеся з <a href="/legal/terms-of-use">Умовами надання послуг Elastic</a>. Ваші особисті дані будуть опрацьовані відповідно з <a href="/legal/privacy-statement">Положеннями про конфіденційність Elastic</a>.</p>

Newsletter GDPR Text - Ukrainian

<p>By submitting you acknowledge that you've read and agree to our <a href="/legal/elastic-cloud-account-terms" target="_self">Terms of Service</a>, and you agree to receive the <a href="/legal/privacy-statement#how-we-use-the-information" target="_self">selected communications</a> at the contact details you provided above. See <a href="/legal/privacy-statement/" target="_self">Elastic’s Privacy Statement</a> for more details or to opt-out at any time.</p>

Communication Preferences - ONLY for /communication-preferences

<p>By submitting you agree to <a href="/agreements/global/cloud-services-monthly" rel="nofollow">Elastic Cloud Terms of Service</a>. Your personal data will be processed in accordance with <a href="/legal/privacy-statement">Elastic's Privacy Statement</a>.</p>

Cloud Trial GDPR Text

Intentionally Blank

<p>By submitting you acknowledge that you've read and agree to our <a href="/legal/serverless-preview-terms" target="_blank">Terms of Service</a>, and that Elastic may <a href="/legal/privacy-statement#how-we-use-the-information" target="_blank">contact you</a> about our related products and services, using the details you provide above. See <a href="/legal/privacy-statement/" target="_blank">Elastic’s Privacy Statement</a> for more details or to opt-out at any time.</p>

Serverless GDPR Text

<p>提交即表示您確認已詳閱並同意我們的<a href="/legal/elastic-cloud-account-terms" target="_blank">服務條款</a>，且同意 Elastic 使用您在上方提供的個人資料<a href="/legal/privacy-statement#how-we-use-the-information" target="_blank">與您聯絡</a>，為您提供相關產品和服務的資訊。如需詳細說明或隨時想停止收到產品和服務資訊，請參閱 <a href="/legal/privacy-statement/" target="_blank">Elastic 的隱私聲明</a>。</p>

Newsletter GDPR Text - TW

<p>By submitting you acknowledge that you've read and agree to our <a href="/legal/elastic-cloud-account-terms" target="_blank">Terms of Service</a>, and that Elastic may <a href="/legal/privacy-statement#how-we-use-the-information" target="_blank">contact you</a> about our related products and services, using the details you provide above. See <a href="/legal/privacy-statement/" target="_blank">Elastic’s Privacy Statement</a> for more details or to opt-out at any time.</p><p>By submitting this form, I agree to Elastic sharing my contact information with Google Cloud. I also agree to Elastic giving my information to the vendor for the purpose of shipping my food package.</p>

GDPR Text + Google Cloud

<p>Inviando la registrazione (1) esprimi il consenso ai <a href="/legal/terms-of-use">termini e condizioni di Elastic</a> e <a href="/legal/privacy-statement">all’informativa sulla privacy</a>, nonché (2) a ricevere saltuarie e-mail.</p>

Newsletter GDPR Text - Italian

<p>By submitting you acknowledge that you've read and agree to our <a href="/legal/terms-of-use" target="_blank">Terms of Service</a>, and that Elastic may <a href="/legal/privacy-statement#how-we-use-the-information" target="_blank">contact you</a> about our related products and services, using the details you provide above. See <a href="/legal/privacy-statement/" target="_blank">Elastic’s Privacy Statement</a> for more details or to opt-out at any time.</p>

Newsletter GDPR Text

Load more

Overview

Can't make it? Register and we'll send you the recording. You'll also receive an email with related content

Can't make it? Register and we'll send you the recording. You'll also receive an email with related content.

You'll also receive an email with related content.

You'll also receive an email with related content

Speakers

Close

See more insights

The content on this page is not available in the selected language. As Elastic grows globally, we continue to support content in multiple languages.

Author

Articles by Josh Bressers

Principal Product Security Engineer, Elastic

How to prevent an Elasticsearch server breach

Tips to secure Elasticsearch clusters for free with encryption, users, and more

Getting started with Elasticsearch security

Analyzing Bitcoin Price Behavior with Elasticsearch & Kibana

How to Secure your Elasticsearch Clusters using Kerberos

Elasticsearch 6.2.0 released

Sense Chrome plugin malware issue

Follow us

About us

Join us

Partners

Trust & Security

Investor relations

Excellence Awards