generating-an-elastic-cloud-enterprise-client-fullbleed.jpg

generating-an-elastic-cloud-enterprise-client.jpg

Generating an Elastic Cloud Enterprise Client

<html><head></head><body><p>With the <a href="https://www.elastic.co/blog/elasticsearch-5-0-0-alpha1-released">first alpha release</a> of Elasticsearch 5.0 comes a ton of new and awesome features, and if you&apos;ve been paying attention then you know that one of the more prominent of these features is the new shiny <a href="https://www.elastic.co/guide/en/elasticsearch/reference/master/ingest.html">ingest node</a>.&#xA0;Simply put, ingest aims to provide a lightweight solution for pre-processing and enriching documents within Elasticsearch itself before they are indexed.
</p><p>We aren&apos;t going to dive into the details of how ingest node works in this blog post (it is recommended that you <a href="https://www.elastic.co/guide/en/elasticsearch/reference/master/ingest.html">read the docs</a>&#xA0;as a prerequisite), but instead we&apos;re going to showcase how to consume the ingest APIs from an Elasticsearch client. In these examples we&apos;re going to use <a href="https://github.com/elastic/elasticsearch-net">NEST</a>, the official .NET client, but keep in mind that these concepts apply to any of the&#xA0;<a href="https://www.elastic.co/guide/en/elasticsearch/client/index.html">official Elasticsearch language clients</a>.&#xA0;So whether you&apos;re C# inclined or not, transferring this knowledge to the language of your choice should be trivial.
</p><h2>Creating an ingestion pipeline</h2><p>In Elasticsearch 5.0, all nodes are ingest nodes by default, so there&apos;s nothing to do in terms of setting up Elasticsearch.&#xA0;Thus, the first step to enriching documents via an ingest node is to create an ingestion pipeline.
</p><p>Let&apos;s use the classic Tweet example (my apologies) and create a <code>tweets</code> index in our Elasticsearch cluster with the following mapping:
</p><pre class="prettyprint">{
  &quot;tweets&quot;: {
    &quot;mappings&quot;: {
      &quot;tweet&quot;: {
        &quot;properties&quot;: {
          &quot;lang&quot;: {
            &quot;type&quot;: &quot;keyword&quot;
          },
          &quot;message&quot;: {
            &quot;type&quot;: &quot;text&quot;
          },
          &quot;retweets&quot;: {
            &quot;type&quot;: &quot;integer&quot;,
            &quot;coerce&quot;: false
          }
        }
      }
    }
  }
}
</pre><p>Here, <code>message</code> contains the actual content of the tweet, <code>lang</code> represents the language code (<code>en</code>, <code>fr</code>, etc...), and <code>retweets</code> is the number of times the tweet has been re-tweeted.
</p><p>Now imagine we have the following C# type to model our <code>tweet</code> documents:
</p><pre class="prettyprint">public class Tweet
{
    public string Message { get; set; }
    public string Lang { get; set; }
    public string Retweets { get; set; }
}
</pre><p>Notice that <code>Retweets</code> is of type <code>string</code>, and perhaps for some reason we cannot to change the type.&#xA0;Well, since we set <code>&quot;coerce&quot;: false</code> in our mapping of the <code>retweets</code> field, if we try to index one of these documents, Elasticsearch is going to throw a parse exception since the incoming value will be a <code>string</code>.
</p><p>So, what can we do?&#xA0;Let&apos;s create a pipeline that converts our <code>retweets</code> field to an <code>integer</code> before indexing it.&#xA0;While we&apos;re at it, let&apos;s also uppercase our language codes since they are of type <code>keyword</code> in our mapping and so case-sensitivity matters.
</p><p>Here&apos;s what this looks like using NEST:
</p><pre class="prettyprint">var client = new ElasticClient();
client.PutPipeline(&quot;tweet-pipeline&quot;, p =&gt; p
    .Processors(ps =&gt; ps
        .Convert&lt;Tweet&gt;(c =&gt; c
            .Field(t =&gt; t.Retweets)
            .Type(ConvertProcessorType.Integer)
        )
        .Uppercase&lt;Tweet&gt;(u =&gt; u
            .Field(t =&gt; t.Lang)
        )
    )
);
</pre><p>So what we&apos;ve done here is used the <a href="https://www.elastic.co/guide/en/elasticsearch/reference/master/put-pipeline-api.html">put pipeline API</a>&#xA0;to create a pipeline with the id &quot;tweet-pipeline&quot; that has two processors. A <a href="https://www.elastic.co/guide/en/elasticsearch/reference/master/convert-processor.html">convert&#xA0;processor</a> for converting <code>retweets</code> from <code>string</code> to <code>integer</code>, and an <a href="https://www.elastic.co/guide/en/elasticsearch/reference/master/uppercase-processor.html">uppercase&#xA0;processor</a> for, you guessed it, upper-casing the value of our <code>lang</code> field.
</p><h2>Indexing documents</h2><p>Now that we&apos;ve created our pipeline, let&apos;s index some documents using the bulk API and enrich them through the pipeline.
</p><pre class="prettyprint">client.Bulk(b =&gt; b
    .Index(&quot;tweets&quot;)
    .Pipeline(&quot;tweet-pipeline&quot;)
    .Index&lt;Tweet&gt;(i =&gt; i
        .Document(new Tweet { Retweets = &quot;4&quot;, Message = &quot;Hello, Twitter!&quot;, Lang = &quot;en&quot; })
    )
    .Index&lt;Tweet&gt;(i =&gt; i
        .Document(new Tweet { Retweets = &quot;32&quot;, Message = &quot;Bonjour, Twitter!&quot;, Lang = &quot;fr&quot; })
    )
    .Index&lt;Tweet&gt;(i =&gt; i
        .Document(new Tweet { Retweets = &quot;&quot;, Message = &quot;Hallo, Twitter !&quot;, Lang = &quot;nl&quot; })
    )
);
</pre><p>Business as usual, except notice we specified a pipeline.&#xA0;This tells Elasticsearch we want to pre-process each document using the &quot;tweets-pipeline&quot; we created earlier before indexing. Here we specified the pipeline for all index commands, but we could specify a different pipeline for each individual index command if we had multiple pipelines.
</p><h2>Handling errors</h2><p>If we inspect the response from our bulk request:
</p><pre class="prettyprint">{
  &quot;took&quot; : 33,
  &quot;ingest_took&quot; : 4,
  &quot;errors&quot; : true,
  &quot;items&quot; : [ {
    &quot;index&quot; : {
      &quot;_index&quot; : &quot;tweets&quot;,
      &quot;_type&quot; : &quot;tweet&quot;,
      &quot;_id&quot; : &quot;AVSl-cCKVD5bKRQTTXNo&quot;,
      &quot;_version&quot; : 1,
      &quot;_shards&quot; : {
        &quot;total&quot; : 2,
        &quot;successful&quot; : 1,
        &quot;failed&quot; : 0
      },
      &quot;created&quot; : true,
      &quot;status&quot; : 201
    }
  }, {
    &quot;index&quot; : {
      &quot;_index&quot; : &quot;tweets&quot;,
      &quot;_type&quot; : &quot;tweet&quot;,
      &quot;_id&quot; : &quot;AVSl-cCKVD5bKRQTTXNp&quot;,
      &quot;_version&quot; : 1,
      &quot;_shards&quot; : {
        &quot;total&quot; : 2,
        &quot;successful&quot; : 1,
        &quot;failed&quot; : 0
      },
      &quot;created&quot; : true,
      &quot;status&quot; : 201
    }
  }, {
    &quot;index&quot; : {
      &quot;_index&quot; : &quot;tweets&quot;,
      &quot;_type&quot; : &quot;tweet&quot;,
      &quot;_id&quot; : null,
      &quot;status&quot; : 400,
      &quot;error&quot; : {
        &quot;type&quot; : &quot;illegal_argument_exception&quot;,
        &quot;reason&quot; : &quot;unable to convert [] to integer&quot;,
        &quot;caused_by&quot; : {
          &quot;type&quot; : &quot;number_format_exception&quot;,
          &quot;reason&quot; : &quot;For input string: \&quot;\&quot;&quot;
        }
      }
    }
  } ]
}
</pre><p>We&apos;ll notice that our last document failed.&#xA0;Why? Well if we take a look at the document we tried to index we&apos;ll notice that the value <code>Retweets</code> was an empty string and our convert processor didn&apos;t know what to do with it.
</p><p>We can address this by telling our processor what to do when it encounters a failure.&#xA0;Each processor has an <code>on_failure</code> property, which accepts more processors that it will execute when an error occurs. These nested processors themselves also have an <code>on_failure</code> property which you can further nest error handling, and so on and so forth.&#xA0;Also, the pipeline itself has its&apos; own <code>on_failure</code> which you can set as a &quot;catch all&quot; error handler for the entire pipeline.
</p><p>For our <code>tweet-pipeline</code> example, let&apos;s just update our <code>convert</code> processor and add a <a href="https://www.elastic.co/guide/en/elasticsearch/reference/master/set-processor.html">set processor</a>&#xA0;to <code>on_failure</code> that will set <code>retweets</code> to <code>0</code> if it fails to convert the original value.
</p><pre class="prettyprint"> client.PutPipeline(&quot;tweet-pipeline&quot;, p =&gt; p
    .Processors(ps =&gt; ps
        .Uppercase&lt;Tweet&gt;(u =&gt; u
            .Field(twt =&gt; t.Lang)
        )
        .Convert&lt;Tweet&gt;(c =&gt; c
            .Field(twt =&gt; twt.Retweets)
            .Type(ConvertProcessorType.Integer)
            .OnFailure(f =&gt; f
                .Set&lt;Tweet&gt;(s =&gt; s
                    .Field(t =&gt; t.Retweets)
                    .Value(0)
                )
            )
        )
    )
);
</pre><p>If we index the document again (this time just using the index API), we&apos;ll get a <code>201</code> back from Elasticsearch:
</p><pre class="prettyprint">client.Index(new Tweet { Retweets = &quot;&quot;, Message = &quot;Hallo, Twitter !&quot;, Lang = &quot;nl&quot; }, i =&gt; i
    .Index(&quot;tweets&quot;)
    .Pipeline(&quot;tweet-pipeline&quot;)
);
</pre><p>Now let&apos;s take a look at the documents in our index:
</p><pre class="prettyprint">&quot;hits&quot;: [
  {
    &quot;_index&quot;: &quot;tweets&quot;,
    &quot;_type&quot;: &quot;tweet&quot;,
    &quot;_id&quot;: &quot;AVSmDI7jVD5bKRQTTXin&quot;,
    &quot;_score&quot;: 1,
    &quot;_source&quot;: {
      &quot;retweets&quot;: 32,
      &quot;message&quot;: &quot;Bonjour, Twitter!&quot;,
      &quot;lang&quot;: &quot;FR&quot;
    }
  },
  {
    &quot;_index&quot;: &quot;tweets&quot;,
    &quot;_type&quot;: &quot;tweet&quot;,
    &quot;_id&quot;: &quot;AVSmDJB6VD5bKRQTTXio&quot;,
    &quot;_score&quot;: 1,
    &quot;_source&quot;: {
      &quot;retweets&quot;: 0,
      &quot;message&quot;: &quot;Hallo, Twitter !&quot;,
      &quot;lang&quot;: &quot;NL&quot;
    }
  },
  {
    &quot;_index&quot;: &quot;tweets&quot;,
    &quot;_type&quot;: &quot;tweet&quot;,
    &quot;_id&quot;: &quot;AVSmDI7jVD5bKRQTTXim&quot;,
    &quot;_score&quot;: 1,
    &quot;_source&quot;: {
      &quot;retweets&quot;: 4,
      &quot;message&quot;: &quot;Hello, Twitter!&quot;,
      &quot;lang&quot;: &quot;EN&quot;
    }
  }
]
</pre><p>All of our language codes are uppercase and retweets for our <code>NL</code> has been set to <code>0</code>!
</p><p>That is &#x2014; in a nutshell &#x2014;&#xA0;ingest node.
</p><h2>Client considerations</h2><h4>Dedicated ingest nodes</h4><p>Since Elasticsearch will automatically reroute ingest requests to ingest nodes, you don&apos;t have to specify or configure any routing information. However, if you&apos;re doing heavy ingestion and have dedicated ingest nodes, it makes sense to send index requests to these nodes directly, to avoid any extra hops in the cluster.
</p><p>The simplest way to achieve this with any Elasticsearch client is to create a dedicated &quot;indexing&quot; client instance, and use it for indexing requests:
</p><pre class="prettyprint">var pool = new StaticConnectionPool(new []
{
    new Uri(&quot;http://ingestnode1:9200&quot;),
    new Uri(&quot;http://ingestnode2:9200&quot;),
    new Uri(&quot;http://ingestnode3:9200&quot;)
});
var settings = new ConnectionSettings(pool);
var indexingClient = new ElasticClient(settings);
</pre><h4>Increasing timeouts</h4><p>When a pipeline is specified, there is an added overhead of document enrichment when indexing a document. For large bulk requests, you might need to increase the default indexing timeout (<code>1m</code>) to avoid exceptions.
</p><p>Keep in mind, that the client may have its own request timeout &#x2014;&#xA0;this should be increased as well, at least to the same value as the Elasticsearch timeout.
</p><pre class="prettyprint">client.Bulk(b =&gt; b
    .Index(&quot;tweets&quot;)
    .Pipeline(&quot;tweet-pipeline&quot;)
    .Timeout(&quot;5m&quot;) // Increases the bulk timeout to 5 minutes
    .Index&lt;Tweet&gt;(/*snip*/)
    .Index&lt;Tweet&gt;(/*snip*/)
    .Index&lt;Tweet&gt;(/*snip*/)
    .RequestConfiguration(rc =&gt; rc
        .RequestTimeout(TimeSpan.FromMinutes(5)) // Increases the request timeout to 5 minutes
    )
);
</pre><h2>Conclusion</h2><p>In this post I&apos;ve covered the basics of the new ingest node feature coming in Elasticsearch 5.0 and how to consume the ingest APIs from an Elasticsearch language client.  I&apos;ve only scratched the surface though, so I highly recommend <a href="https://www.elastic.co/guide/en/elasticsearch/reference/master/ingest.html">reading the docs</a> and watching the <a href="https://www.elastic.co/elasticon/conf/2016/sf/ingest-node-enriching-documents-within-elasticsearch">Ingest Node: Enriching Documents within Elasticsearch</a> talk from Elastic{ON}16.  NEST supports the ingest node APIs since the <a href="https://github.com/elastic/elasticsearch-net/releases/tag/5.0.0-alpha1">5.0.0-alpha1</a> release and will work against any Elasticsearch 5.0.0 alpha release.  Try it out and <a href="https://github.com/elastic/elasticsearch-net/issues">let us know</a> what you think!
</p></body></html>

nest-843231_1280.jpg

Ingest Node: A Client's Perspective

<html><head></head><body><p>This marks the first GA release of our 2.0 client with well over a 1000 commits since 1.7.1 (the currently last released NEST version in the 1.x range).
</p><h2>Back to the drawing board</h2><p>We took some time to go back to the drawing board. NEST was originally started in 2010 and there are many choices that have accumulated in the code base that don&apos;t make sense anymore. So we stepped back to properly formalize how we see the lifetime of a call and worked off of that. Armed with the following diagram, we completely rewrote NEST&apos;s internals; The old <a href="https://msdn.microsoft.com/en-us/library/dd460717.aspx">Task Parallel Library (TPL)</a> based code is now replaced with <a href="https://msdn.microsoft.com/en-us/library/hh191443.aspx">async/await</a> with a much saner approach to exceptions and errors in addition to exposing enough information as an audit trail so you don&apos;t ever have to guess what went down during a call.
</p><p><img src="https://api.contentstack.io/v2/assets/575e4d2b3dc542cb38c09d96/download?uid=bltec66d884d4f8e393?uid=bltec66d884d4f8e393" alt="Request pipeline in NEST 2.0" style="max-width:100%;">
</p><p>Our internals now also reflect this:
</p><ul>
	<li><code>IElasticClient</code> exposes all the Elasticsearch API endpoints e.g <code>client.Search</code> this calls into <code>ITransport</code>&apos;s 2 methods <code>Request</code> and <code>RequestAsync</code> </li>
	<li>the default <code>ITransport</code> uses the passed in <code>IRequestPipelineFactory</code> to create an <code>RequestPipeline</code> which implements <code>IPipeline</code>.</li>
</ul><p>This pipeline now handles all of the failover/sniffing/pinging logic and directly reflects the flow diagram.
</p><p>We also simplified <code>IConnection</code> down just 2 methods. This means the outer edges are clean (<code>ITransport</code> and <code>IConnection</code>) and implementing your own should be really really simple. All of these (and also <code>IMemoryStreamProvider</code> and <code>IDateTimeProvider</code>) can be injected on the constructor of the client.
</p><h2>Test Framework</h2><p>Another huge endeavour is the rework of our test framework. NEST 1.x was always well tested but used 5 different test projects and 5 years worth of changing our minds as how best to write tests and assertions, thus becoming a big hodgepodge of <a href="http://www.nunit.org/">NUnit assertions</a>, <a href="http://www.fluentassertions.com/">Fluent assertions</a>, <a href="https://fakeiteasy.github.io/">FakeItEasy</a>, <a href="http://www.moqthis.com/">Moq</a> combined with several different ways to compare json with object graphs and vice-versa. Trying to write a new test quickly became <a href="http://www.hanselman.com/blog/YakShavingDefinedIllGetThatDoneAsSoonAsIShaveThisYak.aspx">an exercise in yak shaving</a> because there was no clear cut way for how best to write said test.
</p><p>So the first thing we did as part of our 2.0 branch was to completely delete all of our tests. This <em>act of insanity</em> gave us carte blanche during our rewrite.
</p><p>As of 2.0, we have one test project, <strong>Tests</strong>, with all tests written in such a way that they can be run in unit test mode and integration test mode. <em>Write once, run differently</em>.
All of the API endpoint tests test all four request variations - two DSL&apos;s (using fluent and object initializer syntax) with both synchronous and asynchronous variations of each. We also test all of the moving parts of the Elasticsearch DSL (Aggregations, Sorting, IndexSettings, etc) in the same way.
</p><p>In addition to the more formal unit and integration tests, we also implemented a thing we dubbed <em>Literate Testing</em> to allow us to write tests in a more story telling form, with multi-line comments serving as the narrative for our asciidoc documentation while using the <a href="https://github.com/dotnet/roslyn/wiki/Roslyn%20Overview">Roslyn compiler</a> to pick out the interesting bits of code. This gives us the benefit of always compiling our documentation in addition to having only one place where we document, test and assert how a piece of code is supposed to work.
</p><p>Another huge component of our testing framework is the <em>Virtual Cluster</em> that allows us to write tests for any situation and how we expect the client to behave.
</p><p>For example:
</p><pre class="prettyprint">/** we set up a 10 node cluster with a global request time out of 20 seconds.
* Each call on a node takes 10 seconds. So we can only try this call on 2 nodes
* before the max request time out kills the client call.
*/
var audit = new Auditor(() =&gt; Framework.Cluster
  .Nodes(10)
  .ClientCalls(r =&gt; r.FailAlways().Takes(TimeSpan.FromSeconds(10)))
  .ClientCalls(r =&gt; r.OnPort(9209).SucceedAlways())
  .StaticConnectionPool()
  .Settings(s =&gt; s.DisablePing().RequestTimeout(TimeSpan.FromSeconds(20)))
);
audit = await audit.TraceCalls(
  new ClientCall {
    { BadResponse, 9200 }, //10 seconds
    { BadResponse, 9201 }, //20 seconds
    { MaxTimeoutReached }
  },
  /**
  * On the second client call we specify a request timeout override to 80 seconds
  * We should now see more nodes being tried.
  */
  new ClientCall(r =&gt; r.RequestTimeout(TimeSpan.FromSeconds(80)))
  {
    { BadResponse, 9203 }, //10 seconds
    { BadResponse, 9204 }, //20 seconds
    { BadResponse, 9205 }, //30 seconds
    { BadResponse, 9206 }, //40 seconds
    { BadResponse, 9207 }, //50 seconds
    { BadResponse, 9208 }, //60 seconds
    { HealthyResponse, 9209 },
  }
);
</pre><p>This showcases the <em>Virtual Cluster</em> tests combined with <em>Literate Tests</em> and the extensive audit trail information available on each response (or exception).
</p><p>I&apos;m pleased to say we are back at a decent coverage rate (60%) and will continue to iterate and improve this.
</p><h2>Exception handling</h2><p>Another big change in NEST 2.0 is how we deal with exceptions.
</p><p>In NEST 1.x, the client threw a multitude of exceptions: <code>MaxRetryException</code>, <code>ElasticsearchAuthException</code>, <code>ElasticsearchServerException</code>, <code>DslException</code>, etc.. This made it challenging for users to handle exceptions and invalid responses, and understand the root cause of errors. On top of that, the types of exceptions thrown depended on what kind of <code>IConnectionPool</code> was injected, in order to maintain maximum backwards compatibility with NEST 0.x.
</p><p>In NEST 2.x, exceptions are much more deterministic.  The former <code>ThrowOnElasticsearchServerExceptions()</code> setting has been replaced with the more succinct <code>ThrowExceptions()</code>, which determines whether the client should ever throw an exception or not (client side and server exceptions). Furthermore, the types of exceptions have been reduced and simplified down to three types of exceptions:
</p><p><strong>ElasticsearchClientException:</strong> These are <strong>known</strong> exceptions, either an exception that occurred in the request pipeline(such as max retries or timeout reached, bad authentication, etc...) or Elasticsearch itself returned an error (could not parse the request, bad query, missing field, etc...). If it is an Elasticsearch error, the <code>ServerError</code> property on the response will contain the the actual error that was returned.  The inner exception will always contain the root causing exception.
</p><p><strong>UnexpectedElasticsearchClientException:</strong>  These are <strong>unknown</strong> exceptions, for instance a response from Elasticsearch not properly deserialized.  These are usually bugs in the client and <a href="https://github.com/elastic/elasticsearch-net/issues">we encourage you to report them</a>.  This exception also inherits from <code>ElasticsearchClientException</code> so an additional catch block isn&apos;t necessary to handle but can be helpful in distinguishing between the two.
</p><p><strong>Runtime exceptions:</strong> These are CLR exceptions like <code>ArgumentException</code>, <code>NullArgumentException</code> etc. that are thrown when an API in the client is misused.
</p><h1>Breaking Changes</h1><p>Even though a lot of work went into the interior, the exterior did not escape unscathed! On top of the many breaking changes that Elasticsearch 2.0 introduces, there are more then a few NEST 2.0 introduces.
We revalidated all the request and response domain objects against Elasticsearch 2.0.
</p><p>A pretty complete list of breaking changes are available:
</p><ul>
	<li><a href="https://github.com/elastic/elasticsearch-net/blob/master/docs/2.0-breaking-changes/elasticsearch-net-breaking-changes.md">Low level client changes, Elasticsearch.NET</a></li>
	<li><a href="https://github.com/elastic/elasticsearch-net/blob/master/docs/2.0-breaking-changes/nest-breaking-changes.md">High level client changes, NEST</a></li>
</ul><h2>Elasticsearch 2.x support</h2><p>NEST 2.0 supports all the new features in Elasticsearch 2.0 including pipeline aggregations. <a href="https://www.elastic.co/blog/elasticsearch-2-2-0-and-2-1-2-and-1-7-5-released">New features from Elasticsearch 2.2</a> have not yet been mapped.
</p><p>Here we&apos;ll just highlight a couple features that are reflected in the NEST changes to whet your appetite!
</p><h2>Filters are <strong>Gone Gone Gone</strong></h2><p>In Elasticsearch 2.0, <a href="https://www.elastic.co/guide/en/elasticsearch/reference/current/breaking_20_query_dsl_changes.html#_queries_and_filters_merged">query and filter constructs have merged into one concept called queries</a> and NEST 2.0 reflects this. So if you were previously using the <code>AndFilter</code>, this is now&#xA0;the <code>AndQuery</code>; beware however&#xA0;that some of these filters have been obsoleted and chances are high <a href="https://www.elastic.co/blog/all-about-elasticsearch-filter-bitsets">you were using them wrong!</a>.
</p><h2>Isolated descriptors</h2><p>In NEST 1.x we took pride in being a 1-to-1 mapping with the Elasticsearch API. In some cases however, this <em>purity</em> hid the real depth of parameters. As an example,
</p><p>in NEST 1.x you could add sorts on <code>Search()</code> using:
</p><pre class="prettyprint">client.Search&lt;Project&gt;(s=&gt;s
  .SortAscending(...)
  .SortScript(...)
)
</pre><p>in NEST 2.0 you have to stoop down a level first in order to access the same functionality
</p><pre class="prettyprint">client.Search&lt;Project&gt;(s=&gt;s
  .Sort(ss=&gt;ss
    .Field()
    .Script()
    .Ascending()
    .GeoDistance()
  )
)
</pre><p>This encapsulates all of the sort options properly and adheres stricter to the 1-to-1 mapping. NEST 1.x did also contain this full descriptor however the mix and matching
of the convenience methods of the parent meant that some fluent methods were additive whilst others overwrote what was previously set. With NEST 2.0, this discrepancy is gone!
</p><p>This happens in more places e.g index settings and mappings.
</p><h2>Filtered query deprecation</h2><p>With the removal of filters, NEST has added a special construct in its Query DSL to easily create a <a href="https://www.elastic.co/guide/en/elasticsearch/reference/2.0/query-dsl-bool-query.html">bool query</a> with a filter clause
</p><pre class="prettyprint">.Query(q=&gt; +q.Term(p=&gt;p.Name, &quot;NEST&quot;))
</pre><p>the <code>+</code> will cause&#xA0;the term query to be wrapped inside a bool query&apos;s <code>filter</code> clause
</p><p>You can even combine <code>+</code> with <code>!</code>
</p><pre class="prettyprint">.Query(q=&gt; !+q.Term(p=&gt;p.Name, &quot;NEST&quot;))
</pre><p>This will wrap the term query inside a <code>bool filter</code> and subsequently inside a <code>bool must_not</code>&#xA0;clause.&#xA0;This approach also works with the object initializer syntax (OIS)
</p><pre class="prettyprint">!+new TermQuery {}
</pre><h2>Attribute based mapping</h2><p>The single <code>ElasticPropertyAttribute</code> has been broken up into individual attributes per property type.
</p><p>For instance, the following:
</p><pre class="prettyprint">[ElasticType(Name = &quot;othername&quot;, IdProperty = &quot;MyId&quot;)]
public class Foo
{
  [ElasticProperty(Type = FieldType.String)]
  public Guid MyId { get; set; }
  [ElasticProperty(Type = FieldType.String)]
  public string Name { get; set; }
  [ElasticProperty(Type = FieldType.String, Analyzer = &quot;myanalyzer&quot;, TermVector = TermVectorOption.WithOffsets)]
  public string Description { get; set; }
  [ElasticProperty(Type = FieldType.Date, Format = &quot;mmmddyyyy&quot;)]
  public DateTime Date { get; set; }
  [ElasticProperty(Type = FieldType.Integer, Coerce = true)]
  public int Number { get; set; }
  [ElasticProperty(Type = FieldType.Nested, IncludeInParent = true)]
  public List&lt;Bar&gt; Bars { get; set; }
}
</pre><p>becomes
</p><pre class="prettyprint">[ElasticsearchType(Name = &quot;othername&quot;, IdProperty = &quot;MyId&quot;)]
public class Foo
{
  [String]
  public Guid MyId { get; set; }
  [String]
  public string Name { get; set; }
  [String(Analyzer = &quot;myanalyzer&quot;, TermVector = TermVectorOption.WithOffsets)]
  public string Description { get; set; }
  [Date(Format = &quot;mmddyyyy&quot;)]
  public DateTime Date { get; set; }
  [Number(NumberType.Integer, Coerce = true, DocValues = true)]
  public int Number { get; set; }
  [Nested(IncludeInParent = true)]
  public List&lt;Bar&gt; Bars { get; set; }
}
</pre><p>Aside from a simpler and cleaner API, this allows each attribute to only reflect the options that are available for the particular type instead of exposing options that may not be relevant (as <code>ElasticPropertyAttribute</code> did).
</p><h2>Inferred types</h2><p>Many places that only took a string or primitive type now take a more strongly typed object such as <code>Id</code>, <code>Field</code>, <code>Fields</code>, <code>Index</code>, <code>Indices</code>, <code>Type</code>, <code>Types</code>, <code>DocumentPath&lt;T&gt;</code>, etc. It&apos;s good to know that in most cases you can still pass a string or primitive type and it will be implicitly converted to the type where it makes sense.
</p><p>If you are using <code>C# 6</code> you can also statically import the <code>Infer</code> static class using <code>using static Nest.Infer</code>, allowing to write
</p><ul>
	<li><code>Field&lt;Project&gt;(p =&gt; p.Name)</code> any where that takes a <code>Field</code></li>
	<li><code>Field&lt;Project&gt;(p =&gt; p.Name).And(p=&gt;p.Description).And(&quot;field&quot;)</code> anywhere that takes <code>Fields</code></li>
	<li><code>Index&lt;Project&gt;()</code> for <code>Index</code></li>
</ul><p>so on and so forth. If you are using the fluent API using these infer methods is not required since the fluent API is strongly typed through lambda expressions, but they are another tool at your disposal nonetheless.
</p><h2>C# 6 support</h2><p>NEST&apos;s codebase has been largely rewritten to take advantage of all the cool new c# features making almost all the fluent code one liners
</p><pre class="prettyprint">using static Nest.Infer;
//later..
Field&lt;Project&gt;(p =&gt; p.Name);
Index&lt;Project&gt;();
Indices&lt;Project&gt;().And&lt;Developer&gt;();
</pre><h2>DNX Support</h2><p>The 2.0 release has been released with a <code>dotnet5.1</code> version on nuget that can be used on both the Desktop and Core CLR runtimes of DNX rc1. We are actively tracking <a href="https://github.com/aspnet/Home/wiki/Roadmap">rc2</a>, the <a href="https://github.com/dotnet/cli">.NET Commandline interface (dotnet-cli)</a> and the new <code>netstandard</code> NuGet target framework, with a plan to release compatible packages once these hit the scene.
</p><h2>Feedback</h2><p>We&apos;d like to thank everyone who took the alpha and rc releases out for a spin and provided invaluable input while we incubated the 2.0 release. A special shoutout to <a href="https://github.com/niemyjski">Blake Niemyjski</a> for sharing his screen with us so we could see firsthand where the pain points were during an upgrade from 1.x to 2.0.
</p><p>As always we very much welcome all feedback on <a href="https://github.com/elastic/elasticsearch-net/issues">our github issues page.</a>
</p></body></html>

birds-nest.jpg

GA Release of NEST 2.0, our .NET client for Elasticsearch

Start free trial

Blog Footer CTA

Sign up for Elastic Cloud free trial

Greg_Marzouka.jpg

Greg Marzouka

<p>Надсилаючи цю форму ви погоджуєтеся з <a href="/legal/terms-of-use">Умовами надання послуг Elastic</a>. Ваші особисті дані будуть опрацьовані відповідно з <a href="/legal/privacy-statement">Положеннями про конфіденційність Elastic</a>.</p>

Newsletter GDPR Text - Ukrainian

<p>By submitting you acknowledge that you've read and agree to our <a href="/legal/elastic-cloud-account-terms" target="_self">Terms of Service</a>, and you agree to receive the <a href="/legal/privacy-statement#how-we-use-the-information" target="_self">selected communications</a> at the contact details you provided above. See <a href="/legal/privacy-statement/" target="_self">Elastic’s Privacy Statement</a> for more details or to opt-out at any time.</p>

Communication Preferences - ONLY for /communication-preferences

<p>By submitting you agree to <a href="/agreements/global/cloud-services-monthly" rel="nofollow">Elastic Cloud Terms of Service</a>. Your personal data will be processed in accordance with <a href="/legal/privacy-statement">Elastic's Privacy Statement</a>.</p>

Cloud Trial GDPR Text

Intentionally Blank

<p>By submitting you acknowledge that you've read and agree to our <a href="/legal/serverless-preview-terms" target="_blank">Terms of Service</a>, and that Elastic may <a href="/legal/privacy-statement#how-we-use-the-information" target="_blank">contact you</a> about our related products and services, using the details you provide above. See <a href="/legal/privacy-statement/" target="_blank">Elastic’s Privacy Statement</a> for more details or to opt-out at any time.</p>

Serverless GDPR Text

<p>提交即表示您確認已詳閱並同意我們的<a href="/legal/elastic-cloud-account-terms" target="_blank">服務條款</a>，且同意 Elastic 使用您在上方提供的個人資料<a href="/legal/privacy-statement#how-we-use-the-information" target="_blank">與您聯絡</a>，為您提供相關產品和服務的資訊。如需詳細說明或隨時想停止收到產品和服務資訊，請參閱 <a href="/legal/privacy-statement/" target="_blank">Elastic 的隱私聲明</a>。</p>

Newsletter GDPR Text - TW

<p>By submitting you acknowledge that you've read and agree to our <a href="/legal/elastic-cloud-account-terms" target="_blank">Terms of Service</a>, and that Elastic may <a href="/legal/privacy-statement#how-we-use-the-information" target="_blank">contact you</a> about our related products and services, using the details you provide above. See <a href="/legal/privacy-statement/" target="_blank">Elastic’s Privacy Statement</a> for more details or to opt-out at any time.</p><p>By submitting this form, I agree to Elastic sharing my contact information with Google Cloud. I also agree to Elastic giving my information to the vendor for the purpose of shipping my food package.</p>

GDPR Text + Google Cloud

<p>Inviando la registrazione (1) esprimi il consenso ai <a href="/legal/terms-of-use">termini e condizioni di Elastic</a> e <a href="/legal/privacy-statement">all’informativa sulla privacy</a>, nonché (2) a ricevere saltuarie e-mail.</p>

Newsletter GDPR Text - Italian

<p>By submitting you acknowledge that you've read and agree to our <a href="/legal/terms-of-use" target="_blank">Terms of Service</a>, and that Elastic may <a href="/legal/privacy-statement#how-we-use-the-information" target="_blank">contact you</a> about our related products and services, using the details you provide above. See <a href="/legal/privacy-statement/" target="_blank">Elastic’s Privacy Statement</a> for more details or to opt-out at any time.</p>

Newsletter GDPR Text

Load more

Overview

Can't make it? Register and we'll send you the recording. You'll also receive an email with related content

Can't make it? Register and we'll send you the recording. You'll also receive an email with related content.

You'll also receive an email with related content.

You'll also receive an email with related content

Speakers

Close

See more insights

The content on this page is not available in the selected language. As Elastic grows globally, we continue to support content in multiple languages.

Author

Articles by Greg Marzouka

Generating an Elastic Cloud Enterprise Client

Ingest Node: A Client's Perspective

GA Release of NEST 2.0, our .NET client for Elasticsearch

Follow us

About us

Join us

Partners

Trust & Security

Investor relations

Excellence Awards