Serverless computing, or Function-as-a-Service (FaaS), has been a trending architecture pattern in many cloud deployments, often as a complement to existing container, VM, and cloud infrastructures. It enables applications to be run as functions on demand, further simplifying deployments when long-running infrastructure isn’t required, and potentially reducing operational cost with users only being charged when functions are being executed.With more workloads moving into the cloud, it has become increasingly important to harness the ability to easily ingest cloud data in a scalable, reliable, and cost efficient fashion. Serverless compute services like AWS Lambda, Azure Functions, and GCP Functions enable users to take the “Ops” out of DevOps, and the introduction of <a href="https://www.elastic.co/products/beats/functionbeat">Functionbeat</a> now further takes the “Dev” part out as well, leaving you with a notably simple experience for ingesting cloud data into Elasticsearch or <a href="https://www.elastic.co/cloud">Elastic Cloud</a>. The <a href="https://www.elastic.co/cloud/elasticsearch-service">Elasticsearch Service</a> in Elastic Cloud is the only hosted Elasticsearch offering that is built and run by Elastic, the company behind Elasticsearch.<h2>Introducing Functionbeat</h2>Functionbeat is a new addition to the <a href="https://www.elastic.co/products/beats">Beats product suite</a> that can be easily deployed as a function in serverless compute platforms, providing everything you need to configure, deploy, and transform events like logs and metrics coming from your cloud infrastructure. Available as of 6.5 as a beta, it currently supports AWS Lambda and a subset of data sources, namely AWS Cloudwatch Logs and Simple Queue Service (SQS). We have plans to expand support across more cloud providers and data sources as we move forward. Functionbeat is offered under the <a href="https://www.elastic.co/subscriptions">Elastic Basic license</a> which means its freely available for all end users.<img src="https://images.contentstack.io/v3/assets/bltefdd0b53724fa2ce/blt22fcb021761f0796/5c304739682c4c4866a7980e/functionbeat-1.png" data-sys-asset-uid="blt22fcb021761f0796" style="display: block;margin: auto;"/><h3>Ingest from Cloud Data Sources</h3>With flexibility in mind, Functionbeat opens up endless opportunities to effectively ingest, transform, enrich, and ship your cloud data to Elasticsearch. Here are two use cases in which you may leverage Functionbeat to accomplish:<ul><li><a href="https://www.elastic.co/observability/aws-monitoring" target="_self">AWS monitoring</a> with the Elastic Stack - Functionbeat can receive events from a Cloudwatch Log group, extract relevant fields with the <a href="https://www.elastic.co/guide/en/beats/functionbeat/6.5/dissect.html">dissect processor</a> to structure the event, and <a href="https://www.elastic.co/guide/en/beats/functionbeat/6.5/drop-event.html">apply filtering</a> prior to shipping to an <a href="https://www.elastic.co/guide/en/elasticsearch/reference/6.5/ingest.html">Ingest Node</a> pipeline or directly to Elasticsearch.</li><li>Analyze Application Data from SQS with the Elastic Stack - for applications sending JSON encoded events to an SQS queue, Functionbeat can be used to listen, ingest, and <a href="https://www.elastic.co/guide/en/beats/functionbeat/6.5/decode-json-fields.html">decode the JSON event</a> prior to shipping to Elasticsearch.</li></ul><h3>Define Processing Logic without Code</h3>At the core, Functionbeat operates like any other Beat, using the same underlying framework as <a href="https://www.elastic.co/products/beats/filebeat">Filebeat</a> and <a href="https://www.elastic.co/products/beats/metricbeat">Metricbeat</a>, and exposing similar functionality that you’ve already come to know and love. This means that you can add fields to your events, use <a href="https://www.elastic.co/guide/en/beats/functionbeat/6.5/defining-processors.html#conditions">conditionals</a> to drop or route events, or use any <a href="https://www.elastic.co/guide/en/beats/functionbeat/6.5/defining-processors.html">processors</a> to manipulate or enrich your data before sending it to Elasticsearch. What’s great is that all of this can be achieved through a configuration style experience without the need to actually write and compile code!<h2>Ingesting Cloudwatch Logs in the Real World</h2>Let’s say we want to send a subset of log messages from Cloudwatch Logs to our Elasticsearch cluster, and to keep the event count reasonable, we are only interested in messages containing the ERROR keyword. We edit the functionbeat.yml and add the following function configuration for a mylogs Lambda function that will be triggered when new events are generated from the Cloudwatch Logs group /aws/lambda/test matching the ERROR pattern:<pre class="prettyprint">functionbeat.provider.aws: deploy_bucket: myappdeploybucket functions: - name: mylogs enabled: true type: cloudwatch_logs concurrency: 5 description: "A really special lambda" fields: level: "error" triggers: - log_group_name: /aws/lambda/test filter_pattern: "ERROR" output.elasticsearch: enabled: true hosts: ["localhost:9200"] username: "elastic" password: "changeme" </pre>We can now deploy that function to AWS Lambda, but before proceeding we should make sure the AWS environment variables are correctly configured. The function can be pushed into Lambda using the <a href="https://www.elastic.co/guide/en/beats/functionbeat/6.5/command-line-options.html#deploy-command">deploy</a> subcommand:<pre class="prettyprint">./functionbeat deploy mylogs </pre><img src="https://images.contentstack.io/v3/assets/bltefdd0b53724fa2ce/blt7dcd99036ad35eea/5c3047377c04885167e04da1/functionbeat-2.png" data-sys-asset-uid="blt7dcd99036ad35eea" style="display: block;margin: auto;"/><img src="https://images.contentstack.io/v3/assets/bltefdd0b53724fa2ce/blta3ef73adf7f267e5/5c3047355a7621286e3e904b/functionbeat-3.png" data-sys-asset-uid="blta3ef73adf7f267e5"/>After the Lambda function is deployed, new Cloudwatch Logs events added to the log group that match the ERROR pattern will be automatically ingested and available in our Elasticsearch cluster. If this is not the case, we can look up the Functionbeat logs in our Cloudwatch Logs panel to debug any issues. Functions can be updated or completely removed from Lambda at anytime using the Functionbeat CLI <a href="https://www.elastic.co/guide/en/beats/functionbeat/6.5/command-line-options.html#update-command">update</a> and <a href="https://www.elastic.co/guide/en/beats/functionbeat/6.5/command-line-options.html#remove-command">remove</a> subcommands.<h2>Scaling Concurrency</h2>The unit of scale in AWS Lambda is the concurrency and a hard limit of 1000 is used for the concurrency by default. This limit is account wide and represents the maximum number of functions of any kind that can be executed concurrently.The Lambda service will autoscale your functions based on the load and concurrency levels configured for a function and is a good way to control cost and regulate the time it takes to process and execute this work. By default, Functionbeat will use a conservative setting of 5 as the concurrency level for any function it creates, but this can be tuned to more strongly fit the requirements of your use case and scale.<h2>Let’s Get the Party Started!</h2>Now that we understand what Functionbeat is and the problems it solves, we encourage you to go try it out for your own serverless ingest use cases! Although it’s initially been released in beta, we’re actively working towards making it GA production ready. In the meantime, we’re curious to hear your feedback, so please <a href="https://github.com/elastic/beats/issues/new">submit a Github issue</a> (all our code is open!) or post your thoughts in the <a href="https://discuss.elastic.co/c/beats/functionbeat">Functionbeat forum</a>. Check out the <a href="https://www.elastic.co/guide/en/beats/functionbeat/6.5/functionbeat-overview.html">Functionbeat documentation</a> to get started!

Functionbeat: Serverless Ingestion for Elasticsearch

<html><head></head><body>Logstash is a swiss army knife for event ingestion and transformation. You can configure it to stream and transform multiple data sources into Elasticsearch. You can then visualize this data easily with Kibana.&#xA0;
Logstash can also send to more outputs than just Elasticsearch -- we support over 70 outputs. In this post, we will be talking about the <a href="https://www.elastic.co/guide/en/logstash/current/plugins-outputs-s3.html">S3 output</a> which allows you to store your events to an S3 bucket for long term archival or for batch processing at a later time. Logstash supports multiple outputs; so you can send data to Elasticsearch as well as archive it to S3. Using conditional logic, you can even route certain streams of data you want sent to S3. If you want more control on the created S3 structure you can use event data and the field reference syntax to dynamic create the S3 prefixes. We will go in more details in the following examples.
<h3>A &quot;Hello World&quot; S3 archival
</h3>Before we dive into a complex example, a simple way to archive your data to S3. Just define a S3 block in the output section of your configuration, and all your data will be archived.
<pre class="prettyprint">input {
 beats {
 port =&gt; 5544
 }
}
output {
 s3 {
 access_key_id =&gt; &quot;SECRET&quot;
 secret_access_key =&gt; &quot;SECRET&quot;
 bucket =&gt; &quot;mybucket&quot;
 codec =&gt; json_lines
 }
 elasticsearch { }
}
</pre>The codec&#xA0;option controls how your data will be serialized in the S3 item.
<h3>Controlling data archived to S3
</h3>Maybe you only need to send some of your data to S3. You can use our <a href="https://www.elastic.co/guide/en/logstash/current/event-dependent-configuration.html#conditionals">conditional syntax</a>&#xA0;like you do in the filter section to select a subset of your events. You can also use this approach to send events to different buckets for different events. This could mean you have 2 separate S3 output blocks differing in bucket names.&#xA0;
<pre class="prettyprint">input {
 beats {
 port =&gt; 5544
 tags =&gt; [&quot;edge_firewall&quot;]
 }
}
output {
 elasticsearch { }
 # Archive only some events to your s3bucket
 if &quot;edge_firewall&quot; in [tags] {
 s3 {
 access_key_id =&gt; &quot;SECRET&quot;
 secret_access_key =&gt; &quot;SECRET&quot;
 bucket =&gt; &quot;edge_firewall&quot;
 codec =&gt; json_lines
 }
 }
}
</pre>Here, only events tagged <code>edge_firewall</code> will be routed to S3.
<h3>Dynamically choosing prefixes
</h3>One of the popular feature requests in the S3 output was the ability to use field references or values in the event to control the prefix paths for the uploaded documents. This is great for a multi-tenant deployment where you want to separate paths for each tenant on S3. To be able to support that feature we had to do a complete rewrite of the plugin. Other recent changes include a move to an asynchronous upload model and be able to use the latest version of the AWS-SDK.
In the example below, we see how to dynamically choose your S3 destination. Beats by default sends the <code>shipper name</code> and the <code>hostname</code>, so we can use these event values to create the prefixes on your s3 bucket.
<pre class="prettyprint">input {
 beats {
 port =&gt; 5044
 codec =&gt; json
 }
}
output {
 s3 {
 access_key_id =&gt; &quot;SECRET&quot;
 secret_access_key =&gt; &quot;SECRET&quot;
 bucket =&gt; &quot;mybucket&quot;
 codec =&gt; json_lines
 prefix =&gt; &quot;/%{[beat][name]}/%{[beat][hostname]}/%{customerID}&quot;
 }
}</pre>The beats inputs will receive events with different values for fields <code>name</code>, <code>hostname</code>, <code>customerID</code>. These fields can now be used in the <code>prefix</code> option to configure the created path. For minimizing the collisions on the remote server we only allow you to set the prefix dynamically and we will generate the base name of the file for you. Assuming the events will come from multiple clients, the previous configuration will generate a structure similar to this on your S3 bucket:
<img src="https://api.contentstack.io/v2/uploads/59024dc9408ab806083b7b82/download?uid=blt6afc232e41c2f90d" data-sys-asset-uid="blt6afc232e41c2f90d" alt="logtash-documents-in-your-amazon-bucket.png" style="display: block; margin: auto;">
If you expand the directory tree, it will look similar to the one shown below -- each **field reference** is a directory. 
<img src="https://api.contentstack.io/v2/uploads/59024da4408ab806083b7b73/download?uid=bltff5e3451194ce829" data-sys-asset-uid="bltff5e3451194ce829" alt="logstash-document-on-amazon-tree-view.png" style="display: block; margin: auto;">
<h3>Under the hood
</h3>When you use S3 output, data is buffered to disk locally before uploading to S3. Each unique dynamic prefix will generate a file on disk with the matching events, and this file will be periodically and asynchronously uploaded to S3. This is controlled by the <code>rotation_strategy</code>&#xA0;option. By default, it will look for the size and the creation time of the file. These values can be changed by using the <code>time_size</code>&#xA0;and the <code>size_file</code>&#xA0;option.
Each file on disk will respect the rotation options independently of other files, so make sure that these options are configured correctly in relation to your event rate. The files might not be available right away in your bucket if the size or the creation date did not trigger a rotation or if the traffic is low for a specific dynamic prefix.
<h3>Conclusion
</h3>As you&#x2019;ve seen in this post, archiving data using Logstash is super easy! With the recent changes, a few more options were added to give you better control the performance and flexibility. I invite you to review the documentation and give S3 output a try to simplify your archiving needs!
</body></html>

Archiving your event stream with Logstash.jpg

Archiving your events.jpg

Archiving your event stream with Logstash

<html><head></head><body>With the release of Logstash 1.5 we have added the ability to add <a href="https://www.elastic.co/guide/en/logstash/current/event-dependent-configuration.html#metadata">metadata</a> to an event. The difference between
 regular event data and metadata is that metadata is not serialized by any outputs. This means any metadata you add is
 transient in the Logstash pipeline and will not be included in the output. Using this feature, one&#xA0;can add custom data
 to an event, perform additional filtering or add conditionals based on the metadata while the event flows through the
 Logstash pipeline. This will simplify your configuration and remove the need to define temporary fields.
To access the metadata fields you can use the standard field syntax:
<pre class="prettyprint">[@metadata][foo]
</pre><h2>Use Cases</h2>Lets us consider some use cases to illustrate the power of metadata. In all our use cases, will
 be using the rubydebug&#xA0;and the stdout output to check our transformation, so make sure you are correctly defining the
 output codec with the <code>metadata</code> option set to true.
 Note: The <code>rubydebug</code> codec used in the stdout output is currently
the only way to see what is in 
	<code>@metadata</code> at output time.
<pre class="prettyprint">output { 
 stdout { 
 codec =&gt; rubydebug {
 metadata =&gt; true
 }
 }
}
</pre><h2>Date filter</h2>Since logs arrive in a wide variety of formats, grok is used to extract them, and the date filter to convert them to
 ISO8601 and overwrite the 
	<code>@timestamp</code> field with the timestamp from the log event. It happens frequently that users
 omit to remove the source timestamp field after the conversion and overwrite, though.
Here&apos;s a rough example of how the new <code>@metadata</code> field could be used with the date filter and prevent a temporary
 timestamp field from making it into Elasticsearch:
<pre class="prettyprint"> grok {
 match =&gt; {
 &quot;message&quot; =&gt; &apos;%{IPORHOST:clientip} %{USER:ident} %{USER:auth} \[%{HTTPDATE:[@metadata][timestamp]}\] &#x201C;%{WORD:verb} %{DATA:request} HTTP/%{NUMBER:httpversion}&#x201D; %{NUMBER:response:int} (?:-|%{NUMBER:bytes:int}) %{QS:referrer} %{QS:agent}&apos;
 }
 }
 date {
 match =&gt; [ &quot;[@metadata][timestamp]&quot;, &quot;dd/MMM/YYYY:HH:mm:ss Z&quot; ]
 }
</pre>Before Logstash 1.5, you would remove the redundant <code>timestamp</code> field by adding the <code>remove_field</code> line into the date
 filter as I outlined above. Theoretically, that will be a slower operation than this one. That makes using the 
	<code>@metadata</code> field a performance booster!
The <code>@metadata</code> field act like a normal field and you can do all the operations or filtering on it. Use them as a scratchpad if you don&apos;t need to persist the information.
<pre class="prettyprint"># Log sample:
# 213.113.233.227 - server=A id=1234 memory_load=300 error_code=13 payload=12 event_start=1417193566&#xA0;event_stop=1417793586</pre><pre class="prettyprint">input { 
 file {
 sincedb_path =&gt; &apos;/dev/null&apos;
 path =&gt; &quot;/source/test.log&quot;
 start_position =&gt; &apos;beginning&apos;
 }
}
filter { 
 grok {
 match =&gt; {
 &quot;message&quot; =&gt; &quot;%{IP:ip} - %{DATA:[@metadata][components]}$&quot; 
 }
 }
 kv { source =&gt; &quot;[@metadata][components]&quot; }
 date {
 match =&gt; [&quot;event_start&quot;, &quot;UNIX&quot;]
 target =&gt; &quot;event_start&quot;
 }
 date {
 match =&gt; [&quot;event_stop&quot;, &quot;UNIX&quot;]
 target =&gt; &quot;event_stop&quot;
 }
 ruby {
 code =&gt; &quot;event[&apos;@metadata&apos;][&apos;duration&apos;] = event[&apos;event_stop&apos;] - event[&apos;event_start&apos;]&quot;
 }
 if [@metadata][duration] &gt; 100 {
 mutate { 
 add_tag =&gt; &quot;slow_query&quot; 
 add_field =&gt; { &quot;[@metadata][speed]&quot; =&gt; &quot;slow_query&quot; }
 }
 } else {
 mutate { 
 add_field =&gt; { &quot;[@metadata][speed]&quot; =&gt; &quot;normal&quot; }
 }
 }
}
output { 
 stdout { 
 codec =&gt; rubydebug { metadata =&gt; true }
 }
}
</pre><h2>Elasticsearch output</h2>Some plugins leverage the use of the metadata, like the elasticsearch input. It allows you to keep the document
 information in a predefined 
	<code>@metadata</code> field. This information is available to various parts of the Logstash pipeline, but will not be persisted in Elasticsearch documents.
<pre class="prettyprint">input {
 elasticsearch {
 host =&gt; &quot;localhost&quot;
 # Store ES document metadata (_index, _type, _id) in metadata
 docinfo_in_metadata =&gt; true
 }
}
output {
 elasticsearch {
 document_id =&gt; &quot;%{[@metadata][_id]}&quot;
 index =&gt; &quot;transformed-%{[@metadata][_index]}&quot;
 type =&gt; &quot;%{[@metadata][_type]}&quot;
 }
}
</pre><h2>Create your own id from your event data</h2>Out of the box, Elasticsearch provides an efficient way to create&#xA0;unique IDs for every documents that you are inserting. In most cases, you should let Elasticsearch generate the IDs. However, there are scenarios where&#xA0;you would&#xA0;want to generate an&#xA0;unique identifier in Logstash&#xA0;based on the content of the event. Using IDs based on event data lets&#xA0;Elasticsearch perform&#xA0;de-duplication. In our example, we will generate&#xA0;the IDs using&#xA0;the <code>logstash-filter-fingerprint</code> and use the default hash method (SHA1).
To test it, use the following JSON event with this configuration:
<pre class="prettyprint">{ &quot;IP&quot;: &quot;127.0.0.1&quot;, &quot;message&quot;: &quot;testing generated id&quot;}
</pre><pre class="prettyprint">input {
 stdin { codec =&gt; json }
}
filter {
 fingerprint {
 source =&gt; [&quot;IP&quot;, &quot;@timestamp&quot;, &quot;message&quot;]
 target =&gt; &quot;[@metadata][generated_id]&quot;
 key =&gt; &quot;my-key&quot;
 }
}
output {
 elasticsearch {
 protocol =&gt; &quot;http&quot;
 host =&gt; &quot;127.0.0.1&quot;
 document_id =&gt; &quot;%{[@metadata][generated_id]}&quot;
 }
 stdout {
 codec =&gt; rubydebug { metadata =&gt; true }
 }
}
</pre>
	Like in the previous examples, we are using the fieldref syntax to access the <code>generated_id</code> in the <code>@metadata</code> hash.
The Elasticsearch output will use this value as the document id, but the intermediate variable 
	<code>generated_id</code> will not be&#xA0;saved as part of the <code>_source</code> inside Elasticsearch.
If you do a query for the specific document using the generated ID you should see a similar document showing the saved information.
<pre class="prettyprint"># curl -XGET &quot;http://localhost:9200/logstash*/_search?q=_id:5f5b8e63da13c17405e940b5e8db703a19cd4485&amp;pretty=1&quot;
</pre><pre class="prettyprint">{
 &quot;took&quot; : 8,
 &quot;timed_out&quot; : false,
 &quot;_shards&quot; : {
 &quot;total&quot; : 35,
 &quot;successful&quot; : 35,
 &quot;failed&quot; : 0
 },
 &quot;hits&quot; : {
 &quot;total&quot; : 1,
 &quot;max_score&quot; : 1.0,
 &quot;hits&quot; : [ {
 &quot;_index&quot; : &quot;logstash-2015.09.03&quot;,
 &quot;_type&quot; : &quot;logs&quot;,
 &quot;_id&quot; : &quot;5f5b8e63da13c17405e940b5e8db703a19cd4485&quot;,
 &quot;_score&quot; : 1.0,
 &quot;_source&quot;:{&quot;IP&quot;:&quot;127.0.0.1&quot;,&quot;message&quot;:&quot;testing generated id&quot;,&quot;@version&quot;:&quot;1&quot;,&quot;@timestamp&quot;:&quot;2015-09-03T20:27:25.206Z&quot;,&quot;host&quot;:&quot;sashimi&quot;}
 } ]
 }
}
</pre>Similarly, you can also use <code>@metadata</code> as fieldref syntax&#xA0;in your configuration like any other fields:
<pre class="prettyprint">&quot;from server: %{[@metadata][source]}%&quot;
</pre><h2>Conclusion</h2>As you have seen in the examples above, the addition of metadata provides a simple, yet convenient&#xA0;way to store intermediate results. This makes configuration less complex -- you don&apos;t have to use <code>remove_field</code> explicitly. Also, we can&#xA0;reduce&#xA0;storage of unnecessary fields in Elasticsearch which helps reduce the size of your index. Metadata is a powerful addition to your&#xA0;Logstash toolset.&#xA0;Start using this feature today in your configuration!
</body></html>

data.jpg

Make Your Config Cleaner and your Log Processing Faster with Logstash Metadata

We are announcing the <a href="https://www.elastic.co/downloads/logstash">release</a>&nbsp;of logstash 1.5.4 and 1.4.5 which fixes important security issues. Our recommendation is to upgrade immediately if you are using either of the following features:Logstash forwarder: After the release of 1.5.3, users encountered an issue where Logstash Forwarder was unable to communicate to Logstash instance because of SSL/TLS certificate validation errors. This has been fixed.Lumberjack output: Typically used to connect two Logstash instances. In such deployments, one Logstash instance is used to collect logs from a webserver and securely transmit them to a central Logstash instance to perform additional filtering and storing.<h3>Security Fixes</h3>When using SSL/TLS functionality, Lumberjack output from Logstash 1.5.3 and prior versions did not validate certificate presented by the Logstash instance acting as a server. This exposes a man in the middle vulnerability.We have been assigned <a href="http://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2015-5619">CVE-2015-5619</a>&nbsp;for this issue and have added this vulnerability to our <a href="https://www.elastic.co/community/security/">CVE page</a>.Note:&nbsp;Users of Logstash Forwarder are not affected by this particular vulnerability<h2>Enhancements</h2>Elasticsearch Output: Added the ability to update existing ES documents and support of upsert &nbsp;--&nbsp;if document doesn't exist, create it (<a href="https://github.com/logstash-plugins/logstash-output-elasticsearch/pull/116">#116</a>). Thanks to&nbsp;<a href="https://github.com/Da-Wei">David Chauviere</a>&nbsp;for contributing this enhancement!&nbsp;Example configuration:&nbsp;<pre class="prettyprint">output { if [use_case] == "doc_upsert" { elasticsearch { host =&gt; "elasticsearch" protocol =&gt; "http" action =&gt; "update" document_id =&gt; "%{[uid]}" doc_as_upsert =&gt; true } } else if [use_case] == "doc_static_upsert" { elasticsearch { host =&gt; "elasticsearch" protocol =&gt; "http" action =&gt; "update" document_id =&gt; "%{[uid]}" upsert =&gt; '{"static_field": "demo"}' } } else if [use_case] == "doc_dynamic_upsert" { elasticsearch { host =&gt; "elasticsearch" protocol =&gt; "http" action =&gt; "update" document_id =&gt; "%{[uid]}" upsert =&gt; '{"use_case": "%{[use_case]}", "dynamic": { "fieldC": "%{[dynamic_field][fieldC]}"}}' } } } </pre><h2>Bug fixes</h2>Below is a list of bug fixes in core and&nbsp;plugins. For a full list, please check the <a href="https://github.com/elastic/logstash/blob/master/CHANGELOG.md">changelog</a>.<ul><li>Reverted a change in our harden SSL fix, that prevented Logstash Forwarder and Lumberjack output clients to connect to 1.5.3 instances (<a href="https://github.com/elastic/logstash/issues/3657">#3657</a>)</li><li>Updated Concurrent-ruby library usage to suppress deprecation warnings (<a href="https://github.com/elastic/logstash/issues/3662">#3662</a>)</li><li>Lumberjack input: Fixed a scenario where Logstash Forwarder could lose events when dealing with congestion from downstream plugins. We were incorrectly calculating the window size of payload in the acknowledgement stage. (<a href="https://github.com/elastic/logstash/issues/3691)">#3691</a>)</li><li>File input: Fix double ingestion issue when using glob path (<a href="https://github.com/elastic/logstash/issues/3674">#3674</a>)</li><li>AWS mixin: Correctly configure the proxy when using V2 version of the mixin. (<a href="https://github.com/logstash-plugins/logstash-mixin-aws/issues/15">#15</a>)</li><li>Lumberjack ouput: &nbsp;Added better handling of congestion scenario by using a buffered payloads (<a href="https://github.com/logstash-plugins/logstash-output-lumberjack/pull/7">#7</a>)</li></ul><h2>Feedback</h2>Please <a href="https://www.elastic.co/downloads/logstash">download</a>&nbsp;Logstash 1.5.4 and let us know what you think on Twitter (<a href="https://twitter.com/elastic">@elastic</a>)&nbsp;or on our forum. You can report any problems on the GitHub <a href="https://github.com/elastic/logstash/issues/">issues</a>&nbsp;page.

logs-woodpile.jpg

Logstash 1.5.4 and 1.4.5 released

Start free trial

Blog Footer CTA

Sign up for Elastic Cloud free trial

headshot-pier-hugues-pellerin-300x300.jpg

Team Lead

Pier-Hugues Pellerin

Надсилаючи цю форму ви погоджуєтеся з <a href="/legal/terms-of-use">Умовами надання послуг Elastic</a>. Ваші особисті дані будуть опрацьовані відповідно з <a href="/legal/privacy-statement">Положеннями про конфіденційність Elastic</a>.

Newsletter GDPR Text - Ukrainian

By submitting you acknowledge that you've read and agree to our <a href="/legal/elastic-cloud-account-terms" target="_self">Terms of Service</a>, and you agree to receive the <a href="/legal/privacy-statement#how-we-use-the-information" target="_self">selected communications</a> at the contact details you provided above. See <a href="/legal/privacy-statement/" target="_self">Elastic’s Privacy Statement</a> for more details or to opt-out at any time.

Communication Preferences - ONLY for /communication-preferences

By submitting you agree to <a href="/agreements/global/cloud-services-monthly" rel="nofollow">Elastic Cloud Terms of Service</a>. Your personal data will be processed in accordance with <a href="/legal/privacy-statement">Elastic's Privacy Statement</a>.

Cloud Trial GDPR Text

Intentionally Blank

By submitting you acknowledge that you've read and agree to our <a href="/legal/serverless-preview-terms" target="_blank">Terms of Service</a>, and that Elastic may <a href="/legal/privacy-statement#how-we-use-the-information" target="_blank">contact you</a> about our related products and services, using the details you provide above. See <a href="/legal/privacy-statement/" target="_blank">Elastic’s Privacy Statement</a> for more details or to opt-out at any time.

Serverless GDPR Text

提交即表示您確認已詳閱並同意我們的<a href="/legal/elastic-cloud-account-terms" target="_blank">服務條款</a>，且同意 Elastic 使用您在上方提供的個人資料<a href="/legal/privacy-statement#how-we-use-the-information" target="_blank">與您聯絡</a>，為您提供相關產品和服務的資訊。如需詳細說明或隨時想停止收到產品和服務資訊，請參閱 <a href="/legal/privacy-statement/" target="_blank">Elastic 的隱私聲明</a>。

Newsletter GDPR Text - TW

By submitting you acknowledge that you've read and agree to our <a href="/legal/elastic-cloud-account-terms" target="_blank">Terms of Service</a>, and that Elastic may <a href="/legal/privacy-statement#how-we-use-the-information" target="_blank">contact you</a> about our related products and services, using the details you provide above. See <a href="/legal/privacy-statement/" target="_blank">Elastic’s Privacy Statement</a> for more details or to opt-out at any time.By submitting this form, I agree to Elastic sharing my contact information with Google Cloud. I also agree to Elastic giving my information to the vendor for the purpose of shipping my food package.

GDPR Text + Google Cloud

Inviando la registrazione (1) esprimi il consenso ai <a href="/legal/terms-of-use">termini e condizioni di Elastic</a> e <a href="/legal/privacy-statement">all’informativa sulla privacy</a>, nonché (2) a ricevere saltuarie e-mail.

Newsletter GDPR Text - Italian

By submitting you acknowledge that you've read and agree to our <a href="/legal/terms-of-use" target="_blank">Terms of Service</a>, and that Elastic may <a href="/legal/privacy-statement#how-we-use-the-information" target="_blank">contact you</a> about our related products and services, using the details you provide above. See <a href="/legal/privacy-statement/" target="_blank">Elastic’s Privacy Statement</a> for more details or to opt-out at any time.

Newsletter GDPR Text

Load more

Overview

Can't make it? Register and we'll send you the recording. You'll also receive an email with related content

Can't make it? Register and we'll send you the recording. You'll also receive an email with related content.

You'll also receive an email with related content.

You'll also receive an email with related content

Speakers

Close

See more insights

The content on this page is not available in the selected language. As Elastic grows globally, we continue to support content in multiple languages.

Author

Articles by PH Pellerin

Team Lead, Elastic

Functionbeat: Serverless Ingestion for Elasticsearch

Archiving your event stream with Logstash

Make Your Config Cleaner and your Log Processing Faster with Logstash Metadata

Logstash 1.5.4 and 1.4.5 released

Follow us

About us

Join us

Partners

Trust & Security

Investor relations

Excellence Awards