Introducing the BytesLines iterator

The BytesLines iterator's goal is to provide an API for processing logs line by line. It processes logs by:

  • Splitting sub line to treat cmd output embedded as a long oneliner.
  • Working with Read objects, such as file decompressors or network endpoints.
  • Using zero copy slices to optimize memory usage …
more ...

Sprint 2022 Feb 18 to Mar 09 summary

Below are the tasks we worked on during our last sprint.

Opendev

  • We propose "logsender" service that will be parsing log results fetched by Logscraper and send it to the Elasticsearch service. It change the log processing workflow by removing gearman-client, gearman-worker and logstash service
  • We are adding more documentation …
more ...



Improving logreduce with Rust

This article introduces logreduce-tokenizer which leverages the Rust programing language to improve logreduce performance and reporting capabilities.

In this post you will learn:

  • What logreduce is.
  • How it works.
  • Why we need this new function.
  • What the upcoming roadmap is.

Logreduce

Logreduce is a command line tool that can extract …

more ...

Sprint 2021 Jan 07 to Jan 26 summary

Below are the tasks we worked on during our last sprint.

Opendev

  • We finally fix an issue with Reed in Opensearch Opendev instance that index pattern was not visible
  • We reviewed the IBM Cloud VPC driver for nodepool
  • We created a visualisation of the zuul git history with gource for …
more ...




Mitigate CVE-2021-44228

An important Java vulnerability is affecting the following Software Factory service:

  • elasticsearch
  • logstash

Install the mitigation from the install server by running these commands:

ansible elasticsearch -m lineinfile -a "path=/etc/sysconfig/elasticsearch regexp='^ES_JAVA_OPTS=.*' line='ES_JAVA_OPTS=\"-Dlog4j2.formatMsgNoLookups=true\"'"
ansible elasticsearch -m service    -a "name …
more ...