| 24 Sep 2023 |
|  mclovin9929 joined the room. | 14:22:56 |
| mclovin9929 | 
Download Screenshot_2023-09-24_at_8.04.58_PM.png | 14:36:02 |
| mclovin9929 | Hello everyone siddhant this side, I had a doubt regarding my logging operator deployment in the efk-stack,
can someone tell me why I am seeing two fluentd stateful-sets?
The default-logging-simple-fluentd should only be there I guess as it is using the resource that I have specified in the file | 14:36:02 |
| mclovin9929 | Hey thank you so much, just discussed with my teammate and found out he deployed it while trying out single-app one-destinaiton.
The issue has been resolved, thanks for the help. | 14:52:04 |
| 25 Sep 2023 |
|  @martin.mueller:dataport.modular.im changed their display name from 🐦Martin 🔥Müller (Phoenix AMO - away till 2023-09-26) to 🐦Martin 🔥Müller (Phoenix AMO). | 09:54:09 |
| 26 Sep 2023 |
 sbonafe | Hello, I am trying to parse a log that looks like this:
{"instant":{"epochSecond":1695759227,"nanoOfSecond":796395046},"thread":"scheduled:3","level":"INFO","loggerName":"foo","message":"Version: 1.27.5 Total Processed: 342770","endOfBatch":false,"loggerFqcn":"foo","threadId":40,"threadPriority":5}
We are outputting to kafka which displays this:
{
"time":"2023-09-26T20:13:47.796765583Z"
"message":"{"instant":{"epochSecond":1695759227,"nanoOfSecond":796395046},"thread":"scheduled:3","level":"INFO","loggerName":"foo","message":"Version: 1.27.5 Total Processed: 342770","endOfBatch":false,"loggerFqcn":"foo","threadId":40,"threadPriority":5}"
"kubernetes":{
"pod_name":"foo"
"namespace_name":"foo"
"host":"foo"
"container_name":"foo"
}
}
I would like to parse the log before it gets to kafka in order to break up the message into the following fields: time, level, message.
What I have attempted:
Editing the application yaml to have fluentd do the parsing and attempted to edit the flow after that did not work. Any suggestions on how to make this work? | 20:26:46 |
| 27 Sep 2023 |
|  xinity77 joined the room. | 12:29:47 |
| xinity77 | hello there 🙂 | 12:30:23 |
| xinity77 | starting to play with the awesome toolkit | 12:30:33 |
| xinity77 | i've been wondering how does the logging-operator fits with vector ? | 12:31:09 |
| xinity77 | meaning do you think i would be easy to send data from the central fluentd to vector to process and then to ..... | 12:36:38 |
|  gioppoluca joined the room. | 14:28:51 |
| gioppoluca | are there any examples for a ClusterFlow for spring boot? It is difficult to find proper examples on how to parse java logs | 14:29:43 |
| 28 Sep 2023 |
| xinity77 | 
Download logging_operator_flow.png | 08:14:05 |
| xinity77 | weird question but , which tool has been used to make this graph ? : | 08:14:05 |
| xinity77 | i love the design :p | 08:15:38 |
|  foo joined the room. | 11:36:42 |
 gholie | I'm trying to remove dates to indexes, and instead using rollover index with this pattern:
env-namespace-000001. I have set this config:
apiVersion: logging.banzaicloud.io/v1beta1
kind: ClusterOutput
metadata:
name: log-output
spec:
elasticsearch:
request_timeout: 10s
reload_connections: false
reconnect_on_error: true
reload_on_failure: true
include_tag_key: true
rollover_index: true
logstash_format: true
tag_key: tag
logstash_prefix: ${tag[0]}-${tag[1]}
index_date_pattern: ""
logstash_dateformat: ""
deflector_alias: sandbox
log_es_400_reason: true
host: host
port: 9243
scheme: https
ssl_verify: false
ssl_version: TLSv1_2
user: elastic
password:
valueFrom:
secretKeyRef:
name: es-elastic-user
key: elastic
But my index in elasticsearch still looks like this:
env-namespace-date. Any suggestion on what I am doing wrong here? | 11:51:42 |
|  jvaron_52559 joined the room. | 13:55:39 |
|  thanostak joined the room. | 13:58:59 |
| 29 Sep 2023 |
 sg2566 | The logstash_format: true implicitly add the date suffix to your index name. Try to use https://docs.fluentd.org/output/elasticsearch#index_name-optional instead | 11:24:48 |
| gholie | Ah, I thought either the index_date_pattern or logstash_dateformat would remove it. Altough I have also tried using index_name, but I am not getting the trailing digits. Should I set those manually? | 11:26:04 |
 pepov | 📢 Hey folks! I'm happy to announce that the latest release of the Logging Operator is available: https://github.com/kube-logging/logging-operator/releases/tag/4.4.0
Huge thanks for everyone involved in this important milestone where real multi-tenancy becomes a thing among many other great new features, improvements and bug fixes.
Updates to the documentation site are in progress, will share it here once we get it sorted out. | 14:25:29 |
 zipzad | I'd like to send logs from a shared nginx-ingress instances to users specified outputs. I need to ensure they can only access logs from their namespaces.
I thought it would be possible with one ClusterFlow per tenant linking to outputs in their namespaces. However, AFAIK, there is no way to route from ClusterFlow to an Output. Any idea how to achieve that? | 16:26:23 |
| zipzad | * Hey there 🙂 I'd like to send logs from a shared nginx-ingress instances to users specified outputs. I need to ensure they can only access logs from their namespaces.
I thought it would be possible with one ClusterFlow per tenant linking to outputs in their namespaces. However, AFAIK, there is no way to route from ClusterFlow to an Output. Any idea how to achieve that? | 16:26:30 |
| zipzad | * Hey there 🙂 I'd like to send logs from a shared nginx-ingress instances to users specified outputs. I need to ensure they can only access logs from their namespaces.
I thought it would be possible with one ClusterFlow per tenant linking to outputs in their namespaces. However, AFAIK, there is no way to route from ClusterFlow to an Output. Any idea how to achieve that? Thank you | 16:26:38 |
| pepov | Hey there 🙂 I d like to send logs from | 17:32:45 |
| 2 Oct 2023 |
|  @temp4096:matrix.org joined the room. | 03:56:55 |
| pepov | 📢 Hey! Docs are now available for the new release, make sure to check out the "What's new" section!
https://kube-logging.dev/docs/whats-new/ | 07:18:24 |
|  fekete77.robert changed their display name from fekete77.robert to fekete77.robert#0. | 08:09:04 |
