| 4 Jul 2023 |
|  YourMomIsNotMale joined the room. | 11:21:01 |
|  therealrandomstring joined the room. | 11:27:06 |
|  _ollster joined the room. | 13:15:55 |
| 5 Jul 2023 |
|  alexb271 joined the room. | 15:51:33 |
| 6 Jul 2023 |
|  bazsi77 changed their display name from bazsi77 to bazsi77#0. | 12:39:55 |
| bazsi77 changed their display name from bazsi77#0 to bazsi77. | 12:39:56 |
| bazsi77 | Just responded to a question of Stack Overflow on local timezone handling for filenames, I thought I'd share here. https://serverfault.com/questions/1133627/syslog-ng-timezone-change-and-hour-macro?noredirect=1#comment1478382_1133627 | 12:39:56 |
| bazsi77 | * Just responded to a question of ServerFault on local timezone handling for filenames, I thought I'd share here. https://serverfault.com/questions/1133627/syslog-ng-timezone-change-and-hour-macro?noredirect=1#comment1478382_1133627 | 12:40:21 |
| 7 Jul 2023 |
|  hofione changed their display name from HoFi to hofione#0. | 14:48:25 |
| hofione changed their display name from hofione#0 to hofione. | 14:48:26 |
| hofione | Hi,
some more syslog-protocol related questions
i have the following simple source
source s_remote1 {
channel {
# Generic syslog-protocol network source you can feed like
# loggen -S 127.0.0.1 514 --reconnect -r 1 -T -s 180 -P -F
#
source {
network(
ip(localhost)
transport("tcp")
port(514)
flags(
dont-store-legacy-msghdr
)
);
};
parser {
syslog-parser(
flags(
syslog-protocol
dont-store-legacy-msghdr
)
);
};
};
};
i feed it with
loggen -S 127.0.0.1 514 --reconnect -r 1 -T -s 180 -P -F
| 14:48:26 |
| hofione | for some reason the parsing of the produced log entries stops at the APP-NAME (PROGRAM) field and the PROCID (PID) is not processed instead becomes part of he MSG
[2023-07-07T14:36:52.211748] >>>>>> parser rule evaluation begin; rule='#anon-parser0', location='/root/data/PE7/install/etc/syslog-ng.conf:120:4', msg='0x7ffff40148a0', rcptid='35'
[2023-07-07T14:36:52.211767] syslog-parser message processing started; input='2023-07-07T14:36:52+00:00 localhost prg00000 1234 - 10 100.3 seq: 0000000040, thread: 0000, runid: 1688739718, stamp: 2023-07-07T14:36:52 PADDPADDPADDPADDPADDPADDPADDPADD', msg='0x7ffff40148a0', rcptid='35'
[2023-07-07T14:36:52.211807] Setting value; name='PROGRAM', value='prg00000', type='string', msg='0x7ffff40148a0', rcptid='35'
[2023-07-07T14:36:52.211821] Setting value; name='HOST', value='localhost', type='string', msg='0x7ffff40148a0', rcptid='35'
[2023-07-07T14:36:52.211835] Setting value; name='MESSAGE', value='1234 - 10 100.3 seq: 0000000040, thread: 0000, runid: 1688739718, stamp: 2023-07-07T14:36:52 PADDPADDPADDPADDPADDPADDPADDPADD', type='string', msg='0x7ffff40148a0', rcptid='35'
[2023-07-07T14:36:52.211851] Requesting flow control; location='/root/data/PE7/install/etc/syslog-ng.conf:144:2'
could someone please explain why? | 14:48:35 |
| bazsi77 | You parse the message twice | 14:50:08 |
| bazsi77 | Once in the network source (there's no no-parse flag) | 14:50:23 |
| bazsi77 | This means your MSG will only contain part of the original message (we extract it as if it was BSD formatted) | 14:51:18 |
| bazsi77 | Then the second syslog-parser() would try to parse this remaining MSG as rfc5424 | 14:51:42 |
| hofione | i tried no-parse for the fiirst one but it gives me parsing error in that case | 14:55:12 |
| hofione | in that case the produced message looks like this
Jul 7 14:56:31.841212 syslog-ng[70110]: Error processing log message: <#38>1 2023-07-07T14:56:31+00:00 localhost prg00000 1234 ->@< 10 100.3 seq: 0000000072, thread: 0000, runid: 1688739718, stamp: 2023-07-07T14:56:31 PADDPADDPADDPADDPADDPADDPADDPADD | 14:56:52 |
| hofione | i tried a bunch of combinations of the flags both for the first stage, the second and in invocation of loggen using the -P -F options
still cannot produce an rfc5424 compilant result | 14:58:18 |
| hofione | that gave me the best result, at least in the trace log output
[2023-07-07T14:58:44.357669] >>>>>> parser rule evaluation begin; rule='#anon-parser0', location='/root/data/PE7/install/etc/syslog-ng.conf:119:4', msg='0x7ffff40148a0', rcptid='2'
[2023-07-07T14:58:44.357696] syslog-parser message processing started; input='<38>1 2023-07-07T14:58:44+00:00 localhost prg00000 1234 - 10 100.3 seq: 0000000078, thread: 0000, runid: 1688739718, stamp: 2023-07-07T14:58:44 PADDPADDPADDPADDPADDPADDPADDPADD', msg='0x7ffff40148a0', rcptid='2'
[2023-07-07T14:58:44.357834] Setting value; name='HOST', value='localhost', type='string', msg='0x7ffff40148a0', rcptid='2'
[2023-07-07T14:58:44.357852] Setting value; name='PROGRAM', value='prg00000', type='string', msg='0x7ffff40148a0', rcptid='2'
[2023-07-07T14:58:44.357866] Setting value; name='PID', value='1234', type='string', msg='0x7ffff40148a0', rcptid='2'
[2023-07-07T14:58:44.357891] Setting value; name='HOST', value='', type='string', msg='0x7ffff40148a0', rcptid='2'
[2023-07-07T14:58:44.357923] Setting value; name='MESSAGE', value='Error processing log message: <38>1 2023-07-07T14:58:44+00:00 localhost prg00000 1234 ->@< 10 100.3 seq: 0000000078, thread: 0000, runid: 1688739718, stamp: 2023-07-07T14:58:44 PADDPADDPADDPADDPADDPADDPADDPADD', type='string', msg='0x7ffff40148a0', rcptid='2'
[2023-07-07T14:58:44.357940] Setting value; name='PROGRAM', value='syslog-ng', type='string', msg='0x7ffff40148a0', rcptid='2'
[2023-07-07T14:58:44.357956] Setting value; name='PID', value='70113', type='string', msg='0x7ffff40148a0', rcptid='2'
[2023-07-07T14:58:44.357976] Requesting flow control; location='/root/data/PE7/install/etc/syslog-ng.conf:143:2'
| 14:59:29 |
| hofione | but for some reason also led to an 'Error processing log message' too | 15:00:15 |
| hofione | actually that is the reason i stated to try it in 2 passes
if i simply try to parse the output of the loggen -P -F ....
(that otherwise looks perfect for me, like:
'<38>1 2023-07-07T15:30:50+00:00 localhost prg00000 1234 - 10 100.3 seq: 0000000102, thread: 0000, runid: 1688742509, stamp: 2023-07-07T15:30:50 PADDPADDPADDPADDPADDPADDPADDPADD'
and using
source first_run {
network(
ip(localhost)
transport("tcp")
port(514)
flags(
syslog-protocol
dont-store-legacy-msghdr
#no-parse
)
);
};
that gives the same result
an errored output
Jul 7 15:30:52.035155 localhost syslog-ng[70161]: Error processing log message: <38>1 2023-07-07T15:30:52+00:00 localhost prg00000 1234 ->@< 10 100.3 seq: 0000000104, thread: 0000, runid: 1688742509, stamp: 2023-07-07T15:30:52 PADDPADDPADDPADDPADDPADDPADDPADD
| 15:38:18 |
| 8 Jul 2023 |
|  cpuram1 joined the room. | 20:20:08 |
| 10 Jul 2023 |
| hofione | just for the record, the issue was the malformed STRUCTURED-DATA part
sorry for the false alert | 12:42:27 |
|  bruno.bressi changed their display name from bruno.bressi to bruno.bressi#0. | 15:01:55 |
| bruno.bressi changed their display name from bruno.bressi#0 to bruno.bressi. | 15:02:20 |
|  faxmodem#0 changed their display name from faxmodem#5219 to faxmodem#0. | 17:22:10 |
| 11 Jul 2023 |
|  josephsaber joined the room. | 08:50:11 |
|  mranno changed their display name from Anno to mranno#0. | 11:57:43 |
| mranno changed their display name from mranno#0 to mranno. | 11:57:43 |
