NWA50AX logging is buggy

StefanZ · May 21

Not sure if it fits the topic, but there is no general "bug" forum here.

The CEF formatted messages of the NWA50AX are a paint to parse – like many logs of Zyxel devices I might add!

<141>May 21 01:30:26 NWA50AX_A CEF:0|Zyxel|NWA50AX||0|System Monitoring|5|src=0.0.0.0 dst=0.0.0.0 spt=0 dpt=0 msg=cpu=1,memory=40,sessions=40,uptime=9 days, 13:54:40

#1 Why is the msg field blank?

#2 There is no separator to the cpu field (should be a blank)

#3 Why on earth do we then get commas as separators for the rest of the log?

#4 And then the cherry on top is the uptime with a friggin' comma in the value!

Looking at this other message:

<141>May 21 01:50:40 NWA50AX_A CEF:0|Zyxel|NWA50AX||0|System Monitoring|5|src=0.0.0.0 dst=0.0.0.0 spt=0 dpt=0 msg=hostname=NWA50AX_A,modelname=NWA50AX,firmwareversion=V6.29(ABYW.1)

… one can see that there is method to the madness!
Turns out, that the msg value is basically another list of key/value pairs! But comma separated. But WHY?

Same thing on the FLEX200, but with key:value pairs instead of key=value.
Separated with spaces – and value fields that sometimes contain – you guessed it? – SPACES!

name:vlan10 status:Down TxPkts:0 RxPkts:0 Colli.:0 TxB/s:0 RxB/s:0

Also: keys that contain slashes and dots – WHY?
Try to parse that into any system and predict what happens…

Zyxel_Kay · May 29

Hi @StefanZ

Regarding your problem,

#1 Why is the msg field blank?
#2 There is no separator to the cpu field (should be a blank)
#3 Why do we then get commas as separators for the rest of the log?

The msg field is not blank here, it contains the msg “cpu=1,memory=40,sessions=40,uptime=9 days, 13:54:40”. The cpu field appears as part of the msg field, therefore is no a distinct separator. In order to distinguish the rest of log (ex: cpu, memory, session, uptime, and etc,), we get commas for them.

#4 And then the cherry on top is the uptime with a friggin' comma in the value!

The comma in the value here is to differentiate the day and time components day and time.

To better assist you, we would appreciate some additional information:

Could you provide more details about the specific issue or challenge you are facing with your syslog server and the current AP CEF syslog format?
Which syslog server are you using? Knowing the specific syslog server you're working with would help us provide more targeted guidance/solution.

StefanZ · May 29

I am using a current Graylog server.

First issue is, that you cannot use the native CEF input – UDP RAW works, but you already lose some automatic pre-processing. I suspect that is because of the prepended facility code. That code could just live in its' own field.

Generally most of the Zyxel logs are formatted in a way that makes them hard to parse. And then they also differ from each other.

Often you also fail at just putting the important bit into a dedicated field:
"Limited-Admin admin(MAC:) from eap-cfg has logged out Device" (on FLEX200)
the rest of the fields do not tell me if someone logged in or out – I have to run some sort of RegEx or GROK to parse the "human readable" message. Let alone the text is missing the word "of".
Then on the NWA50AX you have Administrator admin http/https login. – same event, new message to parse, also zero additional fields that would help.

As for "message in a message with slightly different markup" – I figured that, but I fail to understand why one would want that?
This results in parsing errors, because the parser looks for = as a delimiter between key/value and finds that msg=next_key=text,another_key=12345 indeed means "msg field is empty" and "next_key value is 'text,'". So unless you replace all the commas with spaces before parsing, you end up with a value that has a trailing comma.

This might work, if you escape the delimiters in msg, but again: Why do that, if you can just have regular fields?

Here msg contains a key "name" – but the CEF header already contains a "name" field: "INTERFACE STATISTICS" (no idea why this one is in all caps).

So the event from the "name" field will be replaced with the interface "name" field, thereby screwing up the message classification / sorting. Renaming it to "interface_name" would help.

And lastly: Having dots or slashes in a key is quite the "inconvenience" in most common programming languages – to put it mildly! They serve zero purpose here, since people looking at those metrics surely will be able to deduce "Colli is an abbreviation for Collisions". But they make log ingestion a game of Russian roulette.

"The comma in the value here is to differentiate the day and time components day and time."

Well no – the commas separate your key/value pairs! You wrote that yourself.
In order to make the comma part of a value, you would have to escape it using a \ – although that might not even be possible or yield unpredictable results. But this would be no issue, if you just put everything into its' own field.

And lastly, there are messages that contain errors:

The value of "ZYuptime" is missing the last seconds digit in every message.
And yes, here the msg= formatting switches over to key:value with a colon while still having a comma in one value – yet another special case to handle.

Some CEF headers are incomplete:

CEF:0|Zyxel|NWA50AX||0|System Monitoring|5|src=0.0.0.0 dst=0.0.0.0 spt=0 dpt=0 msg=cpu=1,memory=39,sessions=39,uptime=22:41:19

Here the device_version key is missing – it exists in all/most other messages from that device.

And then there are redundant messages like this one:

<141>May 29 15:54:06 NWA50AX_A CEF:0|Zyxel|NWA50AX||0|System Monitoring|5|src=0.0.0.0 dst=0.0.0.0 spt=0 dpt=0 msg=hostname=NWA50AX_A,modelname=NWA50AX,firmwareversion=V6.29(ABYW.1)

Here the CEF header is missing the device_version once more, but has fields that will always be 0 – and a message body that just repeats the information that should be included in every header anyway (plus the hostname, but that one is included in the syslog envelope).

Zyxel_Kay · May 30

Hi @StefanZ

Thank you for providing us with such a detailed description of the issues you've encountered. We greatly appreciate your feedback and the time you've taken to outline each point.

Rest assured that we will promptly forward your concerns to our development team for further investigation and resolution.

NWA50AX logging is buggy

Active · Last Updated May 21

Comments

Categories