Debian Normalizers

For Linux logs we can use LP_Unix Normalizers. You could try these normalization packages:

LP_Unix Rsyslogd, LP_Unix Crond, LP_Unix Bash, LP_common unix system.

These are common processes running in linux systems.

Linux/UNIX system logs will generally contain the process name and the normalization packages are generally based on the application. So you could look at the logs that are not being normalized using a query like :

“-norm_id=*”

Then you’d look for the application name in the msg field as shown in the log format below and search for keyword. Here in the example: su

Hi,

Yeah that helped a lot in the first place, now I only have the Problem that some Rsyslogs or systemd logs are still not normalized correctly. Do you have any idea why?

It’s quite possible that the normalisers don’t include a signature for these messages - that does happen from time to time. It could be that we have never seen them, or that they are very generic. Both of the messages in your example are very “free text”.

There are multiple approaches:

1. Do you really need those messages other than in cleartext? Sometimes I find it’s perfectly fine to just ignore some messages if they don’t contain serious payload that I am after for aggregation - but it’s always worth checking from time to time (via a search for -norm_id=*) what messages aren’t getting normalised in case there are new messages appearing.
2. Create you own normaliser or label specifically for the messages you are interested in - that’s often the quickest way to tackle that ONE message that fails to be normalised.
3. Raise a ticket with LogPoint Support including the example messages that don’t normalise - that is particularly the case if they are actually important messages or there are a lot of them. They would then go an incorporate those new signatures into the package.

Which approach to choose is kind of dependent on the number and contents of unnormalised logs...