Join: Restricting "followed by" using time conditions

I have tried similar things in the past and they can be super difficult to achieve due to the “fuziness”. Ideally, there is something distinctive that we can hang our hat on, and in this case that’s probably the device IP, and you match on that exactly between the logs. According to the manual “followed by” can have a “within 2 seconds” clause, although I haven’t tried that myself.

So it would be [search1] as s1 followed by [search2] as s2 within 2 seconds on s1.target_ip=s2.target_ip - I think :)

Another thing that I have is done is to work with lists (that get populated from scheduled alerts or reports). You put the candidates based on a search on one list, make sure that gets updated from time to time, and then use that list for your second query, without doing the actual correlation during the search. I don’t think that really works in your use case though, because you are just trying to monitor everything from what I understand.

Hi,

To get more accurate results, I would look into the possibility to log the ‘X-Forwarded-For’ Header in the HTTP requests that go through the proxy. If it is possible to add this field in the logs from the proxy, the query will be much easier and you won’t have any doubt about the original ip - thus avoiding the ‘fuziness’. More info on the HTTP Header:

https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/X-Forwarded-For

Just as a side note...

@ Nils:

Thanks Nils, didn’t find it or just read over the followed by keyword. This was exactly what i was looking for, so i will mark this topic as closed.

However, the result was rather disappointing, as the search query does not yield the desired results:

• I didn’t think about that “joining” two queries using f ollowed by will only output a login attempt on the device (first search) IF there is ALSO a correlating log from the proxy (second search). As some login attempts come from within the same LAN, there will not be a log in the proxies, so these accesses to the device will not be in the output.
  ( From what i have read a left join may be a solution rather need, as it will always output results of the  “left query” and will add the result of the “right query”. I will take a further look into that)
  
• It seems that our proxy appliances do not create a log for every packet passing through it (seems it only does so when a session is “established”). So i have several login attempts done through the proxy, but no correlated logs in the proxy within a 5 minutes timeframe.

I think i will do a query for the login attempts on the device and will do a left join with the proxy logs on the exact same timestamps. This will give me all the access attempts and “for some of these” also the client IP. Think this is the best result considering the amount of details (or lack thereof) and “fuzziness” of the logs.

@Mike Blomgren

Great hint regarding the X-Forwarded-For Header. Have to admit i wasn’t aware of this one.
Unfortunately, our devices and proxies are 3rd party appliances and we cannot configure the details contained in the logs. The logs are rather “simplistics” and do not contain any HTTP header information.