Project

General

Profile

Bug #216

mod_security cleanup, and compatibility with version 2.0

Added by about 12 years ago. Updated almost 10 years ago.

Status:
Closed
Priority:
Normal
Assignee:
Target version:
Start date:
Due date:
% Done:

0%

Resolution:
fixed

Description

Correct the following:

  • Place the unique_id data in the correct field (remove classification.ident, most likely belongs in additional_data).
  • Correct classification.text where action taken is included (this should be in the action field); remove the action taken to improve correlation capabilities based on classification.text independent of action taken.
  • Audit ruleset for mod_security 2.0

modsecurity.rules - Ruleset for ModSecurity 2.x (4.31 KB) , 07/31/2008 05:09 PM

modsecurity.2.rules - Updated rules for ModSecurity 2.x (including log sample) (6.76 KB) , 08/01/2008 03:37 PM

modsecurity.3.rules - Updated ruleset; Changed LOG: samples (7 KB) , 08/11/2008 10:42 PM

modsecurity.4.rules - Updated ruleset. Changed ids to 31xx range (7 KB) , 08/12/2008 01:19 PM

modsecurity.5.rules - Updated ruleset. Added reference.name and reference.url. (7.12 KB) , 08/12/2008 02:47 PM

modsecurity.6.rules - Updated ruleset. Tested with ModSecurity 2.1.7 and 2.5.6. (12 KB) , 10/09/2008 11:01 PM

History

#1 Updated by about 12 years ago

  • Status changed from New to Assigned

#2 Updated by almost 12 years ago

Hello,

Here is my rule when using [[ModSecurity]](2)

#LOG:May 31 11:51:09 server httpdr10634: [error] [client 10.8.0.128] [[ModSecurity]]: Access denied with code 400 (phase 2). Pattern match "^[\\\\d\\\\.]+$" at REQUEST_HEADERS:Host. [id "960017"] [msg "Host header is a numeric IP address"] [severity "CRITICAL"] [hostname "10.8.0.127"] [uri "/blah"] [unique_id "CcdHJFKU22EAACmK5AcAAAAF"]
regex=\[client ([\d\.]+)\] [[ModSecurity]]: Access denied with code (\d+) \(phase (\d+)\)\. Pattern match "(\S+)" (.+)\. \[id "(\d+)"\] \[msg "(.+)"\] \[severity "(\S+)"\] \[hostname "([\S.]+)"\] \[uri "(.+)"\] \[unique_id "(\S+)"\]; \
 id=3108; \
 revision=1; \
 classification.ident = $11; \
 classification.text=HTTP $5 ($4) Blocked; \
 analyzer(0).name=ModSecurity; \
 analyzer(0).manufacturer=www.modsecurity.org; \
 analyzer(0).class=HIDS; \
 assessment.impact.severity=$8; \
 assessment.impact.completion=failed; \
 assessment.impact.description=mod_security encountered an error: $7.; \
 assessment.action(0).category = block-installed; \
 assessment.impact.description=Access was blocked with HTTP response code $2; \
 source(0).service.iana_protocol_name=tcp; \
 source(0).service.iana_protocol_number=6; \
 source(0).node.address(0).category=ipv4-addr; \
 source(0).node.address(0).address=$1; \
 target(0).node.name=$9; \
 target(0).service.iana_protocol_name=tcp; \
 target(0).service.iana_protocol_number=6; \
 target(0).service.name=http; \
 target(0).service.web_service.url = $10; \
 additional_data(0).type=integer; \
 additional_data(0).meaning=HTTP code returned; \
 additional_data(0).data=$2; \
 additional_data(1).type=integer; \
 additional_data(1).meaning=id; \
 additional_data(1).data=$6; \
 additional_data(2).type=integer; \
 additional_data(2).meaning=phase; \
 additional_data(2).data=$3; \
 last

Regards,

Robin

#3 Updated by almost 11 years ago

Hi Robin,
could you please send us some more logs from modsecurity (or put it somewhere so we can download it)?
We need some input for audit of the ruleset.

Thanks,
Dan

Replying to [comment:2 anonymous]:

Hello,

Here is my rule when using [[ModSecurity]](2)
....
....
Regards,

Robin

#4 Updated by Yoann VANDOORSELAERE almost 11 years ago

Robin apparently changed mail address. Adding the new one to the list.

#5 Updated by over 10 years ago

we appreciate any feedback on our mod_security 2.0 ruleset attached by .
thnx.

#6 Updated by over 10 years ago

I haven't found a reference manual for the prelude-lml ruleset keywords like chained, silent, min-opt-goto. Is it somewhere available?

#7 Updated by Yoann VANDOORSELAERE over 10 years ago

I have just added some documentation concerning rule flow control at the end of the wiki:PreludeLML page.
Please let me know if you have any other question!

#8 Updated by Yoann VANDOORSELAERE over 10 years ago

Replying to [comment:7 ]:

we appreciate any feedback on our mod_security 2.0 ruleset attached by .

Thanks! could you update the ruleset to include some log sample, using the following syntax:

#LOG:May 10 15:19:28 mail clamdr14292: /usr/share/doc/clamav-0.70/test/test2.badext: [[ClamAV]]-Test-Signature FOUND

This is used for regression testing and reviewing generated alerts:

  cd prelude-lml/tests
  ./loggrep.py ../plugins/pcre/ruleset/<name of your ruleset>.rules | ../src/prelude-lml --dry-run --metadata=nowrite,head --batch-mode --no-resolve --pcre --dump-unmatched --config ./prelude-lml.conf --text-output

#9 Updated by over 10 years ago

Replying to [comment:10 yoann]:

Replying to [comment:7 ]:

we appreciate any feedback on our mod_security 2.0 ruleset attached by .

Thanks! could you update the ruleset to include some log sample, using the following syntax:

Yeah, I forgot. Fixed.

This is used for regression testing and reviewing generated alerts:

>   cd prelude-lml/tests
>   ./loggrep.py ../plugins/pcre/ruleset/<name of your ruleset>.rules | ../src/prelude-lml --dry-run --metadata=nowrite,head --batch-mode --no-resolve --pcre --dump-unmatched --config ./prelude-lml.conf --text-output

I tested the ruleset "in action" (apache with modsec, prelude, prewikka) and it worked fine. However, I tried to test the ruleset using command you mentioned but it ends up with this error messages:

$ ./loggrep.py modsecurity.rules | ../src/prelude-lml --dry-run --metadata=nowrite,head --batch-mode --no-resolve --pcre --dump-unmatched --config ./prelude-lml.conf --text-output
Invalid option -- "pcre" (0).
Invalid option -- "dump-unmatched" (0).
./prelude-lml.conf:52: invalid section : "Pcre".
./prelude-lml.conf:53: invalid option "ruleset" in "global" section.
01 Aug 15:44:51 (process:27226) ERROR: couldn't open config file /usr/local/etc/prelude-lml/plugins.rules. (regex.c:168 get_regex_table)
01 Aug 15:44:51 (process:27226) WARNING: error while setting option 'file': No such file or directory.
Traceback (most recent call last):
  File "./loggrep.py", line 29, in <module>
    print i[:-1]
IOError: [Errno 32] Broken pipe

Version of prelude-lml is 0.9.12.2

$ pwd
/home/dkopecek/devel/prelude-lml/F-9/prelude-lml-0.9.12.2/tests
$ make tests
./loggrep.py ../plugins/pcre/ruleset/*.rules | ../src/prelude-lml --quiet --dry-run --metadata=nowrite,head --batch-mode --no-resolve --pcre --dump-unmatched --config ./prelude-lml.conf 2>&1 | /bin/grep -Fvf ignored
grep: ignored: No such file or directory
Traceback (most recent call last):
  File "./loggrep.py", line 29, in <module>
    print i[:-1]
IOError: [Errno 32] Broken pipe
make: *** [tests] Error 2

#10 Updated by Yoann VANDOORSELAERE over 10 years ago

Replying to [comment:11 ]:

Replying to [comment:10 yoann]:

Replying to [comment:7 ]:
This is used for regression testing and reviewing generated alerts:

> >   cd prelude-lml/tests
> >   ./loggrep.py ../plugins/pcre/ruleset/<name of your ruleset>.rules | ../src/prelude-lml --dry-run --metadata=nowrite,head --batch-mode --no-resolve --pcre --dump-unmatched --config ./prelude-lml.conf --text-output

I tested the ruleset "in action" (apache with modsec, prelude, prewikka) and it worked fine. However, I tried to test the ruleset using command you mentioned but it ends up with this error messages:

> $ ./loggrep.py modsecurity.rules | ../src/prelude-lml --dry-run --metadata=nowrite,head --batch-mode --no-resolve --pcre --dump-unmatched --config ./prelude-lml.conf --text-output
> 01 Aug 15:44:51 (process:27226) ERROR: couldn't open config file /usr/local/etc/prelude-lml/plugins.rules. (regex.c:168 get_regex_table)

The previous error are most probably due to the absence of plugin.rules: make sure prelude-lml is installed before running the tests.

<pre>
> $ pwd
> /home/dkopecek/devel/prelude-lml/F-9/prelude-lml-0.9.12.2/tests
> $ make tests
> ./loggrep.py ../plugins/pcre/ruleset/*.rules | ../src/prelude-lml --quiet --dry-run --metadata=nowrite,head --batch-mode --no-resolve --pcre --dump-unmatched --config ./prelude-lml.conf 2>&1 | /bin/grep -Fvf ignored
> grep: ignored: No such file or directory
> Traceback (most recent call last):
>   File "./loggrep.py", line 29, in <module>
>     print i[:-1]
> IOError: [Errno 32] Broken pipe
> make: *** [tests] Error 2
</pre>

The file is missing, you can just _touch ignored_ in order to fixes this problem.

#11 Updated by over 10 years ago

Replying to [comment:12 yoann]:

Replying to [comment:11 ]:

Replying to [comment:10 yoann]:

Replying to [comment:7 ]:
This is used for regression testing and reviewing generated alerts:

> > >   cd prelude-lml/tests
> > >   ./loggrep.py ../plugins/pcre/ruleset/<name of your ruleset>.rules | ../src/prelude-lml --dry-run --metadata=nowrite,head --batch-mode --no-resolve --pcre --dump-unmatched --config ./prelude-lml.conf --text-output

Hi,
I get this warning if I run your test: "WARNING: No alert emited for log entry" (for all LOG: entries). But I'm sure that this ruleset generates alerts, because I tested it in "normal" mode and it worked for each LOG: entry. So, what can be wrong?

#12 Updated by Yoann VANDOORSELAERE over 10 years ago

Replying to [comment:13 dkopecek@

#13 Updated by over 10 years ago

After looking at the issue, it only happen when the mod_security ruleset is used along with others ruleset: the problem is that the [[ModSecurity]] rules ID conflict with other ruleset ID. If running the mod_security ruleset by itself:

> 12 Aug 12:49:07 (process:30577) WARNING: 7 line processed in 0.02 seconds (358.42 EPS), 7 alert emited.

So the mod_security rules ID need to be given an unique range (You can use the old assigned range: 31XX).

Thanks! Now it's working.

12 Aug 13:20:37 (process:5970) WARNING: 7 line processed in 0.00 seconds (3396.41 EPS), 7 alert emited.

#14 Updated by Yoann VANDOORSELAERE over 10 years ago

Overall, the generated alert look good! Here are some feedback & questions:

  • IDMEF require the Reference name and url to not be empty.
  • What is [[ModSec]] Rule ID?
  • What is [[UniqueID]]?
  • What exactly is the log format used in the ruleset log sample (doesn't look like raw log entry, there is no timestamp)?

#15 Updated by over 10 years ago

Replying to [comment:16 yoann]:

Overall, the generated alert look good! Here are some feedback & questions:

  • IDMEF require the Reference name and url to not be empty.

See last attachment.

  • What is [[ModSec]] Rule ID?

This is the ID of the mod_security rule that generated the log entry.

  • What is [[UniqueID]]?

I don't know what this is exactly, but I think that apache identifies sessions with this. mod_unique_id is a separate apache module and mod_security requires it.

  • What exactly is the log format used in the ruleset log sample (doesn't look like raw log entry, there is no timestamp)?

I changed the format back to the version with timestamp (apache error_log). See last attachment.

#16 Updated by Yoann VANDOORSELAERE over 10 years ago

Replying to [comment:17 dkopecek@

#17 Updated by over 10 years ago

Replying to [comment:18 yoann]:

  • What is [[ModSec]] Rule ID?

This is the ID of the mod_security rule that generated the log entry.

Is the [[RuleID]] tied to the "msg" part of the log (aka: one ID per msg)?

Yes.

I am wondering whether we should use the "msg" part directly within classification.text.
...
Do you have any opinion on the matter?

The msg part can be very long and very specific. For example:

"Injection of Undocumented [[ColdFusion]] Tags. Matched signature <%{TX.0}>"

In error_log %{TX.0} is substituted with data from the session that triggered this alert. I thought that the classification.text should contain the 2 main alerts that modsecurity generates: warning and access denied. Should it be more specific?

How many different msg are there in current [[ModSecurity]] rules?

  1. sed -n "s/.*msg:'\([^']*\)'.*/\1/p" /etc/httpd/modsecurity.d/*.conf | sort | uniq | wc -l

64

...and probably more in the latest version.

#18 Updated by over 10 years ago

Replying to [comment:19 dkopecek@

#19 Updated by Yoann VANDOORSELAERE over 10 years ago

Replying to [comment:19 dkopecek@

#20 Updated by over 10 years ago

Replying to [comment:21 yoann]:

The ruleset now look ready to be included in the Prelude-LML repository, should I process with it now, or would you like to perform other modification before this is done?

Yes, you can use this version. I plan to add more rules, but don't have time to do it now. I'll let you now if a new ruleset is available or I simply open a new ticket.

#21 Updated by Yoann VANDOORSELAERE over 10 years ago

  • Status changed from Assigned to Closed
  • Resolution set to fixed

(In r10869) [[ModSecurity]] ruleset rewrite, by Peter Vrabec <> and
Dan Kopecek <>. This ruleset handle [[ModSecurity]] 2.0
output. Thanks for the contribution! Fix #216.

#22 Updated by Yoann VANDOORSELAERE almost 10 years ago

  • Project changed from PRELUDE SIEM to Prelude-LML
  • Category deleted (4)
  • Target version deleted (0.9.13)

Also available in: Atom PDF