Hello,

Here is my rule when using [[ModSecurity]](2)

#LOG:May 31 11:51:09 server httpdr10634: [error] [client 10.8.0.128] [[ModSecurity]]: Access denied with code 400 (phase 2). Pattern match "^[\\\\d\\\\.]+$" at REQUEST_HEADERS:Host. [id "960017"] [msg "Host header is a numeric IP address"] [severity "CRITICAL"] [hostname "10.8.0.127"] [uri "/blah"] [unique_id "CcdHJFKU22EAACmK5AcAAAAF"]
regex=\[client ([\d\.]+)\] [[ModSecurity]]: Access denied with code (\d+) \(phase (\d+)\)\. Pattern match "(\S+)" (.+)\. \[id "(\d+)"\] \[msg "(.+)"\] \[severity "(\S+)"\] \[hostname "([\S.]+)"\] \[uri "(.+)"\] \[unique_id "(\S+)"\]; \
 id=3108; \
 revision=1; \
 classification.ident = $11; \
 classification.text=HTTP $5 ($4) Blocked; \
 analyzer(0).name=ModSecurity; \
 analyzer(0).manufacturer=www.modsecurity.org; \
 analyzer(0).class=HIDS; \
 assessment.impact.severity=$8; \
 assessment.impact.completion=failed; \
 assessment.impact.description=mod_security encountered an error: $7.; \
 assessment.action(0).category = block-installed; \
 assessment.impact.description=Access was blocked with HTTP response code $2; \
 source(0).service.iana_protocol_name=tcp; \
 source(0).service.iana_protocol_number=6; \
 source(0).node.address(0).category=ipv4-addr; \
 source(0).node.address(0).address=$1; \
 target(0).node.name=$9; \
 target(0).service.iana_protocol_name=tcp; \
 target(0).service.iana_protocol_number=6; \
 target(0).service.name=http; \
 target(0).service.web_service.url = $10; \
 additional_data(0).type=integer; \
 additional_data(0).meaning=HTTP code returned; \
 additional_data(0).data=$2; \
 additional_data(1).type=integer; \
 additional_data(1).meaning=id; \
 additional_data(1).data=$6; \
 additional_data(2).type=integer; \
 additional_data(2).meaning=phase; \
 additional_data(2).data=$3; \
 last

Regards,

Robin

Hi Robin,
could you please send us some more logs from modsecurity (or put it somewhere so we can download it)?
We need some input for audit of the ruleset.

Thanks,
Dan

Replying to [comment:2 anonymous]:

Hello,

Here is my rule when using [[ModSecurity]](2)
....
....
Regards,

Robin

Robin apparently changed mail address. Adding the new one to the list.

we appreciate any feedback on our mod_security 2.0 ruleset attached by dkopecek@redhat.com.
thnx.

I haven't found a reference manual for the prelude-lml ruleset keywords like chained, silent, min-opt-goto. Is it somewhere available?

I have just added some documentation concerning rule flow control at the end of the wiki:PreludeLML page.
Please let me know if you have any other question!

Replying to [comment:7 pvrabec@redhat.com]:

we appreciate any feedback on our mod_security 2.0 ruleset attached by dkopecek@redhat.com.

Thanks! could you update the ruleset to include some log sample, using the following syntax:

#LOG:May 10 15:19:28 mail clamdr14292: /usr/share/doc/clamav-0.70/test/test2.badext: [[ClamAV]]-Test-Signature FOUND

This is used for regression testing and reviewing generated alerts:

  cd prelude-lml/tests
  ./loggrep.py ../plugins/pcre/ruleset/<name of your ruleset>.rules | ../src/prelude-lml --dry-run --metadata=nowrite,head --batch-mode --no-resolve --pcre --dump-unmatched --config ./prelude-lml.conf --text-output

Replying to [comment:10 yoann]:

Replying to [comment:7 pvrabec@redhat.com]:

we appreciate any feedback on our mod_security 2.0 ruleset attached by dkopecek@redhat.com.

Thanks! could you update the ruleset to include some log sample, using the following syntax:

Yeah, I forgot. Fixed.

This is used for regression testing and reviewing generated alerts:

>   cd prelude-lml/tests
>   ./loggrep.py ../plugins/pcre/ruleset/<name of your ruleset>.rules | ../src/prelude-lml --dry-run --metadata=nowrite,head --batch-mode --no-resolve --pcre --dump-unmatched --config ./prelude-lml.conf --text-output

I tested the ruleset "in action" (apache with modsec, prelude, prewikka) and it worked fine. However, I tried to test the ruleset using command you mentioned but it ends up with this error messages:

$ ./loggrep.py modsecurity.rules | ../src/prelude-lml --dry-run --metadata=nowrite,head --batch-mode --no-resolve --pcre --dump-unmatched --config ./prelude-lml.conf --text-output
Invalid option -- "pcre" (0).
Invalid option -- "dump-unmatched" (0).
./prelude-lml.conf:52: invalid section : "Pcre".
./prelude-lml.conf:53: invalid option "ruleset" in "global" section.
01 Aug 15:44:51 (process:27226) ERROR: couldn't open config file /usr/local/etc/prelude-lml/plugins.rules. (regex.c:168 get_regex_table)
01 Aug 15:44:51 (process:27226) WARNING: error while setting option 'file': No such file or directory.
Traceback (most recent call last):
  File "./loggrep.py", line 29, in <module>
    print i[:-1]
IOError: [Errno 32] Broken pipe

Version of prelude-lml is 0.9.12.2

$ pwd
/home/dkopecek/devel/prelude-lml/F-9/prelude-lml-0.9.12.2/tests
$ make tests
./loggrep.py ../plugins/pcre/ruleset/*.rules | ../src/prelude-lml --quiet --dry-run --metadata=nowrite,head --batch-mode --no-resolve --pcre --dump-unmatched --config ./prelude-lml.conf 2>&1 | /bin/grep -Fvf ignored
grep: ignored: No such file or directory
Traceback (most recent call last):
  File "./loggrep.py", line 29, in <module>
    print i[:-1]
IOError: [Errno 32] Broken pipe
make: *** [tests] Error 2

Replying to [comment:11 dkopecek@redhat.com]:

Replying to [comment:10 yoann]:

Replying to [comment:7 pvrabec@redhat.com]:
This is used for regression testing and reviewing generated alerts:

> >   cd prelude-lml/tests
> >   ./loggrep.py ../plugins/pcre/ruleset/<name of your ruleset>.rules | ../src/prelude-lml --dry-run --metadata=nowrite,head --batch-mode --no-resolve --pcre --dump-unmatched --config ./prelude-lml.conf --text-output

I tested the ruleset "in action" (apache with modsec, prelude, prewikka) and it worked fine. However, I tried to test the ruleset using command you mentioned but it ends up with this error messages:

> $ ./loggrep.py modsecurity.rules | ../src/prelude-lml --dry-run --metadata=nowrite,head --batch-mode --no-resolve --pcre --dump-unmatched --config ./prelude-lml.conf --text-output
> 01 Aug 15:44:51 (process:27226) ERROR: couldn't open config file /usr/local/etc/prelude-lml/plugins.rules. (regex.c:168 get_regex_table)

The previous error are most probably due to the absence of plugin.rules: make sure prelude-lml is installed before running the tests.

<pre>
> $ pwd
> /home/dkopecek/devel/prelude-lml/F-9/prelude-lml-0.9.12.2/tests
> $ make tests
> ./loggrep.py ../plugins/pcre/ruleset/*.rules | ../src/prelude-lml --quiet --dry-run --metadata=nowrite,head --batch-mode --no-resolve --pcre --dump-unmatched --config ./prelude-lml.conf 2>&1 | /bin/grep -Fvf ignored
> grep: ignored: No such file or directory
> Traceback (most recent call last):
>   File "./loggrep.py", line 29, in <module>
>     print i[:-1]
> IOError: [Errno 32] Broken pipe
> make: *** [tests] Error 2
</pre>

The file is missing, you can just _touch ignored_ in order to fixes this problem.

Replying to [comment:12 yoann]:

Replying to [comment:11 dkopecek@redhat.com]:

Replying to [comment:10 yoann]:

Replying to [comment:7 pvrabec@redhat.com]:
This is used for regression testing and reviewing generated alerts:

> > >   cd prelude-lml/tests
> > >   ./loggrep.py ../plugins/pcre/ruleset/<name of your ruleset>.rules | ../src/prelude-lml --dry-run --metadata=nowrite,head --batch-mode --no-resolve --pcre --dump-unmatched --config ./prelude-lml.conf --text-output

Hi,
I get this warning if I run your test: "WARNING: No alert emited for log entry" (for all LOG: entries). But I'm sure that this ruleset generates alerts, because I tested it in "normal" mode and it worked for each LOG: entry. So, what can be wrong?

Replying to [comment:13 dkopecek@

After looking at the issue, it only happen when the mod_security ruleset is used along with others ruleset: the problem is that the [[ModSecurity]] rules ID conflict with other ruleset ID. If running the mod_security ruleset by itself:

> 12 Aug 12:49:07 (process:30577) WARNING: 7 line processed in 0.02 seconds (358.42 EPS), 7 alert emited.

So the mod_security rules ID need to be given an unique range (You can use the old assigned range: 31XX).

Thanks! Now it's working.

12 Aug 13:20:37 (process:5970) WARNING: 7 line processed in 0.00 seconds (3396.41 EPS), 7 alert emited.

Overall, the generated alert look good! Here are some feedback & questions:

• IDMEF require the Reference name and url to not be empty.
• What is [[ModSec]] Rule ID?
• What is [[UniqueID]]?
• What exactly is the log format used in the ruleset log sample (doesn't look like raw log entry, there is no timestamp)?

Replying to [comment:16 yoann]:

Overall, the generated alert look good! Here are some feedback & questions:

• IDMEF require the Reference name and url to not be empty.

See last attachment.

• What is [[ModSec]] Rule ID?

This is the ID of the mod_security rule that generated the log entry.

• What is [[UniqueID]]?

I don't know what this is exactly, but I think that apache identifies sessions with this. mod_unique_id is a separate apache module and mod_security requires it.

• What exactly is the log format used in the ruleset log sample (doesn't look like raw log entry, there is no timestamp)?

I changed the format back to the version with timestamp (apache error_log). See last attachment.

Replying to [comment:17 dkopecek@

Replying to [comment:18 yoann]:

• What is [[ModSec]] Rule ID?

This is the ID of the mod_security rule that generated the log entry.

Is the [[RuleID]] tied to the "msg" part of the log (aka: one ID per msg)?

Yes.

I am wondering whether we should use the "msg" part directly within classification.text.
...
Do you have any opinion on the matter?

The msg part can be very long and very specific. For example:

"Injection of Undocumented [[ColdFusion]] Tags. Matched signature <%{TX.0}>"

In error_log %{TX.0} is substituted with data from the session that triggered this alert. I thought that the classification.text should contain the 2 main alerts that modsecurity generates: warning and access denied. Should it be more specific?

How many different msg are there in current [[ModSecurity]] rules?

1. sed -n "s/.*msg:'\([^']*\)'.*/\1/p" /etc/httpd/modsecurity.d/*.conf | sort | uniq | wc -l

64

...and probably more in the latest version.

Replying to [comment:19 dkopecek@

Replying to [comment:19 dkopecek@

Replying to [comment:21 yoann]:

The ruleset now look ready to be included in the Prelude-LML repository, should I process with it now, or would you like to perform other modification before this is done?

Yes, you can use this version. I plan to add more rules, but don't have time to do it now. I'll let you now if a new ruleset is available or I simply open a new ticket.