Project

General

Profile

MYSQL DATABASE // LIMITS

Added by Pierre Pichard almost 3 years ago

Hi all,
How much is approximately esteemed the limit of mysql database with OSS version ?
Thank you in advance for your answer.
Pierre


Replies (4)

RE: MYSQL DATABASE // LIMITS - Added by Antoine LUONG almost 3 years ago

Hello,

The OSS database is limited regarding the insertion rate, but essentially the searching and displaying functionalities.
Its schema is not optimal for nowadays volume of alerts, leading to searches that can be slow and get slower the longer you store alerts.

Correlation also is slower as OSS uses only the script engine (the meta-language engine is faster).

Regards

RE: MYSQL DATABASE // LIMITS - Added by Pierre Pichard almost 3 years ago

Thank you for your detailed answer.

Regards

RE: MYSQL DATABASE // LIMITS - Added by Ondřej Macháček over 2 years ago

Hello,
I'd like to ask you about some experience with large prelude SQL DB. We have cca 6 mil alerts in prelude_alerts table (size 0.35GB), cca 30 mil rows in prelude_analyzer (size 3.2GB) and cca 50 mil rows in preude_addres (cca 3,6GB). Whole DB needs 53GB of disk space. We are using 21 agents (prelude-manager, prelude-lml, audisp-prelude and suricata).
When I want to list all agents in Prewikka Agents menu I need to wait cca 5 minutes to get any answer from Prewikka. In this moment Prewikka is generating a large set of SELECT queries searching a lot of tables, repeating this set of SELECTs for every 21 agents-analyzers we are using. Some SELECTs takes 50 seconds to complete, others are faster. For 21 agents there is cca 200 of SELECT and whole procedure takes cca 5 minutes to complete.
I think schema of prelude DB is not optimized to work with large database and I think there is no other way to get quick response, than regularly backup prelude DB and create new empty one. Am I right?
We are using PostgreSQL, but I think that the problem, we have with PostgreSQL, would be the same with MySQL DB.

Ondrej Machacek

RE: MYSQL DATABASE // LIMITS - Added by Antoine LUONG over 2 years ago

Hello,

If the Agents page has performance problems, it can be caused by too many heartbeats in the database.
The heartbeats should be deleted regularly with the preludedb-admin command.

Regards

    (1-4/4)