MYSQL DATABASE // LIMITS
How much is approximately esteemed the limit of mysql database with OSS version ?
Thank you in advance for your answer.
RE: MYSQL DATABASE // LIMITS - Added by Antoine LUONG about 3 years ago
The OSS database is limited regarding the insertion rate, but essentially the searching and displaying functionalities.
Its schema is not optimal for nowadays volume of alerts, leading to searches that can be slow and get slower the longer you store alerts.
Correlation also is slower as OSS uses only the script engine (the meta-language engine is faster).
RE: MYSQL DATABASE // LIMITS - Added by Pierre Pichard about 3 years ago
Thank you for your detailed answer.
RE: MYSQL DATABASE // LIMITS - Added by Ondřej Macháček almost 3 years ago
I'd like to ask you about some experience with large prelude SQL DB. We have cca 6 mil alerts in prelude_alerts table (size 0.35GB), cca 30 mil rows in prelude_analyzer (size 3.2GB) and cca 50 mil rows in preude_addres (cca 3,6GB). Whole DB needs 53GB of disk space. We are using 21 agents (prelude-manager, prelude-lml, audisp-prelude and suricata).
When I want to list all agents in Prewikka Agents menu I need to wait cca 5 minutes to get any answer from Prewikka. In this moment Prewikka is generating a large set of SELECT queries searching a lot of tables, repeating this set of SELECTs for every 21 agents-analyzers we are using. Some SELECTs takes 50 seconds to complete, others are faster. For 21 agents there is cca 200 of SELECT and whole procedure takes cca 5 minutes to complete.
I think schema of prelude DB is not optimized to work with large database and I think there is no other way to get quick response, than regularly backup prelude DB and create new empty one. Am I right?
We are using PostgreSQL, but I think that the problem, we have with PostgreSQL, would be the same with MySQL DB.
RE: MYSQL DATABASE // LIMITS - Added by Antoine LUONG almost 3 years ago
If the Agents page has performance problems, it can be caused by too many heartbeats in the database.
The heartbeats should be deleted regularly with the preludedb-admin command.