as well as message summary (require twisted.names and twisted.internet),

  see the additional dns_max_delay settings parameters in prewikka.conf.

  and show alert messageid.

- Fix exception when rendering ToolAlert.

- Only perform additional database request when using Sensor localtime:

  this bring a performance improvement of about 36% on aggregated query,

- JQuery support: Port most of the javascript code to make use of JQuery.

  Add show/hide effect to CSS popup. More filtering functionality in the

- Cleanup the Authentication class, so that uper Prewikka layer can act

  depending whether the backend support user creation / deletion. Anonymous

- Better integration of CGI authentication allowing user listing and deletion.

- Fix exception if an alert analyzer name is empty.

  (which is a minor issue since the user is already authenticated). Thanks

  to Helmut Azbest <helmut.azbest@gmail.com> for the fix.

  Helmut Azbest <helmut.azbest@gmail.com>).

- Localization fixes, by Sebastien Tricaud <toady@gscore.org>, and

  Bjoern Weiland.

* 2007-05-26, prewikka-0.9.11.2:

          database does not exist, make the error available from the Prewikka

          user to parse its web server log in order to find out the problem).

* 2007-05-25, prewikka-0.9.11.1:

	- Fix incorrect locale switch when accessing certain pages.

* 2007-05-21, prewikka-0.9.11:

        - Prewikka has been internationalized: user might choose the language

          used in their settings tabs. Additionally, you might specify

          a default locale using the "default_locale" configuration keyword.

        - French translation, by Sebastien Tricaud <sebastien@gscore.org>.

        - German translation, by Bjoern Weiland <mail@bjou.de>.

        - Spanish translation, by Carlo G. Añez M. <carlo.anez@gmail.com>.

        - In the Alert/Heartbeat summary view, number analyzers backward so that

          it reflect the ordering in the analyzer list.

        - Various bug fixes.

* 2007-04-05, prewikka-0.9.10:

          provide an expansion link.

        - Add two new view in the Events section: CorrelationAlert and ToolAlert.

          ToolAlert, OverflowAlert).

        - Prewikka can now aggregate by analyzer.

          he attempted to access when the session expired.

        - When an error occur, the default Prewikka layout is now preserved.

          non aggregated view by ~50%).

        - Avoid iterating the list of database result more than needed.

        - Support IDMEF Action, SNMPService, and WebService class.

        - Improved support for small screen resolution.

* 2007-02-06, prewikka-0.9.9:

        - Improve database performance by reducing the number of query. (Paul Robert Marino)

        - Activate CleanOutput filtering (lot of escaping fixes).

        - More action logging.

        - Bug fixes with the error pages Back/Retry buttons.

        - Fix error on group by user (#191).

        - Fix template compilation error with Cheetah version 2 (#184).

* 2006-11-23, prewikka-0.9.8:

	- Save/load user configuration when using CGI authentication mode (#181).

	- Show Prewikka version in the About page (#177).

	  multiple simultaneous handler supported (#113).

	- Fix anonymous authentication.

* 2006-08-18, prewikka-0.9.7.1:

	- Fix filter interface bug introduced in 0.9.7.

	- Improved error reporting on filter creation.

* 2006-08-16, prewikka-0.9.7:

	  of around 3000%.

	  source/target in the message listing.

	- Make host command/Information link available from the Sensor

	  listing.

	  command line arguments.

	- Avoid toggling several popup at once in the HeartbeatListing.

	- New address and node name lookup provided through prelude-ids.com service.

          database (fix #162).

	- Various bug fixes.

	- Show multiple source/target in message listing/summary.

	  which is stored as string (#156).

	- Fix aggregation on IDMEF-Path that are not string.

	- Intelligent display for CorrelationAlert. Include correlated

	  alert information in the alert listing.

	- Fix Cheetah compilation for the heartbeat page.

	- Distribute SQLite schema.

	- Fix exception in the heartbeat analysis view when the heartbeat_count

	- Open external link in new windows by default. Add a configuration option

	  to disable opening external link in new window (#61).

	  use (#117).

	- Sanitize the limit parameter in case the input value is not correct

	- Fix filter saving issue in the heartbeat listing.

	- Add an "Unlimited" timeline option.

	  listing when unwiding alert with classification text containing backslash.

	  empty in the alert listing.

	  summary and on the correlated alert listing.

* 2005-11-30, prewikka-0.9.1:

	  linking the protocol to portdb if there is no port.

	- Ability to setup IDMEF filter using iana_protocol_name and iana_protocol_number.

	- Sanitize timeline years value on system which does not support time

	  exceeding 2^31-1. Fix #104.

	- Ability for the user to save settings for the current view.

	- Fix a bug where clicking the IP address popup would cause

	  Firefox to go back to the top of the page. Fix #112.

	  /usr/bin/env to find it.

	- Minor rendering fix.

	  as well as service.protocol.

	- Show target file in the message listing.

	  Especially useful for users of integrity checker.

	- XHTML conformance in most of the code.

	- Fix possible exception with filtered classification text.

	- Allow filtering on heartbeat.analyzer.name.

	  Fix Javascript warnings. Correct URL escaping. Make it work

	  better in Apple's Safari browser.

	  filter reference non existing criteria. Add the substr operator.

	  the user.

	  listing table head.

	  severity.

	- Fix low/mid/high/none severity filtering.

	- Fix a bug where agents with multiple address would disappear.

	- Add an heartbeat_error_margin configuration keyword.

	- Saving modification to an existing filter now work.

	  initialization step and display an error screen to the user instead of

	  a server internal error.

	- Update Auth cookie expiration time.

	- Fix escaping issue.

* 2005-04-05, prewikka-0.9.0-rc4:

          login for no appearent reasons.

	- Set default session expiration time to 60 minutes.

* 2005-03-31, prewikka-0.9.0-rc3:

	- Installation cleanup / bugfix.

class IDMEFTime(object):

    def __init__(self, res):

        self._res = res

    def __del__(self):

        idmef_time_destroy(self._res)

    def __int__(self):

        return idmef_time_get_sec(self._res)

    def __float__(self):

        return float(idmef_time_get_sec(self._res)) + float(idmef_time_get_usec(self._res)) / 10 ** 6

    def toYMDHMS(self):

        return time_to_ymdhms(time.localtime(idmef_time_get_sec(self._res)))

    def __getattribute__(self, name):

        if name is "sec":

            return idmef_time_get_sec(self._res)

    except KeyError:

        return None

class Message:

    def __init__(self, res, htmlsafe):

        self._res = res

        self._value_list = None

        self._htmlsafe = htmlsafe

    def __del__(self):

        idmef_message_destroy(self._res)

    def __iter__(self):

        if not self._value_list:

            raise TypeError, "iteration over a non-sequence"

        self._list_iterator = 0

        return self

            return idmef_value_get_count(self._value_list)

        return 1

    def next(self):

        next = idmef_value_get_nth(self._value_list, self._list_iterate)

        if not next:

        value = self._convert_value(next, self._root + "(%d)" % self._list_iterate)

        self._list_iterate += 1

        return value

    def _convert_value(self, idmef_value, key):

            value._value_list = idmef_value

            if self._value_list:

                idmef_value_ref(idmef_value)

        elif idmef_value_get_type(idmef_value) != IDMEF_VALUE_TYPE_CLASS:

            value = convert_idmef_value(idmef_value)

            if not self._value_list:

                idmef_value_destroy(idmef_value)

        else:

            if not self._value_list:

                idmef_value_destroy(idmef_value)

            value = Message(idmef_message_ref(self._res), self._htmlsafe)

            value._root = key

        return value

    def _get_raw_value(self, key):

        path = idmef_path_new_fast(key)

        idmef_value = idmef_path_get(path, self._res)

        if idmef_value:

            ret = self._convert_value(idmef_value, key)

        else:

                ret = []

            else:

                ret = None

        idmef_path_destroy(path)

        return ret

            return escape_value(self._get_raw_value(key))

        else:

            return self._get_raw_value(key)

    def match(self, criteria):

        if type(criteria) is list:

            criteria = " && ".join(criteria)

        return ret

        if htmlsafe != None:

            htmlsafe_bkp = self._htmlsafe

            self._htmlsafe = htmlsafe

        if val == None:

                val = default

        if htmlsafe != None:

            self._htmlsafe = htmlsafe_bkp

        return val

    def getAdditionalData(self, searched, many_values=False):

            meaning = self["%s.additional_data(%d).meaning" % (self._root, i)]

            if meaning is None:

                break

            if meaning == searched:

                value = self["%s.additional_data(%d).data" % (self._root, i)]

                if not many_values:

                    return value

                values.append(value)

            i += 1

        self._rows = [ ]

        self._has_cache = False

        self._res, self._len = results

    def __iter__(self):

        if self._has_cache:

            return iter(self._rows)

        else:

            return self

    def __len__(self):

        return self._len

    def __del__(self):

        if self._res:

            self._db_delete(self._res)

    def __getitem__(self, key):

        if isinstance(key, types.SliceType):

            start, stop, step = key.start, key.stop, key.step

            index = start + stop

        else:

            index = key

        if not self._has_cache:

            for r in self:

                if len(self._rows) >= index:

                    break

        return self._rows[key]

        if self._res == None:

            raise StopIteration

            raise StopIteration

        row = self._db_convert_row(values)

        self._rows.append(row)

        return row

class DbResultValues(DbResult):

    def __init__(self, selection, results):

        self._selection = selection

        DbResult.__init__(self, results)

    def _db_get_next(self):

        return preludedb_result_values_get_next(self._res)

    def _db_delete(self, result):

        if self._selection:

            preludedb_path_selection_destroy(self._selection)

        if result:

            preludedb_result_values_destroy(result)

    def _db_convert_row(self, values):

        row = []

        for value in values:

               row.append(None)

           else:

               row.append(convert_idmef_value(value))

        return row

class DbResultIdents(DbResult):

    def _db_get_next(self):

        return preludedb_result_idents_get_next(self._res)

    def _db_delete(self, result):

        if result:

            preludedb_result_idents_destroy(result)

    def _db_convert_row(self, value):

        return value

class IDMEFDatabase:

    _db_destroy = preludedb_destroy

    _db = None

    def __init__(self, config):

        settings = preludedb_sql_settings_new()

        for param in "file", "host", "port", "name", "user", "pass":

                raise "libpreludedb %s or higher is required (%s found)." % (wanted_version, cur)

            else:

                raise "libpreludedb %s or higher is required." % wanted_version

        self._db = preludedb_new(sql, None)

    def __del__(self):

        if self._db:

            self._db_destroy(self._db)

    def _getMessageIdents(self, get_message_idents, criteria, limit, offset, order_by):

        if type(criteria) is list:

            if len(criteria) == 0:

                criteria = None

            else:

                criteria = " && ".join(criteria)

        if criteria:

            criteria = idmef_criteria_new_from_string(criteria)

        idents = [ ]

        if order_by == "time_asc":

            order_by = PRELUDEDB_RESULT_IDENTS_ORDER_BY_CREATE_TIME_ASC

        else:

            order_by = PRELUDEDB_RESULT_IDENTS_ORDER_BY_CREATE_TIME_DESC

            result = get_message_idents(self._db, criteria, limit, offset, order_by)

        except:

            self._freeDbParams(criteria=criteria)

            raise

        if criteria:

            idmef_criteria_destroy(criteria)

        if not result:

        return DbResultIdents(result)

    def getAlertIdents(self, criteria=None, limit=-1, offset=-1, order_by="time_desc"):

        return self._getMessageIdents(preludedb_get_alert_idents2, criteria, limit, offset, order_by)

    def _getLastMessageIdent(self, type, get_message_idents, analyzerid):

        criteria = None

        idents = get_message_idents(criteria, limit=1)

        return idents[0]

        return self._getLastMessageIdent("alert", self.getAlertIdents, analyzer)

        return self._getLastMessageIdent("heartbeat", self.getHeartbeatIdents, analyzer)

    def getAlert(self, ident, htmlsafe=False):

    def _freeDbParams(self, selection=None, criteria=None):

        if selection:

            preludedb_path_selection_destroy(selection)

        if criteria:

            idmef_criteria_destroy(criteria)

    def getValues(self, selection, criteria=None, distinct=0, limit=-1, offset=-1):

        if type(criteria) is list:

            if len(criteria) == 0:

                criteria = None

            else:

                criteria = " && ".join([ "(" + c + ")" for c in criteria ])

        if criteria:

            criteria = idmef_criteria_new_from_string(criteria)

        my_selection = preludedb_path_selection_new()

        for selected in selection:

            my_selected = preludedb_selected_path_new_string(selected)

        except:

            self._freeDbParams(my_selection, criteria)

            raise

        if criteria:

        if not result:

            preludedb_path_selection_destroy(my_selection)

            return [ ]

        return DbResultValues(my_selection, result)

    def _countMessages(self, root, criteria):

        return self.getValues(["count(%s.create_time)" % root], criteria)[0][0]

    def countAlerts(self, criteria=None):

        return self._countMessages("alert", criteria)

                index += 1

            analyzer_paths.append(path)

    def getAnalyzer(self, analyzerid):

        ident = self.getLastHeartbeatIdent(analyzerid)

        heartbeat = self.getHeartbeat(ident)

  });

  \$(".fieldset_toggle2").click(function(){

  });

  \$("td.offline, td.online, td.missing, td.unknown").click(function(){

    <tbody>

      <tr class="table_row_even">

        <td>#echo $analyzer.name or "n/a" #</td>

          #if $analyzer.model

            $analyzer.model

            #if $analyzer.version

            n/a

          #end if

        </td>

          #if $analyzer.ostype

            $analyzer.ostype

            #if $analyzer.osversion

            n/a

          #end if

        </td>

        <td>#echo $analyzer.node_location or "n/a" #</td>

          #if len($analyzer.node_addresses) > 0

            #for $address in $analyzer.node_addresses

              $address<br/>

          #else

            n/a

          #end if

      </tr>

    </tbody>

  </table>

    if time.time() - int(heartbeat_time) > int(heartbeat_interval) + error_margin:

        return "missing", _("Missing")

    return "online", _("Online")

def analyzer_cmp(x, y):

    xmiss = x["status"] == "missing"

    ymiss = y["status"] == "missing"

    if xmiss and ymiss:

        return cmp(x["name"], y["name"])

    elif xmiss or ymiss:

        return ymiss - xmiss

    else:

        return cmp(x["name"], y["name"])

def node_cmp(x, y):

    xmiss = x["missing"]

    ymiss = y["missing"]

    if xmiss or ymiss:

    else:

        return cmp(x["node_name"], y["node_name"])

    def init(self, env):

        self._heartbeat_count = int(env.config.general.getOptionValue("heartbeat_count", 30))

        self._heartbeat_error_margin = int(env.config.general.getOptionValue("heartbeat_error_margin", 3))

    def render(self):

        analyzers = { }

        locations = { }

        nodes = { }

            analyzer = self.env.idmef_db.getAnalyzer(analyzerid)

            parameters = { "analyzerid": analyzer["analyzerid"] }

            analyzer["alert_listing"] = utils.create_link("sensor_alert_listing", parameters)

            analyzer["heartbeat_listing"] = utils.create_link("sensor_heartbeat_listing", parameters)

                analyzer["node_name_link"] = utils.create_link(self.view_name,

                                                               { "filter_path": "heartbeat.analyzer(-1).node.name",

                                                                 "filter_value": analyzer["node_name"] })

            if analyzer["node_location"]:

                analyzer["node_location_link"] = utils.create_link(self.view_name,

                                                                   { "filter_path": "heartbeat.analyzer(-1).node.location",

                                                                     "filter_value": analyzer["node_location"] })

            node_key = ""

            for i in range(len(analyzer["node_addresses"])):

                addr = analyzer["node_addresses"][i]

                node_key += addr

                analyzer["node_addresses"][i] = {}

                analyzer["node_addresses"][i]["value"] = addr

                analyzer["node_addresses"][i]["inline_filter"] = utils.create_link(self.view_name,

                                                                           utils.create_link("Command",

                                                                                             { "origin": self.view_name,

                                                                                               "command": command, "host": addr })))

            analyzer["status"], analyzer["status_meaning"] = \

                                get_analyzer_status_from_latest_heartbeat(analyzer["last_heartbeat_status"],

                                                                          analyzer["last_heartbeat_time"],

            analyzer["last_heartbeat_time"] = utils.time_to_ymdhms(time.localtime(int(analyzer["last_heartbeat_time"]))) + \

                                              " %+.2d:%.2d" % utils.get_gmt_offset()

            node_location = analyzer["node_location"] or _("Node location n/a")

            node_name = analyzer.get("node_name") or _("Node name n/a")

            osversion = analyzer["osversion"] or _("OS version n/a")

            ostype = analyzer["ostype"] or _("OS type n/a")

            addresses = analyzer["node_addresses"]

            node_key = node_name + osversion + ostype

            if not locations.has_key(node_location):

                locations[node_location] = { "total": 1, "missing": 0, "unknown": 0, "offline": 0, "online": 0, "nodes": { } }

            else:

                locations[node_location]["nodes"][node_key] = { "total": 1, "missing": 0, "unknown": 0, "offline": 0, "online": 0,

                                                                "analyzers": [ ],

                                                                "node_name": node_name, "node_location": node_location,

                                                                "node_addresses": addresses, "node_key": node_key }

            else:

                locations[node_location]["nodes"][node_key]["total"] += 1

            status = analyzer["status"]

            locations[node_location][status] += 1

            locations[node_location]["nodes"][node_key][status] += 1

                locations[node_location]["nodes"][node_key]["analyzers"].insert(0, analyzer)

            else:

                locations[node_location]["nodes"][node_key]["analyzers"].append(analyzer)

        self.dataset["locations"] = locations

class SensorMessagesDelete(SensorListing):

    view_name = "sensor_messages_delete"

            if self.parameters.has_key("heartbeats"):

                criteria = "heartbeat.analyzer(-1).analyzerid == %d" % long(analyzerid)

                self.env.idmef_db.deleteHeartbeat(self.env.idmef_db.getHeartbeatIdents(criteria))

        SensorListing.render(self)

    def init(self, env):

        self._heartbeat_count = int(env.config.general.getOptionValue("heartbeat_count", 30))

        self._heartbeat_error_margin = int(env.config.general.getOptionValue("heartbeat_error_margin", 3))

    def render(self):

        analyzerid = self.parameters["analyzerid"]

        analyzer = self.env.idmef_db.getAnalyzer(analyzerid)

        analyzer["last_heartbeat_time"] = str(analyzer["last_heartbeat_time"])

        analyzer["events"] = [ ]

        analyzer["status"] = "abnormal_offline"

        analyzer["status_meaning"] = "abnormal offline"

        start = time.time()

        idents = self.env.idmef_db.getHeartbeatIdents(criteria="heartbeat.analyzer(-1).analyzerid == %d" % analyzerid,

                                                      limit=self._heartbeat_count)

                    analyzer["events"].append({ "value": "sensor is down since %s" % older_time, "type": "down"})

            if newer:

                event = None

                if newer_status == "starting":

                    if older_status == "exiting":

                        event = { "value": "normal sensor start at %s" % str(newer_time),

                    if abs(int(newer_time) - int(older_time) - int(older_interval)) > self._heartbeat_error_margin:

                        event = { "value": "abnormal heartbeat interval between %s and %s" % (str(older_time), str(newer_time)),

                                  "type": "abnormal_heartbeat_interval" }

                if newer_status == "exiting":

                    event = { "value": "normal sensor stop at %s" % str(newer_time),