Project

General

Profile

Prewikka 5.1.1 issues

Added by milan P 26 days ago

Hello,

I installed every components by compiling them from sources but i still have issues with Prewikka.
I spent time to debug before coming here to open this thread.
When prewikka is starting, it loads some plugins. Some of them are failing, resulting (i think) in an inapropriate interface/user experience.

2019-10-18 11:21:31,399 prewikka (pid:27005) prewikka.pluginmanager ERROR: prewikka.views.datasearch.threat: cannot import name grammar
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/dist-packages/prewikka/pluginmanager.py", line 101, in iter_plugins
    plugin_class = i.load()
  File "/usr/lib/python2.7/dist-packages/pkg_resources/__init__.py", line 2411, in load
    return self.resolve()
  File "/usr/lib/python2.7/dist-packages/pkg_resources/__init__.py", line 2417, in resolve
    module = __import__(self.module_name, fromlist=['__name__'], level=0)
  File "/usr/local/lib/python2.7/dist-packages/prewikka/views/datasearch/threat.py", line 30, in <module>
    from . import alert, datasearch
  File "/usr/local/lib/python2.7/dist-packages/prewikka/views/datasearch/alert.py", line 30, in <module>
    from . import idmef
  File "/usr/local/lib/python2.7/dist-packages/prewikka/views/datasearch/idmef.py", line 28, in <module>
    from . import datasearch
  File "/usr/local/lib/python2.7/dist-packages/prewikka/views/datasearch/datasearch.py", line 38, in <module>
    from prewikka.dataprovider.parsers import criteria, lucene
  File "/usr/local/lib/python2.7/dist-packages/prewikka/dataprovider/parsers/criteria/__init__.py", line 27, in <module>
    from . import grammar
ImportError: cannot import name grammar

This issue happens on these plugins :

2019-10-18 11:27:48,010 prewikka (pid:27043) prewikka.pluginmanager ERROR: prewikka.views.datasearch.threat: cannot import name grammar
2019-10-18 11:27:48,011 prewikka (pid:27042) prewikka.pluginmanager ERROR: prewikka.views.datasearch.heartbeat: cannot import name grammar
2019-10-18 11:27:48,013 prewikka (pid:27044) prewikka.pluginmanager ERROR: prewikka.views.datasearch.threat: cannot import name grammar
2019-10-18 11:27:48,015 prewikka (pid:27044) prewikka.pluginmanager ERROR: prewikka.views.datasearch.heartbeat: cannot import name grammar

I tried to troubleshoot the code but i didn't find the issue. It looks like a bug with the lark parser, but i am not sure.

For info:

*python2.7

lark-parser 0.7.7*

Can you help me ?


Replies (4)

RE: Prewikka 5.1.1 issues - Added by Antoine LUONG 26 days ago

Hi,

This is rather strange.

  • Does the /usr/local/lib/python2.7/dist-packages/prewikka/dataprovider/parsers/criteria/grammar.py file exist and have appropriate permissions?
  • Does your Python Path contain the /usr/local/lib/python2.7/dist-packages directory?
  • Does the same error occur when directly importing the module in a Python console?
>>> from prewikka.dataprovider.parsers import criteria

Regards

RE: Prewikka 5.1.1 issues - Added by milan P 23 days ago

Hello,

Thanks for your reply:

grammar.py exists :

ls -al /usr/local/lib/python2.7/dist-packages/prewikka/dataprovider/parsers/criteria/grammar.py
-rw-r--r-- 1 root staff 2117 oct. 8 12:39 /usr/local/lib/python2.7/dist-packages/prewikka/dataprovider/parsers/criteria/grammar.py

PYTHONPATH:

I tried to force it :

PYTHONPATH=/usr/local/lib/python2.7/dist-packages prewikka-httpd -p 80

or via export PYTHONPATH but the same errors.

From Console:

Python 2.7.16 (default, Apr  6 2019, 01:42:57)
[GCC 8.3.0] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> from prewikka.dataprovider.parsers import criteria
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/lib/python2.7/dist-packages/prewikka/dataprovider/parsers/criteria/__init__.py", line 30, in <module>
    _grammar = Lark(grammar.GRAMMAR, start="criteria", parser="lalr")
  File "/usr/local/lib/python2.7/dist-packages/lark/lark.py", line 231, in __init__
    self.parser = self._build_parser()
  File "/usr/local/lib/python2.7/dist-packages/lark/lark.py", line 250, in _build_parser
    parser_conf = ParserConf(self.rules, self._callbacks, self.options.start)
  File "/usr/local/lib/python2.7/dist-packages/lark/common.py", line 23, in __init__
    assert isinstance(start, list)
AssertionError

RE: Prewikka 5.1.1 issues - Added by Antoine LUONG 23 days ago

You may want to either:

  • downgrade lark-parser to 0.7.1
  • change the test in lark-parser:
    diff --git a/lark/lark.py b/lark/lark.py
    index c27f534..2635a5d 100644
    --- a/lark/lark.py
    +++ b/lark/lark.py
    @@ -86,7 +86,7 @@ class LarkOptions(Serialize):
    
                 options[name] = value
    
    -        if isinstance(options['start'], str):
    +        if isinstance(options['start'], (str, unicode)):  # this won't work with Python3
                 options['start'] = [options['start']]
    
             self.__dict__['options'] = options
    
  • or change the call in prewikka:
    diff --git a/prewikka/dataprovider/parsers/criteria/__init__.py b/prewikka/dataprovider/parsers/criteria/__init__.py
    index 7fcfbee..1865462 100644
    --- a/prewikka/dataprovider/parsers/criteria/__init__.py
    +++ b/prewikka/dataprovider/parsers/criteria/__init__.py
    @@ -27,7 +27,7 @@ from prewikka.dataprovider import Criterion, ParserError
     from . import grammar
    
    -_grammar = Lark(grammar.GRAMMAR, start="criteria", parser="lalr")
    +_grammar = Lark(grammar.GRAMMAR, start=["criteria"], parser="lalr")
    
     class CommonTransformer(Transformer):
    

Regards

RE: Prewikka 5.1.1 issues - Added by milan P 22 days ago

Hello Antoine,

I downgraded lark to 0.7.1 and it fixed the issue.
Thanks very much for your help !

    (1-4/4)