Project

General

Profile

Bug #385

Infinite loop in bufpool_add_message

Added by Friedrich Delgado about 9 years ago. Updated almost 4 years ago.

Status:
Closed
Priority:
High
Start date:
08/02/2010
Due date:
% Done:

100%

Resolution:

Description

After some while (a few hours or a few days) our prelude-manager 1.0.0 with several sensors will go into an infinite loop in bufpool_add_message.

In an attached gdb it looks like this loop will never terminate:

        while ( get_total_mem() + len >= on_disk_threshold ) {
                evicted = evict_from_memory();
                if ( evicted == bp )
                        break;
        }

get_total_mem() always returns 0 and evict_from_memory() appears to always return NULL. If I set a breakpoint in the line after the loop, it is never reached.

Specifically, with a breakpoint in line 227 of bufpool.c and display on 'evicted', 'len' and 'mem_msglen' (the value returned by get_total_mem) when the breakpoint is hit it will always look like this:

Breakpoint 5, bufpool_add_message (bp=0x809c0d8, msg=0xb64e6008) at bufpool.c:227
227                     if ( evicted == bp )
3: mem_msglen = 0
2: len = 1124069
1: evicted = (bufpool_t *) 0x0

0001-Prevent-infinite-loop.patch View (1005 Bytes) Yoann VANDOORSELAERE, 08/02/2010 06:38 PM

Associated revisions

Revision f13f3cf3 (diff)
Added by Yoann Vandoorselaere about 9 years ago

Prevent infinite loop

An infinite loop was possible on insertion of an event in the processing
queue in case the event len was higher than the 'on disk threshold' value.
Closes #385.

Thanks Friedrich Delgado <> for reporting the problem,
and the useful debug information!

History

#1 Updated by Friedrich Delgado about 9 years ago

Some additional info:

It appears that it's quite directly dependant on the amount of events coming in, if the manager will go into the loop. After it hasn't been accepting data for a few days and I restart it, it starts receiving lots of messages (at least 288 messages per day per sensor, from about 5 sensors) and then stops pretty quickly. After restarting it once or twice again, it will keep running for a few days.

I've tried to put

sched-buffer-size = 101M

into the prelude-manager.conf as a temporary workaround. This might lead to high memory consumption (101MB of main memory) but if I understand the logic correctly, the loop should not be entered at all.

I'm aware that you might need additional information to be able to reproduce this, but I have no idea which information that might be, so please just feel free to ask!

#2 Updated by Yoann VANDOORSELAERE about 9 years ago

  • File 0001-Prevent-infinite-loop.patch added
  • Status changed from New to Assigned
  • Assignee set to Yoann VANDOORSELAERE
  • Priority changed from Normal to High
  • Target version set to 1.0.1

Thank you for the detailed report! Could you please try the attached patch, and let us know whether this solve your problem?

Regards,

#3 Updated by Yoann VANDOORSELAERE about 9 years ago

  • File deleted (0001-Prevent-infinite-loop.patch)

#5 Updated by Friedrich Delgado about 9 years ago

Thanks for the quick response.

I've applied the patch, recompiled and reinstalled and disabled the workaround.

I'm quite sure that there will be no infinite loop any more, but I'll get back to you in a few days, if I see any further problems.

#6 Updated by Friedrich Delgado about 9 years ago

Patched prelude manager 1.0.0 has been running smoothly for 3 days now, no problems in sight. Thanks! ;)

#7 Updated by Anonymous about 9 years ago

  • Status changed from Assigned to Resolved
  • % Done changed from 0 to 100

#8 Updated by Thomas ANDREJAK almost 4 years ago

  • Target version changed from 1.0.1 to Prelude OSS 1.0.1

#9 Updated by Thomas ANDREJAK almost 4 years ago

  • Status changed from Resolved to Closed

Also available in: Atom PDF