[sr-dev] Race condition in Makefile.libs

[Richard Fuchs]

Hi,

Actually I've looked into this further in the meantime and it seems that
the actual cause of the problem is elsewhere. It seems to be a basic
problem of how modules are made to depend on libs.

For example, avpops and lcr both depend on kcore. Their makefiles
specify this through:

SER_LIBS+=$(SERLIBPATH)/kcore/kcore

which Makefile.rules (line 164) translates into

SER_LIBS_DEPS=$(SERLIBPATH)/kcore/libkcore.so

which in turn gets the following rule and recipe a few lines below:

$(SER_LIBS_DEPS): FORCE
        @$(MAKE) -wC $(dir $@) ..........

So what happens during a parallel build is this: Make's thread A starts
compiling avpops while make's thread B starts compiling lcr at the same
time. Both make threads see the dependency on libkcore.so and thus
execute the sub-make to build it. Which means that you now have two make
children running at the same time in the same directory, trying to
compile the same target. Obviously they get into each others way.

I'm not sure if the make children are supposed to communicate with each
other and if one is supposed to know that the other one is compiling
kcore already or not. Taking out the FORCE dependency seems to help a
little, but it's still ending up with multiple makes trying to build the
same targets.

I'm still playing this this, trying to figure out if there's some make
flags that will fix it or if it's maybe a fundamental problem of make
and/or the way the makefiles are made.

cheers


On 01/30/13 07:15, Daniel-Constantin Mierla wrote:
> Hello,
> 
> haven't looked at it yet, just wondering if the symlink cannot be
> created as extra command in the make target building the object.
> 
> Cheers,
> Daniel
> 
> On 1/26/13 12:30 AM, Richard Fuchs wrote:
>> Hello,
>>
>> There seems to be a race condition in Makefile.libs that occurs
>> sporadically when doing parallel builds (make -j). I believe the culprit
>> part is this:
>>
>>
>>
>> $(NAME): $(LIB_RUNTIME_NAME) $(LIB_LINK_NAME) $(LIBINAME_F)
>>
>> $(LIB_RUNTIME_NAME):
>>         - at ln -s $(LIB_NAME) $(LIB_RUNTIME_NAME)
>>
>> $(LIB_LINK_NAME):
>> ifeq ($(OS), freebsd)
>>         - at ln -s $(LIB_RUNTIME_NAME) $(LIB_LINK_NAME)
>> else
>>         - at ln -s $(LIB_NAME) $(LIB_LINK_NAME)
>> endif
>>
>>
>>
>> The problem is that dependencies (the ln -s commands) get built first,
>> and the actual target ($(NAME)) last. The result is that for a while,
>> the symlinks already exist while the lib itself is not fully built yet.
>> In the meantime, other make children build something else that depends
>> on the lib, see that the symlink already exists and hence think the
>> target has already been built, which then results in various funny
>> errors when the linker tries to read a non-existent or half-finished .so
>> file. At least I think that's what happens.
>>
>> Unfortunately I have no idea how to fix this within the current makefile
>> framework. I tried reordering the dependencies (making the symlink
>> targets depend on the lib instead of the other way around) but those all
>> resulted in build failures.
>>
>> cheers
>>
>>
>>
>> _______________________________________________
>> sr-dev mailing list
>> sr-dev at lists.sip-router.org
>> http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-dev
> 
> -- 
> Daniel-Constantin Mierla - http://www.asipto.com
> http://twitter.com/#!/miconda - http://www.linkedin.com/in/miconda
> Kamailio World Conference, April 16-17, 2013, Berlin
>  - http://conference.kamailio.com -
> 
> 
> 
> _______________________________________________
> sr-dev mailing list
> sr-dev at lists.sip-router.org
> http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-dev

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 900 bytes
Desc: OpenPGP digital signature
URL: <http://lists.sip-router.org/pipermail/sr-dev/attachments/20130130/e8bf79e0/attachment-0001.pgp>