Re: [MERGE] build-test: fixed err detection and added multi-test support

Alex Rousskov wrote:
> On 03/09/2009 04:01 PM, Amos Jeffries wrote:
>> Target string is logged by the master make file _after_ all make targets
>> are completed successfully. Anything else is a bug, either in code or in
>> TestBed.
>>
> This is not true, on several layers:
>
> 1) The string is logged when all-am target is done. The check target,
> among others, is build _after_ all-am. You can grep raw Makefile for
> all-am to see the dependencies.

Not by the testbed. It runs checks first to catch the dependencies, then
all-am last to be sure of the basic build.

>
> 2) The string is logged when make fails if make is told to ignore
> errors. This does not happen in the build-test context (by default), but
> often happens during developement, when I ran "make -k" to get more
> errors than one. It is rather confusing to see the "Build Successful"
> message when there are hundreds of errors above it.

Would you not define a successful run when make finishes all its tasks
as requested to do so?

>
> 3) "Make" is designed to exit with an error when there is an error. We
> should not be re-implementing that logic. The problems with the current
> script and the above caveats are all cases in point.
>
> Overall, it seems strange to me to reject the change that uses the
> correct interface to detect build errors while the current code is using
> a half-broken hack.

I agree we should not use a hack when there is a legit method of
detecting all and any errors. I' just recalling past history of the
development of the testbed when this exact approach left some errors
uncaught.

I'm able to stay up late on IRC tonight so will try to catch you in a
direct conversation about this later.

>
>>> The build is still considered failed if error strings are found
>>> in the log.
>> More Major Warning Bells are going off in my head on that Alex.
>> The first design attempt used this enumerating-badness method. Far too
>> many false-positives were getting through. I'd rather face the odd
>> false-negative like this than hobble the testing.
>>
> I am fine with deleting the code that greps for errors. I just wanted to
> be as conservative as I could when fixing the bigger problem. Note that
> my change does not make the error grepping code better or worse so I am
> not sure why you are talking about bells going off.
>
> Should I submit a revised patch that deletes the old error-grepping code?
>
>> Hmm, on that 'make all check distcheck'....
>> Do you know if doing that is faster than make all && make check && make
>> distcheck? Maybe we should be doing that to speed things up a little.
>>
> Marginally faster.

Okay even marginal improvement is good in this place.

Amos

-- 
Please be using
   Current Stable Squid 2.7.STABLE6 or 3.0.STABLE13
   Current Beta Squid 3.1.0.6