use of 'testing' function from testsuite/testing.sh

[Denys Vlasenko]

On Friday 02 May 2008 16:29, Cristian Ionescu-Idbohrn wrote:
> Documentation:
> 
>  # The "testing" function takes five arguments:
>  #       $1) Description to display when running command
> -#       $2) Command line arguments to command
> +#       $2) Command and command line arguments

Command(s) to run. May have pipes, redirects, etc

>  #       $3) Expected result (on stdout)
>  #       $4) Data written to file "input"
>  #       $5) Data written to stdin
> +# At least one of $4 and $5 should not be empty, but if both are empty
> +# an input file name should be included in $2 (how do we validate that?).

"Sanity checking found bugs. More sanity checking found more bugs
and now even more sanity checking finds bugs in sanity checking.

What will be next?"


Seriously. $4 and $5 _can_ be empty. Lots of commands don't use input.
For example, mdev.

"an input file name should be included in $2". What input file?
Why it should be included? The applets we test are so diverse,
they have wastly different behavior. For example, sleep
applet takes no input. At all. There will be no "input file"
in $2 for it.

And lastly, we don't need sanity checking on sanity checking.

> Tests not specifying $4 or $5 _and_ the input (file) in $2 (when $4 is
> non-empty) may not work at all.  Some still do work due to a
> "feature/bug" in the 'testing' function, which always does this:
> 
>   echo -ne "$5" | eval "$2"
> 
> regardless.  This should be probably done only if both $4 and $5 are
> empty.
> 
> If $4 is non-emty the magic word 'input' should be present in $2, but even
> if it's present, it could be placed in the wrong spot.  And if $5 is
> empty, then only:
> 
>   eval "$2"
> 
> should be executed (no echo).

What will happen if test *will* read stdin?

> Are there cases where we want both stdin and input file(s)?

Yes. grep etc.

> Are there cases where we _don't_ want them both?

Yes. There are. But "testing" doesn't need to bother!
It should just feed the stding and create "input",
and don't worry whether thing being tested uses it or not.

> If both $4 and $5 are non-empy, the magic word 'input' should be present
> in $2, double input should be assumed and the 'testing' function should
> execute:
> 
>   echo "$5" | eval "$2"
> 
> Still, not totally reliable :(

I dont understand. _What_ is not reliable?
--
vda