Comments on: RFC: Making Valgrind easier to use with multi-process programs

By: Nick

Nick — Sat, 02 May 2009 01:05:47 +0000

Thanks for the details. I’m still concerned about the complexity for such a simple case. It’d be nice to achieve “make the common cases easy and the uncommon cases possible”. Black-lists/white-lists achieve the first half of that; your suggestion achieves the last half.

I wonder if it would be possible to provide a couple of predicates for making common cases (such as those equivalent to black-listing/white-listing) easy. Eg. such that you could write this:

–trace-children-predicate=”blacklist ‘/usr/bin/python .*’”

and “blacklist” would take the ‘/usr/bin/python .*’ argument, which is a regexp, and do a regexp match against the full command.

So “blacklist” would receive the regexp as its first arg, and then the remaining args (toolname, PID, exec path, etc) as the 2nd and subsequent args. I think that could be made to work. Likewise for “whitelist”, etc.

I think that would achieve “make the common cases easy and the uncommon cases possible”.

By: John Reiser

John Reiser — Fri, 01 May 2009 18:36:46 +0000

ARRGGGHHH! The blog software stripped my meta-syntactic use of less-than and greater-than signs to specify trailing arguments to the shell and perl in those examples above: toolname, PPID, pathname_to_execve, and argv[]_to_execve

By: John Reiser

John Reiser — Fri, 01 May 2009 18:34:01 +0000

Nick, here’s how the escape character mechanism could work for your case:
valgrind –trace-children-predicate=”| perl -c \” exit ( $ARGV[5] eq \\\“/usr/bin/python\\\” ? 1 : 0 ) \” ” ./my_app
where I have used $ARGV[5] because eventually perl will be invoked with its sixth argument being the pathname to the original execve; see below.

The shell quoting is enough so that vagrind sees:
–trace-children-predicate=| perl -c ” exit ( $ARGV[5] eq \“/usr/bin/python\” ? 1 : 0 ) ”
(which has a trailing space). Then valgrind interprets the pipe symbol as indicating a literal shell command instead of the name of an executable. So valgrind invokes a shell as [each argument on a separate line]:
/bin/sh
-c
perl -c ” exit ( $ARGV[5] eq \“/usr/bin/python\” ? 1 : 0 ) ”

The shell invokes perl with arguments
/usr/bin/perl
-c
exit ( $ARGV[5] eq “/usr/bin/python” ? 1 : 0 )

which looks like the right stuff. Of course this would be simpler if arguments did not suffer re-parsing by the various interpreters.

By: Nicholas Nethercote

Nicholas Nethercote — Thu, 30 Apr 2009 22:51:09 +0000

Ka-Hing, Colin: You are right, Firefox perhaps isn’t the best example to use here, because the extra processes can be easily skipped.

But there are more “genuine” multi-process programs out there (eg. Google Chrome, which I also mentioned). I made these proposals in response to real users asking for this functionality.

By: Nicholas Nethercote

Nicholas Nethercote — Thu, 30 Apr 2009 22:45:48 +0000

John, the escape character idea sounds promising, but I can’t quite see how it would work. If I want to use this snippet of Perl code as the predicate:

exit ( $ARGV[0] eq “/usr/bin/python” ? 1 : 0 )

what would the Valgrind command line look like? I’m particularly unsure if the quoting would work out. Thanks.

By: John Reiser

John Reiser — Thu, 30 Apr 2009 14:17:37 +0000

The exit code of the predicate process could be interpreted further. In addition to 0 for “exec, and trace it”, then 1 could be “exec but do not trace” and 2 could be “do not exec” because the predicate has already done it, such as by invoking a different tool, and thus subsuming the original intent to exec.

By: John Reiser

John Reiser — Thu, 30 Apr 2009 13:46:39 +0000

For making the predicate process mechanism more lightweight, there could be an escape based on the first character of the name. For instance, “strace -o ‘| gzip -c >foo’” creates a pipeline for the output of strace, instead of just a filename. For valgrind, the escape might be just a literal shell script to be invoked as “sh -c “, and with the same arguments as for the heavyweight predicate.

By: Colin Walters

Colin Walters — Thu, 30 Apr 2009 13:46:17 +0000

Firefox is not multiprocess. The extra shell process is an *entirely pointless* waste, and there’s a trivial patch to fix it here for a while which just needs to be reviewed and applied:

https://bugzilla.mozilla.org/show_bug.cgi?id=477724

By: Nick

Nick — Thu, 30 Apr 2009 11:58:15 +0000

John’s idea subsumes black-lists, but doesn’t cover whitelists. One possibility is to have three return values, eg. 0 to trace with the given tool, 1 to trace with Nulgrind, 2 to run natively. Another possibility is to make the stdout control the tracing, eg. produce “trace”, “nulgrind”, “native”. (You could even go so far as to name another tool, and thus trace different process with different tools, although that is probably overkill.) Either way, it would subsume white-lists as well.

So the power is good. But I’m concerned about convenience, particularly for simple cases like “skip all python sub-processes”. Having to create a separate program (even a short script) is much more heavyweight than specifying something like –trace-blacklist=”/usr/bin/python *”. Or is there a way to do the equivalent entirely from the command line that I’m missing?

By: Kostya Serebryany

Kostya Serebryany — Thu, 30 Apr 2009 06:35:44 +0000

+1 for John Reiser’s suggestion.
Leave –trace-children as it is now.
Add another option for a ‘predicate process’.