PHP exec/system interpreting escapes like form feed (bash/dash differences in echo)

Abstract

A pig of a bug to unravel: why executing echo with the system shell seemingly breaks in PHP

When running shell or exec in PHP, the code you pass is run not in bash, but in /bin/sh, which on many systems is instead a symlink to dash, a lighter, faster, and crucially POSIX-compliant shell. For various historical reasons, there are (among other differences) different versions of echo. I have been unraveling this problem for about an hour now, and I hope this is the final solution. It has been posted on the PHP docs website as well. The information is out there, but not specific instructions for safely catching all the edge cases from PHP.

I won’t repeat the background and journey of discovery train of thought here; you can happily learn from the long mailing list discussions and various bug reports that all appeared soon after Ubuntu switched its default shell to dash. The upshot is that certain character sequences like '\f' do not work as you expect with echo. Firstly you should switch to using printf, which always has the same behaviour and avoids tricky bugs with the code working in one shell but not another. Secondly, printf unfortunately has a harder mode of escaping things which needs to be dealt with.

There is some more detail in the answer I posted to the PHP docs site. In brief:

<?php
$input = 'string to be passed *exactly* to the command';
//Escape only what is needed to get by PHP's parser; we want
//the string data PHP is holding in its buffer to be passed
//exactly to stdin buffer of the command.
$cmd = str_replace(array('\\', '%'), array('\\\\', '%%'), $input);
$cmd = escapeshellarg($cmd);

$output = shell_exec("printf $cmd | /path/to/command");
?>

I would not be posting this if I were not extremely confident it handles every control character and possible escape sequence. Here is the torture test:

<?php

$test = 'stuff bash interprets, space: ! # & ; ` | * ? ~ < '.
        '> ^ ( ) [ ] { } $ \ \x0A \xFF. \' " %'.PHP_EOL.
        'stuff bash interprets, no space: !#&;`|*?~<>^()[]{'.
        '}$\\x0A\xFF.\'\"%'.PHP_EOL.
        'stuff bash interprets, with leading backslash: \! '.
        '\# \& \; \` \| \* \? \~ \< \> \^ \( \) \[ \] \{ \}'.
        ' \$ \\\ \\\x0A \\\xFF. \\\' \" \%'.PHP_EOL.
        'printf codes: % \ (used to form %.0#-*+d, or \\ \a'.
        ' \b \f \n \r \t \v \" \? \062 \0062 \x032 \u0032 a'.
        'nd \U00000032)';

echo "These are the strings we are testing with:".
      PHP_EOL.$test.PHP_EOL;
$cmd = $test;
$cmd = str_replace(array('\\',   '%'), 
                   array('\\\\', '%%'), $test);
$cmd = escapeshellarg($cmd);

echo PHP_EOL."This is the output using the escaping ".
             "mechanism given:".PHP_EOL;
echo `printf $cmd | cat`.PHP_EOL;

echo PHP_EOL."They should match exactly".PHP_EOL;