shell's 'errexit' option and variable definition.

Shell's option 'errexit' works unexpectedly when command returning non-zero
is called in `local` variable definition statement.

Variable assignment in bash `eval`

Here is example of variable assignment in eval:

    (
        v="a\" b";
        eval "r=\"$v\"";
        declare -p r
    )

Such assignment lead to error due to unmatched double quote. Here is corrected
version of above example

    (
        v="a\" b";
        eval "r=\"\$v\"";
        declare -p r
    )

And here is example of executing arbitrary command through such incorrect eval
construction:

    (
        v="a\"; ls; \":";
        eval "r=\"$v\"";
        declare -p r
    )

Note a colon (':') in 'v' value: bash does not even complain about incorrect
syntax - all is correct and all will work fine!

So, it's safe (perhaps) to use parameter expansion to form variable's name for
assign to, like

    (
        v="arr";
        eval "${v}[1]=123";
        declare -p arr;
        eval "r=\${${v}[1]}";
        declare -p r
    )

because here bash's own constrains on characters in variable name applied, and
bash checks this. But it's _not_ safe to expand variable up to value in the
main script - do it in eval script instead.

Note, the same also applies to indirect reference: do not use it to expand up
to value in main script in eval constructions. Here is flawed example:

    (
        j=var;
        var="a\"; ls; \":";
        i=r;
        eval "$i=\"${!j}\"";
        declare -p r;
    )

and now corrected version

    (
        j=var;
        var="a\"; ls; \":";
        i=r;
        eval "$i=\"\${$j}\"";
        declare -p r;
    )

or with indirect reference

    (
        j=var;
        var="a\"; ls; \":";
        i=r;
        eval "$i=\"\${!j}\"";
        declare -p r;
    )

[bash] Move array elements

Here is the task:
    1. move all array elements to offset (index) 'off'. I.e. such, that first
       array element will be at index 'off'.
    2. make holes (sequences of unset elements) in the array by moving
       different array parts to different offsets (indexes) off1, off2,
       off3,.. .

This can't be done by

    arr=( [i]="${arr[@]}" )

because "${[@]}" expansion works not as it should: it treats assignment prefix
specially, and in such case works the same as "${[*]}".

So, we should use some workaround, like

    #!/bin/bash

    declare -a arr=(
        1
        2
        3
        4
        5
    )
    declare -i off=15
    declare -p arr
    b=${arr[0]}
    unset arr[0]
    arr=( [off]=$b "${arr[@]}" )
    declare -p arr

This can be done simplier, but i'm not sure which version of bash this
requires

    #!/bin/bash

    declare -a arr=(
        1
        2
        3
        4
        5
    )
    declare -i off=15
    declare -p arr
    arr=( [off]=${arr[0]} "${arr[@]:1}" )
    declare -p arr

Second task can be solved like this

    #!/bin/bash

    declare -a arr=(
        1
        2
        3
        4
        5
        6
        7
        8
        9
        10
        11
        12
        13
    )
    declare -i off1=15 len1=2 i1=0
    declare -i off2=21 len2=3 i2=$((i1 + len1))
    declare -i off3=40 len3=4 i3=$((i2 + len2))
    declare -i off4=70        i4=$((i3 + len3))
    declare -p arr
    arr=( 
        [off1]=${arr[i1]}   "${arr[@]:i1 + 1: len1 - 1}"
        [off2]=${arr[i2]}   "${arr[@]:i2 + 1: len2 - 1}"
        [off3]=${arr[i3]}   "${arr[@]:i3 + 1: len3 - 1}"
        [off4]=${arr[i4]}   "${arr[@]:i4 + 1}"
    )
    declare -p arr

Note, that all existed holes in the array after such operations would be lost.

[bash] Write filenames to array.

Here is two general tasks:
    1. Assign strings (e.g. filenames) separated by '\0' from some input
       stream to corresponding array elements.
    2. Convert array into stream consisting from strings separated by '\0'.

I.e we have some bash script, which somewhere get such stream, e.g. by `find`
    
    find $root -wholename "*/$project" -prune -print0

and then we want to place this filenames into array elements. But there is a
problem: we can't use '\0' in IFS, so we can't split find's output stream
using bash word splitting expansion. And we can't use any other character to
separate filenames, because filename may contain any character.

One possible method (i don't know other, though it may be) is to transform
stream to bash code and then `eval` it.
   
But here is another problem: we can't simply escape string with double or
single quotes, because string may contain un-escaped double or single quotes
inside (find does not escape characters in filenames, and hence we assume,
that string contain un-escpaed characters, i.e written "as is"). For example

     a"'      b.txt

than after escaping with double quotes

    "a"'      b.txt"

or with single quotes

    'a"'      b.txt'

In both cases some part of string remain unescaped and not-matched quote
appears. (Write anothre example with command).


So, we can't escape string by commands like sed or awk, which perfom text
editing without string parsing. But we can use bash itself to parse and escape
string properly, and then output escaped result, like this

    eval "$(find $root -wholename "*/$project" -prune -print0 \
            | sort -z -s \
            | xargs -0 -x bash -c '
                            arr=( "$@" );
                            declare -p arr
                        ' escape_filename)"

(the last argument 'escape_filename' is used as $0. It may be any, but
required for correct work. For details see chapter 7.4.2 from 'info find')

Script for bash instance, invoked by xargs, may do some other operations with
strings, like

    j=15;
    eval "$(find $root -wholename "*/$project" -prune -print0 \
            | sort -z -s \
            | xargs -0 -x bash -c "
                            set -- \"\${@#$root/}\"
                            set -- \"\${@%$project}\"
                            arr2=( [$j]=\"\$1\" \"\${@:2}\" );
                            declare -p arr2
                        " escape_filename)"

Here we delete leading ($root) and trailing ($project) portions of filename
and then assign resulted set of strings to array starting at index $j.
Variables $root, $project and $j are substituted by main bash process before
executing pipeline.

If you use in this script array name you want assign to in main script, no
further editing of output will be needed.


Here is another example to what incorrect quoting may lead to. If we have in
input stream string like this

     a' rm -rf ~ '

then quote it with single quotes and add assignment (with sed, for example)

    var='a' rm -rf ~ ''

when it will be eval-ed it execute command

    rm -rf ~

Here is sample script

    #!/bin/bash

    str="a' ls ~ '"
    to_eval="$(echo "$str" | sed -e"s/^/var='/;s/$/'/")"
    eval "$to_eval"

[bash][part][draft]Return values from bash function

DISCLAIMER. English language used here only for compatibility (ASCII only), so any suggestions about my bad grammar (and not only it) will be greatly appreciated.

UPD. 2010.09.07

If we need to return several values from function, we can return through 
    - either setting some global variables,
    - or through pipe ("stdout").
    - or use both.
      
1. Global variables.

    If we do not want to hardcode these global variable names into script, we
    can pass their names as arguments and then use `eval` to assign values in
    child function,

        f() {
            ..
            eval "$1=\"a    b\""
        }
        f g1
        declare -p g1

    but in this case there is possible name conflicts between local function's
    variables and global ones. If local variable has the same name as global,
    it replaces it and we no longer able to set global variable in this
    child function.
    
2. Pipe.

    If we use pipe, we'll first gather all values from local child function's
    variables and send them into pipe, but then, in caller function, we'll
    parse pipe content to separate these values again and assign to proper
    variables. This is waste. Also, order, in which results will be outputed
    into pipe, should be fixed since cutting of result into separate values
    again and assigning them to proper variables into caller function assumes
    certain order.  And if this order sometimes accidently will change,
    returned values will be messed.

        f3() {
            local l1='a   b'
            local l2='   c d'
            echo "\"$l1\""
            echo "\"$l2\""
            return 0
        }
        eval "$(echo "$(f3)" | sed -e'1s/^/g1=/;2s/^/g2=/')"
        declare -p g1 g2

3. Mixed.

    But we can use both those techniques at the same time to eliminate all
    listed above problems each of them have.

    Names of global variables to be set should be passed through arguments,
    but in the pipe we'll send not raw values, but assignment statements with
    already expanded values, and in caller function we should simply execute
    them (assignment statements) in `eval`.  In such case we get:
        - no name conflicts, since we're no longer need global variables into
          child function: we need only their names to generate proper output,
          but values to them will be assigned into `eval` in caller function.
        - no parsing of pipe by caller, since child function knows names of
          variables to which caller want to assign values, and send the output
          with already prepared assignment statements (values in this
          assignment statements must be expanded in child function, since
          child's local variables will be deleted upon it returns).
        - Order of variables in child function's output may be any (except,
          very special cases, where one variable use the value of previous
          ones), since all assignments already written and no parsing
          required.

        f() {
            local l1='local   1'
            local l2='local 2'
            echo "g1=\"$l1\""
            echo "g2=\"$l2\""
            return 0
        }
        declare g1=''
        declare g2=''
        eval "$(f g1 g2)"
        declare -p g1 g2

Note1:
    - when function is called through command substitution, then it'll be
      executed in the subshell (unlike, when it's called normally, it is
      executed in the main shell), so you can _not_ set any parent shell's
      variable inside it. Thus, all parent shell's variable, which should be
      set must be written into pipe in assignment statements form.
    - you can not reinitialize array in parent shell using 'arr=( ${arr[@]} )'
      syntax, when writing to pipe, if array elements contain shell
      'metacharacters' (not IFS characters!). This is because in `eval`-ed
      script all values will be already expanded (yet before it'll be written
      into pipe) and will not contain any quotes, so they'll be broken into
      tokens (words and operators) by shell 'metacharacters' far before any
      expansions (including word splitting by 'IFS' characters) will take
      place.
    - the only way to reinitialize array in parent shell is to write either
      'arr[index]=value' for each element or `declare -p arr` into pipe.

Here is example.

# ./t.sh {{{

#!/bin/bash

f() {
    echo "SUBSH=$BASH_SUBSHELL" >/dev/tty
    arr1[3]='    g h  '
    arr2[3]='i    k  l'
    IFS=" "
    echo "arr1=( ${arr1[*]} )"
    echo "$(declare -p arr2)"
    return 0
}
declare -a arr1=(
    'a   b'
    'c      '
)
declare -a arr2=(
    '    d'
    '   e  f   '
)
eval "$(f)"
declare -p arr1 arr2

# }}}

Note2:

Even if we don't need any results from function, which write result into pipe,
following considerations should be looked at:
    - such function should always be invoked in subshell, even if we don't
      need its results. Otherwise, it'll be executed in the same shell as
      parent function and its actions with FDs may result in unexpected FD
      table state for parent.
    - FD=1 of such function should always be connected to something like
      '/dev/null'. Otherwise, it can write results somewhere you don't expect
      them for.

Example-1.

Here both functions f() and f2() return result through pipe, but function f()
does not need what f2() returns, so f2() is called (before f() moves pipe)
without subshell and without pipe connected. This results in the following
problems:
    - f2() closes 'fd_t_stdout' in the parent function's FD table, though f()
      thinks it's still open. This resluts in "9: Bad file descriptor error",
      when f() tries to move pipe and f() loses correct value for stdout.
      Hence, f() writes into pipe both output intended for 'stdout' and
      intended for 'pipe'.
    - f2() writes its output intended for 'pipe' into the _same_ pipe, as f()
      uses, because it was invoked before f() moves its own pipe.

# cat ./t.sh {{{
#!/bin/bash

declare -r -i fd_t_stdout=9
declare -r -i fd_t_pipe=7
declare -r log_file='./2.tmp'
declare -r read_file='/dev/zero'
[ -f "$log_file" ] && rm -f "$log_file"
exec 2>>"$log_file" <"$read_file"

f2() {
    echo "f2(): start: \$\$: $$, BASH=$BASHPID, SUBSH=$BASH_SUBSHELL" >>"$log_file"
    lsof -a -p $BASHPID -d'^mem,^txt,^rtd,^cwd' >>"$log_file"
    read -rsn1

    echo "f2(): move pipe" >>"$log_file"
    eval "exec $fd_t_pipe>&1 1>&$fd_t_stdout-"
    lsof -a -p $BASHPID -d'^mem,^txt,^rtd,^cwd' >>"$log_file"
    read -rsn1

    echo "f2(): write to stdout"

    echo "f2(): restore pipe" >>"$log_file"
    exec 1>&$fd_t_pipe-
    lsof -a -p $BASHPID -d'^mem,^txt,^rtd,^cwd' >>"$log_file"
    read -rsn1

    echo "f2(): write to pipe"
}
f() {
    echo "f(): start: \$\$: $$, BASH=$BASHPID, SUBSH=$BASH_SUBSHELL" >>"$log_file"
    lsof -a -p $BASHPID -d'^mem,^txt,^rtd,^cwd' >>"$log_file"
    read -rsn1

    f2
    echo "f(): move pipe" >>"$log_file"
    eval "exec $fd_t_pipe>&1 1>&$fd_t_stdout"
    lsof -a -p $BASHPID -d'^mem,^txt,^rtd,^cwd' >>"$log_file"
    read -rsn1

    echo "f(): write to stdout"

    echo "f(): restore pipe" >>"$log_file"
    eval "exec 1>&$fd_t_pipe-"
    lsof -a -p $BASHPID -d'^mem,^txt,^rtd,^cwd' >>"$log_file"
    read -rsn1

    echo "f(): write to pipe"
}
eval "exec $fd_t_stdout>&1"
v="$(f)"
echo "v: '$v'"
eval "exec $fd_t_stdout>&-"

# }}}
# ./t.sh >|./1.tmp {{{
f(): start: $$: 12681, BASH=12683, SUBSH=1
COMMAND   PID USER   FD   TYPE DEVICE SIZE    NODE NAME
t.sh    12683 root    0r   CHR    1,5          920 /dev/zero
t.sh    12683 root    1w  FIFO    0,6        31967 pipe
t.sh    12683 root    2w   REG    8,7   43 1590776 /home/sgf/new_tree/src/send_sms/2.tmp
t.sh    12683 root    9w   REG    8,7    0 1121662 /home/sgf/new_tree/src/send_sms/1.tmp
f2(): start: $$: 12681, BASH=12683, SUBSH=1
COMMAND   PID USER   FD   TYPE DEVICE SIZE    NODE NAME
t.sh    12683 root    0r   CHR    1,5          920 /dev/zero
t.sh    12683 root    1w  FIFO    0,6        31967 pipe
t.sh    12683 root    2w   REG    8,7  438 1590776 /home/sgf/new_tree/src/send_sms/2.tmp
t.sh    12683 root    9w   REG    8,7    0 1121662 /home/sgf/new_tree/src/send_sms/1.tmp
f2(): move pipe
COMMAND   PID USER   FD   TYPE DEVICE SIZE    NODE NAME
t.sh    12683 root    0r   CHR    1,5          920 /dev/zero
t.sh    12683 root    1w   REG    8,7    0 1121662 /home/sgf/new_tree/src/send_sms/1.tmp
t.sh    12683 root    2w   REG    8,7  805 1590776 /home/sgf/new_tree/src/send_sms/2.tmp
t.sh    12683 root    7w  FIFO    0,6        31967 pipe
f2(): restore pipe
COMMAND   PID USER   FD   TYPE DEVICE SIZE    NODE NAME
t.sh    12683 root    0r   CHR    1,5          920 /dev/zero
t.sh    12683 root    1w  FIFO    0,6        31967 pipe
t.sh    12683 root    2w   REG    8,7 1175 1590776 /home/sgf/new_tree/src/send_sms/2.tmp
f(): move pipe
./t.sh: line 36: 9: Bad file descriptor
COMMAND   PID USER   FD   TYPE DEVICE SIZE    NODE NAME
t.sh    12683 root    0r   CHR    1,5          920 /dev/zero
t.sh    12683 root    1w  FIFO    0,6        31967 pipe
t.sh    12683 root    2w   REG    8,7 1492 1590776 /home/sgf/new_tree/src/send_sms/2.tmp
f(): restore pipe
./t.sh: line 43: 7: Bad file descriptor
COMMAND   PID USER   FD   TYPE DEVICE SIZE    NODE NAME
t.sh    12683 root    0r   CHR    1,5          920 /dev/zero
t.sh    12683 root    1w  FIFO    0,6        31967 pipe
t.sh    12683 root    2w   REG    8,7 1812 1590776 /home/sgf/new_tree/src/send_sms/2.tmp

# }}}
# cat ./1.tmp {{{
f2(): write to stdout
v: 'f2(): write to pipe
f(): write to stdout
f(): write to pipe'

# }}}

Example-2.

If f() moves its pipe before calling f2(), but calls it also without subshell,
this will not be better:
    - f2() will write its output intended for 'pipe' into 'stdout', since
      stdout instead of pipe will be opened at FD=1, when f2() starts.
    - f2() replaces FD='fd_t_pipe' value into parent function's FD table, so
      saved f()'s pipe will be replaced with 'stdout' (since 'stdout' will be
      at FD=1 in f2()). So, f() loses correct value for its pipe and writes
      into 'stdout' both output intended for pipe and intended for 'stdout'.

# cat ./t.sh {{{
#!/bin/bash

declare -r -i fd_t_stdout=9
declare -r -i fd_t_pipe=7
declare -r log_file='./2.tmp'
declare -r read_file='/dev/zero'
[ -f "$log_file" ] && rm -f "$log_file"
exec 2>>"$log_file" <"$read_file"

f2() {
    echo "f2(): start: \$\$: $$, BASH=$BASHPID, SUBSH=$BASH_SUBSHELL" >>"$log_file"
    lsof -a -p $BASHPID -d'^mem,^txt,^rtd,^cwd' >>"$log_file"
    read -rsn1

    echo "f2(): move pipe" >>"$log_file"
    eval "exec $fd_t_pipe>&1 1>&$fd_t_stdout-"
    lsof -a -p $BASHPID -d'^mem,^txt,^rtd,^cwd' >>"$log_file"
    read -rsn1

    echo "f2(): write to stdout"

    echo "f2(): restore pipe" >>"$log_file"
    exec 1>&$fd_t_pipe-
    lsof -a -p $BASHPID -d'^mem,^txt,^rtd,^cwd' >>"$log_file"
    read -rsn1

    echo "f2(): write to pipe"
}
f() {
    echo "f(): start: \$\$: $$, BASH=$BASHPID, SUBSH=$BASH_SUBSHELL" >>"$log_file"
    lsof -a -p $BASHPID -d'^mem,^txt,^rtd,^cwd' >>"$log_file"
    read -rsn1

    echo "f(): move pipe" >>"$log_file"
    eval "exec $fd_t_pipe>&1 1>&$fd_t_stdout"
    lsof -a -p $BASHPID -d'^mem,^txt,^rtd,^cwd' >>"$log_file"
    read -rsn1

    f2
    echo "f(): write to stdout"

    echo "f(): restore pipe" >>"$log_file"
    eval "exec 1>&$fd_t_pipe-"
    lsof -a -p $BASHPID -d'^mem,^txt,^rtd,^cwd' >>"$log_file"
    read -rsn1

    echo "f(): write to pipe"
}
eval "exec $fd_t_stdout>&1"
v="$(f)"
echo "v: '$v'"
eval "exec $fd_t_stdout>&-"

# }}}
# ./t.sh >|./1.tmp {{{
f(): start: $$: 12713, BASH=12715, SUBSH=1
COMMAND   PID USER   FD   TYPE DEVICE SIZE    NODE NAME
t.sh    12715 root    0r   CHR    1,5          920 /dev/zero
t.sh    12715 root    1w  FIFO    0,6        32160 pipe
t.sh    12715 root    2w   REG    8,7   43 1590776 /home/sgf/new_tree/src/send_sms/2.tmp
t.sh    12715 root    9w   REG    8,7    0 1121662 /home/sgf/new_tree/src/send_sms/1.tmp
f(): move pipe
COMMAND   PID USER   FD   TYPE DEVICE SIZE    NODE NAME
t.sh    12715 root    0r   CHR    1,5          920 /dev/zero
t.sh    12715 root    1w   REG    8,7    0 1121662 /home/sgf/new_tree/src/send_sms/1.tmp
t.sh    12715 root    2w   REG    8,7  409 1590776 /home/sgf/new_tree/src/send_sms/2.tmp
t.sh    12715 root    7w  FIFO    0,6        32160 pipe
t.sh    12715 root    9w   REG    8,7    0 1121662 /home/sgf/new_tree/src/send_sms/1.tmp
f2(): start: $$: 12713, BASH=12715, SUBSH=1
COMMAND   PID USER   FD   TYPE DEVICE SIZE    NODE NAME
t.sh    12715 root    0r   CHR    1,5          920 /dev/zero
t.sh    12715 root    1w   REG    8,7    0 1121662 /home/sgf/new_tree/src/send_sms/1.tmp
t.sh    12715 root    2w   REG    8,7  893 1590776 /home/sgf/new_tree/src/send_sms/2.tmp
t.sh    12715 root    7w  FIFO    0,6        32160 pipe
t.sh    12715 root    9w   REG    8,7    0 1121662 /home/sgf/new_tree/src/send_sms/1.tmp
f2(): move pipe
COMMAND   PID USER   FD   TYPE DEVICE SIZE    NODE NAME
t.sh    12715 root    0r   CHR    1,5          920 /dev/zero
t.sh    12715 root    1w   REG    8,7    0 1121662 /home/sgf/new_tree/src/send_sms/1.tmp
t.sh    12715 root    2w   REG    8,7 1349 1590776 /home/sgf/new_tree/src/send_sms/2.tmp
t.sh    12715 root    7w   REG    8,7    0 1121662 /home/sgf/new_tree/src/send_sms/1.tmp
f2(): restore pipe
COMMAND   PID USER   FD   TYPE DEVICE SIZE    NODE NAME
t.sh    12715 root    0r   CHR    1,5          920 /dev/zero
t.sh    12715 root    1w   REG    8,7   22 1121662 /home/sgf/new_tree/src/send_sms/1.tmp
t.sh    12715 root    2w   REG    8,7 1752 1590776 /home/sgf/new_tree/src/send_sms/2.tmp
f(): restore pipe
./t.sh: line 43: 7: Bad file descriptor
COMMAND   PID USER   FD   TYPE DEVICE SIZE    NODE NAME
t.sh    12715 root    0r   CHR    1,5          920 /dev/zero
t.sh    12715 root    1w   REG    8,7   63 1121662 /home/sgf/new_tree/src/send_sms/1.tmp
t.sh    12715 root    2w   REG    8,7 2105 1590776 /home/sgf/new_tree/src/send_sms/2.tmp

# }}}
# cat ./1.tmp {{{
f2(): write to stdout
f2(): write to pipe
f(): write to stdout
f(): write to pipe
v: ''

# }}}

Example-3.

And here is how this can be done correctly. Shortly, we should call f2() like

    ( f2 >/dev/null )

In this case, f2() result will be discarded (but we do not need it, right?), but
f() will output its result as expected.

# cat ./t.sh {{{
#!/bin/bash

declare -r -i fd_t_stdout=9
declare -r -i fd_t_pipe=7
declare -r log_file='./2.tmp'
declare -r read_file='/dev/zero'
[ -f "$log_file" ] && rm -f "$log_file"
exec 2>>"$log_file" <"$read_file"

f2() {
    echo "f2(): start: \$\$: $$, BASH=$BASHPID, SUBSH=$BASH_SUBSHELL" >>"$log_file"
    lsof -a -p $BASHPID -d'^mem,^txt,^rtd,^cwd' >>"$log_file"
    read -rsn1

    echo "f2(): move pipe" >>"$log_file"
    eval "exec $fd_t_pipe>&1 1>&$fd_t_stdout-"
    lsof -a -p $BASHPID -d'^mem,^txt,^rtd,^cwd' >>"$log_file"
    read -rsn1

    echo "f2(): write to stdout"

    echo "f2(): restore pipe" >>"$log_file"
    exec 1>&$fd_t_pipe-
    lsof -a -p $BASHPID -d'^mem,^txt,^rtd,^cwd' >>"$log_file"
    read -rsn1

    echo "f2(): write to pipe"
}
f() {
    echo "f(): start: \$\$: $$, BASH=$BASHPID, SUBSH=$BASH_SUBSHELL" >>"$log_file"
    lsof -a -p $BASHPID -d'^mem,^txt,^rtd,^cwd' >>"$log_file"
    read -rsn1

    echo "f(): move pipe" >>"$log_file"
    eval "exec $fd_t_pipe>&1 1>&$fd_t_stdout"
    lsof -a -p $BASHPID -d'^mem,^txt,^rtd,^cwd' >>"$log_file"
    read -rsn1

    ( f2 >/dev/null )
    echo "f(): write to stdout"

    echo "f(): restore pipe" >>"$log_file"
    eval "exec 1>&$fd_t_pipe-"
    lsof -a -p $BASHPID -d'^mem,^txt,^rtd,^cwd' >>"$log_file"
    read -rsn1

    echo "f(): write to pipe"
}
eval "exec $fd_t_stdout>&1"
v="$(f)"
echo "v: '$v'"
eval "exec $fd_t_stdout>&-"

# }}}
# ./t.sh >|./1.tmp {{{
f(): start: $$: 12777, BASH=12779, SUBSH=1
COMMAND   PID USER   FD   TYPE DEVICE SIZE    NODE NAME
t.sh    12779 root    0r   CHR    1,5          920 /dev/zero
t.sh    12779 root    1w  FIFO    0,6        32627 pipe
t.sh    12779 root    2w   REG    8,7   43 1590776 /home/sgf/new_tree/src/send_sms/2.tmp
t.sh    12779 root    9w   REG    8,7    0 1121662 /home/sgf/new_tree/src/send_sms/1.tmp
f(): move pipe
COMMAND   PID USER   FD   TYPE DEVICE SIZE    NODE NAME
t.sh    12779 root    0r   CHR    1,5          920 /dev/zero
t.sh    12779 root    1w   REG    8,7    0 1121662 /home/sgf/new_tree/src/send_sms/1.tmp
t.sh    12779 root    2w   REG    8,7  409 1590776 /home/sgf/new_tree/src/send_sms/2.tmp
t.sh    12779 root    7w  FIFO    0,6        32627 pipe
t.sh    12779 root    9w   REG    8,7    0 1121662 /home/sgf/new_tree/src/send_sms/1.tmp
f2(): start: $$: 12777, BASH=12784, SUBSH=2
COMMAND   PID USER   FD   TYPE DEVICE SIZE    NODE NAME
t.sh    12784 root    0r   CHR    1,5          920 /dev/zero
t.sh    12784 root    1w   CHR    1,3          898 /dev/null
t.sh    12784 root    2w   REG    8,7  893 1590776 /home/sgf/new_tree/src/send_sms/2.tmp
t.sh    12784 root    7w  FIFO    0,6        32627 pipe
t.sh    12784 root    9w   REG    8,7    0 1121662 /home/sgf/new_tree/src/send_sms/1.tmp
t.sh    12784 root   10w   REG    8,7    0 1121662 /home/sgf/new_tree/src/send_sms/1.tmp
f2(): move pipe
COMMAND   PID USER   FD   TYPE DEVICE SIZE    NODE NAME
t.sh    12784 root    0r   CHR    1,5          920 /dev/zero
t.sh    12784 root    1w   REG    8,7    0 1121662 /home/sgf/new_tree/src/send_sms/1.tmp
t.sh    12784 root    2w   REG    8,7 1410 1590776 /home/sgf/new_tree/src/send_sms/2.tmp
t.sh    12784 root    7w   CHR    1,3          898 /dev/null
t.sh    12784 root   10w   REG    8,7    0 1121662 /home/sgf/new_tree/src/send_sms/1.tmp
f2(): restore pipe
COMMAND   PID USER   FD   TYPE DEVICE SIZE    NODE NAME
t.sh    12784 root    0r   CHR    1,5          920 /dev/zero
t.sh    12784 root    1w   CHR    1,3          898 /dev/null
t.sh    12784 root    2w   REG    8,7 1874 1590776 /home/sgf/new_tree/src/send_sms/2.tmp
t.sh    12784 root   10w   REG    8,7   22 1121662 /home/sgf/new_tree/src/send_sms/1.tmp
f(): restore pipe
COMMAND   PID USER   FD   TYPE DEVICE SIZE    NODE NAME
t.sh    12779 root    0r   CHR    1,5          920 /dev/zero
t.sh    12779 root    1w  FIFO    0,6        32627 pipe
t.sh    12779 root    2w   REG    8,7 2248 1590776 /home/sgf/new_tree/src/send_sms/2.tmp
t.sh    12779 root    9w   REG    8,7   43 1121662 /home/sgf/new_tree/src/send_sms/1.tmp

# }}}
# cat ./1.tmp {{{
f2(): write to stdout
f(): write to stdout
v: 'f(): write to pipe'

# }}}

[draft][part] Shell I/O redirection

DISCLAIMER. English language used here only for compatibility (ASCII only), so any suggestions about my bad grammar (and not only it) will be greatly appreciated.

1. Open file descriptors table maps number (file descriptor - FD) into actual
   file to which output will be written.

    +----+------------+
    | FD | File       |
    +----+------------+
    | 0  |  /dev/vc/1 |
    +----+------------+
    | 1  |  /dev/vc/1 |
    +----+------------+
    | 2  |  /dev/vc/1 |
    +----+------------+
    | ...             |

2. Redirection operators (shell) affect (change) only 'File' column of the
   table. So, if program writes to FD=5 (write(5,..); call) you can _not_
   force it to write to another FD with shell redirection - you can only
   change the content of 'File' column in the FD=5 row (i.e to what file FD=5
   is mapped) and through that change to where output arrives at the end (to
   which file).

3. '/dev/stdout' and '/dev/stderr' are _not_ an actual files, they're _links_
   to what is _now_ opened at FD=1 and FD=2 correspondingly. I.e links to
   file specified in 'File' column of FD=1 and FD=2 rows.

# ls -l /dev/stdout /dev/stderr
lrwxrwxrwx 1 root root 15 Sep 30 07:56 /dev/stderr -> /proc/self/fd/2
lrwxrwxrwx 1 root root 15 Sep 30 07:56 /dev/stdout -> /proc/self/fd/1

4. In construction like 

        f() {
            echo "abc"
            return 0
        }
        res=$(f)

   function f() result are sent through pipe to the caller. I think, this is
   not clear to say, that result are sent through "stdout", though it's
   correct, since "stdout" is _not_ an actual file - it's a link to what is
   now opened at file descriptor 1, and now there will be pipe. By default,
   when function invoked through command substitution (function will be
   executed in a subshell), pipe is opened for writing at FD=1 for child
   process (function) and for reading at FD=3 for caller process.  So, if you
   use something like

        echo "abc"

   "abc" will be written into pipe (because `echo` always writes to FD=1). But
   if you want, that the way how we send result does not affect function's
   code, we should move 'pipe' from FD=1 into some unused FD and restore
   original FD=1 content. 

        exec 7>&1 1>&2
   
   When result will be ready, we move 'pipe' back to FD=1 and `echo` result
   into it.

        exec 1>&7-
        echo "result"

   Note, that for all child subshells pipe will be opened as well.


Example. Illustrates complete implementation, with several stacked functions
returning result through pipe.


#!/bin/bash

log='./t.log'
read_f='/dev/null'
rm -f $log

f2() {
    echo "f2(): SUBSH=$BASH_SUBSHELL" >>$log
    echo "f2(): BASHPID[$BASHPID]:" >>$log
    lsof -a -p $BASHPID -d '^mem,^cwd,^rtd,^txt' >>$log

    echo "f2(): stdout-1: ghi"
    echo "f2(): stderr-1: klm" >/dev/stderr

    echo "f2(): Before moving pipe somewhere" >>$log && read -n1 < $read_f
    eval "exec $save_pipe>&1 1>&$save_stdout-"
    echo "f2(): BASHPID[$BASHPID]:" >>$log
    lsof -a -p $BASHPID -d '^mem,^cwd,^rtd,^txt' >>$log

    echo "f2(): stdout-2: ghi"
    echo "f2(): stderr-2: klm" >/dev/stderr
    echo "f2(): Before exit f2()" >>$log && read -n1 < $read_f
    return 0
}
f() {
    echo "f(): SUBSH=$BASH_SUBSHELL" >>$log
    echo "f(): \$\$[$$]:" >>$log
    lsof -a -p $$ -d '^mem,^cwd,^rtd,^txt' >>$log
    echo "f(): BASHPID[$BASHPID]:" >>$log
    lsof -a -p $BASHPID -d '^mem,^cwd,^rtd,^txt' >>$log

    echo 'f(): stdout-1: abc'
    echo 'f(): stderr-1: def' >/dev/stderr

    echo "f(): Before moving pipe somewhere" >>$log && read -n1 < $read_f
    eval "exec $save_pipe>&1 1>&$save_stdout-"
    echo "f(): BASHPID[$BASHPID]:" >>$log
    lsof -a -p $BASHPID -d '^mem,^cwd,^rtd,^txt' >>$log

    echo 'f(): stdout-2: abc'
    echo 'f(): stderr-2: def' >/dev/stderr

    echo "f(): Before calling f2()" >>$log && read -n1 < $read_f
    eval "exec $save_stdout>&1"
    v=$(f2)
    echo "-$v-"
    eval "exec $save_stdout>&-"

    echo "f(): BASHPID[$BASHPID]:" >>$log
    lsof -a -p $BASHPID -d '^mem,^cwd,^rtd,^txt' >>$log

    echo "f(): Before restoring pipe" >>$log && read -n1 < $read_f
    exec >&$save_pipe-

    echo 'f(): stdout-3: abc'
    echo 'f(): stderr-3: def' >/dev/stderr

    echo "f(): BASHPID[$BASHPID]:" >>$log
    lsof -a -p $BASHPID -d '^mem,^cwd,^rtd,^txt' >>$log
    echo "f(): Before exit f()" >>$log && read -n1 < $read_f
    return 0
}
declare -r -i save_stdout=9
declare -r -i save_pipe=7
eval "exec $save_stdout>&1"
v=$(f)
eval "exec $save_stdout>&-"
echo "in main()"
echo "\$\$[$$]:" >>$log
lsof -a -p $$ -d '^mem,^cwd,^rtd,^txt' >>$log
echo "-$v-"



Here is illustration:

                             (subshell)
  main()                   ....................
 +--------------+          .   f()            .
 | 3r  pipe     | call f() . +--------------+ .
 | 9u  "stdout" |----------->| 1w  pipe(f)  | .
 +--------------+          . | 9u  "stdout" | .
                           . +--------------+ .
                           .     |            .
                           .     | move pipe(f)
                           .     |            .
                           .     v            .           (subshell)
                           . +--------------+ .         ....................
                           . | 1w  "stdout" | .         .   f2()           .
                           . | 7w  pipe(f)  | call f2() . +--------------+ .
                           . | 9-  (closed) |------------>| 1w  pipe(f2) | .
                           . +--------------+ .         . | 7w  pipe(f)  | .
                           .                  .         . | 9u  "stdout" | .
                           .                  .         . +--------------+ .
                           .                  .         .     |            .
                           .                  .         .     | move pipe(f2)
                           .                  .         .     | over pipe(f)
                           .                  .         .     |            .
                           .                  .         .     v            .
                           .                  .         . +--------------+ .
                           . +--------------+ . return  . | 1w  "stdout" | .
                           . | 1w  "stdout" |<------------| 7w  pipe(f2) | .
                           . | 7w  pipe(f)  | .         . | 9-  (closed) | .
                           . +--------------+ .         . +--------------+ .
                           .     |            .         ....................
                           .     | restore pipe(f)
                           .     |            .
                           .     v            .
                           . +--------------+ .
 +--------------+  return  . | 1w  pipe(f)  | .
 | 1u  "stdout" |------------| 7-  (closed) | .
 +--------------+          . +--------------+ .
                           ....................


And here is log (slightly edited)

# ./t.sh >|./1.tmp

f(): SUBSH=1
f(): $$[4794]:
COMMAND  PID USER   FD   TYPE DEVICE SIZE    NODE NAME
t.sh    4794 root    0u   CHR    4,1         5466 /dev/vc/1
t.sh    4794 root    1w   REG    8,7    0 1121662 .../1.tmp
t.sh    4794 root    2u   CHR    4,1         5466 /dev/vc/1
t.sh    4794 root    3r  FIFO    0,6        14132 pipe
t.sh    4794 root    9w   REG    8,7    0 1121662 .../1.tmp
t.sh    4794 root  255r   REG    8,7 2075 1121653 .../t.sh
f(): BASHPID[4796]:
COMMAND  PID USER   FD   TYPE DEVICE SIZE    NODE NAME
t.sh    4796 root    0u   CHR    4,1         5466 /dev/vc/1
t.sh    4796 root    1w  FIFO    0,6        14132 pipe
t.sh    4796 root    2u   CHR    4,1         5466 /dev/vc/1
t.sh    4796 root    9w   REG    8,7    0 1121662 .../1.tmp
f(): Before moving pipe somewhere
f(): BASHPID[4796]:
COMMAND  PID USER   FD   TYPE DEVICE SIZE    NODE NAME
t.sh    4796 root    0u   CHR    4,1         5466 /dev/vc/1
t.sh    4796 root    1w   REG    8,7    0 1121662 .../1.tmp
t.sh    4796 root    2u   CHR    4,1         5466 /dev/vc/1
t.sh    4796 root    7w  FIFO    0,6        14132 pipe
f(): Before calling f2()
f2(): SUBSH=2
f2(): BASHPID[4803]:
COMMAND  PID USER   FD   TYPE DEVICE SIZE    NODE NAME
t.sh    4803 root    0u   CHR    4,1         5466 /dev/vc/1
t.sh    4803 root    1w  FIFO    0,6        14194 pipe
t.sh    4803 root    2u   CHR    4,1         5466 /dev/vc/1
t.sh    4803 root    7w  FIFO    0,6        14132 pipe
t.sh    4803 root    9w   REG    8,7   19 1121662 .../1.tmp
f2(): Before moving pipe somewhere
f2(): BASHPID[4803]:
COMMAND  PID USER   FD   TYPE DEVICE SIZE    NODE NAME
t.sh    4803 root    0u   CHR    4,1         5466 /dev/vc/1
t.sh    4803 root    1w   REG    8,7   19 1121662 .../1.tmp
t.sh    4803 root    2u   CHR    4,1         5466 /dev/vc/1
t.sh    4803 root    7w  FIFO    0,6        14194 pipe
f2(): Before exit f2()
f(): BASHPID[4796]:
COMMAND  PID USER   FD   TYPE DEVICE SIZE    NODE NAME
t.sh    4796 root    0u   CHR    4,1         5466 /dev/vc/1
t.sh    4796 root    1w   REG    8,7   61 1121662 .../1.tmp
t.sh    4796 root    2u   CHR    4,1         5466 /dev/vc/1
t.sh    4796 root    7w  FIFO    0,6        14132 pipe
f(): Before restoring pipe
f(): BASHPID[4796]:
COMMAND  PID USER   FD   TYPE DEVICE SIZE  NODE NAME
t.sh    4796 root    0u   CHR    4,1       5466 /dev/vc/1
t.sh    4796 root    1w  FIFO    0,6      14132 pipe
t.sh    4796 root    2u   CHR    4,1       5466 /dev/vc/1
f(): Before exit f()
$$[4794]:
COMMAND  PID USER   FD   TYPE DEVICE SIZE    NODE NAME
t.sh    4794 root    0u   CHR    4,1         5466 /dev/vc/1
t.sh    4794 root    1w   REG    8,7   71 1121662 .../1.tmp
t.sh    4794 root    2u   CHR    4,1         5466 /dev/vc/1
t.sh    4794 root  255r   REG    8,7 2075 1121653 .../t.sh

[method][draft][part][need_check] (bash) passing arguments to function

DISCLAIMER. English language used here only for compatibility (ASCII only), so any suggestions about my bad grammar (and not only it) will be greatly appreciated.

Status: method.
State: suggestion,draft,part,need_check.
Detailed description: One more method for passing arguments to bash function.

Outline: below under 'passing argument as pointer' i mean passing argument by name:

function f() {
}

declare var='abc'
f var  # not f "$var"

This may be considered as analogy for pointers (in C), i think :-)

1. Passing argument as pointer to function sometimes may require redefinition
   it as local variable in order to preserve referred data. Though, generally
   pointer used to provide to function ability to modify caller's environment,
   in bash it may be used, to skip unnecessary copying of big argument, if we
   want to redefine all positional parameters to local variables (names
   1, 2, 3,.. is bad whenever, and also positional parameters may be used for
   some tricks, so it's better, when they do not contain useful data).

        eval "
                $(declare -p $2 | sed -e's/^[^=]\+=/local -a shortopts=/' );
                $(declare -p $3 | sed -e's/^[^=]\+=/local -a longopts=/');
        "

   Above construction is, seemingly, the only way. To correctly redefine
   pointer to local variable we should honour several contraints:
        1. we should not define any local variable before all pointers will be
           expanded, because otherwise there is a chance, that defined local
           variable name is match with some pointed to variable's name and we
           lose access to it.
        2. we should preserve all special characters inside pointed data.

   To handle first constraint, we need `eval`.

   To handle second, we should properly quote expanded pointed to variable's
   value:

        ${[@]} - split elements correctly only during first expand, but in the
        second (done in `eval`) these boundaries will be lost and bash splits
        elements by 'metacharacters', not by IFS or anything else (info bash,
        ch-2).
        ${[*]} - my be used, perhaps, with IFS=',', and then.. something
        artificial to set up quoting by brace expansion. But this rarely works,
        though (i don't know working example).
        `declare` - seems, only way to obtain correct quoting without many
        evals and artificial tricks. But its output, perhaps, should be
        processed by some program to replace variable name to desired one.

[summary][draft][part] Bash script parsing

DISCLAIMER. English language used here only for compatibility (ASCII only), so any suggestions about my bad grammar (and not only it) will be greatly appreciated.

Status: summary.
State: draft,part.
Detailed description: illustration for info bash ch-2.

Bash script parsing units:

                    token = single unit
                   /     \
                 /         \
               /             \
             /                 \
          word                operator
         / | \                /     \
       /   |   \             /       \
     /     |     \          /         \
reserved  name    ..     control     redirection
word                     operator     operator


              metacharacter - separate words (or tokens?)
              /     \   
            /         \ 
          /             \
       blank           some (why not all?)
                       control operators