DISCLAIMER. English language used here only for compatibility (ASCII only), so any suggestions about my bad grammar (and not only it) will be greatly appreciated.

четверг, 30 сентября 2010 г.

[draft][part] Shell I/O redirection

DISCLAIMER. English language used here only for compatibility (ASCII only), so any suggestions about my bad grammar (and not only it) will be greatly appreciated.

1. Open file descriptors table maps number (file descriptor - FD) into actual
   file to which output will be written.

    +----+------------+
    | FD | File       |
    +----+------------+
    | 0  |  /dev/vc/1 |
    +----+------------+
    | 1  |  /dev/vc/1 |
    +----+------------+
    | 2  |  /dev/vc/1 |
    +----+------------+
    | ...             |

2. Redirection operators (shell) affect (change) only 'File' column of the
   table. So, if program writes to FD=5 (write(5,..); call) you can _not_
   force it to write to another FD with shell redirection - you can only
   change the content of 'File' column in the FD=5 row (i.e to what file FD=5
   is mapped) and through that change to where output arrives at the end (to
   which file).

3. '/dev/stdout' and '/dev/stderr' are _not_ an actual files, they're _links_
   to what is _now_ opened at FD=1 and FD=2 correspondingly. I.e links to
   file specified in 'File' column of FD=1 and FD=2 rows.

# ls -l /dev/stdout /dev/stderr
lrwxrwxrwx 1 root root 15 Sep 30 07:56 /dev/stderr -> /proc/self/fd/2
lrwxrwxrwx 1 root root 15 Sep 30 07:56 /dev/stdout -> /proc/self/fd/1

4. In construction like 

        f() {
            echo "abc"
            return 0
        }
        res=$(f)

   function f() result are sent through pipe to the caller. I think, this is
   not clear to say, that result are sent through "stdout", though it's
   correct, since "stdout" is _not_ an actual file - it's a link to what is
   now opened at file descriptor 1, and now there will be pipe. By default,
   when function invoked through command substitution (function will be
   executed in a subshell), pipe is opened for writing at FD=1 for child
   process (function) and for reading at FD=3 for caller process.  So, if you
   use something like

        echo "abc"

   "abc" will be written into pipe (because `echo` always writes to FD=1). But
   if you want, that the way how we send result does not affect function's
   code, we should move 'pipe' from FD=1 into some unused FD and restore
   original FD=1 content. 

        exec 7>&1 1>&2
   
   When result will be ready, we move 'pipe' back to FD=1 and `echo` result
   into it.

        exec 1>&7-
        echo "result"

   Note, that for all child subshells pipe will be opened as well.


Example. Illustrates complete implementation, with several stacked functions
returning result through pipe.


#!/bin/bash

log='./t.log'
read_f='/dev/null'
rm -f $log

f2() {
    echo "f2(): SUBSH=$BASH_SUBSHELL" >>$log
    echo "f2(): BASHPID[$BASHPID]:" >>$log
    lsof -a -p $BASHPID -d '^mem,^cwd,^rtd,^txt' >>$log

    echo "f2(): stdout-1: ghi"
    echo "f2(): stderr-1: klm" >/dev/stderr

    echo "f2(): Before moving pipe somewhere" >>$log && read -n1 < $read_f
    eval "exec $save_pipe>&1 1>&$save_stdout-"
    echo "f2(): BASHPID[$BASHPID]:" >>$log
    lsof -a -p $BASHPID -d '^mem,^cwd,^rtd,^txt' >>$log

    echo "f2(): stdout-2: ghi"
    echo "f2(): stderr-2: klm" >/dev/stderr
    echo "f2(): Before exit f2()" >>$log && read -n1 < $read_f
    return 0
}
f() {
    echo "f(): SUBSH=$BASH_SUBSHELL" >>$log
    echo "f(): \$\$[$$]:" >>$log
    lsof -a -p $$ -d '^mem,^cwd,^rtd,^txt' >>$log
    echo "f(): BASHPID[$BASHPID]:" >>$log
    lsof -a -p $BASHPID -d '^mem,^cwd,^rtd,^txt' >>$log

    echo 'f(): stdout-1: abc'
    echo 'f(): stderr-1: def' >/dev/stderr

    echo "f(): Before moving pipe somewhere" >>$log && read -n1 < $read_f
    eval "exec $save_pipe>&1 1>&$save_stdout-"
    echo "f(): BASHPID[$BASHPID]:" >>$log
    lsof -a -p $BASHPID -d '^mem,^cwd,^rtd,^txt' >>$log

    echo 'f(): stdout-2: abc'
    echo 'f(): stderr-2: def' >/dev/stderr

    echo "f(): Before calling f2()" >>$log && read -n1 < $read_f
    eval "exec $save_stdout>&1"
    v=$(f2)
    echo "-$v-"
    eval "exec $save_stdout>&-"

    echo "f(): BASHPID[$BASHPID]:" >>$log
    lsof -a -p $BASHPID -d '^mem,^cwd,^rtd,^txt' >>$log

    echo "f(): Before restoring pipe" >>$log && read -n1 < $read_f
    exec >&$save_pipe-

    echo 'f(): stdout-3: abc'
    echo 'f(): stderr-3: def' >/dev/stderr

    echo "f(): BASHPID[$BASHPID]:" >>$log
    lsof -a -p $BASHPID -d '^mem,^cwd,^rtd,^txt' >>$log
    echo "f(): Before exit f()" >>$log && read -n1 < $read_f
    return 0
}
declare -r -i save_stdout=9
declare -r -i save_pipe=7
eval "exec $save_stdout>&1"
v=$(f)
eval "exec $save_stdout>&-"
echo "in main()"
echo "\$\$[$$]:" >>$log
lsof -a -p $$ -d '^mem,^cwd,^rtd,^txt' >>$log
echo "-$v-"



Here is illustration:

                             (subshell)
  main()                   ....................
 +--------------+          .   f()            .
 | 3r  pipe     | call f() . +--------------+ .
 | 9u  "stdout" |----------->| 1w  pipe(f)  | .
 +--------------+          . | 9u  "stdout" | .
                           . +--------------+ .
                           .     |            .
                           .     | move pipe(f)
                           .     |            .
                           .     v            .           (subshell)
                           . +--------------+ .         ....................
                           . | 1w  "stdout" | .         .   f2()           .
                           . | 7w  pipe(f)  | call f2() . +--------------+ .
                           . | 9-  (closed) |------------>| 1w  pipe(f2) | .
                           . +--------------+ .         . | 7w  pipe(f)  | .
                           .                  .         . | 9u  "stdout" | .
                           .                  .         . +--------------+ .
                           .                  .         .     |            .
                           .                  .         .     | move pipe(f2)
                           .                  .         .     | over pipe(f)
                           .                  .         .     |            .
                           .                  .         .     v            .
                           .                  .         . +--------------+ .
                           . +--------------+ . return  . | 1w  "stdout" | .
                           . | 1w  "stdout" |<------------| 7w  pipe(f2) | .
                           . | 7w  pipe(f)  | .         . | 9-  (closed) | .
                           . +--------------+ .         . +--------------+ .
                           .     |            .         ....................
                           .     | restore pipe(f)
                           .     |            .
                           .     v            .
                           . +--------------+ .
 +--------------+  return  . | 1w  pipe(f)  | .
 | 1u  "stdout" |------------| 7-  (closed) | .
 +--------------+          . +--------------+ .
                           ....................


And here is log (slightly edited)

# ./t.sh >|./1.tmp

f(): SUBSH=1
f(): $$[4794]:
COMMAND  PID USER   FD   TYPE DEVICE SIZE    NODE NAME
t.sh    4794 root    0u   CHR    4,1         5466 /dev/vc/1
t.sh    4794 root    1w   REG    8,7    0 1121662 .../1.tmp
t.sh    4794 root    2u   CHR    4,1         5466 /dev/vc/1
t.sh    4794 root    3r  FIFO    0,6        14132 pipe
t.sh    4794 root    9w   REG    8,7    0 1121662 .../1.tmp
t.sh    4794 root  255r   REG    8,7 2075 1121653 .../t.sh
f(): BASHPID[4796]:
COMMAND  PID USER   FD   TYPE DEVICE SIZE    NODE NAME
t.sh    4796 root    0u   CHR    4,1         5466 /dev/vc/1
t.sh    4796 root    1w  FIFO    0,6        14132 pipe
t.sh    4796 root    2u   CHR    4,1         5466 /dev/vc/1
t.sh    4796 root    9w   REG    8,7    0 1121662 .../1.tmp
f(): Before moving pipe somewhere
f(): BASHPID[4796]:
COMMAND  PID USER   FD   TYPE DEVICE SIZE    NODE NAME
t.sh    4796 root    0u   CHR    4,1         5466 /dev/vc/1
t.sh    4796 root    1w   REG    8,7    0 1121662 .../1.tmp
t.sh    4796 root    2u   CHR    4,1         5466 /dev/vc/1
t.sh    4796 root    7w  FIFO    0,6        14132 pipe
f(): Before calling f2()
f2(): SUBSH=2
f2(): BASHPID[4803]:
COMMAND  PID USER   FD   TYPE DEVICE SIZE    NODE NAME
t.sh    4803 root    0u   CHR    4,1         5466 /dev/vc/1
t.sh    4803 root    1w  FIFO    0,6        14194 pipe
t.sh    4803 root    2u   CHR    4,1         5466 /dev/vc/1
t.sh    4803 root    7w  FIFO    0,6        14132 pipe
t.sh    4803 root    9w   REG    8,7   19 1121662 .../1.tmp
f2(): Before moving pipe somewhere
f2(): BASHPID[4803]:
COMMAND  PID USER   FD   TYPE DEVICE SIZE    NODE NAME
t.sh    4803 root    0u   CHR    4,1         5466 /dev/vc/1
t.sh    4803 root    1w   REG    8,7   19 1121662 .../1.tmp
t.sh    4803 root    2u   CHR    4,1         5466 /dev/vc/1
t.sh    4803 root    7w  FIFO    0,6        14194 pipe
f2(): Before exit f2()
f(): BASHPID[4796]:
COMMAND  PID USER   FD   TYPE DEVICE SIZE    NODE NAME
t.sh    4796 root    0u   CHR    4,1         5466 /dev/vc/1
t.sh    4796 root    1w   REG    8,7   61 1121662 .../1.tmp
t.sh    4796 root    2u   CHR    4,1         5466 /dev/vc/1
t.sh    4796 root    7w  FIFO    0,6        14132 pipe
f(): Before restoring pipe
f(): BASHPID[4796]:
COMMAND  PID USER   FD   TYPE DEVICE SIZE  NODE NAME
t.sh    4796 root    0u   CHR    4,1       5466 /dev/vc/1
t.sh    4796 root    1w  FIFO    0,6      14132 pipe
t.sh    4796 root    2u   CHR    4,1       5466 /dev/vc/1
f(): Before exit f()
$$[4794]:
COMMAND  PID USER   FD   TYPE DEVICE SIZE    NODE NAME
t.sh    4794 root    0u   CHR    4,1         5466 /dev/vc/1
t.sh    4794 root    1w   REG    8,7   71 1121662 .../1.tmp
t.sh    4794 root    2u   CHR    4,1         5466 /dev/vc/1
t.sh    4794 root  255r   REG    8,7 2075 1121653 .../t.sh