Help with pipes, buffering and pseudoterminals

[Cameron Simpson]

On 06Apr2015 21:13, Daniel Ellis <ellisd23 at gmail.com> wrote:
>Wow, thank you for the long and detailed reply.
>
>As for the question ptys and C, no, I have no experience in that area.

No worries. The use is the same in Python.

BTW, you may want a fixed width font; there are some ASCII diagrams in this 
message.

>I've read a bit on the master/slave stuff, but I'm still finding it
>confusing.  The manpage for pty states that it creates a "pair" of virtual
>devices, one master and one slave.  It doesn't seem like these are simple
>file descriptors for stdin, stdout, and stderr, given that there is only
>one for the master and one for the slave, so I'm really not understanding
>what it means for these devices to be created.

The following explaination is long; I do get to master/slave at the bottom. But 
I'm covering stdin/stdout/stderr too.

Think about a serial terminal. On one end there is a computer, with wires to 
the terminal. Since the computer runs UNIX, access to the terminal is done with 
read and write calls. A read returns characters typed at the terminal by a 
person and a write sends characters to the terminal which are displayed to the 
person.

         <--eyes-----
  person             terminal <==cable==> computer
         --fingers-->

If someone is logged in on a physical terminal, on the computer's end there 
will be a program acting on what they type. The UNIX command prompt, the shell, 
is what one would normally get after logging in; for moderns users this is 
often bash.

Terminology Digression:

  When you open() a file (or device) in UNIX you get a file descriptor handed 
  to you; this is a single integer which is essentially an index into the 
  process' array of open files. Index 0 is stdin, 1 stdout, 2 stderr, and other 
  open files get further indices.

  It is important to realise an open() creates, internally, a "file handle", 
  the data structure that tracks use of the file from that specific open() 
  call. It has information like the I/O mode (read/write/append) and the file 
  position (where data lands when you write or where data comes from when you 
  read).

  The file descriptor, the index, points into an array of _references_ to file 
  handles. You can dup() a file descriptor to get another file descriptor i.e.  
  another index; it will refer to the _same_ file handle internally.

  process: fd 0 -> tty handle -> tty device
           fd 1 ---^ ^
           fd 2 -----+

Returning...

The operating system arranges the commection of the shell to the terminal. Your 
usual program has by default a stdin, stdout and stderr. These are _all_ the 
same file handle, duplicated to each of the three file descriptors 0, 1 and 2 
respectively. On the operating system side, the OS has performed _one_ open() 
call on the terminal device and handed the caller a single file descriptor. The 
caller then calls dup() (or modernly, dup2()) to present the open terminal as 
stdin, stdout and stderr.

That file handle is open for read and write, as you might imagine.

If you want to see this, type:

  lsof -p $$

at your shell prompt. Observe fd 0, 1, 2.

Why the duplication? This lets you do redirection! You can attach something 
else to a program's stdout, for example:

  ls > files.txt

without disturbing stdin or stderr. And so forth.

A pty behaves the same way, except that there is no physical device involved.  
There is just an abstraction which behaves like a physical device:

    You see, wire telegraph is a kind of a very, very long cat. You
    pull his tail in New York and his head is meowing in Los Angeles.
    Do you understand this?  And radio operates exactly the same way:
    you send signals here, they receive them there. The only difference
    is that there is no cat.
    - Albert Einstein, when asked to describe radio

The main reason for ptys is that there is a class of programs which present a 
terminal interface to users (xterm, sshd, telnetd, etc).

So, something like xterm _draws_ a GUI representation of a terminal for you to 
look at, and collects keystrokes (as GUI events), converts them into bytes and 
writes them to the pty. For example, it might see "Shift-A-Down" and send code 
65 (capital "A"). Similarly it reads from the pty and draws on the screen.

The shell you use in an xterm is on the other side of the pty. The code 65 the 
xterm sent above is seen read as code 65 by the shell and treated as a typed 
"A".

In this scenario, effectively the xterm is a server to the shell: it is 
responsible for accepting things from the shell and displaying them, and 
providing typed GUI events to the shell as input bytes. It also sets the 
initial terminal modes and attaches the shell to the pty.

Correspondingly, when xterm opened a pty for its work the two file descriptors 
it gets are "master" and "slave" file descriptors.

The master is kept by xterm and used to read shell output and write to the 
shell's input.

The slave is used to set up the shell. The basic scheme is that the xterm 
forks. The child xterm instance closes all file descriptors _except_ the slave 
descriptor, and then dup()s that descriptor to fd 0, 1 and 2, providing stdin, 
stdout and stderr. And the pty is made the child's controlling terminal.

When the (parent) xterm writes to the master descriptor those data appear for 
read on the slave file descriptor. Conversely, when the shell writes to its 
output (the slave file descriptor) it appears for read on the master descriptor 
(for the xterm to read and display).

So the slave descriptor _is_ a perfectly ordinary file descriptor.

Feel free to ask further questions. I'm sure my rambling must occasion some.

Cheers,
Cameron Simpson <cs at zip.com.au>

From a programmer's point of view, the user is a peripheral that types when you 
issue a read request.  - Peter Williams