Unix I/O

Created

Intro

Output can go to the terminal to a file.

File Handle

In C, FILE* data type represents a "file handle".

  • It is a source of input and/or destination for output
  • Standard file handles include stdin, stdout, stderr.
  • These can open named file/devices using fopen

Many I/O functions in C will then take a FILE* as a parameter.

File Descriptors

File descriptors refer to files, devices (e.g., terminals), other communication channels (e.g., network connections).

  • Just an integer
  • A C file handle (FILE*) is just a "wrapper" for a file descriptor.

Files and Filesystems

Files

The concept of a file:

  • It's a sequence of bytes.
  • For all files, bytes can be read and written sequentially. For some file, random access is possible.

A "file" has two but distinct meanings:

  • A sequence of bytes stored on a medium such as a disk or SSD
  • Any object or device that can be treated as a file (for reading and/or writing bytes)
    • The first meaning is a special case of the second meaning.

Unix Filesystem

A unix filesystem is a hierarchical namespace for files.

A path names the location of a file or directory in the namespace by describing how to find it, starting from the root directory (/), navigating through a sequence of zero or more intermediate directories.

System Calls, Unix I/O

System Calls

A system call is a mechanism allowing a process to request a service from the operating system.

  • These calls are typically very low-level. Most programming languages provide a run-time library with higher-level functions.
  • E.g., open, read, write are system calls, but fopen, fread, and fwrite are run-time library functions.

To the program, the system calls are just function calls (they are typically implemented using software interrupts).

Signals are Unix mechanism for asynchronous notification.

  • They are similar to hardware interrupts, but are delivered to processes. The program can register a signal handler function to receieve them.
  • Example signals: SIGSEGV (segmentation violation), SIGINT (interruption), SIGALRM (software timer).
  • System calls can be interrupted if a signal is received, which poses issues.

Opening Files

The open system call:

#include <sys/types.h>
#include <sys/stat.h>
#include <fcntl.h>

int open (const char *pathname, int flags, mode_t mode);

This call is used for opening a named file (a file that has a path in the filesystem). It returns a file descriptor (or -1 to indicate error).

  • flags: one of O_RDONLY, O_WRONLY, or O_RDWR, bitwise-or’ed with (optionally) O_CREAT, O_TRUNC, and/or O_APPEND
  • access permission bits (only significant when flags contains O_CREAT, can omit otherwise)

Closing files

The close system call

#include <unistd.h>
int close(int fd);

Closes file named by specified file descriptor (fd).

Reading Data

THe read system call:

#include <unistd.h>
ssize_t read(int fd, void *buf, size_t n);

Read n bytes from specified file descriptor, placing data read in buf. Returns number of bytes read, or -1 on error.

A short read is when fewer than n bytes might be read because

  • end of file was reached
  • some data is not available yet (e.g., reading from a network connection)
  • line buffering by terminal.

This is why you must check return value. Example on how to deal with a short reads.

// Try to read n bytes from fd.
// Returns number of bytes read, or -1 on error.
// Returns fewer than n bytes only if EOF is reached.
ssize_t read_fully(int fd, void *buf, size_t n) {

  char *p = buf;
  while (p < (char *) buf + n) {

    size_t remaining = n - (p - (char *) buf);
    ssize_t rc = read(fd, p, remaining);
    if (rc == 0) {

      break;      // reached end of file
    } else if (rc > 0) {

      p += rc;    // read data successfully
    } else if (errno != EINTR) {

      return -1;  // an error occurred
    }

}

  return (ssize_t) (p - (char *) buf);
}
  • very complicated, which is why higher-level functions are always preferred.

Writing Data

The write system call:

#include <unistd.h>
ssize_t write(int fd, const void *buf, size_t n);

Write n bytes from buf to specified file descriptor. Returns number of bytes written, or -1 on error.

  • Similar to read, it can also return a short write (fewer than n bytes written)

Buffered I/O

Example 1:

int main(int argc, char **argv) {
  int fd = open(argv[1], O_RDONLY);
  char *buf = malloc(10000000);
  for (int i = 0; i < 10000000; i++) {

    if (read(fd, buf + i, 1) != 1) {
      fprintf(stderr, "failed read?\n");
      return 1;
    }
}

close(fd);

return 0; 
}

This program opens up a file where the filename is specified as a command line argument and attempts to read 10 million bytes from that file using the read system call for each singular byte and puts them into buf. This is extremely slow (5.334s).

Example 2:

int main(int argc, char **argv) {
	int fd = open(argv[1], O_RDONLY);
	char *buf = malloc(10000000);
	if (read(fd, buf, 10000000) != 10000000) {
		fprintf(stderr, "failed read?\n");
		return 1; 
		}
	close(fd);
	return 0;
}

In this example, we only make one read call, which is a lot faster (0.010s).

Why is it that there's such a drastic difference?

This is due to the system call overhead.

System calls are not like ordinary function calls. They require a call into the operating system kernel (typically via a software interrupt). There is a lot of work to transition from executing a process in user mode to kernel mode.

  • This transition requires saving and restoring registers, switching processor privilege levels, checking system call arguments, carrying out the system call (IO, data transfer to/from program buffer).

To speed things up, we have a method called buffering, which reduces the number of read and write system calls.

  • writing: store data in a buffer in memory, flush (write to actual file) when buffer has become full or before closing the file
  • reading: read large chunks of data into buffer infrequently, retrieve data from the buffer when requested.

FILE* objects (C file handles) have an internal buffer, so, e..g.,

  • fgetc doesn't call read every time
  • fputc doesn't call write every time.

Example 3:

int main(int argc, char **argv) {
	FILE *in = fopen(argv[1], "rb");
	char *buf = malloc(10000000);
	for (int i = 0; i < 10000000; i++) {
	    int c = fgetc(in);
	    if (c < 0) {
			fprintf(stderr, "fgetc failed?\n");
			return 1; 
		}
		buf[i] = (char) c;
	}
	fclose(in);
	return 0; 
}
  • FILE* has buffer! It will only use the underlying read system call only when buffer is empty
  • The speed is much better (0.042s)

Linked from Computer System Fundamentals