010203040506070809101112131415
ALL MODULESmodule_08.py

File I/O & File Systems

~45 minTheoryLive CodingDemoChallengeQuiz
theory.md

5. Operating System File Descriptors, Storage & I/O Subsystems

5.1OS File Descriptors & Kernel Handles

When your program wants to read or write a file stored on a hard drive or SSD, Python cannot access the physical disk directly. Instead, Python asks your computer's Operating System (OS) for permission. The OS opens the file and hands Python an integer reference code called a File Descriptor (or file handle).

ANALOGY TIME
Think of a file descriptor like a coat check claim ticket at a theater cloakroom. You don't walk into the coat closet yourself; you hand your claim ticket number to the attendant (the Operating System), and they fetch or update your coat (the file) for you!

If a program opens thousands of files without closing them, it consumes all available file descriptors, causing a system crash known as Resource Exhaustion (File Descriptor Leak)!

5.2The Context Manager Protocol (with statement)

To prevent resource leaks, Python provides the Context Manager (with statement). A context manager guarantees that a file is automatically and safely closed the moment your code exits the with block—even if an unexpected crash or error occurs inside!

Under the hood, any object used in a with statement must implement two special methods:

  • __enter__(): Opens the resource and returns the file object.
  • __exit__(): Cleans up and closes the file handle safely.

5.3File Modes & Line Streaming (Preventing Out-of-Memory Crashes)

When opening files with open(), Python supports different modes: 'r' (read text), 'w' (write/overwrite text), 'a' (append to end), and 'b' (binary mode for images/audio).

  • The Out-of-Memory (OOM) Danger: Calling file.read() loads the ENTIRE file into RAM all at once. If you try to open a 10 Gigabyte log file on a laptop with 8 Gigabytes of RAM, your computer will freeze and crash!
  • Line Streaming: Instead of reading the whole file at once, Python files are iterable. Using a for line in file: loop reads the file one line at a time from disk, using virtually zero RAM!
  • Analogy: Like drinking water from a straw one sip at a time versus trying to swallow an entire bucket of water in one giant gulp!

5.4Cross-Platform Paths (pathlib.Path) & Atomic Writing

Different operating systems write file directory paths differently: Windows uses backslashes (C: older ile.txt), while Linux and Mac use forward slashes (/folder/file.txt).

  • `pathlib.Path`: Python's modern built-in module for working with file paths cleanly across Windows, Mac, and Linux without writing custom slash code.
  • Atomic Writing: Writing data to a temporary file first and swapping it into place in one instant step so that a power failure never leaves you with a half-written corrupt file.

5.5Code Demo 1: Safe File I/O with Context Managers

Here is the recommended, bulletproof way to read and write files in Python:

PYTHON

5.6Code Demo 2: Streaming Line Filter (OOM Prevention)

Filter huge log files while keeping memory consumption tiny:

PYTHON
GOTCHA 1

Opening files using f = open(...) without a with context manager, leading to leaked file descriptors if an exception is raised before f.close().

GOTCHA 2

Opening a file in write mode 'w' assuming it will append. 'w' instantly truncates (erases) all existing file contents! Use 'a' to append.

GOTCHA 3

Forgetting to specify explicit encoding='utf-8'. Platform defaults (e.g. Windows CP1252) can cause UnicodeDecodeError when processing UTF-8 text.

GOTCHA 4

Reading huge log files into memory using f.read() or f.readlines(), which can exhaust RAM on large production files. Iterate over f line-by-line instead.

WHERE THIS SHOWS UP IN REAL CODE

Applications continuously interact with file systems—reading configuration files (.env, .json), streaming web server access logs, generating CSV export reports, and persisting data buffers.

TRY THIS YOURSELF

Modify Demo 2 to filter and print lines that contain 'WARN' in addition to 'ERROR'.

live_coding — Log File Summarizer

Notice how pathlib and open work seamlessly together.

Python not loaded yet.
demo — Demonstrate file creation, context managers, line streaming, and pathlib utilities.

Demonstrate file creation, context managers, line streaming, and pathlib utilities.

challenge_1

Let's practice writing to and reading from files safely!

Steps to Complete:

  1. Open data.txt in write mode and write 3 lines: `Line 1

Line 2 Line 3`.

  1. Open data.txt in read mode and count the number of lines.
  2. Print Total Lines: 3.

Expected Output:

TEXT
Python not loaded yet.
challenge_2

Let's build an environment configuration file parser!

Steps to Complete:

  1. Write `PORT=5432

HOST=localhost to env.txt`.

  1. Read env.txt line by line, splitting by = into a dictionary.
  2. Print Config Host: localhost.

Expected Output:

TEXT
Python not loaded yet.
challenge_3

Let's build an audit log filter that counts failed HTTP requests!

Steps to Complete:

  1. Write 4 lines to access.log: `200 OK

404 NOT_FOUND 200 OK 500 SERVER_ERROR`.

  1. Read access.log and count lines that do not start with 200.
  2. Print Failed Requests: 2.

Expected Output:

TEXT
Python not loaded yet.
quiz — 6 questions
30s question timers are paused. Start individually or all together.
# question 1

Why is it best practice to open files using the `with open(...)` statement?

# question 2

What happens when you open an existing file in write mode `'w'`?

# question 3

Why should line-by-line iteration (`for line in f:`) be used when processing multi-gigabyte files?

# question 4

Which module provides object-oriented, cross-platform file path manipulation in Python?

# question 5

What is the purpose of specifying `encoding='utf-8'` when opening text files?

# question 6

Which file open mode allows you to append data to an existing file without deleting prior text?

discussion

# comments · files

  • @priya · 2h ago

    The typewriter hero is such a nice touch.

    • @alex · 1h ago

      Totally agree! Made it super engaging.

  • @sam · 1d ago

    Finally a course where I can run code without leaving the page.

# further_reading