Time and Space Complexity - DSA | Chapter: Understanding Logarithmic Time Complexity ($O(\log N)$)

Logarithmic time complexity, denoted as $O(\log N)$, represents one of the most efficient runtime behaviors in computer science, commonly found in divide-and-conquer algorithms like Binary Search. This chapter deconstructs how logarithmic complexity arises from iterative loops that scale exponentially (either by doubling or halving) rather than linearly. Through step-by-step mathematical proofs and intuitive tracing, we will establish a permanent mental model for identifying and calculating $O(\log N)$ runtimes.

1. The Problem Statement: Analyzing a Non-Linear Loop

In standard linear loops, the loop counter increases by a constant addition (e.g., i++ or i += 1). However, to understand logarithmic time complexity, we must analyze a loop where the counter increases multiplicatively.

Consider the following Java implementation of the loop discussed in the lecture:

public class LogarithmicExample {
    public static void main(String[] args) {
        int n = 64; // Example boundary value
        
        // Loop variable 'i' doubles in every iteration
        for (int i = 1; i <= n; i = i * 2) {
            System.out.println("Chamka"); 
        }
    }
}

The Core Question

How many times will the statement "Chamka" print as a function of $n$? Because the loop control variable i does not increment by $1$, we cannot simply state that the loop runs $n$ times. We must trace the exact progression of i to find the mathematical relationship between the number of iterations and the input size $n$.

2. Step-by-Step Trace of the Doubling Loop

Let us trace the value of i and count how many times the print statement executes.

Assume $n$ is a power of $2$ for clean mathematical tracking. Let's observe the progression:

Observation Table

From this trace, we can see a clear pattern: when the loop variable $i$ is represented as $2^k$, the print statement has executed exactly $k + 1$ times.

3. Mathematical Derivation of $O(\log N)$

To find the exact time complexity, we need to solve for the total number of iterations ($k + 1$) in terms of $n$.

The loop terminates when the value of $i$ exceeds $n$. At the very last valid iteration, we can approximate that $i$ has reached $n$:

To solve for $k$, we apply the logarithm with base 2 ($\log_2$) to both sides of the equation:

Using the logarithmic power rule ($\log(x^y) = y \cdot \log(x)$):

Since $\log_2(2) = 1$, we get:

Calculating Total Operations

The total number of times "Chamka" is printed is $k + 1$. Substituting the value of $k$:

Asymptotic Analysis ($O$-Notation)

When expressing this in Big-O notation, we drop the lower-order constant terms ($+1$) because they become insignificant as $n$ grows infinitely large:

Note on Logarithmic Bases: In computer science, if the base of the logarithm is not explicitly written, it is generally assumed to be $2$ (i.e., $\log n$ means $\log_2 n$).

4. The Core Mental Model: Doubling vs. Halving

The creator highlights a crucial rule of thumb that allows you to identify logarithmic behavior instantly without writing out formal proofs every time.

Pattern A: The Doubling Pattern (Bottom-Up)

When a process starts at $1$ and repeatedly doubles until it reaches $n$, the number of steps required is always $\log_2 n$.

Pattern B: The Halving Pattern (Top-Down)

Conversely, if you start at a large number $n$ and repeatedly divide it by $2$ until you reach $1$, the number of steps required is also exactly $\log_2 n$.

This halving pattern is the foundational mechanism behind Binary Search and Merge Sort (during the division phase).

5. Generalizing to Other Bases (e.g., Tripling)

What happens if the loop variable multiplies by $3$ instead of $2$ in each iteration?

for (int i = 1; i <= n; i = i * 3) {
    // Constant time operations
}

Following the same mathematical derivation:

1. The values of $i$ will progress as: $3^0, 3^1, 3^2, 3^3, \dots, 3^k$.
2. At the boundary: $3^k = n$.
3. Solving for $k$ by taking $\log_3$ on both sides:

Why Base Doesn't Matter in Big-O

According to the logarithmic base change formula:

This means:

Since $\frac{1}{\log_2 3}$ is a constant factor ($\approx 0.63$), it is ignored in asymptotic complexity analysis. Therefore:

Regardless of whether you multiply/divide by $2$, $3$, or $100$ in each step, the time complexity remains $O(\log n)$.

6. Tabular Synthesis of Loop Patterns

To help solidify your ability to analyze loops, the table below contrasts linear loop patterns with logarithmic loop patterns:

Summary and Key Takeaways

• Logarithmic Definition: $O(\log n)$ time complexity occurs when the size of the problem (or the loop range) is divided or multiplied by a constant factor at each step.
• The Doubling Rule: Starting at $1$ and doubling up to $n$ ($1 \to 2 \to 4 \to 8 \dots \to n$) takes exactly $\log_2 n$ steps.
• The Halving Rule: Starting at $n$ and halving down to $1$ ($n \to n/2 \to n/4 \dots \to 1$) takes exactly $\log_2 n$ steps.
• Mathematical Proof: By setting the progression $2^k = n$ and applying logarithms to both sides, we isolate the number of steps $k = \log_2 n$.
• Base Independence: In Big-O notation, the base of the logarithm (whether $2$, $3$, or $10$) is omitted because base-change factors are constants, meaning all logarithmic complexities are written simply as $O(\log n)$.
• No Best/Worst Case Divergence: In simple loops without conditional break statements, the average, best, and worst-case complexities are identical ($\Theta(\log n) = \Omega(\log n) = O(\log n)$).