Time and Space Complexity: Analyzing Multi-Variable and Fractional Loop Bounds

This chapter focuses on advanced techniques for analyzing the time complexity of nested loops. We will explore how to calculate complexity when loop variables increment exponentially (with varying bases), how to handle multi-variable nested loops with independent bounds, and how to analyze complex dependent loops where step sizes change dynamically, leading to mathematical series like the Harmonic Series.

1. Exponential and Multiplicative Loop Increments (Base $k$)

When a loop control variable does not increment linearly (e.g., i++) but instead multiplies by a constant factor at each step, its growth is exponential. Consequently, the number of steps required to reach the termination condition is logarithmic.

The General Rule for Multiplicative Loops

If a loop variable starts at $1$ and is multiplied by a constant factor $k$ at each iteration until it reaches $n$:

• For $k = 2$ (i = i * 2): The loop runs $\approx \log_2 n$ times.
• For $k = 3$ (i = i * 3): The loop runs $\approx \log_3 n$ times.
• For $k = 4$ (i = i * 4): The loop runs $\approx \log_4 n$ times.

Mathematical Derivation

Let $i$ be the loop iterator starting at $1$. Let $x$ be the number of steps taken. At each step, $i$ is multiplied by $3$:

Step 0: i = 1
Step 1: i = 1 * 3 = 3^1
Step 2: i = 3 * 3 = 3^2
Step 3: i = 9 * 3 = 3^3
...
Step x: i = 3^x

The loop terminates when $i$ exceeds or equals $n$:

Taking the logarithm with base 3 on both sides:

Thus, the total number of iterations (steps) is $O(\log_3 n)$.

2. Multi-Variable Independent Nested Loops

When analyzing nested loops, a common mistake is assuming that every nested loop structure automatically results in a quadratic $O(n^2)$ or cubic $O(n^3)$ time complexity. To analyze them accurately, we must check if the inner loops depend on the outer loop variables.

Code Scenario 1: Independent Nested Loops

for (int i = n / 2; i <= n; i++) {
    for (int j = 1; j <= n / 2; j++) {
        for (int k = 1; k <= n; k++) {
            cout << "chamka";
        }
    }
}

Step-by-Step Analysis

1. Outer Loop ($i$):

   • Starts at $n/2$ and runs up to $n$ with a linear increment of i++.
   • Number of iterations: $n - \frac{n}{2} = \frac{n}{2}$ times.

2. Middle Loop ($j$):

   • Starts at $1$ and runs up to $n/2$ with a linear increment of j++.
   • Number of iterations: $\frac{n}{2}$ times.
   • This loop is completely independent of $i$.

3. Inner Loop ($k$):

   • Starts at $1$ and runs up to $n$ with a linear increment of k++.
   • Number of iterations: $n$ times.
   • This loop is completely independent of both $i$ and $j$.

Total Complexity Calculation

Since the loops are independent, we multiply their individual iteration counts:

In Big-O notation, we drop constant coefficients:

3. Nested Loops with Logarithmic Bounds & The Multiplication Rule

Let's analyze a nested loop structure where the inner loops grow exponentially (multiplicatively) rather than linearly.

Code Scenario 2: Logarithmic Nested Loops

for (int i = n / 2; i <= n; i++) {
    for (int j = 1; j <= n; j = 2 * j) {
        for (int k = 1; k <= n; k = 2 * k) {
            cout << "chamka";
        }
    }
}

Step-by-Step Analysis

1. Outer Loop ($i$):

   • Runs from $n/2$ to $n$ linearly.
   • Iterations: $\frac{n}{2}$ times.

2. Middle Loop ($j$):

   • Starts at $1$, doubles at each step (j = 2 * j), and stops when $j > n$.
   • Iterations: $\log_2 n$ times.

3. Inner Loop ($k$):

   • Starts at $1$, doubles at each step (k = 2 * k), and stops when $k > n$.
   • Iterations: $\log_2 n$ times.

Total Complexity Calculation

Multiplying the independent work of each loop:

Applying Big-O notation:

Note on Dropping Terms: We cannot ignore or drop the $\log n$ terms here. While it is true that $\log n$ grows much slower than $n$, they are multiplied together ($n \times \log n \times \log n$).

• If the relationship was additive ($n + \log n$), we would drop the lower-order term $\log n$ to get $O(n)$.
• Because they are multiplicative, every term contributes to the growth rate and must be preserved in the final complexity.

4. Dependent Loops with Variable Step Jumps (The Harmonic Series)

Now we will analyze a highly classic and non-trivial pattern where the inner loop's step size is directly determined by the outer loop's iterator value.

Code Scenario 3: Variable Step Jumps

for (int i = 1; i <= n; i++) {
    for (int j = 1; j <= n; j = j + i) {
        cout << "chamka";
    }
}

Step-by-Step Trace

Let's analyze how many times the inner loop runs for different values of $i$:

• When $i = 1$:

  • The inner loop update is j = j + 1.
  • The loop runs: $1, 2, 3, 4, \dots, n$.
  • Total iterations: $n$ times (or $\frac{n}{1}$).

• When $i = 2$:

  • The inner loop update is j = j + 2.
  • The loop runs: $1, 3, 5, 7, \dots, n$.
  • Total iterations: $\frac{n}{2}$ times.

• When $i = 3$:

  • The inner loop update is j = j + 3.
  • The loop runs: $1, 4, 7, 10, \dots, n$.
  • Total iterations: $\frac{n}{3}$ times.

• When $i = k$:

  • The inner loop update is j = j + k.
  • Total iterations: $\frac{n}{k}$ times.

• When $i = n$:

  • The inner loop update is j = j + n.
  • The loop runs exactly $1$ time (since the next step immediately exceeds $n$).
  • Total iterations: $\frac{n}{n} = 1$ time.

Mathematical Summation

To find the total number of operations across all iterations of the outer loop, we sum the iterations of the inner loop for each step of $i$:

We can factor out $n$ from the entire expression:

The Harmonic Series

The summation inside the parentheses is the famous mathematical Harmonic Series:

Asymptotically, the sum of the first $n$ terms of a Harmonic Series is bounded by the natural logarithm of $n$:

Substituting this back into our total operations formula:

Thus, even though the inner loop step size increases rapidly as $i$ grows, the overall time complexity of this nested structure is $O(n \log n)$.

5. Synthesis and Comparison of Loop Patterns

Summary and Key Takeaways

• Logarithmic Base Rule: When a loop variable is multiplied by a constant factor $k$ at each step, the time complexity is $O(\log_k n)$. The base of the logarithm is determined by the multiplier.
• Independence Rule: If the loop boundaries and step updates of inner loops do not contain any variables from outer loops, you can calculate their complexities individually and multiply them.
• Multiplication vs. Addition in Big-O: Lower-order terms can only be dropped when they are added to higher-order terms (e.g., $n + \log n \to O(n)$). When terms are multiplied (e.g., $n \times \log n$), they must all be kept in the final complexity expression.
• Harmonic Series Complexity: Loops where the step size of the inner loop increases by the outer loop index $i$ (i.e., j = j + i) yield the summation $n \sum \frac{1}{i}$. This evaluates directly to $O(n \log n)$ due to the properties of the Harmonic Series.
• Trace-and-Sum Strategy: For complex dependent loops, the safest analytical approach is to manually trace the first few steps ($i=1, 2, 3$), write down the work done as a mathematical series, and find its closed-form sum.