--- comments: true --- # 2.2 Iteration and recursion In algorithms, the repeated execution of a task is quite common and is closely related to the analysis of complexity. Therefore, before delving into the concepts of time complexity and space complexity, let's first explore how to implement repetitive tasks in programming. This involves understanding two fundamental programming control structures: iteration and recursion. ## 2.2.1 Iteration Iteration is a control structure for repeatedly performing a task. In iteration, a program repeats a block of code as long as a certain condition is met until this condition is no longer satisfied. ### 1. For loops The `for` loop is one of the most common forms of iteration, and **it's particularly suitable when the number of iterations is known in advance**. The following function uses a `for` loop to perform a summation of $1 + 2 + \dots + n$, with the sum being stored in the variable `res`. It's important to note that in Python, `range(a, b)` creates an interval that is inclusive of `a` but exclusive of `b`, meaning it iterates over the range from $a$ up to $b−1$. === "Python" ```python title="iteration.py" def for_loop(n: int) -> int: """for loop""" res = 0 # Loop sum 1, 2, ..., n-1, n for i in range(1, n + 1): res += i return res ``` === "C++" ```cpp title="iteration.cpp" [class]{}-[func]{forLoop} ``` === "Java" ```java title="iteration.java" /* for loop */ int forLoop(int n) { int res = 0; // Loop sum 1, 2, ..., n-1, n for (int i = 1; i <= n; i++) { res += i; } return res; } ``` === "C#" ```csharp title="iteration.cs" [class]{iteration}-[func]{ForLoop} ``` === "Go" ```go title="iteration.go" [class]{}-[func]{forLoop} ``` === "Swift" ```swift title="iteration.swift" [class]{}-[func]{forLoop} ``` === "JS" ```javascript title="iteration.js" [class]{}-[func]{forLoop} ``` === "TS" ```typescript title="iteration.ts" [class]{}-[func]{forLoop} ``` === "Dart" ```dart title="iteration.dart" [class]{}-[func]{forLoop} ``` === "Rust" ```rust title="iteration.rs" [class]{}-[func]{for_loop} ``` === "C" ```c title="iteration.c" [class]{}-[func]{forLoop} ``` === "Kotlin" ```kotlin title="iteration.kt" [class]{}-[func]{forLoop} ``` === "Ruby" ```ruby title="iteration.rb" [class]{}-[func]{for_loop} ``` === "Zig" ```zig title="iteration.zig" [class]{}-[func]{forLoop} ``` The flowchart below represents this sum function. ![Flowchart of the sum function](iteration_and_recursion.assets/iteration.png){ class="animation-figure" }
Figure 2-1 Flowchart of the sum function
The number of operations in this summation function is proportional to the size of the input data $n$, or in other words, it has a linear relationship. **This "linear relationship" is what time complexity describes**. This topic will be discussed in more detail in the next section. ### 2. While loops Similar to `for` loops, `while` loops are another approach for implementing iteration. In a `while` loop, the program checks a condition at the beginning of each iteration; if the condition is true, the execution continues, otherwise, the loop ends. Below we use a `while` loop to implement the sum $1 + 2 + \dots + n$. === "Python" ```python title="iteration.py" def while_loop(n: int) -> int: """while loop""" res = 0 i = 1 # Initialize condition variable # Loop sum 1, 2, ..., n-1, n while i <= n: res += i i += 1 # Update condition variable return res ``` === "C++" ```cpp title="iteration.cpp" [class]{}-[func]{whileLoop} ``` === "Java" ```java title="iteration.java" /* while loop */ int whileLoop(int n) { int res = 0; int i = 1; // Initialize condition variable // Loop sum 1, 2, ..., n-1, n while (i <= n) { res += i; i++; // Update condition variable } return res; } ``` === "C#" ```csharp title="iteration.cs" [class]{iteration}-[func]{WhileLoop} ``` === "Go" ```go title="iteration.go" [class]{}-[func]{whileLoop} ``` === "Swift" ```swift title="iteration.swift" [class]{}-[func]{whileLoop} ``` === "JS" ```javascript title="iteration.js" [class]{}-[func]{whileLoop} ``` === "TS" ```typescript title="iteration.ts" [class]{}-[func]{whileLoop} ``` === "Dart" ```dart title="iteration.dart" [class]{}-[func]{whileLoop} ``` === "Rust" ```rust title="iteration.rs" [class]{}-[func]{while_loop} ``` === "C" ```c title="iteration.c" [class]{}-[func]{whileLoop} ``` === "Kotlin" ```kotlin title="iteration.kt" [class]{}-[func]{whileLoop} ``` === "Ruby" ```ruby title="iteration.rb" [class]{}-[func]{while_loop} ``` === "Zig" ```zig title="iteration.zig" [class]{}-[func]{whileLoop} ``` **`while` loops provide more flexibility than `for` loops**, especially since they allow for custom initialization and modification of the condition variable at each step. For example, in the following code, the condition variable $i$ is updated twice each round, which would be inconvenient to implement with a `for` loop. === "Python" ```python title="iteration.py" def while_loop_ii(n: int) -> int: """while loop (two updates)""" res = 0 i = 1 # Initialize condition variable # Loop sum 1, 4, 10, ... while i <= n: res += i # Update condition variable i += 1 i *= 2 return res ``` === "C++" ```cpp title="iteration.cpp" [class]{}-[func]{whileLoopII} ``` === "Java" ```java title="iteration.java" /* while loop (two updates) */ int whileLoopII(int n) { int res = 0; int i = 1; // Initialize condition variable // Loop sum 1, 4, 10, ... while (i <= n) { res += i; // Update condition variable i++; i *= 2; } return res; } ``` === "C#" ```csharp title="iteration.cs" [class]{iteration}-[func]{WhileLoopII} ``` === "Go" ```go title="iteration.go" [class]{}-[func]{whileLoopII} ``` === "Swift" ```swift title="iteration.swift" [class]{}-[func]{whileLoopII} ``` === "JS" ```javascript title="iteration.js" [class]{}-[func]{whileLoopII} ``` === "TS" ```typescript title="iteration.ts" [class]{}-[func]{whileLoopII} ``` === "Dart" ```dart title="iteration.dart" [class]{}-[func]{whileLoopII} ``` === "Rust" ```rust title="iteration.rs" [class]{}-[func]{while_loop_ii} ``` === "C" ```c title="iteration.c" [class]{}-[func]{whileLoopII} ``` === "Kotlin" ```kotlin title="iteration.kt" [class]{}-[func]{whileLoopII} ``` === "Ruby" ```ruby title="iteration.rb" [class]{}-[func]{while_loop_ii} ``` === "Zig" ```zig title="iteration.zig" [class]{}-[func]{whileLoopII} ``` Overall, **`for` loops are more concise, while `while` loops are more flexible**. Both can implement iterative structures. Which one to use should be determined based on the specific requirements of the problem. ### 3. Nested loops We can nest one loop structure within another. Below is an example using `for` loops: === "Python" ```python title="iteration.py" def nested_for_loop(n: int) -> str: """Double for loop""" res = "" # Loop i = 1, 2, ..., n-1, n for i in range(1, n + 1): # Loop j = 1, 2, ..., n-1, n for j in range(1, n + 1): res += f"({i}, {j}), " return res ``` === "C++" ```cpp title="iteration.cpp" [class]{}-[func]{nestedForLoop} ``` === "Java" ```java title="iteration.java" /* Double for loop */ String nestedForLoop(int n) { StringBuilder res = new StringBuilder(); // Loop i = 1, 2, ..., n-1, n for (int i = 1; i <= n; i++) { // Loop j = 1, 2, ..., n-1, n for (int j = 1; j <= n; j++) { res.append("(" + i + ", " + j + "), "); } } return res.toString(); } ``` === "C#" ```csharp title="iteration.cs" [class]{iteration}-[func]{NestedForLoop} ``` === "Go" ```go title="iteration.go" [class]{}-[func]{nestedForLoop} ``` === "Swift" ```swift title="iteration.swift" [class]{}-[func]{nestedForLoop} ``` === "JS" ```javascript title="iteration.js" [class]{}-[func]{nestedForLoop} ``` === "TS" ```typescript title="iteration.ts" [class]{}-[func]{nestedForLoop} ``` === "Dart" ```dart title="iteration.dart" [class]{}-[func]{nestedForLoop} ``` === "Rust" ```rust title="iteration.rs" [class]{}-[func]{nested_for_loop} ``` === "C" ```c title="iteration.c" [class]{}-[func]{nestedForLoop} ``` === "Kotlin" ```kotlin title="iteration.kt" [class]{}-[func]{nestedForLoop} ``` === "Ruby" ```ruby title="iteration.rb" [class]{}-[func]{nested_for_loop} ``` === "Zig" ```zig title="iteration.zig" [class]{}-[func]{nestedForLoop} ``` The flowchart below represents this nested loop. ![Flowchart of the nested loop](iteration_and_recursion.assets/nested_iteration.png){ class="animation-figure" }Figure 2-2 Flowchart of the nested loop
In such cases, the number of operations of the function is proportional to $n^2$, meaning the algorithm's runtime and the size of the input data $n$ has a 'quadratic relationship.' We can further increase the complexity by adding more nested loops, each level of nesting effectively "increasing the dimension," which raises the time complexity to "cubic," "quartic," and so on. ## 2.2.2 Recursion Recursion is an algorithmic strategy where a function solves a problem by calling itself. It primarily involves two phases: 1. **Calling**: This is where the program repeatedly calls itself, often with progressively smaller or simpler arguments, moving towards the "termination condition." 2. **Returning**: Upon triggering the "termination condition," the program begins to return from the deepest recursive function, aggregating the results of each layer. From an implementation perspective, recursive code mainly includes three elements. 1. **Termination Condition**: Determines when to switch from "calling" to "returning." 2. **Recursive Call**: Corresponds to "calling," where the function calls itself, usually with smaller or more simplified parameters. 3. **Return Result**: Corresponds to "returning," where the result of the current recursion level is returned to the previous layer. Observe the following code, where simply calling the function `recur(n)` can compute the sum of $1 + 2 + \dots + n$: === "Python" ```python title="recursion.py" def recur(n: int) -> int: """Recursion""" # Termination condition if n == 1: return 1 # Recursive: recursive call res = recur(n - 1) # Return: return result return n + res ``` === "C++" ```cpp title="recursion.cpp" [class]{}-[func]{recur} ``` === "Java" ```java title="recursion.java" /* Recursion */ int recur(int n) { // Termination condition if (n == 1) return 1; // Recursive: recursive call int res = recur(n - 1); // Return: return result return n + res; } ``` === "C#" ```csharp title="recursion.cs" [class]{recursion}-[func]{Recur} ``` === "Go" ```go title="recursion.go" [class]{}-[func]{recur} ``` === "Swift" ```swift title="recursion.swift" [class]{}-[func]{recur} ``` === "JS" ```javascript title="recursion.js" [class]{}-[func]{recur} ``` === "TS" ```typescript title="recursion.ts" [class]{}-[func]{recur} ``` === "Dart" ```dart title="recursion.dart" [class]{}-[func]{recur} ``` === "Rust" ```rust title="recursion.rs" [class]{}-[func]{recur} ``` === "C" ```c title="recursion.c" [class]{}-[func]{recur} ``` === "Kotlin" ```kotlin title="recursion.kt" [class]{}-[func]{recur} ``` === "Ruby" ```ruby title="recursion.rb" [class]{}-[func]{recur} ``` === "Zig" ```zig title="recursion.zig" [class]{}-[func]{recur} ``` Figure 2-3 shows the recursive process of this function. ![Recursive process of the sum function](iteration_and_recursion.assets/recursion_sum.png){ class="animation-figure" }Figure 2-3 Recursive process of the sum function
Although iteration and recursion can achieve the same results from a computational standpoint, **they represent two entirely different paradigms of thinking and problem-solving**. - **Iteration**: Solves problems "from the bottom up." It starts with the most basic steps, and then repeatedly adds or accumulates these steps until the task is complete. - **Recursion**: Solves problems "from the top down." It breaks down the original problem into smaller sub-problems, each of which has the same form as the original problem. These sub-problems are then further decomposed into even smaller sub-problems, stopping at the base case whose solution is known. Let's take the earlier example of the summation function, defined as $f(n) = 1 + 2 + \dots + n$. - **Iteration**: In this approach, we simulate the summation process within a loop. Starting from $1$ and traversing to $n$, we perform the summation operation in each iteration to eventually compute $f(n)$. - **Recursion**: Here, the problem is broken down into a sub-problem: $f(n) = n + f(n-1)$. This decomposition continues recursively until reaching the base case, $f(1) = 1$, at which point the recursion terminates. ### 1. Call stack Every time a recursive function calls itself, the system allocates memory for the newly initiated function to store local variables, the return address, and other relevant information. This leads to two primary outcomes. - The function's context data is stored in a memory area called "stack frame space" and is only released after the function returns. Therefore, **recursion generally consumes more memory space than iteration**. - Recursive calls introduce additional overhead. **Hence, recursion is usually less time-efficient than loops.** As shown in Figure 2-4, there are $n$ unreturned recursive functions before triggering the termination condition, indicating a **recursion depth of $n$**. ![Recursion call depth](iteration_and_recursion.assets/recursion_sum_depth.png){ class="animation-figure" }Figure 2-4 Recursion call depth
In practice, the depth of recursion allowed by programming languages is usually limited, and excessively deep recursion can lead to stack overflow errors. ### 2. Tail recursion Interestingly, **if a function performs its recursive call as the very last step before returning,** it can be optimized by the compiler or interpreter to be as space-efficient as iteration. This scenario is known as tail recursion. - **Regular recursion**: In standard recursion, when the function returns to the previous level, it continues to execute more code, requiring the system to save the context of the previous call. - **Tail recursion**: Here, the recursive call is the final operation before the function returns. This means that upon returning to the previous level, no further actions are needed, so the system does not need to save the context of the previous level. For example, in calculating $1 + 2 + \dots + n$, we can make the result variable `res` a parameter of the function, thereby achieving tail recursion: === "Python" ```python title="recursion.py" def tail_recur(n, res): """Tail recursion""" # Termination condition if n == 0: return res # Tail recursive call return tail_recur(n - 1, res + n) ``` === "C++" ```cpp title="recursion.cpp" [class]{}-[func]{tailRecur} ``` === "Java" ```java title="recursion.java" /* Tail recursion */ int tailRecur(int n, int res) { // Termination condition if (n == 0) return res; // Tail recursive call return tailRecur(n - 1, res + n); } ``` === "C#" ```csharp title="recursion.cs" [class]{recursion}-[func]{TailRecur} ``` === "Go" ```go title="recursion.go" [class]{}-[func]{tailRecur} ``` === "Swift" ```swift title="recursion.swift" [class]{}-[func]{tailRecur} ``` === "JS" ```javascript title="recursion.js" [class]{}-[func]{tailRecur} ``` === "TS" ```typescript title="recursion.ts" [class]{}-[func]{tailRecur} ``` === "Dart" ```dart title="recursion.dart" [class]{}-[func]{tailRecur} ``` === "Rust" ```rust title="recursion.rs" [class]{}-[func]{tail_recur} ``` === "C" ```c title="recursion.c" [class]{}-[func]{tailRecur} ``` === "Kotlin" ```kotlin title="recursion.kt" [class]{}-[func]{tailRecur} ``` === "Ruby" ```ruby title="recursion.rb" [class]{}-[func]{tail_recur} ``` === "Zig" ```zig title="recursion.zig" [class]{}-[func]{tailRecur} ``` The execution process of tail recursion is shown in Figure 2-5. Comparing regular recursion and tail recursion, the point of the summation operation is different. - **Regular recursion**: The summation operation occurs during the "returning" phase, requiring another summation after each layer returns. - **Tail recursion**: The summation operation occurs during the "calling" phase, and the "returning" phase only involves returning through each layer. ![Tail recursion process](iteration_and_recursion.assets/tail_recursion_sum.png){ class="animation-figure" }Figure 2-5 Tail recursion process
!!! tip Note that many compilers or interpreters do not support tail recursion optimization. For example, Python does not support tail recursion optimization by default, so even if the function is in the form of tail recursion, it may still encounter stack overflow issues. ### 3. Recursion tree When dealing with algorithms related to "divide and conquer", recursion often offers a more intuitive approach and more readable code than iteration. Take the "Fibonacci sequence" as an example. !!! question Given a Fibonacci sequence $0, 1, 1, 2, 3, 5, 8, 13, \dots$, find the $n$th number in the sequence. Let the $n$th number of the Fibonacci sequence be $f(n)$, it's easy to deduce two conclusions: - The first two numbers of the sequence are $f(1) = 0$ and $f(2) = 1$. - Each number in the sequence is the sum of the two preceding ones, that is, $f(n) = f(n - 1) + f(n - 2)$. Using the recursive relation, and considering the first two numbers as termination conditions, we can write the recursive code. Calling `fib(n)` will yield the $n$th number of the Fibonacci sequence: === "Python" ```python title="recursion.py" def fib(n: int) -> int: """Fibonacci sequence: Recursion""" # Termination condition f(1) = 0, f(2) = 1 if n == 1 or n == 2: return n - 1 # Recursive call f(n) = f(n-1) + f(n-2) res = fib(n - 1) + fib(n - 2) # Return result f(n) return res ``` === "C++" ```cpp title="recursion.cpp" [class]{}-[func]{fib} ``` === "Java" ```java title="recursion.java" /* Fibonacci sequence: Recursion */ int fib(int n) { // Termination condition f(1) = 0, f(2) = 1 if (n == 1 || n == 2) return n - 1; // Recursive call f(n) = f(n-1) + f(n-2) int res = fib(n - 1) + fib(n - 2); // Return result f(n) return res; } ``` === "C#" ```csharp title="recursion.cs" [class]{recursion}-[func]{Fib} ``` === "Go" ```go title="recursion.go" [class]{}-[func]{fib} ``` === "Swift" ```swift title="recursion.swift" [class]{}-[func]{fib} ``` === "JS" ```javascript title="recursion.js" [class]{}-[func]{fib} ``` === "TS" ```typescript title="recursion.ts" [class]{}-[func]{fib} ``` === "Dart" ```dart title="recursion.dart" [class]{}-[func]{fib} ``` === "Rust" ```rust title="recursion.rs" [class]{}-[func]{fib} ``` === "C" ```c title="recursion.c" [class]{}-[func]{fib} ``` === "Kotlin" ```kotlin title="recursion.kt" [class]{}-[func]{fib} ``` === "Ruby" ```ruby title="recursion.rb" [class]{}-[func]{fib} ``` === "Zig" ```zig title="recursion.zig" [class]{}-[func]{fib} ``` Observing the above code, we see that it recursively calls two functions within itself, **meaning that one call generates two branching calls**. As illustrated below, this continuous recursive calling eventually creates a recursion tree with a depth of $n$. ![Fibonacci sequence recursion tree](iteration_and_recursion.assets/recursion_tree.png){ class="animation-figure" }Figure 2-6 Fibonacci sequence recursion tree
Fundamentally, recursion embodies the paradigm of "breaking down a problem into smaller sub-problems." This divide-and-conquer strategy is crucial. - From an algorithmic perspective, many important strategies like searching, sorting, backtracking, divide-and-conquer, and dynamic programming directly or indirectly use this way of thinking. - From a data structure perspective, recursion is naturally suited for dealing with linked lists, trees, and graphs, as they are well suited for analysis using the divide-and-conquer approach. ## 2.2.3 Comparison Summarizing the above content, the following table shows the differences between iteration and recursion in terms of implementation, performance, and applicability.Table: Comparison of iteration and recursion characteristics