CPS222 Lecture: Recursion - Last revised 1/13/2015 Materials: 1. Simple expression grammar projectable plus interpreter demo 2. Excerpt from Java grammar 3. Towers of Hanoi toy 4. Projectable of towers of hanoi program 5. Demos of towers of hanoi program - plain and graphic versions 6. Projectable of permute program 7. Demo of permute program Objectives: 1. To introduce recursion and define "recursive" 2. To give examples of uses of recursion 3. To introduce the various classes of recursive algorithms 4. To introduce reasons why it might be desirable to avoid recursion. I. Introduction - ------------ A. Today we discuss a algorithm design technique that is very powerful. The technique is known as recursion. B. Some definitions: 1. A recursive DEFINITION is one that defines some entity partially in terms of itself. 2. A recursive METHOD is one that can call itself. 3. Recursive methods often arise in programs when one is dealing with entities that are defined recursively. C. An example: 1. The factorial operation can be defined mathematically as follows: For all integers N >=0: N! = IF N <= 1 THEN 1 ELSE N*(N-1)! 2. In Java or C++, this can be coded as follows: int factorial(int n) { if (n <= 1) return 1; else return n * factorial(n-1); } 3. The book developed a diagram called a recursion trace for tracing how a recursive algorithm works. The diagram is especially simple if recursion only occurs once in the code. Let's develop the diagram. GIVE class time to work on in pairs for factorial(3) ASK volunteer to draw on the board Initial call | v ----------------- | factorial (3) | ----------------- | v ----------------- | factorial (2) | ----------------- | v ----------------- | factorial (1) | ----------------- D. Note well that a recursive definition/procedure differs from a circular definition/procedure. A recursive definition/procedure must have at least one non-recursive component; and it must be guaranteed that the definition/procedure will terminate after a finite number of recursions for any (finite) input. 1. The case(s) where no recursive calls is/are needed are called BASE CASE(S). There must be at least one base, but there can be more. Example: For factorial, the base case is n <= 1. 2. The case(s) that involve recursive calls are called GENERAL CASE(S). Every general case must have the property that each recursion moves closer to a base case. Example: For factorial, the recursive cases have n > 1, and reduce n by one on the recursion, thus moving closer to the base case. 3. The correct operation of a recursive algorithm is often demonstrated by the use of MATHEMATICAL INDUCTION. Example: Thm: The recursive definition/function for factorial will terminate after n-1 recursions for all n >= 1. Proof: By induction: a. Basis: When n=1, the algorithm terminates after 0 recursions. b. Hypothesis: Suppose that there exists a k such that for all n 1 <= n <= k the algorithm terminates in n-1 recursions. c. Induction step: When n = k+1 the algorithm terminates after n-1 = k recursions. Proof: We calculate k+1 factorial as (k+1) * k factorial. But calculating k factorial involves k-1 recursions by our hypothesis. Therefore k+1 factorial involves k recursions QED. The correctness of the final result is ensured because of conformity to the definition of factorial, which is, itself, recursive II. Why use recursion? -- --- --- --------- A. Some entities are most naturally defined using recursion: 1. Factorial 2. Fibonacci numbers: Fib(N) = IF N <= 2 THEN 1 ELSE Fib(N-1) + Fib(N-2) Class exercise: write code based on the definition int fib(int n) { if (n <= 2) return 1; else return fib(n-2) + fib(n-1); } 3. Syntax of languages - both artificial and natural a. Ex: You will see in CPS320 how recursive definitions can be used to define several classes of formal language b. Ex: Syntax for a simple expression grammar PROJECT Demo interpreter based on this grammar c. This is true - though in a more complex way - of the full grammar of a language like Java PROJECT 4. Many Artificial Intelligence applications use recursive lists 5. Trees: A tree consists of a root and zero or more subtrees, each of which is a tree. (Draw) B. Some problems are most easily solved using recursion. 1. Ex: Towers of Hanoi a. Illustrate using model. b. Observe: i. For the case N=1 the problem is trivial. ii. For N>1 the problem can be solved in terms of two subproblems of size N-1: To move N disks from peg A to peg B: (1) Move top N-1 from peg A to peg C (2) Move 1 disk from peg A to peg B (3) Move N-1 disks from peg C to peg B c. Demonstrate for case N=3 d. A portion of a program: PROJECTABLE OF PLAIN VERSION void hanoi(int n, char start, char finish, char help) /* Recursive procedure that does the work. Prints directions for * moving n disks from start to finish using help. */ { if (n <= 1) cout << "Move disk from peg " << start << " to peg " << finish << endl; else { hanoi(n-1, start, help, finish); cout << "Move disk from peg " << start << " to peg " << finish << endl; hanoi(n-1, help, finish, start); } } void main(int argc, char * argv []) { int size; cout << "How many disks? "; cin >> size; hanoi(size, 'A', 'B', 'C'); } e. DEMO program: plain version graphic version 2. In general, recursion is an appropriate way to tackle a problem if: a. It has a "size". b. It is trivial for a certain size. c. A "big" problem can always be reduced to one or more subproblems of lesser size. d. Examples: i. Find the Greatest Common Divisor of A,B (A >= B) (1) Let B be the "size" (2) The problem is trivial when B is 0. (3) For B > 0, the problem can be reduced to GCD(B, A MOD B) (Euclid's Algorithm - for proof see Dromey p 97 f) Now clearly A MOD B < B; therefore we have reduced the size of the problem Class exercise - develop code based on definition: int gcd(int a, int b) { if (b == 0) return a; else return gcd(b, a % b); } ii. Sorting a list of non-equal items into ascending order: (1) Let the number of items in the list be the size (N). (2) The problem is trivial when N=1 (3) For N>1 we can reduce the problem to 2 subproblems of smaller size, as follows: (a) Choose an arbitrary item from the list - say the first. Call it M. (b) Divide the list into two sublists - one consisting of all items smaller than M and one consisting of all items greater than M. (c) Clearly each sublist contains less than N items, since M is not a member of either. (d) Sort the original list by sorting each sublist and then gluing back together: sorted list of items smaller than M; M; sorted list of items larger than M. (4) This is the basis of a standard sorting algorithm known as quicksort, which we will look at later in the course. iii. Generating all permutations of a list of distinct items. (1) Let the number of items in the list be the size (N). (2) The problem is trivial when N=1 (3) For N > 1 we can reduce the size of the problem by 1 as follows: (a) Let each of the N items, in turn, serve as the last item (b) For each case, form all the permuations to the N-1 remaining items, then append the chosen item at the end. (4) Class exercise: develop code based on above PROJECT Code void exchange(char & c1, char & c2) /* Exchange two characters in an array */ { char temp = c1; c1 = c2; c2 = temp; } void permute(char values[], int n) /* Finds all the permutations of the first n entries of values. * When n = 1 (base case) prints out the current contents of values */ { if (n == 1) { // BASE CASE cout << values << endl; } else { for (int i = 0; i < n; i ++) { exchange(values[i], values[n-1]); permute(values, n - 1); exchange(values[i], values[n-1]); } } } int main(int argc, char * argv[]) { int N; cout << "Enter N - realize that there are N! permutations!: "; cin >> N; char values[N+1]; for (int i = 0; i < N; i ++) values[i] = 'A' + i; values[N] = '\0'; permute(values, N); } DEMO FOR N = 4 III. Classes of Recursive Algorithms: --- ------- -- --------- ---------- A. A linear recursive algorithm is one in which each non-trivial call to the recursive procedure gives rise to one new call to that procedure - i.e. the procedure calls itself only at one point: Examples: ASK factorial, gcd - An important subcategory of linear recursive is tail-recursive. In a tail-recursive algorithm, the recursive call is the very last step in the algorithm. Tail-recursive algorithms can be easily converted to non-recursive form, replacing the recursion with a loop. Example of transformation to a non-recursive version: gcd int gcd(int a, int b) { while (b != 0) { int oldA = a; a = b; b = oldA % b; } return a; } [ Note how the code becomes more complex due to the need to keep track of both "old" and "new" values of a; having multiple "versions" of a is handled automatically in the recursive version. ] B. A binary recursive algorithm is one in which each non-trivial call to the recursive procedure gives rise to two new calls to that procedure- ie the procedure calls itself at two points. Many of the most important recursive algorithms fall into this category - including those based on a "divide and conquer" approach. Ex: Fibonacci, towers of Hanoi, quicksort C. A multiple recursive algorithm is one in which each non-trivial call to the recursive procedure gives rise to a varying number of new calls to that procedure (often more than two). Frequently, such a procedure calls itself from within a loop. Ex: permute D. A mutually recursive algorithm is one which contains several recursive procedures which call themselves indirectly - e.g. A calls B, B calls C, and C calls A. Parsers for languages (artificial and natural) are often mutual-recursive. Ex: In the grammar for expressions we looked at earlier, we have a several cases of simple linear recursion, plus a simple case of mutual recursion PROJECT Expressions are defined in terms of terms Terms are defined in terms of factors One alternative for a factor is a parenthesized expression IV. Some Comments on the Efficiency of Recursion -- ---- -------- -- --- ---------- -- --------- A. Recursion is often a good strategy for finding a solution to a problem (and may be about the only way to find a solution.) B. Often, recursive solutions are more efficient than ones that might be discovered other ways. In particular, "divide and conquer" algorithms exploit this property. We will see several examples of this throughout the course. C. However, recursion can also lead to inefficient solutions. 1. Tail recursive algorithms can always be easily transformed to non-recursive ones using a simple loop, and linear recursive algorithms can frequently be transformed this way as well. When recursion can be easily replaced by a a loop, the resulting algorithm is almost always more efficient, because there is significant overhead associated with method calls. Example: Non-recursive (and preferred) calculation of factorial: int factorial(int n) { int result = 1; for (int i = n; i > 1; i --) result *= i; return result; } 2. Sometimes, recursion yields a really bad solution that can be replaced by a much more efficient one using a loop. (This is the strategy called dynamic programming, which we will discuss at the end of the course.) For example, the recursive version of Fibonacci requires fib(N) iterations to calculate fib(N), which has exponential complexity. An iterative solution that has linear complexity is as follows: int fib(int n) { if (n <= 2) return 1; int last = 1; int nextToLast = 1; int current = 1; for (int i = 3; i <= n; i ++) { current = nextToLast + last; nextToLast = last; last = current; } return current; }