Overview
Administration
Calendar
Lecture Notes
Assignments
Links
Change Log
Projects

The Theory of Computing (TOC) course is a 3 credit introduction to formal languages and automata, computability theory, and complexity theory with the goal of developing understanding of the power and limits of different computational models. Topics covered include: regular languages and finite automata; context-free grammars and pushdown automata; Turing machines; undecidable languages; the classes P and NP; NP completeness.

Course Topics:

• Regular languages and finite automata
• Context-free grammars and pushdown automata
• Turing machines; decidable and undecidable languages
• The classes P and NP; NP completeness

Course Rationale: Instructor's View

While philosophy has no answers to the deep questions of life, Computer Science has some very rigorous answers to the deep questions of computing, and this course will introduce you to the heart of such answers, in a way that WILL directly affect your careers as future computer scientists and engineers (rather than just programmers).

As an example, when I was an undergrad here at ND (back when dinosaurs roamed the earth) I spent one summer working on an interpreter for FORTRAN, the programming language we taught to students at the time. This interpreter gave far more diagnostic information to the student than the regular compiler. My job for one summer was to handle EQUIVALENCE statements: statements that said that two or more variables were to map to the same location. I spent the bulk of the summer trying to find an efficient solution for the general case, only to give up. 10 years later when the theory of NP-Completeness came into vogue, I would have known that what I was trying to do was equivalent to the marriage problem when there are three or more genders - for which we now believe there is no efficient algorithm. If I had taken 34151 at the time I would have known that the 2-gender version of EQUIVALENCE is solvable simply (and is the most common case), and just flagged more complicated cases as "too hard."

For you CS students, TOC will not only get you to these questions of computational hardness, but also give you a solid introduction to the grammar, computational power, and limits of different classes of programming languages.

To the CPEG students, I love building, not using, computers, and TOC gets to the heart of what computers are and do. In a very, very real sense, we will discuss the architecture of three classes of computer architectures: finite-state machines, push-down automata, and Turing machines. This includes what they are good for (their inherent computational properties), how to understand their operation, and equally importantly the structure of programming paradigms that can utilize such machines fully. Closely coupled with these architecture classes will be a discussion of what kinds of "languages" are "accepted" by each class. A language in this discussion is a set of all strings that obey some set of grammar rules. A machine designed to accept a language is one that when given strings of characters, can distinguish between those that obey the rules of grammar for those languages, and those that do not.

Our first topic, Finite-state machines (or "finite automata" FA) in particular are at the heart of virtually all digital logic, and TOC will give you deep insight into designing them. This will be excellent preparation for real-world state machine and CPU controller design. The regular expressions you use when you grep a text file for string matches are exactly the class of languages for which FAs are a good and complete target.

Our second topic, push-down machines (PDA) add the common data structure "stacks" to an FA, giving us direct insight into the operation of such things as the Java Virtual Machine (JVM). The class of languages that match PDAs are called "Context-Free" and are often used to describe many common programming languages.

Our third topic, Turing Machines (TM), introduce a memory that is expressed as an "infinite" magnetic tape that can be read and written by a FA. Theory will show that anything that is "computable" is computable by an algorithm written for such machines, and thus any computer algorithm (expressed in any programming language) can be converted to run on a TM, and that in the limit there is nothing that makes any one current (or future) language like C, C++, Java, Python, …. inherently more powerful than any other.

Our final topic, Decidability and Complexity theory, deals with the properties of any algorithm that might be developed to address classes of problems, and is mind-bending in its applications.

Student Educational Objectives

1. Understand and be able to write rules of grammar for several classes of languages.
2. Design FAs, PDAs, and TMs that follow these rules to determine if character strings input to the machines are in fact valid examples of the languages expressible by specific grammar rules.
3. Convert non-deterministic grammars (and their matching "non-deterministic" FA) into deterministic FA.
4. Follow and create proofs about the true or falsity of mathematical statements dealing with computability or complexity, including proofs by construction, contradiction, and induction.
5. Follow and develop proofs that certain problems are computable or not by some TM.
6. Describe the complexity of an algorithm in terms of steps executed by some TM, and when specific problems fall into a particular class of problems.