CSC262 Computer Vision

Spring 2020

Synopsis: We explore some fundamentals of image processing and understanding, considering several image representations for extracting information and survey algorithms for solving a wide variety of problems, such as image de-noising, panorama creation, 3-D reconstruction, segmentation, and object recognition.

Instructor:	Jerod Weinman
E-mail:	[`weinman`]

Virtual Office:: https://grinnellcollege.webex.com/meet/weinman
Course web page:: http://weinman.cs.grinnell.edu/courses/CSC262/2020F
Class meetings:: MWF, 10:00 am - 11:50 am

1 Accommodations
2 Overview
3 Texts
4 Class attendance
5 Activities
    5.1 Reading
    5.2 Pre-recorded lecture
    5.3 Laboratory exercises and write-ups
6 Grading
7 Academic honesty
8 Deadlines
9 Getting Help
    9.1 Discussion with Piazza
    9.2 Office Hours
    9.3 Email

1 Accommodations

If you have any disability that requires accommodations, please meet with me right away so that we can work together to find accommodations that meet your learning needs. You will also need to provide documentation of your disability to the Coordinator of Disability Services, John Hirschmann, located on the 3rd floor of Goodnow Hall (x3089).

2 Overview

How can we make computers understand images? What information is contained in an image and how can a meaningful representation be extracted? The problems of computer vision are two-fold. First, how can we take a two-dimensional image and reconstruct the three-dimensional world it came from? Second, how can we understand and recognize objects from those three-dimensional worlds?

Computer vision draws on a surprisingly large and diverse set of other fields, including psychology, neuroscience, mathematics, photogrammetry, optics, physics, signal processing, pattern recognition, artificial intelligence, and philosophy. We will try to find a way to glean what we need from each of these in a truly liberal (as in "liberal arts") fashion.

In this course, we will examine computational approaches to problems in low-level visual processing as well as some applications in higher-level vision.

Our major objectives for this course include:

Understanding the fundamental problems in computer vision
Learning both new and classical approaches to vision problems
Finding and appreciating the limitations and caveats of models and algorithms

Why take it?

Computers can do many things much better than humans, but this area is one where humans still excel. If you are interested in an exciting interdisciplinary field with many open problems and practical applications, this course is for you.

What do I need to know?

This course assumes you have

experience with programming in both the functional and imperative paradigms.
fluency in some basic computational ideas like encapsulation and code re-use
mathematical maturity, especially familiarity with multivariate calculus and linear algebra.¹

3 Texts

Due to discrepancies between topics or details appropriate to an undergraduate course, we have no single, official textbook. Instead, we will use selected readings from a variety of texts, which will be available on the course web page, PioneerWeb, and E-Reserves:

David Forsyth and Jean Ponce, Computer Vision: A Modern Approach, Prentice Hall, 2003.
Richard Szeliski, Computer Vision: Algorithms and Applications, Springer, 2020 (Draft Second Edition).
Emanuele Trucco and Alessandro Verri, Introductory Techniques for 3-D Computer Vision, Prentice Hall, 2008.
Ian Goodfellow, Yoshua Bengio, and Aaron Courville, Deep Learning, MIT Press, 2016.

Our laboratory exercises will be done in MATLAB. This software is available on the MathLAN, but if you wish to have MATLAB on your own personal computer, a student version may be purchased from http://www.mathworks.com/store, http://www.journeyed.com, or http://www.academicsuperstore.com.

Two well-organized, regularly updated, additional resources on the web include a manually categorized research bibiliography and a compendium.

CVonline: The Evolving, Distributed, Non-Proprietary, On-Line Compendium of Computer Vision
Annotated Computer Vision Bibliography

4 Class attendance

Class meetings will involve questions about readings and lecture videos followed by collaborative lab work.

Because is an active, collaborative course, your presence is integral to your learning. Although video lectures are pre-recorded for asynchronous viewing, and I will not explicitly grade based on your lab attendance, I want to be abundantly clear that I do expect you to attend our synchronous meetings.

Except in case of dire emergencies, I require you to let me know beforehand of your class absence so I can make any necessary adjustments to lab partner assignments. You do not need to give me a reason for your absence (I understand that current circumstances present innumerable complexities), but I want ensure you are heeding the fact that you will have (make-up) work to do in order to meet our desired learning outcomes. You should expect to hear from me if I find you missing significant amounts of class, particularly if you are not communicating with me about it.

5 Activities

Grinnell College expects you to work a total of 180 hours for a 4CR course. Over our 7.5 week term, that total amounts to 24 hours of work per week for this course. We will spend six of those hours in synchronous lab meetings, and the rest of your time will be occupied with a cycle of the following activities: reading, video lecture, lab, and write-up.

5.1 Reading

The first step in the process will be for you to engage ideas through reading. While our lab sessions will Our class meetings will be heavily discussion and lab-based, and this will require a significant amount of preparation on your part.

You should check the class schedule for updates. Reading texts entails the following:

Overview: You should quickly skim through the reading once to get an overview of the material to be covered, paying particular attention to subject headings and topic introductions. This first "reading" can (and should) be quick. (Expected time: 5 to 10 minutes.)
In-Depth: Next, read the material closely. Try to understand what individual steps of algorithms or mathematical equations are accomplishing. Not everything will make sense at this point, but hopefully many things will. (Expected time: 40 to 50 minutes.)
Final Notes: After carefully reading the material, mentally review and try making a few notes to yourself about what you think are the most important concepts being covered, as well as any . Bring your questions to class. I will ask for them. (Expected time: 5 to 10 minutes.)

Most readings for this class contain mathematics or algorithms that require a moderate amount of study. While I realize not everything is learned best by reading, you are asked to make your best effort to understand watching the video. Thus, to help you prepare, you may consider answering the following questions to be part of your daily homework:

Identify the section or concept from the reading that you find most confusing.
Briefly explain what you find confusing about it. (Doing so often yields revelations or other new understandings.)

5.2 Pre-recorded lecture

Before watching the corresponding lecture on the topic, I recommend you take at least a brief break after your hour of reading and study. This will allow your brain a small opportunity to consolidate the new ideas it has encountered (and a little bit of forgetting before practicing recall, also enhances your learning). Review your reading questions before watching the video so that you can be attentive to the material that may answer them.

Since these lectures are not interactive, you won't have the opportunity to ask questions immediately as they arise. Thus, as you watch the lectures, I recommend you pause the video to write out your questions in your notebook. The first part of our synchronous class meetings will be an opportunity for you to bring remaining questions from the text or class videos for us to discuss and address together as a class.

The videos are collected on a Microsoft Stream channel associated with a group for the class, and these are linked from within MS Teams.

5.3 Laboratory exercises and write-ups

All of our class meetings will involve collaborative work on laboratory exercises. The "hands-on" labs will provide an instructor-assisted setting for you concretely engage in the material introduced in readings and lecture. Labs will be posted in advance of the date they are begun in class; because our in-class lab time is limited, it is very important for you to come prepared by , bringing any questions to resolve at the beginning of class. However, you should not work on the labs before class.

Contents

You will be expected to produce a formal write-up of your results, including images and figures as appropriate, a description of your method(s), and providing some conclusions. You will also submit your code. Please see the course web page (and individual labs) for further details on what you are expected to submit. Although we will begin the labs in class, you might not finish the technical material in class; it is unlikely you will finish the write-up.

Collaboration

You will complete these labs in pairs or groups, typically assigned on a rotating basis. Everyone whose name appears on a submitted group lab report has the responsibility to ensure everyone fully understands the submission.

While you are welcome to discuss course concepts with others, solutions and any work you do and submit should be that of you and your group alone. (See Academic honesty, section 7 below.)

You are encouraged to use Piazza (see Section 9.1) for questions related to the course. If a post is related to an assignment, it must adhere to the standards of collaboration for that particular assignment.

Grading

Up to one fourth (25%) of the required labs may be graded on a completion basis only; such labs will weigh one-third of a fully assessed lab in the final grade calculation. You will not be told which labs are completion-graded because the instructor does not know in advance. Therefore, you should assume all labs will receive a full evaluation and produce a complete write-up for each. (Doing so will maximize your learning anyhow.)

6 Grading

My goal is for everyone taking this course to be able to demonstrate familiarity and fluency with the course concepts. I would be very happy if you all met the goals above and received "A"s.

Grading will be based on the College's Grading System with the following brackets proposed:

Average at least	Receives	Grade Points	Definition
3.75	A	4.00	Excellent
3.50	A-	3.67	Excellent
3.16	B+	3.33	Good
2.83	B	3.00	Good
2.50	B-	2.67	Good
2.16	C+	2.33	Satisfactory
1.50	C	2.00	Satisfactory
0.50	D	1.00	Passing
0.00	F	0.00	Failing

To compensate for the unpredictability of learning exercises' outcomes, the bracket minimums (left column) may be adjusted downward (but not upward).

Note that the "average" used may not be an arithmetic mean, but a geometric mean.

The logistics of grading will happen via the Gradescope (https://www.gradescope.com) website. Since the College login will not be used during this term, please ensure the password you choose adheres to the Grinnell College Password Policy (https://grinco.sharepoint.com/sites/its/ServiceCatalogPages/Password%20Information.aspx)

7 Academic honesty

As students, you are members of the academic community. Both the College and I expect the highest standards of academic honesty. (See the Grinnell College Student Handbook). Among other things, this means clearly distinguishing between work and ideas that are your own, and those that should be attributed to others. It is expected that the collaboration policies given in this syllabus and on particular assignments will be followed. In particular:

When you explicitly work as part of a group or team, you need not identify the work of each individual (unless I specify otherwise).
You may discuss (algorithms, ideas, approaches, etc.) described in the readings, during class, or explained in the lab assignments with anyone.
You may only discuss homework assignments (algorithms, solutions, write-ups, code, debugging, etc.) with your group members or the instructor.
All the work submitted (code, experimental data, write-ups, etc.) must be your [group's] own. Code or documentation provided by the instructor should be attributed, but no other code or written work (from any source) may be shared with others or copied for your own use.
All non-syntax consultations (including the textbook and language references) require formal citation within the related program or write-up.
Any conceptual contributions by others (not in your group) must be acknowledged and attributed in your report. That is you must give specific attribution for any assistance you receive. The suggested acknowledgment format is

"[Person X] helped me to do [thing Y] by [explaining Z]."
Any program results or output must be faithfully recorded, not forged. (A thoughtful explanation of unexpected behavior can often be a worthwhile submission and is much better than the alternative.)
You are responsible for safeguarding your work from being copied by others. This requires you to take reasonable precautions with hard copy printouts as well as file system permissions.

As an instructor, I will meet my obligation to bring any work suspected to be in violation of the College's Academic Honesty Policy to the attention of the Committee on Academic Standing, after which there is no recourse with me.

8 Deadlines

Assignments are due at the specified time and date. Namely, a lab assignment (code and write-up) is due at the beginning of the class following the lab session. That is, Monday's lab is due at 10 am on Wednsday, Wednesday's lab is due at 10 am on Friday, and Friday's lab is due at 10 am on Monday. While the pace is both regular and relentless, the spacing should give you enough flexibility to meet with your partner(s), while still allowing room for reading the text, watching the videos, and reading the next lab assignment.

Assignments due on days for which you have a prior excused absence must still be submitted by the deadline.

A late penalty of one letter grade will be deducted in each subsequent twenty-four hour period after the deadline.

In the software production world, missing critical release deadlines can have significant consequences for individuals, teams, or even entire companies. Planning carefully with deadlines in mind is a good habit to develop. After all, in the academic world, missing assignment deadlines has consequences not only for your grade, but impacts your time available for other coursework and can negatively affect your overall, general wellness.

Exception: Deadlines for MathLAN computer-based assignments will automatically be extended by at least one twenty-four hour period if MathLAN is down for an unscheduled period of three or more hours during the week preceding the assignment due date.

9 Getting Help

9.1 Discussion with Piazza

For class discussion, we will use Piazza, which is designed to get you help fast and efficiently from your classmates and myself. Rather than emailing questions, please to post your questions on Piazza, which is linked directly from the P-Web course or at https://piazza.com/grinnell/fall2020/csc262/home.

9.2 Office Hours

Please come by during my office hours to discuss the course content, get any extra assistance, or just talk about how the course is going. Note that if multiple students have similar questions or issues, we may work together as a group. If you cannot attend a scheduled office hour, you may also email me to schedule an appointment; please include 3-4 possible meeting times so that I can pick one that works for me.

I enjoy getting to know my students, but I prefer to reserve office hours for academic matters. If you would like to have a more informal conversation, I would be delighted to accept an invitation to eat lunch with you at the Marketplace.

9.3 Email

Email is also a reliable way to contact me, but please allow 24 hours for a response (except on weekends, when I do not regularly read email).

With thanks to Janet Davis for the "Reading Suggestions" and other key policies.

Footnotes:

¹If linear algebra is a distant memory, I strongly urge you to study Eero Simoncelli's "A Geometric Review of Linear Algebra " http://www.cns.nyu.edu/~eero/NOTES/geomLinAlg.pdf.