CSC262 Computer Vision

Fall 2024

Description: How can computers understand images? This course covers principles in computational vision and their relationship to human visual perception. Topics may include geometry of image formation, image filtering and representation, texture analysis, 3-D reconstruction from stereo and motion, image segmentation, object detection, and recognition. Students implement and evaluate methods on real images in laboratory exercises and an independent project. Synopsis: We explore some fundamentals of image processing and understanding, considering several image representations for extracting information and survey algorithms for solving a wide variety of problems, such as image de-noising, panorama creation, 3-D reconstruction, segmentation, and object recognition.

M/W 10:00-11:50 am Science 3815
Instructor:	Jerod Weinman
Office:	Noyce 3825
Phone:	x9812
E-mail:	[weinman]

Course web page: http://weinman.cs.grinnell.edu/courses/CSC262/2024F

Contents

english 1  Accommodations
2  Overview
3  Texts
4  Class attendance
5  Schedule of topics
6  Activities
    6.1  Reading
    6.2  Participation
    6.3  Laboratory exercises and write-ups
    6.4  Exams
    6.5  Independent project
7  Electronic Devices
8  Grading
9  Academic honesty
    9.1  Policy on AI-based Tools
10  Deadlines
11  Getting Help
    11.1  Discussion with Piazza
    11.2  Office Hours
    11.3  Email

1  Accommodations

If you have any disability that requires accommodations, please meet with me right away so that we can work together to find accommodations that meet your learning needs. You will also need to provide documentation of your disability to Disability Resources staff, located in Steiner Hall 209 (x3089 or [access]). Please also note that I require your accommodations. The synthetic fragrances found in deodorants, lotions, after shave, body sprays, laundry products, perfume, cologne, etc. make many people who suffer with asthma, allergies, environmental sensitivities, cancer, and migraines much sicker. I am sensitive to many such chemicals you may not even notice, so please try to avoid using such scented products before coming to class and especially if you visit my office. Grinnell College is committed to compliance with Title IX and to supporting the academic success of pregnant and parenting students and students with pregnancy related conditions. If you are a pregnant student, have pregnancy related conditions, or are a parenting student (child under one-year needs documented medical care) who wishes to request reasonable related supportive measures from the College under Title IX, please email the Title IX Coordinator at titleix@grinnell.edu. The Title IX Coordinator will work with Disability Resources and your professors to provide reasonable supportive measures in support of your education while pregnant or as a parent under Title IX.

2  Overview

How can we make computers understand images? What information is contained in an image and how can a meaningful representation be extracted? The problems of computer vision are two-fold. First, how can we take a two-dimensional image and reconstruct the three-dimensional world it came from? Second, how can we understand and recognize objects from those three-dimensional worlds? Computer vision draws on a surprisingly large and diverse set of other fields, including psychology, neuroscience, mathematics, photogrammetry, optics, physics, signal processing, pattern recognition, artificial intelligence, and philosophy. We will try to find a way to glean what we need from each of these in a truly liberal (as in "liberal arts") fashion. In this course, we will examine computational approaches to problems in low-level visual processing as well as some applications in higher-level vision. Our major objectives for this course include:

• Understanding the fundamental problems in computer vision
• Learning both new and classical approaches to vision problems
• Finding and appreciating limitations and caveats of models and algorithms
• Developing technical writing skills

Why take it?

Computers can do many things much better than humans, but this area is one where humans still excel. If you are interested in an exciting interdisciplinary field with many open problems and practical applications, this course is for you.

What do I need to know?

This course assumes you have

• experience with programming in both the functional and imperative paradigms.
• fluency in some basic computational ideas like encapsulation and code re-use
• mathematical maturity, especially familiarity with multivariate calculus and linear algebra.¹

3  Texts

Due to discrepancies between topics or details appropriate to an undergraduate course, we have no single, official textbook. Instead, we will use selected readings from a variety of texts, which will be available on the course web page, PioneerWeb, and E-Reserves:

David Forsyth and Jean Ponce, Computer Vision: A Modern Approach, Prentice Hall, 2003.

Richard Szeliski, Computer Vision: Algorithms and Applications, (Second Edition) Springer, 2022.

Emanuele Trucco and Alessandro Verri, Introductory Techniques for 3-D Computer Vision, Prentice Hall, 2008.

Antonio Torralba, Phillip Isola, and William T. Freeman, Foundations of Computer Vision, MIT Press, 2024.

Our laboratory exercises will be done in MATLAB. This software is available on the MathLAN, but if you wish to have MATLAB on your own personal computer, a version may be downloaded for College-related work from https://www.mathworks.com/academia/tah-portal/grinnell-college-31597603.html. Two well-organized, regularly updated, additional resources on the web include a manually categorized research bibiliography and a compendium.

CVonline: The Evolving, Distributed, Non-Proprietary, On-Line Compendium of Computer Vision

Annotated Computer Vision Bibliography

4  Class attendance

Class meetings will involve a mix of collaborative activities, labs, discussions, and lectures. In short: You are expected to attend and actively participate in class. I am expected to make class worth attending. Because is a collaborative, lab and discussion-based course, your presence is integral to your learning. Thus, 1.5% (0.06 grade points) will be deducted from your overall grade for each absence. I know that sometimes "things happen." Therefore, you will be granted one unexcused absence from class without penalty. However, this rebate is cancelled upon a second unexcused absence. If you are going to be absent, you must send a written explanation (email is appropriate) before class except in the (rare) case of dire emergency. (Consider the analogy for phoning in sick for work; you wouldn't do it at 4 PM if your supervisor expected you at 8 AM.) Because I do not wish you to risk harm to yourself or others, I typically moderate or annul absence penalties in case of illness, provided you notify me in advance. If you know in advance that you will be absent for any planned reason (e.g., athletics, interviews, etc.), you must notify me in writing (again, email is fine) at least 7 days in advance to make arrangements for considering your absence excused. Our discussions benefit from your contributions. However, if you do miss a class, you should first talk to a classmate about any material that you may have missed. After that, you may follow up with the instructor about any further questions or concerns.

5  Schedule of topics

The following is an approximate schedule of topics to be discussed during the course. See the web page schedule for daily details.

6  Activities

The Grinnell College definition of a credit hour states that a four credit course such as this translates to a minimum of 180 hours of total time both in and out of the classroom.² That we meet for four hours of class each week implies you should spend at least eight hours each week working outside of class.

6.1  Reading

Our class meetings will be heavily discussion and lab-based, and this will require significant preparation on your part. Most of this preparation will consist of careful reading and reflection on the material. You should check the class schedule for updates and read any material that has been assigned before coming to class. Reading texts entails the following:

Overview

You should quickly skim through the reading once to get an overview of the material to be covered, paying particular attention to subject headings and topic introductions. This first "reading" can (and should) be very quick. (Expected time: 5 to 10 minutes.)

In-Depth

Next, read the material closely. Try to understand what individual steps of algorithms or mathematical expressions are accomplishing. Not everything will make sense at this point, but hopefully many things will. (Expected time: 40 to 50 minutes.)

Final Notes

After carefully reading the material, mentally review and try making a few notes to yourself about what you think are the most important concepts being covered, as well as any questions you have. Bring your questions to class. I will ask for them. (Expected time: 5 to 10 minutes.)

Most readings for this class contain mathematics or algorithms that require a moderate amount of study. While I realize not everyone learns best by reading, you are asked to make your best effort and come to class with questions. Then we can proceed with discussion, examples, and exercises that enhance and clarify the material in class. Thus, you may consider answering the following questions to be part of your daily homework:

• Identify the section or concept from today's reading that you find most confusing.
• Briefly explain what you find confusing about it.

6.2  Participation

Because much of our work in this course involves collaboration and discussion, you are expected to demonstrate full participation. Participating in class involves:

• being present in class (physically and mentally)
• coming to class on time
• coming to class prepared
• asking questions when appropriate
• making positive contributions to class discussion by volunteering and when called upon
• staying on task during collaborative exercises, and
• working effectively with your group.

Failure to participate regularly or participating in counterproductive ways (e.g., by dominating the conversation or making inappropriate comments) may negatively impact a student's grade.

6.3  Laboratory exercises and write-ups

Almost every class meeting will involve laboratory exercises. The "hands-on" labs will provide an instructor-assisted setting for you concretely engage in the material introduced in class and readings. Labs will be posted in advance of the date they are begun in class; because our in-class lab time is limited, it may be helpful for you to prepare by reading the lab assignment beforehand, bringing any questions to resolve at the beginning of class. However, you should not work on the labs before class. Contents   You will be expected to produce a formal write-up of your results, including images and figures as appropriate, a description of your method(s), and providing some conclusions. You will also submit your code. Please see the course web page (and individual labs) for further details on what you are expected to submit. Although we will begin the labs in class, you might not finish the technical material in class; it is unlikely you will finish the write-up. Collaboration   You will typically complete these labs in pairs or groups, assigned on a rotating basis. Everyone whose name appears on a submitted group lab report has the responsibility to ensure everyone fully understands the submission. While you are welcome to discuss course concepts with others, solutions and any work you do and submit should be that of you and your group alone. (See Academic honesty, section 9 below.) You are highly encouraged to use Piazza (see Section 11.1) for questions related to the course. If a post is related to an assignment, it must adhere to the standards of collaboration for that particular assignment. Grading   Up to one third (33%) of the required labs may be graded on a completion basis only; such labs will weigh one-third of a fully assessed lab in the final grade calculation. You will not be told which labs are completion-graded because the instructor does not know in advance. Therefore, you should assume all labs will receive a full evaluation and produce a complete write-up for each. (Doing so will maximize your learning anyhow.)

6.4  Exams

As opportunities for you to demonstrate your image interpretation prowess and grasp of vision principles there will be two mid-semester exams, one before fall break and the other near the end of the semester. Although exam 2 is not intended to be cumulative, certain topics build on each other naturally and cannot be excluded.

6.5  Independent project

In the second half of the semester, you will propose and deploy an independent project that uses and builds on the principles we are learning in the class. You will have several weeks to work on the project and will be expected to conduct a small amount of mentored, but self-directed learning. Several example possibilities will be provided to get your imagination running. You will propose a project including a description of how a successful deployment will be produced, as well as milestones and benchmarks. You will share the results of your work in a final report and a presentation during finals week. Further details will be given near fall break.

7  Electronic Devices

With the exception of collaborative computer-based lab activities that use the existing MathLAN computers, the use of electronic devices (i.e., smartwatches, phones, tablets, and laptops) will be prohibited during class. Why?

• Taking notes on laptops is generally less effective for learning than writing longhand, which forces you to consolidate information as you process it.³
• Distractions (email, messages, social media, etc.) hinder your own learning⁴⁵ as well as that of those around you.⁶
• Even the mere presence of such devices impedes cognition.⁷

When class begins, please put your electronic devices away until after the conclusion of the class. Violations of this policy will negatively impact a student's grade. Exceptions to this policy can be made as an accomodation for those with documented disabilities. An exception may be made at the conclusion of lab classes, when you may wish to trade contact information with your partner or establish a schedule to meet for out-of-class collaboration.

8  Grading

My goal is for everyone taking this course to be able to demonstrate familiarity and fluency with the course concepts. I would be very happy if you all met the goals above and received "A"s. The following weighting will provide a basis for evaluation. Note that this means you may need to unlearn sensitivity to a traditional grading system. A grade of 2.5 is not 62% and failing, but a B− ("Good"). To compensate for the unpredictability of learning exercises' outcomes, the bracket minimums (left column) may be adjusted downward (but not upward). The logistics of grading will happen via the Gradescope (https://www.gradescope.com) website.

9  Academic honesty

As students, you are members of the academic community. Both the College and I expect the highest standards of academic honesty. (See the Grinnell College Student Handbook). Among other things, this means clearly distinguishing between work and ideas that are your own, and those that should be attributed to others. It is expected that the collaboration policies given in this syllabus and on particular assignments will be followed. In particular:

1. When you explicitly work as part of a group or team, you need not identify the work of each individual (unless I specify otherwise).
2. You may discuss concepts (algorithms, ideas, approaches, etc.) described in the readings, during class, or explained in the lab assignments with anyone.
3. You may only discuss homework assignments (algorithms, solutions, write-ups, code, debugging, etc.) with your group members or the instructor.
4. All the work submitted (code, experimental data, write-ups, etc.) must be your own. Code or documentation provided by the instructor should be attributed, but no other code or written work (from any source) may be shared with others or copied for your own use.
5. All non-syntax consultations (including the textbook and language references) require formal citation within the related program or write-up.
6. Any conceptual contributions by others (not in your group) must be acknowledged and attributed in your report. That is you must give specific attribution for any assistance you receive. The suggested acknowledgment format is
   1. "[Person X] helped me to do [thing Y] by [explaining Z]."
7. Any program results or output must be faithfully recorded, not forged. (A thoughtful explanation of unexpected behavior can often be a worthwhile submission and is much better than the alternative.)
8. You are responsible for safeguarding your work from being copied by others. This requires you to take reasonable precautions with hard copy printouts as well as file system permissions.

As an instructor, I will meet my obligation to bring any work suspected to be in violation of the College's Academic Honesty Policy to the attention of the Committee on Academic Standing, after which there can be no recourse with me.

9.1  Policy on AI-based Tools

Code   You are not permitted to use ChatGPT, GitHub Copilot, or any other programming assistance tool that is trained on others' code for this course. If you aren t already aware, GitHub Copilot is a machine learning system that suggests code as you type in VSCode. Copilot is trained on code hosted by GitHub, and offers suggestions that go far beyond the usual API-based autocomplete suggestions many programming editors produce (e.g., VSCode's Intellisense, which you are welcome to use for this course). There are major legal questions around Copilot's use of other code, particularly as it connects to open source licenses that place restrictions on the licensing of derivative works.⁸ For the purposes of this class, there is no ambiguity about the use of AI tools: code suggested by AI tools (e.g., Copilot, ChatGPT, etc.) is a direct violation of honesty policy item #4 above. If you happen to use your own computer for any programming in this class I ask that you do not install GitHub Copilot, or uninstall it if you have already done so. If you prefer not to remove Copilot from your computer you are welcome to complete all work for this course on MathLAN machines, which will not have Copilot installed. Prose   Similarly, you are not permitted to use ChatGPT or other language models for producing or editing prose text in your lab write-ups. Technical writing skills will be developed through iterations of practice and feedback.

10  Deadlines

Assignments are due at the specified date and time (usually 10 pm for lab writeups). Assignments due on days for which you have a prior excused absence must still be submitted by the deadline. A late penalty of one letter grade will be deducted in each subsequent twenty-four hour period after the deadline. Exception: Deadlines for MathLAN computer-based assignments will automatically be extended by at least one twenty-four hour period if MathLAN is down for an unscheduled period of three or more hours during the period preceding the assignment due date.

11  Getting Help

11.1  Discussion with Piazza

For class discussion, we will use Piazza, which is designed to get you help fast and efficiently from your classmates and myself. Rather than emailing questions, please to post your questions on Piazza, which is linked directly from the P-Web course or at https://piazza.com/grinnell/fall2024/csc262/home.

11.2  Office Hours

Please come by during my office hours to discuss the course content, get any extra assistance, or just talk about how the course is going. Note that if multiple students have similar questions or issues, we may work together as a group. If you cannot attend a scheduled office hour, you may also email me to schedule an appointment; please include 3-4 possible meeting times so that I can pick one that works for me. I enjoy getting to know my students, but I prefer to reserve office hours for academic matters. If you would like to have a more informal conversation, I would be delighted to accept an invitation to eat lunch with you at the Marketplace.

11.3  Email

Email is also a reliable way to contact me, but please allow 24 hours for a response (except on weekends, when I do not regularly read email). You may also call me in my office (x9812) for more urgent matters (e.g., you will be missing a lab due to illness). With thanks to Sam Rebelsky for the "Participation" section, and Janet Davis for the "Reading Suggestions" and other key policies.

---

Footnotes:

¹If linear algebra is but a distant memory, I strongly urge you to study Eero Simoncelli's "A Geometric Review of Linear Algebra " http://www.cns.nyu.edu/~eero/NOTES/geomLinAlg.pdf. ²Grinnell College. (2016) Grinnell College Semester Credit Definition. ³Mueller, P. A., & Oppenheimer, D. M. (2014). The pen is mightier than the keyboard: Advantages of longhand over laptop note taking. Psychological science, 25(6), 1159-1168. ⁴Carter, S. P., Greenberg, K., & Walker, M. S. (2017). The impact of computer usage on academic performance: Evidence from a randomized trial at the United States Military Academy. Economics of Education Review, 56, 118-132. ⁵Glass, A. L., & Kang, M. (2018). Dividing attention in the classroom reduces exam performance. Educational Psychology, 1-14. ⁶Sana, F., Weston, T., & Cepeda, N. J. (2013). Laptop multitasking hinders classroom learning for both users and nearby peers. Computers & Education, 62, 24-31. ⁷Ward, A. F., Duke, K., Gneezy, A., & Bos, M. W. (2017). Brain drain: the mere presence of one's own smartphone reduces available cognitive capacity. Journal of the Association for Consumer Research, 2(2), 140-154. ⁸You can read more about these issues at https://www.plagiarismtoday.com/2021/07/08/github-copilot-and-the-copyright-around-ai/.