CSC 499 Mentored Advanced Project

Synopsis: A mentored advanced project (MAP) is an intensely collaborative research effort between students and a faculty member that is likely to lead to a scholarly product for external presentation. It is both an opportunity to enhance faculty scholarship and to train students in research methods.


Instructor:		Jerod Weinman
Office:		SCI 3825
Office Phone:		x9812
Lab:		SCI 3828
Lab Phone:		x4457
E-mail:		[`weinman`]

http://www.cs.grinnell.edu/~weinman/courses/CSC499

1 Overview

In addition to teaching, a major component of a professor's work is scholarly in nature. A professor spends a significant amount of time establishing new knowledge by identifying important research questions that are unanswered, working to determine explanations, and publishing results for the community at large.

As students, you have already been trained in some fundamentals of computer science. The MAP is an opportunity for you to begin to engage in active research within an established research program. You will work alongside the faculty member in an operational, state of the art laboratory facility to develop and practice important research skills with the goal of contributing to a result that will be submitted for publication.

Perhaps the key word for this endeavour is that it is mentored. This mentorship may also be thought of as an apprenticeship. As your mentor, my goal is to establish a framework for activities that will allow you to emulate, experience, and learn about advanced study in computational science. As apprentices, it will be your duty to embrace these opportunities by actively listening, participating, and asking questions at all times, and not just when a particular problem confronts you.

Our major objectives will include:

Practicing research methods and the scientific process
Learning about advanced topics in computer science
Developing tools for use by the larger research community

2 Environment

2.1 Computing

Our research environment will include the MathLAN computer system, as well as two non-MathLAN compute servers (boltzmann and gibbs) that I administer for research purposes. You will make use of an SVN repository for managing shared program code, and a data repository for recording experimental designs and results as well as other important data. You may need the public MathLAN machines for some software tools, but most computation will be done on our compute servers.

boltzmann features a Tesla C1060 GPU, which has 240 processors, but requires specially-written software to use. It otherwise has two Intel Xeon E5520 Quad-Core 2.26 GHz processors and 48GB of 1066 MHz DDR3 RAM with 1.5TB of available disk using a hardware RAID-1 setup.

gibbs has two Intel Xeon E5450 Quad-Core 3.0 GHz processors and 24GB of 667 MHz DDR2 RAM with 3TB of disk using a hardware RAID-5 setup.

Both of these machines have their own data repositories, which are not shared or automatically synchronized.

2.2 Physical

We have our own research lab facilities in SCI 3828. I will provide you with the key code. Though you are welcome to keep the door open while you work, please be sure you leave the room locked when no one is in it. You may store your work materials here during the summer, making use of the bookshelves and/or counter space as needed. However, please keep the area orderly and free of excessive debris and clutter.

Do not use food at the computer terminals and please do not dispose of any food waste in the lab. Instead, use the commons (SCI 3817) or the waste baskets by the elevator, as these are emptied more regularly.

boltzmann is a rather loud machine; if you prefer to do your work in a classroom for the summer because of this, please let me know. These too are shared spaces and you should also keep them tidy as you work.

3 Expectations

I have the highest expectations of my MAP students. This means I expect you to work hard, be productive, practice good technique, and communicate well in both written and oral formats. I will strive to give you regular feedback on all of these aspects, but you are encouraged to regularly inquire as well.

3.1 Schedule

Academics likely work harder than you think, and it is certainly not for the pay.

A summer MAP is a full time job. Therefore, I expect you to work at least 40 hours per week, or 8 hours per day. Since we are working as a group, shared hours are important, therefore you should plan to be present from 8:30-5(+), except for a lunch break. If you take a longer lunch, you should arrive earlier or stay later. I do pay attention to your hours and will not hesitate to let you know if you seem not to be meeting expectations. Like any other job, you should ask about excusing any absence or occasionally shifting work hours to accommodate some other activity.

I do not consider working longer hours a necessity for excellence, but clear shirking of full-time work hours is not a hallmark of an A student.

3.2 Engagement

An apprenticeship is a bi-directional activity. Thus, you are expected to diligently undertake the activities set forth, but also to take the initiative to explore and pursue useful avenues of inquiry that are not explicitly given to you. I consider regular demonstrations of such initiative to be the hallmarks of an A student.

Your peer(s) will also provide a confidential evaluation of your level of engagement with the project.

3.3 Collegiality

Research is rarely an individual activity. In this context, you are participating as a part of a larger research group, a fact that carries some additional responsibilities. I expect you to maintain an environment that is supportive of our collective efforts and free from distraction.

Some resources, such as compute servers and software licenses, are limited. These should be used with consideration for and perhaps in communication with others in the group.

Physical scientists have very clear lab materials and spaces that must be maintained; computer science is hardly different. I expect you to keep clean physical and electronic work spaces. The experimental data repository is a highly shared resource that should be documented, organized, and constructed well to allow for colleagues and peers to learn from and build upon work done in it. Though their frequency may vary, contributions that are clear, thoroughly documented, and well organized are certainly hallmarks of excellence.

3.4 Communication

A scientist may have the most wonderful ideas and results, but they must be able to communicate them clearly and effectively to have impact. Much of your work will involve producing written artifacts. As Grinnell students, you are already aware of the college's high standards for writing. The work you produce for the MAP will also be held to this high standard. You should be constantly considering your audience and the questions they will approach your writing with, as well as practicing good compositional style, and of course grammar, spelling, etc.

For feedback and assistance beyond what I give, you may consider contacting the Writing Lab staff to help you focus and sharpen your writing. Even at the advanced level, writing can always be improved. Perhaps more than any other activity, good writing is a rare hallmark of excellence that often clearly distinguishes an A student.

4 Activities

We will engage in several regular activities throughout the MAP to help promote accountability, establish good practice, and ensure reasonable progress. You have many tasks demanding your time. It will therefore be imperative for you to practice good time management to maximize your research productivity and communication efficiency.

4.1 Today Messages

A today message is a simple e-mail consisting of a brief, bulleted list of your activities and accomplishments for the day. This has the advantage of keeping everyone aware of each other's activities, provides a searchable record, and also affords frequent, semi-public accountability.

As members of a scholarly community of computer scientists, we can all benefit from such an activity. A mailing list with current active MAP students and advisors is maintained on the MathLAN (today at cs). You should send your today message to this list at the conclusion of every work day.

Here are two examples.

Example 1

Today I ...

Reviewed a re-submitted journal article (much improved) and submitted a (much shorter) 1.5 page review
Read comments on our book chapter proposal and forwarded them to co-authors
Studied some of the NVIDIA CUDA reduce code ... realized I am in over my head and don't understand enough to progress individually
Contacted a research collaborator to try and establish next steps for integrating said reduce code
Looked over a submitted bibliography and found three papers I have not read and should look at
Emailed fellow tutor about possibly linking tutorials
Fixed a network configuration on gibbs so that Matlab would work

Example 2

Today I ...

Learned vi this morning
Learned how to navigate in the terminal a little better
Learned a few basics/history to the creation and inner-workings of the Scheme programming language
Slew a tree as I printed out a textbook on scheme and the r5rs for Scheme
Figured out the purpose of syntactic extension
Found more clear ways that macros/syntactic extension could help us in our project.
Began coding my own syntactic extension examples
Began putting together an outline to help teach/explain what i've been learning on macros to the rest of the group.

4.2 Meetings

We will hold three regular meetings during the week, though additional impromptu and informal dialogues will almost certainly occur. For all of our formal meetings, you should prepare questions or other important matters for discussion by recording them beforehand. You should also prepare other materials as appropriate. For meetings to be productive, it is imperative that you take notes. Suggestions will be made, work plans will be modified, and other feedback may be given; for these to be of any use, you must record them.

4.2.1 Monday

We will start the week together as an opportunity for me to remind you of my own goals for you for the week. This time will also allow you (with your partner) to establish your milestones. In a short "presentation" format, I expect you to document (i.e., write on the board) what you plan to accomplish during the week. This is to ensure that everyone (especially you) has an explicit, written record to refer to and use as a guide.

You should be as specific as possible, describing methods and means more than end goals. This should encourage you to think about what you are explicitly going to do and hopefully why. Inexperienced researchers often place too much focus on the goal, rather than on potential means of achieving it, typically putting that achievement in peril. For instance,

Bad:: Fix over-fitting problem
[goal only]
Better:: Try smoothing to prevent over-fitting
[suggests a method]
Good:: Implement and explore Gaussian and Laplace smoothing to prevent over-fitting
[recognizes some development is needed and suggests specific alternative techniques]
Best:: Implement Gaussian and Laplace smoothing and use several parameters to chart performance on training and testing data during learning to determine behavior and find an optimum, if one exists
[describes an explicit process, outcome, and motivation]

I will critique your plans, offering alternatives, suggesting more if it is inadequate for a week's work, or paring things down if they are too ambitious or unlikely to succeed.

This meeting should not be your first activity of the day. Rather, you should plan your agenda carefully beforehand.

4.2.2 Wednesday

Our midweek meeting will provide a definite opportunity for checking in, just in case we don't do anything informal in between. You can briefly provide any updates on your progress and (more importantly) we can address questions that have arisen in your work. This will also be an opportunity to explicitly discuss research methods. Our Wednesday meetings are when I will teach and introduce important concepts for your coming written milestones.

4.2.3 Friday

Our final meeting of the week has a joint "looking back/looking forward" agenda.

In "looking back", you will give a brief (10-15 minutes) individual review outlining your accomplishments and results for the week. I expect you to anchor your review with appropriate board work so that a written record may be produced. If electronic visual aids are needed (e.g. for graphs, tables, images, etc.) please use them, but you do not need to spend additional time preparing elaborate materials. This is an "on the ground" opportunity for sharing and engaging in discussion. Your materials should be legible, but not as formal as a conference presentation.

In addition to sharing your results, you should interpret them at two levels: what you learned from them directly (as in the scientific importance or their relevance to the outcome of the project) and what you learned from the activities at a meta level. That is, do some debugging of your own process. For instance, "I spent too much time writing, letting my perfectionist tendencies overwhelm all of my other tasks for the week."

Finally, in "looking forward," you should outline the new questions or issues and potential directions you are planning to explore in the week to come. At this stage the "bad" or "better" goal examples given above in the Monday meeting is cast at a perfectly acceptable level. Having formulated an issue, you can then spend some time thinking about how to translate that into more concrete next steps. With the rest of the afternoon, you should have ample time to explore and consider means of doing that translation.

4.2.4 Timing

Summer: Our meetings will be Monday at 9 am, Wednesday at 4 pm, and Friday at 1 pm.
Academic Year: Ongoing meeting times will be established as schedules permit.

4.3 Lab Notebooks

You should keep a written record of your activities in a laboratory notebook. This is a bound notebook with either lined or graph paper (your preference; I prefer lines) whose pages cannot be removed. In it, you should keep track of what it is you are doing. Record questions that arise, the processes you are following, ideas you have, and any answers you find or conclusions you reach.

Record the date of each entry and an appropriate brief title (at most one line) to give context to your work and make it easily "searchable." This should stand out visually, perhaps by writing it in all capital letters, underlining or boxing, and/or setting it apart from other material by an extra line or two.

Your notebooks should provide a legible, reasonable record of your activities. Keep them from looking sloppy. You may wish to attach graphs or other results that are germane to your record.

I will review your notebooks bi-weekly, using the following criteria for assessment.

check +: Relevant details are concisely documented, organization and aesthetics provide a clear and complete record of work
check: Record of activities is adequate, including most major details and sufficient organization to find and track ideas and progress
check -: Element(s) appear to be missing (e.g., motivation, line of thought, process details) or organization and aesthetics are lacking
zero: Little or no record and utter lack of organization

4.4 Peer Review

Another important skill of scholars is being able to communicate with others in such a fashion that critique of the scholarship can be accepted, thereby improving the final results.

When there is overlap with other MAP students, you will give brief weekly presentations describing (or introducing, as appropriate) the state of your work. These should not exceed 15 minutes, and should allow at least 5 minutes for questions and discussion with your peers and scientific colleagues. Assist your audience by providing a visual outline of your presentation. This may be done on a whiteboard, but it could also be in an electronic format. Either way, these should be brief so as to enhance and anchor what you say verbally, rather than distract from it. Other visual aids (charts, images, etc.) may be used as appropriate.

It is likely that your presentation will be a hybrid between an oral distillation of your previously completed written milestone (see below), and your current undertakings (as outlined in Monday's meeting). As a result, you will have already done much preparation in terms of content, and even organization. You should therefore not spend more than 1 hour (hopefully even less) making final preparations for your presentation.

You will also be listening to others' presentations of the same format. You are expected to actively engage in these and participate in the discussion. After all, you are the peers; it is your job to provide the review.

Your oral presentations will be assessed (using the same check +/- system) on the following criteria:

Motivation: Is the context for the key ideas clearly established?
Clarity: Is the content easily understood by the audience?
Preparation: Are the presenters adequately knowledgeable about material (especially for questions)?
Materials: Are the visual aids clear, correct, and helping to anchor the presentation?
Oral Skills: Do the presenters make eye contact and enunciate well without distracting mannerisms?
Engagement: Is there dialogue with other presenters on their material?

In addition to the oral review, you may be called on to write reviews of written material and consider the peer reviews of your own written material (especially in producing your final report).

5 Deliverables

It is my hope that the work we do and our time spent together will be fun. However, since this is a formal course, several items will be evaluated over the course of the MAP to provide a means for assessment and feedback.

5.1 Written Milestones

As part of your apprenticeship activities, there will be written reports to help guide you through the research process in general, but particularly as it relates to this project. These are described in brief below, but more details will follow for each assignment throughout the term.

Research Description: (Summer Week 2) An overview and breakdown of the system, task, and environment that will influence the behavior of the study's artifacts.
Assessment of Current Knowledge: (Summer Weeks 2 and 3) Review of relevant primary articles from the scientific literature.
Behavioral Exploration: (Summer Week 5) Investigation of behavior with respect to key variables as a means for finding research questions.
Research Proposal: (Summer Week 6) Identification of a central research question and falsifiable hypotheses.
Experimental Design: (Summer Week 7) Description of the variables, protocol, and analysis method for testing a hypothesis.
Experimental Results: (Summer Week 9) Tabulated outcomes of the experimental test of a hypothesis.

5.2 Final Report

A final report will synthesize your results in the style of a scientific technical report using materials from the incremental milestones and feedback from peer and instructor reviews.

Our major objectives include both practicing scientific development and communicating those results. In computer science, the major means of communicating to the scientific community is via conferences. Your paper will hopefully form the basis of a submission to a regional, national, or even international conference.

5.3 Poster

You will also create a poster conveying the substance of your project. Your results will be communicated locally during the Science Division Student Research Session (part of Family Weekend, typically the first weekend in October). Your research poster will also be displayed with some perpetuity in the hallways of Noyce.

Guidelines on poster size and content design will be given at a later time.

6 Grading

My goal is for you to become able practitioners in the science of computing. With diligent effort in the process and detailed care applied to your writing, you can receive an A. However, an A is a mark of exceptional, excellent work; if most of the categories below do not meet this standard, you will be evaluated accordingly.

The following weighting will provide a basis for evaluation.

Lab Notebooks	5%
Peer Review Activities	5%
Written Milestones (6)	60%
Citizenship	10%
Experimental Code	10%
Poster	5%
Final Report	10%

7 Academic Honesty

You, as students but particularly as apprentices, are members of the academic community. Both the College and I expect the highest standards of academic honesty. (See the Grinnell College Student Handbook, e.g.,
http://www.grinnell.edu/offices/studentaffairs/shb/section3/academichonesty).
Among other things, this means clearly distinguishing between work that is your own, and work that should be attributed to others. Furthermore, any program results or output must be faithfully recorded, not forged. (A thoughtful explanation of unexpected behavior can often be a worthwhile submission and is much better than the alternative.)

As an instructor, I will meet my obligation to bring any work suspected to be in violation of the College's Academic Honesty Policy to the attention of the Committee on Academic Standing, after which there is no recourse with me.

8 Contacting Me

Summer

During the summer, I am in my office nearly every day. I typically work from 7 am until 4 pm, though I may arrive or stay later than this. When my door is open (which it usually is), you are welcome and encouraged to stop by to discuss any questions or problems that arise.

Academic Year

Please come by during my posted office hours to discuss any concerns. If you cannot attend a scheduled office hour, you may also email me to schedule an appointment; please include 3-4 possible meeting times so that I can pick one that works for me.

Email is also a reliable way to contact me, but please allow 24 hours for a response (except on weekends, when I often do not regularly read email). You may also call me in my office (x9812).

9 Accommodations

9.1 For You

If you have any disability that requires accommodations, please meet with me right away so that we can work together to find accommodations that meet your learning needs. You will also need to provide documentation of your disability to the Dean for Student Academic Support and Advising, Joyce Stern, located on the 3rd floor of the Rosenfield Center (x3702).

9.2 For Me

Please also note that I too require a form of accommodation and may need to ask for your polite consideration.

The chemical fragrances found in lotions, perfume, cologne, after shave, body sprays, scented laundry products, etc. make many people who suffer with asthma, allergies, environmental sensitivities, cancer, and migraines much sicker. While I look forward to seeing you, if you are wearing any scented products, please plan to visit me another time.

I am sensitive to many chemicals you may not even notice, so thank you for understanding if I need to ask you to make alternative arrangements. Please do not take it personally.

Acknowledgements: With thanks to Ben DeRidder for inspiration of elements in the Overview section, and Janet Davis's MAP Student for Today Message Example 2.