Image Processing

In this all-inclusive module students will learn how to use matrices and vectors to manipulate and edit images. The students will be provided with their own image editing activity where they will create their own classroom “Instagram feed.” There is also an extension activity that incorporates some basic Python programming into this module. The authors would like to acknowledge the support of the National Science Foundation through the grant DMS-1845406

on vectors; DCS.N.1.3 Implement procedures to find the inverse of a matrix; DCS.N.2.1 Organize data into matrices to solve problems; DCS.N.2.2 Interpret solutions found using matrix operations in context; DCS.N.2.3 Represent a system of equations as a matrix equation; DCS.N.2.4 Use inverse matrices to solve a system of equations with technology.
v Assessments: Ø Feel free to select portions of the guided notes to serve as out-of-class activities. Ø Any problem set contained within guided notes could be given as homework assignments. Ø As an alternative to the final project in this module, you could choose to give students a standard test or quiz on the skills that have been learned.
v Online delivery suggestions: Ø For asynchronous online delivery, create instructional videos to take students through the guided notes. Ø For synchronous online delivery, display the guided notes on your screen and take students through the activities while you annotate on your screen (or writing on paper and using a document camera). Ø Share all prepared documents through a learning management system so that students would have access to them at home Ø For the project, you can have the students create the matrices using Microsoft word (or if they have access to a tablet and a note taking app, they can create the matrices on there), they can use this website to create the pictures, and then they can screen snip the pictures out and paste them into the final document. They can then submit the project fully through an online submission. Students read the opening problem (e.g., Textbook Problem or other context of interest) from a handout and/or projected on a screen. In groups of 2-3, students talk briefly about how they would answer the question from the teacher. (~5 minutes) The teacher brings back students to share out with the class. (~3 minutes) Students are provided guided notes to document new terms (e.g., matrix, dimension, row, column, etc.) They will complete the notes through the discussion conducted by the teacher. (~5 minutes) Teacher hands out a sheet of paper with the opening problem written on it and/or projects it on the screen. Teacher opens with the question, "How might we organize this information in a way that allows us to answer questions about the university's inventory?" After students discuss, the teacher solicits students' responses.
If students do not suggest a matrix, the teacher will introduce the name and ask students if they are familiar with the term. If not, the teacher will define it through one of the matrices used to organize the information in the problem. Teacher circulates the room to observe/monitor students' work.

Explore I
Teacher then poses question: "How could we use these matrices to determine the total inventory of books at the university?" Once students have some time to answer question, teacher returns to full class discussion to ask how we could define matrix addition. Teacher then poses question: "How could we use these matrices to determine the inventory of books at the university if the librarian would like to double the inventory?" Once students have some time to answer question, teacher returns to full class discussion to ask how we could define scalar multiplication..

Explain II
Personalize/Differentiate as needed Adjust along teacher/student centered continuum Provide vocabulary Clarify understandings ~5 min Students offer their ideas on how to define scalar multiplication.
Teacher conducts discussion on scalar multiplication.

Extend
Apply knowledge to new scenarios Continue to personalize as needed Consider grouping homogeneously ~15 min Students complete class problem set in groups of 2-3 to apply their new knowledge.
Teacher circulates the room and observes/monitors students' work.

Evaluate Formative Assessment
How will you know if students understand throughout the lesson?
N/A Students will complete guided notes and a problem set for practice. Students turn in their solutions to the last problem in the problem set as an exit ticket (e.g., Stereo Problem) Teacher will review students' work as they circulate room and monitor progress, engaging students who may be having difficulty in discussion to probe their thinking.

Summative-like Assessment
Questions similar to final assessment

Matrix Addition, Subtraction and Scalar Multiplication
A university is taking inventory of the books they carry at their two biggest bookstores.
Definitions: A matrix is a rectangular table of entries and is used to organize data in a way that can be used to solve problems. The following is a list of terms used to describe matrices: • A matrix's size (or dimension) is written by listing the number of rows "by" the number of columns.
• The values in a matrix, A, are referred to as entries or elements. The entry in the "m th " row and "n th " column is written as !" .
• A matrix is square if it has the same number of rows as it has columns.
• If a matrix has only one row, then it is a row vector. If it has only one column, then the matrix is a column vector.
• The transpose of a matrix, A, written A T , switches the rows with the columns of A and the columns with the rows.
• Two matrices are equal if they have the same size and the same corresponding entries.

Adding and Subtracting Matrices
In order to add or subtract matrices, they must first be of the same size. The result of the addition or subtraction is a matrix of the same size as the matrices themselves, and the entries are obtained by adding or subtracting the elements in corresponding positions.
In our campus bookstores example, we can find the total inventory between the two bookstores as follows: Matrix addition is commutative. This is because the operation is based in the addition of real numbers, as the entries of each matrix are added to their corresponding entries in the other matrix/matrices. Since addition of real numbers is commutative, so is matrix addition.
Question: Is matrix subtraction commutative ( . . , − = − )? Why or why not? Matrix subtraction is not commutative. This is because the operation is based in the subtraction of real numbers, as the entries of each matrix are subtracted from their corresponding entries in the other matrix/matrices. Since subtraction of real numbers is not commutative, neither is matrix subtraction.
Question: Is matrix addition associative ( . . , ( + ) + = + ( + ))? Why or why not? Matrix addition is associative. This is because the operation is based in the addition of real numbers, as the entries of each matrix are added to their corresponding entries in the other matrix/matrices.
Since addition of real numbers is associative, so is matrix addition.
Question: Is matrix subtraction associative ( . . , ( − ) − = − ( − ))? Why or why not? Matrix subtraction is not associative. This is because the operation is based in the subtraction of real numbers, as the entries of each matrix are subtracted from their corresponding entries in the other matrix/matrices. Since subtraction of real numbers is not associative, neither is matrix subtraction.

Scalar Multiplication
Multiplying a matrix by a constant (or scalar) is as simple as multiplying each entry by that number!   Students complete class problem set in groups of 2-3 to apply their new knowledge.
Teacher will review students' work as they circulate room and monitor progress, engaging students who may be having difficulty in discussion to probe their thinking.
Teacher brings class back together to engage in debrief on the problem set.

Evaluate Assessment
How will you know if students understand throughout the lesson?

~10 min
Students work on exit ticket problem and turn it in.
Teacher poses exit ticket problem for students to turn in.

Matrix Multiplication
The Metropolitan Opera is planning its last cross-country tour. It plans to perform Carmen and La Use multiplication and addition to find the total cost for each troupe to travel each of the airlines. It turns out that we can solve problems like these using a matrix operation, specifically matrix multiplication!
We first note that matrix multiplication is only defined for matrices of certain sizes. For the product of matrices and , where is an matrix, must have the same number of rows as has columns. So, must have size . The product will have size .

Exercises
The following is a set of abstract matrices (without row and column labels): List at least 5 orders of pairs of matrices from this set for which the product is defined. State the dimension of each product. The current yields on these investments are 7.5% for bonds, 11.25% for mortgages, and 6% for consumer loans. Use matrix multiplication to find the total earnings for each state.

Total earnings for each state (in millions of dollars):
.
2. Several years ago, Ms. Allen invested in growth stocks, which she hoped would increase in value over time. Calculate the total value of Ms. Allen's stocks at the end of each year. b. During August and September, production is increased by 40%. How many Budget, Economy, Executive, and President models are produced daily during these months?

Total value of the stocks (in dollars) at the end of each year:
c. It takes 5 person-hours to produce the Budget model, 7 person-hours to produce the Economy model, 6 person-hours for the Executive model, and 7 person-hours for the President model. Determine the number of employees needed to maintain the normal production schedule, assuming everyone works an average of 7 hours each day. How many employees are needed in August and September?
Define the matrices for the inventory parts ( ) and the daily manufacturing goal ( ) as j k The number of labor hours needed per week is given by = With 7-hour workdays, the number of employees needed is = . , which implies that 36 employees are needed to maintain full production. For August and September, we want .
4. The president of the Lucrative Bank is hoping for a 21% increase in checking accounts, a 35% increase in savings accounts, and a 52% increase in market accounts. The current statistics on the number of accounts at each branch are as follows: What is the goal for each branch in each type of account? (HINT: multiply by a 3 × 2 matrix with certain nonzero entries on the diagonal and zero entries elsewhere.) What will be the total number of accounts at each branch?
The goal for each branch in each type of account is given by: . . Note: this answer can also be obtained by just adding up the entries in each row of the previous matrix. Students work in groups of 2-3 through guided notes provided by the teacher. The teacher will work through the first example with the students and then give them time to work the rest.
The guided notes would include both problems on drawing images based on matrices and vice versa.
Teacher demonstrates how to draw an image based on a matrix with the students (displaying the first matrix on the students' guided notes on the screen for students to follow.) Teacher circulates the room monitoring students' progress and addressing questions along the way. The teacher brings students back to share and confirm their results. Students work in groups of 2-3 through guided notes provided by the teacher. The teacher will work through the first example with the students and then give them time to work the rest.

Explain Ia
Teacher demonstrates how to convert matrices into vectors (displaying the first matrix on the students' guided notes on the screen for students to follow.) Teacher circulates the room monitoring students' progress and addressing questions along the way.
The teacher brings students back to share and confirm their results. Students apply their new understanding to a problem set involving specific images.

Explain II
Teacher will review students' work as they circulate room and monitor progress, engaging students who may be having difficulty in discussion to probe their thinking.
Teacher brings class back together to engage in debrief on the problem set.

Evaluate Formative Assessment
How will you know if students understand throughout the lesson?

~10 min
Students work on exit ticket problem and turn it in.
Teacher poses exit ticket problem for students to turn in.

Summative-like Assessment
Questions In this lesson, we will discuss representations of images and methods to manipulate those images.

Figure:
Image of a raccoon in grayscale, binary and copper coloring

Matrix representations:
• A grayscale image is a 2-dimensional array of numbers. An 8-bit image has entries between 0 and 255.
• The value 255 represents a white color, and the value 0 represents a black color.
• Lower numbers translate to darker pixels, while higher numbers translate to lighter pixels.
• For an image that has pixels (i.e., "picture elements"), we represent that image using a matrix of size . The entries of the matrix indicate the pixel value of the corresponding part of the image.
Example: This table represents an image that has 4 5 pixels.
Example: This 4 4 image of the raccoon is represented by the matrix on the right below: Example: Write the matrix that represents the following image: Color images can be stored in a similar fashion to a grayscale image. Instead of one number (0-255) per pixel, one stores three numbers per pixel -these three numbers denote the "amount" of red, "amount" of green, and "amount" of blue in each pixel. These three numbers can be used to depict a wide range of colors.

Image transformations:
Once images are represented as matrices, we can describe many transformations of those images using basic matrix operations! Example: Suppose we have two separate images that can be represented by the matrices = 9 255 0 0 255 @ and = 9 0 0 255 0 @.
1. Draw the images that correspond to and .

Exercises:
1. a. Describe an image whose matrix representation contains entries of all zeros.
The image would be entirely black.
b. Describe an image whose matrix representation contains entries of all 255.
The image would be entirely white.

Reflecting/rotating images
Let us consider how we might reflect an image over the x or y-axis. Here's an example! How would we represent these reflections using matrices?
From our earlier work, we know that the image represented by the matrix = 9 255 0 0 255 @ would be: If we were to reflect this image about the x-axis, we would obtain: The matrix ′ that represents this image is = 9 0 255 255 0 @. We can generalize this result!

Exercises:
1. Which matrix represents the image that is the reflection of the matrix = U
Question: From our work above, we know that the matrix representing the image reflected about the -axis is 2 2 4 1 3 <. What matrix would we need to multiply by on the left in order to obtain the matrix of the image reflected about the -axis? Show that your answer gives you the correct result.
2 0 1 1 0 < 2 0 1 1 0 < 2 1 3 2 4 < = 2 2 4 1 3 < Question: From our work above, we know that the matrix representing the image reflected about the -axis is 2 3 1 4 2 <. What matrix would we need to multiply by on the right in order to obtain the matrix of the image reflected about the -axis? Show that your answer gives you the correct result.
2 0 1 1 0 < 2 1 3 2 4 < 2 0 1 1 0 < = 2 3 1 4 2 < The matrix you used above is a form of a permutation matrix (here referred to as a reverse identity matrix). This matrix is always square, and its size is determined by the size of the matrix it is multiplied to (i.e. in order for the matrix multiplication to be performed).

Exercises:
1. What size does the reverse identity matrix need to be to be used to reflect an image represented by a 5 3 matrix about the -axis? Why?
The matrix must be a matrix, so that it can left-multiply the matrix. The reverse identity matrix is square, and it must have the same number of columns as the matrix has rows.
2. What size does the reverse identity matrix need to be to be used to reflect an image represented by a 5 3 matrix about the -axis? Why?
The matrix must be a matrix, so that it can right-multiply the matrix. The reverse identity matrix is square, and it must have the same number of rows as the matrix has columns.
3. Use the reverse identity matrix of the appropriate size to reflect the images represented by the following matrices about the -axis.

Reflecting images using matrix-vector multiplication
We can also manipulate images using matrix-vector multiplication. This method is a bit more powerful than matrix-matrix multiplication, because it allows us with more flexibility in ways you can manipulate those images.
Using this method, we first convert our matrices into vectors (i.e., single column or single row matrices).

Blurring images
We can also blur images to get an effect we're interested in, and we use linear algebra to do it! The image on the left below is the picture, and the image on the right is the blurred version.
One way to do this is to average each entry's "neighbors" (including the entry itself). The definition of these neighbors can be determined by the user themselves. For example, a neighbor could be any entry in the matrix representation that is directly left, right, above and below the original entry, along with the entry itself.

Lesson Plan
Teacher: Subject: This will be a teacher led activity especially if students are not familiar with Python. If students have previous Python programming experience they could try the coding on their own.
Topic/Day: Using Python to Manipulate Photos Content Objective: Use Matrices to solve real world problems.
Materials Needed: Python notebook-nothing has to be downloaded but you will need to allow "colab" to run as an extension in Google Chrome to run the program.

Evaluate Formative Assessment
How will you know if students understand throughout the lesson?

Share their outcomes with their peers
Facilitate the students sharing their outcomes with their peers You can choose to do whole group presentations or partner groups and have them share with only one group at a time and do a rotation (if you do this, be sure to move around and listen to each groups presentation) Recommended: Hang the outcomes on the wall and create a classroom "Instagram feed" on the wall. You can also have the students give the photos likes/comments and choose a classroom influencer based on the likes/comments that each photo receives.

Matrix Addition, Subtraction and Scalar Multiplication
A university is taking inventory of the books they carry at their two biggest bookstores.
Definitions: A __________ is a rectangular table of entries and is used to organize data in a way that can be used to solve problems. The following is a list of terms used to describe matrices: • A matrix's ___________________________ is written by listing the number of rows "by" the number of columns.
• The values in a matrix, A, are referred to as ______________ or ______________. The entry in the "m th " row and "n th " column is written as !" .
• A matrix is ______________ if it has the same number of rows as it has columns.
• If a matrix has only one row, then it is a row ____________. If it has only one column, then the matrix is a column ______________.
• The __________________ of a matrix, A, written A T , switches the rows with the columns of A and the columns with the rows.
• Two matrices are _____________ if they have the same size and the same corresponding entries.
The inventory of the books at the East Campus bookstore can be represented with the following 2 x 4 matrix: Similarly, the West Campus bookstore's inventory can be represented with the following matrix: = " …

Adding and Subtracting Matrices
In order to add or subtract matrices, they must first be of the same ______________________.
The result of the addition or subtraction is a matrix of the same size as the matrices themselves, and the entries are obtained by adding or subtracting the elements in corresponding positions.
In our campus bookstores example, we can find the total inventory between the two bookstores as follows:

Matrix Multiplication
The Metropolitan Opera is planning its last cross-country tour. It plans to perform Carmen and La Use multiplication and addition to find the total cost for each troupe to travel each of the airlines.
It turns out that we can solve problems like these using a matrix operation, specifically matrix multiplication!
We first note that matrix multiplication is only defined for matrices of certain sizes. For the product of matrices and , where is an matrix, must have the same number of rows as has columns. So, must have size ______ . The product will have size ________________.

Exercises
The following is a set of abstract matrices (without row and column labels): The current yields on these investments are 7.5% for bonds, 11.25% for mortgages, and 6% for consumer loans. Use matrix multiplication to find the total earnings for each state.
4. Several years ago, Ms. Allen invested in growth stocks, which she hoped would increase in value over time. She bought 100 shares of stock A, 200 shares of stock B, and 150 shares of stock C. At the end of each year she records the value of each stock. The table below shows the price per share (in dollars) of stocks A, B, and C at the end of the years 1984, 1985, and 1986.
Calculate the total value of Ms. Allen's stocks at the end of each year.

Figure:
Image of a raccoon in grayscale, binary and copper coloring

Matrix representations:
• A grayscale image is a 2-dimensional array of numbers. An 8-bit image has entries between ______ and _______.
• The value ______ represents a white color, and the value ______ represents a black color.
• Lower numbers translate to _____________ pixels, while higher numbers translate to _____________ pixels.
• For an image that has pixels (i.e., "picture elements"), we represent that image using a matrix of size _____________. The entries of the matrix indicate the pixel value of the corresponding part of the image.
Example: This table represents an image that has 4 5 pixels.
Example: This 4 4 image of the raccoon is represented by the matrix on the right below:

! "
Exercises: 1. Write a matrix that represents each of the following images. c. d.
2. Draw a visual representation of the following matrices. Color images can be stored in a similar fashion to a grayscale image. Instead of one number (0-255) per pixel, one stores three numbers per pixel -these three numbers denote the "amount" of red, "amount" of green, and "amount" of blue in each pixel. These three numbers can be used to depict a wide range of colors.

Image transformations:
Once images are represented as matrices, we can describe many transformations of those images using basic matrix operations!  If we were to reflect this image about the x-axis, we would obtain: The matrix ′ that represents this image is = 9 0 255 255 0 @. We can generalize this result!

Exercises:
1. Which matrix represents the image that is the reflection of the matrix = "
Question: From our work above, we know that the matrix representing the image reflected about the -axis is ! ". What matrix would we need to multiply by on the left in order to obtain the matrix of the image reflected about the -axis? Show that your answer gives you the correct result.
Question: From our work above, we know that the matrix representing the image reflected about the -axis is ! ". What matrix would we need to multiply by on the right in order to obtain the matrix of the image reflected about the -axis? Show that your answer gives you the correct result.
The matrix you used above is a form of a __________________________ matrix (here referred to as a reverse identity matrix). This matrix is always _____________, and its size is determined by the size of the matrix it is multiplied to (i.e. in order for the matrix multiplication to be performed).

Exercises:
5. What size does the reverse identity matrix need to be to be used to reflect an image represented by a 5 3 matrix about the -axis? Why?
6. What size does the reverse identity matrix need to be to be used to reflect an image represented

Reflecting images using matrix-vector multiplication
We can also manipulate images using matrix-vector multiplication. This method is a bit more powerful than matrix-matrix multiplication, because it allows us with more flexibility in ways you can manipulate those images.
Using this method, we first convert our matrices into vectors (i.e., single column or single row matrices). •

Exercises:
3. Convert each of the following matrices into vectors.
Question: From our earlier work, we know that the reflection of the image with matrix about the -axis would be = 2 2 4 1 3 <. So, ( ) = " …. What matrix would we need to multiply ( ) by on the left in order to obtain ( )? Show that your answer gives you the correct result.
Question: From our earlier work, we know that the reflection of the image with matrix about the -axis would be = 2 3 1 4 2 <. So, ( ) = " …. What matrix would we need to multiply ( ) by on the right in order to obtain ( )? Show that your answer gives you the correct result.

Blurring images
We can also blur images to get an effect we're interested in, and we use linear algebra to do it! The image on the left below is the picture, and the image on the right is the blurred version.
One way to do this is to average each entry's "neighbors" (including the entry itself). The definition of these neighbors can be determined by the user themselves. For example, a neighbor could be any entry in the matrix representation that is directly left, right, above and below the original entry, along with the entry itself.
Example: Consider an image whose matrix representation is = L 1 1 2 2 1 1 3 1 2 M. Suppose we define a "neighbor" as an entry that is directly above, below, next to or diagonal from that entry. For example, the entries that are neighbors of 1/ = 3 in the matrix are 2/ = 2, 22 = 1, 12 = 1, and 1/ = 3. So the entry 1/ in the new matrix would be

Project
You are going to edit your given image to be Instagram ready! You will need to use a combination of reflections, blurs, and brightness changes to post this image and get LOTS of likes and comments.
Step 1: • Come up with an Instagram handle (Note: all Instagram handles must be school appropriate and approved by the teacher before you can choose your image for step 2) Step 2: • Choose an image from your teacher to utilize throughout this project. First come first serve, so be sure to pick your image quick!
Step 3: • As a group, decide what types of reflections, inversions, and brightness changes you would like to complete to post your picture. You MUST use one of each type of edit!
Step 4: • Create matrices and a new image for each edit that you complete. You will need to turn this into your teacher at the end of the project.
Step 5: • Once your edits are complete, you must come up with a perfect caption for this photo (Note: all captions must be school appropriate).
o Remember, the better the caption, the more likely you are to receive more likes and comments!