\(\DeclarePairedDelimiterX{\Set}[2]{\{}{\}}{#1 \nonscript\;\delimsize\vert\nonscript\; #2}\) \( \DeclarePairedDelimiter{\set}{\{}{\}}\) \( \DeclarePairedDelimiter{\parens}{\left(}{\right)}\) \(\DeclarePairedDelimiterX{\innerproduct}[1]{\langle}{\rangle}{#1}\) \(\newcommand{\ip}[1]{\innerproduct{#1}}\) \(\newcommand{\bmat}[1]{\left[\hspace{2.0pt}\begin{matrix}#1\end{matrix}\hspace{2.0pt}\right]}\) \(\newcommand{\barray}[1]{\left[\hspace{2.0pt}\begin{matrix}#1\end{matrix}\hspace{2.0pt}\right]}\) \(\newcommand{\mat}[1]{\begin{matrix}#1\end{matrix}}\) \(\newcommand{\pmat}[1]{\begin{pmatrix}#1\end{pmatrix}}\) \(\newcommand{\mathword}[1]{\mathop{\textup{#1}}}\)
Vectors as Matrices
Real Linear Combinations
Needed by:
Matrix-Vector Products
Neural Networks
Real Matrix-Matrix Products
Sheet PDF
Graph PDF

Real Matrix-Vector Products


We explore matrix-vector multiplication.


Given a matrix $A \in \R ^{m \times n}$ and a vector $x \in \R ^{n}$, the product of $A$ with $x$ is the vector $y \in \R ^{m}$ defined by

\[ y_i = \sum_{j = 1}^{n} A_{ij} x_j, \quad i = 1, \dots , m. \]


We denote the product of $A$ with $x$ by $Ax$. With which we concisely write the system of linear equations $(A, b)$ as $b = Ax$.

This notation suggests both algebraic and geometric interpretations of solving systems of linear equations. The algebraic interpretation is that we are interested in the invertibility of the function $x \mapsto Ax$. In other words, we are interested in the existence of an inverse element of $A$. The geometric interpretation is that $A$ transforms the vector $x$.

Conversely, we can view $x$ as transforming (acting on) $A$. Let $a^j \in \R ^m$ denote the $j$th column of $A$, then

\[ Ax = \sum_{j = 1}^{n} x_j a^j. \]

In other words, $y$ is linear combination of the columns of $A$.


We call the function $f: \R ^n \to \R ^m$ defined by $f(x) = Ax$ the matrix multiplication function (or matrix-vector product function) associated with $A$. $f$ is satisfies the following two important properties:

  1. $A(x + y) = Ax + Ay$
  2. $A(\alpha x) = \alpha Ax$.
We call such a function $f$ linear. In other words, the matrix multiplication function is linear. Conversely, if $g: \R ^n \to \R ^m$ is linear, there exists a matrix inducing $g$.

Let $f: \R ^n \to \R ^m$ be linear. Then there exists a unique $A \in \R ^{m \times n}$ satisfying $f(x) = Ax$ for all $x \in \R ^n$.
Evaluate $f$ at the standard unit vectors $e_i$. The $i$th component of $e_i$ is 1 and all other components are 0.

Moreover, this matrix representation of $f$ is unique.

If $A, B \in \R ^{m \times n}$ are two matrices so that $f(x) = Ax = Bx$, then $A = B$.
We have $Ax - Bx = 0$ so $(A - B)x = 0$ for every $x$. In particular $y^\top (A - B)x = 0$ for every $x \in \R ^{n}, y \in \R ^m$. In particular, $e_{i}^\top (A - b)e_{j} = 0$. Conclusion: $A_{ij} - B_{ij} = 0$, and conclude that $A_{ij} = B_{ij}$. Thus, $A = B$. Evaluate $f$ at the standard unit vectors $e_i$. The $i$th component of $e_i$ is 1 and all other components are 0.
Copyright © 2023 The Bourbaki Authors — All rights reserved — Version 13a6779cc About Show the old page view