March 12, 2025
There seems to be some uncertainty in the computer science community about exactly what a tensor is. This short note clarifies this by giving names to functions and composing them. The mathematical definitions can be translated directly into any computer language that allows functions to be defined and called.
The vector space \bm{R}^n is the set of tuples x = (x_1,\ldots,x_n) where x_i\in\bm{R}, 1\le i\le n. Scalar multiplication is defined by a(x_1,\ldots,x_n) = (ax_1,\ldots,ax_n) for a\in\bm{R}. Vector addition is defined by (x_1,\ldots,x_n) + (y_1,\ldots,y_n) = (x_1 + y_1,\ldots,x_n + y_n).
Definitions involving dots are not ammenable to computer implementation. Every vector x\in\bm{R}^n determines a function \bm{x}\colon\bm{n}\to\bm{R} by \bm{x}(i) = x_i for i\in\bm{n} = \{1,\ldots,n\}. Scalar multiplication and vector addition can be defined pointwise by (a\bm{x})(i) = a(\bm{x}(i)) and (\bm{x} + \bm{y})(i) = \bm{x}(i) + \bm{y}(i), 1\le i\le n.
The set of tuples \bm{R}^n is not the same as the set of functions \bm{R}^\bm{n} but they are in one-to-one correspondence. Define a map \iota\colon\bm{R}^n\to\bm{R}^\bm{n} by \iota x(i) = x_i, x\in\bm{R}^n, i\in\bm{n}.
Exercise. Show if \iota x = \iota y then x = y, x,y\in\bm{R}^n
Hint: Use x = y if and only if x_i = y_i for all i\in\bm{n}.
This shows \iota is one-to-one, or injective.
Exercise. Show for every \bm{x}\in\bm{R}^\bm{n} there exists x\in\bm{R}^n with \iota x = \bm{x}.
Hint: Given \bm{x}\in\bm{R}^\bm{n} let x_i = \bm{x}(i), i\in\bm{n}.
This show \iota is onto, or surjective. A function that is one-to-one and onto is a one-to-one correspondence, or bijective.
Exercise. Show \iota(ax) = a(\iota x) and \iota(x + y) = \iota x + \iota y for a\in\bm{R}, x,y\in\bm{R}^n.
This shows \iota is a linear operator, a function from one vector space to another that preserves the linear structure. If a linear operator is bijective we call it an isomorphism.
Every finite dimensional vector space over the real numbers can be identified with the set exponential \bm{R}^I of all functions from an index set I to \bm{R}. Elements of \bm{R}^I are called arrays.
A view of \bm{R}^I is a function \sigma\colon J\to I and is used to define a function \circ\sigma\colon\bm{R}^I\to\bm{R}^J by right composition (\circ\sigma)x = x\circ\sigma = x\sigma so (x\sigma)(j) = x(\sigma(j)), x\in\bm{R}^I, j\in J.
Exercise. Show \circ\sigma is a linear operator.
Hint: Show a(x\sigma) = (ax)\sigma and (x + y)\sigma = x\sigma + y\sigma.
Exercise. Show if \sigma is one-to-one then \circ\sigma is onto.
Hint: Show there exists \sigma^\dashv\colon I\to J with \sigma^\dashv\sigma = 1_J. If x\in\bm{R}^J then x\sigma^\dashv\in\bm{R}^I and x\sigma^\dashv\sigma = x.
Exercise. Show if \sigma is onto then \circ\sigma is one-to-one.
Hint: Show there exists \sigma^\vdash\colon J\to I with \sigma\sigma^\vdash = 1_I. If x\sigma = y\sigma, x,y\in\bm{R}^I, then x\sigma\sigma^\vdash = y\sigma\sigma^\vdash and x = y.
This shows if \sigma is one-to-one and onto then \circ\sigma is an isomorphism.
If J\subset I we write [J]\colon J\to I for the inclusion map and call x[J] the projection on J of x\in\bm{R}^I.
A matrix is an array where the index set is the cartesian product of two sets \bm{R}^{I\times J}.
A vector is usually represented on a computer as a contiguous array of memory with elements of the same type. The real numbers are uniquely characterized mathematically as a complete Archemedean ordered field. Computers can only model real numbers as a finite number of bits. The most common representation is 64-bit or 32-bit IEEE 754 floating point.