Scalars, Vectors, Matrices, and Tensors for Machine Learning
Summary
This tutorial introduces the notation and basic objects of linear algebra used in machine learning and quantitative finance. It defines scalars, vectors, matrices, and higher-order tensors, explains their dimensions and indexing, and gives examples such as feature vectors for financial instruments and matrices of neural network weights. Matrices are presented as linear maps that can transform vectors and encode geometric operations.
The article also explains why this mathematical vocabulary matters in practice: model parameters and features can be handled compactly, and vectorized operations can run efficiently on CPUs and GPUs. It previews matrix decompositions such as LU, QR, and singular value decomposition, noting their role in methods including least squares and principal component analysis. This is an introductory treatment focused on concepts and notation; it does not derive matrix operations or demonstrate a trading strategy. It also notes that mathematical quantities must be represented with finite precision in computing, creating possible overflow and underflow issues.
Key ideas
- Scalars, vectors, matrices, and higher-order tensors differ by the number of dimensions used to represent them.
- Vectors can encode feature values, while matrices organize data and represent linear transformations.
- Neural network inputs and weights can be expressed as vectors and matrices for compact computation.
- Vectorized linear algebra supports efficient parallel computation on modern hardware.
- Matrix decompositions can simplify computations and are used in methods such as least squares and principal component analysis.
Tags
This summary was written by Stratmill's research agent from the original; it is not a copy of the source.