A vector space is a collection of objects that can be combined in two ways.
Operation 1 takes two objects, and gives another object.
Operation 2 takes one object, and one number, and returns another object.
That’s about it.
We’re free to define operation 1 and operation 2 as we wish, and what is part of a collection and what isn’t is also our wish. All we need to ensure is that doing operation 1 and operation 2 always gives us another object that is also part of the collection, and that the operations follow some consistency rules (associativity, identity, distributivity)
One can’t do much with these basic primitives, you might imagine. Well, so did I. But a subset of humans known as Mathematicians are very creative, and they’ve found weird and wonderful games we can play just with these rules.
The first question one can ask is - what is the smallest number of “seed” objects from which I can make all the other objects in the collection by doing a sequence of operations. This is the dimension of the vector space.
If we imagine our collection as a graph, with each object as a node, then a operation is an edge which takes us between the objects in the graph. Note that there are usually an infinite number of objects in a graph, even if we didn’t put them there, because numbers are infinite. So unless we specifically arrange for multiple numbers to land to the same object, usually we’ll end up with an infinite number of objects very quickly just by application of the basic two operations.
However, the dimension of the vector space is usually finite. In the games Physicists play they play with infinite vector spaces too, but the games that we’ll be playing - teaching machines how to learn, we’ll usually deal with finite vector spaces, they’re going to be more than enough clay for us to make many a Frankensteins.
Graphs is an okay metaphor to start, but it incorrectly overemphasizes discreteness.
A better metaphor is paints.
We can also now name our operations - operation 1 is called addition, and operation 2 is called scaling. But remember, they are called addition or scaling, but they can be any operation that follows the consistency rules and closure.
So with the paint metaphor, we can think of starting with our seed paints - the dimension of our space. Then we can take an arbitrary amount of each paint (by scaling it with a number) and add two paints (by mixing them) and thus can smoothly vary across a continuous space of possible colors.
Even this metaphor breaks. For example, we can’t have a negative amount of a paint, but numbers can be negative.
Notice how geometry has not entered the picture (sic). In fact, we can even define coordinates without needing geometry.
First, decide on a set of paints that will serve as our seed. These are called the basis vectors, and together they form a basis.
Remember, a requirement of this basis is that the paints in it should be the smallest possible set (that is, no two paints in the basis should be expressible as a combination of some other paints in the basis) that should allow us to reach all the possible paints in our universe (i.e., the “span” of the basis should cover the entire collection)
By definition, we can then express any paint in our universe as a combination of these basis paints.
Because of the various consistency rules we added to our operations, we can mix and match in any order, in any arbitrary sequence, but at the end the operations can be rearranged so that they all work like
- Pick a basis paint
- Select the amount of it to use (possibly zero)
- Add it to the mixture
- Do this for all the basis paints
Or to put it more briefly, every color in our universe is expressible using some mix of basis paints. The amount of basis paints we need are the “coordinates” of our specific paint.