We introduce three "Cayley-Klein" families of Lie algebras through
realizations in terms of either real, complex or quaternionic matrices. Each
family includes simple as well as some limiting quasi-simple real Lie algebras.
Their relationships naturally lead to an infinite family of $3\times 3$
Freudenthal-like magic squares, which relate algebras in the three CK families.
In the lowest dimensional cases suitable extensions involving octonions are
possible, and for $N=1, 2$, the "classical" $3\times 3$ Freudenthal-like
squares admit a $4\times 4$ extension, which gives the original Freudenthal
square and the Sudbery square.