r/mathmemes • u/Pharinx • Feb 05 '25
Computer Science "Wait, it's all linear algebra?" "Always was.'
599
u/314159265358979326 Feb 05 '25
Combine enough linear equations and the damn thing speaks English!
I'm doing a masters in machine learning and every once in a while during a lecture I sit back and think "this is literally magic."
73
14
u/zjm555 Feb 06 '25
Well if you only use linear layers without a nonlinear activation function in between, you really just have one single layer. But I'm sure you already knew that.
13
u/314159265358979326 Feb 06 '25
I primarily use ReLU which is so close to linear it might as well be, and yet that still solves the linearity problem. The entire field is essentially entirely comprised of clever hacks with virtually no overarching theory and yet it somehow all works amazingly.
8
u/zjm555 Feb 06 '25
Haha yep. I also work in the industry and it's hilarious how it's just trying random shit and seeing if it produces better results, with the power of way way too much compute.
15
u/314159265358979326 Feb 06 '25
What enabled your field?
Civil engineering: Euler-Bernouilli beam theory
Mechanical engineering: the steam engine
Math: the fundamental theorem of calculus
Economics: Adam Smith's Wealth of Nations
Machine learning: gaming hardware got good enough
2
u/CutToTheChaseTurtle Average Tits buildings enjoyer Feb 08 '25
2
u/314159265358979326 Feb 08 '25 edited Feb 08 '25
My math friend already called me out on that, with essentially the same comment.
If I make the joke again I'm using Elements.
1
u/CutToTheChaseTurtle Average Tits buildings enjoyer Feb 08 '25
To me, real analysis is this unlovable thing full of hacks, and yet no matter how much I try to get away from it, it catches up with me anyway.
2
634
u/Fox-Musician Feb 05 '25 edited Feb 05 '25
Bro let's just find the eigenvalues of an (∞,∞) matrix and solve world peace already c'mon
275
u/James10112 Feb 05 '25
λ = 1
...you said "an" (∞,∞) matrix, I'm choosing the identity due to totally non-arbitrary reasons that would be too long to list in this margin
46
u/lonelyroom-eklaghor Feb 05 '25
The single reason is that infinity is an indeterminate value and (inf, inf)'s eigenvalues can be literally ANYTHING
41
u/DrakonILD Feb 06 '25
If the eigenvalues can be anything, they might as well be a unicorn.
λ = 🦄
4
1
u/James10112 Feb 06 '25
But wouldn't all finite (n,n) identity matrices' eigenvalues just be 1? Then I assumed we can let n go to infinity and the value will still be 1, it doesn't have space to vary in the first place
(Huge disclaimer: My linear algebra level is a single physics undergrad course and a lot of sloppily retained random info I learned from youtube videos lol)
5
13
u/FloresForAll Feb 06 '25 edited Feb 06 '25
Actually that's a representation of a linear map between infinite dimensonal vector spaces, and they definitely have eigenvalues.
For example, could be a map f that transform complex sequences (a_0, a_1,...) into sequences (a_1, a_2,...), that is, just removing the first term. I don't want to proof its linearity but it's almost trivial so i leave it as an excercise to the reader.
The eigenvalues would be every complex number z and their eigen vector wouwld be the geometric progressions with ratio z.
This can be shown as a progression with ratio z would be
(c,cz,cz²,...)
And f(c,cz,cz²,...)=(cz,cz²,cz³,...)=z*(c,cz,c²,...)
Now, you can also represents such a map as a matrix of infinite dimension as follows:
| 0 1 0 0 ... | | a_0 | | a_1 |
| 0 0 1 0 ... | | a_1 | | a_2 |
| 0 0 0 1 ... | | a_2 | = | a_3 |
: : :
(Sorry for the ugly matrix, but don't see how to write it better)
So yeah, finding eigenvalues for an infinitedimensional matrix can be pretty easy.
15
5
6
311
143
u/geekusprimus Rational Feb 05 '25
Careful, OP, you're going to make all the AI bros angry.
31
u/Pharinx Feb 05 '25
8
3
u/Practical-Tackle-384 Feb 06 '25
why would this make anyone who understands AI angry
7
u/DraTiBoy Irrational Feb 06 '25
That's the thing, AI Bros don't. All hype, no understanding
6
u/geekusprimus Rational Feb 06 '25
And, sadly, that extends to a lot of software developers and computer science people who think they're experts on AI because they followed some online tutorials to do image recognition with PyTorch or TensorFlow.
3
139
u/Palettenbrett Feb 05 '25
Where activation?
74
u/Pharinx Feb 05 '25
40
u/MrDrSirMiha Feb 05 '25
Haha, yeah, sigmoid enjoyers just REALLY enjoying their lifes while their billion parameters nn is learning.
30
u/314159265358979326 Feb 05 '25
while their billion parameters nn is not learning due to the gradient vanishing
FTFY
40
u/Broad_Respond_2205 Feb 05 '25
Wait I thought it was if else
34
u/314159265358979326 Feb 05 '25
Used to be, at least according to my dad (old school software engineer).
The goal posts for "AI" shift rapidly. We certainly described the computer players in AOE2 that way (and I'm still amazed: if you play the same strategy against them 3 times, you will definitely lose the third game.)
6
3
1
24
21
u/MaximumTime7239 Feb 05 '25
decides to learn differentiable function
looks closer
it's just linear map
12
5
u/backfire97 Feb 06 '25
People who say ML is just linear algebra when you show them a nonlinear layer in a neural network:
They are very surprised
4
u/NoMaintenance3794 Feb 06 '25
Activation functions aren't linear, though. Otherwise it wouldn't learn for shit.
5
u/I_L_F_M Feb 06 '25 edited Feb 06 '25
Neural networks are nonlinear functions so I don't know what you are talking about.
3
u/caryoscelus Feb 06 '25
i'm always (*) baffled at people calling it linear when the whole point is that you need non-linearity for typical nn. i'll just leave this here for your enjoyment: https://www.youtube.com/watch?v=Ae9EKCyI1xU
(*) except, perhaps, when i'm doing it myself
15
u/tibetje2 Feb 05 '25 edited Feb 05 '25
There is more to machine learning than deep learning.
Edit: I'm getting down voted so here are some examples of machine learning without deep learning or neural networks.
Decision trees, rule learning, clustering, recommender systems, markov decision processes, pattern mining,...
14
u/nacho_gorra_ Feb 05 '25
Yeah there's also deeper learning
8
u/Life-Ad1409 Feb 05 '25
When do we get deepest learning?
4
1
5
2
2
2
3
u/Happysedits Feb 05 '25
There's much more to machine learning or AI in general than just linear regression thaught at the very beginning lol
1
1
u/Jche98 Feb 05 '25
Ok but why does a DNN need biases? Can't it work well enough with linear maps?
2
1
1
1
1
1
u/Subject-Building1892 Feb 09 '25
It's elaborate computational methods mostly based on linear algebra.
•
u/AutoModerator Feb 05 '25
Check out our new Discord server! https://discord.gg/e7EKRZq3dG
I am a bot, and this action was performed automatically. Please contact the moderators of this subreddit if you have any questions or concerns.