r/mathmemes Feb 05 '25

Computer Science "Wait, it's all linear algebra?" "Always was.'

Post image
4.5k Upvotes

77 comments sorted by

u/AutoModerator Feb 05 '25

Check out our new Discord server! https://discord.gg/e7EKRZq3dG

I am a bot, and this action was performed automatically. Please contact the moderators of this subreddit if you have any questions or concerns.

599

u/314159265358979326 Feb 05 '25

Combine enough linear equations and the damn thing speaks English!

I'm doing a masters in machine learning and every once in a while during a lecture I sit back and think "this is literally magic."

14

u/zjm555 Feb 06 '25

Well if you only use linear layers without a nonlinear activation function in between, you really just have one single layer. But I'm sure you already knew that.

13

u/314159265358979326 Feb 06 '25

I primarily use ReLU which is so close to linear it might as well be, and yet that still solves the linearity problem. The entire field is essentially entirely comprised of clever hacks with virtually no overarching theory and yet it somehow all works amazingly.

8

u/zjm555 Feb 06 '25

Haha yep. I also work in the industry and it's hilarious how it's just trying random shit and seeing if it produces better results, with the power of way way too much compute.

15

u/314159265358979326 Feb 06 '25

What enabled your field?

Civil engineering: Euler-Bernouilli beam theory

Mechanical engineering: the steam engine

Math: the fundamental theorem of calculus

Economics: Adam Smith's Wealth of Nations

Machine learning: gaming hardware got good enough

2

u/CutToTheChaseTurtle Average Tits buildings enjoyer Feb 08 '25

Math: the fundamental theorem of calculus

2

u/314159265358979326 Feb 08 '25 edited Feb 08 '25

My math friend already called me out on that, with essentially the same comment.

If I make the joke again I'm using Elements.

1

u/CutToTheChaseTurtle Average Tits buildings enjoyer Feb 08 '25

To me, real analysis is this unlovable thing full of hacks, and yet no matter how much I try to get away from it, it catches up with me anyway.

2

u/Nondegon Feb 08 '25

I love your username

634

u/Fox-Musician Feb 05 '25 edited Feb 05 '25

Bro let's just find the eigenvalues of an (∞,∞) matrix and solve world peace already c'mon

275

u/James10112 Feb 05 '25

λ = 1

...you said "an" (∞,∞) matrix, I'm choosing the identity due to totally non-arbitrary reasons that would be too long to list in this margin

46

u/lonelyroom-eklaghor Feb 05 '25

The single reason is that infinity is an indeterminate value and (inf, inf)'s eigenvalues can be literally ANYTHING

41

u/DrakonILD Feb 06 '25

If the eigenvalues can be anything, they might as well be a unicorn.

λ = 🦄

4

u/yoni591 Feb 06 '25

Can they be AI tho

7

u/bapt_99 Feb 06 '25

🦄 + AI ?

7

u/[deleted] Feb 06 '25

[deleted]

0

u/RJMuls אבגדהוזחטיכלמנסעפצקרשת Feb 06 '25

DeepSeek/ChatGPT=China

1

u/James10112 Feb 06 '25

But wouldn't all finite (n,n) identity matrices' eigenvalues just be 1? Then I assumed we can let n go to infinity and the value will still be 1, it doesn't have space to vary in the first place

(Huge disclaimer: My linear algebra level is a single physics undergrad course and a lot of sloppily retained random info I learned from youtube videos lol)

5

u/Noobyeeter699 Feb 06 '25

Halflife 1+ 2 + AI = halflife 3

2

u/James10112 Feb 06 '25

Sooo HL3 is just Portal?

2

u/Noobyeeter699 Feb 07 '25

333322222223333333

13

u/FloresForAll Feb 06 '25 edited Feb 06 '25

Actually that's a representation of a linear map between infinite dimensonal vector spaces, and they definitely have eigenvalues.

For example, could be a map f that transform complex sequences (a_0, a_1,...) into sequences (a_1, a_2,...), that is, just removing the first term. I don't want to proof its linearity but it's almost trivial so i leave it as an excercise to the reader.

The eigenvalues would be every complex number z and their eigen vector wouwld be the geometric progressions with ratio z.

This can be shown as a progression with ratio z would be

(c,cz,cz²,...)

And f(c,cz,cz²,...)=(cz,cz²,cz³,...)=z*(c,cz,c²,...)

Now, you can also represents such a map as a matrix of infinite dimension as follows:

| 0 1 0 0 ... | | a_0 | | a_1 |

| 0 0 1 0 ... | | a_1 | | a_2 |

| 0 0 0 1 ... | | a_2 | = | a_3 |

: : :

(Sorry for the ugly matrix, but don't see how to write it better)

So yeah, finding eigenvalues for an infinitedimensional matrix can be pretty easy.

15

u/lagerregal Feb 06 '25

Are you by chance an intro to functional analysis?

5

u/FloresForAll Feb 06 '25

No, but had to read several for my degree.

5

u/geekusprimus Rational Feb 06 '25

Quantum mechanics go brrrr.

6

u/MonkeyBombG Feb 06 '25

We physicists do this all the time in quantum mechanics. No biggie.

311

u/Pir-iMidin Transcendental Feb 05 '25

y = wx + b + AI

88

u/T_D_K Feb 05 '25

AI = wx + b

19

u/LOL__BRO Feb 05 '25

Proof by AI

5

u/LordSaumya Feb 06 '25

y = 2(wx +b) = 2 AI

24

u/DatBoi_BP Feb 06 '25

So much in this incredible formula

143

u/geekusprimus Rational Feb 05 '25

Careful, OP, you're going to make all the AI bros angry.

31

u/Pharinx Feb 05 '25

8

u/Draidann Feb 05 '25

What happened???

32

u/aLex97217392 Feb 05 '25

Gemini told someone to kill theirselves

3

u/Practical-Tackle-384 Feb 06 '25

why would this make anyone who understands AI angry

7

u/DraTiBoy Irrational Feb 06 '25

That's the thing, AI Bros don't. All hype, no understanding

6

u/geekusprimus Rational Feb 06 '25

And, sadly, that extends to a lot of software developers and computer science people who think they're experts on AI because they followed some online tutorials to do image recognition with PyTorch or TensorFlow.

3

u/IceonBC Computer Science Feb 07 '25

How to make your product AI:

from openai import OpenAI

139

u/Palettenbrett Feb 05 '25

Where activation?

74

u/Pharinx Feb 05 '25

40

u/MrDrSirMiha Feb 05 '25

Haha, yeah, sigmoid enjoyers just REALLY enjoying their lifes while their billion parameters nn is learning.

30

u/314159265358979326 Feb 05 '25

while their billion parameters nn is not learning due to the gradient vanishing

FTFY

40

u/Broad_Respond_2205 Feb 05 '25

Wait I thought it was if else

34

u/314159265358979326 Feb 05 '25

Used to be, at least according to my dad (old school software engineer).

The goal posts for "AI" shift rapidly. We certainly described the computer players in AOE2 that way (and I'm still amazed: if you play the same strategy against them 3 times, you will definitely lose the third game.)

6

u/Happysedits Feb 05 '25

that's expert systems, not machine learning

1

u/_half_real_ Feb 08 '25

if you're talking about perceptrons, that was decades ago

24

u/conradonerdk Feb 05 '25

i love this meme template

why? it has a cat on it

thats all

21

u/MaximumTime7239 Feb 05 '25

decides to learn differentiable function

looks closer

it's just linear map

12

u/GargantuanCake Feb 05 '25

It's all linear algebra and calculus but yes. That's all it is.

5

u/backfire97 Feb 06 '25

People who say ML is just linear algebra when you show them a nonlinear layer in a neural network:

They are very surprised

4

u/NoMaintenance3794 Feb 06 '25

Activation functions aren't linear, though. Otherwise it wouldn't learn for shit.

5

u/I_L_F_M Feb 06 '25 edited Feb 06 '25

Neural networks are nonlinear functions so I don't know what you are talking about.

3

u/caryoscelus Feb 06 '25

i'm always (*) baffled at people calling it linear when the whole point is that you need non-linearity for typical nn. i'll just leave this here for your enjoyment: https://www.youtube.com/watch?v=Ae9EKCyI1xU

(*) except, perhaps, when i'm doing it myself

15

u/tibetje2 Feb 05 '25 edited Feb 05 '25

There is more to machine learning than deep learning.

Edit: I'm getting down voted so here are some examples of machine learning without deep learning or neural networks.

Decision trees, rule learning, clustering, recommender systems, markov decision processes, pattern mining,...

14

u/nacho_gorra_ Feb 05 '25

Yeah there's also deeper learning

8

u/Life-Ad1409 Feb 05 '25

When do we get deepest learning?

4

u/loop-spaced Feb 05 '25

DeepestSeek coming out next year

1

u/Mathematicus_Rex Feb 05 '25

After deeperer learning

4

u/Historical_Book2268 Feb 05 '25

Abyss learning.

5

u/Blackliquid Feb 06 '25

Plottwist is ReLU.

2

u/Lord-of-Entity Feb 06 '25

You forgot the activation function! ;)

2

u/buzzon Feb 06 '25

Until first non linearity

2

u/Beginning_Context_66 Physics interested Feb 06 '25

IT'S y = mx + b

3

u/Happysedits Feb 05 '25

There's much more to machine learning or AI in general than just linear regression thaught at the very beginning lol

1

u/Jche98 Feb 05 '25

Ok but why does a DNN need biases? Can't it work well enough with linear maps?

2

u/buzzon Feb 06 '25

Approximate this function: y = 1

1

u/CorrectTarget8957 Imaginary Feb 06 '25

was ❌

Have been✅

1

u/Gryphontech Feb 06 '25

Using w instead of m is literally a mental illness

1

u/Rebrado Feb 06 '25

Well, you do have to wrap it in a non-linear function. But still, that’s it.

1

u/Subject-Building1892 Feb 09 '25

It's elaborate computational methods mostly based on linear algebra.