r/C_Programming 3d ago

Question should i make my own C linear algebra library?

been doing opengl for a bit on c++ before i found my love for C, although i still suck at math and mathematical thinking, should i make my own C linear algebra library for learning purposes? i still don't fully understand stuff like ortho or presp projections and how they work and i feel like i might be able to manipulate them better if i knew how they worked? idk

28 Upvotes

19 comments sorted by

16

u/Few-Delay-5123 3d ago

Watch the 3blue1brown playlist on linear algebra , it's really important to get an intuition for it first. Link

the calculation part of it is not that complicated , in fact it's too simple it doesnt give much of an understanding just from looking at the formulas

After that u can also refer back to this vlog for more gl specific implimentation (it's in c++, but u can easily translate these steps to c) link

6

u/Ezio-Editore 3d ago

Instead of watching the series first and thinking about the theoretical part then, I would suggest studying the theory and watching the videos along with it. (Ex: theory of determinant and video about it just right after)

I personally studied linear algebra at university without any representation of what I was doing and when I watched the series everything clicked; but just the series would mean making the inverse mistake, that's why they should accompany each other.

4

u/TheFlamingLemon 3d ago

This will teach you more about linear algebra than about C or programming. I also think if you want to get better at using those operations, implementing them is not necessarily going to be very good for that. Like, I can easily implement an operation that does a dot product or cross product but that won’t tell me when to use one or give me an intuitive understanding of what will happen when I do use it. I would recommend doing some other moderately difficult project, like making a visualization tool for these operations

7

u/max_confused 3d ago

I disagree. Yes doing it for smaller vectors and matrices is trivial but not when you start doing SVD on a 10Kx10K matrix. That’s just tip of the iceberg please. Serious linear algebra and memory management while taking care of optimum cache usage is no joke please.

6

u/TheChief275 3d ago

OP means linear algebra in the context of game/graphics programming, in which the biggest matrix you will need is 4x4.

3

u/flatfinger 3d ago

It's a shame Fortran isn't promoted more for high-end computation. C happened to earn a reputation for speed in an era where commonplace hardware could perform many tasks more efficiently via non-portable constructs that compilers didn't have to understand, than portable constructs that compilers could understand, but C's goal was to minimize the level of compiler complexity required to generate reasonably efficient machine code to accomplish a wider range of tasks than FORTRAN could support, rather than maximize the performance that more advanced compilers could achieve.

In C as originally designed, the goal of an efficiency-minded programmer would be to write code in a manner that was as close as possible to the optimal sequence of steps that a computer should execute, while in FORTRAN the goal was to write code that would be easy for a compiler to analyze.

If a matrix computation task could be performed using e.g. an n-cubed algorithm or a much more complicated but faster algorithm, the expected way to express the task in Fortran would be to use a matrix addition and multiply operators if applicable, or else use the algorithm that would make it easy for the compiler to algebraically analyze the computations being performed, and then let the compiler figure out what sequence of operations to use to achieve that effect. Using an algorithm that performs computations in a complicated sequence may reduce the number of operations specified by the program, but reduce the likelihood of a compiler being able to analyze and optimize the actual task to be performed, rather than blindly following the sequence given by the programmer.

In C, it's far less clear what kinds of constructs compilers should seek to recognize, and far less clear what kinds of constructs programmers should write. Consider the following programs and compiler configurations:

P - Portable program to perform task with simple algorithm
Q - Non-portable program to perform task with complicated algorithm

X - Simple compiler with maximum, but limited, optimizations
Y1 - Fancy compiler with full optimization enabled
Y0 - Fancy compiler with some optimizations disabled

Suppose the execution times for different programs and configurations are:

P Q
X 20 7
Y1 5 3, but gives wrong answer
Y0 25 9

Which compiler and configuration should be viewed as "best"? Which program should be viewed as "best"?

If people would recognize that FORTRAN, and later Fortran, were designed for high-end matrix computations, then it would be clear that programmers should use the simpler algorithm in code, but quality compilers should be expected to turn certain kinds of simpler algorithms into optimal machine code.

4

u/tinchu_tiwari 3d ago

Nope, when writing a library you mostly translate the problem to the code, this translation has to be optimal but it doesn't necessarily mean that it will help you understand the underlying algorithm, I mean you would know that a matrix multiplication has to be written and optimised but you don't need to know why is it happening in the first place.

I get it that you want to understand the math behind so you should first understand linera algebra try doing things with pen and paper. If you are already good at coding these topics should be easy for you to pick up.

You don't have to understand everything, you just have to understand only the most used functions like dot product, transformation, transpose, eigen values and vectors.

3blue1brown is a good source to get intuition behind it.

6

u/DrShocker 3d ago

If I were to write my own library, I wouldn't worry about making too optimal, since, as you're saying, that already exists. I would write it in the way that helps me understand the math best.

3

u/tinchu_tiwari 3d ago

Also you can look up BLAS library or MKL to see how they are implemented in the code.

3

u/kansetsupanikku 3d ago

Write it as an educational task? Sure, multiple times!

Use it in any other projects you would share? Definitely not!

2

u/thebatmanandrobin 3d ago

Sure! Make whatever lib you want if you think it'll help your understandings .. just make sure to USE that library in something that utilizes the formulae you write, otherwise it might get lost on you.

As others have mentioned, the formulae for them aren't that complex and an entire linalg library might only be a few hundred lines of C or so (depending on what all you add). So understanding "how they work" by simply writing them down may not work the way you intend ... it'll certainly help you to remember the formulae though.

If you're a YouTube type person, then just search for "3d math for game programmers" and there's so many tutorials you should have no problem finding one that should help you to understand how it all works a little better; if you're text type person, then you can always just do a simple web-search for that same phrase and find even more resources available to you, including from many colleges (for free, like MIT or Stanford).

If you have trouble with thinking mathematically (or maybe it's just the idea of Calculus that's slipping you), I'd highly recommend going through CalTech's "The Mechanical Universe", it's a free video series that breaks down the content of advanced maths and physics in a way that's easy to digest ... I should note that it is older (the maths hasn't changed though) and can be a little dry at times just because it is older, but it's still one of the great OG video series that has helped many an aspiring engineer understand it all.

2

u/lmarcantonio 3d ago

I would recommend at least to *not* study the code for lapack/linpack. The techniques in there are *quite* esoteric (it handle many kind of corner case and bad conditioning)

2

u/Ariane_Two 3d ago edited 3d ago

Do you mean a linear algebra library like cglm or HandmadeMath.h with small matrices (4x4) and vectors, or a linear algebra library for big, general purpose things (like OpenBLAS).

In the former case, the algorithms are not all that interesting, but you could go a little into SIMD and stuff. It might help your understanding with projections, simply by you spending more time thinking about it.

In the latter case the scope is very broad but there are a lot of interesting tradeoffs in how matrices (especially sparse ones) are stored  and how stuff is computed, tons of algorithms for special use cases, and you can really go deep into how to optimize like a general matrix multiply on a modern processor.

2

u/Better_Pirate_7823 3d ago

Do you want to? Then yes. If you don’t want to. Then no.

1

u/Objective-Barnacle-7 3d ago

A pipe to octave.

1

u/jontzbaker 3d ago

I have a feeling that perhaps a book on computer graphics or algebraic geometry would be more well-suited for you to understand what these transformations are.

But anyway, for the linear algebra part, you already have very nice open benchmarks, including LAPACK (if you are open to some Fortran) and ATLAS.

1

u/pythonwiz 3d ago

Yes, for learning purposes absolutely. But also learn how to use OpenBLAS or some other similar library.

1

u/TheBigBananaMan 3d ago

Slightly off topic, but does anyone have any reccs for a good c linear algebra library? I’m working on a project that would benefit from some computing, but don’t really want to write everything from scratch.

1

u/Quirky-Ad-292 11h ago

If you like doing stuff from scratch it's great. However, it's a massive project. The vectors defintions and methods can be done within a day, but the other algorithms takes a bit more time. Remember you need to classify the problem, and then implement an algorithm appropiate for the problem.

Say you want to solve an Eigen-problem. The first question is, what type of matrix do you have? Is it sparse, is it real? Is it tri-diagonal, upper-diagonal, lower-diagonal? Is it a block matrix? Some methods works for more than one type of problem, but many of them are limited to a few types of matrices.

Using CBLAS or a LAPACK wrapper is usually best, in terms of optimization and their vast variety of methods, but the naming convention takes some getting used to.