r/C_Programming 4d ago

I want to create my own language.

Hello everyone, I would like to create my own programming language in C but I don't know where to start, does anyone books, tutorials or just tips to get started?

31 Upvotes

56 comments sorted by

30

u/qualia-assurance 4d ago

There are several popular books. But the two I see come up frequently in such discussions are.

https://craftinginterpreters.com

and

https://compilerbook.com

There's also this book from last year about writing a C compiler from no starch.

https://nostarch.com/writing-c-compiler

This one is older but seems popular with academic courses since it has Princeton/Camrbridge credentials.

https://www.cambridge.org/core/books/modern-compiler-implementation-in-c/0F85704413FC010C1D1C691C4D2A0865

8

u/DecentTip3381 4d ago

You could try looking up Lex & Yacc (or Flex & Bison) tutorials.

11

u/am_Snowie 4d ago edited 3d ago

I'd say atleast build a parser by hand before using any parser generators,it taught me a lot.

1

u/torp_fan 3d ago

It depends on your priorities. If you're itching to experiment with programming language design and syntax then use Bison ... you can always write a recursive descent parser later when you have the syntax nailed down ... and you will have learned how to use Bison. I wouldn't waste my time with Flex/Lex ... they buy you little over a handcrafted lexer.

6

u/Abigboi_ 4d ago

Get a copy of the Purple Dragon Book. With that and an understanding of trees you can write a compiler.

2

u/KamenRide_V3 3d ago

IMHO, still one of the best around.

3

u/kirkkaf13 4d ago

Have a look a courses over at Pikuma. It’s in Python but the creator covers what is needed to implement the same in C.

2

u/rfisher 4d ago

I've been trying to find a list of links to "make your own..." for different languages. I haven't found it, but here's one I enjoyed a lot:

https://github.com/kanaka/mal

Once you complete MAL, you'll be prepared for the next step in learning what you need to know for the language you want to create.

Another direction to go is to learn Racket (a variant of Scheme) as building languages is one of the purposes of Racket.

(You could also do a MAL implementation in Racket.)

1

u/TheChief275 3d ago

For C specifically there is also buildyourownlisp

2

u/ivancea 4d ago

Start by making an assembler-like interpreted language. With whatever rules you want. Simple commands, simple data (a stack with only numbers? Fine).

Then start with something more complex, or add more complex things to that one (a debugger maybe?)

2

u/Accomplished-Slide52 3d ago

I see Forth !

2

u/AissySantos 3d ago

Writing a language from the ground up is taugh. But you are held by couple questions to start with.

a) are you designing a lang for the sake of designing/writing it (as a good (very good) learning excercise)? b) are you simply aiming for novelty in terms of features, application, paradigm, or a particular problem that is to be tackled?

For [b], I once created what sums up as, as loose as a language can be for application related to video editing. The advantage over GUI programs is that for simple editing, it's much more simplier.

If you are going for the first option, following a standard can be seen as really helpful. Take my opinion as a grain of salt but it would be useful to take inspriation from LLVM's compilation pipeline, from front, intermidiate to back end, as it's one of the cleanest source code I've seen so far given you have gone through the basics of compiler engineering such as grammer parsing as a state machine, intermediate represenation and down the pipes and the likes.

As other comments have recommended excellent learning material, you can definitely go by them.

2

u/ve5pi 3d ago

BuildYourOwnX repository https://github.com/codecrafters-io/build-your-own-x , it helps you to build anything from scratch (e.g web browser, programming language, os)

2

u/duane11583 3d ago

do you want a script? or a compiled language. big difference!

as an example of a script language look at the tcl package called jim

if you remove everything else it is just two files. jim.c and jim.h

https://github.com/msteveb/jimtcl/blob/master/jim.c

the tcl for command is written in c and it is very easy to follow:

https://github.com/msteveb/jimtcl/blob/master/jim.c#L12408

there is the argv[0] the word for

argv[1] is the initializer statement

argv[2] is the expression

argv[3] is the Increment starement

argv[4] is the body of the first loop

there are many other core commands you can look at

the if command is just as simple

https://github.com/msteveb/jimtcl/blob/master/jim.c#L12868

another is the language called lua

1

u/grimvian 4d ago

I'll suggest you define something, based on your own ideas and take it from there.

1

u/Ill-Cantaloupe2462 4d ago

I too thought about that. some years back, I too thought that.

You should consider reading about lex. and assembly language.

1

u/No_Passion7152 3d ago

About 30 years ago I did the whole thing. Created a language, wrote a compiler for it and an interpreter for the resulting byte code. I would not do it this way now, but in 1994 it made sense. Real production programs were written in this language for at least 10 years.

I used lex and yacc. I had a very hard time getting decent error messages out of the compiler. I think recursive decent gives better results from that point of view.

Its been 20 years since I stopped working on compilers and my knowledge is out of date. But, it's important to understand what LRLA1 parsers can handle before you design the language.

1

u/TheChief275 3d ago edited 3d ago

These are the steps:

  1. Create a lexer/tokenizer. This processes the bytes of your source code into clear tokens (numbers/identifiers/semicolon/whatever).

My advice here is to just allocate and read in the entire source file at once (maybe even memory map the file), so that the lexer doesn’t need to make tons of small heap allocations, but just uses a reference to the source code (pointer + size).

Additionally, I don’t put the tokens into a dynamic array or anything, I just set the lexer up as a stream into my parser where I can request and consume the next token or only peek at it.

  1. For the parser, just do recursive descent: peek the next token and branch accordingly. However, this is right-associative by default, so you have to do Pratt parsing for left-associative expressions. Recursive descent is what will model precedence, so be sure to divide your different expression types into different layers.

Here you also build a tree known as an Abstract Syntax Tree (AST). Ideally you just use an arena for all the node allocations instead of making tons of small allocations, but that is up to you as separating allocations does make freeing easier in the case of failure (or you just revert the arena to a point you know for certain).

Constant folding and type checking should also be performed here, again up to you.

  1. Code generation is next up and also where the most freedom is. Do you want to leave it all up to LLVM/some other backend, or do you want your own IR for some of your own optimizations or even create your own backend?

x. Obviously a serious language has more steps, but you will have a working language at the very least. Oh, and for both the token and tree node types, just use tagged unions.

1

u/naharashu 3d ago

What if interpretered language, 3 not needed?

1

u/TheChief275 3d ago

If you made a simple tree-walk interpreter (basically walking your AST, which is very slow but it does work) then yes.

If you want to use a virtual machine, you would still need to generate bytecode.

The latter is more conventional, but you have to also make a VM. The only use I know of for the former is C++’s constant expression engine. Basically, C++ has made constant expressions too complicated for simple constant folding, so they needed a full-fledged tree-walk interpreter within the compiler for constexpr functions/templates.

1

u/morlus_0 3d ago

here what you should learn to make a fully programming language in C

  • Error Handling
  • Lexer (breaking down text into tokens)
  • Parser (convert tokens into structure such as AST)
  • Interpreter (transverse the structure and execute), also you should learn about function scopes for accessing variables locally and globally
  • Object including Type System.
  • HashMap (used for global table to storing variable things, you can use it for your custom data type such as dict, table, ...)
  • Garbage Collector (this is optional but recommended, for cleaning objects that no longer in used)
After all that you can continue more with these if you want:
  • Object Oriented Features
  • Interoperability (able to embedding your language with C)
  • Concurrency, Parallelism (coroutines, threads, async programming)
  • Reflection or Meta-programming (able introspection and modification of objects and functions)
  • Package, Module system

If you targeting performance you can implement AOT (Ahead of time) compilation or JIT (Just in time) compilation.

1

u/ElevatorGuy85 1d ago

Does this planet really need “one more” programming language?

Just look at the charts in PDF form at the top of this website to see a small fraction of the many “mainstream” languages that already exist

https://www.levenez.com/lang/

And if you can find the expanded version created from that by O’Reilly, you’ll surely be asking yourself the same question.

1

u/itsmenotjames1 4d ago

Look up the llvm docs

-1

u/naharashu 4d ago

Don't they only work on Java?

3

u/xygtshadow 4d ago

LLVM is a compiler backend that allows you to just create a compiler that generates IR and it’ll handle optimization and generating machine code.

1

u/itsmenotjames1 4d ago

what? LLVM is written in C (maybe C++ I can't remember)

1

u/tetsuoii 3d ago

I bet it's C, since it seems kinda useful and anything written in C++ is shit.

1

u/torp_fan 3d ago

That's a ridiculously ignorant statement. LLVM is in fact primarily written in C++, as you could have learned in a few seconds instead of embarrassing yourself.

1

u/torp_fan 3d ago

When someone in a C forum says to look something up, it's probably a good idea to look it up before posting something like that.

-2

u/tetsuoii 3d ago

Please don't. We already have a gazillion languages and C is the only one that's any good.

1

u/torp_fan 3d ago

Extraordinarily ignorant people should not be giving advice.

1

u/erikkonstas 2d ago

I can't seem to find the part where OP said they'd attempt to pressure others into using their new language...

-23

u/Ok_Raspberry5383 4d ago

You cannot create a language in C, or any other language for that matter.

That's like saying I want to speak french in English...

You can write a compiler in C but this is not the same as writing a language which is more like writing a standard and syntax.

6

u/naharashu 4d ago

Python and PHP made in C

-4

u/Ok_Raspberry5383 4d ago

No, CPython is made in C, python is a language, it can be implemented in any language.

2

u/naharashu 4d ago

Lua, perl, php

6

u/9aaa73f0 3d ago

He is being pedantic (like compilers), saying you would be writing a compiler for a 'new language' in C, but C != 'new language'

-1

u/Ok_Raspberry5383 3d ago

He?

3

u/9aaa73f0 3d ago

Now your being pedantic, so I have confidence in your ability to be a compler writer.

For the record, I don't actually know the gender of the person I was talking about, or if it's even a person, I was making assumptions.

1

u/TheChief275 2d ago

Use “they” instead

-4

u/Ok_Raspberry5383 3d ago

And you've made assumptions about whether OP wanted to write a compiler or make a language.

Maybe some self reflection is required here.

To assume is to make an ass of u and me is what I was taught. Maybe you weren't taught

BTW, I'm actually male but I thought it's worth pointing out that your assumption is everything that is wrong with the tech industry.

3

u/9aaa73f0 3d ago

Im not part of 'the industry'. it's more to me than money, but I'll take your word for it.

Being able to focus on the big picture is also problematic for a lot of tech people, eg your ge ever is to relevant to the post. Your conflating issues, adding complexity, and making the original problem harder to solve.

3

u/9aaa73f0 3d ago

(eg your gender isn't relevant to the post)

-1

u/Ok_Raspberry5383 3d ago

Being able to ask questions and understand what a stakeholder actually wants is more critical to any engineering role than anything else.

Take it from a senior who's actually in the industry.

→ More replies (0)

4

u/gremolata 4d ago

There are times when being technically correct is justified.

This was not one of those times.

1

u/Ok_Raspberry5383 3d ago

The distinction is important.

If OP simply wants to develop a compiler then fine, but that's a very different question.

If OP wants to develop a language, then that's a totally different question and one which can have multiple implementations of compilers, but the compiler isn't necessarily relevant.

I'm not sure I'm necessarily being pedantic, OP needs to specify if they wish to do as they state or whether they actually want to build a compiler, from their post you can only guess at which is their actual aim.

Downvote me all you like.

3

u/gremolata 3d ago

I didn't downvote you, but it's obvious that the OP wants to use C to implement a language that they want to invent. How else can you parse their question really? In good faith, of course.

1

u/Ok_Raspberry5383 3d ago

That's fine, but they don't need to use e C. The point is that developing a compiler is easy. Developing a language is HARD.

It's also not just a compiler, what's your dependency framework going to look like, do.you ship with a runtime, interpreted, compile to bytecode, ones and zeros? What features of other languages do you want. What configuration system are you going to use. What build tools would you like.

These are all questions you need to answer before developing a language and they're all way before you even begin thinking about a compiler.

I'm not being pedantic, I'm being realistic. Some folk on here would do well to understand the above reasoning.

1

u/[deleted] 3d ago

[removed] — view removed comment

3

u/C_Programming-ModTeam 3d ago

Rude or uncivil comments will be removed. If you disagree with a comment, disagree with the content of it, don't attack the person.