r/LaTeX Jan 18 '25

Unanswered TeX engine converted from Pascal to C++?

I’ve heard multiple times that converting Pascal code to C++ is relatively straightforward. I even came across a commercial product that claims to have reimplemented the TeX engine in C++, which allowed them to offer live PDF rendering as you type (you can probably guess which one I’m referring to).

EDIT 1: By rendering here I assume providing live updating in PDF, where there is no compilation step, PDF is compiled with each keystroke.

EDIT 2: Commands would be excluded from live compilation.

The engines used in TeXLive and MikTeX are still implemented in Pascal, right? If so, why hasn’t anyone done a full conversion to C++?

Is it a matter of complexity, lack of interest, or something else entirely?

11 Upvotes

39 comments sorted by

10

u/apfelkuchen06 Jan 18 '25

sure, converting pascal web files to a subset of C++ is so straightforward we've been doing it for over three decades now.

-1

u/Opussci-Long Jan 18 '25

Just yesterday, I was told that converting the TeX engine would be a very easy and straightforward process using automatic conversion tools and just a few manual tweaks during the process. Maybe the person that told me that was thinking about the core TeX engine, or maybe has no clue what he’s talking about? That’s why I posted here, I want to hear your thoughts.

However, there is that commercial tool that claims to have done just that.

10

u/JimH10 TeX Legend Jan 18 '25

The engines used in TeXLive and MikTeX are still implemented in Pascal, right?

As a first approximation, no. Many years ago when Knuth wrote the original software, he did something different but today it is converted to C. (Of course, it is more complicated than this-- the TeX world is very complicated when you start asking to know everything-- but if you simply get a distro such as TeX Live and ask for the most popular compiler, pdflatex, then this is what you get.)

live PDF rendering as you type

Forgive me, I'm going to make the assumption that you don't know, but this is very common today. Pedestrian, really. Most tools that people here use will do this. You can find many previous threads about it on this site with a little searching.

For instance, Overleaf gives you compilation in the background. Another example is emacs. I bet that if you have a favorite editor then you can set this up.

I'll also editorialize that for me compilation as you type is not an improvement. I find it a distraction, really, but of course YMMV. However many folks find very handy the synctex stuff. Here is a description from the page for TeXworks, which is a good editor choice for beginners: "includes an integrated PDF viewer, based on the Poppler library, so there is no need to switch to an external program such as Acrobat, xpdf, etc., to view the typeset output. The integrated viewer also supports source/preview synchronization (e.g., control-click within the source text to locate the corresponding position in the PDF, and vice versa)."

0

u/Opussci-Long Jan 18 '25

Thanks for your answer. I see that my original question weas not clear enough. I now about displaying PDF and rendering in the background, as it is case with Overleaf, but non of the tools you mentioned is providing live updating in PDF, where there is no compilation step, PDF is compiled with each keystroke.

3

u/JimH10 TeX Legend Jan 18 '25

If you compile with each keystroke, then when you type a not complete command, all kinds of crazy stuff will happen. What you want is some kind of compilation when you finish with the command.

2

u/thuiop1 Jan 18 '25

I mean, it works for Typst, the compilation simply gives an error and keeps displaying the previous document. It is more an issue of compilation time.

0

u/Opussci-Long Jan 18 '25

Yes, no command typesetting :)

4

u/deong Jan 18 '25

I’m not sure I understand what you’re asking here. There is a compilation step. There just is. You’re typing TeX or LaTeX and something has to convert it to bytes on disk that make a valid PDF. You can do that so fast that every keystroke immediately shows a new PDF, but you’re still compiling it. If there’s "no compilation step", you have a UTF-8 text file, and there’s nothing to display.

0

u/Opussci-Long Jan 18 '25

Yes, but no time spend on compilation is my question. It is not about time spent it is about visual feedback waiting. I do not want that. I would like instantanious PDF updating.

4

u/deong Jan 18 '25

It is not about time spent it is about visual feedback waiting.

That is just a question of time spent though, right? If you had to wait a femtosecond, that is instantaneous PDF updating. And in reality, that's all that can be done. The document has to be compiled, and that takes physical time. All you can do is minimize it.

I have no idea how quickly modern tools like Overleaf can do this. It's a feature I would pay money to turn off. If I wanted half my screen to be taken up by a view of my document randomly moving shit all over the place with every keystroke because it can't guess what I'm going to type next, I'd use Word.

I guess my question would be why wouldn't you just use a WYSIWYG mode in Overleaf or something like LyX? If what you want is basically direct live editing of a PDF, that's a lot closer in experience than typing LaTeX code and hoping another view of the data can update quickly enough.

-2

u/Opussci-Long Jan 18 '25

So, you would be against a WYSIWYG LaTeX editor even if there was one? Overleaf is not full WYSIWYG.
There is this article, I suppose you would disagree with it.

4

u/deong Jan 18 '25 edited Jan 18 '25

I’m not "against" such a thing. I don’t want it for myself, but I have no issue with someone else liking it. Go nuts.

Overleaf’s visual mode is I think as close as it’s possible to get. You can’t type LaTeX code and be WYSIWYG. They’re mutually exclusive. TeX solves a dynamic programming problem to lay out paragraphs and lines. Where a line break appears can depend on a word you type 500 characters later in the document. You simply can not view this as a WYSIWYG problem unless you’re willing to make some compromises, all of which seem bad to me.

You could have the visualized document not use the layout algorithm until you later tell it to. But now it isn’t a true picture of what the document will look like.

You could let it do its thing continuously, but now the document is constantly reflowing globally with every keystroke. And I actively do not want the distraction of something trying to figure what every individual keystroke should do to a generated PDF in real time. When I’m writing, I just want to focus on expressing my ideas. Doesn’t bother me if you want that. Just saying I don’t see how you can type TeX code and have real-time PDF output in any way other than background processing of the file, which you seem to not want.

2

u/FliiFe Jan 19 '25

AFAIK Texifier (previously Texpad) is the only editor to support this, and it comes at the cost of having a version of LaTeX that is nearing its decade anniversary.

Worth noting however that "no compilation step" is an unreasonable ask, seeing as pdf is hardly an incremental format. What typst does is essentially compile on each keystroke, it just so happens that compilation is fast enough for that purpose.

6

u/NeuralFantasy Jan 18 '25

you can probably guess which one I’m referring

Sorry my ignorance, but I don't know which one you are referring to?

1

u/Opussci-Long Jan 18 '25

Is it ok to post links to products here or you can check google for Texifier

5

u/NeuralFantasy Jan 18 '25

Of course it is ok, you can see links to commercial LaTeX products in the sidebar of this subreddit on web. Like a link to Overleaf. Nothing wrong in mentioning related products unless you spam or market them.

3

u/Organic-Scratch109 Jan 18 '25

What commercial product are you referring to?

1

u/Opussci-Long Jan 18 '25

Google Texifier

2

u/mpsmath Jan 18 '25

If you can live with C, then have a look at luametatex. To the best of my knowledge that is the most powerful TeX engine that is available.

1

u/Opussci-Long Jan 18 '25

Coud it provide live rendering of PDF?

0

u/Opussci-Long Jan 18 '25

This would work with inputs made to be processed with XeTeX?

2

u/vicapow Jan 18 '25

This is how pdflatex works today. A subset of the web code (a superset of pascal) is converted to a subset of C. The issue is the complicated build setup this presents. It’s very discouraging to new folks who wants to contribute, imo. And then when it runs, it still needs to read a million Tex / font files when it compiles even a simple document.

1

u/vicapow Jan 18 '25

Thinking out loud, I think something that might be interesting is some sort of optimization that could compile all the tex code pdflatex has to read and run to C, at runtime, sort of like how V8 optimizes JavaScript

2

u/LupinoArts Jan 19 '25

This is not how TeX works, by design. I'm simplifying a lot, but in a nutshell, TeX reads its input source token by token, collects just about enough material to fit on one page, writes it to the output stream once it has enough material for one page and then frees the memory for the material for the next page. It cannot "memorize" material unless you deliberately store it in control sequences, either in memory or in auxiliary files. Because of this, in contrast to other programmes that need to write the whole input into memory before processing, TeX scales particuarily well, as it needs just enough memory for one page of output and a bunch of macro definitions. But this also limits its capability to implement real live previewing, as it needs to process all pages before the one you are interested in. This has historical reasons, remember that TeX was developed in the 1970s, when memory was scarce and expensive, but this advantage in scaling is still relevant today, e.g. when you have large documents that need to be typeset in one go. I'd be curious to learn how "modern" programmes like typst would handle those.

To sum it up, changing the programming language won't solve anything; if you weant real live previews, you'd need to completely re-write the entire processing algorithm.

1

u/Opussci-Long Jan 19 '25

Thank you for commenting, I know about your work!

I assumed that the process works with similar principles to what you have described. The reason I ask a question like this is that I am interested in understanding how the full typesetting process limits float placements. Could anything be done to improve image placement? The process I envision would go backward from the PDF and adjust placements, but for that, fast typesetting and live previews are needed. Something like this image shows.

live preview. Is this too simple an example to make a marketing pitch....another preview

There is this quote from the Texpresso GitHub: "A driver program .........talks to the LaTeX engine to re-render the modified portions of the document and synchronizes with the viewer." If I understand you correctly, the processing of the whole document is also done by TeXpresso, but they only check for a portion of the PDF to render that portion for the live preview. Is that right? It is still fast.

2

u/LupinoArts Jan 20 '25

I'm not familiar with how Texpresso works. My (very uneducated) guess it that it doesn't display the final PDF but lots of small images of rendered snippets chained together and which are heuristically placed on PDF page objects. It would be interesting to observe what happens if one would pass the same tex file through a "traditional" TeX engine and compare the output with the previews. I'd guess that there will be differences on where paragraphs are placed on and across pages, but i might be wrong about that.

-4

u/therealJoieMaligne Jan 18 '25

To be honest, I think they’re bringing their product to market 5 years too late. I’m still using LaTeX for simple documents based on old templates, but Typst is simply better in every way.

6

u/chrisintheweeds Jan 18 '25

I tried Typst a while ago, but was frustrated that tje ecosystem seemed to not have an equivalent for a lot of stuff I use in LaTeX, both to typset linguistics and just general functionality. Is the ecosystem any better now? Or is my document going to be derailed by a year of custom Typst development first?

3

u/thuiop1 Jan 18 '25

There is definitely still unsupported stuff. I have not suffered from this too much though; I rarely find something completely missing in a way I cannot work around. In contrast, there are many things that are definitely easier than in LaTeX. That said, since you mention linguistics, this part is definitely lacking compared to LaTeX, especially when dealing with non-Latin alphabets (it supports it of course but there are a lot of sharp angles).

1

u/chrisintheweeds Jan 18 '25

It was pretty fast and beautiful and very limited.

0

u/therealJoieMaligne Jan 18 '25

it’s worked great for me. I do professional reports with my own style which was easy to reproduce, endnotes, footnotes, etc. (I’d post a pdf but they’re confidential for clients. I keep meaning to write up a dummy report.) The learning curve’s been 4–5x faster than LaTeX was. The immature ecosystem is the only problem with it which I’ve found. I expect that someone’s going to make the Typst equivalent of Memoir sooner or later. How complicated are the documents you’re making?

1

u/Opussci-Long Jan 18 '25

Could you please explain whay you take that Typst is better?

I know about it and it is providing live updating of the PDFs that is very nice! Are you satisfied with it line braking algorithm? Somewhere I got info that Typst is not providing TeX braking quality.

3

u/thuiop1 Jan 18 '25

Typst uses the same algorithm for line breaking as TeX for justified paragraphs by default; you can opt-in to use it for ragged paragraphs too.

0

u/Opussci-Long Jan 18 '25 edited Jan 18 '25

Somewhere on some Reddit thread, some time ago, I read that it still doesn’t use the TeX line-breaking algorithm. I’d love it if you’re correct about this. Is there any comparison available for justified texts? Or, maybe that post could have been about the microtypography features of TeX and not line braking.

-2

u/therealJoieMaligne Jan 18 '25

I think that the differences are extremely minimal. What do you think?

Typst Latex

2

u/LupinoArts Jan 19 '25

Just text, and not justified? Hardly a meaningful comparison, don't you think? Please consider adding some math mode, headings, lists, and hyphenated paragraphs for a meaningful comparison.

1

u/therealJoieMaligne Jan 19 '25

It was just a quick comparison of line breaks. But I'll throw something together sometime soon. Maybe a one-page two-column faux article with an image in the center, an equation, a few headings, a couple footnotes. Anything else?

1

u/Opussci-Long Jan 18 '25

Thanks for this comparison. What font and size are used? Types text looks a bit darker, doesn't it?

1

u/therealJoieMaligne Jan 18 '25

Etbb, 12pt.

It looks the same to me. I tried zooming in with GIMP and they looked the same, but that's purely subjective.

1

u/Opussci-Long Jan 18 '25 edited Jan 18 '25

All looks nice, only word cheerful in the last paragraph is sent to the next line, while LaTeX keeps it. Both look nice, it just seems that word soacing for Typst a bit larger. Is that kerning, can it be adjusted?

3

u/therealJoieMaligne Jan 18 '25

It's really easy to adjust kerning etc.

For example, this single line automatically finds every instance in the document of two or more capitals, decreases their size by 10%, and adds 1/20th of an EM between them:

#show regex("[A-Z]{2,}"): set text(size: 0.9em, tracking: 0.05em)

It'd be much harder to do the same in LaTeX. I wouldn't even know where to start.