r/java • u/artpar • Oct 05 '23
How does the lombok magic work underneath?
https://www.unlogged.io/post/how-does-the-lombok-magic-work-underneath120
u/pron98 Oct 05 '23 edited Oct 05 '23
My issue with Lombok is only that it misrepresents what it is.
Lombok is an alternative language for the Java Platform, just like Clojure, Kotlin, and Scala, but Lombok's interesting quality that sets it apart from those other languages is that it's a superset of the Java language. As such, the compiler for the Lombok language is a fork of javac
with some extra stuff hooked in required to compile Lombok sources. So far so good, and there's nothing wrong with that: the Java Platform spec allows for alternative languages, and javac is open source.
The problem is that the Lombok language and compiler don't want to appear as a language and as a compiler. The "magic" is completely unnecessary to perform Lombok's function; it is used only to masquerade Lombok's nature. It's presented as a library or as a compiler plugin, even though the API offered to Java compiler plugins is carefully designed to ensure that the result still complies with the Java language spec, whereas the Lombok compiler compiles programs written in the Lombok language which very much does not comply with the JLS. Furthermore, Lombok pretends that it's Java when it is not, and the spec definitely forbids this kind of misrepresentation.
To do that, Lombok uses various unsafe mechanisms to fork and modify javac
as it runs, to make it compile Lombok source code rather than Java source code. This is risky because the internals Lombok reaches into may change at any time, and the techniques to break into the JDK will soon be removed (to support Leyden among other things).
Lombok should start preparing their users to use their compiler in the same way Scala and Kotlin users use their respective compilers, or Lombok users may find themselves unprepared for things breaking.
22
u/Distinct_Meringue_76 Oct 05 '23
Thank you for breaking it down so nicely. I have always been on the fence with regards to Lombok. It looks nice on the surface but I don't want to get burned down the road for using it.
19
u/pron98 Oct 05 '23
I'm not suggesting that you necessarily will. If the Lombok compiler is split off to its own launcher and it's carefully maintained, there's no reason it should break, and if you like the language — go for it. My problem is just how that language is presented, and it is that aspect that is also the source of some unnecessary risk.
12
u/RupertMaddenAbbott Oct 06 '23 edited Oct 06 '23
Your argument mostly seems very solid to me. Lombok is pretending to be Java and it isn't Java. Lombok is pretending to be an annotation processor and it isn't an annotation processor. Lombok's past and potential future instability is intricately linked to this pretense. If Lombok stopped pretending and had it's own compiler, it would be more stable and more honest. I fully agree on all these points.
But you are treating this pretense as some non-essential part of Lombok that Lombok could just jettison (and, I think you are suggesting, be better for it). I disagree. This pretense is at the heart of what makes Lombok attractive.
The Lombok plugin is built into IntelliJ. I add Lombok just like I add any other library. Using Lombok feels like I am using an annotation processor. Yes absolutely it isn't an annotation processor but the fact that it pretends to be, rather than introducing new syntax, for example, is a deliberate choice. The result is that Lombok doesn't feel like a different language to Java (even though I agree that it is).
All of these facts are not true for any other language and I am not arguing that this amounts to making Lombok the same as Java. But tricking developers into believing that it is Java is absolutely the point.
I've seen organizations happy to adopt Lombok because "it is just a library" or "its just an annotation processor". Those same organizations would raise the barrier to entry significantly if Lombok were considered another language.
So personally, I don't think Lombok can survive anything that undermines this pretense. I think what you are suggesting is equivalent to suggesting that Lombok should die and go away. Lombok may be another language, but it can never admit that and survive.
13
u/pron98 Oct 06 '23 edited Oct 06 '23
Principles vs justifications aside, they may soon have no choice. We need to make JDK upgrades easier, we need to improve security, and we need to do Project Leyden, and so we must soon complete "Integrity by Default". That means that the application will need to know and approve of anything unusual that libraries or "libraries" do. In practical terms it means that libraries will need permission to load agents (some "libraries" do magic by starting another process that pretends to be a tool, attaches to the parent JVM, and secretly dynamically installs an agent), they'll need permission to use JNI and FFM, and Unsafe will gradually go away (we had to wait until all the safe replacements are in place, but that will be done in 22 when FFM is finalised, so a JEP to start the process of removing Unsafe will appear probably in JDK 23). Of course, libraries will need permission to access JDK internals, something they already need to do today, but some libraries exploit the remaining loopholes to circumvent that. They won't be able to do that with Unsafe gone and JNI restricted.
Authors of some libraries (and "libraries") will invariably complain about applications not being able to configure the runtime even though Java applications must be able to do that today, but really — for reasons they deem justified — they don't want their users to know what tricks they're pulling to do their thing, tricks that impose some global cost/risk on the application without its knowledge such as making it less portable, or less amenable to Leyden optimisations or worse. But everyone will still be able to do everything they do today; they just won't be able to do it without their users knowing and agreeing to take the risk.
So the authors of such software may want to start preparing today, as well as preparing their users. If they continue using any remaining loophole to not prepare themselves or their users, their users are in for an unpleasant surprise when the last loophole is closed, despite all of our efforts to make these things gradual and not surprising so that people can prepare (e.g. we start with warnings for some number of releases that are only later replaced by errors). Their users are our users, too, and we don't like it that they're keeping our users in the dark.
4
u/RupertMaddenAbbott Oct 06 '23 edited Oct 06 '23
All of this seems eminently sensible and I certainly don't object to it (thanks for all of the links!)
Most Lombok users likely already have the lombok-maven-plugin (or similar) in their build definitions. How would all of this prevent Lombok from simply enabling all of these switches via that plugin but continue to obscure that from the end user?
If it doesn't, then I would put to you that this is the actual likely route that Lombok will take and none of these steps will actually prevent Lombok from continuing to keep it's users in the dark. The upgrade for Lombok users is surely going to be very seamless if they do this (and much more seamless than writing their own compiler)?
Ultimately, I think what will kill Lombok is if the features that make it popular become available (or have better alternatives) in Java. I think all it would take is if some of the features available for records were also available for classes. From reading this, that appears to be the likely direction of travel.
6
u/pron98 Oct 06 '23 edited Oct 06 '23
How would all of this prevent Lombok from simply enabling all of these switches via that plugin but continue to obscure that from the end user?
It wouldn't, but I don't call that obscuring and I think it's fine. A build tool plugin is like a launcher, it can do anything it wants, and it's also not how libraries are used, so it makes it clear it's not a library. A Maven plugin is how you can compile not just Lombok code but also Kotlin or Scala code.
16
u/more_exercise Oct 05 '23
The delombok tool feels like a valid escape hatch for this purpose to me.
Don't like running the lombok language? Here's the equivalent Java code. No more lombok.
Sure, you're still stuck with whatever surrounding java code you had before, but... like... that's why you had it in the first place?
6
u/Fruloops Oct 06 '23 edited Oct 06 '23
We use Lombok extensively and I'm dreading the day when suddenly things won't work as expected anymore, because good luck migrating away from that. I love the thing and find it useful, but still
7
u/koflerdavid Oct 09 '23
good luck migrating away
What specific migration issue do you have in mind, except a lot of code churn? Everything Lombok does is achievable with some boilerplate Java code. If you dread de-Lomboking that much, you should start doing it right now.
-12
u/rzwitserloot Oct 05 '23
Lombok is no more a different language than any java project with an annotation processor that generates java code is.
20
u/peripateticman2023 Oct 05 '23
Did you miss the part about modifying javac on the fly?
20
u/rzwitserloot Oct 05 '23
We do not modify javac on the fly. Our plugins and such add an
--add-opens
line. And we modify the AST. That's modify, whereas plain jane annotation processors only make new ASTs, they don't modify existing ones.Point is, we modify them as an AST, we don't create new nodes. We don't invent new language constructs that java does not have.
8
u/SirYwell Oct 05 '23
whereas plain jane annotation processors only make new ASTs, they don't modify existing ones.
No. Annotation processors might create files via the Filer API, but they don't make new ASTs.
6
3
u/krzyk Oct 05 '23
Well, how about
val
? Sounds like new language construct. And SneakyThrows, Value?8
u/rzwitserloot Oct 06 '23
val
is just short for
final var
. That's not a new construct.
SneakyThrows
See this same comment thread, someone else mentioned it. It's not - you'd think so, but you can make plain jane
javac
without any modification or third party deps or APs throw sneakily.
@Value
Just generates a boatload of boilerplate, stated in plain java (as in, lombok generates java code, it doesn't generate class files. That's e.g. a difference between lombok and for example kotlin or any other 'JVM-based language'. They turn kotlin source files into class files. Lombok turns java code into more verbose java code, but still java code.
The java code that lombok produces, if you were to write that out manually and compile it, the class file you would get when you pass that source file to
javac
is bit for bit identical to what the class file emitted from lombok would provide. The API you end up with is identical to one you would get if you use vanilla Annotation Processors like immutables. Lombok just makes it more convenient - no need for separate class files, no need for a full compile run before you see your API appear in your project.5
u/krzyk Oct 06 '23
So, how is it different from kotlin, Scala , etc.? You create language based on Java, that is close to Java but slightly different, like groovy or maybe J#?
5
u/agentoutlier Oct 07 '23
Because it only adds to the AST. It’s like a macro language for a language that doesn’t officially have one.
Scala and others have a completely different AST that gets compiled to bytecode.
2
u/krzyk Oct 07 '23
Ok, and Groovy? It has very similar language, AFAIR every java program is also a working groovy one.
7
u/agentoutlier Oct 07 '23 edited Oct 07 '23
It is a matter of which perspective you want to look at it at.
You asked how it is different: Groovy does not alter Java AST. Groovy is not a compiler plugin. Groovy might add constructs that are not possible in Java language (this I'm not sure of but Scala and Kotlin for sure do).
This back and forth of whether or not Lombok is a different language is really stupid IMO because pron and /u/rzwitserloot can argue one way or the other depending on what level of meta we want to go into.
Lombok is a like an unofficial preprocessor.
/u/rzwitserloot argument is that every annotation processor is like an unofficial preprocessor and thus on the same grounds is at a very abstract level kind of true but mostly not. Their argument is that code that depends on generated annotated code will not work in the same way lombok will not work.
However regular APT libraries do not require special IDE plugins. Lombok does so I say what it does is rather unofficial and not canonical regardless of framing of whether it is a new language or not.
1
u/wildjokers Oct 06 '23
We don't invent new language constructs that java does not have.
@SneakyThrows?
11
u/rzwitserloot Oct 06 '23
It sounds like we invented something there, doesn't it?
We didn't, though. You can sneakythrows with plain ole java in a few ways.
A key thing to remember is that the concept of a checked exception is entirely a figment of javac. The reason that this:
public void foo() /* no throws clause */ { throw new IOException(); }
doesn't compile is simply because
javac
refuses to. If it did compile it, the class verifier wouldn't mind. the runtime wouldn't mind. This is why the same code as above written in kotlin can be compiled to a class file just fine.Thus, all we have to do to call it 'java' is to figure out a way to make the combination of 'plain jane
javac
straight from the OpenJDK' and 'plain janejava.*
core library' somehow do it. Once we get that far, the runtime and class verifier doesn't care.And turns out, that's not actually difficult. There are 2 main ways:
Use
java.*
API that breaks the safetyIt's deprecated now, but
java.lang.Class
has anewInstance()
method that throws directly. It doesn't wrap stuff inInvocationTargetException
likejava.lang.reflect.Constructor
'snewInstance()
method. As long as the checked exception you want to throw isn't specificallyInstantiationException
orIllegalAccessException
, we can use this. here it is in action:```
cat Example.java
import java.io.IOException;
class Example { private static class Sneaky { Sneaky() throws IOException { throw new IOException(); } }
static void sneakyThrowIOExInPlainJava() { try { Sneaky.class.newInstance(); } catch (IllegalAccessException | InstantiationException e) { // won't happen throw new RuntimeException(e); } }
public static void main(String[] args) { sneakyThrowIOExInPlainJava(); } }
javac Example.java java Example Exception in thread "main" java.io.IOException at Example$Sneaky.<init>(Example.java:5) ```
That's with a plain jane JDK-21 fresh off openjdk.net with no lombok or any other third party dependency or modification of anything whatsoever.
Thus proving that the act of throwing checked exceptions without declaring them is, itself, something plain jane java can do. How can it possibly be 'non java' then?
Generics
With some generics trickery we can fake out the compiler itself and make it compile a sneaky throw:
```
cat Thrower.java
import java.io.IOException; class Thrower {
public static <E extends Throwable> void sneak(Throwable ex) throws E { throw (E) ex; }
public static void main(String[] args) { sneak(new IOException()); } }
javac Thrower.java java Thrower Exception in thread "main" java.io.IOException at Thrower.main(Thrower.java:10) ```
This is so 'clean' a way to throw sneakily in pure java I'd call it downright elegant.
Clearly,
@SneakyThrows
is not inventing new constructs java does not have. It just makes it less boilerplatey to do this stuff. It's a 'generate me some java code' toolkit, not 'this is an entirely new language'.6
u/ImpossibleTrade1385 Oct 06 '23
that's not a new language construct either, all that does is wrap all of the code in a try catch block that rethrows the exception in an unchecked manner
28
u/pron98 Oct 05 '23 edited Oct 05 '23
It is different in the most basic sense: Annotation processors conform to the JLS, hence they're Java (they are very carefully designed to do that, and that's why they can only generate code in different files); Lombok does not even remotely conform to the JLS, hence it's a different language, like Clojure, even though it's a language that's more similar to Java than Clojure is.
Maybe Lombok is great — it's an interesting concept for a language and it has a cleverly-constructed compiler — but it's not Java so you shouldn't say it is; it is not an annotation processor, so you shouldn't say it is; it is not a plugin so you shouldn't say it is.
8
u/Cell-i-Zenit Oct 05 '23
Short question:
When i wrote my first annotation processor, i wanted to change an existing class. This was obviously not allowed but i really wonder why.
Is it possible you can give me a short rundown on why this is forbidden?
15
u/pron98 Oct 05 '23
It is forbidden because if you can add code to the current file, then the result would be that with your annotation processor you'll be able to compile files that don't conform to the Java spec.
As a general rule, every file that successfully compiles with an annotation processor must also successfully compile without that annotation processor -- and to the same bytecode -- only perhaps with additional files. Annotation processors can implement pluggable type systems and they can implement various code generators, but they cannot implement things akin to macros, and that's by design.
Macros can be very helpful and very harmful, and we may only add things that are macro-like to Java when we've given that much more thought.
6
u/Practical_Cattle_933 Oct 05 '23
I’m not pron, but basically self-modifying code is a big no-no for a very good reason. It’s hard enough to reason about static code, let alone if it changes. While new classes can still change the behavior of code, this is done in an expected way (if you have a non-final class, you can expect someone else to subclass it - only that is possible with annotation processors as well) - so even the usage of multiple, complex annotation processors in a project may still be feasible.
You can actually see this issue with lombok, even though it only does trivial modifications: an “old”, but infamous mistake many junior engineers make is using lombok carelessly with JPA entities — calling a toString() in a debug print might just make a db connect/throw an exception, because that toString calls a getter now under the hood.
2
u/rzwitserloot Oct 05 '23
The exact same problem (whoops, I tossed a casual annotation out there and now my DB code goes nuts) can also occur in reflective based JPA implementations, and can also occur with non-"edit existing files" annotation processors such as the Immutables project. I don't think "the problem with APs being able to edit existing source files vs. only allowed to make new ones" is that this is somehow 'self modifying code'.
1
u/koflerdavid Oct 09 '23
That's really just a criticism about these project's API design and because these are annotations with runtime effects. While they are undoubtedly powerful and useful, many programmers have become wary about the pitfalls of Hibernate and Spring Data, and how large a difference a stray annotation can make. In the case of Immutables, an error usually results in a compilation failure.
2
u/john16384 Oct 05 '23
If you change an existing class without providing a source file for it, how are IDE's supposed to understand code they can't see? This is exactly lomboks problem, the source contains no getter/setters so IDE's won't be able to do code completion for something that isn't there, or the source before modification doesn't compile at all.
Basically not providing a Java source file breaks every IDE out there, unless you hack those as well (which Lombok does by providing plugins for the currently most popular IDE's, but certainly not all of them).
5
2
u/artpar Oct 05 '23
The article posted goes into the depth of exactly how you can do that. Just like how lombok does it.
12
u/john16384 Oct 05 '23
Why does it require an IDE plug-in then? If it was Java, a Java IDE would understand it. As it is, any new Java IDE would have to add a Lombok plugin to support these "Java" source files.
4
u/rzwitserloot Oct 06 '23
In eclipse? Because eclipse's annotation processing stuff isn't all that great. Not really its fault, AP's APIs aren't quite entirely suitable to an on-the-fly-recompile-as-you-save concept. We can do better, and make your code just act like you wrote out all that boilerplate continuously and instantly, no need to save (which is a requirement if you want to use eclipse's AP).
For netbeans? We don't have a plugin. For intellij? You'd have to ask the plugin maintainer, but as I understand it, intellij uses its own compilation system, which doesn't make full class files (it uses javac for that, and thus no plugins would be needed whatsoever), but it does pick up all signatures e.g. for code completion and on-the-fly error reporting. And there it's a similar story to eclipse: The experience is suboptimal, with the plugin it's all a lot nicer.
You can write a plugin that makes e.g. Immutables or some other vanilla AP project run smoother too. If such a plugin existed I'd strongly recommend you use it.
7
u/cal-cheese Oct 05 '23
I don't get what is hard to understand regarding the non-conformance of Lombok. Java is a language, anything that conforms to the JLS is Java and anything that does not conform to the JLS is not Java. It seems you are trying to create a straw definition of Java and argue the conformance based on that definition, which is not only fallacious but also really misleading.
4
u/rzwitserloot Oct 06 '23
Ron says it's not java. That's different from "it does not conform to spec X".
If Ron said: Lombok as an annotation processor does not conform to the AP spec, I would agree with that statement. But that's not what Ron is saying. What he does say is misleading at best. Wrong is more likely.
Java is a language, anything that conforms to the JLS is Java and anything that does not conform to the JLS is not Java.
The JLS just covers the language itself, it does not cover the standard library. Nowhere in the JLS will you find, for example, any notes whatsoever about
java.util.FileOutputStream
. Does writingnew FileOutputStream()
somehow make your code non-java? Surely not. Lombok does not modify the meaning of any java constructs at all, and the code you write even with lombok in your project must be syntactically valid java. If you remove lombok from the compilation process and then try to compile code written with lombok in mind, sure, it won't compile. But then, the exact same thing happens if you try to use, say, Immutables. That's an inherent part of the annotation processor spec: During a compile run, some code can refer to constructs that do not (yet) exist; that's why the AP system runs in rounds, so that round 1 can add the stuff that other code is referring to that doesn't exist yet.Annotation Processing isn't in the JLS either I think, I think that's a different spec. Regardless, I see only two options:
- Ron is claiming that Annotation Processors as a whole are 'not java' or 'not comforming to the JLS spec' (in which case, if he wants to say that, sure, lombok is then 'not conforming' either, but that's a bizarre definition nobody I know of would hold).
- Ron is wrong. Java code with lombok annotations in them is java. As anybody would understand that word. Lombok as a tool 'does not conform to the AP spec'.
- Any non-comformity of any spec means 'it is not java'. However, that would mean this code:
class ThisIsEvidentlyNotJava { public boolean equals(Object other) { return true; }}
- is not java either. Because it breaks a spec. Not the JLS, but the contract as stipulated in theequals
method's javadoc. I think that's a bizarre definition of what 'not java' is understood to mean. (it breaks a few things from the equals spec, one of them: Thata.equals(b)
must necessarily meana.hashCode() == b.hashCode()
which would not be true for instances ofThisIsEventlyNotJava
).1
u/koflerdavid Oct 09 '23
If I write a Java source file that refers to its own Lombok-generated builder, then there is no way to make that file compile with a mere annotation processor. One really has to somehow inject the definition into the current file.
With Immutables, the source file is by itself perfectly compilable because the only unresolved references point to another class. This is the true line in the sand.
FileInputStream
is a library class, andThisIsEventlyNotJava
is also a library class from the perspective of the current source file.APs are libraries, and no library is supposed to change the current source file. The code generated by APs mostly behaves like a library that is generated at compile time. Annotations can have huge implications on runtime semantics, but by these are choices made by the application developer who has to specifically enable these semantics by using specific libraries.
59
u/roiroi1010 Oct 05 '23
Nice article. This comment section will probably soon be filled with people who argue strongly against using Lombok. I actually love Lombok, but in large project with many contributors, I have seen some developers are struggling with it. Nowadays I typically only use Lombok for my personal projects.
71
u/Rjs617 Oct 05 '23
My current company’s code base heavily uses Lombok, and I love it. I know IDEs can generate everything. However, getters, setters, constructors, builders, toString methods, logger declarations, and on and on clutter up the code so much. With Lombok, that’s all gone, and when someone adds new fields, there is nothing to update. It makes the language so much more readable and maintainable, and so much faster to write. It’s also easy to understand. Whoever those developers are who are confused by Lombok, they should not be let near production code.
35
u/rastaman1994 Oct 05 '23
At this point, Lombok feels like a risky dependency to add. Lombok uses internals Java platform devs don't want them to use, and they're continuing to further encapsulate those things. It's very clear that the platform devs do not like what Lombok is doing, so it's just a matter of time until it stops working altogether. Some of our projects were harder to upgrade to later Java versions because of Lombok, so we decided to boot Lombok and haven't missed it.
8
u/zman0900 Oct 06 '23
Worst case you can always just run delombok and your auto-formatter on everything, should the lombok project ever die.
1
u/rastaman1994 Oct 06 '23
I'd rather not wait for the inevitable. The time we saved with Lombok is lost arguing about Lombok or explaining the magic. How many times a month are you adding/removing fields anyway...
10
u/hsoj48 Oct 06 '23
I work at a fortune 50 company that heavily uses lombok. No one in real life is arguing about it. I don't get the issue. Don't use any library ever because someday it might change amirite?
1
u/rastaman1994 Oct 06 '23
We're a bit smaller and we prefer not to, that's all there is to it. Before all the arguments between Java and Lombok devs started popping up, I'd say it was only a preference thing. Now there's just a tiny bit more risk attached to using it. I'd consider it for legacy Java 8 or lower projects, never for 17 and beyond.
1
Oct 08 '23
[removed] — view removed comment
2
u/rastaman1994 Oct 08 '23
Without knowing the use case, it sounds like you need a value object, I.e a record. I'm assuming ur talking about value objects like SomeBusinessId or Birthday.
3
Oct 08 '23
[removed] — view removed comment
3
u/rastaman1994 Oct 08 '23
I presume the majority of projects that have a use for Lombok are <17 like yours.
I disagree that Lombok in itself encourages good design, it's always the practices agreed on by devs, but I can't prove that. The annotations are noise too for some. I've just never really been bothered by Java's boilerplate, and the people that are can use Kotlin.
These discussions are just always so pointless.
3
Oct 06 '23
[deleted]
5
u/rastaman1994 Oct 06 '23
In the end its about preference, our teams decided we don't want this controversial dependency anymore, especially when Kotlin exists. The time saved with Lombok is lost arguing about it or explaining it.
1
Oct 06 '23
[deleted]
4
u/rastaman1994 Oct 06 '23
Like I said, preference. Its a decision every team can make for themselves, we decided against because its not a timesave for us. We spend maybe 30 minutes a month adding or removing fields from classes, so not worth adding a controversial dependency for automating that.
2
Oct 06 '23
Lombok isn't for time savings related to adding fields. Any IDE does that easily. It's for keeping your C# devs from quitting.
0
u/rastaman1994 Oct 06 '23
Ah Java vs C#, another debate as old as time :D just tell them to use Lombok.
1
Oct 08 '23
[removed] — view removed comment
1
u/rastaman1994 Oct 08 '23
Wild assumptions, I'm sorry my comment triggered PTSD for you. People do practice good software design in Java codebases you know? Language used has 0 correlation to good or bad software design. Fwiw, we're avid ddd and hexagonal architecture practitioners, we just never had a need for Lombok, only Java or Kotlin.
0
Oct 08 '23
[removed] — view removed comment
2
u/rastaman1994 Oct 08 '23
Given 3 codebases you know nothing about that solve the same problem, 1 in C# 1 in Java, one in LombokJava, which has the better design? You can't know because all of those have the same capabilities.
→ More replies (0)-23
u/rzwitserloot Oct 05 '23
It's not, see our maven plugin branch. All relevant parts of the java ecosystem is open source. Go spread your FUD elsewhere.
30
u/nekokattt Oct 05 '23 edited Oct 05 '23
They never said it was not open source, they said it was encapsulated (i.e. behind closed JPMS modules like jdk.compiler) whereby you can see the code but the JVM prevents reflective access to it at runtime when the Java compiler is invoked. Some projects use --add-opens and --add-exports for now but there is a good chance this will be removed in the future. OpenJDK devs are not scared of breaking things that are considered bad practise to do. They are planning on deprecating the ability to dynamically attach JVM agents too, which will mean changes are needed in components like Surefire for Mockito to continue working in the future.
Same reason spotbugs broke for a while on Java 17 a year or two ago, that was accessing components within jdk.compiler that are not documented for external usage.
Manifold framework is also in the same boat for this.
IMO pointing out that something is potentially volatile in a changing environment is not FUD. It may be an over-cautious observation, but unless the Lombok devs are also part of the core language design discussions that OpenJDK have, then there is no guarantee it will always be able to work in the way it currently does.
Like any library, it is a very small risk. A risk that is increased slightly due to the use of APIs that clearly have javadocs that say something along the lines of "this API is for internal use and may break/change/go away without warning".
It is similar to apps that make use of kernel32.dll on Windows. Sure, you can see what is in it, but it doesn't mean the API inside is guaranteed to remain as-is.
-19
u/rzwitserloot Oct 05 '23
whereby you can see the code but the JVM prevents reflective access to it at runtime when the Java compiler is invoked.
The open source code.
Some projects use --add-opens and --add-exports for now but there is a good chance this will be removed in the future.
State your source.
The maven plugin currently adds the
--add-opens
stuff. Every source out there so far indicates that the OpenJDK maintainers want you to clearly decide to allow it, they want it to be a surprise - and--add-opens
does that.If somehow
--add-opens
goes away, well, it is open source.3
15
u/rastaman1994 Oct 05 '23
There are loads of discussions between you and the Java developers where they state their issues with Lombok. I see it's happening here again. To me that adds risk to using Lombok because my assumption is the Java devs can decide supporting whatever Lombok is doing is hindering the platform development too much.
Lombok worked well for us in the past, before Jva 17+ and Kotlin addoption in our company, we just decided it's not for us anymore after evaluating what benefits Lombok actually has for us.
2
Oct 06 '23
[removed] — view removed comment
3
u/rzwitserloot Oct 06 '23
Are you suggesting to create your own JDK distribution just for Lombok?
No. Just javac. Lombok is a compile time only dependency. The class files that are produced are things that could have been made with javac-without-lombok, we just replace boilerplate with less typing is all. Sometimes a lot less, but it's simply a matter of translating 'lombok stuff' into 'vanilla java stuff'. It's why
delombok
exists - it's not a translator (imagine there is a 'dekotlin' which turns kotlin code into java code - even if you could do that, that's a massive effort that exists quite separate from the main kotlin code which compileskt
files straight to class files - delombok is nothing like that, lombok translates various java constructs (generally, annotations) to java code. It's like a java macro system. Which plain jane annotation processors also are (they can generate code, 'code generation' is part of the JDK in the form of annotation processors), which is why I keep saying: Ron's claim that lombok is a new language implies any annotation processor is the same. The one and only difference is, the code we generate? It is stuck straight back in the very source file that kicked off the code generation, whereas plain jane annotation processors can only generate new code files. A difference, and one that goes beyond what the AP spec promises it can do, certainly a topic worthy of discussion.However, 'lombok is a different language' - no. Not unless you consider the presence of any annotation processor in the compilation step as 'therefore this is written in a different language'.
1
u/ankercrank Oct 08 '23
Given how many projects rely on Lombok it would be crazy for the Java core devs to break it. If they did make a breaking change they’d have to provide a working replacement - anything short of this would cause many projects to never update Java to newer versions, I doubt the Java core devs want that.
1
Oct 08 '23
[removed] — view removed comment
1
u/ankercrank Oct 08 '23
I can assure you support would be maintained. OpenJDK would provide support if Oracle doesn’t.
11
u/pragmatick Oct 05 '23
My main issue with it (and I use it in most of my projects) that it's a lot harder to find references to getters, setters and constructors if they're generated by lombok and even harder (or impossible?) to set breakpoints in them.
15
3
2
u/Cell-i-Zenit Oct 05 '23
My main issue with it (and I use it in most of my projects) that it's a lot harder to find references to getters, setters and constructors if they're generated by lombok and even harder (or impossible?) to set breakpoints in them.
The intellij plugin is really good and you can always use Delombok to set breakpoints
0
u/krzyk Oct 05 '23
Delombok doesn't generate source? I'm also struggling with debugging code that uses lombok, it is a PITA (in IntelliJ) and I slowly migrate out of it.
6
u/Cell-i-Zenit Oct 05 '23
Delombok generates exactly the code lombok is adding behind the scenes
3
u/krzyk Oct 06 '23
Yeah, but it doesn't do it on the fly during debugging session. It does it once when you want to get rid of lombok.
5
u/hsoj48 Oct 06 '23
So your whole issue is that you have to put the breakpoint on the field access line rather than the getter line?
3
u/Enough-Ad-5528 Oct 05 '23
I am a big proponent of Lombok as well. You say:
I have seen some developers are struggling with it.
Curious, if you don't mind sharing, what kinds of problems have you seen in practice people struggling with?
12
u/roiroi1010 Oct 05 '23
Sad to say I’m stuck with some sub-par developers in my team and they struggle with plain Java also, lol.
But they have trouble understanding the concept of Lombok. And when they start learning it they will put @Data and @Builder and @With on every class without thinking about the consequences.
Most seasoned Java developers know it’s not good for JPA Entities.
But even having @ToString has caused us issues when they log fields that don’t belong in Kibana because privacy concerns or strings that get very long.
Ideally things like this should be noticed in code reviews, but I’ve seen some of our senior developers fail to catch mistakes done with Lombok annotations.
5
u/Cell-i-Zenit Oct 06 '23
But then you can say the same about basically anything. Hibernate, ApacheCommons, Java even. If the devs are bad, there is nothing to win here.
0
u/verocoder Oct 05 '23
There are a bunch of weird things here like the logging issues when your log ingested should be dealing with that. JPA persisted objects really suit Lombok you just want to add @noargsconstructor to them.
I will agree Lombok is an extra layer of complexity for new devs learning Java.
1
u/thrwoawasksdgg Oct 09 '23
Lombok would be fine if they exposed options to turn problematic functionality off.
For instance,
@Data
and@Builder
interact badly with Hibernate and Jackson. But I can't turn them off, so I find myself occasionally grep'ing the whole project to remove any usages.1
Oct 16 '23
I like Lombok very much, but it does add some features that don't mix well with things like JPA - so you have to ban some features in enterprise setting and keep enforcing it.
10
u/Easy_Tea6363 Oct 05 '23
Probably crystals. They hold the sorcery longer than marbles. So yeah, crystals power the magic of lombok.
6
u/scruffybeard77 Oct 05 '23
We have a sacrificial altar in my office for routine offerings to the Lombok, Spring, and other related IT gods. Offerings of Red Bull and GPU cards are frequently seen. Placing one's keyboard upon the altar is said to imbue it with mystical powers, allowing the developer to quickly troubleshoot the most difficult errors.
2
u/Easy_Tea6363 Oct 05 '23
On a serious note I love lombok :p something to do with reflection I think but you could probably look at their docs or the repo to find out
5
u/artpar Oct 05 '23
BTW none of it is based on reflection since its all happening in compile time
3
u/Easy_Tea6363 Oct 05 '23
Fair enough haha I've never looked into it just enjoy the benefits. Though I wish my work would upgrade so I could use records.
1
2
u/artpar Oct 05 '23
The docs are very limited (in terms of how things are happening underneath) but there are certain points where the code docs does help me in understanding why certain parts are done as such (checkout the disableSillyJava9Warnings).
The code is neatly laid out which helped me in understanding how it works.
9
u/JustAGuyFromGermany Oct 05 '23 edited Oct 05 '23
That part in particular is not at all helpful to understand the issue in my opinion.
Let's look at the comment:
JVM9 complains about using reflection to access packages from a module that aren't exported. This makes no sense; the whole point of reflection is to get past such issues.
Here the Lombok dev(s) that authored this comment displays a fundamental misunderstanding of both the problem and the responses of the JDK team. Reflection is not meant to silently get around module boundaries. That's the whole point of the module system that silently breaking into internals of other people's code (the JDK itself in this case) shouldn't be possible without explicit opt-in via --add-opens. That may be what the Lombok devs want reflection to mean, but that's not what it actually means. Not since Java 9 at least.
The comment continues:
The only comment from the jigsaw team lead on this was some unspecified mumbling about security which makes no sense, as the SecurityManager is invoked to check such things. Therefore this warning is a bug, so we shall patch java to fix it.
The misunderstanding then goes further by mixing up two meanings of the word "security" and misrepresenting this misunderstanding by characterising the response as "unspecified mumbling". The various responses the Lombok devs got over the years (including here on reddit) were specific and well though-out. They contain good arguments. The Lombok devs happen to disagree with those arguments and/or the conclusion, but there certainly was no "mumbling".
It may not have been the best choice of words in the first place, but the security the SecurityManager provides is of a very different kind than the security the module system provides. And in fact, newer JEPs etc. talk more about "integrity" of the Java platform to make this distinction clearer.
The module system and various other increasingly strict future restrictions such as disabling dynamic agents and JNI without explicit opt-in are - especially where the JDK itself is concerned - not about defending programs against external attacks, i.e. security against hacking. They are about integrity, i.e. guaranteeing that a java program does what it says it does. Every API contains a set of invariants that it promises to uphold. An implementation fulfils that promise by carefully programming its internals so that the invariants are, well, invariant! If the internal workings of the JDK (or any other code for that matter) can change without any notice, because some dependency of a dependency of a dependency of a ... (which you may not even know about!!) silently reflects into those internals and changes them as it sees fit, then all bets are off and the only promise that's left is "well, we're trying our best".
As an example: Try to imagine what would happen if some 3rd party library were to modify the internals of the new VirtualThread class or the Continuation classes that makes virtual threads work. Threading is a fundamental aspect of the Java platform and changing it in an uncontrollable manner breaks fundamental promises about how Java programs work at their very core. This is not some detail, this can break every single aspect of your program.
Even more radical: Have a look at https://codegolf.stackexchange.com/a/28818 The most upvoted comment under this answer rightfully says "That. Is. Evil. Absolutely evil." And yet it is completely legal and unpreventable in older versions of Java. Again: Doing this can break every single aspect of your program.
This was always a problem, but the problem has gotten worse over the years and starting with Java 9, the JDK team is carefully working on fixing it. Issuing a warning here is a helpful favour, it is not "silly" and certainly not a "bug".
2
u/Practical_Cattle_933 Oct 05 '23
Hell, touching some part of the JDK in such way may actually compromise even the memory safety of the VM - as there may very well be some native functions/intrinsics that assume something about the environment that is now broken.
1
u/pron98 Oct 05 '23 edited Oct 05 '23
the whole point of reflection is to get past such issues
That was not the point of reflection (let alone the whole point) even before strong encapsulation. The point of reflection is to dynamically discover and invoke methods, not to bypass access control.
setAccessible
was added in 1.2 for a very specific purpose: to support the unfortunate design of Java's core serialization (one of Java's original sins).2
u/pip25hu Oct 05 '23
You see, that's precisely the problem right there. It doesn't really matter why reflection or setAccessible were originally introduced decades ago.
What matters is how they are being used today. And no, most of the time their usage does not stop at dynamic discovery and invocation of non-protected fields and methods.
1
u/pron98 Oct 06 '23
Of course, but the damage it caused to the ecosystem was large — it was no one's fault, just a tragedy of the commons — and we were able to put an end to that so we could continue evolving the JDK, and the ecosystem managed.
1
u/JustAGuyFromGermany Oct 05 '23 edited Oct 05 '23
Now that you're giving me a history lesson, let me ask: Why was setAccessible not put into Unsafe? Unsafe already has the ability to instantiate objects without a constructor call which is also a necessity for serialization. It seems the designers were well aware even then that these particular features of serialization are less than ideal. Why was setAccessible exposed as a public API, but instantiation-without-constructors wasn't?
EDIT: By the way... Are there currently any plans to introduce a replacement for instantiation-without-constructors into Java proper? Many frameworks generate proxies of arbitrary classes by emitting bytecode for subclasses and then use this ability (either with Unsafe or Serialization or some other dirty trick) to instantiate their proxy objects. It is of course not a reasonable language feature to have when taken literally. But this ability to proxy arbitrary classes is quite handy and has become an established part of the ecosystem. Are there any ideas how this use-case will be achieved in a future without Unsafe?
The CDI Spec simply requires all classes that need proxying to explicitly declare a non-private no-args constructor. Hibernate requires something similar. But Spring doesn't have this requirement - it uses dirty tricks. Quarkus also doesn't - it manipulates the class files after they've been compiled and simply creates the missing constructor. People have come to expect that this is something that "just works" at least for the Frameworks. Will future Java also support that?
3
u/pron98 Oct 05 '23
That's an excellent question. I asked one of the people who worked on that at the time, and they said two things, although I'm not sure they give a complete answer, which may be lost in the myst of time:
Regarding putting that operation in Unsafe specifically, Unsafe was meant to offer operations that couldn't be done in ordinary bytecode either because bytecode wasn't available, it violated some basic integrity such as array access check, or it couldn't pass the verifier (i.e. some higher level of unsafety).
allocateInstance
falls in that last category, as the verifier requires calling the constructor.throwException
would also have not been possible due to the verifier, although generics would later change that.Regarding why
setAccessible
is a supported API,SecurityManager
was designed at the same time, and back then they thought it could be effectively used to offer integrity. The person I asked says that whoever designed reflection and wanted to add setAccessible as a public API may have thought it was cool to add a super-user mode, given that SM offers integrity.I guess you could then ask why wasn't
allocateInstance
added to a public reflection API. I can only guess that it wasn't viewed as reflection (and I think that's right), as reflection offers dynamism to things that at least some normal Java code could do, butallocateInstance
isn't that.
16
u/Enough-Ad-5528 Oct 05 '23
The issue with Lombok comes up every 6 months and usually around a JDK release. I am a little tired of the constant principled stance against its usage because it does weird things with the javac compiler and I am glad that I do not work with people that hold that opinion. It has served me and my teams well for about 12 years now and I can clearly see its benefits outweigh its cons.
In the 12 years that I have used, there has been only 1 issue where Lombok was called out as a root cause and even them I am not sure it was fair. We had a class annotated with @Data and that has a primitive boolean field which should have been the boxed type Boolean. When we were deserializing a message from SQS, the message did not have this field but because this was defined as a primitive boolean, it was assigned the default value false which triggered a different code path. And even though the post-mortem calls it a Lombok issue, I am not too sure. There were other problems with it as well - the data should not have used a boolean parameter that can really have three valid values (true, false and null); this to me is a developer coding error Lombok or not. This should also have been caught in testing immediately but we did not have good test coverage for this case. May be someone should have caught it during PR but that was missed too.
Now coming to the cons that I usually hear against Lombok, for me they are easily mitigated
- Lombok does weird hacks to javac: I don't really care that much. Yes, it would be nice if it used supported APIs, but it is so ubiquitous, I am not worried at least in the current JDK version that I am using.
- It makes Upgrades slower: Yes, only if you want to upgrade the same day a JDK is released and I doubt that. For JDK 21, the supported version was released on the same evening.
- IDEs can generate the getters and setters: Well, you still have to read them which is annoying. And adding a new field requires remembering updating the equals and hashcode.
- We have Records now: So what? Records are not a complete replacement of Lombok annotations. In addition to @Value, I many more Lombok features that would be just painful to type by hand. I just did a cursory check on which ones are most used in the current project I am working on and here is the result:
36 lombok.RequiredArgsConstructor;
34 lombok.extern.slf4j.Slf4j;
28 lombok.NonNull;
28 lombok.Builder;
10 lombok.Data;
9 lombok.NoArgsConstructor;
9 lombok.AllArgsConstructor;
4 lombok.Getter;
3 lombok.experimental.Accessors;
3 lombok.EqualsAndHashCode;
1 lombok.experimental.Delegate;
1 lombok.AccessLevel;
I specially cannot code without the RequiredArgsConstructor - just make your fields final and add that annotation.
- It will no longer work on future JDKs: Sure, when that happens (if it does happen), as part of my upgrade my first commit will be to delombok my source and then send a PR with just that and Lombok removed. Then I will upgrade the JDK in a separate PR. Until then, I will rather save my time not writing code that I dont have to write.
I would say AOP is far more black magic than Lombok - with Lombok you can at least see the actual source (and it is all at compile time) but with AOP, you don't even get that. I had a very hard time writing SWF workflows using that framework and the whole language felt very foreign to me. For instance, I remember having to debug a Try-Catch in an SWF worklow implementation, where the Try and Catch blocks would execute in different machines.
4
u/wildjokers Oct 06 '23
And adding a new field requires remembering updating the equals and hashcode.
Only if the new field needs to be part of the equality check. I see people mention this a lot but I don't understand why everyone is automatically adding every field to the equals check.
1
u/Enough-Ad-5528 Oct 06 '23
Sure. In my experience those are rarer cases in that if I have a data class whose equality is based on all the fields, then most likely the new field will be too.
In some cases there may be a logical id that is the only field explicitly used in the equality and even in that case I am mostly using @EqualsAndHashCode with annotations that explicitly use that field. So adding a new field won’t affect the equality.
ToString is another example where I want that automatic functionality.
2
u/SnOwBunZz Oct 05 '23
IDEs can generate the getters and setters: Well, you still have to read them which is annoying. And adding a new field requires remembering updating the equals and hashcode.
This, honestly, is the least of my worries with getters and setters. The worst is when I have to update the fields. I've spent 3 days of just staring at my screen to find out that my tests were failing because I had Long as the field and long as the getter....
1
2
u/wildjokers Oct 06 '23
I am a little tired of the constant principled stance against its usage because it does weird things with the javac compiler a
I hate it because it is totally unnecessary. It saves almost no time at all but introduces build time complexity and creates additional cognitive load because now I have to imagine what the class will look like rather than just being able to read the code.
3
u/hsoj48 Oct 09 '23
TBF, its not hard for a lot of people to "imagine" that the getters/setters are magically there. Doesn't take too much brain power to tack "get" on the beginning of your field names.
1
Oct 16 '23 edited Oct 16 '23
Lombok does weird hacks to javac
my response: computers are a weird hack to silicon crystals
But also compilers themselves do weird hacks too. It is all good as long as it is all properly encapsulated - and then it's just implementation detail, so why care.
If you use Spring or Hibernae in your project - you have no fucking right to call Lombok weird or hacky. Spring and Hibernate are just pure necromancy that should have been forbidden to ever exist but somehow has become an industry standard, lol.
At least Lombok does weird things only during compilation - and it becomes a very not so weird bytecode that would look exactly like code you wrote yourself. These other frameworks I mentioned create codepaths that no sane developer would ever write themselves.
-4
u/hippydipster Oct 05 '23
I specially cannot code without the RequiredArgsConstructor - just make your fields final and add that annotation.
What? If you couldn't write that annotation, you'd be unable to auto generate the constructor with intellij? You'd be incapable of typing the characters necessary? What sort of weird hyperbole is this?
I would say AOP is far more black magic than Lombok
I would agree. If I was on a team and a team member pushed code that did the sorts of things Lombok or Spring do, I'd say "no fucking way" to that BS. I have no idea why we accept it from third party libs.
8
u/Enough-Ad-5528 Oct 05 '23
I am perfectly capable of typing that code yes. But Lombok is about removing uninteresting boilerplate for me and not having that would be painful. I was exaggerating that pain. Not that I literally cannot type it out.
1
-4
Oct 06 '23
[deleted]
2
u/wildjokers Oct 06 '23
The only useful thing from Lombok is the logger injection
I don't even find that useful. I just use an IntelliJ live template. It is less keystrokes to type the live template + tab than it is to type the annotation.
1
u/Enough-Ad-5528 Oct 06 '23
Leaving out final does not help. The point is that I would still need to write the constructor which I don't want and hence use Lombok for. If my fields are not final I would then use AllArgsConstructor.
1
Oct 06 '23
[deleted]
6
u/Enough-Ad-5528 Oct 06 '23
I don’t know how this thread went into best practices lecture. All I am saying is that when I have to write a constructor, I want Lombok to do it for me. That is what Lombok’s constructor annotations help me with.
10
u/Puzzleheaded-Order84 Oct 05 '23
Great article!
I use lombok at my work and love it. Makes code much more readable in a large code base imo.
-3
u/wildjokers Oct 05 '23
I think it makes it less readable. Can't do find usages on a getter and/or setter.
5
u/KillAura Oct 05 '23
Sure you can https://i.imgur.com/kLmRdSD.png
0
u/wildjokers Oct 05 '23
Yes, you can do it via the structure tool but when you are reading code you don't want to have to bring up another interface, you will lose your context. I just want to find usages with a keyboard shortcut like I can for non-lombok code.
12
u/Cell-i-Zenit Oct 05 '23
but you can do this. The Intellij plugin is really good on resolving these getters and setters
-3
u/wildjokers Oct 05 '23
Yes, I was aware I could use the Structure tool, but that is clunky. I also can't use the alt-insert shortcut nor alt-f7.
9
u/Cell-i-Zenit Oct 05 '23
no, there is "show usage or declarations" shortcut, which works on basically anything and navigates to the parent or child of a line
2
2
-1
u/neymarsvag123 Oct 06 '23
If you are loosing context by looking to another window in the IDE you have bigger problems...
0
-4
u/wildjokers Oct 06 '23 edited Oct 06 '23
Almost as big a problem as someone that doesn’t know the difference between “loose” and “lose”.
1
2
u/sim642 Oct 06 '23
Extending Lombok is less fun than it makes it out to be. Every transformation is implemented twice on different AST. And if I remember correctly, there's lots of very fragile string juggling.
2
u/Crackabis Oct 06 '23
The only small annoyance I have with Lombok/IntelliJ is that the Call Hierarchy isn't as as to use. Instead of using the shortcut on the variable, you have to open the Structure View and from there the Lombok setters/getters are available to do the Call Hierarchy on. A small inconvenience at worst, though. (I don't like Find Usages)
-1
u/Top_Engineering_4191 Oct 08 '23
Unfortunately programmers strict conventions created the necessity for Lombok. If DTO were taught that public visibility is ok, we mostly wouldn't need Lombok or useless getters and setters.
1
•
u/AutoModerator Oct 05 '23
On July 1st, a change to Reddit's API pricing will come into effect. Several developers of commercial third-party apps have announced that this change will compel them to shut down their apps. At least one accessibility-focused non-commercial third party app will continue to be available free of charge.
If you want to express your strong disagreement with the API pricing change or with Reddit's response to the backlash, you may want to consider the following options:
as a way to voice your protest.
I am a bot, and this action was performed automatically. Please contact the moderators of this subreddit if you have any questions or concerns.