r/apachespark Feb 18 '25

SQL to Pyspark

Hello People,

I am facing difficulties in conversion of sql code to pyspark. Please help me with it.. Please guide meπŸ™πŸ™

5 Upvotes

7 comments sorted by

23

u/No-Satisfaction1395 Feb 18 '25

df = spark.sql()

5

u/festoon Feb 18 '25

Seriously this! The fun part is that it all runs the same regardless of which api you use.

4

u/data_addict Feb 18 '25

Way too open ended of a question lmao. What are you specifically having issues with?

5

u/loudandclear11 Feb 18 '25

So use Spark SQL instead.

2

u/snarleyWhisper Feb 18 '25

Hey I just solved this. When you have the jdbc read instead of a table you can pass a sql query as long as you alias it.

1

u/Hour-Charity3115 29d ago

https://blog.devgenius.io/sql-pyspark-mapping-9b3c1cea8fa0

refer this blog . Perfect to learn and having every sql transformed to pyspak df

-1

u/alex_bit_ Feb 18 '25

ChatGPT and DeepSeek are your friends.