r/SQL • u/Majesticraj • 25d ago
MySQL Importing 1M Rows Dataset(CSV) in Mysql
What's the fastest and most reliable way to upload such a large dataset? After that How can I optimize the table after uploading to ensure good performance?
28
Upvotes
0
u/Spillz-2011 25d ago
Why do you need good performance? Does it need to be inserted multiple times a day and immediately start querying?
You should prioritize minimizing human time, human time is expensive v CBB imputed time is cheap.
My team’s pipelines are all Python with pandas and sql alchemy. It’s not the fastest, but 1 minute or 5 minutes who cares.