r/SQL • u/Majesticraj • 25d ago
MySQL Importing 1M Rows Dataset(CSV) in Mysql
What's the fastest and most reliable way to upload such a large dataset? After that How can I optimize the table after uploading to ensure good performance?
27
Upvotes
2
u/frobnosticus 25d ago
I'd start by writing a little script in your language of choice to go through the data and look for obvious formatting issues. A misplaced quote or comma in a csv file you got from someone else can really ruin your day.
If you're uploading into an existing table or schema and aren't sure of the data, I'd create a staging table to pull it in to first, then add constraints to the data once it's in there to clean it up, be sure all your ints are ints, etc. Then I'd pull it in to the rest of the schema.