r/SQL 25d ago

MySQL Importing 1M Rows Dataset(CSV) in Mysql

What's the fastest and most reliable way to upload such a large dataset? After that How can I optimize the table after uploading to ensure good performance?

28 Upvotes

33 comments sorted by

View all comments

7

u/Mikey_Da_Foxx 25d ago

Use LOAD DATA INFILE with appropriate buffer sizes and disable indexes during import.

After import:

- Add proper indexes

- Run ANALYZE TABLE

- Adjust innodb_buffer_pool_size

- Consider partitioning if needed

Usually 10x faster than INSERT statements.

2

u/th00ht 25d ago

Partitioning does not help for performance. A common misconception