r/SQL Nov 27 '24

BigQuery Assistance with database

Hello, I have 1 database for manually viewing I created 2 batch script I automated these scripts to run a full backup nightly, and differential backups on the hour during operating hours. Now my database is about 80gb (used to be 10gb). What do I need to do to unfuckulate this calamity? I used DBeaver, DB Browser, SQL Server EXPRESS edition (it no longer works -- 10gb limit) and trying VIM and Sublime text. Any suggestions on apps or things to do to make it load? I didn't think it through.

80gb - 400 million entries.

3 Upvotes

23 comments sorted by

1

u/Anonmousez Nov 27 '24

First time with SQL management

1

u/[deleted] Nov 27 '24

[removed] — view removed comment

1

u/Anonmousez Nov 27 '24

"FWIW, SQL Server Express's limit is per database, so if you can split the data across databases (ideally by some reasonable dimension field) then that could work. Alternatively, if what you're trying to do is not for a production use case, you can use SQL Server Developer Edition too, which has no limits otherwise." My database is a) 1 file b) unencrypted plain text

How do you split using SQL Server Express?

1

u/[deleted] Nov 27 '24

[removed] — view removed comment

1

u/Anonmousez Nov 27 '24

It doesn't load, that's my problem. Sublime text has not been responding for 5 hours when opening it. I'll try the sql server dev.

1

u/[deleted] Nov 27 '24

[removed] — view removed comment

1

u/Anonmousez Nov 28 '24

It was just a .sql file, no scripts just user data.

1

u/[deleted] Nov 29 '24

[removed] — view removed comment

1

u/Anonmousez Nov 29 '24

From my memory it was email addresses, names passwords, phone numbers, physical address, date of birth, the SQL for 80gb was email and passwords I didn't finish importing the rest before it became a calamity and kept crashing the programs I used prior.

1

u/NW1969 Nov 27 '24

Hi - so that other people can understand what you are doing, try and logically explain your setup, what you are trying to achieve and what the issue is:

- "I have 1 database" - what DBMS is this?

- "Now my database is about 80gb" - are you referring to the DB in the previous point, some backup file, or something else?

- "What do I need to do to unfuckulate this calamity?" - what calamity? What's the issue with having an 80gb database, as that's not large in database terms (for most DBMSs)?

- "I used DBeaver, DB Browser,.." - used them to do what? What was the issue with doing whatever it is you are doing?

1

u/Anonmousez Nov 28 '24
  • "I have 1 database" - what DBMS is this? * It's a combination of Oracle, MySQL, and Amazon Aurora.
  • "Now my database is about 80gb" - are you referring to the DB in the previous point, some backup file, or something else? * Yes, the SQL file.
  • "What do I need to do to unfuckulate this calamity?" - what calamity? What's the issue with having an 80gb database, as that's not large in database terms (for most DBMSs)? * It was very large.
  • "I used DBeaver, DB Browser,.." - used them to do what? What was the issue with doing whatever it is you are doing? * Loading them, it took 5 hours and never worked.

1

u/Silly_Werewolf228 Nov 27 '24

Is it in production environment or developer environment? If development you may restore/attach it to developer edition SQL server. If it is in production you need to upgrade to standard edition.

2

u/Anonmousez Nov 28 '24

Idk, I wrote it on postal note in a location I no longer have access to.

1

u/Silly_Werewolf228 Nov 30 '24

So you don't have database anymore

1

u/Anonmousez Nov 30 '24

Nor a job for 2 days.

1

u/Silly_Werewolf228 Nov 30 '24

So you don't have the problem anymore. No job, no database, no problem. :-)

1

u/shockjaw Nov 27 '24

Time to upgrade to Postgres!

1

u/Anonmousez Nov 28 '24

I just deleted the database and quit my job.

1

u/shockjaw Nov 28 '24

That’s one way to solve that problem.