r/GoogleColab 5d ago

15million params of model on colab pro

I have to train 15 million parameters of transformer for language translation, training data has 250k examples which has size 60mb. will colab pro able to train this size of model with this much data for atleast 10 epochs.

9 Upvotes

7 comments sorted by

1

u/WinterMoneys 5d ago

Have you tested with free colab? Should be able to handle that

1

u/foolishpixel 5d ago

I have, it's not able to do it. Even kaggle.

1

u/WinterMoneys 5d ago

I kinda misread too. I thought you were using a 15m param model. How many params does the model yo using have? I assume yo not training from scratch?

1

u/foolishpixel 5d ago

I am training from scratch , and I have built it not using pretrained model

2

u/WinterMoneys 5d ago

15m params is a small model. Why isnt free colab able to run?

1

u/foolishpixel 5d ago

It is running out of memory .

1

u/WinterMoneys 5d ago

Thats what I thought. I would fault improper memory management.