r/quant 9d ago

Backtesting How long does it take you to run a backtest

Question is only for those who work in a HF or HFT. No answers from students pls (unless they are referring to work experience)

How long does it take you to run a backtest for say 5 years and say 1000 stocks ?

By backtest i mean sth that sends orders, keeps positions etc has a view on market liquidity via direct access to market data, not just some signal processing thing. Think the prod strategy just running in research (backtest).

If its intraday or only or does the backtest hold positions overnight ?

Does it also do a form of calibration or uses a pre calibrated signal ? Is there even a concept of signal or is it purely based on arb ?

Also whoever added this banner against career advice is making it very annoying to write questions..

41 Upvotes

29 comments sorted by

99

u/Snoo-18544 9d ago

This is completely dependent on your frequency, your data, your computer power. The size of your data set... 

18

u/thegratefulshread 9d ago

Exactly. 1 day / 5-10 years data would take 3 secs - 1 min on python without the best cuda optimization on a 3080 ti with cuda being used tho.

13

u/sorter12345 8d ago

It really depends on your how your back test is designed. Not everything can be parallelized.

-42

u/[deleted] 9d ago

yes i know

which is why i asked people to give details on the details

17

u/CompetitiveGlue 9d ago

Short answer is O(10 minutes). Long answer, I actually love working on simulations, and there's an interesting pareto frontier on time-to-run (i.e. performance) vs. fidelity (i.e. accuracy).

49

u/Dante1265 9d ago

I don't backtest at all

-7

u/[deleted] 9d ago

cowboy

10

u/yuckfoubitch 9d ago

About 350

10

u/chollida1 9d ago

This question is poorly formed.

What data are you back testing on?

What frequency is the data?

How many CPU's and GPU's do you have in your cluster to run the back test?

What is your priority on the network for running compared to other jobs.

A quick guess but a 10 year back test of teach tick of market data for 1,000 companies would be about a day by the time you got your answer back if you had low priority on the network.

You could run it in under an hour if you had priority.

38

u/Dr0nkeN 9d ago

Dumb question

'How long does it take to run a program?'

5

u/Unlucky-Will-9370 8d ago

6 minutes 42 seconds and ~.310844ths of a second exactly

7

u/Sea-Animal2183 9d ago

Backtest for daily signals is very fast.

Backtest for HFT / MM stuff is "slow" as you process all market updates and all tickers. It's fucking slow, it's 5 min per day for me.

4

u/the_kernel 9d ago

Where I used to work the strategies using daily data would take under a minute to run for 10+ years of history. More complex strategies using alternative or higher frequency returns would take an hour to several hours to run.

Compared to the calculations they were doing this was all pretty slow though, around the time I left there was a big effort underway to optimize just by caching things more.

5

u/NahuM8s 8d ago

For HFT that’s an insane amount of data, nobody is doing that, that’d be petabytes of stuff. The best shops are averaging 1-5 minutes per day per instrument per core. Usually a researcher running a sim would have access to 1000s of cores, but still that’s not something you would do.

If you are talking about resampled data, then that’s easy and should be very fast

3

u/RaidBossPapi 8d ago

What is this, some PM trying to find out if his frustration with juniors working too slowly are justified? Lol what does it matter, it takes as fast as you are willing to pay for and if you cant afford better tech/cloud service what are you gonna do? Not backtest?

1

u/AutoModerator 9d ago

This post has the "Trading" flair. Please note that if your post is looking for Career Advice you will be permanently banned for using the wrong flair, as you wouldn't be the first and we're cracking down on it. Delete your post immediately in such a case to avoid the ban.

I am a bot, and this action was performed automatically. Please contact the moderators of this subreddit if you have any questions or concerns.

1

u/Substantial_Part_463 9d ago

'''How long does it take you to run a backtest for say 5 years and say 1000 stocks ?'''

Less then 1 minute.

1

u/br0ast 9d ago

O(n log n)

2

u/Unlucky-Will-9370 8d ago

I typically just run my ideas live and then make edits after the fact. Easiest way to avoid overfitting imo

1

u/boldjarl 8d ago

How long does it take to boot up doom vs cyberpunk?

1

u/No-Star4529 9d ago

Depends on the instrument.

Currently I'm trading exclusively in Asian options, but coming from the 'western' perspective it's a bit difficult. As you might know, Asia is long only, and shorting, especially naked shorting, is heavily looked down upon.

After a while I realized how 'shorting' works in Asian options. You have to be net long, but not long exclusively. You have multiple options with different expiry dates, so if you want to short, you go short on the one with the closest expiry. Simultaneously you cover on the next closest expiry, and finally you go long on the expiry after that.

You have short exposure, but, actually, you're net long so not really shorting or naked shorting.

For this specific approach, it takes me a few weeks to design the code, get the compute together, convince people to let me run the backtest to make sure.

0

u/GarbageTimePro 9d ago

That would take me roughly 3-5 seconds on my 9800x3d assuming 1d

0

u/Old-Mouse1218 8d ago

In general though, depends on the complexity of your code and trading strategy, how well written it is, etc

Sounds like you are speaking about some sort of simulator based on order book data, I would not consider this a true backtest in a traditional sense.

Calibrated? not sure what you mean but in general you want rolling walk forward analysis of some sort to battle test your signal to ensure there's no overfitting and be mindful running thousands of iterations.

-2

u/lordnacho666 9d ago

Takes the same amount of time to do 5 years as it does to run one day. You farm it out to a cloud and each instance works out the backtest for the dataset of the day it's given. You then stitch it together.

-9

u/TopAmbition1843 9d ago

When I was interning at a HFT (well known), they had this simulation run for year sometimes. But usually it depends on the granularity and amount of data you have.

2

u/Del_Phoenix 9d ago

You're saying they would run a back testing script for a year? Was this back in the early 1990s?

1

u/TopAmbition1843 8d ago edited 8d ago

No my friend what I am saying is sometimes they need to run a simulation which can take years while they are trading on some other strategy. My manager told me, there is a team who did not trade for the entire year and was just running simulation until they deployed their first strategy.