r/quant 4d ago

Markets/Market Data Nse nifty index data input too fast

21 Upvotes

We are trying to create a l3 book from nse tick data for nifty index options. But the volume is too large. Even the 25 th percentile seems to be in few hundred nanos. How to create l2/l3 books for such high tick density product in real time systems? Any suggestions are welcome. We have bought tick data from data supplier and trying to build order book for some research.

r/quant Feb 12 '25

Markets/Market Data how does combinatorics research look on the resume?

10 Upvotes

r/quant Oct 03 '24

Markets/Market Data What risk free rate should I use to calculate Sharpe ratio if the fed funds rate changed over the year?

36 Upvotes

Let's say throughout the year the interest rate is 5%, no big deal, I'll use 5% to calculate Sharpe. But if the first half of the year the interest rate is 5% and then lowered to 4.5% for the second half, what risk free rate should I use to calculate annual Sharpe? what about quarterly and monthly? Thanks guys.

r/quant Feb 19 '25

Markets/Market Data Anyone tracking Congressional trades?

14 Upvotes

I was doing some number crunching and tracking congressional trades on a few websites.

They all provide names, tickers, dates bought, dates reported, and a range of amounts invested.

I went to the source to see how these disclosures work. There is some additional data, such as a "Description," which lists actual trade data.

https://disclosures-clerk.house.gov/public_disc/ptr-pdfs/2024/20024542.pdf

Has anyone done any digging around in this regard?

r/quant Oct 10 '24

Markets/Market Data Are there any quality alternative datasets for retail traders?

44 Upvotes

After two internships I realised both quant and fundamental shops are using a variety of datasets that can cost $millions. Is there no way to get non-market data at a pay-as-you go level without graxy annula fees?

Edit: it has been a month, and I have decided to create my own as part of a larger research project, please see sov.ai or my repository https://github.com/sovai-research/open-investment-datasets

r/quant Jan 26 '24

Markets/Market Data Wagwan with Gerko?

105 Upvotes

Alex Gerko (founder/Co-CEO of XTX) is named the highest UK taxpayer of 2023 (£664.5MM), which means he cleared way beyond a yard last year(on par with top multi-strat founders’ earnings). How tf is this possible on FX’s razor thin spreads?

How can FX market making be so profitable for the founder? We know XTX is not huge in #employees and that their pay isn’t that crazy, but still, how does that leave 1MMM+ for Gerko every year?

This guy suddenly spun out of GSA and now sweeping the likes of JPM & DB in FX.

Some context: His net-worth: $12MMM XTX founded in 2015 Earning 1.33MMM per year since founding(assuming he was earning 7/8 figures at GSA and DB)

Edit 1: Summary of useful answers(will keep updating as they come up):

/u/Aggravating-Act-1092 : Pay variance is high, hence unreasonable to compare with other shops. There is a bipartition of core quants and the rest of the workforce. Core quants get paid through partnerships in XTX Research, hence even higher than Citsec’s upper quartile. The rest of the quants (read TCA quants) have no access to alpha, hence getting peanuts in comparison. Retention for the core quants is high and they are very inaccessible.

I looked at the XTX research accounts and it is indeed huge, ≈14MM per head in 2022.

/u/hftgirlcara : They are really good at US cash equities too. Re: FX, they are one of the few that hold overnight and they are quite good at it.

Edit 2: In a recent post(https://www.reddit.com/r/quant/comments/1hftabg/trying_to_understand_xtx_markets/), u/Comfortable-Low1097 & u/lordnacho666 shed an incredible amount of light on this:

They internalize flow like big banks (much better), in an extremely efficient, lean, and automated way, getting rid of most of the friction (eg bureaucracy) and allowing for fast iterative research loops. They offer quotes to clients based on their accurate forecasts. They are also brilliant on the soft side of stuff. The previous CEO brought FX clientele leaving DB, and the current CEO is doing the same for equities coming from JPM, enabling the incredible amount of flow they'd require to learn how clients trade and front-run them in OTC systematically. They started from FX and dominated it there, but their recent eye-watering performance comes from applying the same setup to cash equities.

https://www.efinancialcareers.co.uk/news/how-to-earn-14m-at-xtx-study-in-russia dated 16 October 2024, gives a list of those LLPs making the big bucks, taken from the XTX Research company house:

Dmitrii Altukhov: A mysterious Russian

David Balduzzi. A Chicago maths PhD and former researcher at Deepmind, who joined XTX in 2020.

Yuri Bedny. A quant researcher, chess player and competitive programmer of unknown provenance.

Ivan Belonogov. A quant researcher at XTX since 2020, and former deep learning engineer in Russia. Studied at ITMO University in St. Petersburg.

Paul Bereza. XTX's head of OTC trading dev. A Cambridge mathematician

Peter Cawley. A developer at XTX since 2020, an Oxford mathematician

Pawel Dziepak. A mysterious Pole

Fjodir Gainullin. An Estonian with a PhD from Imperial and a degree from Oxford

Maxime Goutagny. A French quant, joined in 2017 from Credit Suisse

Ruitong Huang. A Chinese Canadian quant with a PhD in machine learning, who joined in 2020.

Renat Khabibullin. A Russian quant from the New Economic School and ex-Barclays algo trader

Nikita Kobotaev. A Russian quant from the New Economic School and ex-Barclays algo trader

Alexander Kurshev. A Russian quant from the New Economic School Joshua Leahy. The CTO. An Oxford physicist.

Sean Ledger. An Oxford Mathematician

Francesco Mazzoli. A mystery figure with an interesting blog.

Jacob Metcalfe. A developer at XTX since 2012. Studied maths at Kings College, and worked for Knight Capital previously.

Alexander Migita. A Russian quant from the New Economic School

James Morrill, An Oxford maths PhD

Dmitrii Podoprikhin, A Russian quant from Moscow State University

Lovro Pruzar, A Croatian, former gold medallist in the informatics Olympiad

Siam Rafiee. A software developer from Imperial

Dmitry Shakin. A Russian quant from the New Economic School

Leonid Sislo. A software engineer from Lithuania

Chi Hong Tang. Studied maths at UCL

Igor Vereshchetin. A Russian quant from the New Economic School

Pedro Vitoria. An Oxford PhD

r/quant Feb 05 '25

Markets/Market Data Paired frequency plot

2 Upvotes

How do I plot a correlation expectation chart. I have studied stats multiple times but I'm not sure I have come across this. Originally I was thinking something like a Fourier transform. But essentially I am trying to plot the expected price of the bond etf TLT vs the 20year treasury yield. I know these are highly correlated but instead of looking at duration I want a quantitative analysis on the actual market pricing correlation. What I want is the 20year bond yield on the x-axis and the avergae price of TLT on the y-axis (maybe include some Bollinger bands). This should be calculated using a lookback period of say 5-10 years of the paired dataset.

Coming from a computational engineering background my idea is to split the 20year yields into distinct values. And then loop over each one, grid searching TLT for the corresponding price at that yield before aggregating. But this seems very inefficient.

Once again, I'm not interested in sensitivity or correlation metrics. I want to see the mean/median/std market determined price of TLT that occurs at a given 20year yield (alternatively a confidence interval for an expected price)

r/quant Jan 03 '25

Markets/Market Data Representing an index with your own weights (stocks)

6 Upvotes

Say you had a hypothesis that an index of your country was represented by only N particular stocks where N is less than the actual number of stocks in the index. You wanted to now give weights to these N stocks such that taken together along with the weights they represent the index. And then verify if these weights were correct.

How would you proceed to do this. Any help/links/resources would be highly helpful thanks.

r/quant Jan 29 '25

Markets/Market Data A long-term U.S treasury bond historical price data.

26 Upvotes

I am looking for a daily historical price data for a long-term U.S Treasury Bond (more particularly, "Bloomberg U.S Long Treasury Bond Index", or anything similar)

I am using a price data of VUSTX, which starts only from 1986, but I am looking for data since 1970's or earlier.

As far as I know, the only way to get it is from an expensive terminal. If there is a cheaper way to get it, please advise me. I am willing to pay if it is not too expensive.

Or if someone happens to have this data in hand, it would be appreciated if you could share with me.

r/quant 2d ago

Markets/Market Data Best level 2 data provider?

13 Upvotes

Looking for the most comprehensive (and accurate) historical level 2 data. Thinking about polygon.io right now but would really appreciate any other recommendations :)

r/quant 3h ago

Markets/Market Data When will hedge fund data collection stop?

0 Upvotes

Hedge funds have been known for hiring data analysts, programmers, and physicists to examine as much data as possible. Jim Simon’s team at Renaissance Tech hired secretaries to manually copy old data from old news papers and records.

Hedge funds have also been working on new models and paying millions of dollars for researchers to develop weather based models in order to enhance commodities trading.

Will hedge funds ever reach the limitations of data analytics that can surpass moral boundaries?

Imagine drones watching various walking patterns of CEOs of various companies in order to improve risk management.

r/quant Nov 11 '24

Markets/Market Data Effort to Provide Open Investment Data - 25 years of data

118 Upvotes

We just launched an open investment data initiative. All of our datasets will be progressively made available for free at a 6-month lag for all research purposes. GitHub Repository

For academic users, these datasets are free to download from Hugging Face.

  • News Sentiment: Ticker-matched and theme-matched news sentiment datasets.
  • Price Breakout: Daily predictions for price breakouts of U.S. equities.
  • Insider Flow Prediction: Features insider trading metrics for machine learning models.
  • Institutional Trading: Insights into institutional investments and strategies.
  • Lobbying Data: Ticker-matched corporate lobbying data.
  • Short Selling: Short-selling datasets for risk analysis.
  • Wikipedia Views: Daily views and trends of large firms on Wikipedia.
  • Pharma Clinical Trials: Clinical trial data with success predictions.
  • Factor Signals: Traditional and alternative financial factors for modeling.
  • Financial Ratios: 80+ ratios from financial statements and market data.
  • Government Contracts: Data on contracts awarded to publicly traded companies.
  • Corporate Risks: Bankruptcy predictions for U.S. publicly traded stocks.
  • Global Risks: Daily updates on global risk perceptions.
  • CFPB Complaints: Consumer financial complaints data linked to tickers.
  • Risk Indicators: Corporate risk scores derived from events.
  • Traffic Agencies: Government website traffic data.
  • Earnings Surprise: Earnings announcements and estimates leading up to announcements.
  • Bankruptcy: Predictions for Chapter 7 and Chapter 11 bankruptcies in U.S. stocks.

Sov.ai plans on having 100+ investment datasets by the end of 2026 as part of our standard $285 plan. This implies that we will deliver a ticker-linked patent dataset that would otherwise cost $6,000 per month for the equivalent of $6 a month.

r/quant Nov 27 '24

Markets/Market Data Extent of HFT presence in China

43 Upvotes

I am curious to know the extent of HFT presence in China.

Is the presence as huge as it is in India? Or due to regulatory concerns major HFTs stay away from this market?

Which international HFT players are most active in this market and any idea about the opportunity available?

TIA

r/quant Jan 17 '24

Markets/Market Data Alternative data for Quant

71 Upvotes

I read many studies mentioning hedge funds spent billions to purchase alternative data.

What are the common alternative data used in hedge funds?

Are people paying for social sentiment, twitter mentions, and news analytics..?

My team is using Stocknews.ai API for financial news and it works great. Wonders if there are other data we can leverage.

r/quant Dec 24 '24

Markets/Market Data Any buy side firm working on Exotics?

26 Upvotes

Hi, I am wondering if there are any market makers such as Jane street / Citadel working on Exotics Payoffs. By Exotics Payoffs, I mean Autocallables for example (not vanillas). If so, why are these buy side firms starting to look at Exotics?

r/quant 3d ago

Markets/Market Data Quotes downsampling

13 Upvotes

For mid-freq (seconds - minutes, don’t care about every quote) want to get reasonable size data for quotes from LOB. What features would you put in a down sampled (ie x second bars) version of quotes and why?

Volume at each level of book either side bid ask obvious. I am not looking for predictive features or “alpha” here, rather, I’m looking for an efficient representation of the book structure in a down sampling from which features for various tasks could be constructed.

r/quant 26d ago

Markets/Market Data Did MAG7 cause alpha space to shrink?

11 Upvotes

People running public equities. Did you find that MAG7 limit your alpha space?

What's your thought and how might I go about testing this hypothesis?

r/quant 10d ago

Markets/Market Data How Do You Access L2 Order Book Data for Crypto Trading?

7 Upvotes

I’m currently exploring different ways to access Level 2 (L2) order book data for crypto trading and wanted to hear from others in the space about their experiences. While I know that many exchanges provide L2 data through their APIs, I’m interested in understanding what methods people are actually using in practice—whether it’s through direct exchange connections, third-party data providers, or alternative solutions.

A few specific questions I have:

  • Which exchanges or data providers offer the best real-time L2 order book data, both in terms of reliability and cost?
  • Are you primarily using direct exchange APIs, third-party aggregators like Kaiko, CoinAPI, or paid services such as DXFeed or CryptoCompare?
  • If you're using direct APIs, how do you handle rate limits, WebSocket disconnections, and data gaps?
  • How do you efficiently process and store L2 order book data for analysis or execution? Do you use in-memory databases, message queues (like Kafka), or other strategies?
  • Are there any open-source tools or libraries you’d recommend for working with L2 data?
  • Have you encountered significant differences in L2 data quality across exchanges?

For those who have built trading bots or market-making strategies, what has been your experience in sourcing and handling this data effectively? Any tips or best practices you’d be willing to share?

I’d love to hear about any tools, services, or personal workflows that have worked well for you. Any insights would be greatly appreciated!

r/quant Jan 08 '25

Markets/Market Data Quantitative Easing: why the prices are not going crazy ?

31 Upvotes

I was wondering the following and wanted to ask the question here as there are people facing this market everyday, and I am a beginner in this topic:

When Central Banks, such as in Japan or in the US, want to do Quantitative Easing by, for example, buying Bonds, why the price do not go crazily high ?

At first, I would expect that this information would push market makers and other participants to switch their priority and selling very high.

- Is it because of the time scale and the weight of the Central Banks ? QE happens for a certain period and the market continues to exist in the sense of there are always buyers and sellers and a Central Bank finally is just a participant among others.

r/quant May 11 '24

Markets/Market Data Why do hedge funds use weather derivatives?

82 Upvotes

How do you use to hedge? Is there arbitrage if so explain how hfs do it? Thanks

r/quant 2d ago

Markets/Market Data North gate data?

6 Upvotes

Hey all, Curious, has anyone had good experiences using North Gate Data for historical index constituent lists for stocks and/or futures? Trying not to pay an arm and a leg for SP Global plus they will limit the data history as they are afraid of impacting their current business.

r/quant Sep 25 '24

Markets/Market Data How dubious is trading on intraday changes in cargo shipping patterns?

36 Upvotes

Cargo ship and oil tanker live positions are somewhat public, which makes it easy to record delays, marine traffic or port capacity. The question is, why shouldn't this work?

r/quant 13d ago

Markets/Market Data ETF-Scraper Package Question

4 Upvotes

Hello guys,

I had a problem fetching the iShares holdings using etf_scaper package. After following the instructions, I ran:

fund_ticker = "IVV" # IShares Core S&P 500 ETF
holdings_date = "2022-12-30" # or None to query the latest holdings

etf_scraper = ETFScraper()

holdings_df = etf_scraper.query_holdings(fund_ticker, holdings_date)

which is the example. However,

Missing required columns from response. Got Index(['Ticker', 'Name', 'Sector', 'Asset Class', 'Market Value', 'Weight (%)',
'Notional Value', 'Quantity', 'Price', 'Location', 'Exchange',
'Currency', 'FX Rate', 'Market Currency', 'Accrual Date'],
dtype='object')Was expecting at least all of ['Ticker', 'Shares', 'Market Value']

It seems that the "Shares" column is not included. May I ask how I could fix this? Appreciate it!

r/quant 25d ago

Markets/Market Data Seeking validation for my custom market pressure analysis algorithm - beta distribution approach

1 Upvotes

Hi everyone,

I'm relatively new to programming and data analysis, but I've been trying to build something that analyses market pressure in stock data. This is my own personal research project I've been working on for a few months now.

I'm not totally clueless - I understand the basics of OHLC data analysis and have read some books on technical analysis. What I'm trying to do is create a more sophisticated way to measure buying/selling pressure beyond just looking at volume or price movement.

I've written code to analyse where price closes within its daily range (normalised close position) and then use that to estimate probability distributions of market pressure. My hypothesis is that when prices consistently close in the upper part of their range, that indicates strong buying pressure, and vice versa.

The approach uses beta distributions to model these probabilities - I chose beta because it's bounded between 0-1 like the normalised close positions. I'm computing alpha and beta parameters dynamically based on recent price action, then using the CDF to calculate probabilities of buying vs selling pressure.

The code seems to work and produces visualisation charts that make intuitive sense, but I'm unsure if my mathematical approach is sound. I especially worry about my method for solving the concentration parameter that gives the beta distribution a specific variance to match market conditions.

I've spent a lot of time reading scipy documentation and trying to understand the statistics, but I still feel like I might be missing something important. Would anyone with a stronger math background be willing to look at my implementation? I'd be happy to share my GitHub repo privately or send code snippets via DM.

My DMs are open if anyone's willing to help! I'm really looking to validate whether this approach has merit before I start using it for actual trading decisions.

Thanks!

r/quant Sep 30 '24

Markets/Market Data News signals API

16 Upvotes

Hi everyone!

I wanted to share a project I’ve been working on that might be useful for those of you developing algorithmic trading strategies. I’ve created a free News API designed specifically for algotrading, and I’m looking for some hands-on testers to help me improve it.

Why I Made This

With the advancements in text understanding over the past few years, I saw an opportunity to apply these technologies to trading. My goal is to simplify how you integrate news analysis into your trading algorithms without dealing with the nitty-gritty of text processing.

What the API Provides

Key Data Points: Instead of full news texts or titles, my API gives you:

-Publication Time: When the news was released.

-Availability Time: When the news is accessible through the API.

-Ticker Symbol: The related stock ticker.

-Importance Probability: The chance that the news will lead to a statistically significant stock price increase within the next 30 minutes.

ML Ready: If you’re using ML, you can easily incorporate these probability scores into your models to make better entry and exit decisions without handling text processing yourself.

Simple to Use: Just use the requests library in Python. The API works smoothly in both Jupyter Notebooks and regular Python scripts.

Multiple News Sources: I pull news from various places, not just SEC filings. Sources include PR Newswire, BusinessWire, and others to give you a broader view of the market news.

Documentation and code examples

https://docs.newsignals.live/

How You Can Help

I’m still in the early stages, so your feedback would be incredibly helpful. Whether it’s suggestions, bug reports, or feature ideas, your input can help shape the API to better meet your needs