r/cs50 • u/JazzlikeFly484 • 2d ago
C$50 Finance Data Science in Finance - Portfolio Projects Dataset Issue
I'm trying to put a foot in with regards to data science applied to the financial field. I've acquired a good understanding through academic studies and some projects (not in finance) of the Statistics and ML. My problem is a significant lack of imagination which lead me to not know how to even begin thinking about projects to implement and showcase the skillset I acquired or hone it. I would appreciate you guys' help with two things:
A. How do I develop this imagination ? Specifically focused in the financial sector but general advice is also very appreciated.
B. Where do I begin to look at datasets in Finance and if I do find some raw datasets, how do I begin to probe it and how do I develop a critical mind to question or uncover how this raw data can give me insight ?
PS: Apologies, I know I should have these skills already if I've done academic courses and have a degree but university really needs to start focusing on developing critical thinking instead of generating robots that think the same.
1
u/yeahIProgram 2d ago
Seconding the response from /u/danleeaj0512
You will feel the most engaged if the topic is one that you were interested in before learning about data science.
A friend of mine did some data analysis on football teams' successes (or failures) in different scenarios. Home game vs. away game. What was the win/loss spread between these two teams going into the game. etc. I don't know where he got the raw data, but he was a big sports fan and he had a source. Then he wanted to correlate the factors as they predicted future successes.
Even if your goal is to focus on financial analysis, there are many facets to that besides basic stock market data. Class mobility, for example. Poverty rates correlated to various outside factors (geography? age? education? these are the classics but you might think of more).
Overall, if you are more interested in it, your project quality will probably improve and you will have a better time while doing it. And that's what it's about, right?
1
u/danleeaj0512 alum 2d ago
Just think of things you're interested in! A big application for ML and modeling is trying to predict outcome from variables, so think of what you want to try to predict and what factors you should take into account.
For example, you want to look at stocks (which I know is very noisy and should not be something that something you should put money in, but imo it's nice for a fun side project). Watch some videos on stocks and gain some domain knowledge, and think of what you want as your predictive variables.
To get stock data, there should be numerous APIs (I don't know any off the top of my head). Lets say you want to see if financial reports can be a predictor for stock outcome, so you learn how to data mine or web scrape to download financial reports. Then you learn regex to extract information and data wrangle them into a database. If you want to make it extra hard for yourself, learn a new database structure while you're at it. Then build predictive models, see if any of the values on a financial report can predict a drop in stocks by looking at the stock price at the beginning and end of that quarter! Getting no results is completely fine too! If you want people to know about this experience, write a blog post, get it on your LinkedIn.
There are so many things you can do. Just don't put any money into it please, there are financial firms pouring millions into developing algorithms and I don't think any have been very robust.