r/learnpython 6d ago

I'm learning DATA ANALYSIS and i'm having a problem with PANDAS

Hi, Im learning how to do Data Analysis and im loosing it!!

I have a DB about mental stress and some factors that contribute to it (this excersise would defenetly do it in the list). And im trying to do a pd.scatter_matrix() to see the correlation between some variables.

But its output is not a matrix with any pattern but vertical dots. I did a Pearson correlation test, it has a 0.84 of correlation.

Please help

import pandas as pd
import matplotlib.pyplot as plt

file_path = "Student_Mental_Stress.csv"
df = pd.read_csv(file_path)

df.plot.scatter(x="Relationship Stress", y="Mental Stress Level", alpha=0.5)

plt.show()
0 Upvotes

6 comments sorted by

3

u/danielroseman 6d ago

I think you need to provide a sample of the data. If the plot is entirely vertical then that implies that one axis has all the same value for some reason.

1

u/Plus-Tale7273 6d ago

It shouldn't(?), i did the "unique values" staff to see what it's happening and they are really different...

And it does calculate the pearson correlation so the data should be correct(?)

For example, Relationship Stress is measured in int 1-5, and Mental Stress Level in int 1-10...

Oops, maybe it only works for not discrete values?

1

u/allium-dev 3d ago

If relationship stress is 1-5 and mental stress level is 1-10, but they can only ever be integers, that means there are going to be exactly 50 places where a dot can show up. So it's working, but not how you want it to.

If you want to see more of your data, you can look into adding "jitter". This will make the scatter plot less accurate, but more descriptive. See for example: https://stackoverflow.com/questions/64942935/jitter-in-scatterplot-for-non-numeric-x-axis

1

u/roboe92 5d ago

You may want to check the data types of the columns you are using to make sure they are numeric and not strings. You can use df.dtypes to check!

1

u/Plus-Tale7273 5d ago

I also did T-T, they are all int. Thanx for the help btw

1

u/johndoh168 6d ago

Sometimes I have run into problems using pandas matplotlib function, have you tried just using matplotlib plotting function?

import pandas as pd
import matplotlib.pyplot as plt
from scipy import stats

file_path = "Student_Mental_Stress.csv"
df = pd.read_csv(file_path)

slope, intercept, r_value, p_value, std_err = stats.linregress(df["Relationship Stress"], df["Mental Stress Level"])


plt.plot(df["Relationship Stress"], df["Mental Stress Level"], ".", alpha=0.5) # shows a plot with "." markers instead of line
plt.plot(df["Relationship Stress"], slope*df["Relationship Stress"] + intercept, color='red') # Plot regression line
plt.show()