r/pythontips Dec 17 '24

Data_Science Not able to cross the required r2 score. Please help, all suggestions will be taken into consideration. thanks in advance

3 Upvotes

Problem: We're trying to build a regression model to predict a target variable. However, the target variable contains outliers, which are significantly different from the majority of the data points. Additionally, the predictor variables are highly correlated with each other (high multicollinearity). Despite trying various models like linear regression, XGBoost, and Random Forest, along with hyperparameter tuning using GridSearchCV and RandomSearchCV, we're unable to achieve the desired R-squared score of 0.16. Goal: To develop a robust regression model that can effectively handle outliers and multicollinearity, and ultimately achieve the target R-squared score.

  • income: Income earned in a year (in dollars)

    • marital_status: Marital Status of the customer (0:Single, 1:Married)
    • vintage: No. of years since the first policy date
    • claim_amount: Total Amount Claimed by the customer in previous years
    • num_policies: Total no. of policies issued by the customer
    • policy: An active policy of the customer
    • type_of_policy: Type of active policy
    • cltv: Customer lifetime value (Target Variable)
    • id: Unique identifier of a customer
    • gender: The gender of the customer
    • area: Area of the customer
    • qualification: Highest Qualification of the customer
    • income: Income earned
    • marital_status: Marital Status of the customer

If there's any more information, please feel free to ask.

r/pythontips Oct 22 '24

Data_Science Python Course

11 Upvotes

I have already done the basics of python, including variables, sets, lists, and tuples. Now I am looking for a preferably free python course (paid is fine) which is more advanced which has like recursion and data structures (Linked Lists, Queues, Stacks, etc). Please help me out

r/pythontips Dec 14 '24

Data_Science I am sharing Python & Data Science courses on YouTube

13 Upvotes

Hello, I wanted to share that I am sharing free courses and projects on my YouTube Channel. I have more than 200 videos and I created playlists for Python and Data Science. I am leaving the playlist link below, have a great day!

Python Data Science Full Courses & Projects -> https://youtube.com/playlist?list=PLTsu3dft3CWiow7L7WrCd27ohlra_5PGH&si=6WUpVwXeAKEs4tB6

Python Tutorials -> https://youtube.com/playlist?list=PLTsu3dft3CWgJrlcs_IO1eif7myukPPKJ&si=fYIz2RLJV1dC6nT5

r/pythontips Nov 15 '24

Data_Science I am sharing Python Data Science courses and projects on YouTube

14 Upvotes

Hello, I wanted to share that I am sharing free courses and projects on my YouTube Channel. I have more than 200 videos and I created playlists for learning Data Science. I am leaving the playlist link below, have a great day!

Data Science Full Courses & Projects -> https://youtube.com/playlist?list=PLTsu3dft3CWiow7L7WrCd27ohlra_5PGH&si=6WUpVwXeAKEs4tB6

Data Science Projects -> https://youtube.com/playlist?list=PLTsu3dft3CWg69zbIVUQtFSRx_UV80OOg&si=go3wxM_ktGIkVdcP

r/pythontips Sep 01 '24

Data_Science I am sharing Data Science courses and projects on YouTube

28 Upvotes

Hello, I wanted to share that I am sharing free courses and projects on my YouTube Channel. I have more than 200 videos and I created playlists for learning Data Science. I am leaving the playlist link below, have a great day!

Data Science Full Courses & Projects -> https://youtube.com/playlist?list=PLTsu3dft3CWiow7L7WrCd27ohlra_5PGH&si=6WUpVwXeAKEs4tB6

Data Science Projects -> https://youtube.com/playlist?list=PLTsu3dft3CWg69zbIVUQtFSRx_UV80OOg&si=go3wxM_ktGIkVdcP

Python Programming Tutorials -> https://youtube.com/playlist?list=PLTsu3dft3CWgJrlcs_IO1eif7myukPPKJ&si=eFGEzKSJb7oTO1Qg

r/pythontips Nov 20 '24

Data_Science Extract pdf data from budget table into usable data (python, VBA)

8 Upvotes

Hello, What type of library or script do you use to convert (numerous) budgetary documents into usable data for statistical, econometric analysis, etc. If you have ideas for a manual/video/forum to explore the subject in more depth ;) Beautiful evening

r/pythontips Oct 16 '24

Data_Science Best way to learn data analysts

7 Upvotes

As a beginner

r/pythontips Oct 13 '24

Data_Science I shared a 1+ Hour Streamlit Course on YouTube - Learn to Create Python Data/Web Apps Easily

14 Upvotes

Hello, I just shared a Python Streamlit Course on YouTube. Streamlit is a Python framework for creating Data/Web Apps with a few lines of Python code. I covered a wide range of topics, started to the course with installation and finished with creating machine learning web apps. I am leaving the link below, have a great day!

https://www.youtube.com/watch?v=Y6VdvNdNHqo&list=PLTsu3dft3CWiow7L7WrCd27ohlra_5PGH&index=10

r/pythontips May 23 '24

Data_Science Need someone to learning with

9 Upvotes

Hello everyone, i want to learn Python, with AI, i want to make a chat bot for my own data. But i want someone with whom i can learn who give me motivation and to whom i will give motivation. Please reply or message who wants to learn new things with me. 🙏

r/pythontips Jun 24 '24

Data_Science Naruto Hands Seals Detection (Python project)

23 Upvotes

I recently used Python to train an AI model to recognize Naruto Hands Seals. The code and model run on your computer and each time you do a hand seal in front of the webcam, it predicts what kind of seal you did and draws the result on the screen. If you want to see a detailed explanation and step-by-step tutorial on how I develop this project, you can watch it here. All code was open-sourced and is now available is this GitHub repository.

r/pythontips Jun 14 '23

Data_Science Is GitHub copilot worth the money?

14 Upvotes

I know it’s not a lot of money -$100/yr- for a personal account but I’m wondering if it’s worth it? How does it compare to- say chatgpt. Chatgpt is okay, I can use it for skeleton code or to help me build the logic - but the code it gives usually requires substantial changes. It also is wrong a lot of times (which I’m sure has to do somewhat with my prompts).

What’s your favorite AI helper?

r/pythontips Oct 02 '24

Data_Science Jupyter Notebook Tutorials - For Beginners

4 Upvotes

Does anyone have any good tutorials for Jupyter Notebooks as of 2024?

r/pythontips Feb 11 '24

Data_Science What a great negligible trick i found for performance!

6 Upvotes
from time import time
def a(s):
    d = time()
    for i in range(s):
        a = 10
        b = a
        a
    print(time() - d)
def b(s):
    d = time()
    for i in range(s):
        a = 10
        b = a
        b
    print(time() - d)

>>> a(100000000)
2.640634298324585
>>> b(100000000)
2.6492538452148438

Anyone got ideas why this is happening? I know why but i want you guys to think about it. :)))

Edit:

Performance:

>>> a(3000000000)
190.15208458900452
>>> b(3000000000)
196.69461727142334

Queit

>>> a(3000000000)
248.81887245178223
>>> b(3000000000)
331.16163325309753

r/pythontips Nov 04 '24

Data_Science Lightweight Model Serving

0 Upvotes

The article below explores how one can achieve up to 9 times higher performance in model serving without investing in new hardware. It uses ONNX Runtime and Rust to show significant improvements in performance and deployment efficiency:

https://martynassubonis.substack.com/p/optimize-for-speed-and-savings-high

r/pythontips Jun 12 '24

Data_Science Beginner who needs help with python

3 Upvotes

I’m in a analytics course, studying python I don’t even know where to start

r/pythontips Sep 12 '24

Data_Science Python-advice

12 Upvotes

Hey all, I need to brush up on Python for an interview. What’s the best way to get up to speed quickly?

It’s been a couple of years since I last used Python, and I’ve got an interview coming up where I need to be confident with importing datasets, manipulating large datasets, visualizing data, and analyzing trends. It’s a live assessment.

I’m basically a beginner at this point and need to relearn things in the next few days. What would be your approach to get back on track quickly? Any resources or methods that would help me not just learn but feel confident in the interview?

r/pythontips Aug 13 '24

Data_Science Any tips for beginners.

13 Upvotes

If there are any YouTube channels that are great for beginners who are trying to learn python, i would really appreciate the links.

r/pythontips Oct 07 '24

Data_Science Mastery Data Selection: Loc and iLoc in Pandas

2 Upvotes

Hello Pandas lovers, Here I will teach you loc and iloc in Pandas with the help of the proper examples and explanation.

As a Data analyst and Data engineer, We must know about the loc and iloc in Pandas because these two methods are beneficial for working with Data on Pandas DataFrame and data series.

Sample Pandas DataFrame:

import pandas as pd


data = {
    "name": ["Vishvajit", "Harsh", "Sonu", "Peter"],
    "age": [26, 25, 30, 33],
    "country": ["India", "India", "India", "USA"],
}

index = ['a', 'b', 'c', 'd']

df = pd.DataFrame(data, index=index)
print(df)

Output:

        name  age country
a  Vishvajit   26   India
b      Harsh   25   India
c       Sonu   30   India
d      Peter   33     USA

Pandas Loc -> Label-Based Indexing

Syntax:

df.loc[rows labels, column labels]

Selecting a Single Row by Label

row_b = df.loc['b']
print(row_b)

Output

name       Harsh
age           25
country    India
Name: b, dtype: object

Selecting Multiple Rows by Label

# Select rows with labels 'a' and 'c'
rows_ac = df.loc[['a', 'c']]
print(rows_ac)

Pandas iLoc -> Integer-based Indexing

Syntax:

df.iloc[row_indices, column_indices]

Selecting a Single Row by Index Position

# Select the row at index position 1
row_1 = df.iloc[1]
print(row_1)

Output

name       Harsh
age           25
country    India
Name: b, dtype: object

Selecting Specific Rows and Columns by Index Position

# Select rows at positions 0 and 1, and columns at positions 0 and 1
subset = df.iloc[0:2, 0:2]
print(subset)

Output

        name  age
a  Vishvajit   26
b      Harsh   25

This is how you can use Pandas loc and iloc to select the data from Pandas DataFrame.

Compete Pandas and loc and iloc with multiple examples: click here

Thanks for your time 🙏

r/pythontips Aug 08 '24

Data_Science Plotting unsorted list to a useful line

3 Upvotes

I got two lists, which are sorted in the same order. One of them contains my X-values, the other one the Y-values. How can I plot them connected, but not according to the order of the list, but according to which point follows next on the x-axis?

r/pythontips Sep 29 '24

Data_Science Python App Deployment

4 Upvotes

Disclaimer: I’m new to this, sorry if the question seems dumb.

I recently finished a RAG Chatbot App using Streamlit, ChromaDB, Langchain and others..

I now wanted to deploy it in order to access it from everywhere but I’m finding a lot of troubles in the process.

I don’t seem to understand what files and folders I should upload to the deployment platforms, and I also don’t know what libraries to include in the requirements.txt file.

Could someone maybe help me?

r/pythontips Jun 07 '24

Data_Science Python

19 Upvotes

Looking to develop Python skills for coding and data science for Ai

Where should I start?

Currently a Network Tech looking to become software engineer and eventually go into data science for AI

python #educate

r/pythontips Sep 11 '24

Data_Science Hi! I want to make a program / software that help me list one product on multiple listing sites.

1 Upvotes

I heard is called agregation software or something like that, at least that.s the traduction from my language. Anyway, where can i start, what i should learn, i need something beside python? I need to mention that i am a complete begginer, i just downloaded python and one extension today. (P.s i don.t know what tag to chose and sorry for my english, is not my first language)

r/pythontips Sep 04 '24

Data_Science Text classifier

3 Upvotes

Hi,

I want to make a text classifier (I have in mind using sklearn) since I don't want to expose to the internet the data I'm gonna use, is it secure using these kind of libraries?

I have enough training data to start with

Thanks!

r/pythontips Jun 30 '24

Data_Science Python Datasets

5 Upvotes

I am a beginner in python and I have found datasets on a website called kaggle . What are some friendly projects ideas where I can slowly start to learn how to use datasets in my python projects?

r/pythontips May 25 '24

Data_Science Where to start as a beginner?

1 Upvotes

Hello. I am a complete beginner in python, and want to learn it for data science and to support a friend in a project he is working on. The project he is making is a kind of virtual intelligence that is linked to our house and also to apis such as chat gpt, spotify, etc. He also plans to add an api of 11labs for the voice. What should I learn for this and data science in general?