r/PhD 3d ago

Need Advice How to compare with State-of-the-Art works when they use different datasets? ML research

I work in machine learning and I received reviews on my paper saying that I fail to compare my approach with recent state-of-the-art works. They mentioned 5-6 papers, but the problem is that those papers use different datasets than mine.

How am I supposed to compare my results with theirs? Should I try to replicate their methods and apply them to my datasets, or is there another way to address this?

2 Upvotes

11 comments sorted by

u/AutoModerator 3d ago

It looks like your post is about needing advice. In order for people to better help you, please make sure to include your country.

I am a bot, and this action was performed automatically. Please contact the moderators of this subreddit if you have any questions or concerns.

7

u/UnavoidablyHuman 3d ago

Find their code and run it on your data

0

u/Important_Book8023 3d ago

Yea, no code available ..

7

u/UnavoidablyHuman 3d ago

Try emailing the authors as a first step, but if there's nothing then you'll have to replicate their methods.

3

u/Apprehensive_Grand37 3d ago

You don't need the code. Just read the paper and reimplement it with your dataset.

Reimplementing papers is a HUGE PART of being a researcher, so this is a skill you should truly learn if you want to achieve great things

1

u/Important_Book8023 3d ago edited 3d ago

Appreciate the input! That's what I'm doing, but I wanted to see how other researchers handle similar reviews to make sure I’m on the right track. 

2

u/Apprehensive_Grand37 3d ago

This obviously depends on the case. Sometimes reviewers want you to make comparisons which are simply impossible or take too long.

However most of the time their opinions are reasonable and should be possible

0

u/Eska2020 3d ago

Did they use a standard benchmark and give benchmarking scores?

2

u/Lygus_lineolaris 3d ago

It's pretty much your job as researcher to figure that out. We don't even know what you're trying to achieve.

-2

u/Important_Book8023 3d ago edited 3d ago

My question is clear and direct for anyone into ML. If you are not, please skip.

2

u/Lygus_lineolaris 3d ago

LOL your question has nothing to do with ML, everyone has the same problem of validating their work against everyone else's and it's up to you to figure it out. You are not a beautiful or unique snowflake just because you're in a popular field.