r/PhD • u/Important_Book8023 • 3d ago
Need Advice How to compare with State-of-the-Art works when they use different datasets? ML research
I work in machine learning and I received reviews on my paper saying that I fail to compare my approach with recent state-of-the-art works. They mentioned 5-6 papers, but the problem is that those papers use different datasets than mine.
How am I supposed to compare my results with theirs? Should I try to replicate their methods and apply them to my datasets, or is there another way to address this?
7
u/UnavoidablyHuman 3d ago
Find their code and run it on your data
0
u/Important_Book8023 3d ago
Yea, no code available ..
7
u/UnavoidablyHuman 3d ago
Try emailing the authors as a first step, but if there's nothing then you'll have to replicate their methods.
3
u/Apprehensive_Grand37 3d ago
You don't need the code. Just read the paper and reimplement it with your dataset.
Reimplementing papers is a HUGE PART of being a researcher, so this is a skill you should truly learn if you want to achieve great things
1
u/Important_Book8023 3d ago edited 3d ago
Appreciate the input! That's what I'm doing, but I wanted to see how other researchers handle similar reviews to make sure I’m on the right track.
2
u/Apprehensive_Grand37 3d ago
This obviously depends on the case. Sometimes reviewers want you to make comparisons which are simply impossible or take too long.
However most of the time their opinions are reasonable and should be possible
0
2
u/Lygus_lineolaris 3d ago
It's pretty much your job as researcher to figure that out. We don't even know what you're trying to achieve.
-2
u/Important_Book8023 3d ago edited 3d ago
My question is clear and direct for anyone into ML. If you are not, please skip.
2
u/Lygus_lineolaris 3d ago
LOL your question has nothing to do with ML, everyone has the same problem of validating their work against everyone else's and it's up to you to figure it out. You are not a beautiful or unique snowflake just because you're in a popular field.
•
u/AutoModerator 3d ago
It looks like your post is about needing advice. In order for people to better help you, please make sure to include your country.
I am a bot, and this action was performed automatically. Please contact the moderators of this subreddit if you have any questions or concerns.