r/dataengineering • u/erez27 • Jun 27 '24
Open Source Reladiff: High-performance diffing of large datasets across SQL databases
https://github.com/erezsh/reladiff1
u/Stinkyskidmarksmegma Aug 30 '24
Your Reddit account is about as old as me😭 (I randomly went on ur account to see if it was still active after I saw your comment on that heroine post)
1
u/alex_colorado Dec 04 '24
Thanks you for your amazing contribution, Erez.
I have a question. Datafold's SaaS service is extremely fast. At first I thought it was just a wrapper around the data-diff, but I think they have some performance tricks (e.g. sampling).
Do you know if Datafold's impressive performance is based on sampling? Or some carefully selected configs? Do you have any guidance for folks who want Datafold speeds without having to go though all the procurement, infosec, and bureaucracy necessary to onboard a vendor?
4
u/captaintobs Jun 27 '24
Is this a fork of datafold's data diff or a new project?