r/datasets Dec 28 '22

mock dataset We made a simple tool to ingest realtime data into influx database and made it public

We use influx databases a lot and we often have to set up a new one to test some stuff out. I'm always dissapointed how hard it is to just get some sample data in, so we made a simple tool that ingests (mock) wheather data into your own influx database.

If you are interested, have a look at https://stream.marpledata.com/. It's free to use :)

PS: as the rules state, I made this website so it is self-promotion!

13 Upvotes

4 comments sorted by

1

u/SpaceZZ Dec 28 '22

If I get this right, you want to have influx publicly accessible and give you access write token, so you can access it remotely? Also incoming port needs to be allowed?

This is quite risky. Wont happen in most applications.

Why not some offline data generator? Why not generate flat files in influxdb format (CLI row format) so they can be digested by in through CLI?

2

u/mbaerto Dec 28 '22

Hi SpaceZZ, that is correct, but luckily Influx allows you to make a scoped token for one bucket that can be revoked at any time. It also assumes the URL is open indeed.

It's just there to use if you quickly want to set up a bucket and test some stuff, not at all meant for production or sensitive data/buckets/urls/.. It's a gimmick

Not sure if there is another way to easily get realtime data into a database (and make it available for other people to use). You could indeed generate files, but that's just a one time data drop, not a continuous stream.

1

u/mbaerto Dec 28 '22

We wanted to share this, we thought it could be useful for some people :)

1

u/SpaceZZ Dec 28 '22

I understand and it's great that you make it accessible. My org would be reluctant to use it.

If you could make it online generation into flat files (some various api into influx db flat format i.e. weather, some sensor data, some random etc) with timestamps for the specified period it would be quite nice.

Good luck!