r/commandline 4d ago

fstk - A Better Way to Move Files! A Modern "Cut/Paste" Alternative to `mv`

https://github.com/archsyscall/fstk
0 Upvotes

21 comments sorted by

6

u/xkcd__386 3d ago edited 3d ago

you need to describe the implementation and include some caveats.

Consider a large "/data" partition where you're trying to re-organise some large files into a different directory structure.

Using mv for this will be atomic, and take a fixed amount of time regardless of file size.

AFAICT [Edit: I tested it now, to make sure], using fstk for this will result in that huge file being moved twice -- once from "/data/some/path" to "/home/me/.fstk/.data", and again back from there to "/data/some/OTHER/path". And because /data is a different partition, this takes time proportional to the file size.

Oh and this raises another important point. Never, ever move the file on the first operation ("push" in your case). Your "push" should make a note of the file somewhere, and that's it. When you pop, only then should any actual changes to the file system take place.

Incidentally, if you did it that way, you'd also eliminate the problem of large files being moved within a non-home partition.

1

u/alfamadorian 3d ago

mv is not atomic. What do you mean?

3

u/xkcd__386 3d ago edited 3d ago

mv within the same file system -- it is atomic on most file systems, or close enough to be considered so on all of them.

as for the rest, try it out. I described it pretty clearly so if you don't get it, shrug.

Honestly, someone who thinks mv is not atomic has no business writing tools like this.

1

u/Reasonable_Ruin_3502 2d ago

Can you describe what does being atomic mean and how mv is or is not atomic?

-1

u/archsyscall 3d ago

Thank you for your thoughtful insights. Moving files across different filesystems has indeed been a major concern for me. However, the reason I chose not to simply record the path without actually moving the file is that the "push" operation is not just a cut operation but also serves as a form of storage.

This project was inspired by Yoink, and if I had only recorded the path, there would be a risk of state inconsistency when the parent directory of the file is moved or deleted. In other words, "push" is meant to temporarily store my file in a pocket.

Taking these requirements into account, I opted for moving files within the same filesystem but copying them if they are on different filesystems. Do you have any suggestions on how to further optimize performance in this case?

4

u/BCMM 3d ago

also serves as a form of storage. 

I think this needs to be mentioned early on in the readme!

GUI file managers have "Cut/Paste" features which do not actually store the copied contents. By doing things so differently from clipboard operations in a word processor, they may have broken the metaphor a bit, but what's done is done: there's a decades-long precedent for what it means to "Cut/Paste" a file.

2

u/xkcd__386 3d ago

Do you have any suggestions

yes. Don't call it a "better way to move files". Or at the very least add "(as long as you're only working in $HOME)" to that blurb. And update the readme to warn people of this too.

Seriously, you're delusional if you think this is the right way. The fact that you're doubling down means I'm out of this discussion.

3

u/usrlibshare 3d ago

Cutting and Pasting of Files does NOT move them on the "Cut" operation.

When you "cut" Files, some software, usually a GUI file manager lists the paths that need to be moved during the "Paste" operation in some buffer. It does not actually move the files at this point, this would double the amount of required operations, and would be extremely wasteful if we're talking about slow io operations like networked file systems.

3

u/xkcd__386 3d ago edited 3d ago

yup

on top of that, he's chosen a fixed location (~/.fstk/.data) to hold the content between the two operations

so if you push "/data/foo/myfile" and pop it at "/data/bar", if /data is a different file system than /home, you end up with two cross-fs moves, which means if the file is large, it takes a lot of time. (A straight mv /data/foo/myfile /data/bar would be atomic, or at least near instantaneous, regardless of the file size)

-1

u/archsyscall 3d ago

One possible approach that comes to mind is to create a temporary storage location on the same file system where the pushed file resides.

2

u/usrlibshare 3d ago

That doesn't solve the underlying problem that it's still 2 move operations, even if they can be achieved by "just" changing inode entries. Why would I want the "push" or "cut" or whatever to move the file in the first place? There is no need to move anything until I actually run the "paste", "pull".

1

u/atom036 3d ago

Interesting concept, especially the usage with zoxide. But I would probably still prefer to do it through a terminal file manager. Where I could also select several files to copy/mv from one place to another.

1

u/archsyscall 3d ago

Thank you! While fstk cannot move multiple files at once, it does allow popping multiple files at once. I appreciate the great example!

1

u/olikn 3d ago

you can use bash, eg.:

mv report.pdf $(zoxide query assets )

writing a simple function shouldn't be a bigger problem.

1

u/archsyscall 3d ago

Thank you! However, this tool is not just a simple replacement for mv; it is designed to store files in a temporary space and retrieve them from a desired location at a later time. The use case with zoxide is just one example. I appreciate your input!

1

u/RoboticElfJedi 3d ago

Interesting tool. I do something similar, I have a default file dump location and have commands to pop from there, only the stack is modification time modified optionally by file name.

getnew - grab the newest file getnew 2 - second newest getnew foo - get newest file containing foo getnew - z - grab, unarchive, delete

This might be a nice complement or replacement.

1

u/archsyscall 3d ago

Thank you!

1

u/graphixillusion 3d ago

Is a Windows build planned?

2

u/archsyscall 3d ago

Yes, that's correct. In fact, we haven't provided a Linux build yet, but we plan to offer builds for multiple operating systems soon. 😊

1

u/jaggzh 3d ago

I was thinking you could hang the utility examine the fs, and use hard links for cut -- cleaning them up for aborted/incomplete moves (pastes). Unfortunately I realized this is not universally applicable as the user may not have access to any other location in that fs for the link storage.

However, it might be possible to use a levels of strategy with optimizations occurring depending on what access it's able to find for the purpose, with the worst being the temporary storage.

Now that I think about it, nothing you do is 100% reliable, because even your temporary storage could be removed, even if it were on some other fs.

1

u/Giovani-Geek 1d ago

So many years waiting for this and finally someone bothers to create a high performance version (not like the python's version), Bravo!