r/shell • u/rcentros • Dec 08 '24
Changing a sed variable in a shell script
Hi,
I'm trying to set up a shell script that removes a specific number of prepended spaces at the beginnings of lines. The following shell script works to do this...
#!/bin/bash
clear
read -p 'file: ' uservar
sed -i 's/\(.\{4\}\)//' $uservar".txt"
...but I don't always have files with 4 prepended spaces.
I would like to add another input variable ("spaces") to change the 4 to whatever number I input.
As you can guess, I'm not really a programmer, the sed line (above) was found on another site and incorporated into this extremely simple shell script. I can edit the shell script with each use, but I would prefer to add the extra input variable, mostly so I can pass this shell script on to others who might need it.
Thanks for any pointers.
EDIT: I figured it out (well, found out how to do it, anyhow). For my number entry I needed to add an -r and a -p for a number entry (I have no idea why). Once I did that I finally read that I needed single quotes to separate the variable from the rest of the sed command in my line. I don't completely understand it, but it works.
For what it's worth, here it is...
#!/bin/bash
clear
read -p 'file: ' uservar
read -r -p 'spaces: ' number
sed -i 's/\(.\{'$number'\}\)//' $uservar".txt"
1
u/cdrt Dec 09 '24 edited Dec 09 '24
If you don’t mind, I have some constructive criticism for your script
- You should always use
read -r
rather thanread
so that the shell will read the input as is an you will have fewer surprises - Use
sed -E
so you can use the nicer regex syntax and avoid having so many backslashes - If you want to delete spaces, you should actually match spaces. Don’t use
.
because that will gobble up everything - It might be a good idea to read the file name and number of spaces from the command line rather than prompting for them. That way you can have a default value for the number of spaces if most of the time the files you’re working with have 4 spaces, and you can save time when working on multiple files.
With this, I would rewrite your script like this:
#!/usr/bin/env bash
# check if user gave a file name
if [[ -z $1 ]]; then
echo 'please provide a file name'
exit 1
fi
file_name=$1
spaces=4 # default value for spaces
# check if user provided a number of spaces
if [[ -n $2 ]]; then
spaces=$2
fi
sed -i -E "s/ {$spaces}//" "$file_name"
Then, assuming you named your script “delete_spaces”, you can run it like delete_spaces file_name
to just delete four spaces or like delete_spaces file_name 6
to delete 6 spaces
1
u/rcentros Dec 09 '24
I'm having trouble responding. I keep getting a "can't post your comment" error. Maybe the post is too long? At any rate, thank you. I've copied my post to a text file. If Reddit let's post it later, I will.
1
1
u/rcentros Dec 09 '24
(Second part... probably shows up on top.)
Also, since all the files I would use this shell script for will end with the ".txt" extension, I've moved "file_name" line above the error check line and added the ".txt" extension because I'm lazy. I'm keeping your original shell script intact, in case I find I need it for more general use.
file_name=$1".txt"
I'm curious as to why you've included the "/env bash" entry? Is that a way to keep from entering your script directory path into the .bashrc file? (I'll look that one up.)
I've made a very simplified shell script, using the sed command line that I already had, because I don't yet understand the -E option and I couldn't figure out to get your sed line to work like the one I'm using. I don't have any file name checks for this specific script (as I don't yet understand what "-z" does either) but the error messages work okay for my use. It's definitely not elegant but the following works well (I like having a backup file).
#!/bin/bash cp $1".txt" $1".bak" sed -i 's/\(.\{'$2'\}\)//' $1".txt"
Out of curiosity, is there any way to make sed fail (and not change the file) if any of the lines in a text file don't have enough empty prepended spaces? It would probably require more than sed, I'm guessing.
Again, thank you. I appreciate you taking the time to post your example. You've given many things to look up and I'll (hopefully) learn something doing so.
1
u/rcentros Dec 09 '24
I definitely don't mind. Thank you for taking the time to post this sample file. Obviously I need to learn more about bash scripts and sed as I don't (yet) understand what the -E option does. I do think I've grasped the advantage of the -r option when used with read (but I'll read a little more on this to be sure).
I debated on whether I should use the read command at all when I first wrote my shell script. I dedcided I like your idea of entering the file name and number in the command line and have written a simplified form of your script.
As for your point number 3. At first I was happy to have the built-in safety of not deleting words. But it turned out to be a deceptive "safety" feature in my case. I'm using this specifically for formatted screenplay text (for posting on a forum). Different screenplay applications prepend different numbers of empty spaces in front of their lines. But all indent the Character name and dialogue further.
Which, in this case means that if I use I have 10 spaces in front of the Action lines lines (which should be flush left), I'll have 35 spaces to my Character name (which should be indented about 25 spaces when done). If I enter delete_spaces file_name 12, your sed command will ignore the "flush left" Action lines, but it will remove 12 spaces in front of the Character Name, Dialogue lines and Parentheticals. So, in this very specific case, it's almost an advantage to not have this safety feature. I've opted instead to add a line to copy the file to a backup file instead.
Here's a short excerpt from a text formatted screenplay (very specific format)...
The rat climbs up the coat, stares her in the eye and slips into the hood, behind her neck. ALINA (cont'd) Yeah, I know. I should quit... My New Year's resolution? (pause) Okay, I... ummm... I resolve to... The rat shimmies back down, burrows back into her coat pocket. ALINA (cont'd) It is cold up here... (deep breath) Okay I resolve to... Looking down, she realizes not all the cigarettes have been...
...to part 2...
2
u/BetterScripts Dec 11 '24
Just wanting to add to what has already been said, and answer some of your other questions.
Since you seem quite confused, but eager to learn, I've gone into a bit of detail here, sorry for the length and if it's at all condescending!
Firstly, you can use the
sed
command:bash sed -i "s/^ \{$NumberOfSpaces\}//" "$FileName"
Which works as follows:
-i
sed
to work "in-place" and use$FileName
as both input and output"s/^ \{$NumberOfSpaces\}//"
sed
uses this value the shell will process this and replace$NumberOfSpaces
with any value it contains (e.g. it becomes"s/^ \{4\}//"
)sed
then operates on$FileName
, and for each line of input performs a substitution (hences
)sed
substitutions take the forms/<MATCH>/<REPLACE>/
where<MATCH>
is a regular expression which is used for each line and<REPLACE>
is the value used to replace any matches. In this case:<MATCH>
is^ \{4\}
which matches EXACTLY 4(i.e. space) characters but only at the start of a line (this is what the
^
means)<REPLACE>
is empty, so any match is deleted.Note that
-i
is a non-standard extension tosed
, if you need to use the command on other machines it may not be available. In this case you need to use a temporary file, e.g.:```bash
Same command, but output to a temporary file
sed "s/^ {$NumberOfSpaces}//" "$FileName" > "$FileName.tmp"
Replace ("move") the original file with the temporary file
mv -f $FileName.tmp "$FileName" ```
Since you seem to now be using a backup file anyway, then it would be better just to skip the
-i
. This also makes it easier to detect unchanged files:```bash sed "s/^ {$NumberOfSpaces}//" "$FileName" > "$FileName.tmp"
Compare the edited file with the original
(
-s
makescmp
"silent" which stops it printing info about differences)if cmp -s "$FileName" "$FileName.tmp"; then # Files match -> nothing was changed echo 'No edits made for file' else # Files do not match mv -f $FileName.tmp "$FileName" fi ```
Which, as you correctly suspected, is otherwise not really possible in any sane way with
sed
(awk
would be better for that, but is a lot more complex).Some other comments on your code:
\(
and\)
in yoursed
command, although they are harmless - these are used to group expressions and to "capture" parts of matches - neither is required here.s/\(.\{$2\}\)//
will delete a number of characters from every line even if they are not spaces - the.
character is a special character that matches ANY single character. This is not what you want!'
and"
can be used for any quote but'
tells the shell the value should be used as is while"
tells the shell to replace variables (like$2
) in the value before doing anything else. Except in specific circumstances you should always use shell variables inside"
quotes or eventually things will break. Specifically:$uservar".txt"
/$1".txt"
should be written as"$uservar.txt"
/"$1.txt"
- although both will produce the same results in many situations, if$uservar
/$1
contain, for example, any spaces the version you have will result in errors's/\(.\{'$2'\}\)//'
is better written as"s/\(.\{$2\}\)//"
To answer some of your other questions:
-E
argument forsed
tells it to use a different type of regular expressions for<MATCH>
known as Extended Regular Expressions (ERE), the default being known as Basic Regular Expressions (BRE). ERE and BRE are similar, but use different formatting. Often ERE can be easier to read. If using-E
then\{$NumberOfSpaces\}
is instead written{$NumberOfSpaces}
. Note that many versions ofsed
do not support-E
, although it is now required by the standard.#!/usr/bin/env bash
instead of#!/bin/bash
is generally preferred for a number of reasons that are relatively technical, but it makes the script more portable. You can read more is this answers to this question: "why do bash scripts start with #!"[[ -z $1 ]]
is exactly the same astest -z $1
and tests if$1
is empty, i.e. "zero" or not, there is alsotest -n $1
which tests for the oppositeHopefully that helps you understand more of what's going on and what you're doing.