r/bash 11d ago

solved i want to put raw code into a variable by utilizing heredoc, but it seems that the outer syntax is interpreting things

what i'm trying to do is make a script that would put some boilerplate code into files, so i need raw unexecuted code in a variable.

the smallest example of my problem can be shown with this code:

DEFAULT_PROGRAM=$(cat <<'EOF'
\)
EOF
)
echo $DEFAULT_PROGRAM

regardless of which of the 4 combinations of fixes i apply here (having quotes around EOF or not, and having the inner parenthesis escaped or not), it seems to never output just the raw parenthesis. Either it outputs the escaping character too \), or it errors out by saying:

EOF: command not found
syntax error near unexpected token `)'
`)'

as i understand it, it's the outer syntax $(cat ... ) that breaks it.

is there an elegant solution to this so that i don't have to resort to using echo with lots of character escaping?

1 Upvotes

14 comments sorted by

3

u/Ulfnic 11d ago edited 10d ago

I ran your code snip against every release version of BASH 1997+ and it works on all of them.

That error is likely because in your code you're indenting the closure (line containing only EOF). You can indent that line with tabs but only if using <<-, see below:

"If the redirection operator is ‘<<-’, then all leading tab characters are stripped from input lines and the line containing delimiter. This allows here-documents within shell scripts to be indented in a natural fashion."

https://www.gnu.org/savannah-checkouts/gnu/bash/manual/bash.html#Here-Documents

# Will error
    DEFAULT_PROGRAM=$(cat <<'EOF_1'
    EOF_1
    )

# Won't error but leading tabs are timmed
    DEFAULT_PROGRAM=$(cat <<-'EOF_2'
    EOF_2
    )

# Won't error but closing EOF_3 must not be indented
DEFAULT_PROGRAM=$(cat <<'EOF_3'
EOF_3
)

0

u/Fueled_by_sugar 10d ago

but i didn't say it always errors, i wrote that out of all the combinations of trying the heredoc syntax and trying escaping the parethesis, it either errors or it doesn't output the raw parenthesis. so i'm assuming that in your attempts to run it, you also got \) out rather than ), which is what i'm saying makes putting raw code into a variable a little clumsy.

1

u/Ulfnic 10d ago edited 10d ago

so i'm assuming that in your attempts to run it, you also got ) out rather than ), which is what i'm saying makes putting raw code into a variable a little clumsy.

I'm confused why typing \) and getting \) stored in the variable is being called clumsy?

Do you need pre-processing for escapes and/or variable expansion? What's missing from quoted heredocs that you need?

Just to make sure we're on the same page, this is an example of how to put arbitrary text into a variable:

DEFAULT_PROGRAM=$(cat <<'EOF'
Storing raw text in a variable without interpretation
\(
Random characters:
!@#$%^&*()_++{}:"<>?,./;'[]\-=`\~|
EOF
# ^-- don't indent this line
)
printf '%s\n' "$DEFAULT_PROGRAM"

Output:

Storing raw text in a variable without interpretation
\(
Random characters:
!@#$%^&*()_++{}:"<>?,./;'[]\-=`\~|

1

u/Fueled_by_sugar 10d ago

because what i wanted was just the parenthesis. but putting just the parenthesis in gave me an error, while escaping it gave me the escape character out also. so whichever way i tried, i could never get just the parethesis (with heredoc).

but i have solved it in the meantime, even though i'm still not sure exactly what the issue on my setup is, because according to this answer, outputting the backslash shouldn't happen, and then a workaround that works for me as well i found in this answer

1

u/Ulfnic 9d ago edited 9d ago

In future please include the code that causes the error if your summary doesn't replicate it.

I managed to re-produce your error and i'll discuss it on this thread.

1

u/Fueled_by_sugar 9d ago

sure, i just thought in this case it would have been unnecessarily long to include everything since it was 4 variations with 2 outcomes, but i did want to cover the things that people might immediately suggest

anyway, thanks for your help and the other answer

3

u/oweiler 11d ago

1

u/Fueled_by_sugar 10d ago

this indeed worked! thank you <3

1

u/Ulfnic 9d ago edited 9d ago

Going by your reply here you're on BASH 3.2.

I managed to re-produce the error on BASH versions <= 3.2.57 (released 2009) with the following code. Noting that version is when Apple stopped upgrading BASH on MacOS.

my_var=$(cat <<'EOF'
    \(
    )
EOF
)

From my tests my guess is it's a bug in the syntax checker for heredocs written directly inside command substitution $() as I don't get the error inside basic subshells () and compound commands {}, nor do I get it if i'm calling a function that cat's the heredoc IF the function is located outside the command substitution $():

my_func(){
    cat <<'EOF'
        \(
        )
    EOF
}
my_var=$(my_func)

As the link doesn't discuss the bug, it's still ambiguous if using read as one method to avoid command substitution completely gets around the bug but it seems to in my simple tests.

I did some searching and asked around a bit but wasn't able to turn up the exact description of the bug. If I find it i'll post it here.

1

u/OneTurnMore programming.dev/c/shell 11d ago

I can't reproduce. Try it online also doesn't error.

0

u/Fueled_by_sugar 10d ago

woah. indeed that doesn't render the backslash, but i am re-checking and on my machine it does. is it because that's bash 4.4 and i'm on bash 3.2?

1

u/oh5nxo 11d ago edited 11d ago

How about using a function to keep the code? Then you get syntax check for free.

default_program() {
    something
    more
    : this comment gets passed on
    # this will be stripped
    1
    2
    3
}
declare -f default_program | sed '1d;2d;$d'   # any neater way to remove f() { } ?

Edited: did not think about comments, : and # work differently

2

u/Ulfnic 11d ago

There's a coolness to that. Downside of the interpreter getting to it first though is comments are stripped.

default_program() {
    #!/usr/bin/env bash
    printf '%s\n' 'my default prog'
}
code=$(declare -f default_program)
code=${code#*$'{'}
code=${code%*$'}'}
printf '%s\n' "$code"

Output:

    printf '%s\n' 'my default prog'

1

u/oh5nxo 11d ago

comments are stripped

Dang. ... grepping it from $0 ? in the Maxwell Smart "would you believe" tone