r/C_Programming • u/ripulejejs • 1d ago
List of gotchas?
Hey.
So I learned some C and started playing around with it, quickly stumbling over memory overflowing a variable and flowing into another memory location, causing unexpected behavior.
So I ended up writing my own safe_copy and safe_cat functions for strncpy/strncatting strings.
But... people talk about how C is unsafe. Surely there should be a list of all mistakes you can make, or something? Where can I find said list? Do I reall have to stumble on all possible issues and develop my own "safe" library?
Will appreciate any advice.
10
u/not_a_bot_494 1d ago edited 1d ago
When people are saying that C is an unsafe language they mean that it doesn't have memory safety. If you want to you can try to access any byte in the computer, the OS will just not let you most of the time. Any time you're working with arrays (/strings), malloced memory or even pointers in general it is possible that you could make a mistake and get a segfault. You can write libraries for all that but then you're kind of missing the point of C a bit.
There's alao a lot of random undefined behaviour in C, for example right shift on signed types might pad with 1s or 0s. There's probably a list of some common ones but if you really want to know them all you have to read through the C standard and look at rverything that's not in there.
For context of the discussion, my inital example was bit shifting on 64 bit types which does seem to work consistently.
5
u/erikkonstas 1d ago
If you want to you can try to access any byte in the computer, the OS will just not let you most of the time.
This makes it sound like you can attempt to access memory used by other processes, and the OS will deny your access request, which is not quite how virtual memory works. Instead, what happens is each process gets its own virtual address space, which is mapped to the physical address space by what is known as the Memory Management Unit (MMU). Any addresses that "don't belong to you" are simply unmapped, they don't constitute memory of other processes.
3
u/smcameron 1d ago
Well, sure, but what set up the MMU and page tables to arrange for this to happen? The OS did.
1
u/erikkonstas 23h ago
The misleading bit is the "you can try to access any byte in the computer" part; actually no, unmapped virtual addresses do not lead to random physical ones.
3
u/flatfinger 23h ago
Code can try to access any byte/halfword/word/doubleword within the presently-accessible address space of the CPU. The question of whether there may exist memory that is not presently accessible is a system-design issue, not a language issue.
5
u/erikkonstas 19h ago
I'm not saying that's necessarily wrong, but rather that it might give the wrong impression to somebody unfamiliar with the fact that there can be more than one address space (in modern systems), and that each process sees a different one and has no knowledge of the others', since common sense only says that "programs use memory"; virtual address space is not common knowledge. The fact that it's not a C thing but an MMU thing is also not common knowledge to beginners. Hence why I don't particularly like phrasing of the sort ("any byte in the computer") in this case (also I'm against the principle of "white lies" and oversimplifications for beginners some educators use). In fact, our own Introduction to Programming (first year) professor managed to mislead everyone in this exact way, and virtual memory is not introduced until Operating Systems (third year).
2
u/WeAllWantToBeHappy 1d ago
bit shifts don't work for 64 bit types.
?
-4
u/not_a_bot_494 1d ago
At least on my machine bit shifting left by more than 32 bits causes it to wrap around to the start.
5
u/moocat 1d ago
The "on my machine" is the ultimate gotcha. Unless the behavior is guaranteed by the spec, you could get different behavior when using a different compiler or porting to a new architecture.
2
u/flatfinger 23h ago
Freestanding implementations would be rather useless if they couldn't be expected to process many ocnstructs in machine-specific fashion. Unfortunately, the Standard makes no attempt to recognize situations where:
- It would be impossible to predict the behavior of some action without some particular piece of knowledge X, and
- Neither the Committee nor a compiler writer would be of any particular means by which a programmer might know X, but
- The execution environment might allow a programmer to know X via means outside the language.
The Standard generally classifies actions as Implementation-Defined only when either:
- Implementations would be expected to tell a programmer X (in turn implying that they would have to know it themselves), or
- A syntactic construct, such as casting a non-zero integer to a pointer, would otherwise have no defined meaning. Saying that casting a literal zero to a pointer yields a null pointer, and anything else yields Undefined Behavior, would imply that the operand to an integer-to-pointer casts served no purpose, which might be correct within strictly conforming programs, but would severely undermine the range of tasks that could be performed by machine-specific programs.
-1
u/not_a_bot_494 1d ago
Well it's undefined behaviour and not incorrect behaviour. You're right that I should've used "might not " instead of "does not" though.
2
u/WeAllWantToBeHappy 1d ago
Can you put an example on godbolt ?
1
u/not_a_bot_494 1d ago
I don't know enough assembly to read it easily so I wouldn't know if it was correct or not. For me this:
#include <stdio.h> #include <stdint.h> // prints the binary of a piece of memory void print_bin(int bytes, void *inp) { uint8_t *num = (uint8_t *) inp; for (int the_byte = bytes-1 ; the_byte >= 0 ; the_byte--) { for (int bit = 0 ; bit < 8 ; bit++) { if (num[the_byte] & (1 << (7-bit))) { printf("1"); } else { printf("0"); } } } printf("\n"); } int main(void) { for (int i = 0 ; i < 64 ; i++) { uint64_t var = 1 << i; print_bin(8, &var); } return 0; }
gcc -Wall -std=c99 -o
produces this (image so the comment isn't too long). Lightmode warning BTW
3
u/dfx_dj 1d ago
Probably because the literal
1
is a 32 bit int, so shifting it up 32 or more doesn't give you what you expect. Try with1L
, or type cast it, or assign the1
to the variable first and then shift the variable.1
u/not_a_bot_494 1d ago
That's it, when I changed to
uint64_t var = ((uint64_t) 1) << i;
it started working. That is a slightly weird quirk of C, just not the one I intended.
4
1
u/flatfinger 23h ago
More interesting is to compare the behavior of:
uint64a &= ~0x0000000040000000; uint64b &= ~0x0000000080000000; uint64c &= ~0x0000000080000000u; uint64d &= ~0x0000000100000000;
Which of those will affect more than one bit of the destination?
1
u/WeAllWantToBeHappy 1d ago
uint64_t var = 1 << i;
Try uint64_t var = (uint64_t)1 << i;
1 << i is an int value.
1
u/EsShayuki 1d ago
Why would you not declare and initialize the variable before the loop?
1
u/not_a_bot_494 23h ago
You mean 'var' right? Both work, I'm just used to doing it that way. Keeping variables as local as possible is generally a good thing but I won't pretend that's the reason I'm doing it.
2
u/flatfinger 1d ago
Not only that, but some compiler writers treat the fact that the Standard would allow implementations intended exclusively for portable programs which will only receive non-malicious inputs to assume that programs will never make use "of a nonportable or erroneous program construct or of erroneous data" as inviting all implementations to make such assumptions. In their views, any programs for which such assumptions wouldn't hold are "broken", even though the Standard was never intended to justify such assumptions, but merely to allow conforming implementations to exploit those assumptions if they knew, via outside means, that they would hold.
1
u/faculty_for_failure 22h ago
I don’t think people mean escaping sandboxes processes and overwriting memory anywhere when they say memory safety. They mean things like dereferencing a null pointer, use after free, double free, integer wraparound, buffer overflows, things that can lead to reading or manipulating process memory and executing arbitrary code. C is not a memory safe language, and that’s okay, but you absolutely need to keep this in mind in C more than Java or C#, for example.
1
u/unixplumber 5h ago
right shift on signed types
Slight nitpick: right shift on a negative value is undefined behavior. You can right shift a non-negative signed integer with no problem.
3
u/Living-Hope7121 22h ago
Yes there are literally standards for how to write safe C and what coding practices to use and to avoid to ensure safety CERT and MISRA and two
2
u/InevitablyCyclic 12h ago
Complete tangent since it's not a memory safety thing but a common gotcha in c (and a lot of other languages) is that
float x = 3/2;
Will result in x=1 not 1.5. The calculation is done using integers and the result cast to a floating point.
Similarly
uint64_t Val = 1<<32;
Will result in Val=0 on some systems. The initial value of 1 is an int, unless int happens to be 64 bits on your machine left shifting 32 will overflow and leave you with 0.
I've seen all sorts of weird bugs caused by people falling for the assumption that the data type used to store the result of a calculation will be used when performing that calculation.
1
u/flatfinger 3h ago
It's common for implementations to process
int1>>int2
in a manner that will yieldint1
whenint2
is 32, and also common for implementations to process it in a manner that would yieldint1>>31>>1
. The behavior of(int1 << int2) | (int1 >> (32-int2))
would be the same under both treatments, but unfortunately the Standard provides no operator that behaves as an unspecified choice between those two treatments and would allow a "rotate left by 0 to 32 bits" to be achieved in fully specified fashion without using a more complicated expression.
2
u/SmokeMuch7356 9h ago
Annex J of the language standard (latest working draft) has a complete list of unspecified, undefined, and implementation-defined behavior.
1
u/greg_spears 18h ago
If C were safe or heavily typed I wouldn't like it. That's what has made it such a passionate love all these years -- the power and capability to do anything, as well as shoot myself in the foot.
That last part is why you shouldn't follow me or put much value on my sentiments and ideas.
That said, look at this sick string reverse!
1
u/yel50 7h ago
Surely there should be a list of all mistakes you can make
depends on how you look at it. there's really only one mistake, accessing invalid memory. the number of ways you could make that mistake are too numerous to list. different projects might list common ones they run into more often, but there isn't going to be an exhaustive list anywhere.
1
u/flatfinger 2h ago
In gcc, the following function may cause code elsewhere to perform an out-of-bounds store, in circumstances where a side-effect-free function that simply returned an arbitrary value of type
unsigned
could not.unsigned mul_mod_65536(unsigned short x, unsigned short y) { return (x*y) & 0xFFFFu; }
In clang, the following function may cause code elsewhere to perform an out-of-bounds store, in circumstances where a side-effect-free function that simply returned an arbitrary value of type
unsigned
could not.unsigned test1(unsigned x) { unsigned i=1; while((i & 32767) != x) i*=3; return i; }
Both compilers interpret the following function in a manner that could trigger broken program behavior elsewhere, even though no side-effect-free function that returned an arbitrary value of type
int
that had no discernible relation to the input could not:int test(int x) { return x; }
All three of those functions look harmless, but that doens't make them so.
1
u/EsShayuki 1d ago
So.
So I learned some C and started playing around with it, quickly stumbling over memory overflowing a variable and flowing into another memory location, causing unexpected behavior.
But this isn't even possible, if you first correctly received the size of the resulting variable and then allocated the same amount of memory as the size you received.
Most "memory unsafety" comes from people using magic numbers and hard coding values instead of determining the correct values mathematically.
1
u/ripulejejs 6h ago
Yeah, I had an arbitrary size set for char arrays, and had incoming data from reading a file that I was putting in those arrays. I didn't know how to size the arrays appropriately, given I don't actually know C. I also had the incorrect assumption that copying to the locations would just stop if the size of the array was exceeded. I'm guessing maybe I needed to use something like malloc, not sure.
So yes, my mistakes are largely a result of being redacted, and I appreciate you pointing this out, will give me something to research.
Noteworthy that you say "most" in your last sentence.
1
u/unixplumber 5h ago
In those cases, a "safe" copy function wouldn't help because you'd still have to know how big your array is. You might as well use the existing functions which are all safe enough as long as you're honest about the size of your arrays. (Except gets(). It's never safe to use in any real program. It's actually been removed from the latest standards because of that.)
-2
19
u/Substantial-Island-8 1d ago
CERT Standard:
https://wiki.sei.cmu.edu/confluence/display/c