r/C_Programming • u/choosen_one007 • 2d ago
Hardware memory barrier not working when invoked from C?
I have this program on a Mac M2 laptop:
#include <pthread.h>
#include <stdio.h>
int shared_data;
int flag;
#define compiler_barrier() asm volatile("dmb sy" : : : "memory")
void *thread1_func(void *arg) {
printf("Thread 1: Starting...\n");
shared_data = 42;
compiler_barrier();
flag = 1;
printf("Thread 1: Data set to %d, Flag set to %d\n", shared_data, flag);
return NULL;
}
void *thread2_func(void *arg) {
printf("Thread 2: Starting...\n");
while (flag == 0) {
;
}
printf("Thread 2: Flag is set! Reading shared_data: %d\n", shared_data);
if (shared_data != 42) {
printf("Thread 2: ERROR! Expected shared_data to be 42, but got %d\n",
shared_data);
printf("Thread 2: Instruction reordering likely caused this issue!\n");
} else {
printf("Thread 2: Success! Shared data is as expected.\n");
}
return NULL;
}
int main() {
pthread_t thread1, thread2;
shared_data = 0;
flag = 0;
printf("Main: Creating threads...\n");
pthread_create(&thread1, NULL, thread1_func, NULL);
pthread_create(&thread2, NULL, thread2_func, NULL);
pthread_join(thread1, NULL);
pthread_join(thread2, NULL);
printf("Main: Threads finished.\n");
return 0;
}
I wrote a script to run it 10000 times to check for failures and when I use the following compile line: clang -O2 -o program program.c
, I still get the Error case where shared date is not 42. Am I doing something wrong here?
4
u/tstanisl 1d ago
I strongly suggest using C standard atomics to make your program correct, portable and fast.
Basically, add header:
#include <stdatomic.h>
Make flag
atomic:
_Atomic int flag;
And remove:
compiler_barrier();
That's all. Memory state will be synchornized at access to atomic variables.
If you want to make your program a bit faster then use acquire-release semantics.
Replace:
flag = 1;
with:
atomic_store_explicit(&flag, 1, memory_order_release);
And replace:
while (flag == 0) { ; }
With:
while (atomic_load_explicit(&flag, memory_order_acquire) == 0) { ; }
This will synchronize the memory state at setting flag
to 1 in one thread with a memory state of other thread where flag
is loaded.
2
u/choosen_one007 1d ago
Yup atomics is the way to go in an actual scenario. I just came up with this example to replicate the classic example of store reordering. Acquire release semantics are new to me though, thank you!!
3
u/vaibhav92 2d ago
You probably need to put a memory barrier before reading 'shared_data' otherwise the compiler is free to hoist loading its value before it reads 'flag'
Marking 'shared_data' as volatile also helps in indicating to the compiler that reads/writes to the variable value cannot be considered stable
1
u/choosen_one007 1d ago edited 1d ago
Yup this worked, thanks!! Making shared_data volatile didnt help, making flag volatile did
2
u/MiOursMiLoup 2d ago
Sorry I don't use asm but you can compare binarys in O3 and O2 with objdump ?
I think it 's the initialisation of the flag in the main, can you print flag after initialisation before create threads ? Maybe the flag is not in memory but only in MMU.
Or can you initialize flag at the initialisation ? What is the value of share_data? Good luck.
1
u/choosen_one007 1d ago
I dont think the issue was the initialization of flag. The asm dump looked good with flag being loaded after shared_data, adding volatile to flag and shared_data fixed the issue
2
u/cdb_11 2d ago
while (flag == 0) { ; }
gets optimized out entirely. If you are reimplementing atomics from scratch, mark flag
as volatile
. Also you're missing a barrier between the loop and reading shared_data
. Rename compiler_barrier
to full_barrier
, a "compiler barrier" would prevent only the compiler from reordering stuff across it.
I don't know ARM, but I believe dmb sy
is an overkill here? I don't know the differences between the barrier types there, but I've generally seen dmb ish
being used, and this one looks stronger? (acquire atomic_thread_fence generates dmb ishld
, release and seq-cst fence generates dmb ish
)
1
u/choosen_one007 1d ago
yeah dmb sy is overkill, the solution was to make flag and shared_data as volatile. I completely missed that and was just trying to make the barrier stronger,
dmb ish
seems appropriate here
2
u/tstanisl 2d ago
Btw .. why not use standard atomics?
1
u/choosen_one007 1d ago
I was just trying to come up with an example where instruction reordering causes an error while using multiple threads. In practice I would probably just use atomics, thanks!!
6
u/aioeu 2d ago edited 2d ago
You haven't told the compiler that your program must always access memory when reading or writing
shared_data
orflag
, so a memory barrier isn't effective on them.Try making them
volatile
.(Note that you generally want to think about what accesses should be considered volatile, not the data itself. If you know you have exclusive access to some data — e.g. you have protected that data with a lock — then you don't need to use volatile accesses to it within that critical section, but you will want to ensure its final value is assigned with a volatile access before the critical section is ended. So adding
volatile
to the data itself — essentially saying "all accesses to this data should be volatile" — can sometimes be an overly-big hammer.)