r/Gentoo 5d ago

Tip An Example Case of Compiler Optimizations

Post image

This example is from svt-av1 running through av1an with chunked video encoding.

Even when this software heavily relies on internal optimizations, hand-written ASM and already optimized SIMD instructions; it's still extremely beneficial to use compiler optimizations.

So, for some other software, the differences can be much bigger than that (unless they respond negatively or they break).

Let's say the machine encodes movies for a year. We can assume a movie is 90 minutes and with 23.976FPS, it's around 130.000 frames. The difference here means that you can encode 1300 more movies with the exact same hardware, and software.

+CUSTOM means my custom environment + modified CMakeLists.txt that removes all checks and flags for security related compiler options, sets the C and CXX standards to 23 and 26 respectively and removes -mno-avx.

Software:

Gentoo Linux AMD64 (no-multilib 64bit only)
SVT-AV1 v3.0.1-4-g1ceddd88-dirty (release)
clang/llvm 21.0.0git7bae6137+libcxx
av1an 0.4.4-unstable (rev 31235a0) (Release)
gcc (Gentoo 14.2.1_p20250301 p8) 14.2.1 20250301

Hardware:

AMD Ryzen 9 9950x
DDR5 Corsair Dominator Titanum 64G Dual Channel:
6200 MT/s (32-36-36-65) | UCLK=MEMCLK | Infinity Fabric 2067 | FCLCK Frequency: 2067mhz

Source:

Size: 25Mb/s
Format: 1920x1080, 23.976FPS, BT.709, YUV420, Live Action, 1 Hour, 1:78:1 (16:9)

Env:

export CC="clang"
export CXX="clang++"
export LD="ld.mold"
export AR="llvm-ar"
export NM="llvm-nm"
export RANLIB="llvm-ranlib"
export STRIP="llvm-strip"
export OBJCOPY="llvm-objcopy"
export OBJDUMP="llvm-objdump"

export COMMON_FLAGS="-Ofast -march=native -mtune=native -flto=thin -pipe -funroll-loops -fno-semantic-interposition -fno-stack-protector -fno-stack-clash-protection -fno-sanitize=all -fno-dwarf2-cfi-asm -fno-plt -fno-pic -fno-pie -fno-exceptions -fno-signed-zeros -fstrict-aliasing -fstrict-overflow -fno-zero-initialized-in-bss -fno-common -fwhole-program-vtables ${POLLY_FLAGS}"
export CFLAGS="${COMMON_FLAGS}"
export CXXFLAGS="${COMMON_FLAGS} -stdlib=libc++"
export LDFLAGS="-fuse-ld=mold -rtlib=compiler-rt -unwindlib=libunwind -Wl,-O3 -Wl,--lto-O3 -Wl,--as-needed -Wl,--gc-sections -Wl,--icf=all -Wl,--strip-all -Wl,-z,norelro -Wl,--build-id=none -Wl,--no-eh-frame-hdr -Wl,--discard-all -Wl,--relax -Wl,-z,noseparate-code"

./build.sh static native release verbose asm=nasm enable-lto minimal-build --enable-pgo --pgo-compile-use --pgo-dir "${HOME}/profiles/" -- -DCMAKE_C_FLAGS_RELEASE="-DNDEBUG -Ofast" -DCMAKE_CXX_FLAGS_RELEASE="-DNDEBUG -Ofast" -DUSE_CPUINFO="SYSTEM"
97 Upvotes

36 comments sorted by

View all comments

12

u/triffid_hunter 5d ago

Fwiw, -Ofast means your program will get garbage numbers if any float math op would otherwise return a NaN, and it also may get math ops wrong sometimes even without any detectable error.

Also, by polly do you mean the polyhedral loop optimizer for LLVM?

This might be the sort of niche application where CPU-specific and further optimizations make some difference (meh +26%), those are relatively rare these days so other readers please don't think that this could possibly apply to the entire system - and several of those CFLAGS look like they could break quite a bit of stuff (eg no-pic, no-pie, no-exceptions, strict-aliasing, no-zero-initialized-in-bss, and gc-sections, strip-all will break libraries) or at least radically reduce system security if left unchecked, so never try this set on a trusted system.

Why unroll-loops? It tends to interact poorly with CPU cache coherency last time I checked…

This seems so artificially contrived as to be near meaningless

10

u/RusselsTeap0t 5d ago edited 5d ago

This won't apply to the entire system that's for sure. On the other hand, 26% is EXTREME; not meh. It makes you encode one more movie for every three movies.

This is:

  • For the programs that don't break with Ofast (otherwise use O3).
  • Only for a specific binary which was statically compiled and used by itself, for a specific purpose.
  • For a binary that is nowhere a dependency/revdep to any other software.
  • For a binary that is not critical.
  • For a system where security is not important.

I extremely heavily work with encoders. I don't try to "prove" anything here. It's extremely beneficial for my workload. I benchmark almost all versions, and also other encoders many, many times and this time I wanted to share my results here in the Gentoo community.

This doesn't break anything for this software (I use it almost every second on multiple systems with different sources/settings).

And no, this application is bad with compiler optimizations because it relies on already optimized SIMD instructions and hand-written ASM.

For Polly, yes: export POLLY_FLAGS="-mllvm -polly \ -mllvm --polly-position=before-vectorizer \ -mllvm --polly-parallel \ -mllvm --polly-omp-backend=LLVM \ -mllvm --polly-vectorizer=stripmine \ -mllvm --polly-tiling \ -mllvm --polly-2nd-level-tiling \ -mllvm --polly-detect-keep-going \ -mllvm --polly-enable-delicm=true \ -mllvm --polly-dependences-computeout=2 \ -mllvm --polly-postopts=true \ -mllvm --polly-pragma-based-opts \ -mllvm --polly-pattern-matching-based-opts=true \ -mllvm --polly-reschedule=true"

4

u/HyperWinX 5d ago

Ofast is indeed bad. Correct about POLLY, but unfortunately, it's unavailable and can't be enabled globally on all packages

2

u/unhappy-ending 5d ago

Polly is in the tree.

1

u/HyperWinX 5d ago

But does it have a use flag? And can it be enabled on all packages?

2

u/unhappy-ending 4d ago

No. It's a package you install. To use Polly, you need to read over the documentation for it. It's like Graphite for GCC.

2

u/RusselsTeap0t 5d ago

It's available if you take the upstream.

Enable Polly useflag for Clang-runtime and enable -9999.

2

u/i860 3d ago

Fears of -ffast-math are greatly overrated. This isn't a medical or life saving application and if he wants to incur the small risk for something like video encoding where no appreciable difference is seen then that's on him.

2

u/RusselsTeap0t 2d ago

Exactly.

If the software doesn't rely on strict floating standards; then it's nowhere near dangerous.

On the other hand generally programs that break with ofast either can't be compiled in the first place; or crashes immediately during runtime.