Bugzilla – Bug 11869
Add LLVM support up to 3.9
Last modified: 2022-03-10 01:07:02 EST
Created attachment 7271 [details] llvm 3.7 support Those three patches were made by Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com> I just forward them upstream
Created attachment 7272 [details] llvm 3.8 support
Created attachment 7273 [details] llvm 3.9 support
Thanks for passing these along! For now, with 0.99.3 right around the corner, we'll have to target these for review in 0.99.4 alongside our other planned llvm work.
We'll try to take a look at it for 0.99.3 and see what can be done, but may still need to hold off until 0.99.4.
It looks like this won't make it into 0.99.3. Was able to successfully configure, make, install on ubuntu when using apt-get to install llvm 3.7, 3.8. (didn't try 3.9). However, if llvm is built from source, the clamav build threw errors. I did my testing for this on macos 10.12. The first round of errors was because "llvm/Config/config.h" didn't exist for llvm 3.7. I got past the compile errors after after adjusting the LLVM_VERSION ifdefs to say < 37 instead of < 38. However, it then threw linker errors with undefined symbols. It may be just an easy of a fix with as the above LLVM_VERSION change, but we will not have the time/resources to figure it out and verify it on each of our supported platforms for each LLVM version before the 0.99.3 release.
*** Bug 11913 has been marked as a duplicate of this bug. ***
(In reply to Micah Snyder from comment #5) > It looks like this won't make it into 0.99.3. > > Was able to successfully configure, make, install on ubuntu when using > apt-get to install llvm 3.7, 3.8. (didn't try 3.9). > > However, if llvm is built from source, the clamav build threw errors. > I did my testing for this on macos 10.12. The first round of errors was > because "llvm/Config/config.h" didn't exist for llvm 3.7. I got past the > compile errors after after adjusting the LLVM_VERSION ifdefs to say < 37 > instead of < 38. However, it then threw linker errors with undefined > symbols. > > It may be just an easy of a fix with as the above LLVM_VERSION change, but > we will not have the time/resources to figure it out and verify it on each > of our supported platforms for each LLVM version before the 0.99.3 release. Thanks for looking into this. You could shorten the list of supported llvm versions. llvm 2.8 is what you have in tree and this was released Oct 2010. You could lift the minimal requirement to 3.9 or so. There is no reason to support the old and ancient ones. The whole ifdefery does not ease the maintaince :) On the other hand, newer llvm broke 32bit-powerpc support. However if nobody is going to work on it then it is probably not that important (ppc64le is supported). Sebastian
Related ticket: https://bugzilla.clamav.net/show_bug.cgi?id=11606 Basically this ticket is a dup of that ticket, but since they both have some good info in them, I guess we can keep both tickets open until we update LLVM support.
Created attachment 7404 [details] Combined_LLVM-3.9.patch I'm not a coder, so the patch may not be true. But I used three patches of Sebastian A. Siewior as a reference and tried to make Combined_LLVM-3.9.patch. I hope you find it informative. ---------------------------------------------------- 0. Source Code: from latest /dev/0.100 (commit 4553e63651efb5187f518887d762c8473f5d18f2) 1. Applied Combined_LLVM-3.9.patch 2. Build Envioroment Mac OS X 10.12.6 Darwin Kernel Version 16.7.0 pkgutil version: 9.2.0.0.1.1510905681 LLVM 3.9.0 : /usr/local/clang+llvm-3.9.0-x86_64-apple-darwin pcre2-10.31 : /usr/local/pcre libressl-2.7.0 : /usr/local/libressl 3. Configuration: export CFLAGS="-O3 -march=nocona" export CXXFLAGS="-O3 -march=nocona" export CPPFLAGS="-I/usr/local/pcre/include -I/usr/local/libressl/include" ./configure --prefix=/usr/local/clamXav --build=x86_64-apple-darwin`uname -r` --with-pcre=/usr/local/pcre --with-openssl=/usr/local/libressl--with-system-llvm=/usr/local/clang+llvm-3.9.0-x86_64-apple-darwin/bin/llvm-config --with-llvm-linking=static 4. Results: make && make check complete with no error, but freshclam aborts as follws: -------------------------- freshclam --version dyld: Library not loaded: @rpath/libc++.1.dylib Referenced from: /usr/local/clamXav/bin/freshclam Reason: image not found Abort trap: 6d 5. For your reference: -------------------------- clamd --version ClamAV 0.100.0-rc/24418/Sun Mar 25 13:24:35 2018 -------------------------- clambc --version Clam AntiVirus Bytecode Testing Tool 0.100.0-rc LLVM (http://llvm.org/): LLVM version 3.9.0 Optimized build. Default target: x86_64-apple-darwin16.7.0 Host CPU: westmere
(In reply to Eiichi Yokota from comment #9) > Created attachment 7404 [details] > Combined_LLVM-3.9.patch > > I'm not a coder, so the patch may not be true. > But I used three patches of Sebastian A. Siewior as a reference and tried to > make Combined_LLVM-3.9.patch. This is not why submitted *three* patches. Also you lost Authorship. Sebastian
(In reply to Sebastian A. Siewior from comment #10) > This is not why submitted *three* patches. Also you lost Authorship. > > Sebastian Sebastian, I'm sorry for your feeling of discomfort. I only wanted to check whether your patches work well, and I'm not going to violate your Authorship. Sorry again and sorry for my bad English. Eiichi
Created attachment 7405 [details] Configure.patch for LLVM I've tested to build clamav-0.100.0-rc under varying conditions. I hope you find it informative. A. Build Environment Mac OS X 10.12.6 Darwin Kernel Version 16.7.0 pkgutil version: 9.2.0.0.1.1510905681 LLVM 3.7.0: /usr/local/clang+llvm-3.7.0-x86_64-apple-darwin LLVM 3.8.0: /usr/local/clang+llvm-3.8.0-x86_64-apple-darwin LLVM 3.9.0: /usr/local/clang+llvm-3.9.0-x86_64-apple-darwin pcre2-10.31: /usr/local/pcre libressl-2.7.1: /usr/local/libressl B. LLVM 3.7 ---------------------------------------------------------------------- Configuration: --with-system-llvm=/usr/local/clang+llvm-3.7.0-x86_64-apple-darwin/bin/llvm-config --with-llvm-linking=static ________________________________________________ Case (1) Applied: 'Add-support-for-LLVM-3.7.patch' ====================== Results: configure fails configure: error: Failed to configure LLVM, and LLVM was explicitly requested ________________________________________________ Case (2) Sources + 'Add-support-for-LLVM-3.7.patch' + 'Configure.patch' ====================== Results: configure && make && make check succeed All 6 tests passed C. LLVM 3.8 ---------------------------------------------------------------------- --with-system-llvm=/usr/local/clang+llvm-3.8.0-x86_64-apple-darwin/bin/llvm-config --with-llvm-linking=static ________________________________________________ Case (1) Applied: 'Add-support-for-LLVM-3.7.patch' + 'Add-support-for-LLVM-3.8.patch' ====================== Results: configure fails configure: error: Failed to configure LLVM, and LLVM was explicitly requested ________________________________________________ Case (2) Applied: 'Add-support-for-LLVM-3.7.patch' + 'Add-support-for-LLVM-3.8.patch' + 'Configure.patch' ====================== Results: configure succeeds but make fails clang: warning: no such sysroot directory: '/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.10.sdk' [-Wmissing-sysroot] bytecode2llvm.cpp:24:10: fatal error: 'pthread.h' file not found #include <pthread.h> ^~~~~~~~~~~ 1 error generated. make[5]: *** [libclamavcxx_la-bytecode2llvm.lo] Error 1 make[4]: *** [all] Error 2 make[3]: *** [all-recursive] Error 1 make[2]: *** [all] Error 2 make[1]: *** [all-recursive] Error 1 make: *** [all] Error 2 D. LLVM 3.9 ---------------------------------------------------------------------- --with-system-llvm=/usr/local/clang+llvm-3.9.0-x86_64-apple-darwin/bin/llvm-config --with-llvm-linking=static ________________________________________________ Case (1) Applied: 'Add-support-for-LLVM-3.7.patch' + 'Add-support-for-LLVM-3.8.patch'+ 'Add-support-for-LLVM-3.9.patch' ====================== Results: configure fails configure: error: Failed to configure LLVM, and LLVM was explicitly requested ________________________________________________ Case (2) Applied: 'Add-support-for-LLVM-3.7.patch' + 'Add-support-for-LLVM-3.8.patch'+ 'Add-support-for-LLVM-3.9.patch' + 'Configure.patch' ====================== Results: configure && make && make check succeed All 6 tests passed And details are as noted above in Comment 9.
Additional Info: Building ClamAV 0.100.0-rc with LLVM 3.9.0 I hope you find it informative. I found the following: 1. Original permission of /usr/local/clamXav/bin/freshclam is: -rwxr-xr-x 1 root wheel 162140 Apr 1 15:46 freshclam In this case 'freshclam --version' works properly. ClamAV 0.100.0-rc/24439/Sun Apr 1 13:20:07 2018 2. To compatible with ClamXav(https://www.clamxav.com/), I set permission of /usr/local/clamXav/bin/freshclam as follws: 1) Change owner from root to clamav: -rwxr-xr-x 1 _clamav wheel 162140 Apr 1 16:01 freshclam In this case 'freshclam --version' works properly. ClamAV 0.100.0-rc/24439/Sun Apr 1 13:20:07 2018 2) Add s-bit to owner: -rwsr-xr-x 1 _clamav wheel 162140 Apr 1 16:01 freshclam And in this case 'freshclam --version' aborts as follws: -------------------------- freshclam --version dyld: Library not loaded: @rpath/libc++.1.dylib Referenced from: /usr/local/clamXav2/bin/freshclam Reason: image not found Abort trap: 6 3. ClamAV 0.100.0-rc with LLVM 3.7.0 is compatible with ClamXav.
Thanks for your work testing the patches. I look forward to spending more time on Clam improvements such as this after the 0.100 release. With regards to the s-bit issue with freshclam. That appears to be an unrelated issue. Care to create a new bugzilla ticket to report it?
(In reply to Micah Snyder from comment #14) > With regards to the s-bit issue with freshclam. That appears to be an > unrelated issue. Care to create a new bugzilla ticket to report it? Thanks a lot. I submitted Bug 12074.
Thanks!
The packaged 101 in debian still caries the llvm 3.7 to 3.9 patches. https://sources.debian.org/src/clamav/0.101.0+dfsg-1/debian/patches/ Has this been overseen?
ping :) We (in Debian) still ship these patches but don't use them since llvm in Debian moved. In current stable (Buster) we have llvm 6 and 7 while the next stable release has currently 8, 9 and 10 with 9 as the default version. Are there any plans to update the llvm support? Sebastian
pong! Yes! Andy, our newest engineer, has been working hard on upgrading the bytecode compiler first. Andy came with a lot of LLVM experience so we were excited to have him clean up the compiler and bring it up to date with llvm + clang. However, Andy is currently taking a break from the compiler to complete 0.103 tasks and meet deadlines. After 0.103 is published, and once the compiler overhaul is complete, his next task will be updating our llvm bytecode runtime support to llvm 8/9/10. I guess I may as well assign this ticket his way, though it'll probably be a few months before he takes a look the libllvm runtime support. -Micah
We just merged support for LLVM 8 - 12: https://github.com/Cisco-Talos/clamav/pull/366 Versions older than 8 will not be supported. There are some outstanding issues with LLVM debug builds and with LLVM detection on for some packages where libPolly.a appears to be missing. So for now we won't recommend using LLVM as the runtime and will continue to have the bytecode interpreter as the default runtime. But it does work now!