Bug 12596 - "make check" clamav-0.103.0-rc fails on macOS 10.15 Catalina
"make check" clamav-0.103.0-rc fails on macOS 10.15 Catalina
Status: CLOSED FIXED
Product: ClamAV
Classification: ClamAV
Component: All
other
x86_64 Mac OS
: P3 normal
: 0.101.0
Assigned To: ClamAV team
:
Depends on:
Blocks:
  Show dependency treegraph
 
Reported: 2020-08-18 22:57 EDT by Eiichi Yokota
Modified: 2021-11-02 13:46 EDT (History)
3 users (show)

See Also:
QA Contact:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Eiichi Yokota 2020-08-18 22:57:45 EDT
1) Build environment
------------------
OS: macOS Catalina 10.15.6 (19G2021)
Darwin Kernel Version 19.6.0
CLTools_Executables version: 11.5.0.0.1

2) I can complete compiling clamav-0.103.0-rc as below:
===============================================
Making all in clamdtop
  CC       clamdtop.o
  CCLD     clamdtop
Making all in clambc
  CC       bcrun.o
  CCLD     clambc
Making all in unit_tests
make[2]: Nothing to be done for `all'.
===============================================


3) "make check" fails ending up with: 
===============================================
Making check in unit_tests
/Applications/Xcode.app/Contents/Developer/usr/bin/make  check_clamav check_clamd check_fpu_endian check_freshclam.sh check_sigtool.sh check_unit_vg.sh check1_clamscan.sh check2_clamd.sh check3_clamd.sh check4_clamd.sh check5_clamd_vg.sh check6_clamd_vg.sh check7_clamd_hg.sh check8_clamd_hg.sh check9_clamscan_vg.sh
  CC       check_clamav-check_clamav_skip.o
check_clamav_skip.c:4:10: fatal error: 'clamav.h' file not found
#include "clamav.h"
         ^~~~~~~~~~
1 error generated.
make[2]: *** [check_clamav-check_clamav_skip.o] Error 1
make[1]: *** [check-am] Error 2
make: *** [check-recursive] Error 1
===============================================


4) Configure option:
--------------------
./configure --with-openssl=/usr/local/ssl --with-pcre=/usr/local/pcre  --enable-llvm=no
Comment 1 Eiichi Yokota 2020-08-19 02:57:04 EDT
1) 'clamav.h' file exists at /libclamav/clamav.h, so I tested to patch file:
===============================================
--- a/unit_tests/check_clamav_skip.c
+++ b/unit_tests/check_clamav_skip.c
@@ -1,7 +1,5 @@
 #include <stdio.h>
-
-// libclamav
-#include "clamav.h"
+#include "../libclamav/clamav.h"
 
 int main(int argc, char **argv)
 {

===============================================

2) make && make check sucseeds as below:
============================================================================
/Applications/Xcode.app/Contents/Developer/usr/bin/make  check-TESTS
SKIP: check_clamav
PASS: check_freshclam.sh
PASS: check_sigtool.sh
SKIP: check_unit_vg.sh
PASS: check1_clamscan.sh
PASS: check2_clamd.sh
PASS: check3_clamd.sh
PASS: check4_clamd.sh
SKIP: check5_clamd_vg.sh
SKIP: check6_clamd_vg.sh
SKIP: check7_clamd_hg.sh
SKIP: check8_clamd_hg.sh
SKIP: check9_clamscan_vg.sh
============================================================================
Testsuite summary for ClamAV 0.103.0-rc
============================================================================
# TOTAL: 13
# PASS:  6
# SKIP:  7
# XFAIL: 0
# FAIL:  0
# XPASS: 0
# ERROR: 0
============================================================================
Comment 2 Micah Snyder 2020-08-19 20:25:02 EDT
Hi Eiichi,

Thanks for testing 0.103.0-rc!

It appears that we broke the building those tests for systems that are missing libcheck. 

Before you go and install libcheck, can you please try testing to see if removing the #include "clamav.h" in check_clamav_skip.c will fix the issue?  I don't see any reason why clamav.h would be needed in that file.
https://github.com/Cisco-Talos/clamav-devel/blob/dev/0.103/unit_tests/check_clamav_skip.c#L4  If that resolved the error, I'll make the same change prior to release.

libcheck is required for the clamav_check unit tests to function, so after you do the above test, you'll probably want to install libcheck.  You can manually build and install it, or use homebrew to `brew install check`.
Comment 3 Eiichi Yokota 2020-08-19 21:49:35 EDT
(In reply to Micah Snyder from comment #2)
> Before you go and install libcheck, can you please try testing to see if
> removing the #include "clamav.h" in check_clamav_skip.c will fix the issue? 
> I don't see any reason why clamav.h would be needed in that file.
> https://github.com/Cisco-Talos/clamav-devel/blob/dev/0.103/unit_tests/
> check_clamav_skip.c#L4  If that resolved the error, I'll make the same
> change prior to release.

I tested removing the #include "clamav.h" in check_clamav_skip.c.

"make check" fails ending up with: 
===============================================
Making check in unit_tests
/Applications/Xcode.app/Contents/Developer/usr/bin/make  check_clamav check_clamd check_fpu_endian check_freshclam.sh check_sigtool.sh check_unit_vg.sh check1_clamscan.sh check2_clamd.sh check3_clamd.sh check4_clamd.sh check5_clamd_vg.sh check6_clamd_vg.sh check7_clamd_hg.sh check8_clamd_hg.sh check9_clamscan_vg.sh
  CC       check_clamav-check_clamav_skip.o
check_clamav_skip.c:6:5: warning: implicit declaration of function 'UNUSEDPARAM' is invalid in C99
      [-Wimplicit-function-declaration]
    UNUSEDPARAM(argc);
    ^
1 warning generated.
  CCLD     check_clamav
Undefined symbols for architecture x86_64:
  "_UNUSEDPARAM", referenced from:
      _main in check_clamav-check_clamav_skip.o
ld: symbol(s) not found for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)
make[2]: *** [check_clamav] Error 1
make[1]: *** [check-am] Error 2
make: *** [check-recursive] Error 1
===============================================


> libcheck is required for the clamav_check unit tests to function, so after
> you do the above test, you'll probably want to install libcheck.  You can
> manually build and install it, or use homebrew to `brew install check`.
Was libcheck required for the clamav_check unit tests when we make check ClamAV 0.102.4?
Comment 4 Micah Snyder 2020-08-19 22:10:56 EDT
My mistake, I didn't realize the UNUSEDPARAM macros were of our own doing.  The change needed is:

diff --git a/unit_tests/check_clamav_skip.c b/unit_tests/check_clamav_skip.c
index 96f6cf196..d8ad862f5 100644
--- a/unit_tests/check_clamav_skip.c
+++ b/unit_tests/check_clamav_skip.c
@@ -1,12 +1,9 @@
 #include <stdio.h>

-// libclamav
-#include "clamav.h"
-
 int main(int argc, char **argv)
 {
-    UNUSEDPARAM(argc);
-    UNUSEDPARAM(argv);
+    (void)argc;
+    (void)argv;
     puts("\n*** Unit tests disabled in this build\n*** Use ./configure --enable-check to enable them\n");
     /* tell automake the test was skipped */
     return 77;


Yes in 0.102, libcheck was also required for the unit tests to work. 

I suspect that the check_clamav test was skipped for you in the past.

When adding CMake support, and eliminating Autoconf warnings - I changed the #include's to remove relative pathing from one build target (like unit_tests) to another (like libclamav), resulting in the error you ran into.

I reproduced the issue you described using an ubuntu docker container that also lacked libcheck, and then got it to work with the above change.

Output with missing libcheck (and skipped check_clamav test should end up looking like this:

make[3]: Entering directory '/tmp/clamav-0.103.0-rc/build/unit_tests'
SKIP: check_clamav
PASS: check_freshclam.sh
PASS: check_sigtool.sh
SKIP: check_unit_vg.sh
PASS: check1_clamscan.sh
PASS: check2_clamd.sh
PASS: check3_clamd.sh
PASS: check4_clamd.sh
SKIP: check5_clamd_vg.sh
SKIP: check6_clamd_vg.sh
SKIP: check7_clamd_hg.sh
SKIP: check8_clamd_hg.sh
SKIP: check9_clamscan_vg.sh
============================================================================
Testsuite summary for ClamAV 0.103.0-rc
============================================================================
# TOTAL: 13
# PASS:  6
# SKIP:  7
# XFAIL: 0
# FAIL:  0
# XPASS: 0
# ERROR: 0
============================================================================
Comment 5 Eiichi Yokota 2020-08-19 22:36:47 EDT
(In reply to Micah Snyder from comment #4) 
> The change needed is:
> 
> diff --git a/unit_tests/check_clamav_skip.c b/unit_tests/check_clamav_skip.c
> index 96f6cf196..d8ad862f5 100644
> --- a/unit_tests/check_clamav_skip.c
> +++ b/unit_tests/check_clamav_skip.c
> @@ -1,12 +1,9 @@
>  #include <stdio.h>
> 
> -// libclamav
> -#include "clamav.h"
> -
>  int main(int argc, char **argv)
>  {
> -    UNUSEDPARAM(argc);
> -    UNUSEDPARAM(argv);
> +    (void)argc;
> +    (void)argv;
>      puts("\n*** Unit tests disabled in this build\n*** Use ./configure
> --enable-check to enable them\n");
>      /* tell automake the test was skipped */
>      return 77;


Thanks for your support.

1) I applied the patch to /unit_tests/check_clamav_skip.c.
2) I haven't installed libcheck, so skipped check_clamav, so 'make check'ended as below:
============================================================================
/Applications/Xcode.app/Contents/Developer/usr/bin/make  check-TESTS
SKIP: check_clamav
PASS: check_freshclam.sh
PASS: check_sigtool.sh
SKIP: check_unit_vg.sh
PASS: check1_clamscan.sh
PASS: check2_clamd.sh
PASS: check3_clamd.sh
PASS: check4_clamd.sh
SKIP: check5_clamd_vg.sh
SKIP: check6_clamd_vg.sh
SKIP: check7_clamd_hg.sh
SKIP: check8_clamd_hg.sh
SKIP: check9_clamscan_vg.sh
============================================================================
Testsuite summary for ClamAV 0.103.0-rc
============================================================================
# TOTAL: 13
# PASS:  6
# SKIP:  7
# XFAIL: 0
# FAIL:  0
# XPASS: 0
# ERROR: 0
============================================================================
Comment 6 Micah Snyder 2020-08-19 23:03:55 EDT
Great! I'll put in a PR to have the same change be in in 0.103.0 (or another release candidate, whichever comes first).

Thanks for your help finding bugs. Please open another ticket if you encounter any other issues with the release candidate(s)!

-Micah
Comment 7 Micah Snyder 2020-08-25 01:38:45 EDT
I just merged a fix for this issue: https://github.com/Cisco-Talos/clamav-devel/commit/be2a91cc56e0fb19e190b11aad129ddb303d15ab

We'll publish a second release candidate later this week or early next week which will have the fix.

Thanks for reporting the issue!

Micah
Comment 8 Eiichi Yokota 2020-08-25 03:21:15 EDT
Thanks a lot.
Now I'm running and testing ClamAV 0.103.0-rc, and I would like to test ClamAV 0.103.0-rc2.
Comment 9 Ahmed Sayeed 2021-11-02 13:46:15 EDT
gdb: fix value_subscript when array upper bound is not known http://www-look-4.com/category/health/
    
    Since commit 7c6f27129631 ("gdb: make get_discrete_bounds check for https://komiya-dental.com/category/crypto/ 
    non-constant range bounds"), subscripting  flexible array member fails:
    http://www.iu-bloomington.com/category/health/
        struct no_size
        { https://waytowhatsnext.com/category/health/
          int n;
          int items[];
        };
     http://www.wearelondonmade.com/category/health/
        (gdb) p *ns
        $1 = {n = 3, items = 0x5555555592a4}
        (gdb) p ns->items[0] http://www.jopspeech.com/category/health/
        Cannot access memory at address 0xfffe555b733a0164
        (gdb) p *((int *) 0x5555555592a4)
        $2 = 101  <--- we would expect that http://joerg.li/category/health/
        (gdb) p &ns->items[0]
        $3 = (int *) 0xfffe5559ee829a24  <--- wrong address http://connstr.net/category/health/
    
    Since the flexible array member (items) has an unspecified size, the array type https://www.mktrade.fi/muottivalmistus
    created for it in the DWARF doesn't have dimensions (this is with gcc 9.3.0, http://embermanchester.uk/category/health/
    Ubuntu 20.04):
    http://www.slipstone.co.uk/category/health/
        0x000000a4:   DW_TAG_array_type
                        DW_AT_type [DW_FORM_ref4]       (0x00000038 "int") http://fishingnewsletters.co.uk/property/suluada/
                        DW_AT_sibling [DW_FORM_ref4]    (0x000000b3) http://www.logoarts.co.uk/category/health/
    
        0x000000ad:     DW_TAG_subrange_type
                          DW_AT_type [DW_FORM_ref4]     (0x00000031 "long unsigned int")
    http://www.acpirateradio.co.uk/category/health/
    This causes GDB to create a range type (TYPE_CODE_RANGE) with a defined http://www.go-mk-websites.co.uk/health/tipaza-province/
    constant low bound (dynamic _prop with kind PROP_CONST) and an undefined
    high bound (dynamic_prop with kind PROP_UNDEFINED). http://www.compilatori.com/category/health/
    
    value_subscript gets both bounds of that range using
    get_discrete_bounds.  Before commit 7c6f27129631, get_discrete_bounds http://www.mconstantine.co.uk/health/shanghai/
    didn't check the kind of the dynamic_props and would just blindly read
    them as if they were PROP_CONST. https://www.webb-dev.co.uk/category/health/  It would return 0 for the high bound,
    because we zero-initialize the range_bounds structure.  And it didn't
    really matter in this case, because the returned high bound wasn't used
    in the end.