any feedback on this? This change will not break openssl 1.0.2h but is required for the upcomming 1.1.0 release currently (re)scheduled for 25th August.

Yes thanks once again for providing a patch. Targeting for 0.99.3.

Tested and added for inclusion in 0.99.3

https://github.com/vrtadmin/clamav-devel/commit/fa15aa98c7d5e1d8fc22e818ebd089f2e53ebe1d

*** Bug 11635 has been marked as a duplicate of this bug. ***

*** Bug 11646 has been marked as a duplicate of this bug. ***

Mickey, please review/comment on the issue below.

From [clamav-users] list:

I'm building clam 0.99.3/head,

    git branch -a | grep \*
        * 0.99.3

    git log | head
        commit 6f8290632b6e1ddcf08b3a64c6cbc9d8b98571e3
        Author: Steven Morgan <stevmorg@cisco.com>
        Date:   Wed Nov 29 17:38:57 2017 -0500

            ClamAV 0.99.3 beta2 versioning.

        commit 0a320049f1fe058dbed05606c925bb2ec2584264
        Author: Steven Morgan <stevmorg@cisco.com>
        Date:   Wed Nov 29 17:18:42 2017 -0500

The build FAILs -- as it has for over a year -- when linking against
OpenSSL 1.1.0x libs/api, due to reference of deprecated symbols,

    ...
    -L/usr/local/lib64  -Wl,-rpath,/usr/local/lib64  -o clamscan output.o
getopt.o optparser.o actions.o misc.o clamscan.o manager.o  ../libclamav/
libclamav.la -lpthread
    libtool: link: rm -f .libs/clamscan.nm .libs/clamscan.nmS
.libs/clamscan.nmT
    libtool: link: rm -f ".libs/clamscan.nmI"
    libtool: link: (cd .libs && /usr/bin/gcc-7 -O3 -Wall -fstack-protector
-funwind-tables -fasynchronous-unwind-tables -fmessage-length=0
-grecord-gcc-switches -march=native -mtune=native
-I/usr/local/openssl11/include -I/usr/local/include -I/usr/local/include
-D_LARGEFILE_SOURCE -D_LARGEFILE64_SOURCE -D_FILE_OFFSET_BITS=64 -c
-fno-builtin "clamscanS.c")
    libtool: link: rm -f ".libs/clamscanS.c" ".libs/clamscan.nm"
".libs/clamscan.nmS" ".libs/clamscan.nmT" ".libs/clamscan.nmI"
    libtool: link: /usr/bin/gcc-7 -O3 -Wall -fstack-protector
-funwind-tables -fasynchronous-unwind-tables -fmessage-length=0
-grecord-gcc-switches -march=native -mtune=native
-I/usr/local/openssl11/include -I/usr/local/include -I/usr/local/include
-D_LARGEFILE_SOURCE -D_LARGEFILE64_SOURCE -D_FILE_OFFSET_BITS=64 -Wl,-rpath
-Wl,/usr/local/openssl11/lib64 -Wl,-rpath -Wl,/usr/local/lib64 -Wl,-rpath
-Wl,/usr/local/lib64 -o .libs/clamscan output.o getopt.o optparser.o
actions.o misc.o clamscan.o manager.o  -L/usr/local/openssl11/lib64
-L/usr/local/lib64 ../libclamav/.libs/libclamav.so
-L/usr/local/openssl11/lib -L/lib64 -L/usr/local/lib /usr/lib64/libxml2.so
-llzma -lbz2 /usr/lib64/libltdl.so -ldl /usr/local/lib64/libpcre2-8.so -lm
/usr/local/lib64/libpcrecpp.so /usr/local/lib64/libpcre.so
/usr/local/lib64/libcurl.so /usr/local/lib64/libnghttp2.so -lpsl -lz -lssl
-lcrypto -lssh2 -lpthread -pthread
    ../libclamav/.libs/libclamav.so: undefined reference to
`X509_CRL_get_nextUpdate'
    ../libclamav/.libs/libclamav.so: undefined reference to
`SSL_library_init'
    ../libclamav/.libs/libclamav.so: undefined reference to
`ERR_load_crypto_strings'
    ../libclamav/.libs/libclamav.so: undefined reference to
`OpenSSL_add_all_algorithms'
    ../libclamav/.libs/libclamav.so: undefined reference to `EVP_cleanup'
    ../libclamav/.libs/libclamav.so: undefined reference to
`OpenSSL_add_all_digests'
    ../libclamav/.libs/libclamav.so: undefined reference to
`SSL_load_error_strings'
    ../libclamav/.libs/libclamav.so: undefined reference to
`OpenSSL_add_all_ciphers'
    collect2: error: ld returned 1 exit status
    Makefile:611: recipe for target 'clamscan' failed
    make[2]: *** [clamscan] Error 1
    make[2]: Leaving directory '/usr/local/src/clamav-devel/clamscan'
    Makefile:767: recipe for target 'all-recursive' failed
    make[1]: *** [all-recursive] Error 1
    make[1]: Leaving directory '/usr/local/src/clamav-devel'
    Makefile:596: recipe for target 'all' failed
    make: *** [all] Error 2

Mod'ing the build by applying changes similar to a 3rd-party patch (
https://github.com/patch-exchange/openssl-1.1-transition/tree/master/clamav)
for v0.99.2x, also available for over a year now,


https://github.com/patch-exchange/openssl-1.1-transition/blob/master/clamav/clamav-0.99.2-openssl-1.1.patch

clam* build/linked with OpenSSL 1.1.0,

    ldd `which clamdscan` `which clamd` | egrep "ssl|crypto"
        libssl.so.1.1 => /usr/local/openssl11/lib64/libssl.so.1.1
(0x00007fbda5a85000)
        libcrypto.so.1.1 => /usr/local/openssl11/lib64/libcrypto.so.1.1
(0x00007fbda55dc000)
        libssl.so.1.1 => /usr/local/openssl11/lib64/libssl.so.1.1
(0x00007f08b5a00000)
        libcrypto.so.1.1 => /usr/local/openssl11/lib64/libcrypto.so.1.1
(0x00007f08b5557000)

and exec OK

    systemctl status clamd.service
        ● clamd.service - clamd scanner daemon
           Loaded: loaded (/etc/systemd/system/clamd.service; enabled;
vendor preset: disabled)
           Active: active (running) since Thu 2017-11-30 15:46:05 PST; 1h
20min ago
         Main PID: 14070 (clamd)
            Tasks: 2 (limit: 512)
           CGroup: /system.slice/clamd.service
                   └─14070 /usr/local/sbin/clamd -c
/usr/local/etc/clamav/clamd.conf

        Nov 30 17:07:21 dev.loc clamd[14070]: SelfCheck: Database status OK.
        Nov 30 17:07:22 dev.loc clamd[30292]: Portable Executable support
enabled.
        Nov 30 17:07:22 dev.loc clamd[30292]: ELF support enabled.
        Nov 30 17:07:22 dev.loc clamd[30292]: Mail files support enabled.
        Nov 30 17:07:22 dev.loc clamd[30292]: OLE2 support enabled.
        Nov 30 17:07:22 dev.loc clamd[30292]: PDF support enabled.
        Nov 30 17:07:22 dev.loc clamd[30292]: SWF support enabled.
        Nov 30 17:07:22 dev.loc clamd[30292]: HTML support enabled.
        Nov 30 17:07:22 dev.loc clamd[30292]: XMLDOCS support enabled.
        Nov 30 17:07:22 dev.loc clamd[30292]: HWP3 support enabled.
        Nov 30 17:07:22 dev.loc clamd[30292]: Self checking every 1800
seconds.

    clamscan -d /var/lib/clamav
        ...
        ----------- SCAN SUMMARY -----------
        Known viruses: 9380408
        Engine version: 0.99.3-beta2
        Scanned directories: 1
        Scanned files: 33
        Infected files: 0
        Data scanned: 4.41 MB
        Data read: 1.93 MB (ratio 2.29:1)
        Time: 27.834 sec (0 m 27 s)


openssl references in git log suggest openssl 110 readiness, as well as the
option to link to local instances of it,

    ...
    commit a4013285691478f165f1fe2de070ff32f34093fc
    Author: Micah Snyder <micasnyd@cisco.com>
    Date:   Fri Nov 17 09:00:06 2017 -0500

        Regargeting openssl solution to match the other projects.
    ...
    commit 89c6504289cd54e2db60e9e04e5752c553d4449c
    Author: Steven Morgan <smorgan@sourcefire.com>
    Date:   Fri Jul 14 16:50:12 2017 -0400

        fix for linking to openssl fo x64.
    ...
    commit 950be7e5eb93cdafc1349d85813c125a53886ee5
    Author: Steven Morgan <smorgan@sourcefire.com>
    Date:   Wed Dec 21 17:16:39 2016 -0500

        Change Windows build to use OpenSSL 1.1.0.c
    ...
    commit dd1b59482dab05f732b8116218eea9d187c41031
    Author: Mickey Sola <msola@sourcefire.com>
    Date:   Tue Aug 9 15:48:31 2016 -0400

        bb11594 - allow for compilation against openssl 1.1.0
    ...
    commit 3f40439f56ba179107afea9e349441fa57cbeb84
    Author: Kevin Lin <klin@sourcefire.com>
    Date:   Thu Oct 22 14:50:41 2015 -0400

        fix for openssl build with specific openssl location (needs autogen)
    ...


But attempting to view that bug#11594 for more detail, we're refused:


    @ https://bugzilla.clamav.net/show_bug.cgi?id=11594

        "You are not authorized to access bug #11594"


What's needed to get full OpenSSL 1.1.0 compat into master branch?

> ... comment here or on the ticket with your OS details (gcc and autotools versions would also be helpful) ...

fyi:

lsb_release -rd
	Description:    openSUSE Leap 42.3
	Release:        42.3

uname -rm
	4.14.2-4.gb5596a5-default x86_64

gcc --version
	gcc (SUSE Linux) 7.2.1 20171020 [gcc-7-branch revision 253932]

autoconf --version
	autoconf (GNU Autoconf) 2.69

libtoolize --version
	libtoolize (GNU libtool) 2.4.6

automake --version
	automake (GNU automake) 1.15.1

openssl version
	OpenSSL 1.1.0g  2 Nov 2017

openssl configure:

	./config \
	 --api=1.1.0 \
	 --prefix=/usr/local/openssl11 \
	 --openssldir=/usr/local/openssl11 \
	 --libdir=lib64 \
	 -D_GNU_SOURCE \
	 -DOPENSSL_NO_BUF_FREELISTS \
	 -DOPENSSL_NO_HEARTBEAT \
	 -DPURIFY \
	 -DSSL_FORBID_ENULL \
	 -DTERMIO \
	 -Wa,--noexecstack \
	 -Wl,-z,relro,-z,now \
	 -Wall \
	 -Wl,-rpath=/usr/local/openssl11/lib64 \
	 -fno-common \
	 threads shared \
	 no-comp no-zlib no-zlib-dynamic \
	 enable-ec_nistp_64_gcc_128 \
	 enable-rfc3779 \
	 enable-ecdsa \
	 no-weak-ssl-ciphers

pkg-config --libs openssl
	-L/usr/local/openssl11/lib64 -lssl -lcrypto

pkg-config --cflags openssl
	-I/usr/local/openssl11/include

clamav configure:

	export LDFLAGS+=" -L/usr/local/openssl11/lib64 -Wl,-rpath,/usr/local/openssl11/lib64 -L/usr/local/lib64 -Wl,-rpath,/usr/local/lib64 -L/usr/local/lib64 -Wl,-rpath,/usr/local/lib64"

	export CFLAGS+=" -I/usr/local/openssl11/include -I/usr/local/include -I/usr/local/include"

	export LIBS+=" -lssl -lcrypto -lpcre -lpcrecpp -lcurl"

	export CPPFLAGS+=" -I/usr/local/openssl11/include -I/usr/local/include -I/usr/local/include"

	./configure \
	--disable-debug \
	--prefix=/usr/local \
	--libdir=/usr/local/lib64 \
	--sysconfdir=/usr/local/etc/clamav \
	 --with-dbdir=/var/lib/clamav \
	 --with-user=clamav --with-group=clamav \
	--enable-rpath \
	--enable-shared --disable-static \
	--enable-no-cache \
	--enable-clamav \
	--enable-clamdtop \
	--enable-libfreshclam \
	--enable-milter \
	--enable-yara \
	--with-openssl=/usr/local/openssl11 \
	--with-pcre=/usr/local \
	--with-libcurl=/usr/local \
	--disable-llvm \
	--with-gnu-ld \
	--disable-zlib-vcheck

ldd `which clamd` | egrep -i "curl|ssl|crypto"
	libcurl.so.4 => /usr/local/lib64/libcurl.so.4 (0x00007f3d0c364000)
	libssl.so.1.1 => /usr/local/openssl11/lib64/libssl.so.1.1 (0x00007f3d0baa0000)
	libcrypto.so.1.1 => /usr/local/openssl11/lib64/libcrypto.so.1.1 (0x00007f3d0b5f6000)

ldd /usr/local/lib64/libcurl.so.4 | egrep -i "ssl|crypto"
	libssl.so.1.1 => /usr/local/openssl11/lib64/libssl.so.1.1 (0x00007fe41fe68000)
	libcrypto.so.1.1 => /usr/local/openssl11/lib64/libcrypto.so.1.1 (0x00007fe41f9be000)

also

NOTE:
@ https://www.openssl.org/docs/man1.1.0/crypto/ERR_load_crypto_strings.html
  "All of the following functions are deprecated from OpenSSL 1.1.0 ..."

fyi, my patch (note, *not* general case)

--------------
diff -ur clamav-devel.ORIG/libclamav/crypto.c clamav-devel/libclamav/crypto.c
--- clamav-devel.ORIG/libclamav/crypto.c	2017-03-23 10:20:29.590645016 -0700
+++ clamav-devel/libclamav/crypto.c	2017-03-23 10:18:15.455247547 -0700
@@ -36,6 +36,7 @@
 #include "clamav-config.h"
 #endif

+#include <openssl/ssl.h>
 #include <stdio.h>
 #include <stdlib.h>
 #include <string.h>
@@ -120,19 +121,13 @@

 int cl_initialize_crypto(void)
 {
-    SSL_load_error_strings();
-    SSL_library_init();
-    OpenSSL_add_all_digests();
-    OpenSSL_add_all_algorithms();
-    OpenSSL_add_all_ciphers();
-    ERR_load_crypto_strings();
-
+    OPENSSL_init_ssl(0, NULL);
     return 0;
 }

 void cl_cleanup_crypto(void)
 {
-    EVP_cleanup();
+    return 0;
 }

 unsigned char *cl_hash_data(char *alg, const void *buf, size_t len, unsigned char *obuf, unsigned int *olen)
@@ -1111,7 +1106,7 @@
     if ((x)) {
 	ASN1_TIME *tme;

-	tme = X509_CRL_get_nextUpdate(x);
+	tme = X509_CRL_get0_nextUpdate(x);
 	if (!tme || X509_cmp_current_time(tme) < 0) {
 		X509_CRL_free(x);
 		return NULL;
diff -ur clamav-devel.ORIG/m4/reorganization/libs/openssl.m4 clamav-devel/m4/reorganization/libs/openssl.m4
--- clamav-devel.ORIG/m4/reorganization/libs/openssl.m4	2017-03-23 10:20:29.606644944 -0700
+++ clamav-devel/m4/reorganization/libs/openssl.m4	2017-03-23 09:54:07.397505020 -0700
@@ -30,9 +30,9 @@
 LIBS="$LIBS $SSL_LIBS"

 if test "$LIBSSL_HOME" != "/usr"; then
-    SSL_LDFLAGS="-L$LIBSSL_HOME/lib"
-    SSL_CPPFLAGS="-I$LIBSSL_HOME/include"
-    LDFLAGS="-L$LIBSSL_HOME/lib"
+    SSL_LDFLAGS="-L/usr/local/openssl11/lib64 -Wl,-rpath,/usr/local/openssl11/lib64"
+    SSL_CPPFLAGS="-I/usr/local/openssl11/include"
+    LDFLAGS="-L/usr/local/openssl11/lib64 -Wl,-rpath,/usr/local/openssl11/lib64"
     CFLAGS="$SSL_CPPFLAGS"
 else
     SSL_LDFLAGS=""
@@ -44,7 +44,7 @@

 AC_LINK_IFELSE(
 	       [AC_LANG_PROGRAM([[#include <openssl/ssl.h>]],
-				[[SSL_library_init();]])],
+				[[OPENSSL_init_ssl(0, NULL);]])],
 	       [have_ssl="yes";],
 	       [AC_MSG_ERROR([Your OpenSSL installation is misconfigured or missing])])
--------------

Any additional information needed from this end to help move this forward?

(In reply to PGNd from comment #9)
> Any additional information needed from this end to help move this forward?

At the moment we're focused on testing general features across a range of operating systems, versions, and architectures and that's been really time consuming.  We'll take a look at updating to the latest APIs as soon as we have the time. 

In the meantime, if you want to have a working build, recompile OpenSSL using standard configuration options that allow deprecated APIs.  Then ClamAV will compile/link fine with OpenSSL 1.1.0.

I _have_ a working build; that's the point of the patch(es) in this bug, and getting to a reliable release that can be distributed.

Re: priorities, noted.

As for actionable information: if you can show the patch will not break 1.0.2h inclusion will be must faster.

(In reply to PGNd from comment #9)
> Any additional information needed from this end to help move this forward?

This patch is only needed if you do disable the compat functions. Was this your doing or is this OpenSuSE?

(In reply to PGNd from comment #8)
> fyi, my patch (note, *not* general case)
> 
> --------------
> diff -ur clamav-devel.ORIG/libclamav/crypto.c clamav-devel/libclamav/crypto.c
> --- clamav-devel.ORIG/libclamav/crypto.c	2017-03-23 10:20:29.590645016 -0700
> +++ clamav-devel/libclamav/crypto.c	2017-03-23 10:18:15.455247547 -0700
> @@ -36,6 +36,7 @@
>  #include "clamav-config.h"
>  #endif
> 
> +#include <openssl/ssl.h>
why?

>  #include <stdio.h>
>  #include <stdlib.h>
>  #include <string.h>
> @@ -120,19 +121,13 @@
> 
>  int cl_initialize_crypto(void)
>  {
> -    SSL_load_error_strings();
> -    SSL_library_init();
> -    OpenSSL_add_all_digests();
> -    OpenSSL_add_all_algorithms();
> -    OpenSSL_add_all_ciphers();
> -    ERR_load_crypto_strings();
> -
> +    OPENSSL_init_ssl(0, NULL);

as noted in comment #12 you need provide a fallback which works with 1.0.2

>      return 0;
>  }
> 
> diff -ur clamav-devel.ORIG/m4/reorganization/libs/openssl.m4
> clamav-devel/m4/reorganization/libs/openssl.m4
> --- clamav-devel.ORIG/m4/reorganization/libs/openssl.m4	2017-03-23
> 10:20:29.606644944 -0700
> +++ clamav-devel/m4/reorganization/libs/openssl.m4	2017-03-23
> 09:54:07.397505020 -0700
> @@ -30,9 +30,9 @@
>  LIBS="$LIBS $SSL_LIBS"
> 
>  if test "$LIBSSL_HOME" != "/usr"; then
> -    SSL_LDFLAGS="-L$LIBSSL_HOME/lib"
> -    SSL_CPPFLAGS="-I$LIBSSL_HOME/include"
> -    LDFLAGS="-L$LIBSSL_HOME/lib"
> +    SSL_LDFLAGS="-L/usr/local/openssl11/lib64
> -Wl,-rpath,/usr/local/openssl11/lib64"
> +    SSL_CPPFLAGS="-I/usr/local/openssl11/include"
> +    LDFLAGS="-L/usr/local/openssl11/lib64
> -Wl,-rpath,/usr/local/openssl11/lib64"

If you have installed openssl in a non-standard path you need to provide that to configure and not hardcoding it here.


Sebastian

You can see, in detail, exactly what's been done in C7:

  https://bugzilla.clamav.net/show_bug.cgi?id=11594#c7

The referenced symbols are DEPRECATED in OpenSSL 1.1.0

> If you have installed openssl in a non-standard path you need to provide that to configure and not hardcoding it here.

Right.  As I've pointed out, this is 'my' patch, and is NOT a general case.

I've done it this way because it simply successfully, temporarily works around the invalid assumptions in the current code.

We will revisit the discussion of improving OpenSSL detection and API usage during the 0.99.4 development.

*** Bug 11810 has been marked as a duplicate of this bug. ***