Bug 11498 - [clamav-users] ScanOLE2 yes disables macro virus detection
[clamav-users] ScanOLE2 yes disables macro virus detection
Status: RESOLVED DUPLICATE of bug 11513
Product: ClamAV
Classification: ClamAV
Component: libclamav
stable
x86_64 GNU/Linux
: P3 normal
: ---
Assigned To: ClamAV team
:
Depends on:
Blocks:
  Show dependency treegraph
 
Reported: 2016-02-08 16:10 EST by Steven Morgan
Modified: 2016-04-24 23:00 EDT (History)
5 users (show)

See Also:
QA Contact:


Attachments
Malware: doc with macros that download exe by http and run it (91.00 KB, application/octet-stream)
2016-02-14 22:45 EST, David
bugzilla1: sensitive+
Details
signature file for the attached doc to exercise bug (191 bytes, text/plain)
2016-02-14 23:21 EST, David
no flags Details

Note You need to log in before you can comment on or make changes to this bug.
Description Steven Morgan 2016-02-08 16:10:30 EST
From David Shrimpton, Information Technology Services | The University of Queensland:

Hi,

I found some problems with the way clamav handles OLE2 containers.
This is causing many macro virus sigatures to not work and many viruses
to be missed:

If ScanOLE2 is set to yes, clamav only appears to scan the decompressed
macro files in OLE2 containers.
It does not scan any of the other files in the container.

If ScanOLE2 is set to no, clamav does not appear to extract any
files from the OLE2 container.  It just runs signatures against the
container file itself.

This can be tested  with clamav --debug --leave-temps and
--scan-ole2 yes or no  and writing some signatures against the uncompressed
macro files and the other files in the container.

This behaviour contradicts signatures.pdf:

2 = file inside OLE2 container (e.g. image, embedded executable, VBA
script)

which implies signatures for file type 2 will be run against files
extracted from OLE2 containers.   Which implies files ought be extracted
from OLE2 containers and scanned.
But this is not the case, whether ScanOLE2 is yes or no.

All file type 2 appears to denote for a signature is that that signature
should only be run against files recognized as an OLE2 container.  It does
not mean that the the signatures will be run against each file in
an OLE2 container.

Setting ScanOLE2 yes, disables the running of signatures of type 2 or any type
at all ie type 0,  against the OLE2 container itself and causes only signatures
of type 2 or 0 to be run against decompressed macros from the OLE2 container.

But most of the badmacro or other unofficial virus signatures written to
detect macro virus are written against the container itself which has the
compressed macro code in it.  They are not written against the uncompressed
macro code, so setting ScanOLE2 yes will disable these signatures.

A check of some macro viruses received in last 6 months shows none detected by
clamav official signatures.

The work going into macro virus detection appears
to be in the unofficial databases.

But the signatures in these databases
are simple byte matching signatures and not based upon detecting actual VBA
code and looking for the patterns of logic used by virus writers in their code.
The signatures will fail when matching uncompressed actual vba code when
ScanOLE2 is yes.   The signatures won't match at all when ScanOLE2 is yes.
They only match if ScanOLE2 is no.  But then signatures written against
vba code in the uncomressed macros are not matched.

We are seeing OLE2 virus files where the macros extract the malicious code
from other files in the  ole container at run time.

These viruses are completely missed when ScanOLE2 is yes , no matter what
signature you write, as the non macro
files in the OLE2 container are not scanned and the scanned files ie the
uncompressed macro vba code, don't contain the malicious code.

The non scanning of 'other' files in OLE containers if ScanOLE2 is yes or the
non scanning of decompressed macros if ScanOLE2 is no is causing macro
viruses to be not detected and this appears to be a bug or limitation
of clamav (or maybe a misconfiguration ?).

Ideally clamav would scan the OLE2 container itself, all the contained files
and also the uncompressed macros .

Also there is some confusion as to what file type 2 means.   Should this denote
run signatures against a file recognized as OLE2 or should it mean run
signatures against files within an OLE2 container.  eg a signature of type 2
would not be expected to be run against a file inside an OLE2 unless that
file is also an OLE2 container.  signatures.pdf is misleading in its description
of file type 2.

It seems the OLE2 handling needs to be split into 2 parts , 1 . scanning
of the container and all the contained files.   2.  Scanning of uncompressed
macros .  If this were done a  special file type to restrict
scanning to only uncompressed macros would be useful to prevent false positive.

If it can't be fixed then ome clearer explanation of the OLE2 scanning would be helpful as its misleading at present.

--
David Shrimpton
Information Technology Services | The University of Queensland
Comment 1 Steven Morgan 2016-02-08 16:27:55 EST
NEEDINFO: signatures and sample files.
Comment 2 Steven Morgan 2016-02-08 18:35:38 EST
clamav-users list discussion:

http://www.gossamer-threads.com/lists/clamav/users/65316
Comment 3 David 2016-02-14 22:45:00 EST
Created attachment 7059 [details]
Malware: doc with macros that download exe by http and run it

Malware: doc with obfuscated macros that download an uri

hxxp://iamnickrobinson dot com/u56gf2d/k76j5hg.exe 

by http and run it.

The macro code in macro: Module2 extracts part of the malicious code '.../WScript.Shell/...' from a non macro file Macros/UserForm1/o in the ole2 container.

'bug1': clamscan -z --scan-ole2=yes  with signatures for Module2  and the file 'o' will not scan file 'o' after hit on Module2.

clamscan -z --scan-ole2=no will hit on file 'o' sig but not Module2 as no
macro extraction.  (This is expected behaviour)

clamdscan -z (ScanOLE2 yes) will not scan file 'o' after a hit on a sig for Module2.
clamdscan -z (ScanOLE2 yes) will not scan file 'o' if no sig for Module2 but Heuristics.OLE2.ContainsMacros is hit due to OLE2BlockMacros yes

'bug2': HeuristicScanPrecedence 'no' broken as file 'o' not scanned

'bug3': (bug report exists for this already) clamscan missing OLE2BlockMacros option
Comment 4 David 2016-02-14 23:21:46 EST
Created attachment 7060 [details]
signature file  for the attached doc to exercise bug

MACRO_SIGNATURE.macro.ndb.1   detects code from the uncompressed Module2 macro
it is not matched on the compressed macro and so not on the .doc itself
It is only matched if scan-ole2=yes.

USERFORM_SIGNATURE.macro.ndb.1 matches bytes in the file 'Macros/UserForm1/o'
from the OLE2 container.  It also matches the container itself as these bytes
are the same (there is no uncompression as with the macro)  

USERFORM_SIGNATURE.macro.ndb.1 is only matched if scan-ole2=no and therefore MACRO_SIGNATURE.macro.ndb.1 is not matched.  If scan-ole2=yes and -z , MACRO_SIGNATURE.macro.ndb.1 is hit and file 'Macros/UserForm1/o' is not scanned
and the container itself is not scanned, so USERFORM_SIGNATURE.macro.ndb.1 is not hit.
This is a 'bug/limitation'.

Note the file type of the signature must be considered.

MACRO_SIGNATURE.macro.ndb.1 is type 2 ie OLE2.   This is confusing as it it matching a file which is an uncompressed macro within an ole2 container.  It does not match a file that is an ole2 container itself.  Note type 2 does not denote extract files and macros in ole2 . The scan-ole2=yes option denotes this.

The file 'Macros/UserForm1/o' from within the ole2 container is a binary file and not of type 2.   Therefore if USERFORM_SIGNATURE.macro.ndb.1 is type 2 , it is run against the container itself but not against the file 'Macros/UserForm1/o' if this were being scanned.

If USERFORM_SIGNATURE.macro.ndb.1 is type 0 ie any file, it is matched against the ole2 container itself and against file 'Macros/UserForm1/o' and should hit twice with -z , if signature MACRO_SIGNATURE.macro.ndb.1 were removed.
Comment 5 Kevin Lin 2016-04-20 15:45:22 EDT
This is documentation issue that has been fixed as part of a similar bug. Target type 2 is meant for OLE2 containers themselves and associated macros; target type 2 DOES NOT include OLE2 file contents. To specifically target OLE2 contents, use "Container:CL_TYPE_OLE2" inside a .ldb target description block.

For extra information, please refer to bb#11513.

*** This bug has been marked as a duplicate of bug 11513 ***
Comment 6 David 2016-04-24 23:00:26 EDT
CL_TYPE_OLE2  produces error

CL_TYPE_MSOLE2  appears to be correct.

Container:CL_TYPE_MSOLE2  in ldb  does not work.

Only the macro files are scanned even with Target:0 .
No other files in the container are scanned.

So the original bug remains.  It is not just a documentation problem.