PS - can be reproduced with ClamScan as well:

clamscan --detect-pua=yes --exclude-pua=packed ...

Hi Andy,

Our list of PUA categories on the FAQ is out of date. I did a little research and found that these are the PUA signature prefixes found in our current database:

PUA.Andr.Adware
PUA.Andr.Downloader
PUA.Andr.Dropper
PUA.Andr.Tool
PUA.Andr.Trojan
PUA.Andr.Virus
PUA.Cert.Revoked
PUA.Doc.Dropper
PUA.Doc.Packed
PUA.Doc.Tool
PUA.Doc.Trojan
PUA.Email.Phishing
PUA.Email.Trojan
PUA.Embedded.File
PUA.Html.Exploit
PUA.Html.Tool
PUA.Html.Trojan
PUA.Java.Exploit
PUA.Java.Packer
PUA.Js.Exploit
PUA.Osx.File
PUA.Osx.Trojan
PUA.Packed.Tool
PUA.Pdf.Exploit
PUA.Pdf.Trojan
PUA.Php.Trojan
PUA.Rtf.Exploit
PUA.Spy.Tool
PUA.Swf.Spyware
PUA.Tool.Tool
PUA.Unix.Adware
PUA.Unix.Coinminer
PUA.Unix.Downloader
PUA.Unix.File
PUA.Unix.Malware
PUA.Unix.Tool
PUA.Unix.Trojan
PUA.Unix.Virus
PUA.Win.Adware
PUA.Win.Coinminer
PUA.Win.Downloader
PUA.Win.Dropper
PUA.Win.Exploit
PUA.Win.File
PUA.Win.Ircbot
PUA.Win.Joke
PUA.Win.Keylogger
PUA.Win.Malware
PUA.Win.Packed
PUA.Win.Packer
PUA.Win.Proxy
PUA.Win.Ransomware
PUA.Win.Spyware
PUA.Win.Tool
PUA.Win.Trojan
PUA.Win.Virus

In addition to "Packed" try adding "Packer" or "Win.Packer" (I'm not sure which).

Please let me know if that resolves the issue.

Regards,
Micah

Hi Micah,

first - thank you for that inventory list. 

I conducted a series of test and after a first "hooray", the results are actually NOT confidence-inspiring.

clamscan --detect-pua=yes --exclude-pua=Packed c:\work\test
c:\work\test\CDPlay.exe: PUA.Win.Packer.BorlandDelphi-5 FOUND
c:\work\test\StartViewer.exe: PUA.Win.Packer.BorlandDelphi-5 FOUND

clamscan --detect-pua=yes --exclude-pua=Packer c:\work\test
c:\work\test\CDPlay.exe: PUA.Win.Packer.BorlandDelphi-5 FOUND
c:\work\test\StartViewer.exe: PUA.Win.Packer.BorlandDelphi-5 FOUND

So for a "Packer" category - excluding ONLY "Packer" or "Packed", without the "Win." will NOT fix.


However this works:

clamscan --detect-pua=yes --exclude-pua=Win.Packer c:\work\test
c:\work\test\CDPlay.exe: OK
c:\work\test\StartViewer.exe: OK

which seems intuitive, at first glance.


But ... then I tried:

clamscan --detect-pua=yes --exclude-pua=Win.Packed c:\work\test
c:\work\test\CDPlay.exe: OK
c:\work\test\StartViewer.exe: OK

So THAT is worrysome, because clearly "Win.Packed" is NOT a string match for "Win.Packer"?


At this point I worried, if it ACTUALLY compared the entire string - so I tried "Win" ONLY:

clamscan --detect-pua=yes --exclude-pua=Win c:\work\test
c:\work\test\CDPlay.exe: OK
c:\work\test\StartViewer.exe: OK

So this now leaves the question WHAT the category comparison ACTUALLY validates against? Does specifying "Win.Packed" possibly only compare the first token and it (quite unintended!) excludes ALL "Win" signatures?

This whole topic is a documentation black hole, leaving the user to make (possibly dangerous) guesses, causing either less or MORE signatures to be excluded than what the user intended to accomplish.

Hm - even "Win.Pack" will work, which would be logical IF it does a string compare in the length of the parameter string.

But... then "Win.Packed" should NOT work for a "Win.Packer" signature. 

So, again, the behavior is completely non-transparent to a user without any meaningful documentation on what the code is actually doing "behind the scenes":

clamscan --detect-pua=yes --exclude-pua=Win.Pack c:\work\test
C:\work\test\CDPlay.exe: OK
C:\work\test\StartViewer.exe: OK

(In reply to Andy Schmidt from comment #4)
> Hm - even "Win.Pack" will work, which would be logical IF it does a string
> compare in the length of the parameter string.
> 
> But... then "Win.Packed" should NOT work for a "Win.Packer" signature. 
> 
> So, again, the behavior is completely non-transparent to a user without any
> meaningful documentation on what the code is actually doing "behind the
> scenes":
> 
> clamscan --detect-pua=yes --exclude-pua=Win.Pack c:\work\test
> C:\work\test\CDPlay.exe: OK
> C:\work\test\StartViewer.exe: OK

Very strange!  That's definitely not what I would have expected either.  Category exclusion is definitely string compare logic. It seems it is only comparing the first part, "Win" in this case, and ignores everything after the ".". 

I suspect that the signature names used for regular and PUA signatures morphed over time and that the behavior of the category exclusions is no longer useful because the old categories aren't being properly used. I'll have a discussion with the malware team next week about it. Either way, I or someone on my team will need to fix the exclusion behavior so the exclusions work with the current signature name categories and we'll update the documentation.

Hi Andy,

I've put in a PR to the FAQ to provide meaningful documentation on the use of IncludePUA/ExcludePUA.  I hope this helps: https://github.com/Cisco-Talos/clamav-faq/issues/134

I do think there are improvements that could be made to support subcategories but isn't something I expect our team will work on.  It would be a good contribution opportunity for anyone who wants to improve the PUA config options.  

I haven't heard much from our malware research team about cleaning up the signatures with malware-names (like Trojan, Virus, etc).  I'll ask about it again when we meet next week. 

Anyways, if you feel the documentation update is sufficient I would like to close this ticket.

-Micah

(In reply to Micah Snyder from comment #6)
> I've put in a PR to the FAQ to provide meaningful documentation on the use
> of IncludePUA/ExcludePUA.  I hope this helps:
> https://github.com/Cisco-Talos/clamav-faq/issues/134

Hi Micah, thanks for making the effort and at least documenting the actual behavior, so that one will get predictable results.

I think it addresses my original trouble, which essentially was one of incorrect documentation. Feel free to close this ticket.

Best Regards,
Andy

Ok thanks Andy!  I'll check with the other team again when we meet tomorrow to see if we can get some of the improper category names in PUA signatures fixed.