Exchange 2013 is around for quite a while now and there are some deployments happening across the globe. I was asked few questions by some architects about Exchange 2013 Anti-Malware scanning a couple of days back and some of them were related to operations and troubleshooting. Well, I did not question them back on why they needed to know troubleshooting but that gave me enough food for thought.
With my labs on I got some details gathered on how to troubleshoot anti-malware scanning if it goes mad. So, first thing first, What is anti-malware scanning in Exchange 2013?
Anti-malware scanning is a built-in feature in Microsoft Exchange 2013 and kind of replaces the Forefront Protection for Exchange. You have an option to install it or go without it. Although it consists of a single scanning engine, it does not do a file level scanning of servers. It is primarily designed to look at the transport hygiene factors. So, far I found it a pretty good solution for those who do not have a third-party message hygiene and scanning appliances or software. Although not an extremely impressive stuff, it knows what to do. Anti-malware scanning runs a transport agent.
Okay, I and you both now understand what this thing is. What’s next? How to troubleshoot this guy if he loses his mind and starts acting weird.
Anti malware protection registers an agent and can be easily seen using Get-TransportAgent
You can see Malware Agent list and enabled. For anti malware scanning to work this agent should be enabled.
Event logs have always been a best place to start troubleshooting. Anti malware scanning uses Application Event logs to record some of the limited information.
If you encounter a problem with your inbound or outbound messages and Application Log is unable to show you enough of information, you can use diagnostic logging on the anti malware scanning by using few powershell commands:
Set-EventLogLevel “MSExchange Antimalware\General” -Level High
Set-EventLogLevel “MSExchange Antimalware\Init” -Level High
Set-EventLogLevel “MSExchange Antimalware\ScanResults” -Level High
Set-EventLogLevel “MSExchange Antimalware\ScanError” -Level High
Level parameter accepts four levels Low, Medium, High, Expert. Based on your selection of level event logs be generated. Higher the value you specify more the number of details you get but more the number of events are flooded in.
Once you have set the logging levels High you should be able to see some events in application log. To test whether your antimalware agent works, you can use the application log to find out what is going on. If your anti malware scanning is working you should be able to see some events as below
Log Name: Application
Source: MSExchange Antimalware
Date: 7/20/2013 12:35:12 PM
Event ID: 3810
Task Category: ScanResults
The anti-malware agent has detected malware. MessageId: email@example.com Message sent: 7/20/2013 7:05:08 AM From: firstname.lastname@example.org Size: 926 KB Engine: Microsoft : Antimalware Engine (1.137.1393.0) Malware name: DOS/EICAR_Test_File Action taken: DeleteMessage
Like any other AV application malware agent also requires updates be fetched from upstream server. If your servers do not have access to Microsoft upstream servers, you may see some errors in applications logs which indicate a problem in update process.
Log Name: Application
Date: 7/20/2013 2:34:17 PM
Event ID: 6027
Task Category: None
User: NETWORK SERVICE
MS Filtering Engine Update process was unsuccessful in contacting the Primary Update Path. Update Path: http://forefrontdl.microsoft.com/server/scanengineupdate
Servers where logs are generated
Exchange 2013 does not have HT and UM roles anymore. Organization wide mail flow is handled by mailbox servers, inbound and outbound mail flow is now handled by CAS servers using Front-End Transport Service.
If you are troubleshooting a anti malware related issues for internal mail flow you should be able to see these logs on mailbox servers.
If you are facing anti malware related issues for inbound/outbound (external) mail flow, mailbox servers and CAS servers both should be looked at based on the scenario.
E.g. If an outbound email is being deleted or has any other issues and you identified anti malware protection may be a cause behind it, you should be looking at application logs on mailbox servers.
If an inbound email is being deleted and you want to see what has happened to that email, you should be looking at CAS server application logs.
Malware Agent is written to work on OnSubmittedMessage transport event and it intercepts the messages even before they are routed. That mean, if a user mailbox is on server named EGIEX05; and an email sent by him contains a virus and it is deleted the logs will be generated on EGIEX05 and not on the destination server where recipient’s mailbox is.
Is there a way to manage Anti Malware protection?
Yes, there is a way to manage it but the graphical interface is very limited. Anti Malware Protection has a built in anti virus scanning engine that cannot be managed via graphical interface at all. The only way to manage the AV engine is powershell. Before you attempt to manage see the settings or change the settings you need to add the management snapin in powershell. I will write another follow up post on managing anti malware protection which shall cover the management options.
Honestly, I did not know anything of such kind really exists but I ran Get-PSSnapin on one of the Exchange 2013 servers and found it. Out of curiosity; I added it and I got only four commands by adding this powershell snap in.
Other than above commands there are few more those can be used for management. I will talk about them in a follow up article.
Enable Disable Anti Malware Protection
In some cases you may need to disable anti malware protection agent on an individual or all servers. Prebuilt scripts can help you disabling or enabling the anti malware agents on a one or more servers. The script named Enable-AntimalwareScanning.ps1 and Disable-AntimalwareScanning.ps1 are located at C:\Program Files\Microsoft\Exchange Server\V15\Scripts or the path where exchange binaries are installed.
Find out whether it was Anti Malware Agent to kill your message
So, how do I find out if a message is being really killed by anti malware agent? Fairly simple, message tracking logs are our friend here. If you receive a complaint that a message was not received by its intended recipient, you can simply use message tracking log to find out whether it was deleted by anti malware agent.
Get-MessageTrackingLog -Sender -Recipient -MessageSubject “Message Subject”
If an email was intercepted by anti malware agent and deleted, EventID should tell you that the message that you are looking for has FAILED. You can then use |FL to find more details about it and confirm the possibility.
Well, that is all for today, I will put across a couple of follow-up articles on managing anti malware protection on Exchange 2013 to discuss more about how to manage and use it.
EDIT: Corrected the text in Is there a way to manage Anti Malware protection? section.