Perfection Series – Forgotten data in your logs (Log Redaction service)

From business standpoint, leaking sensitive information into your logs is not only bad, but could lead to compliance, liability and risk disaster sooner than you think. While there are solutions, including DLP, out there to inspect the data traffic to help capture sensitive data leakage, not many solutions out there are proactive and intrusive enough inspect the backplane of your systems for sensitive data leakage or regulatory appliance analysis. This becomes more pronounced when you have multiple ways you allows users (especially the admin users) to access your system – such as browser, command line, XML interface, etc. You need to not only worry about the logs for each of those interfaces, but also you need to worry about the types are logs that are kept and where they may go in the future; i.e. – record of such as trace log, transactional log, exception log, command log, admin log. Etc.

Recently Intel / McAfee Service Gateway (ESG/MSG) have seen a lot of activity and interest in providing API/ services in/from the cloud. One of the major issues they all faced was the fact that the log, which is stored in the cloud, might contain information that is sensitive or a compliance issue, especially when you offer this as a service (*aaS) and exposed 24×7 to the hackers in the cloud. While this detailed logging may not be an issue for the enterprise customers, where the actual log information stored in a centralized secure storage, most times this becomes an issue when you offer a multi-tenant environment and you share resource with other users. In order to provide more control to the customers in the cloud we introduced a new log redaction future, which can be used either in the enterprise version or in the cloud version. This helps customers with sensitive information such as PAN data (especially when there are some credit card information is sent over for processing), personal data PII, names, addresses, SS#, DOB and other pertinent information including passwords in verbose modes. Intel solution allows that data to be removed/masked with ease and it is user definable, both for patterns and for masking specifics.

I was talking to a customer of mine few days ago on this very topic and I told them how cool this is. He was like, “I think our system is very secure and we have taken “extra measures” given that they deal with multiple compliance standards and issues.” So I suggested to perform a log spider scan. He called me 2 days later panicked with what he found. If you don’t know whether you should worry about this or not there are spider scan tools available on the net, just Google it and see what your logs tell you. If you don’t like what you see don’t blame me :).

While most customers take extra care of their transactional messages, I have seen a lot of customers a bit lax in regards to logging and administrative interfaces. I had recently blogged about an incident with a customer with an exposed data in the logs which you can read here.

Our solution allows them to tighten their logs up multiple notches. We come with about 30 or so pre-defined filters, with an option for customers to build more on their own, using a simple visual tool. It can be applied to any level of logs including the most verbose levels. The masks are user defined and are flexible. Once you turn them on, define them, there is no need to restart your system, it is always on after that until you explicitly turn it off. What is more cool is that the logs would be instantly cleaned once you push the config and the push is cluster wide into all node points. Imagine the power of controlling the sensitive log data in all edge devices (whether Enterprise edge or extended to the cloud) in one push of a button.

In reality, the logging system normally logs the content given from many different underlying components, such as from Input Server, Invocation Agent, runtime Workflow, mediation Engine, Security engine, etc. This makes it complicated to manage when you deal with so many components as when you log things from input side, at some verbose log levels (such as detailed trace), it can log the wire data which can be anything (imagine that most solutions out there log everything comes on the wire for auditing purposes). So it is very hard to prevent the sensitive data logged in the log without special handling such as these contrary to what you think.

Imagine if you are dealing with PCI, HIPAA, etc and have an edge device to define saying I need my logs to be clean from these sensitive data and define masking / encryption on transactional data as well. You can be sure that from your edge inside, or going out, your message and logs are all cleanup to your satisfaction and for compliance.

If you need more information on this or on our solutions in general please check out or reach out to me.

About Andy Thurai
My website is

2 Responses to Perfection Series – Forgotten data in your logs (Log Redaction service)

  1. Pingback: Your personal data is now yours….. maybe? « SOA, Cloud, Identity & Security Blog

  2. Pingback: Your personal data is now yours… maybe? by Andy Thurai « Intel XML Gateway – SOA Expressway Blog

Leave a Reply

Please log in using one of these methods to post your comment: Logo

You are commenting using your account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: