John Melton's Weblog
Java, Security and Technology
Java, Security and Technology
What is it and why should I care?
Application layer intrusion detection is a simple concept that I believe is very, very powerful when it comes to protecting applications. Most of the topics I’ve covered thus far have focused on the development portion of the software life-cycle, but this topic really covers the entire span of an application, from the requirements and planning to sun-setting.
The basic concept is that you plan for, implement and monitor “bad” things that occur in your application. With this type of system in place, you look for events that appear to be undesirable in some way and then keep track of them. Over time, you can make decisions about whether those individual events turn into an actual attack.
Many developers actually do most of the work of detection already. Consider the following pseudo-code:
if (user has access to record) {
get data
redirect to view/edit page
} else {
log exception
send user error message
}
I’ve seen code just like this lots of times. The problem here is the handling exceptional condition. In general, people don’t review logs, so if there’s an attacker trying to break your application, the only person seeing the error you’ve caught is the _attacker_. With one quick addition of sending a message to your intrusion detection engine, you can start tracking these events and actually gaining knowledge into the real-time (and historical if you choose to store it) usage of your application. After you’ve detected an actual intrusion, you also have the ability to respond to the activity in any [legal] way you see fit. Popular options include ideas like: increased logging, manipulating user account (logout, disable), or even blocking access to certain functionality.
What should I do about it?
Let’s assume I’ve sold you on the idea of implementing something like this (hopefully I have). What now?
Well, you have a few options on how to proceed that I’m aware of: ESAPI, AppSensor or roll-your-own.
ESAPI does have an intrusion detection engine built-in that performs some of these ideas. It is admittedly not extensive, but the core is there and can certainly be extended.
AppSensor is one such extension of the ESAPI intrusion detection engine. The implementation is more extensive than what’s available for ESAPI. Additionally, the project offers a book about the overall idea, as well as significant documentation in addition to the code. Lastly, there is actually a significant update being worked on currently on the project to update both the documentation and the code.
Rolling your own analysis engine can be a small or very large project depending on your needs. Nevertheless, you can certainly take the ideas and implement them in your applications and get significant benefit.
By just adding a little bit of effort, you can gain significant insight into the overall security health of your application(s). You can see who attacked/is attacking your application in real-time or the past, and you can actually respond to events as they occur. Who wouldn’t like that?
Author note: I work on the AppSensor project, so this whole topic is near and dear to me. Please take advantage of the idea whether it’s in our implementation or not!
References
———–
https://www.owasp.org/index.php/ApplicationLayerIntrustionDetection
http://www.jtmelton.com/2010/11/10/application-intrusion-detection-with-owasp-appsensor/
http://www.owasp.org/index.php/OWASP_AppSensor_Project
http://www.owasp.org/
http://www.youtube.com/watch?v=6gxg_t2ybcE
http://www.clerkendweller.com/2010/11/12/Application-Intrusion-Detection-and-Response-Planning-Methodology
https://www.owasp.org/index.php/Category:OWASP_Enterprise_Security_API
This article is a basic introduction to AppSensor, an OWASP project that’s been gaining a lot of traction recently. It’s a fairly simple concept, and one that I think (and hope) will be implemented in lots of applications in the near future. If you’d rather watch a video about AppSensor, here is a good one from Michael Coates, the project lead. Alternatively, here is a very quick video of a demo of AppSensor. So, let’s get started …
AppSensor is an implementation of an idea called application layer intrusion detection. The concept is very simple. While we have controls like intrusion detection at the network layer and web application firewalls at the web layer to detect attacks, these tools work outside the application where they are missing some context. We don’t currently have a good paradigm to represent attack detection and possible response inside an application, unless you completely roll your own. There is actually an implementation of this concept in ESAPI, (see the IntrusionDetector interface) and AppSensor’s implementation can be used as a drop-in replacement for the default handler provided by ESAPI, but more on that later. For now, let’s talk about why you need AppSensor.
So, you do good threat modeling, good security architecture, design, coding, code reviews, white/black box security scans, etc. … great, but you still have security holes. It’s a fact of life that today’s complex applications, and the heavily constrained timelines under which they are built, lead to security holes. If you have an application of any decent size, you are almost assuredly vulnerable to something. You should absolutely continue good secure development practices – these are crucial – but we need more.
Let’s consider those folks that actually end up breaking into our apps. It turns out that attackers vary in how good they are. Some are better than others; some are phenomenal, though the bar has continually been lowered due to automated tools. However, any attacker, bad, good or great, almost always has to try several things in order to successfully break into your app (for many classes of vulnerabilities). If they get in on the first try, they’re either lucky, or your app is so insecure, there’s not much that can help it. So, what if we could differentiate the bad guys before they actually get in? That’s the idea of AppSensor. Let’s assume that it takes an attacker 20 tries to find a field he can manipulate to get an XSS bug, but you can detect that he’s trying to find an XSS bug in 5 tries. Now, we have the advantage – we’ve determined he’s a bad guy, and can respond before a successful attack occurs.
The AppSensor project represents both the concept and implementation of an application layer intrusion detection system. First, a little info (and a few references) about the conceptual pieces. There are 3 basic concepts in AppSensor:
1. Detection
2. Evaluation
3. Response
Detection is the process of actually recognizing that an attacker is performing an attack. This involves adding code (which we’ll show a sample of below) to points that are deemed valid “detection points”. These are points in the code where we know the expected reasonable path(s) a user should be performing, and have a way to detect the user has varied from the path. Consider the case where a user submits a GET request when the form is setup for a POST. That should NEVER happen in a normal circumstance unless the user is doing some modification. A collection of these detection points has been created by the team in order to help you get started in determining reasonable locations to check for attacks.
Evaluation is the process of evaluating all the actual attack detections across the system for all users and determining when it is appropriate to trigger certain response actions. This is, for lack of a better term, the “engine” of the system, and is really behind-the-scenes code for anyone who wants to implement AppSensor. The interface to this would be the threshold setup (shown below)
Response is the activity of actually responding when it is determined an attack has actually occurred. A collection of these response actions has been created by the team in order to help you get started in determining reasonable responses for attacks.
These are the basic components of the system, but in order to really understand the system, feel free to read the book. This was a mammoth effort to document this concept, and as open source projects go, is some of the best documentation available.
My key part in the project has been from the development side. For a quick “getting started” version of using AppSensor that has some good information (I wrote it 
), see the developer guide. Here I’ll show the few basic steps to getting going. I’m going to show the setup here as if you’re using OWASP ESAPI in your project. This is by no means required, but since there is already the concept of intrusion detection, it makes life a bit simpler. Also, since ESAPI is a set of distinct controls, you can use some and not others. So, here are the steps:
1. Replace default ESAPI intrusion detector with AppSensor intrusion detector.
ESAPI.IntrusionDetector=org.owasp.esapi.reference.DefaultIntrusionDetector becomes ESAPI.IntrusionDetector=org.owasp.appsensor.intrusiondetection.AppSensorIntrusionDetector
2. Add threshold configuration. There is an appsensor.properties file (see dev guide for details) that you can use to configure thresholds for when an attack is considered to “trigger” a response. Here’s an example configuration for a threshold.
# http://www.owasp.org/index.php/AppSensor_DetectionPoints#ACE2:_Modifying_Parameters_Within_A_POST_For_Direct_Object_Access_Attempts # number of intrusions in a specified segment of time that constitutes the upper threshold - once crossed, it's considered an "attack" IntrusionDetector.ACE2.count=3 # segment of time (in seconds) IntrusionDetector.ACE2.interval=3 # list of actions you want executed in the specified order as the threshold for this intrusion is met - ie. log the first time, logout the user the second time, etc. IntrusionDetector.ACE2.actions=log,logout,disable,disableComponent # some integer - duration of time to disable IntrusionDetector.ACE2.disableComponent.duration=30 # some measure of time, currently supported are s,m,h,d (second, minute, hour, day) IntrusionDetector.ACE2.disableComponent.timeScale=m # some integer - duration of time to disable IntrusionDetector.ACE2.disableComponentForUser.duration=30 # some measure of time, currently supported are s,m,h,d (second, minute, hour, day) IntrusionDetector.ACE2.disableComponentForUser.timeScale=m
3. Add detection points into your code. This is the hardest actual work – it requires finding those places in your codebase where you can check for an attack. This example shows an instance where we’ve decided that 404 requests could be considered an attack. In this case, a single request or even a handful might be legitimate, but dozens or more probably point to a scanner of some type looking for vulnerable code.
//This example snippet might be placed on a jsp that handles HTTP 404 errors.
//When the page is accessed, this code notifies AppSensor that an invalid page request was made.
//Notice that the exception is created, not thrown
new AppSensorException("ACE3", "Invalid request", "Attacker is requesting a non-existent (404) page (" + requestedURI + ")");
So, that’s it – pretty simple. AppSensor is another tool in the toolbelt for the security developer/practitioner. It is an idea that can be implemented in an application fairly easily and with great effect. It has the power to detect and prevent an attack before it is successful, and has the key advantage of context within an application, allowing us to respond to the user that performed the attack.
That’s it for now. I may post more detailed information about AppSensor as the project progresses. For now, feel free to comment here and let me know what you think.
I should briefly mention the other members of the core team: Michael Coates is the project lead, and Colin Watson is another key member of the team.
AppSensor Project Page
AppSensor Book
AppSensor Developer Guide