The OWASP Top Ten and ESAPI – Part 5 – Insecure Direct Object Reference

This article will describe how to protect your J2EE application from direct object reference (DOR) attacks using ESAPI. As with all of the detail articles in this series, if you need a refresher on OWASP or ESAPI, please see the intro article The OWASP Top Ten and ESAPI.

What’s the problem?

What is direct object reference (DOR)? Other than being a somewhat quirky, generic name … here’s what OWASP says it is:

“A direct object reference occurs when a developer exposes a reference to an internal implementation object, such as a file, directory, database record, or key, as a URL or form parameter. An attacker can manipulate direct object references to access other objects without authorization, unless an access control check is in place. ” [from here]

So, what does that mean? Well for most (not all) J2EE developers, it will mean database key references, so that’s what we’ll deal with here. However, the discussion is applicable no matter what your reference.

Consider the following database table called USERS.

Now, let’s say that this database table were queried in a normal application. Then, the results are presented on the page, and each record in the result list would probably have a link to more detail about each user, having their pictures, their list of friends, etc. This is the typical master-detail interaction and is prevalent in pretty much any web app, not just J2EE. But how do we get to the detail page? How do we pass along which user we want to see in detail from the “master” table to “detail” page? Well, we could start by using the first name, but that’s not unique, or we could use the last name, but same issue. We could include both, which is less likely to produce duplicates but it’s still possible. In most situations, what happens is each table in the database has a column setup as a unique identifier. Each record in the table will have a different ID. So, that’s what we use. Now, here’s what the url might look like to go view greater detail about Bill:

/users/viewDetail?id=2

Ok, great, so what we have is the identifier being passed in the url to represent Bill. That way when we get to the view detail code, we simply check the id that was passed in and pull up that record in detail and display the information.

So where’s the issue? Well, everything works fine in this instance b/c every user is able to see the detail, but let’s look at what this might look like for a bank, which is the classic DOR example. In a bank, all records for all users are stored in the same table (at least in our example), however each user can only view his or her account. They should not be able to simply pick a random account to view or manipulate. But what if the URL to my account looks like:

/accounts/viewDetail?id=3540

What would happen if I changed this to

/accounts/viewDetail?id=3541

If there’s no protection in place, I can see someone else’s account!

Well, this is the crux of the DOR problem. Any attacker can manipulate url or form parameters at will when they submit a request, which means this issue obviously applies to both GET and POST requests, as well as exposed parameters, hidden fields, etc. If it’s being sent from the client to the server, the client can alter it. That means when you use the ID as the reference, and that ID can be manipulated, the attacker can gain access to data that they should not have access to. The ID in this case is the direct reference (id) to the object (record in the table) that you are trying to access.

Where do we go from here? ….

Well, ok, that’s not super encouraging. We have an issue (that most people have never heard of) that is affecting most applications . Banking is the obvious example, but this could be a problem for many different types of applications. Take heart, there is a solution (actually there are two)! You have 2 options for how to properly deal with this issue when it exists in your application.

1. Use ESAPI’s AccessReferenceMap (or something similar)
2. Perform authorization on the view detail page
Bonus. Use both! (the only way to go for paranoid people like me)

Let’s discuss the basic concepts of each of the options.

AccessReferenceMap

The AccessReferenceMap is an interface for a map which has methods that allow you to add a “direct” reference (which generates an “indirect” reference), then retrieve an indirect reference using a direct reference, or retrieve a direct reference using an indirect reference.

Confused yet? Me too. Actually it’s quite simple. Let’s use a contrived (and insecure) example just to illustrate the concept (again, this simplified version is NOT secure).

Let’s say you have a table in the database with 4 records, and their id’s are 1,2,3,4. Now let’s say you have the situation above where you display all these records in a master table, and each row is a link to more detail about the individual record. Let’s say you want to hide these numbers and reference them on the UI as letters instead. The simple mapping would be 1=A, 2=B, etc… In this case, when you retrieve the data from the database, your ids are 1,2,3,4. However, when you show them on the screen, their ids are A,B,C,D. Finally from the detail page (after the user clicks the link), you would just lookup what C matches, find that it is 3, then retrieve that record from the DB and display it. It’s that simple.

Now, in practice, obviously a letter -> number matrix is too simple to be effective. What works better, as usual in computer science, is randomness. That’s why ESAPI already contains a class called RandomAccessReferenceMap. Here’s an example of how you might use it.

MyObject obj;	 	// generate your object
Collection coll; 	// holds objects for display in UI

//create ESAPI random access reference map
AccessReferenceMap map = new RandomAccessReferenceMap();

//get indirect reference using direct reference as seed input
String indirectReference = map.addDirectReference(obj.getId());

//set indirect reference for each object - requires your app object to have this method
obj.setIndirectReference(indirectReference);

//add object to display collection
coll.add(obj);

//store collection in request/session and forward to UI
...

As you can see the code is fairly simple. Beyond this, the only other code is when it is displayed on screen, you use the indirect reference to be the id that gets you to the view detail page. From the view detail page, you retrieve the id that was passed in, then iterate over the collection in memory until you find the object from the map matching the indirect reference and then retrieve the full data from the DB by mapping back to the original ID.

A couple notes here:
- If you get an id passed in that doesn’t match any in your collection, it means either the code is wrong, or the user has tampered with it, so it should be a security issue
- You’ll note my object has a method called setIndirectReference. All my value objects in my applications extend a base class with some helper methods, and this is one of them. This is obviously not required and there are many ways to accomplish the usage of the RandomAccessReferenceMap, but I build my apps knowing I’m going to use this technique, so it’s not a problem for me.

Per-Page Authorization

Per page authorization is another technique that has a similar goal to the AccessReferenceMap. The idea here is that you’ve already done the appropriate authorization on the master page. Again, for simplicity, let’s use the bank account example. Let’s say you have a checking and savings account with a particular bank, and both of those accounts show up on your “master” accounts page. The code has to check what you have access to (authorization) in order to only display the appropriate 2 accounts to you on that page.

Unfortunately, that’s where many applications stop. They assume since they’ve only given you 2 accounts to choose from, you’ll have to choose 1 of the 2. This doesn’t take tampering into account, however, and tampering can allow the attacker to input any account they choose.

What the per-page authorization says is that if and when the attacker tampers with the data sent to the detail page, it still shouldn’t be possible for him to see data that is not his. The code that brings up detail about a specific account should first include a check that you have access to that account (authorization). Again, if he doesn’t have access to that account, either the code is broken, or he is tampering and that is a security incident.

So, which option should I choose

Well, either or both. They are both viable options to solve the DOR problem, but I’ll try to give a couple of points about both that may sway you one direction or the other.

AccessReferenceMap
1. *May* not require an extra DB hit to do detail page authorization – depends on how you’ve coded it.
2. Shields actual IDs from end user. In case you ever have a vulnerability on another page, and haven’t protected for it properly, the attacker won’t know your IDs.
3. Requires a bit of extra coding and some extra information to pass around the application in order to maintain access to the reference maps.
4. Currently not very popular, so not very well understood yet.

Per-Page Authorization
1. *May* require an extra DB hit to do detail page authorization – depends on how you’ve coded it.
2. Exposes actual IDs from end user. In case you ever have a vulnerability on another page, and haven’t protected for it properly, the attacker could know your IDs.
3. Doesn’t requires extra coding or the passing of information around the application in order to maintain access to the reference maps.
4. Very well understood paradigm.

Well, there you have it. There’s really not that much to it. While direct object access is a significant issue in many of today’s applications, it can be fairly simply corrected. I’ve presented a couple of options that can be done separately or combined together as a solution. I like to use them both, but separately, they still do their job properly.

Just one final safety note. While I’ve only discussed these techniques in the context of database queries, whichever one or both that you choose should additionally be applied when referencing any sensitive internal resources, including files, internal objects, classes, etc.

Well, that’s it. Hope you’ve found this article helpful. Let me know if you have any questions or suggestions to improve any of my ideas.

Other articles in this series:
Part 1: The OWASP Top Ten and ESAPI
Part 2: The OWASP Top Ten and ESAPI – Part 2 – Cross Site Scripting (XSS)
Part 3: The OWASP Top Ten and ESAPI – Part 3 – Injection Flaws
Part 4: The OWASP Top Ten and ESAPI – Part 4 – Malicious File Execution

Technorati Tags: Direct Object Reference, ESAPI, J2EE, Java, OWASP, Security

The OWASP Top Ten and ESAPI – Part 4 – Malicious File Execution

This article will describe how to protect your J2EE application from malicious file execution attacks using ESAPI. As with all of the detail articles in this series, if you need a refresher on OWASP or ESAPI, please see the intro article The OWASP Top Ten and ESAPI.

What’s the problem?

So what exactly is malicious file execution? It could mean different things to different people. Here, I want to limit the discussion to the execution, either immediate or delayed, of files or file handles that can be manipulated in some way by user input. This is a fairly broad definition that can cover a few different situations. Let’s first discuss a few of the different types of issues that can come up. This collection is by no means exhaustive, but should give us decent coverage of the types of issues that arise when dealing with file input.

1. Included file name is partially or wholly determined by dynamic input (think dynamic include files)
2. File is uploaded and written to disk with “as is” (no validation of uploaded file – just write to disk)
3. File name to display or retrieve is partially or wholly determined by dynamic input
4. Command / data file is uploaded (think a batch process being kicked off with data from an uploaded spreadsheet)

So, what types of issues do these particular situations present us with. In general, an attacker can manipulate this process to gain increased privileges, access data for which they are not authorized and even execute code. Since these aren’t the only 4 examples of issues that exist, we won’t specifically delve into each issue in detail, but we’ll try to extract some common problems that are present in one or more of the scenarios listed above. This set of problems is related to common file upload / processing situations.

1. Missing or insufficient input validation – This is almost always true with file processing. This also is a broad topic, since it deals both with the file name and path, as well as the file contents. Many times, this could express itself as uploading a file as is and simply writing it out with a filename that is either the name of the file as uploaded, or a name specified by the user. This could cause issues like directory traversal (“../../etc/passwd” – overwriting the passwd file – sorry for the old cliche example), or overwriting other files in the filesystem (“../../jsps/login.jsp” – overwriting the login page). Something like this could allow the attacker to break the site significantly up to and including owning the site, and even the server it’s hosted on. While most file processing code I’ve seen is not bad enough to allow something like this, some of it is just that bad .. scary.

2. No virus scanning – This goes along with input validation to some extent, but if you are uploading a file that is going to be executed in any way at any time, it should be virus scanned. There are engines out there, even free ones, that you can call out to in order to scan the file. If the scan fails, the file should be deleted, and the incident should be logged as a security incident. At that point, the normal security incident response process in your organization can be executed.

3. No size checks – One of the simplest tasks you can perform that is often overlooked is to check the file size of the uploaded file. There is most likely a reasonable limit to the size of your accepted files. For instance, a spreadsheet would have to have quite a bit of data to be any more than a few MB, but an attacker could upload files of many times more than that to try and fill up the file system, or just hang processing on the server to block out other valid users.

4. Invalid file type processing – This task is a bit more difficult, but makes sense in many cases. If you are an image hosting site, there’s no reason to accept Word files. This should involve creating a whitelist of the types of files you will accept and then verifying that only those are uploaded. This does NOT mean file extension checks. Although these can be used as a superficial first test, they are trivial to change and provide no security assurance. Often times, this check will involve output encoding (see #7).

5. Giving too much control over file name input – This occurs when users are allowed to influence the final filename that is output to the filesystem. This could happen by allowing a user to type in the desired filename or by just accepting their filename and using that on the server side. However, it is much safer to generate the filename that is used to save the file to the filesystem. Generally, this would have some securely random string of characters, as well as the author, date, time, etc. If there is a specific requirement to accept input from the user as to what the filename should be, just ensure that the name given is validated against an acceptable whitelist before writing the file out.

6. Direct object reference (DOR) problems – While the DOR issue affects more than just filenames, it is relevant here so we’ll quickly discuss it. DOR is simply an issue where the actual filename is pointed to directly. This filename could be in a hidden field, stored in a cookie, or some other place that is not directly seen by the user on the screen. Typically the filename is taken directly and used to retrieve or execute the file. This could cause issues of broken authorization (attacker changes filename to access another user’s file), elevation of privileges (touching a file belonging to a user with higher privileges or the system user), or many other problems. This topic is covered in greater detail in the next article in the series.

7. No output encoding – This issue doesn’t always exist since not every filetype can be thought of in this way. The basic issue is when a file is not validated properly before being written out. There are several filetypes that are valid, that could have extra information in them. However, of those, there are some file types which can be read in, and then written out again to ensure the “bad stuff” is not still there. This does not apply to all “bad stuff” that can be contained in those files, nor does it apply to every file type, but it can be useful when the option is available, though it rarely if ever is used in many applications.

8. Not authorizing access – This issue exists because many applications contain reasonably appropriate authentication, but horrible if any authorization. Once a user is authenticated, many times they can perform any function in the application. The classic example of this is not restricting access to a certain page, let’s say the admin screen. The home page might not show the link, but if they type the link into the url bar, there’s nothing stopping them from getting there. Depending on the type of site, this could be especially harmful with file upload. Often times, only “trusted” people are expected to be able to upload files, but if the authorization is broken, it could allow anyone to upload files at will. Again, this depends on the type of site you have – it may be perfectly fine for everyone to upload files (like flickr or youtube).

Where do we go from here? ….

So, we’ve discussed lots of issues that can crop up when dealing with uploading files, so how does ESAPI and general best practices say we should deal with these issues? Let’s take them one by one.

1. Missing or insufficient input validation – As discussed in part 2 of this series, ESAPI makes it fairly simple to do proper input validation through the framework. The code is very simple, and extensible through the addition of new regular expressions. Here is an example of how ESAPI checks the filename of the uploaded file to verify that it is valid.

if (!ESAPI.validator().isValidFileName("upload", filename, allowedExtensions, false)) {
	throw new ValidationUploadException("Upload only simple filenames with the following extensions " + allowedExtensions, "Upload failed isValidFileName check");

}

2. No virus scanning – ESAPI does not directly support virus scanning through the base APIs, but several of the antivirus vendors support API access for Java. I’ve heard that there are also free AV tools that offer this feature as well.

3. No size checks – There is no built in support for this in ESAPI. However, several of the web frameworks do provide support for this. Additionally, most if not all file upload libraries also provide support for this. Lastly, it’s trivial to perform your own check even if your specific library does not support it directly. Just load the uploaded file into a file object and call the length() method on it to determine the size of the file in bytes. Here is what ESAPI uses to prevent large files. The following is a snippet:

ServletFileUpload upload = new ServletFileUpload(factory);

upload.setSizeMax(maxBytes);

4. Invalid file type processing – This task is supported by the input validation portion of ESAPI again. First, the filename could be validated by the application. Then, each input from the file that is processed could again be validated by the application. Think of processing a spreadsheet. The filename should first be tested, then not only each line, but each cell should be checked for validity before being used by the application.

5. Giving too much control over file name input – In order to at least partially solve this issue, ESAPI has the SafeFile object. This is a trivial extension to the java.io.File object that additionally performs checks of the file and directory names to check for unsafe characters. However, this is a blacklist check. It is likely that you could do a better job if you know what characters are allowed in your filenames and only allow those (whitelist), though the characters in the ESAPI blacklist should minimally be disallowed.

6. Direct object reference (DOR) problems – Again, this will be covered in greater detail in the next article, but here I will just mention that ESAPI does have a mechanism to hide DORs. This involves essentially creating a map of direct -> indirect references and then using the indirect references for display, then mapping back during the processing. Come back for the next article for more details on how this is actually done in code.

7. No output encoding – There is no built in support for this in ESAPI. However, there are ways of doing this that *may* work for different filetypes. The most obvious one is images. If an image is uploaded, you can open the file, load the contents into an image object, then save that object. This guarantees that the image is a valid object. Otherwise, loading the image will fail. (Note: this is the expected behavior, though there have been some defects that may cause this not to work – ensure your library version functions correctly here). It may also be possible to do this with the help of some 3rd party libraries depending on the file type you are processing.

8. Not authorizing access – ESAPI does have authorization coverage in the framework, but it deserves its’ own discussion. Actually, the last article in this series will cover restricting URL access which will discuss the authorization capabilities of ESAPI. Additionally, CSRF (Cross Site Request Forgery) protections can help deal with this issue as well, and this is also going to be covered in a later article in the series.

I’d like to mention a couple of other helpful things that are more systems management related when dealing with uploaded files.

- Use a safe base directory when saving your uploaded files that is outside the standard web directory that your site is served from. This will force your application to be the only thing that can access these files. That way, an attacker can’t drop a file in an accessible directory.

- Use the file-system access controls that are offered in your operating system and possibly your web/application server. Ensure that only the appropriate users have privileges and that they have the least privilege necessary to do their jobs. This is a necessary stopgap in case your other protections fail.

- Audit properly. This can’t be stressed enough. If you audit and log properly, and then *READ* those logs, then you can notice when strange behavior occurs. This can help you locate things that need to be resolved.

Hopefully this article has been useful in helping you recognize the types of issues related to file upload and some best practices for dealing with those. As in all the other articles, I recommend trying to be as safe as possible, and only having your app do what is necessary. If you don’t need to upload files, don’t do it. If you have to do it, spend time making sure it’s as safe as you can make it. Let me know if you have other suggestions or best practices you use with file upload.

Editorial Note: I realize it’s been quite a while since I posted any articles in this series. However, I plan to finish up the series in the next few months by posting an article every week or two now that I can spend a bit more time on it. I also realize that in the gap of time I had off, the new OWASP Top Ten (2010) has been released and it supercedes the 2007 list. However, all the issues in the 2007 list either still exist in the 2010 list, are covered by it, or are still issues that may not make that high of a ranking anymore, but are still important. For these reasons, and the fact that I’d already started using the 2007 list, I’ll finish up with the 2007 list. I may go back at the end and catch any new ones from the 2010 list if relevant.

Other articles in this series:
Part 1: The OWASP Top Ten and ESAPI
Part 2: The OWASP Top Ten and ESAPI – Part 2 – Cross Site Scripting (XSS)
Part 3: The OWASP Top Ten and ESAPI – Part 3 – Injection Flaws

Technorati Tags: ESAPI, J2EE, Java, Malicious File Execution, OWASP, Security, Validation

The OWASP Top Ten and ESAPI – Part 3 – Injection Flaws

This article will describe how to protect your J2EE application from injection (SQL and others) attacks using ESAPI. As with all of the detail articles in this series, if you need a refresher on OWASP or ESAPI, please see the intro article The OWASP Top Ten and ESAPI.

What’s the problem

Now to discuss injection. Again, let’s begin the the definition of injection from OWASP:
“Injection flaws, particularly SQL injection, are common in web applications. Injection occurs when user-supplied data is sent to an interpreter as part of a command or query. The attacker’s hostile data tricks the interpreter into executing unintended commands or changing data.”

Here I should make a quick note about “input”. While the vast majority of attacks are usually due to malicious user-supplied input, often directly through the browser, this is not necessary for the attack to work. Dangerous (whether malicious or not) could come from a variety of sources, like the filesystem, or the database, or even other applications. The rule is, if you take in input, always understand it could be dangerous. OK, back to injection …

If you look at injection alongside XSS, you’ll notice that they are very similar types of vulnerabilities in spirit. Consider this working definition: “The application takes in input and without proper data validation or encoding, outputs that data to a destination where certain values or entities have special meaning”. This could be describing XSS or Injection. In the case of XSS, our output destination (we’ll use OWASP’s “interpreter” term from here on out) happens to be a browser. For injection, it could be a database, LDAP server, operating system or any other number of interpreters. Each of the interpreters has its’ own semantics and processing rules for what it understands and what symbols and characters have semantic meaning as input.

For the browser, for instance, the bold tag has semantic meaning, which results in making certain text bold-faced.

<b>bold-faced</b>

As for a SQL compliant database, the keyword SELECT has semantic meaning in that it performs a read of some portion of the database.

SELECT first_name, last_name FROM employees;

As we saw previously in the XSS article, it is possible for a user to insert (inject) data that will cause issues if the input is not properly validated and/or encoded (my strong suggestion is to do both). The same thing is true with injection attacks. If the input is not valid for the interpreter, or more specifically contains some commands that will be processed by the given interpreter, then the user can cause the interpreter to perform functions on the user’s behalf that were not intended to be allowed by the application.

Let’s take a classic SQL injection attack as an example to clarify the issue. Here’s some simple Java code creating a SQL string to perform an authentication check so that a user can login. Let’s assume the Java code gets the results back and tests to see if there are any results, and if so, allows the user to be authenticated.

String sqlString = "SELECT * FROM users WHERE fullname = '"
    + form.getFullName() + "' AND password = '" + form.getPassword() + "'";

So, let’s assume the user is presented with a simple login screen, and let’s also assume the user is benevolent. The user might enter something like John Melton for the username and 123pass for the password. That would create the following SQL string to be executed against the database after it is processed by Java (ie. this is what you would see as the SQL string if you watched it through the debugger, or printed it out after the values have been populated at runtime).

SELECT * FROM users WHERE username = 'John Melton' AND password = '123pass'

OK, so far, everything is great right? Sure, for those specific inputs no problem. Now, to pay homage to a good friend of mine, I’ll include his oft-quoted refrain – “The web wasn’t made for Irishmen”.

Let me explain that. For those who are unaware, many Irishmen have names like O’Brien or O’Connor. Note the apostrophe in the last name. Now, let’s show what the SQL string looks like after being processed now.

SELECT * FROM users WHERE username = 'Paul O'Malley' AND password = '789pass'

Are we still good? Nope, this time, we’re not so lucky. What will happen here is a SQLException will be thrown because the SQL cannot be processed by the database interpreter. The issue is in the extra apostrophe in the name. The apostrophe is used in SQL, among other reasons, to open and close a string representation. So, what the SQL interpreter sees in this case is that the username string is populated with Paul O and then closed, and the actual SQL string starts again, only <b> is not valid SQL syntax, so processing stops and an exception is thrown.

(Note: I’m using interpreter generically. In the cases I’m describing, it actually would get booted by the parser, but that’s quibbling :>. )

All right, now that we’ve covered the basic “it works” case and also seen there is a way to break it, how can we break it to our advantage if we wanted to attack the site? Let’s say an attacker sends these two crafted inputs.

Full Name: blah blah
Password: ' OR '1' = '1

Update: modified code above according to Ben’s comment below.

Now that’s a funny looking string to be entered in the password field – let’s see what the SQL string produced looks like.

SELECT * FROM users WHERE username = 'blah blah' AND password = ' ' OR '1' = '1'

What the above SQL actually does is returns all records from the users table that use the given username and password OR where 1 = 1, which is … EVERY RECORD!!!. This grants the attacker access to the application as an authenticated user. This was an actual issue in a real (fairly well-known at the time) web application many years ago. It would check the results that were returned and take the first one as the user to log in. In the case where they were being attacked and all records were returned, it took the first result, which like most other systems first user, was the admin account! Most decent sites have closed this issue on the login screen thankfully, but it is a very prevalent issue in systems across the world.

The above is a simple example of what can happen with SQL injection. The vast majority of reported issues with injection have to do with SQL injection. The others are not as popular, probably because they’re less common types of systems (b/c almost every app uses a DB) and SQL injection is so easy and prevalent, there’s no real driver for attackers to do anything else. Additionally, most of the sensitive data is likely stored in the DB anyhow, so we might as well attack it directly if possible.

Where do we go from here? ….

So, now that we see the problem, how do we fix it? Well there are essentially 2 reasonably reliable techniques as I see it. The first option has to do with using PreparedStatements with bound parameters, and the second option uses the ESAPI framework.

Update: From Jeff’s (the chair of OWASP) comment below, it should be clear that PreparedStatements and the ESAPI output encoding mechanisms are not mutually exclusive. They can be complementary and used together for increased assurance. It is a task left up to the development organization securing their app to make a decision over which option (or both) to choose. As with most choices, each option (PS, ESAPI, or both) has its’ advantages and disadvantages.

Note: with either technique, white-list input validation should be used since it’s relatively easy to do, and helps avoid so many types of problems. Additionally, standard security measures like least privilege for the user account connecting to the DB for your application should also be utilized.

Before getting into the detail of the 2 options, I want to point out that the ESAPI folks have put together another article called the SQL Injection Prevention Cheat Sheet and it is a great resource about this topic, specifically SQL injection.

PreparedStatements (with bound parameters)
——————
Those familiar with DB programming in java will recognize the 3 statement types in Java: Statement, PreparedStatement and CallableStatement. These 3 types represent varying levels of security. The standard Statement class is not a good idea – there are no security measures built-in and you are on your own if you try and use this class. As for the CallableStatement, it is used with stored procedures and functions. *Typically* stored procs are secure. However, it is very possible to make them insecure by not validating the input properly or even constructing dynamic sql within the procedure for execution. Finally, there is the PreparedStatement. When used properly (with a safe driver and bound variables), these are typically viewed as the most secure option.

There are 2 important things to remember about PreparedStatements when it comes to security.
1. SQL executed with PreparedStatements is only as safe as the drivers that implement the classes. The actual runtime class that implements the PreparedStatement interface is part of a 3rd party package. There are some out there that are generally thought to be safe, and some that aren’t. The most popular ones expectedly tend to be more respected (not necessarily earned, but true nonetheless). The only suggestion I can give here is that you can look at the code of popular open source drivers yourself and make up your mind about their safety. If you use a commercial driver, understand the support structure in case there is a security issue that must be resolved.
2. The code leveraging the PreparedStatements must use them properly. This involves the use of strongly typed bound parameters. This means that if you concatenate your SQL string together using dynamic input, the PreparedStatement does you no good. The safety comes into play when using the setXXX methods on the PreparedStatement class. The snippet below shows how to properly use dynamic input (which should already be validated) as input to the SQL string by doing parameter substitution. The setInt method below (if the driver is coded properly) should be DB specific and will escape any necessary characters for that DB to prevent breaking out of the interpreter and allowing the user to specify code that will be run by the DB.

myPrepStmt = conn.prepareStatement("SELECT name FROM users WHERE id = ?");
myPrepStmt.setInt(1, validatedUserId);
...
//execute statement and get results

and a query with multiple input parameters looks very similar …

myPrepStmt = conn.prepareStatement
    ("SELECT name FROM users WHERE id = ? AND date_created >= ?");
myPrepStmt.setInt(1, validatedUserId);		//first param
myPrepStmt.setDate(2, validatedStartDate);	//second param
...
//execute statement and get results

As you can see, using the PreparedStatement can solve this issue, but only when used properly. One very important thing to note here is about ORM frameworks. Most of these frameworks *can* be used securely, but you have to know what you’re doing. Hibernate, for instance, can be used securely or insecurely depending on which classes you choose. Be sure to do your homework, read solid tutorials, pick good frameworks, and above all understand what you’re using when you let a framework do some of the work for you. OK, now on to our other option …

ESAPI
——————
ESAPI takes a similar approach to solving this issue, but the code will look slightly different. The approach is very simple, and boils down to escaping/encoding all dynamic input to the SQL statement using an appropriate encoder. For this section of the article, I relied heavily on the ESAPI section of the SQL Injection Prevention Cheat Sheet at OWASP, so it’s a worthwhile time to go read it if you haven’t already. Let’s look at one quick example to show how it works. We’ll modify the second query above to use ESAPI instead of PreparedStatements.

//ESAPI version of query
Codec ORACLE_CODEC = new OracleCodec();		//we're using oracle
String query = "SELECT name FROM users WHERE id = " +
   ESAPI.encoder().encodeForSQL( ORACLE_CODEC, validatedUserId)
   + " AND date_created >= '"
   + ESAPI.encoder().encodeForSQL( ORACLE_CODEC, validatedStartDate) +"'";
myStmt = conn.createStatement(query);
...
//execute statement and get results

As you can see above, all that we did was encoded the input data (already validated) for SQL by using the OracleCodec, since we are using the Oracle database for our application. This has some obvious drawbacks if you are using multiple databases, and can use the ORM features of cross-database query generation, but that is rare for many apps. Most applications know the 1 database they are going to be running on, and I would argue that’s a good thing from a security perspective because then you can better understand the security ins and outs of that one DB.

Looking at the code, it’s fairly simple to encode most anything – you just have to be diligent to do so. As for the other types of injection, you may have to get your hands dirty and write an encoder for your specific application, or you may not. You’ll just have to check what the ESAPI team has already done. Additionally, it might be possible to get on the mailing list and request a new codec. Certainly, if you have any that you’ve written, I’m sure the dev team would love to at least see them, and possibly include them – you could be helping others out!.

Clearly, using the ESAPI framework, it’s possible to safely and securely prevent injection from the DB as well as other types of interpreters if you use the framework properly.

In summary, I’ll say that while I think either of these techniques work from a purely technical perspective, there are a couple of things you should consider before implementing protection. First, PreparedStatements are far more prevalent in industry. They are a solid solution to the SQL injection problem, and are widely understood and available. They’ve also been very widely tested both for performance and security. Essentially, they are a known entity. However, they do not solve any of the other types of injection. The equivalent to PreparedStatements do not exist for XML or XPath injection or others. You’d have to handle them differently. As for ESAPI, it will also work well, and has been tested for security and performance, but not nearly as much. Additionally, it does not have the industry reach that PreparedStatements do. However, ESAPI has a solution for the other types of injection as well, and a clear model for implementing new encoding mechanisms for new interpreter types. You can even write your own custom encoders if you choose. No matter which you choose (or both), my hope is that you’ve seen there is a need to be responsible and protect resources from injection. These solutions make it fairly trivial to implement solid protection for applications against injection.

Other articles in this series:
Part 1: The OWASP Top Ten and ESAPI
Part 2: The OWASP Top Ten and ESAPI – Part 2 – Cross Site Scripting (XSS)

Update: Added inline comments referencing changes made regarding Jeff and Ben’s comments below.

Technorati Tags: ESAPI, Injection, J2EE, Java, OWASP, Prepared Statement, Security, SQL Injection, Validation

The OWASP Top Ten and ESAPI – Part 2 – Cross Site Scripting (XSS)

This article will describe how to protect your J2EE application from XSS using ESAPI. As with all of the detail articles in this series, if you need a refresher on OWASP or ESAPI, please see the intro article The OWASP Top Ten and ESAPI.

OK, so on to XSS. Here is a slightly modified definition of XSS from OWASP:
XSS flaws occur whenever an application takes untrusted (typically user supplied) data and sends it to a web browser without first validating or encoding that content. XSS allows attackers to execute script in the victim’s browser which can hijack user sessions, deface web sites, possibly introduce worms, etc.

As you can see, XSS essentially allows an attacker to splash whatever they want on the screen since the application doesn’t do any input validation or output encoding. This is not a big deal when you have benevolent users, but an attacker could, say, input some nasty JavaScript and cause quite a few problems. This is typically what happens – JavaScript is output and generally executes in the background so the user is unaware of what’s occurring.

In general there are 3 types of XSS:
1. Stored – The dangerous data is stored in a permanent data store and shown repeatedly on the site – think online forums. If an attacker were to save a forum comment that had an XSS exploit in it, that comment would then be displayed to anyone who visited the page, without requiring any further interaction from the attacker.
2. Reflected – The dangerous data is sent along with the URL typically, and is not stored in a data store but is displayed (reflected) back to the user, which launches the attack. If an attacker sent a URL in an email, and a user clicked on it b/c the site was “trusted”, say that user’s bank, but the site had an XSS exploit, the attacker could cause many issues up to and including processing transactions on behalf of the user at the bank site (more on this in a follow-on article).
3. DOM – This is the most recently named type of attack, and simply represents an entirely client side issue. While the previous 2 types of XSS have to do with the server outputting data to the browser, this has to do with the browser manipulating the DOM and data being moved in and out of context. This can result in an XSS attack if data is not properly validated or encoded. This type of XSS issue has become more prevalent with web 2.0 and the heavy proliferation of JavaScript frameworks being used to do both serious functionality and DOM manipulation in web pages.

XSS, by some accounts, is the most common vulnerability and definitely is one of the most dangerous in the wild. It is extremely prevalent due to a lack of education for the most part, but it can at times be tricky to solve correctly. There are essentially 2 options for how to deal with XSS. Both could be used individually to solve the problem entirely … in theory … but that would require detailed knowledge of every possible input/output vector both now and in the future. Since this is rarely feasible, it is recommended to use both approaches. These approaches are Input Validation and Output Encoding.

Input Validation
—————-
Input validation is simply that – checking each input for validity. This can mean many things, but in the typical and simplest case, it means to check the type and length of the data. For instance, if you are accepting a standard US zip code from a text box, you would know that the only valid type is a digit (0-9) and that the length should be 5, no more and no less. Not all cases are this simple, but many are similar.

Consider this example for what can go wrong. There is a simple search engine and on the page is the search box. A user types in a query only to find on the results page that his search terms are printed on the screen, something like – You searched for “free stuff”. That’s all well and good, but what happens when the user inputs a bit of JavaScript into the search box? If the application doesn’t handle the input/output properly, the screen will print the JavaScript out in all it’s glory, and it will get put inline in the web page response, and treated just as if the web page developer had put that bit of code in there. That’s where the problems come in – now how do we solve these issues?

Here’s an image from OWASP showing their architecture for input validation. The key here is that everything is validated, all input that doesn’t originate within the application (including user input, request headers, cookies, database data, ldap, really everything …).

So how do we use this validation framework to actually validate our data. There are 2 basic types of methods in the validator interface that can be used. They are listed below:

getValidInput(java.lang.String context, java.lang.String input, java.lang.String type,
	int maxLength, boolean allowNull, ValidationErrorList errors)
isValidInput(java.lang.String context, java.lang.String input, java.lang.String type,
	int maxLength, boolean allowNull)

The first, getValidInput, returns canonicalized and validated input data along with a list of errors (ValidationErrorList) if any validation issues occurred. The second is similar, but does not return errors, and rather just returns a boolean as to whether or not the input is valid. Here are a couple real examples of these being used.

String validatedFirstName = ESAPI.validator().getValidInput("FirstName",
	myForm.getFirstName(), "FirstNameRegex", 255, false, errorList);
boolean isValidFirstName = ESAPI.validator().isValidInput("FirstName",
	myForm.getFirstName(), "FirstNameRegex", 255, false);

Both of the samples above deal with the first name field from a typical web form, but this could just as easily be a request header or parameter, or cookie value, or anything else.

In the end, there is great value in general in validating ALL of your application inputs. It will help solve the XSS issue, but will also solve other problems, including some we probably haven’t even thought up yet.

Output Encoding
—————-
On the flip side of input validation is output encoding (also known as “escaping”). Here’s a quick definition from OWASP: “Escaping” is a technique used to ensure that characters are treated as data, not as characters that are relevant to the interpreter’s parser. There are lots of different types of escaping, sometimes confusingly called output “encoding.” Some of these techniques define a special “escape” character, and other techniques have a more sophisticated syntax that involves several characters. Escaping is the primary means to make sure that untrusted data can’t be used to convey an injection attack. There is no harm in escaping data properly – it will still render in the browser properly. Escaping simply lets the interpreter know that the data is not intended to be executed, and therefore prevents attacks from working.

What does all of this mean to the developer? In order to prevent “bad” data from causing XSS issues on the screen when rendered, we can’t just out.println them to the screen – we have to “encode/escape” them. There are many libraries out now that do some form of encoding, most are minimal. c:out and jstl both do encoding – some of the struts tags, jsf, spring. All do some minimal encoding. However, ESAPI takes this to a different, but necessary level.

There are 2 issues with the previously mentioned frameworks when it comes to their encoding schemes. 1 – They don’t encode enough characters – they miss some things. 2 – They only encode for 1 context and miss the other 4. As mentioned in the cheat sheet, there are actually 5 output contexts for the browser:
1. HTML entity (this is the standard HTML output that the above frameworks at least partially handle)
2. HTML Attribute
3. JavaScript
4. CSS
5. URL

This image from OWASP shows their architecture for output validation. The important notion here is that before any data is output in the application, the context for output is considered, and the data is encoded.

All of these contexts have some special handling rules. I suggest you reference the cheat sheet in order to learn those. I’ll give a couple of simple examples here just for reference.

First, a simple example to output to an HTML entity.

//performing input validation
String cleanComment = ESAPI.validator().getValidInput("comment",
	request.getParameter("comment"), "CommentRegex", 300, false, errorList);

//check the errorList here ...
...

//performing output encoding for the HTML context
String safeOutput = ESAPI.encoder().encodeForHTML( cleanComment );

Now, an example of creating a URL that is safe for output.

//performing input validation
String cleanUserName = ESAPI.validator().getValidInput("userName",
	request.getParameter("userName"), "userNameRegex", 50, false, errorList);

//check the errorList here ...
...

//performing output encoding for the url context
String safeOutput = "/admin/findUser.do?name="
	+ ESAPI.encoder().encodeForURL(cleanUserName);

Above, you can see that is it very simple to encode output for a given context, as long as you know the context you’re going to. It does take discipline to use this throughout your application, but it will pay you back many-fold in rewards. Additionally, ESAPI does have tag libraries that wrap each of the output encoding mechanisms available for use.

One final note regarding output encoding: you should always explicitly set the character encoding for all your pages (ISO-8859-1 or UTF8 are popular choices).

As you can see, there is quite a bit to solving XSS. It can become tricky, especially in the instances where multiple contexts are involved, like trying to safely pass a parameter to a javascript function inside an HTML attribute handler event, but it is doable. The big takeaway should be that input validation and output encoding, while they do solve XSS, are in general excellent practices that should be exercised for ALL input to an application.

A final XSS note: A great reference for understanding what is occurring in XSS and how to protect against it is the OWASP XSS Prevention Cheat Sheet. Various references in this article drew from the info on this site.

Other articles in this series:
Part 1: The OWASP Top Ten and ESAPI
Part 3: The OWASP Top Ten and ESAPI – Part 3 – Injection Flaws

Note: Article updated on 11/18 per Jim Manico’s catch of improper url output encoding since ESAPI’s url encoding is intended for parameter values, not the entire url. Also added input validation as Jim’s comment pointed out was missing.

The OWASP Top Ten and ESAPI

This article will be the first in an 11-part series (yes eleven!) about the OWASP Top 10 and ESAPI (Enterprise Security API). This article will be a general introduction to the topic, while the follow-on articles will each cover one of the Top Ten web application security vulnerabilities and the associated usage of ESAPI (or another useful framework) to correct that vulnerability securely.

Before we get into the Top Ten, I should first mention what OWASP is for the unfamiliar. OWASP (http://www.owasp.org) is the Open Web Application Security Project. It is the result of a group of generous web application security folks offering their time and effort to build lots of useful security documentation as well as products. There are a large number of projects that are maintained by the group in varying levels of support, ranging from documentation (Top Ten) to teaching tools (webgoat) to proxies (webscarab) to a secure development library (ESAPI) and so on. This group is doing excellent work, and some of the smartest security folks around are running the show, so it’s a good group to get involved with if you are tasked with doing serious web development, or are involved on the security side of things.

Now, let’s move on to discuss the OWASP Top Ten. The Top Ten project is a list of the 10 most critical web application security issues. The list is updated every few years (most recently 2007) and is compiled with lots of input from the industry. The security issues on the list represent all at once those that are generally very simple, dangerous, well-known and commonly exploited in the wild. Here is a link to the Top Ten project at OWASP. http://www.owasp.org/index.php/Category:OWASP_Top_Ten_Project

As a harsh generalization, web developers are not very well equipped with security knowledge. On the other hand, attackers are. Since the sophistication level of many of the Top Ten attacks is so low in today’s environment, developers need to strengthen themselves quickly on two fronts: education and tools. That’s where OWASP and ESAPI comes in. As for education, there is an abundance of good educational material (see the secure development guide – which offers web application security best practices) on the OWASP site with new information put up frequently. From the tool perspective, ESAPI (Enterprise Security API) is a web application security tool (or rather a set of APIs that can be used as a tool) that is very effective at thwarting attacks when used properly.

ESAPI functions as a framework that developers can utilize to put protections in their applications that will prevent many if not all of the attacks openly used today. Even better, when used properly, it will also prevent “0-day” attack variants of known exploits and will very likely prevent new classes of vulnerabilities to some extent. Here is a link to the ESAPI project at OWASP. http://www.owasp.org/index.php/Category:OWASP_Enterprise_Security_API

The image below shows how ESAPI fits into your application design.

The image below shows how the features of ESAPI match up to the Top Ten vulnerabilities.

One great benefit to getting involved with OWASP and ESAPI is the community support. There is a great group of leaders in OWASP, and an equally great group of volunteer contributors. The level of education and toolsets being produced by the community is excellent, and getting better and more complete every day.

Although this quick introduction just barely skims the surface, hopefully it has been useful to explain what the Top Ten and ESAPI are generally useful for in their given contexts. Future articles in this series will cover each of the Top Ten vulnerabilities and the techniques that can be used to protect against them. Please come back and join me for the remainder of the series.

Other articles in this series:
Part 2: The OWASP Top Ten and ESAPI – Part 2 – Cross Site Scripting (XSS)
Part 3: The OWASP Top Ten and ESAPI – Part 3 – Injection Flaws

Writing a custom role based access control framework that integrates with Struts

This article will tell you a fairly simple way to put a custom access control framework in place for your J2EE application that will let you integrate it with the struts framework role based access control protection mechanisms. There are a few pieces to this solution:
1. A storage mechanism for the accesses (roles) per user
2. Writing these accesses into the HttpSession per user (generally in the form of an easily accessible object with helper functions on the session)
3. Override the processRoles method of the struts RequestProcessor class in order to recognize your custom roles
4. Add roles=”…” to your struts actions to protect them based on the configured roles

Ok, in order then, here we go.

For #1 – this is usually done via a relational database – whatever application database you’re using, you’ll generally store the accesses for a given user. However, you could store these in flat files, ldap, or some other mechanism if you wanted to – that’s up to you.

For #2 – here’s an object you can use to hold the roles.

public class UserAccesses {
    private String userName;
    private String[] roles;

    public UserAccesses(String userName, String[] roles) {
        this.userName = userName;
        this.roles = roles;
    }

    public boolean hasRole(String role) {
        if (roles.length > 0) {
            for (int i = 0; i < roles.length; i++) {
                if (role.equals(roles[i]))
                    return true;
            }
        }
        return false;
    }

    public String toString() {
        StringBuffer sb = new StringBuffer(1024);
        sb.append(userName);
        sb.append(" has roles: ");
        for(int i=0; i

    List allAccessesList = new ArrayList();
    //if the user has the write stories access from the db
    allAccessesList.add(Constants.WRITE_STORIES);
    //if the user has the read stories access from the db
    allAccessesList.add(Constants.READ_STORIES);
    //if the user has the delete stories access from the db
    allAccessesList.add(Constants.DELETE_STORIES);

    String[] allUserAccesses =
	(String[])allAccessesList.toArray(new String[0]);
    UserAccesses userAccessesObj =
	new UserAccesses (userId, allUserAccesses);
    HttpSession session = request.getSession();
    session.setAttribute(Constants.USER_SESSION_ACCESSES, userAccessesObj);

public class RolesRequestProcessor extends RequestProcessor {
    private static final Logger log =
	LogFactory.getLog(RolesRequestProcessor.class);
    public RolesRequestProcessor() {
        super();
    }

    /**
     * @see org.apache.struts.action.RequestProcessor
     *		#processRoles(javax.servlet.http.HttpServletRequest,
     *      javax.servlet.http.HttpServletResponse,
     *      org.apache.struts.action.ActionMapping)
     */
    protected boolean processRoles(HttpServletRequest request,
		HttpServletResponse response, ActionMapping mapping)
        	throws IOException, ServletException {

        //Obtain the list of roles from action config
        String roles[] = mapping.getRoleNames();
        if ((roles == null) || (roles.length < 1)) {
            log.debug("No role is required for running this operation.");
            return (true);
        }
        //verify user possesses role
        StringBuffer sb =
		new StringBuffer("Check for required roles ....... n");

        //checking in ldap roles
        for (int i = 0; i < roles.length; i++) {
            sb.append(roles[i]);
            sb.append(", ");

            if (request.isUserInRole(roles[i])) {
                sb.append("Found a matching role in the ldap roles: " + roles[i]);
                return (true);
            }
        }

        // checking in custom database roles
        // Check the current user against the list of required roles
        HttpSession session = request.getSession();
        UserAccesses userAccessesObj = (UserAccesses )
        	session.getAttribute(Constants.USER_SESSION_ACCESSES);
        if (userAccessesObj == null) {
            return false;
        }
        for (int i = 0; i < roles.length; i++) {
            if (userAccessesObj .hasRole(roles[i])) {
                sb.append("Found a matching role in the custom roles: " + roles[i]);
                return (true);
            }
        }
        //end custom roles check

        //invoke declarative exception handling
        sb.append("n");
        sb.append("User does not have the required role");
        log.debug(sb.toString() + " ");
        ActionForward forward = null;
        try {
            forward = processException(request, response,
                new my.application.AuthorizationViolationException(),
		null, mapping);
        } catch (Throwable e) {
            e.printStackTrace();
        }
        // forward user to the authorization error page...
        if (forward != null) {
            log.info("Forward to authorization error page");
            processForwardConfig(request, response, forward);
        }
        return (false);
    }
}

    <action path="/myApplication/deleteStory"
		type="my.actions.DeleteStoryAction" roles="DELETE_STORIES,ADMIN">
        <forward name="deleteSuccess" path="/myApplication/deleteSuccess.do" />
    </action>