Month: August 2013

Fun with Active Directory

As we all know Active Directory can cause quite a few headscratching moments in general, now I’m trying to conclude some of my findings so maybe others won’t suffer from the same thing.

Creating user fails with LDAP 53

LDAP 53 is UNWILLING_TO_PERFORM, now there is two main candidate reason for this to happen during an ADD operation:

  • SSL isn’t used. AD will not process LDAP operations that involves setting/modifying the unicodePwd attribute if the request was made on a non-secure connection. Always make sure you use LDAPS when connecting to AD.
  • The provided password does not satisfy the password policies configured in AD. This is a bit harder to identify, but for the sake of testing try to use complex passwords and see if the user gets created like that.

Changing password fails

As stated already: AD requires SSL connection when sending the unicodePwd attribute. If this wouldn’t be already enough you have to also enclose the password value with double quote characters and the password should be in a UTF-16LE encoded format. A sample code snippet would look something like:

public byte[] encodePassword(String password) {
    return (""" + password + """).getBytes(Charset.forName("UTF-16LE"));

There is two main way to change passwords by the way:

  • A non-administrative password change: you BIND as the user whose password needs to be changed, and then you send a ModifyRequest with a DELETE and an ADD ModificationItem containing the old and the new password respectively (in the encoded format of course).
  • Administrative password reset: in this case you BIND as an admin user and then send a ModifyRequest with a REPLACE ModificationItem only containing the new password value.

Easy, right?


It appears that the DELETE/ADD combination can fail with the following error:

Constraint Violation: 0000052D: AtrErr: DSID-03190F80, #1:
	0: 0000052D: DSID-03190F80, problem 1005 (CONSTRAINT_ATT_TYPE), data 0, Att 9005a (unicodePwd)

In our case the root cause was the default password minimum age policy in AD (1 days). Once I’ve ran the following command from an Administrator CMD:

net accounts /MINPWAGE:0

AD started to like my ModifyRequests.

Identity Repositories

Identity Repositories (or Data Stores if you’d like) are almost the most important part of OpenAM. They enable you to manage identities within your organization (to a certain degree), and are involved in many processes in general:

  • during authentication it is often necessary to contact the IdRepo to get informations about the user (like the e-mail address, so an OTP code can be sent out)
  • after authentication takes place the user’s profile is looked up (for example to check that the user account is not inactivated)
  • once you have a session you can easily configure a PA to retrieve profile attributes in HTTP headers
  • when you authenticate via SAML, the IdRepo is contacted to retrieve the values for the SAML attributes in the assertion
  • etc

Some (boring) background about IdRepos

As you can see IdRepo is quite an important part of the product, so let’s try to get a better understanding of it.
In general, a given identity repository basically represents a single source of identity data. In this case identity can mean users, groups (and with ODSEE it can even mean roles and filtered roles). While it is common to use a regular relational database to store application data, in enterprise systems it is much more common to use some LDAP-capable directory server (OpenDJ, Active Directory, etc) instead. Since OpenAM is meant to be used as an enterprise solution it has been designed to work mainly with directory servers, but it is not limited to those (there is also Database IdRepo implementation, and with some patience any other sort of implementation can be written just as well). To make it simple let’s only talk about directory servers for now…

So as you already can guess, OpenAM has an LDAPv3Repo named IdRepo implementation which basically should be able to communicate with any LDAPv3 compliant directory server. While this sounds great, this also means that the implementation has grown quite big with time. There are parts for example where special handling is required for a given directory server type (yes, I’m talking about AD)..
In case of versions prior to 10.2.0, the LDAPv3Repo implementation has been using the quite old Netscape LDAP SDK to perform the LDAP operations. Although this solution works, I must say that support for the Netscape SDK is pretty much non-existent (and on top of that the whole SDK has been basically repackaged within OpenAM), so when it comes to debugging, it can become quite difficult to tackle down a given problem (again, we are talking about legacy code here). Because of this, and the fact that the OpenDJ team has started to develop an own LDAP SDK (which by the way ROCKS!), we decided to drop Netscape from the product. The good news is that this change already starts with 10.2.0 (though unfortunately there will be still parts of OpenAM using the old SDK, but hopefully not for too long).

So how do IdRepos work?

You can define any amount of Identity Repositories (Data Stores) within a realm, the only caveat is that all of the configured IdRepos will be contacted whenever an operation needs to be performed. You may ask why, well here it is:
OpenAM uses IdRepos to abstract away from identity repository differences, and within a realm it uses AMIdentityRepository to abstract away from the multiple IdRepo concept. This means basically that OpenAM only knows about the single AMIdentityRepository, which then knows everything about the different IdRepo configurations within the realm. To complicate things AMIdentityRepository is using IdServices to access all the IdRepo implementations, so let’s see the 4 different IdServices implementation for reference:

  • IdServicesImpl – the most basic implementation, this one just looks through the IdRepo list and executes the given operation on all of them. At the end it combines the results from the different IdRepos.
  • IdCachedServicesImpl – this implementation stores operation results in a cache, and when a given requested data cannot be found in the cache (which by the way is invalidated based on persistent search results), it just asks IdServicesImpl to contact the IdRepos for it.
  • IdRemoteServicesImpl – a remote implementation which is using JAX-RPC to contact an OpenAM server (which in turn will use Id(Cached)ServicesImpl) to perform operations remotely
  • IdRemoteCachedServicesImpl – as its name suggests: caches on the client side and asks OpenAM via IdRemoteServicesImpl if there is a cache miss.

Because IdRepos are implemented this way, you can hopefully see why we say that OpenAM wasn’t meant to be used for identity management. Creating/reading/updating/deleting entries from all the configured IdRepos cannot always fulfill all the requirements, and the expected behavior is poorly defined (for example what happens if you run a delete on an entry that only exists in one IdRepo?).

Alright, alright, but what’s new in OpenAM 10.2.0?

We have implemented a new IdRepo implementation, which is only using OpenDJ LDAP SDK to perform LDAP operations, keeping in mind that the behavior should be similar to the old Netscape-based IdRepo implementation. The main changes however are:

  • The LDAP failover code has been rewritten, and it should be much more reliable with the new release.
  • In the old version the user search attribute was misused, and it was basically handled as the naming attribute of the entry (i.e. first attribute name in the DN), this has been changed, so now the user search attribute is only used when performing search operations in the IdRepo, and the already existing naming attribute setting is used as naming attribute now.
  • Two new option has been introduced: heartbeat interval and heartbeat timeunit. These settings control the interval for sending out heartbeat search requests for idle connections. This means that if the connection is not idle, there won’t be any heartbeat request either.
  • Because of this the idle timeout setting has been removed. When there is a network component closing idle connections after a certain period of time, you should just configure the heartbeat interval correctly, and that should basically prevent the connection becoming idle.
  • Data store level cache has been removed (not implemented).

I think we can all agree on the fact that 10.2.0 will be an exciting release. πŸ™‚

Java EE agent internals

Not so long ago I’ve been working on JBoss v7 agent, so let me try to sum up some of my findings with Java EE agents in general.
A Java EE agent basically stands from 4 main components:

  • Installer
  • Container specific JAAS integration
  • Java EE Filter implementation
  • Agent application

Let’s go through each one of these and see how they really work.

Agent installer

As its name suggest, the agent installer helps to install the Policy Agent (PA) on the chosen Java EE container. The main behavior of the installer is defined within config/configure.xml file. This file is basically a big XML descriptor of the followings:

  • interactions – what should be asked from the user when performing install/custom-install/migrate/uninstall
  • tasks – what tasks needs to be performed as part of the agent installation, things like backup, creating directory layout, etc

So basically this XML describes how the installer should work in general. Now let’s go even more deeper…


During an agent installation all user input is being stored in a thing called IStateAccess (well sometimes there is a persistent=”false” interaction which isn’t persistent, but that part I don’t fully grasp yet πŸ™‚ ).
As part of an interaction it is possible to set up different validators (even custom ones), so for example this makes it possible that a given agent is actually installed on a supported container version. For example with JBoss we are running “JBOSS_HOME/bin/instancename.(bat|sh) –version” command to get back the version number of the JBoss instance, parse it and if it matches, we let the installer further.


Tasks are usually just a well defined “change” that needs to be performed by the installer. Implementing a task is not just performing a given change, rollback is just as important part of it. In order to make an agent work, it is very likely that some container specific configuration file needs to be modified (for example to configure/enable the JAAS integration), the necessary file paths are all calculated based on the user input during the interactions (IStateAccess).
Usually an agent installer performs the following tasks:

  • creating backups for the container’s configuration files
  • perform modifications to the container configuration
  • create agent directory layout (i.e. creating the Agent_00x folder structure)
  • encrypt the agent profile’s password and save it in the agent bootstrap config
  • generate the configuration files based on the values provided to the installer
  • in certain scenarios: deploy agentapp.war in the container
  • in case of custom-install: create agent profile if needed

As you can probably see there is nothing super magical about the agent installer, its sole purpose is basically to make the deployer’s life easier by automating this whole process.
I feel we heard enough about the installer for now, so let’s go a bit further and see what the JAAS integration is really all about.

Container specific JAAS integration

JAAS in general isn’t that new really, it’s been part of JRE 1.4 already, so let’s look at it very shortly (here is a longer version):

  • You can implement a custom authentication module
  • You can define authentication “chains” using built-in/custom modules in a configuration file

Well hopefully this concept doesn’t sound too new for you, since the whole OpenAM authentication system is based on JAAS. πŸ˜‰
As an extra, application servers tend to have the concept of “JAAS realm”, which is basically a collection of users (to be sparse as much as possible). Unfortunately there is not much standard around implementing JAAS support for application servers, so this part of the agents is different per container (and sometimes even across container versions).

Advantages of JAAS

Well the main advantage of JAAS is that it can help making the OpenAM integration as less intrusive as possible, but there are others as well:

  • Retrieving the logged in user’s name is quite simple, you can call request#getRemoteUser() or request#getUserPrincipal().
  • By setting up Java EE Security it is possible to define roles in the application’s descriptor files and later on in your application you can just simply check against these using request#isUserInRole(String) (authorization).
  • In case you later on decide to use an LDAP based JAAS module instead, the idea is that you can simply switch out the JAAS login module (of course you’ll need to configure it to work similarly to OpenAM), without actually modifying any part of your application code.
  • While this is already good enough, I think JAAS really pays off when EJBs are used in applications. In that case you can use the @DeclareRoles and @RolesAllowed annotations and magically your EJB method will be protected, only those in the configured role can actually invoke the method (anything else results in failure), which sounds quite neat. πŸ™‚

So what about agents?

The Java EE agents support JAAS, they integrate well with the container and within the agent configuration it is possible to set up role mapping based on pretty much any arbitrary data (session property, profile attribute). To enable JAAS support though you must ensure that you configure the agent in J2EE_POLICY or ALL mode. While it is possible to define security-constraints in web.xml to protect access to pages, usually that’s a bit cumbersome, instead it’s probably simpler to just define OpenAM policies (i.e. use the agent in ALL mode), and only use JAAS for the previously described goodies.

Java EE Filter implementation

The agent itself is implemented as a Java EE filter, which is (obviously) standard, so this part is common for all the agents and as such it is part of the agent SDK. When the filter intercepts a given request, it will basically go through a list of “TaskHandlers” and execute them in a predefined order. If a given TaskHandler returns with an AmFilterResult, then the processing stops and the result will be handled. Here is a small example subset of the available TaskHandlers:

  • CDSSOTaskHandler – detects if the user needs to authenticate using CDSSO and redirects to CDCServlet if necessary
  • CDSSOResultTaskHandler – processes the incoming LARES response and creates the cookie on the application’s domain
  • URLPolicyTaskHandler – checks the currently visited URL against configured URL policies in OpenAM, and blocks access if necessary
  • InitialPDPTaskHandler – saves POST data if the user hasn’t authenticated yet, so later on the POST can be resubmitted once authentication took place.
  • XSSDetectionTaskHandler – checks incoming request parameters for malicious characters

Since the agent is implemented as a filter, you can see how it can alter the outcome of a given incoming request. If the user doesn’t have session yet, it will prevent further processing of the page, and redirect straight to OpenAM instead. This however means that in order to protect the application fully, you must set the agent’s filter as the first one, otherwise it may be possible that a filter earlier in the chain alters the response, or returns earlier, which could basically prevent the agent to protect the application properly.

Agent application

And here is the last component which is pretty simple, the agent application. The purpose of this application is to provide a unique contextroot for the agent, so the agent can receive notifications from OpenAM, and also LARES responses in case of CDSSO. To be fair the agentapp only has the agent’s filter defined, and that basically handles everything. If there is an incoming notification, then the filter just checks if the requested URL is actually the notification URL, and processes it if so.
As a sidenote I would mention that deploying agentapp.war is not necessary when using global web.xml with Tomcat 6. That is simply because the global web filter already includes the agent filter, hence any incoming request to the /agentapp context will be catched by the filter already.


Agents are fun, but that doesn’t mean they are not complex. πŸ™‚