One-Time Passwords – HOTP and TOTP

About One-Time Passwords in general

One-Time Passwords (OTP) are pretty much what their name says: a password that can be only used one time. Comparing to regular passwords OTP is considered safer since the password keeps on changing, meaning that it isn’t vulnerable against replay attacks.

When it comes to authentication mechanisms, usually OTP is used as an additional authentication mechanism (hence OTP is commonly referred to as two factor authentication/second factor authentication/step-up authentication). The main/first authentication step is still using regular passwords, so in order your authentication to be successful you need to prove two things: knowledge and possession of a secret.

Since the OTP is only usable once, memorizing them isn’t quite simple. There is two main way of acquiring these One-Time Passwords:

  • hardware tokens: for example YubiKey devices, which you can plug in to your USB port and will automatically type in the OTP code for you.
  • software tokens: like Google Authenticator, in this case a simple Android application displays you the OTP code which you can enter on your login form.

There is two main standard for generating One-Time Passwords: HOTP and TOTP, both of which are governed by the Initiative For Open Authentication (OATH). In the followings we will discuss the differences between these algorithms and finally we will attempt to use these authentication mechanisms with OpenAM.

Hmac-based One-Time Password algorithm

This algorithm relies on two basic things: a shared secret and a moving factor (a.k.a counter). As part of the algorithm an HmacSHA1 hash (to be precise it’s a hash-based message authentication code) of the moving factor will be generated using the shared secret. This algorithm is event-based, meaning that whenever a new OTP is generated, the moving factor will be incremented, hence the subsequently generated passwords should be different each time.

Time-based One-Time Password algorithm

This algorithm works similarly to HOTP: it also relies on a shared secret and a moving factor, however the moving factor works a bit differently. In case of TOTP, the moving factor constantly changes based on the time passed since an epoch. The HmacSHA1 is calculated in the same way as with HOTP.

Which one is better?

The main difference between HOTP and TOTP is that the HOTP passwords can be valid for an unknown amount of time, while the TOTP passwords keep on changing and are only valid for a short window in time. Because of this difference generally speaking the TOTP is considered as a more secure One-Time Password solution.

So how does OpenAM implement OTP?

In OpenAM currently there is two authentication module offering One-Time Password capabilities: HOTP and OATH. So let’s see what is the difference between them and how to use them in practice.

HOTP authentication module

The HOTP module – as you can figure – tries to implement the Hmac-based One-Time Password algorithm, but it does so by circumventing some portions of the specification. Let me try to summarize what’s different exactly:

  • The shared secret isn’t really shared between the user and OpenAM, actually it is just a freshly generated random number, each HOTP code will be based on a different shared secret.
  • The moving factor is statically held in OpenAM, and it just keeps on incrementing by each user performing HOTP based authentications.
  • A given HOTP code is only valid for a limited amount of time (so it’s a bit similar to TOTP in this sense).

These differences also mean that it does not work with a hardware/software token since the OTP code generation depends on OpenAM’s internal state and not on shared information. So in order to be able to use the generated OTP code, OpenAM has to share it somehow with the user: this can be via SMS or E-mail or both.

So let’s create an example HOTP module using ssoadm:

openam/bin/ssoadm create-auth-instance --realm / --name HOTP --authtype HOTP --adminid amadmin --password-file .pass

Now we have an HOTP module in our root realm, let’s configure it:

openam/bin/ssoadm update-auth-instance --realm / --name HOTP --adminid amadmin --password-file .pass -D

Where contains:


NOTE: The names of the service attributes can be found in the OPENAM_HOME/config/xml/amAuthHOTP.xml file.

At this stage I’ve modified the built-in demo user’s e-mail address, so I actually receive the OTP codes for my tests.

Last step is to create an authentication chain with the new HOTP module:

openam/bin/ssoadm create-auth-cfg --realm / --name hotptest --adminid amadmin --password-file .pass
openam/bin/ssoadm add-auth-cfg-entr --realm / --name hotptest --modulename DataStore --criteria REQUISITE --adminid amadmin --password-file .pass
openam/bin/ssoadm add-auth-cfg-entr --realm / --name hotptest --modulename HOTP --criteria REQUIRED --adminid amadmin --password-file .pass

Here the first command created the chain itself, the other two commands just added the necessary modules to the chain.

We can test this now by visiting /openam/UI/Login?service=hotptest . Since I don’t like to take screenshots you just have to believe me that the authentication was successful and I’ve received my HOTP code in e-mail, which was in turn accepted by the HOTP module. :)

OATH authentication module

The OATH module is a more recent addition to OpenAM which implements both HOTP and TOTP as they are defined in the corresponding RFCs. This is how they work under the hood:

  • Both OTP mode retrieves the shared secret from the user profile (e.g. it is defined in an LDAP attribute of the user’s entry).
  • HOTP retrieves the moving factor from the user profile.
  • TOTP does not let a user to authenticate within the same TOTP window (since a given TOTP may be valid for several time windows), hence it stores the last login date in the user profile.

For the following tests I’ve been using the Google Authenticator Android app to generate my OTP codes.

Testing OATH + HOTP

First set up the authentication module instance:

openam/bin/ssoadm create-auth-instance --realm / --name OATH --authtype OATH --adminid amadmin --password-file .pass
openam/bin/ssoadm update-auth-instance --realm / --name OATH --adminid amadmin --password-file .pass -D

Where contains:


NOTE: here I’m using the givenName and the sn attributes for my personal tests but in a real environment I would use dedicated attributes for these of course.

Again let’s create a test authentication chain:

openam/bin/ssoadm create-auth-cfg --realm / --name oathtest --adminid amadmin --password-file .pass
openam/bin/ssoadm add-auth-cfg-entr --realm / --name oathtest --modulename DataStore --criteria REQUISITE --adminid amadmin --password-file .pass
openam/bin/ssoadm add-auth-cfg-entr --realm / --name oathtest --modulename OATH --criteria REQUIRED --adminid amadmin --password-file .pass

In order to actually test this setup I’m using a QR code generator to generate a scannable QR code for the Google Authenticator with the following value:


NOTE: The Google Authenticator needs the secret key in Base32 encoded format, but the module needs the key to be in HEX encoded format (“48656c6c6f576f726c644f664f415448″) in the user profile.

After all these changes bring up /openam/UI/Login?service=oathservice and just simply ask for a new HOTP code on your phone, you should see something like this:

Google Authenticator - HOTP

Enter the same code on the login screen and you are in, yay. :)

Testing OATH + TOTP

For testing TOTP I’ll be reusing the authentication modules and chains from the OATH HOTP test:

openam/bin/ssoadm update-auth-instance --realm / --name OATH --adminid amadmin --password-file .pass -D

Where contains:


We need to generate a new QR code now with the following content:


Again, let’s test it at /openam/UI/Login?service=oathtest and this time we should have a slightly different UI on the Google Authenticator:

Google Authenticator - TOTP

As you can see there is a nice indicator now showing us how long this TOTP code will remain valid.

I think now we know about OTP codes more than enough, hope you’ve found this article useful. ;)

Creating space & sandcastles

Boxed in

ForgeRock is hiring lots of people these days. Great for the company, and a fantastic opportunity for those who we bring on. In the process of hunting for future ForgeRock talent however, I have been struck by how many candidates are boxed in at their current companies. By boxed in I mean that they get put into a particular role and position with little room for development until more senior staff step aside, or the team is disbanded.

If life is a journey then this compares to standing patiently in a queue, waiting for the day it is your turn. Food for a patient soul, rather than an adventurous one.

Opening the box

It is natural for teams and organizations to have a structure; anarchy is seldom fun for long. In the engineering team at ForgeRock we manage that structure by creating personal roles and creating room for personal development. Typically in a software team you will find team leads, architects, senior developers, etc – fairly fixed roles (although I have never heard the same definition of an architect role twice!). By hiring to the team with the expectation that everyone will partake in these activities we open up new possibilities where individuals can explore and grow without the pressure of a binary decision. Sure, it takes a special kind of team to make this work, but that is what we are all about – building a company of great people.

Creating space

But how to provide an opportunity for the developer who would like to get more involved in design questions, or have a more customer-facing role? The problem with being boxed in is that there is no-where to go, no space to move into. That space is filled by other people already doing the job (engineers are not known for elbowing their way forward) – and that is what this boils down to: making sure that the big personalities and roles don’t do the jobs that others could grow into. Like a sandy beach, you may want the water to fill the moat around your sandcastle but instead of waiting for the tide, you remove some sand to make room for the water to fill.

I’ll conclude this thought for the day with Jimi Hendrix’s Castles made of Sand:

Zero Trust and the Age of Global Connectivity

Global connectivity is omnipresent when it comes to smart phones and tablets.  It's not so much a case of looking for a power adapter when on the road, it's constantly about 3G and 4G signal strength or availability of contract hotspot wifi services.  However, global connectivity has also had a profound impact on enterprises.  There is no longer a rudimentary partitioning of network infrastructure into
public and private areas.  The firewalls of old have been replaced by application firewalls, data loss prevention operations and advanced tracing, tracking and event monitoring.  The internal 'trusted' network no longer exists.  Employees often pose the biggest threat to information assets, even though they are trusted with legitimate accounts on protected internal machines.

Zero Trust as a New Model

Zero Trust is a recent security approach that looks to move away from network segmentation and focus more on data and resources and who can access them, when and from where.  This helps to remove the antiquated approach of being on trusted grounds, which often helps create a singularity point which malware and hackers can focus upon.  By defining more context around individual information assets or services, allows for the opening up of those resources to globally connected devices, whilst securing access based on the who, where and why and not just their network location.  Access is permitted on the traditional 'need to know' basis, whilst being under continual review.  This would require all access to start from a minimal (if none-existent) level, whilst every connection being tracked and monitored.

Internet of Things & Modern Connectivity

I wrote recently of Protection & The Internet of Things and how, with the proliferation of previously 'dumb' devices enriching the Internet, comes a need for increased security context and reliance on the identity of things.  By extending a 'zero trust' model to this brave new world of increased interconnectedness, we can start to see the benefits of things like personalised search results, personalised home and environment settings, dynamic ordering and choice removal.  All devices, services and assets should start from a place of zeroaccess, with trust relations being built between identities and data which the devices can help bridge and create connections.

Zero Trust or Zero Protection?

But should the assumption be of zero trust or zero protection?  Many penetration testing organisations and web security auditors, promote the message that an organisation will be hacked at some point, so it's advisable to put in place recovery plans.  By focusing simply on prevention, an organisation can be opened up to irreversible damage if a breach were to occur.  So, do we take that approach to all services, devices and identities?  Perhaps.  With the increased level of services, API's, identity providers and data being created and consumed, existing models for security relationships are open to many potential failures that could impact the Confidentiality, Integrity and Availability paradigm of traditional security.  Do we follow a zero trust model or simply say, well my phone will be hacked at some point, so I will not rely on it so explicitly?  Time will tell.

By Simon Moffatt

Ansible roles for OpenDJ

My colleague Warren, who I had the pleasure to work with at Sun and again with ForgeRock, has been playing with Ansible and has produced 2 roles to install OpenDJ and to configure replication. Check Warren’s blog post for the details, or go directly to the Ansible Galaxy.

Filed under: Directory Services Tagged: ansible, devops, directory, directory-server, ForgeRock, ldap, opendj, roles

Troubleshooting 101

When running an application in a production environment I would say the most important thing is to ensure that the application keeps behaving correctly and the provided services remain in a nice and working state. Usually people use some sort of a monitoring solution (for example Icinga) for this, which periodically checks the health of the service, and sometimes even asserts that the service produces expected results. While this is an important part of the administration process, today we are going to talk about something else (though closely related), namely what to do when things go wrong: the service becomes unavailable, or operates with degraded performance/functionality. In the followings I will try to demonstrate some possible problems with OpenAM, but within reasonable limits the troubleshooting techniques mentioned here should be applicable to any Java based applications.


As the very first step we always need to determine what is actually not working, for example: is OpenAM accessible at all? Does the container react to user requests? Is it that just certain components are not functioning correctly (e.g. authentication or policy), or is everything just completely “broken”?

The second step is just simply: DON’T PANIC. The service is currently unavailable, users can be already affected by the outage, but you need to stay calm and think about the root causes. I’ve seen it way too many times that a service was restarted right away after outage without collecting any sort of debug information. This is almost the worst thing you can possibly do to resolve the underlying problem, as without these details it is not possible to guarantee that the problem won’t reoccur again some time later. Also it is not even guaranteed that a service restart resolves the problem.. So basically if you look at it, in the end you have two choices really:

  • Either restart the service and potentially you end up missing crucial debug data to identify the root cause of the issue, and essentially you’re risking to run into this problem again causing yet another outage for you.
  • OR collect the most (and hopefully relevant) debug data from your system for later investigations (bearing in mind that during this period users are still unable to access the application), and then restart the service.

I hope I don’t have to tell you, the second option is the good choice.

In any case

Always look at the application logs (in case of OpenAM, debug logs are under the debug folder and audit logs are under the log directory). If there is nothing interesting in the logs, then have a look at the web container’s log files for further clues.

When functionality is partially unavailable

Let’s say authentication does not work in OpenAM, e.g. every user who tries to authenticate gets an authentication failure screen. In this case one of the first things you would need to look at is the OpenAM debug logs (namely Authentication), and determine which of the followings cause the problem:

  • It could be that a dependent service (like the LDAP server) is not operational, causing the application level error.
  • It could be that there is a network error between OpenAM and the remote service, e.g. there is a network partition, or the firewall decided to block some connections.
  • It could be that everything else works fine, but OpenAM just is in an erroneous state (like thinking that the LDAP server is not accessible, but actually it is).
  • Or my favorite one: a combination of these. :)

Based on the findings you are either going to need to look at the remote service or maybe even at the network components to see if the service is otherwise accessible. In the local scenario it may be that the logs is all you got, so preferably you should get as much debug information out of the working system as you can, i.e. enable message level debug logging (if amadmin can still authenticate), and then reproduce the scenario.

Upon finding clues (through swift and not necessarily thorough analysis) it may become straightforward that some other debugging information needs to be collected, so collect those and hope for the best when restarting the system.

When everything is just broken :(

Well this is not the best situation really, but there are several things that you can check, so let’s go through them one by one.

Dealing with OutOfMemoryError

Firstly OutOfMemoryErrors are usually visible in the container specific log files, and they tend to look like:

java.lang.OutOfMemoryError: Java heap space

If you see this sort of error message you should:

  • Verify that you have configured the process to use the minimal requirements for the applications (for example OpenAM likes to run with -Xmx1g -XX:MaxPermSize=256m settings as a minimum).
  • Try to collect a heap dump using the jmap command, for example:
    jmap -dump:live,format=b,file=heap.bin <PID>

    if that doesn’t work, then try to use the force switch:

    jmap -F -dump:format=b,file=heap.bin <PID>

To make sure that you have good debug data you should also do the followings (potentially before the problem actually happens):

  • Set the following JVM properties to enable automatic heap dump generation upon OOME:
    -XX:+HeapDumpOnOutOfMemoryError	-XX:HeapDumpPath=./oome.hprof	
  • Enable GC logging with the following JVM properties, so you can see the memory usage of the application over time:
    -Xloggc:/home/myuser/gc.log -XX:+PrintGC -XX:+PrintGCTimeStamps -XX:+PrintGCDetails -verbose:gc

In case of an OutOfMemoryError usually the container stops responding to any requests, so in such cases it is useful to check the container logs or the heap dump path to see if there was an OOME.

Handling deadlocks

Deadlocks are mostly showing up quite similarly to OOMEs, as in the container stops responding to requests (or in certain cases they only affect a certain component within the application). This is why if you don’t get any response from the application, then it may as well be due to a deadlock. To handle these cases it is advised to collect several thread dumps from the JVM using the jstack tool:

jstack <PID>

NOTE: that jstack generally seems to work better when it is run by the same user as who runs the target JVM. Regardless, if it still doesn’t want to give useful input, try to use:

jstack -F <PID>

If it still doesn’t want to work, you can still attempt to run:

kill -3 <PID>

In this case the java process is not really killed, but it is instructed to generate a thread dump and print it to the container logs.

Generate a few thread dumps and save each output to different files (incremental file names are helpful), this way it should be possible to detect long running or deadlocked threads.

High CPU load

In this case the service is usually functional, however the CPU usage appear to be unusually high (based on previous monitoring data of course). In this case it is very likely that there is an application error (like endless loop), but in certain cases it can be just simply the JVM running GCs more often than expected. To hunt down the source of the problem three thing is needed:

  • thread dumps: tells you which thread is doing what exactly.
  • system monitoring output: tells you which application thread consumes the most CPU.
  • GC logs: this will tell you how often did the JVM perform GC and how long did those take, just in case the high CPU utilization is due to frequent GCs.

On Linux systems you should run:

top -H

and that should give you the necessary details about per-thread CPU usage. Match that information with the thread dump output and you got yourself a rogue thread. :)


Monitoring is really nice when actively done, and sometimes can even help to identify if a given service is about to go BOOM. When there is a problem with the service, just try to collect information about the environment (use common sense!), and only attempt to restore things afterwards.

Ansible roles to install ForgeRock’s OpenDJ LDAP server

Ansible is a really nice "dev-ops" automation tool in the spirit of Chef, Puppet, etc.  It's virtues are simplicity, an "agentless" installation model and a very active and growing community.

One of the neat features of Ansible is the concept of "roles". These are reusable chunks of dev-ops code that perform a specific task. Ansible "Playbooks" orchestrate a number of roles together to perform software installation and configuration.

Roles by themselves are not sufficient to drive reusability.  We need a way to collaborate and share roles.    Enter Ansible Galaxy, the central repository for Ansible roles.

If you have ever used apt or yum, galaxy will appear quite familiar. For example, to install and use the "opendj" role, you issue the following command:

$ ansible-galaxy install warren.strange.opendj

(Roles are prefixed with a contributor name to avoid name collisions).

If you want to install ForgeRock's OpenDJ server, here are two new Ansible roles:

  • opendj  - Downloads and installs the OpenDJ server
  • opendj-replication - sets up replication between two OpenDJ instances.

Here is a sample Ansible playbook that installs two instances on a single host and replicates between them:

# Example of installing two OpenDJ instances on the same host (different ports)
# and enabling replication between them
# Most of the variables here are defaulted (see the role opendj/defaults/main.yml for defaults)
- remote_user: fr
sudo: yes
hosts: ois
- { role: warren.strange.opendj, install_root: "/opt/a" }
- { role: warren.strange.opendj, install_root: "/opt/b", opendj_admin_port: 1444, opendj_ldap_port: 2389,
opendj_ldaps_port: 2636 , opendj_jmx_port: 2689, opendj_service_name: "opendj2" }
- { role: warren.strange.opendj-replication, install_root: "/opt/a", opendj_host2: localhost, opendj_admin_port2: 1444 }

This is my first attempt at an Ansible role. Feedback is most welcome! 

Save the date for the 2014 ForgeRock Identity Relationship Management Summit

The date has been set, the 2014 ForgeRock summit in United States will take place on the week of June 2nd, in Phoenix AZ.

Make sure you block the date in your calendar ! I hope to see you there.

And if you’re in Europe, don’t panic ! We are also planning an EMEA summit in the fall. The date and location will be announced later.

Filed under: General Tagged: conference, ForgeRock, identity, Identity Relationship Management, IRM, openam, opendj, openidm, security, summit