Stupid Oracle vktm tricks to improve VirtualBox performance



In the process of creating a demo VirtualBox image running OEL 6 and the Oracle database 11.2.0.3.0 I noticed the idle CPU consumption was quite high (8% on the guest, 35% on the host).

The culprit turned out to be the Oracle database vktm process. This is a time keeping process - and it calls gettimeofday() *very* frequently.  This can have a negative performance impact in virtualized environments.

A colleague who is a database whiz suggested the following trick:


sqlplus / as sysdba
alter system set "_high_priority_processes"='LMS*' scope=spfile; 

This removes the vktm process from the list of high priority processes.

After this change (you need to bounce the database) the idle CPU consumption comes down to 1-2% or so. A nice improvement!

It goes without saying that this is:

a) Totally unsupported
b) Probably dangerous. This will most certainly break things in the database - such as statistics, auditing, etc.
c) For demo/development use only. If you care about your data don't do this!

YMMV.


Getting bitten by SELinux and sshd authorized_keys




TL;DR:  If you can't ssh using a public key, it could be a SELinux thing.


Logging in to a server with ssh using your public key is pretty handy.  While setting up an OEL 6 VM  I ran into a strange error where sshd would not let me log in with a public key, even though my key was in ~oracle/.ssh/authorized_keys.  Password logins worked just fine.

Somewhat puzzling: I could ssh into the root account using my public key and without a password.

Nine times out of ten, this is a permission problem. Sshd is picky about the permissions on your home directory, ~/.ssh, and the authorized_keys file.  I carefully checked this over - but in this instance permissions were not the problem.

The standard advice to debug SSH problems is to run sshd in the foreground with debugging turned on:

service sshd stop
/usr/sbin/sshd -dD

And of course my problem promptly disappeared. Hmmm, so it works in debug mode, but not when running as a daemon. It also works fine to ssh into the root account, but not ~oracle.

My initial google-fu skills were weak, but on a hunch I googled "sshd SELinux".

Bingo:

http://serverfault.com/questions/50573/selinux-preventing-passwordless-ssh-login


In my case disabling SELinux did the trick (not good for production, but acceptable for my purposes).



Oracle Identity Federation: Federate yourself!


A customer asked me how they could test their OIF IdP configuration without standing up another relying party.

Since OIF can act in both roles (IdP and SP), in turns out you can configure OIF to federate against itself.  It's seems somewhat crazy, and its not all that intuitive, so I thought I would include a few notes on how to set this up.

The key is that you must export OIFs SP and IdP metadata and re-import it back into OIF as configured federations. The "/fed/user/testspsso" test page can be then be used to initiate the federation.

Here are the basics of how to accomplish this.

Step 1: Export your SP and IdP metadata. This is done from the em console.
Administration -> Security and Trust:





Step 2) Import the exported meta data back into OIF:
Administration->Federations 

Click on the "Choose File" button and select the meta-data files you exported in step 1




You should now have an IdP and SP configured under your federations.  You may want to edit your federation settings to enable additional attributes to be mapped or change the default request format (POST, for example). See your OIF Admin guide for the details.


Step 3) Set your default SSO Identity Provider to be the newly created IdP that you just imported.

Administration->Service Provider->Common






Step 4) You are now ready to test your IdP

There is a neat Firefox plugin called "SAML Tracer" that is great for debugging SAML isses.  I highly recommend installing it.

Navigate to the test SP SSO page. This is located at:
 http://your-oif-server:7499/fed/user/testspsso

You should see something like this:




Click on "Start SSO".

You will be redirected to your OIF login page. Assuming you have configured LDAP authentication (Administration->Auth Engines -> LDAP), you will see this:


Login with your LDAP credentials, and you will see the response from the IdP:





If you are using the SAML tracer you can see the SAML request (OIF acting the SP role) and the response (OIF acting in the IdP role).  Here is an example request:









Congratulations. You have now federated with yourself!




How to use Certificate module with DAS

Previously I have described how to install the DAS properly. Today’s entry is all about using the Certificate authentication module in a deployment where you have a DAS instance. So here are the steps required to make this work:

  • Follow the DAS install guide, but with the only difference being that you install the DAS on a HTTPS URL (OpenAM can keep running on HTTP). Since you are now using HTTPS, you need to make sure that the OpenAM server trusts the DAS’s certificate, otherwise OpenAM won’t be able to send notifications to the DAS, which could result in strange situations when a given session remains valid for a short period of time even though the user has logged out on a different DAS instance.
  • Go to Access Control – realm – Authentication page and Add a new Module Instance called cert with type Certificate
  • Open the Certificate module configuration, and change the followings:
    • Set the LDAP Server Authentication User/Password to correct values
    • For evaluation purposes set the Trusted Remote Hosts to “all”
  • Generate a new self signed certificate by following this guide, but make sure that in the CSR you set the CN to “demo”.
  • Create PKCS#12 keystore for the freshly generated private key and certificate:
    openssl pkcs12 -export -inkey server.key.org -in server.crt -out server.p12
  • Install the PKCS#12 keystore in your browser (Guide for Firefox)
  • Enable Client Authentication on the DAS’s container. For example on GlassFish 3.1.2 Admin Console you would have to go to Configurationsserver-configHTTP serviceHttp Listenershttp-listener-2 then open the SSL tab and check the Client Authentication option.
  • Install the public certificate into the container’s truststore, so the container will actually trust your certificate:
    keytool -import -keystore glassfish3/glassfish/domains/domain2/config/cacerts.jks -file server.crt -alias mycert

    You need to make sure that you install the certificate to the actually used truststore, some containers may use the JVM’s truststore instead having one on their own.

  • On the DAS enable the Request/Response serialization by setting the following property in the ~/FAMDistAuth/*.properties file:
    openam.remoteauth.include.reqres=true
  • Restart the DAS’s container
  • Open /auth/UI/Login?module=cert and choose your certificate
  • Profit

Some useful shell scripts for setting up Oracle Linux



Do you frequently need to setup new instances of Oracle Linux to host the database, Weblogic, and  IAM Middleware?

I nabbed some great scripts from my colleagues Chris Johnson (of Fusion Security fame) and Art Mattson. I threw in a couple of my own scripts (great artists steal, right?), and have assembled these as a github project.

If you are setting up a new system grab the scripts with:

wget https://github.com/wstrange/Oracle-IAM-Scripts/zipball/master

Standard disclaimer: These scripts come with no warranty. Use at your own risk!

If you have some useful scripts that you would like to contribute drop me a line.

Integrating Oracle Identity Federation (OIF) with Service-Now



Oracle Identity Federation (OIF) provides SAML and OpenId federation services. OIF helps organizations to leverage SAAS  as part of their IT strategy by providing identity federation.

As part of  a recent POC we completed an OIF integration with Service-Now,  a cloud base IT Service management platform.

John Andersen is a rock star consultant from Service-Now who worked with me on the integration. We have documented the procedure and included a few screen shots (see link below).  Your environment will be a little different - but this recipe should get you going. Comments or corrections  are welcome!

Reference:  Configuring OIF and Service-Now


A Custom OIF Authentication Engine



Oracle Identity Federation (OIF) provides a flexible architecture that enables new authentication engines to be plugged in to the IdP flow.

There are a number of standard Authentication Engines that come out of the box such as Oracle Access Manager (OAM), LDAP and Oracle SSO (OSSO). Most deployments will use the OAM integration, but there are some scenarios where another mechanism is desired.

This posting will show you how to create and configure a custom authentication engine for OIF. In this example we will demonstrate the engine calling out to a simple web service that implements the authentication logic.

Authentication Flow

The authentication flow for our custom auth module is shown below.


















Recall the Fedlet from our previous article, which is a light weight SAML relying party implementation. The interesting bits here are the CustomAuth engine and the WebService that  implements our authentication logic.

Project Sample Code

The sample code for this project can be viewed and downloaded at https://github.com/wstrange/OIFAuthEngineDemo

The AuthSvc/ folder contains our authentication web service. The relevant snippet is


    @WebMethod(operationName = "authenticate")
public AuthInfo authenticate(@WebParam(name = "userid") String userid,
@WebParam(name = "password") String password) {

AuthInfo a = new AuthInfo();

if (userid != null && userid.startsWith("test")) {
a.setZipcode("12345");
a.setStatus("OK");
a.setSubscriberid(userid + "1234");
return a;
}
a.setStatus("ERROR-Auth");
return a;
}

This is a simple web service that authenticates any user whose id starts with "test" (no multi factor here!).  We  return an AuthInfo object that contains attributes for OIF to insert into the SAML assertion for a partner. Don't worry about the details of this web service - it is purely illustrative.

Under the engine/ folder for the project you will find the relevant jsp files for OIF integration. These jsp files are the entry hooks that OIF will call out to during user session creation. They must be deployed in the same container that OIF is running on as Servlet forwarding is used to pass data in and out your custom auth engine.

The integration hooks are documented in the OIF Admin guide.  The provided examples should get you bootstrapped.

loginpage.jsp is where the action gets started. This jsp is responsible for displaying the credential collection dialog to the user before forwarding control back to OIF.

Next, OIF will call out to the next JSP in the chain: forward.jsp. This jsp invokes our sample web service to authenticate the user. If authentication succeeds, the attributes returned from the web service are put in the SSO session. OIF will pick these up and add them to our SAML assertion for our partner.  If the authentication fails we redirect the user back to OIF where the login page will be tried again.

Here is the relevant java code from forward.jsp:

String refid = request.getParameter("refid");
String authnMethod = "oracle:fed:authentication:password-protected";
String userID = request.getParameter("username");
String password = request.getParameter("password");
// you get this id from OIF->AuthEngine->Custom page
String TEST_ENGINE_ID = "8172C8E5A7";
Date now = new Date();

AuthSvc_Service authsvc = new AuthSvc_Service();
AuthSvc svc = authsvc.getAuthSvcPort();
AuthInfo info = svc.authenticate(userID,password);

if( info == null || info.getStatus().startsWith("ERROR") )
{
String message = info.getStatus();

String redirectURL = "/engine/loginpage.jsp?refid=" +
(refid != null ? URLEncoder.encode(refid) : "") +
"&message=" + URLEncoder.encode(message);
response.sendRedirect(redirectURL);
return;
}

Map attrs = new HashMap(); // Attribute map to put in the SSO sesion
// Attributes from the sample web service
attrs.put("zipcode", info.getZipcode() );
attrs.put("subscriberid", info.getSubscriberid() );
// You can also pass back multi-valued attributes
HashSet s = new HashSet();
s.add("Bill");
s.add("Sue");
attrs.put("children", s);

request.setAttribute("oracle.security.fed.authn.attributes", attrs);
request.setAttribute("oracle.security.fed.authn.engineid", TEST_ENGINE_ID);
request.setAttribute("oracle.security.fed.authn.userid", userID);
request.setAttribute("oracle.security.fed.authn.refid", refid);
request.setAttribute("oracle.security.fed.authn.authnmech", authnMethod);
request.setAttribute("oracle.security.fed.authn.authntime", now);
request.getSession().getServletContext().getContext("/fed").
getRequestDispatcher("/user/loginsso").forward(request, response);

The final piece of the puzzle is the logout hook.  logout.jsp simply tells OIF to terminate the SSO session for the user.

The above jsp are packaged up into a war file, and must be deployed to the same Weblogic instance that is running OIF (wls_oif1, for example). Note this is *NOT* the admin server instance.

A tip: While debugging your engine deploy this as an "exploded war" file and enable hot jsp reload. This will allow you to edit your engine without bouncing the app server.

Configuring OIF 

Log on to the OIF console (http://localhost:7001/em for example), and navigate to
Administration->Authentication Engines

Click on the "Custom Authentication Engines" tab, and click on "+" to create a new engine. This is where we define our integration points.  Here is a sample:



Note the Engine ID column. This generated unique ID must be used in your forward.jsp to identify the engine to OIF.

Once you have defined your Authentication Engine remember to set it as the default engine.

Finally you will want to update the federation definition for you partner to pass back the desired attributes in the SAML assertion.  Navigate to Administration -> Federations , select your partner and click edit.

Click on "Attribute Mapping and Filters". This is what my example looks like:



Recall that the "zipcode" attribute was passed back from our web service and placed into the OIF session by forward.jsp.  This attribute is being made available to our partner in the SAML assertion where it will be called "zipcode".

If you have a user data store defined in OIF you may also pull additional attributes for the  user. The user id that authenticated will be correlated to a user in the data store. This is not a requirement (you can set the datastore to "None").

You are now ready to test your custom authentication engine using the Fedlet.  If you run into problems have a look at the OIF instance logs. Errors in your forward.jsp will usually show up as exceptions in this log.


Using the Fedlet with Oracle Identity Federation (OIF)




The Fedlet was first introduced in Sun's OpenSSO product. In a nutshell the Fedlet is a light weight SAML relying party implementation that is "pre-baked" with all of the right meta data needed to join a federation.

Typically a large Identity Provider creates a Fedlets for smaller partners that wish to leverage the providers identity infrastructure. These partners do not need to know the intricacies of SAML as the Fedlet is pre-configured. The Identity provider packages a Fedlet as a Java .war file or a .Net assembly that the partner can deploy.

The Fedlet has a simple API that is used to log the user in at the Identity provider and obtain information about the federated user that the provider has chosen to share.

As a prerequisite OIF must be installed, and the Fedlet distribution downloaded and unzipped.  The latest version of the Fedlet is 11.1.1.3 and can be downloaded from Oracle EDelivery (look under Fusion Middleware).

You also need a servlet container to deploy the sample fedlet war file. Any container with JSP support will do the trick (Tomcat, Glassfish, Weblogic, etc.).

The Fedlet Deployment Process

The high level steps to deploy a Fedlet are:
  1. Export OIF's IdP SAML meta data to a file. This meta-data is used by the Fedlet  generation script to parameterize the Fedlet (part of the "pre-baking" process").
  2. Run the Fedlet script to create a custom Java war file and additional meta-data. 
  3. The Fedlet script outputs the relying party meta-data to a file called sp.xml. This is the partner half of the federation. This meta data is imported into OIF to configure the partner link. You could do this manually - but importing the SP data will save time.
  4. Deploy the generated Fedlet war file to a servlet container
  5. Profit!

Getting Ready

  • Unzip the Fedlet distribution to a base directory (example ~/fedlet-install). The distribution contains templates for Java and .Net. 

Update OIF IdP Settings

Log in to your OIF EM Admin console (for example at  http://oif:7001/em).


Navigate to OIF -> Administration -> Identity Provider -> SAML 2.0
  • In “Assertion Settings” section: Set “Transient/One-Time Identifier” as default Name ID Format
  • In “Protocol Settings” check “Enable Attribute Query Responder” flag, and “Use Identity Federation for Attribute Response” flag.
  • In “Protocol Settings” section, “Messages to Send/Require Signed” table: 
  • Uncheck the flag “Require Signed” for “Requester- HTTP Redirect” 
  • Apply the changes.

Export OIF Idp Meta Data

You can export OIF meta in two ways - from the EM Console and from a meta data URL presented by OIF.  The fedlet configuration script shown later uses the URL. Below is the procedure for using the EM console:
  • Navigate to  OIF->Administration->Security and Trust
  • Select Identity Provider and SAML 2.0 
  • Press Generate

A screen shot is show below:




Run the Fedlet Configuration Script

The next step is to run the Fedlet configuration program to generate the custom war file.  The Fedlet distribution does not come with a shell script to invoke the program. Below is sample that you can modify for your environment (link for non JS browsers https://gist.github.com/1340523#file_configfedlet.sh)



The shell script demonstrates how to get the IdP meta data from OIF. The Fedlet configurator will prompt for the meta data location.

Importing SP meta data

The Fedlet configurator generates the Fedlet configuration files to the target directory you specified. The generated file sp.xml is the meta data for OIF that describes the partner relationship. Import this file into OIF using the OIF Enterprise Manager console:

  • Navigate to Admin -> Federations
  • Select the "+ Add" button
  • Select the "Browse" button and import the generated sp.xml file

Mapping Attributes

The Fedlet can pass along attributes from the IdP along with the SSO assertion. These are useful for the service provider to identify the user and offer personalized service.
  • Navigate to Admin->Federations and select your newly added partner, and click on “Edit”.
  • Under the tab “Oracle Identity Federation Settings”, Click on “Edit” next to “Attribute Mappings and Filters”
    • This  panel will allow you to add mappings from the IdPs attributes  (ldap attributes for the authenticated user) to attribute names to be passed to the partner. 
The following screen shot shows an example:




The above example shows passing on the ldap attribute "giveName" in the SAML assertion. It will be called "firstName" when the partner gets the assertion.

To complete the SP configuration you must perform the following steps:

  • Under “Enable Attributes in Single Sign=On(SSO)”, make sure “Transient/One-Time Identifier” is selected
  • Under “Protocol Settings”, “Messages to Send/Require Signed” table, enable the check mark under “Send Signed” for “Response with Assertion – SOAP”

Deploy and Test the Fedlet 

The generated war file is deployed to your container.  For example copy~/fedlet/fedletsample.war to ~/tomcat/webapps

When you bring up the fedlet page you should see this:



This is the sample Fedlet home page that is meant to serve as a starting point for your partner. Have a look through the various JSP files  to see the how the Fedlet API works.

If you click on any of the sample "Initiate SSO links" you will be redirected to OIF for authentication.

In this example we have OIF integrated with Oracle Access Manager (OAM) to perform the SSO session creation, so the login screen is from OAM:






Upon succesfull Federation you will be redirected back to the Fedlet.

The mapped attributes returned from the IdP are displayed:



Only the attributes mapped for this partner will be returned.

You can initiate Single Logout at the IdP by clicking on any of the "Logout" links at the bottom.


Trouble Shooting

The fedlet will log information to the "debug" directory under the fedlet configuration directory. Unless you override it, the Fedlet configuration directory is:

  • ${user.home}/fedlet 
So the log files will be in ~/fedlet/debug

The file FederationConfig.properties can be used to adjust the behavior of the Fedlet:
  • The property ${com.sun.identity.fedlet.home} sets the default config file location. 
  • You can also increase the logging verbosity to "message" by setting the com.iplanet.services.debug.level property.












A simple weblogic startup script for interactive use



The Oracle IAM suite requires starting quite a few admin and managed server instances.

During development you often want to watch the server output to diagnose errors.

Here is a sample script to launch an Admin server and the associated managed servers. The output for each server will go to a new gnome-terminal window.

You might want to create a master shell script that invokes this script for each of your instances.


Federation with Shibboleth SP (Apache module)

When you want to achieve Federation between different organizations, you often find yourself in a situation where the products used by the parties are different. Before we go any further I should make the terminology clear:
Identity Provider (IdP): The IdP holds all information about the user (for example in LDAP), and also it is the IdP’s job to authenticate the users, and decide on what kind of informations it shares about the users with other providers.
Service Provider (SP): The easiest way to think about the SP as an extra layer in front of the webapplication. The SP’s job is to authorize pagerequests, and if there is no authenticated session at the SP, initiate an authentication request to the IdP.

Today the goal is to achieve SSO between an OpenAM IdP and a Shibboleth SP with the simplest settings as possible.
This tutorial assumes that you already have a configured OpenAM instance running under idp.example.com.

Configure the IdP

  • Log in to the admin console and on the Common Tasks pane click on the Create Hosted Identity Provider link.
  • Select No for Do you have metadata for this provider
  • Use http://idp.example.com:8080/openam as Name
  • Select the default test Signing Key
  • Use cot as the name of the New Circle of Trust
  • Leave the Attribute mapping table empty
  • Press the Configure then the Finish button

Install and configure Shibboleth SP (on Ubuntu)

This tutorial was done with Ubuntu 11.04. If you have other OS/version it’s possible that the paths/steps will be different for you.

  • Install the Shibboleth SP Apache module:
    apt-get install libapache2-mod-shib2
  • Open the /etc/shibboleth/shibboleth2.xml configuration file using a text editor
  • In SPConfig -> InProcess -> ISAPI -> change the Site tag to:
    <Site id="1" name="sp.example.com"/>
    
  • In SPConfig -> RequestMapper -> RequestMap change the Host tag to:
    <Host name="sp.example.com" applicationId="sp.example.com" />
    
  • In SPConfig change the ApplicationDefaults opening tag to:
    <ApplicationDefaults id="default" policyId="default" 
    entityID="http://sp.example.com/shibboleth" 
    REMOTE_USER="eppn persistent-id targeted-id" 
    signing="false" encryption="false">
    
  • In SPConfig -> ApplicationDefaults -> Sessions change the /Login SessionInitiator‘s opening tag to:
    <SessionInitiator type="Chaining" Location="/Login" 
    isDefault="true" id="Intranet" relayState="cookie" 
    entityID="http://idp.example.com:8080/openam">
    
  • In SPConfig -> ApplicationDefaults -> MetadataProvider create a MetadataProvider:
    <MetadataProvider type="XML" file="idp.xml" />
    
  • In SPConfig -> ApplicationDefaults create an ApplicationOverride:
    <ApplicationOverride id="sp.example.com" 
    entityID="http://sp.example.com/shibboleth" /> 
  • Save the configuration file
  • Make sure that the files referred in SPConfig -> ApplicationDefaults -> CredentialResolver actually exist, and if necessary, generate a self signed certificate (using this guide for example).
  • Open http://idp.example.com:8080/openam/saml2/jsp/exportmetadata.jsp in your browser and save the XML as /etc/shibboleth/idp.xml (as configured in the MetadataProvider tag). (See this post for more information about exporting metadata.)
  • Open the /etc/shibboleth/attribute-map.xml config file and add the following line:
    <Attribute name="urn:oid:0.9.2342.19200300.100.1.1" id="HTTP_UID"/>
    

Prepare the Apache configuration

You need to configure Apache as well to make this setup work:

  • Make sure, that there is an sp.example.com VirtualHost in your Apache config
  • If you want to use .htaccess you need to enable the AllowOverride AuthConfig in the VirtualHost config
  • Create an index.php file in your docroot containing:
    <?php
    phpinfo();
    ?>
    
  • Create a .htaccess file in your docroot containing:
    AuthType shibboleth
    ShibRequireSession On
    require shibboleth
    
  • Enable the Apache Shibboleth module:
    a2enmod shib2
    
  • Restart Apache:
    /etc/init.d/apache2 restart
    

Registering the SP at the IdP

If you’ve done everything right so far, then you can access your Shibboleth SP Metadata at http://sp.example.com/Shibboleth.sso/Metadata. In case the Metadata does not contain a certificate check the logs at /var/log/shibboleth/shibd.log, also please remember that whenever you change the Shibboleth config you need to restart the Shibboleth service:

/etc/init.d/shibd restart

NOTE: OPENAM-792 can cause you troubles while importing the metadata. Make sure you either have the fix for this issue, or you have removed the Extensions tag from the Metadata before uploading it.

If everything is OK with your Metadata open the OpenAM admin console and click on the Register Remote Service Provider link on the Common Tasks pane.

  • You can either upload the Metadata or provide the URL on the config page.
  • Leave the attribute mapping empty
  • Click on the Configure button
  • Go to the Federation page then open the SP’s page in the Entity Providers table
  • Go to the Assertion Processing tab
  • In the Attribute Map list add the following value:
    urn:oasis:names:tc:SAML:2.0:attrname-format:uri|
    urn:oid:0.9.2342.19200300.100.1.1=uid
    
  • Save the configuration and log out

How to test

You just need to open a random URL under sp.example.com, and the htaccess config you created will make sure that the user is authenticated at the IdP. Opening such URLs should result in a redirect to the IdP presenting a login screen for you. After submitting the valid credentials you should be redirected back to the SP application to the originally requested URL. On the phpinfo page you should see the HTTP_UID server variable holding the user’s name.

In case you want to use the REMOTE_USER CGI variable in your applications, you can achieve that by modifying the Shibboleth configuration: in SPConfig -> ApplicationDefaults add HTTP_UID to the beginning of the REMOTE_USER attribute.