Security is often a forgotten concern in Big Data environments. However, as these technologies are being embraced by companies with sensitive data (think, for example, about banks or insurance companies), security is a growing requirement. In Stratio, we are aware of our clients’ needs, so we are studying the development of an integrated security solution for our platform.
We have chosen Apache Shiro as the main component of our future security solution, because it is a well-known and tested platform that (almost) fits our needs. Of course, we have performed some extensions for supporting specific requirements in our platform. First of all, we aim to provide our security API in a distributed and scalable way, so we have implemented an actor system based in Akka for managing the underlying Apache Shiro library. Also, as we expect to provide users with fine-grained permissions and due to the amount of elements expected (i.e., a huge numbers of users with different permissions on a huge numbers of tables), we also have implemented a distributed cache for improving the performance of the system. Those points will be treated in future posts.
Authentication and authorization
Shiro supports out-of-the-box multiple realm authentication and authorization. But we need to have the ability to provide that for each or our platform’s modules. We have implemented our own custom Shiro Realm supporting authentication and authorization, and aggregating an arbitrary number of realms (LDAP, JDBC, file-based realms, or custom realms) performing an authentication strategy over all of them and authorizing the platforms users. A remarkable point is that, with our current custom realm implementation, the aggregating realms can share specific authentication and authorization systems. For example, service1 and service2 realms can reuse the same LDAP for performing authentication and the same JDBC repository for permission checking.
Creating the custom StratioRealm
Since we want to support both authentication and authorization, our realm must extend Shiro class AuthorizingRealm (which also extends AuthenticatingRealm), and override the doGetAuthenticationInfo and doGetAuthorizationInfo methods, where we will perform our authentication and authorization operations. Our StratioRealm also includes a service name attribute and two lists of authenticating and authorizing realms.
This way, we achieve our original goal of supporting multiple realm security operations for each of our platform modules.
private String service; private List authenticatingRealms; private List authorizingRealms;
Within the overridden methods, we perform authentication and authorization against each one of the configured realms and return a merged result (principals, in the case of authentication; roles and its associated permissions in the case of authorization). We found one caveat here: the method performing authorization for specific authorization realms is protected, so we can’t call it directly inside our code. Our workaround is implementing a wrapper for AuthorizingRealm with the same package name, and exposing the desired methods as public there.
@Override
protected StratioAuthenticationInfo doGetAuthenticationInfo(AuthenticationToken authenticationToken) {
StratioAuthenticationInfo authInfo = null;
try {
for (AuthenticatingRealm realm: authenticatingRealms) {
AuthenticationInfo auth = realm.getAuthenticationInfo(authenticationToken);
if (MergableAuthenticationInfo.class.isInstance(auth)) {
if (authInfo == null) {
authInfo = new StratioAuthenticationInfo(auth);
} else {
authInfo.merge(auth);
}
} else {
throw new AccountException("Impossible to merge AuthenticationInfo");
}
}
authInfo.setMainRealm(this);
return authInfo;
} catch (AuthenticationException e) {
throw new AuthenticationException(e);
}
}@Override
protected AuthorizationInfo doGetAuthorizationInfo(PrincipalCollection principalCollection) {
SimpleAuthorizationInfo authInfo = null;
try {
for (Realm realm: authorizingRealms) {
AuthorizingRealmWrapper authorizingRealmWrapper = new AuthorizingRealmWrapper((AuthorizingRealm) realm);
SimpleAuthorizationInfo auth = (SimpleAuthorizationInfo) authorizingRealmWrapper.doGetAuthorizationInfo(principalCollection);
if (authInfo == null) {
authInfo = auth;
} else {
if (authInfo.getRoles() != null) {
authInfo.addRoles(auth.getRoles());
}
if (authInfo.getStringPermissions() != null) {
authInfo.addStringPermissions(auth.getStringPermissions());
}
if (authInfo.getObjectPermissions() != null) {
authInfo.addObjectPermissions(auth.getObjectPermissions());
}
}
}
return authInfo;
} catch (Exception e) {
throw new AuthorizationException(e);
}
}
Implementing an AuthenticationStrategy
We have introduced a new parameter (the service/module name) in the authentication process. Thus, we have to create a specific authentication strategy to take this into account. As we did with the previous StratioRealm, we must extends a Shiro class (AbstractAuthenticationStrategy) and implement and override its methods. In this case, we expect to receive an authentication token containing an username, a password, and a service name. We have included an additional condition: that service name must match the service name configured in the realm we want to authenticate against. Otherwise, we throw an AuthenticationException and the user is not allowed to use the system. Also, with our current implementation, we enforce the user authentication in every defined subrealm, throwing an error otherwise, but this behaviour can be also customized.
@Override
public AuthenticationInfo afterAllAttempts(AuthenticationToken token, AuthenticationInfo aggregate) {
if (aggregate == null || CollectionUtils.isEmpty(aggregate.getPrincipals()) || !matchServiceRealm(aggregate,
token)) {
throw new AuthenticationException("Authentication token of type [" + token.getClass() + "] " +
"could not be authenticated by any configured realms. Please ensure that at least one realm can " +
"authenticate these tokens.");
}
return aggregate;
}
private boolean matchServiceRealm(AuthenticationInfo aggregate, AuthenticationToken token) {
UsernamePasswordServiceToken serviceToken = (UsernamePasswordServiceToken) token;
StratioAuthenticationInfo auth = (StratioAuthenticationInfo) aggregate;
return auth.getMainRealm().getService().equals(serviceToken.getService());
}
Sample configuration
The last step for having our custom solution working is specifying the configuration we want Shiro to use for our set of realms. A simple example of Shiro configuration file:
# Define JDBC datasources
ds = com.mysql.jdbc.jdbc2.optional.MysqlDataSource
ds.serverName = server1
ds.user = stratio
ds.password = pwd1
ds.databaseName = authdb
ds2 = com.mysql.jdbc.jdbc2.optional.MysqlDataSource
ds2.serverName = server2
ds2.user = stratio
ds2.password = pwd2
ds2.databaseName = authdb2
# Define the authenticating realms
userLdapRealm = org.apache.shiro.realm.ldap.JndiLdapRealm
userLdapRealm.userDnTemplate = uid={0},ou=users,dc=stratio,dc=com
userLdapRealm.contextFactory.url = ldap://ldap1
deepLdapRealm = org.apache.shiro.realm.ldap.JndiLdapRealm
deepLdapRealm.userDnTemplate = uid={0},ou=deep,dc=stratio,dc=com
deepLdapRealm.contextFactory.url = ldap://ldap2
crossdataLdapRealm = org.apache.shiro.realm.ldap.JndiLdapRealm
crossdataLdapRealm.userDnTemplate = uid={0},ou=admins,dc=stratio,dc=com
crossdataLdapRealm.contextFactory.url = ldap://ldap3
# Define the authorizing realms
jdbcRealm=org.apache.shiro.realm.jdbc.JdbcRealm
jdbcRealm.permissionsLookupEnabled=true
jdbcRealm.dataSource = $ds
jdbcRealm.userRolesQuery = SELECT role.shortcut FROM auth LEFT JOIN auth_role ON auth_role.auth_id = auth.id LEFT JOIN role ON role.id = auth_role.role_id WHERE auth.name = ?
jdbcRealm.permissionsQuery = SELECT permission.shortcut FROM role JOIN role_permission ON role_permission.role_id = role.id JOIN permission ON permission.id = role_permission.permission_id WHERE role.shortcut = ?
jdbcRealm2=org.apache.shiro.realm.jdbc.JdbcRealm
jdbcRealm2.permissionsLookupEnabled=true
jdbcRealm2.dataSource = $ds2
jdbcRealm2.authenticationQuery = SELECT name FROM auth WHERE name = ?
jdbcRealm2.userRolesQuery = SELECT role.shortcut FROM auth LEFT JOIN auth_role ON auth_role.auth_id = auth.id LEFT JOIN role ON role.id = auth_role.role_id WHERE auth.name = ?
jdbcRealm2.permissionsQuery = SELECT permission.shortcut FROM role JOIN role_permission ON role_permission.role_id = role.id JOIN permission ON permission.id = role_permission.permission_id WHERE role.shortcut = ?
# Define a realm for Stratio Deep Module with two authenticating realms and two authorizing realms.
deepRealm = com.stratio.datagov.security.authc.realm.StratioRealm
deepRealm.service = deep
deepRealm.authenticatingRealms = $deepLdapRealm, $userLdapRealm
deepRealm.authorizingRealms = $jdbcRealm, $jdbcRealm2
# Define a realm for Stratio Crossdata Module with two authenticating realms and an authorizing realm.
crossdataRealm = com.stratio.datagov.security.authc.realm.StratioRealm
crossdataRealm.service = crossdata
crossdataRealm.authenticatingRealms = $crossdataLdapRealm, $userLdapRealm
crossdataRealm.authorizingRealms = $jdbcRealm
# Configure the custom realms into Shiro’s Security Manager
securityManager.realms= $adminRealm, $crossdataRealm
#Configure the custom authentication strategy
authcStrategy = com.stratio.datagov.security.authc.pam.CustomAuthenticationStrategy
securityManager.authenticator.authenticationStrategy = $authcStrategy
Conclusions and future work
Some conclusions extracted from this post:
- Security has a growing value for Big Data systems.
- There are several interesting open source projects that might be used for securizing our systems.
- It is more than likely that you should extend some points of your tool of choice. We have done it with our solution adding actor support and customizing the authentication and authorization processes.
- We expect to develop an integrated user management solution, with capabilities for quarantining users, expiring sessions and fully configuring the authentication and authorization realms.
- Another step is developing a solution for auditing every user operation inside the platform.
We look forward to reading your questions and suggestions. Feel free to comment!
