Page 13 – Radiant Logic | Unify, Observe, and Act on ALL Identity Data

A historic stone bridge with three arches spans a rocky river, set against a backdrop of green hills and jagged mountains under a vibrant blue sky with scattered clouds.

From SQL to LDAP/Graph: Bridging the Two Worlds

March 15, 2013/in Blog Michel Prompt/by Josue Ochoa

I’ve been blogging on Gartner analyst Ian Glazer’s recent video on whether we need to kill IdM in order to save it. It seems appropriate to begin these with words of Martin Kuppinger, in response to the Glazer video and my first post on this topic:

…. I believe in approaches that build on existing investments. IAM has to change, no doubt about that. But there will still be a lot of “old school” IAM together with the “new school” parts. Time and time again it has been proven that change without a migration path is an invitation to disaster. Embrace and extend is the classical migration methodology for classical technical transformative strategies.

Anyone who’s been following along understands that Martin echoes my perspective here—I am all about taking what still works and making it work better. So before we dive into the global picture about what is dysfunctional with current IdM deployments amid the clamor for access (SSO and cloud), access rights (authorization), and governance, I want to share some final thoughts on identity representation and storage.

SQL and LDAP: Strengths and Weaknesses

As I mentioned in my most recent post, if we’re focusing only on finding the most flexible, non-redundant, easy-to-maintain system to represent and record identity (and information in general), it’s tough to beat the relational model. And as a result, the majority of our applications are based on SQL, which means the largest chunk of key identity information is also managed by SQL.

While it’s one thing to accurately capture information, however, querying and retrieving this data is another story. Fast lookups—especially those involving some form of contextual navigation and link-hopping—require a lot of “joins.” And joins are a very slow and expensive operation with SQL, because each join means a “multiplication of links” between objects. So after two or three hops, you have exponential growth of your database operations—and even on the fastest computer in the world, an open and potentially infinite number of operations will take forever.

The key problem with navigating hierarchies or graphs in SQL is that the paths are generated on the fly—and this is precisely where hierarchical or a graph database is strongest. In such tools, the links are hard coded and stored. Because the paths are pre-defined through indexation, you can always pick your starting point and navigate quickly, thanks to these pre-established paths.

And here we have the dilemma: on one hand, we have a system that reflects reality in a faithful, non-redundant way, not only capturing the information about a given object, but also reflecting the relationships around that object. And it can also generate all the paths to and from that object, whether in a graph or a tree. But if there are a lot of relationships—a lot of context surrounding that object—those queries will be slow and, in some cases, with a response time that’s difficult to determine. And that doesn’t work in a world where users expect immediate access to resources.

Or you have a system that’s easy to navigate, once you get the data in—but far from optimal when it comes to write and update. Redundancy is one big challenge, and being able to flexibly modify the structure—deleting or updating a link—is another.

What If: Imagining a World without LDAP

I know LDAP is no longer fashionable: Ian is leading the charge on that, and on many points he is right. LDAP is a kind of lost art; for the younger web 2.0 generation—the JSON/REST crowd—anything that is not based on a browser and HTTP is like Cobol (aka, the dinosaurs). Of course, you have to be careful with fashion and trends, because there’s always a next generation. For instance, the Apps/APIs IOS/Android crowd looks at HTTP and HTML 5.0 as “designed by committee,“ and uncool, to boot.

Let’s imagine for a second, though, that we killed all those LDAP servers and their derivatives. First, many of our portals would stop authenticating and authorizing, and that’s only considering the external users—the “Sun” side of the market. If we ended Active Directory, as well, many of our SharePoint portals would stop in their tracks. I cannot even imagine the state of our internal networks, our enterprise intranets.

And finally there is a lowly function that we cannot just ignore: the email address. Yes, those weird conglomerations that often act as a proxy to your unique identification, your “login.” Most email servers are using LDAP or some form of LDAP for email address lookup. To some extent, the problems with LDAP are a bit like the problems we have with DNS (yet another tricky naming service…). We hate them, but we cannot just kill them—or not that easily. As Steve Jobs used to say, you only know the value of something once you lose it.

Embrace and Extend: The Identity Bridge to Somewhere

As Martin says above, I believe it’s better to embrace and extend, to evolve. So let’s look at how we can use two of the mainstays of our identity infrastructure—SQL and LDAP—and give them new life by bridging them through virtualization and federation. Let SQL record and capture identity and information as it does behind our multiple applications and then “publish” that data in a specialized identity look-up and attribute-oriented store, such as our old friend LDAP. (By the way, this virtualization layer approach would also work going from SQL to graph, if graph databases suddenly replaced LDAP.)

Such a system offers the best of both worlds. Identity management, or the identity representation and storage side, at least, would be reconciled with the rest of the infrastructure: “a part of” and not just “apart,” as Ian said. Before we dive into the analysis of how such a SQL-to-LDAP bridge could be built, I would like to reiterate that this approach— what we call “model-driven virtualization”—is not applicable only to the pair SQL/LDAP. It also works going from “N variation of LDAP” to “canonic LDAP” (such as AD-to-LDAP or Novell-to-LDAP), as well as from application APIs or web services mapped to LDAP (WS-to-LDAP or API-to-LDAP).

So think about the approach I am describing as a typical building block. The idea is to find a systematic approach to building identity bridges, so that we stop building a bridge to nowhere, as in this diagram of a typical fragmented identity infrastructure:

And end up with a rational federated view of identity acting as a hub, as illustrated by this diagram:

This is what we at Radiant call a “federated identity service based on virtualization,” and I look forward to diving into what this means and how it works in my next post…stay tuned!

A worker wearing safety gear climbs a tall communications tower high above a forest, surrounded by large antennas and equipment, with trees visible far below.

The Best of Both from SQL and LDAP

March 7, 2013/in Blog Michel Prompt/by Josue Ochoa

Last week, we discussed Ian Glazer’s video on killing identity in order to save it. And this week I want to follow up on some of the questions I raised at the end of my post:

What makes hierarchies, along with graphs, so important (think contextual relationships).
How SQL databases can capture information without redundancy, but still need a materialization layer (think cache) to deliver contextual views at the speeds required for security and context-aware applications (think mobile).
Why virtualizing both identity and context is key to breathing new life into IdM, while respecting your current identity investments (think Radiant).

These can be regrouped into two topics: The first is identity representation and how we should record it, which is a problem of structure and storage. What is the ideal directory structure and storage for identity and how could we better integrate this life cycle with the larger field of data management? Or, as Ian stresses, how could IdM be a part of IT, instead of apart from it? The second is architecture and real life deployment or how in practice we could and should manage the identity lifecycle in a world of silos. I’d argue that the current fragmented identity infrastructure landscape requires a more rational solution, one that would involve what we at Radiant call a federated identity service, based around a virtualization layer for identity and context.

Today, we’ll be considering the structure and storage side of this equation: how to build the most modern, flexible, and future-oriented way of representing, storing, and sharing essential identity information.

The Rise (and Fall) of the Comma: On SQL, LDAP, and Graphs

As I mentioned in my previous post, although Ian wants to “kill the comma”—and don’t get me wrong, I agree with most of his critiques of the system—I believe there’s more life left in LDAP. To make my case, let’s take a trip back in time, because those who do not know their history are doomed to repeat yesterday’s mistakes—and miss yesterday’s lessons.

Now, we all know that in terms of programming languages, there’s been much evolution since the sixties/seventies, including C, C++, logic programming, functional programming, Java, aspects programming, Erlang, Scala, and many more, with each language offering its own strengths, specializations, and limitations. But we have not seen the same exuberant dynamism in the world of databases. In the earliest days, the hierarchical systems emerged, then some forms of the network database, but from the seventies on, SQL has dominated the landscape. Sure, there were some attempts at object-oriented databases in the late eighties and early nineties, but these were specialized niche products that never cracked the mainstream.

Given this history, we know that around the year 2000 or so, SQL remained the workhorse, the predominant database system in enterprises everywhere. So why did we see the emergence of LDAP, itself a simplification of x500, a kind of “object-oriented” hierarchical database? Why did a bunch of very smart people sitting on a standards committee commissioned by many organizations—including the largest telcos in the world—bypass SQL standards to implement a look-up mechanism based on a hard-coded hierarchy? Why did they create a distributed hierarchical database for email addresses and other information about people (including pictures and preferences—hello, Facebook!), which later evolved into a repository for PKI and source of authentication for internal networks?

Could it be that the hierarchy had some relevance that they weren’t getting from the relational model alone? I believe the much-maligned comma, or rather the DNs (and don’t you love the weird jargon from the X500 world?) still has something to tell us, even now, as identity environments grow increasingly fragmented and demand is going mobile and reaching into the cloud. So let’s look at why, where, and how LDAP is still very relevant for today’s identity infrastructures. But first we need a detour through SQL.

A Thought Experiment: Trees Inside of Tables

Imagine that we could implement a directory model—a complete LDAP schema and directory tree—in a set of SQL tables following best data management practices. Such an implementation would offer several major benefits:

The information kept in the system would be extremely flexible. You could modify and generate as many directory trees as you need from your LDAP schema. You could also generate all the graphs you need from these schemas, including graphs containing loops, which are no longer strictly trees.
The information in your system would be non-redundant and easy to maintain, following the normalization rules of the relational model. This is a key advantage of the relational system and one of the biggest weaknesses of the hierarchical and graph/network models.
Your identity would be managed like all other essential business data. Basing your identity infrastructure on SQL technology also means that your IdM is now “a part” of classic data management, and your DB team is working in their most comfortable environment.

Given these known advantages of SQL, why did our committee from above decide to implement the X500 data model (and hence LDAP) as a specialized hierarchical “pseudo object-oriented“ database? Because of one of the prime advantages of the hierarchical model: fast look-up (or “queries”) and fast attribute value-based search. (And here’s where the link with Hadoop, HDFS, and other big data “NO SQL” implementations will become apparent for the advanced reader.)

The Dark Secret of SQL: It Takes Forever to Navigate Complex Hierarchies and Graphs

You see, the best system to record and update data supporting the so-called ACID properties is also terrible at answering very complex queries. The beauty of the relational model is that it can capture any relationship (whether in graphs or trees) and organize data so it’s captured only once and the updates are consistent, with any change in the real world reflected faithfully in the database. Once recorded in SQL, any graph, hierarchy, or relation between entities and data can be re-generated. As a result, there is no question that the relational system cannot answer. But there is also an important hidden cost: While you always get an answer to your SQL query, receiving that answer can take an incredibly long time. Isn’t it ironic that the world’s leading provider of SQL databases is named “Oracle”? Because, just as in Delphi, when it comes to graphs, hierarchies, or rich contextual data, your questions will always be answered—at some stage, when the fates are ready to tell you.

Pictures Tell the Story: Graph/Trees as a Recursive Relation

Still not convinced? Let’s take a look at how we might represent the “graph” of a person and all his or her friends in the relational world. In the following diagrams, we can see the flow of the representation from the most abstract entity level into a complete instantiation at the network or record level.

At the entity (or model) level, we see a recursive relationship in a relational store, where the recursive “friend of” link loops on the person structure. This very abstract and elegant model can generate both graphs and trees, proving my point that “a relational database can be the mother of all trees and graphs.” One way it does that is representing a tree or a graph by a recursive relationship, where the entity loops on itself.

Implemented at the table level, we see that each row is a person. We can use this table to anchor and encode all the relationships between people. (Of course, we didn’t expand all the duplicate rows for the purpose of illustration.)

At the relational level, when you navigate those rows from picture 2, we see how we can generate an infinite number of trees and graphs from a relational table. This is a tremendous capability—and one that we could and should leverage in the IdM of tomorrow. But, as I mentioned, it comes at quite a cost. While the relational model can represent any tree or graph you want, every time you have to navigate those links, you are joining across nodes—and that’s a very expensive operation. So we can represent any tree by a relationship, but while there’s no problem recording that tree, navigating through the tree will be very slow. That’s just a fact of SQL: you can record and update everything in a really robust system, but queries take forever to navigate. In fact, when in comes to context and graph navigation, “SQL” stands for “Slow Query Language.” (Luckily, there’s a way around the slow-down—stay tuned for my next post.)

And finally, at the network level, we see that in a complex table with lots of people and links, you’ll get to an image similar to this one, which I borrowed from Mark Dixon’s excellent response to Ian’s video. So you can imagine the performance in a complex network such as Facebook.

Everything Old is New Again: What About the Graph Database?

Okay, so we see how SQL can deliver all the trees and graphs we need—but at a snail’s pace. So should we try some thing new? Ian raises the issue of a specialized hierarchical or graph database that would be optimized for faster navigation/search. So let’s say we kill hierarchies and replace them with graphs, for a full network, which is a lot richer. So now we have fast access, but it comes with two huge problems:

Trading one specialized storage for another: Implementing a new array of graph databases means we still have a specialized storage that sits apart from the SQL mainstream, which is the source of reference for most of your identity data anyway. Basically, we just replace the LDAP ghetto for a graph ghetto.
Reconciling the system across multiple nodes: A graph database does not follow the normalization rules of the relational model, which make it simple to maintain a single version of the truth. While graph databases are very fast to navigate, it doesn’t take advanced math to see that information updates are hell in a network with umpteen nodes like the one we see in picture 4, where duplicate users may play many different roles. Imagine trying to maintain consistency when a person’s phone number changes and that update must be made at all n aliases across the network. The clean model of SQL, where each entity is only recorded once, starts to look pretty good, right?

So do we rely on the old workhouse SQL, a flexible and well-understood “update and record“ engine that’s mainstream for most of your applications? One that contains much of the information we need about identities and their diverse contextual attributes, but which is slow at tying them together in a contextual way? Or do we invest in a new set of specialized databases supporting hierarchies and graphs that are fast at queries (just like LDAP!), but not great at write and update—and devoid of all the information that’s been captured by your mainstream applications?

In my third post in response to the Glazer video, we’ll look at the practice side of this equation: how you can make identity work, given legacy investments and growing demands. I’ll focus on the fact that our existing infrastructure already has the two players we need—SQL and LDAP—and show what other player is needed (think cache-enabled virtualization) to make them work together to deliver all the trees, graphs, and—wait for it—context we could ever need.

White, raised, curving ridges form a stylized, abstract fingerprint pattern on a smooth, light gray background. The image has a clean, modern, and minimalistic look.

Why Kill Identity Management When You Can Virtualize It Instead?

February 20, 2013/in Blog Michel Prompt/by Josue Ochoa

Our good friend Ian Glazer from Gartner is out with an excellent but provocative new video, where he proposes that we need to kill identity in order to save it. Now, I prefer evolution to revolution, but this video should spark a conversation that’s long overdue—about how yesterday’s identity infrastructure can meet the demands of today’s Internet, where we’re all a hundred different “personas” on a thousand different sites. Where the forces of access, security, and privacy are all dueling it out (and privacy is losing). Where the world is smaller than ever before, but also overflowing with information, opportunity, and—yes—risk.

There’s a great discussion of this new reality in last week’s The Economist, about who’s going to profit from online identity, and how acting as a certified, trusted identity provider is a key function that should not be left to government entities or loosely managed by Facebook or other new social networking “properties.” But today, I want to focus on Ian’s battle cry. In this video, he has come down the mountain with tablets for a new identity, ready to wage war against the “comma.” It’s time to replace the existing identity management—based on LDAP—with REST, JSON, OpenID Connect, OAuth 2.0 and SCIM, he tells us. IdM is all over but the shouting.

Now, don’t get me wrong. I welcome those standards—in fact, our latest version of RadiantOne supports SCIM and a REST interface to our federated identity system, while our CFS service supports OpenID Connect and OAuth 2.0. And I totally buy into Ian’s analysis about the core issues facing our current approach to identity and access management. But I don’t think we need to sacrifice anything.

What Ian Gets Right: The Issues Facing IdM Today

Ian begins with the notion that today’s identity is static, which no longer works when everything is interconnected. We must be able to mirror the relationships, the graphs, the networks that are driving everything around us, he says. And he’s absolutely right! I love how he looks at the current provisioning life cycle, saying that it doesn’t move as quickly as our partners or their customers—or even our own employees—which is really the crux of the challenge. To make identity a competitive advantage instead of an IT cost, it must enable you to collaborate across and outside the corporate walls, and to expand your customer base beyond your current channels. And true, this is a huge challenge, given current constraints. But then Ian jumps to some observations, recommendations, and conclusions that strike me as controversial.

Where He Goes Wrong: Is the Remedy Worse than the Disease?

According to Ian, comma-separated values (CSV) are too simplistic a mechanism for data representation and data transfer in today’s web-driven world—and LDAP is way too inflexible with its hierarchical infrastructure. Instead, we need a graph, a full network of the relationships between nodes/entities. The solution, he says, are new standards, such as OAuth 2.0 and REST. And we need to kill our current IdM systems to get there.

We could not agree more with Ian’s analysis of the state of IdM—but we disagree on the solution. I’ll be looking at why over the rest of this post, then developing these ideas in greater depth over the next few weeks. (I’ll even give a defense of the poor old comma, which gets quite beaten up through the course of Ian’s video.)

Meet the New Standards: Same as the Old Standards?

Like any IdM veteran, I understand how static the current system is, with its jumble of application silos and data stores, where any changes take months of your life, big bites of your budget—and often hairs off your head. You need a system where the definition of identity attributes can evolve depending on context. Do I like the definition of an identity OAuth 2.0 style? Sure, as much as I like LDAP’s “InetOrgperson” in some use cases or even Microsoft AD’s definition of a “User” in other circumstances. There is no ultimate definition of identity—it all depends on needs and requirements. Here at Radiant, we’re pretty agnostic about how identities are described. We like them all, in the same way we’ll like whatever’s part of OAuth 3.0, or any other future standard.

The only certainty is that the definition of identity will evolve as we develop new standards to solve new challenges—and if our definition of identity itself evolves, why not our identity management system? That’s why our federated identity service is based on virtualization, so you can define, remap, and deliver any view of identity you need.

As for REST, we love it and support it in our upcoming release of RadiantOne version 6.1 (look for it in March!). But we also love SOAP when and where it’s not smart, secure, or scalable to use HTTP as a transport protocol. And we even love plain old RPC, when it’s the only mechanism supported in some outmoded, legacy system that’s currently deployed. Like I said, we always try to work with what we have to, to get where we need to be—and that’s a big part of our evolutionary strategy based on virtualization.

Hierarchies, Networks, and Relations: Looking to History to Lead Us Forward

Ian bangs the anti-hierarchical drum in his video, claiming that the hierarchies of LDAP are insufficient and incomplete. A graph is better, he says, because a network of relationships is the most faithful representation of reality. And sure, that’s totally true… But I do think this raises a key issue: as it stands right now, IdM standards (and LDAP as a hierarchical repository) stand apart from the rest of IT, and especially data management. After all, LDAP is not SQL.

However, Ian is short on details here. If he does not want hierarchies, is he thinking about some sort of graph or network database to store those multiple identity relationships? If this is the case, his thought about a graph database—a database that captures all the links of a full network of relationships—should give a big jolt to any relational database practitioner. Did we not settle this debate 40 years ago between SQL and Codasyl and other IMS hierarchical databases? I thought that the first lesson in Database 101 is about how a relation (hence, a relational database system) can represent any graph and, as a consequence, any hierarchy? So we all know that SQL can generate all possible hierarchies and networks. And this is the main reason why the relational database and SQL won the war and now dominates data management as a key mechanism to capture and store application information.

But why then, around the end of the 1990s and beginning of 2000, did we see this resurgence of the hierarchical database model in the X500, which finally yielded LDAP? And why did we in the IT realm—who were fully aware of the limitations of hierarchy versus relational—accept this move, which has led to the current IdM situation? Could there be something useful about hierarchies that we were hoping to capture back then? Couldn’t that value still exist, even in our siloed, static identity environments?

Even in our IdM-isolated world, we should not ignore the growing momentum for attribute and keyword search on all kinds of “Big Data” stored in systems such as Hadoop and other “NoSQL” databases, nor the advantages these technologies might bring. What we need is a good way to unlock all those advantages.

Unless we analyze our past clearly, we are condemned to repeat the same mistakes in the future. So I think it’s essential to step back and look at:

What makes hierarchies, along with graphs, are so important (think contextual relationships);
How SQL databases can capture information without redundancy, but still need a materialization layer (think cache) to deliver contextual views at the speeds required for security and context-aware applications (think mobile); and
Why virtualizing both identity and context is key to breathing new life into IdM, while respecting your current identity investments (think Radiant).

I told you I was about evolution, not revolution. Now, I’ve addressed some of these topics before, but I want to dig a little deeper in coming posts—stay tuned!

And thanks, Ian, for your thought-provoking call to arms!

Aerial view of two parallel highways over a partially frozen river, with one truck on each road, surrounded by snow-covered ground.

RadiantOne VDS 6 Launches!

July 13, 2012/in Blog Michel Prompt/by Josue Ochoa

We are proud to announce that VDS 6, the newest version of our flagship RadiantOne Virtual Directory Server product, is now shipping!

VDS 6 is breaking new ground in identity management, becoming the world’s first true federated identity service. By virtualizing all of an organization’s identity stores, modeling their structures, and joining those silos, VDS 6 creates a logical and unified view of your identity data that every application can access securely. Finally, enterprises large and small can harness the power of single sign-on (SSO) to federate their cloud services without painful compromises to security and flexibility, and without costly investments in new infrastructure.

What’s new in version 6?

Wizard-driven workflow manages your identity system and groups: We’ve overhauled our entire user experience, making simple point-and-click activities of complex tasks, such as:
Aggregating identity sources and correlating identities
Configuring new auto-generating user groups and remapping old ones
Extending existing data hierarchies with data from other data sources
Flexible caching options ensure data reliability: Whatever the priority–mission-critical event-driven instant updates, or low-impact periodic refreshes–VDS 6’s caching will keep your directory up-to-date and ready to meet your demands
Web-based control panel simplifies administration: Everything you need to monitor and manage your server has been consolidated into one neat browser window; now you can manage VDS 6 from anywhere–including while you’re on the go, from your mobile device

VDS 6 works seamlessly with RadiantOne’s new Cloud Federation Service, forming a fully federated identity hub that bridges every data source and application your enterprise employs, both locally and in the cloud. The result: an end-to-end federation solution, where every user across your organization can enjoy an easy, secure SSO experience, with the very minimum in administration and configuration overhead.

Already working with another federation solution? VDS 6 integrates easily with your identity provider to bring all the added security, stability, and flexibility benefits of a federated identity service to your organization.

We’re proud of the work that has gone into making VDS 6 such a powerful piece of technology, and we want to take system design off your identity management to-do list for good.