There are many good reasons to use an open source-based LDAP server. First, obviously, is cost. But features and scalability also play an important part in the decision to use an LDAP directory as the identity storage of choice. As an authentication source, LDAP directories have a number of advantages over SQL. For example, its protocol is optimized to read and search attributes more quickly, scale more efficiently, and install more affordably. By virtue of its simple hierarchical structure, LDAP is easily partitioned into distinct branches that can run in parallel on commodity servers, or replicated to run on redundant distributed servers.

This simplicity and flexibility means the “scale out” story of LDAP is pretty good—even without special optimizations or a brand name on the box. At Radiant, we believe the advantages of proprietary LDAP servers over open source are oversold. The best-optimized of these so-called “carrier class” proprietary LDAP servers yield an improvement of no more than 30% in performance; let’s be generous and imagine a heavily-tweaked system that delivers 50% better performance over a plain vanilla open source implementation. That may sound impressive, and in some sense it is, but it’s also fighting a battle that is already lost. Why? Because this optimization approach is based on a “scale up” strategy, where the extra speed comes from endless tweaking in pursuit of incremental improvements to the code.

Scaling Out, Not Up: Getting to the Cloud Quickly and Cheaply

However smart your code may be, it is ultimately limited by your server’s processor speed. This dependency on high-end hardware guarantees regular costly investments in and migrations onto new infrastructure, and totally fails to take advantage of one of the most powerful transformations in IT: the move to the cloud. The hallmark of this trend is the use of many commodity servers in coordination, where each node is optimized to be powerful enough, while remaining as cheap as possible. In short, the increasingly dominant strategy is to create scalability by scaling out (increasing the number of “good enough” nodes) instead of scaling up (investing heavily in a single high-performance node). This strategy is a perfect fit for LDAP, which, by its efficient support for partitioning, scales beautifully on distributed systems.

We’ve seen this approach taken in other areas of data management with architectures such as Hadoop. Buy a bunch of simple, robust nodes, and you can achieve nearly limitless scalability, delivering “galactic class” performance in a rented cloud with only a fraction of the cost of overly-tweaked software and high-end hardware.

Using a cheap, but proven and reliable implementation is the key to this approach; fortunately, the LDAP server is such a well-known technology that using a vanilla open source implementation makes perfect sense. For this reason, the cost/benefit tradeoff favors the open source LDAP solutions over either heavily customized LDAPs, or SQL-based implementations. That’s why we use open source LDAP technology at the core of our own caching.

What’s the catch? Why do many businesses, especially Internet-based ones, miss out on these advantages? In large part, it’s because familiarity and inertia have kept them relying on SQL to store their identity data, despite its slower query response time and more fragile scalability.

There’s Always a Bigger Picture

LDAP does have its limitations, though. The format can be inflexible; once the schema is designed, implementing changes to the hierarchy is a huge challenge. Important tasks like populating, updating, and remapping data can be time-consuming, and synchronizing other data sources with LDAP is a headache.

This is where a directory virtualization layer—and the federated identity system it enables—really shines. The virtualization layer allows you to remap and manipulate the underlying data sources at will, designing the perfect LDAP structure that each application needs. The virtualization layer then generates the logic to populate and synchronize your source with the open source directory LDAP to become a simple, easily replicated and distributed identity storage resource.

A federated identity system based on virtualization—like our VDS—sits above all your diverse data stores, regardless of format or schema, and pulls them into a coherent whole. With a smart caching system and seamless joining of data from across silos, a virtualization package can augment your data sources—open source or proprietary, LDAP, AD, SQL, or otherwise—and unify them into the ideal authentication and authorization attribute server.