You are not signed in. or register.

OpenSolaris Project: HoneyComb Fixed Content Storage

View the leaders for this project
Project Observers

Endorsing communities

Storage

SPOTLIGHT

The ST5800 Storage Plugin for Fedora Commons allows for the use of the Sun StorageTek 5800 as the underlying back end storage for the Fedora Commons Open Source Repository.

StorageTek 5800 Open Edition Now Available! (Licensed under the BSD License) is now available for download.This includes the prototype XAM capability as well.

The ST5800 1.1 SDK is available for download too.

OVERVIEW

The Honeycomb project will store and manage large amounts of fixed content - data that never changes over the course of its life (YouTube videos, x-rays, digital books, etc.). Recent studies have shown that 80% of the data that's created today will not be modified during its lifetime. And the amount of fixed content is predicted to accelerate growing at a 90% increase through 2010 while mutable (updated) data will grow at a 60% increase (Hal Varian, UC Berkeley).

This project proposes to manage this explosion of data by providing an object-oriented storage system with traditional Java and C interfaces later expanded by a "StorageBean" Java interface. Large data-repository applications access the fixed content through these interfaces which are designed to manage data collections that can total up to 100 million objects or petabytes of storage.

Goals of the project

The goal of this OpenSolaris project is to provide client and server implementations for the Honeycomb fixed content system. Although there are some existing Honeycomb C and Java APIs, we anticipate that OpenSolaris and other open source communities will help shape and define these APIs. We also want to encourage OpenSolaris members to contribute their ideas and code to this project. You can join the conversation over on the OpenSolaris storage discussion list.

Another goal of this project is to add the standard XAM fixed content interface to both HoneyComb and Solaris itself.

Related open source communities

There are a number of open source large data-repository applications used by health care, scientific, and educational communities that need a fixed content storage system. These open source communities include:

Fedora Commons

Fedora open source software gives organizations a flexible service-oriented architecture for managing and delivering their digital content. At its core is a powerful digital object model that supports multiple views of each digital object and the relationships among digital objects. Digital objects can encapsulate locally-managed content or make reference to remote content. Dynamic views are possible by associating web services with objects. Digital objects exist within a repository architecture that supports a variety of management functions. All functions of Fedora, both at the object and repository level, are exposed as web services. These functions can be protected with fine-grained access control policies. Some examples of applications that are built upon Fedora include library collections management, multimedia authoring systems, archival repositories, institutional repositories, and digital libraries for education.

Dspace.org

The DSpace digital repository system captures, stores, indexes, preserves, and distributes digital research material. Research institutions use DSpace as an institutional repository, a learning object repository, for records management, and more.

EPrints for Digital Repositories

Open source EPrints software provides a Web-based Institutional Repository and is established as the easiest and fastest way to set up repositories of open access research literature, scientific data, theses, reports and multimedia. We are also creating the environment in which Open Access will become the norm for distributing research.

Storage standard efforts

There is a storage industry standards effort to create a common access and management interface for fixed-content systems.

Storage Networking Industry Association XAM Initiative

The XAM (eXtensible Access Method) Interface specification defines a standard access method (API) between "Consumers" (application and management software) and "Providers" (storage systems) to manage fixed content reference information storage services. XAM includes metadata definitions to accompany data to achieve application interoperability, storage transparency, and automation for ILM-based practices, long term records retention, and information security. XAM will be expanded over time to include other data types as well as support additional implementations based on the XAM API to XAM conformant storage systems.

More about the proposed API approach...

News

Product review: Sun's StorageTek Honeycomb is sticky and sweet | InfoWorld | 03/24/2008

Most storage solutions are optimized for fast access and frequent updates, a formula that fits the requirements of transactional applications to a tee but isn't necessarily well suited to archiving files that — whether by law, or policy, or practice – either must not or will not be changed. As a sweeping simplification, this "fixed content" – such as legal documents, financial data cutoff, engineering diagrams, medical images, audio files, and video files – are ready for archiving the instant they're created. Using a tiered storage approach to managing them makes little sense. Moreover, these files typically produce large archives that must be maintained for a long time, which makes the conventional approach to data protection, namely tape backup, difficult and expensive to implement.

Sun Open Sources "Honeycomb" Disk Array Software | The IT Jungle | 03/04/2008

The product took two years to move from a project inside the labs to become the StorageTek 5800 disk array, but only a few months after delivering the petabyte-scale disk array that was code-named "Honeycomb" and that is a key part of the Constellation InfiniBand clustered system, Sun Microsystems has decided to open source the software that comprises the guts of the Honeycomb system.

The death knell for RAID? | InfoWorld | 02/29/2008

Defining the single-most significant change in storage I have seen throughout the years could easily lead to multiple discussion threads. After all, technological achievements are plentiful. Take the emergence of SATA and SAS, for example; or the staggering increase in capacity of a wide range of storage media; or the ongoing acceptance of Ethernet as an alternative to Fibre Channel for block storage connectivity.

Library? Cybrary? A Shifting Paradigm at Stanford | Innovating@Sun | 04/03/2007

The thinking around what libraries are and how we can utilize them is changing, as is evidenced by some innovative work being done at the Stanford University Library. Hal Stern, Sun's VP of Global Systems Engineering talks with Stanford's Michael Keller in this latest edition of Innovating@Sun about the shifting paradigm in the world of research, knowledge attainment, and sharing.

Honeycomb CAS: Coming to Open Source | Sun HPC Watercooler | 02/09/2007

Sun Microsystems Inc has earmarked its Thumper disk array as a future NAS and VTL platform, and said that it plans to make its Honeycomb CAS software open source.

OpenSolaris