Customer Solution Case Study
State of Washington Deploys First-of-a-Kind Digital Archive
Overview
Country or Region: UnitedStates
Industry: State government
Customer Profile
The state of Washington has 6 million people and more than 3,000 local and statewide agencies serving them.
Partner Profile
Based in the Dallas area, EDS provides services in information technology, applications, business processes, and data transformation to a worldwide client base.
Business Situation
The state of Washington needed a way to archive millions of historical and legally significant documents produced annually and make them accessible to the public.
Solution
A solution based on Microsoft® Windows Server System™ and Microsoft .NET technologies imports documents in diverse formats and assembles them into a consistent and accessible archive.
Benefits
n Preserving history
n Streamlining government operations
n Showcasing a pioneering state endeavor / “Regardless of technology changes,….we’ll always be able to pull up and view a document and learn what we need to know about that historical record.”
Adam Jansen, Digital Archivist, State of Washington
The state of Washington faced a challenge common among governmental entities everywhere: the need to store historical and legal records securely while making them easily accessible to the public. To address this challenge, officials developed a solution based on Microsoft® Windows Server System™ and Microsoft .NET technologies and, working with experts from EDS and Microsoft, deployed it on schedule and under budget. Today, the solution imports millions of records in dozens of different formats from thousands of state agencies and assembles them into a coherent, consistent, and Web-accessible archive¾all while strictly maintaining the documents’ security, integrity, and content.
Situation
In the state of Washington, officials recently faced a problem that is long-standing and pervasive among government organizations everywhere: the need to securely maintain historical and legally significant documents and make them accessible to the public. Like the governments of most states, Washington officials had stored such documents for more than a century in their paper form.
But this was only a partial solution because to provide public access was costly and inefficient. As such documents became increasingly available only in electronic form, archiving became an even greater challenge. Like their paper counterparts, electronic documents could be stored on digital tapes, but providing public access to tapes stored in remote vaults was equally costly and inefficient.
The state also had begun maintaining an archive of sorts on server hard drives, which made the documents more easily accessible, but also made the storage itself problematic because hard drive space was limited. Officials estimated that for this reason, about half of the potentially historical or legally significant documents produced in the state since the 1970s were lost. For the documents that did continue to be maintained in storage, providing access was also administratively expensive because the documents resided in hundreds of databases and in dozens of different formats.
In 2001, the state made moves to address these challenges. At the urging of just-elected Secretary of State Sam Reed, the legislature allocated funds for the deployment of a solution to store the millions of documents produced each year by more than 3,300 agencies in 39 counties.
There were three primary requirements: the solution would need to work in an automatic and seamless fashion as to avoid imposing administrative costs on the agencies; the solution would need to make the documents easy to find, authenticate, and view on the Web while also ensuring their security; and to comply with the allocated budget, the solution would also need to be deployed within less than a year.
Solution
In considering software for the solution, officials faced a challenge common among pioneers. As Assistant Secretary of State Steven Excell says, “It hasn’t really been done before. There are folks working on this problem at the national level, and there are people working abroad. There was no pre-packaged solution for this.”
Initially, the team evaluated content management systems but found them lacking in the functionality required to manage a mass storage environment. The officials also considered data warehousing systems but found them insufficiently flexible to accommodate the diverse platforms in use among the state and local agencies that generated the documents. Data warehousing also did not provide a quick and easy way to address the expectations of the public, who would be accessing the archive.
“We needed an approach that could handle hundreds of terabytes of data and be sufficiently robust to dish out content to the Web rapidly and in a form the users could easily handle,” Excell emphasizes.
Discovering a Microsoft–Based Approach
Because the state of Washington was a major user of Microsoft technologies, Excell and his colleagues then considered a solution based on Microsoft® Windows Server System™ integrated server software. Through architectural design sessions and proof-of-concept demonstrations, they found much to like.
For example, the officials observed how Microsoft BizTalk® Server 2004, at the time in beta release, could be used as a core technology to import records from multiple sources in multiple file formats and convert them into a consistent and searchable database. They saw that Microsoft SQL Server™ 2000 was sufficiently scalable to store data that they envisioned would ultimately reach nearly a petabyte in volume. They understood how the modular nature of the Microsoft .NET Framework, not to mention its support for Web services, could be ideal for meeting an ambitious deployment timeline and stringent budget constraints.
The officials also considered the long-term budgetary advantages of using the Microsoft Visual Studio® .NET development system and other tools within the Microsoft platform that are widely familiar to developers, especially student developers. “The digital archive facility would be located on a university campus, and we knew that students would help to maintain the solution, so it was ideal to base it on an environment that so many of them already knew,” says Bill Kilcullen, Chief Technologist at EDS, which provided digital-asset management expertise to the deployment team.
In addition to receiving assistance from EDS, state of Washington developers worked closely with the Microsoft Enterprise and Partner Group, Microsoft Services, and the Microsoft sales team to deliver a robust solution in just three months from proof-of- concept to initial production deployment¾and under budget. For this, Digital Archivist Adam Jansen gives a lot of credit to the team effort.
“EDS brought in resources who were very experienced with BizTalk Server 2004 and familiar with the .NET environment,” Jansen says. “And Microsoft brought the tools and skills not only to deploy initially, but also to help ensure that we have the resources we need to continue to develop out.”
Secretary of State Sam Reed concurs, stating, “Having the teamwork and the technology has enabled us to do something in just a few months that in the past might have required an enormous volume of internal resources and perhaps a matter of years.”
Using Web Services with ASP.NET Controls to Protect Data Integrity
Online since late 2004, the solution uses nearly two dozen servers to maintain high availability, supports clustering and load-balancing for failover, and accommodates up to 800 terabytes of data (equivalent to 200 billion pages of text) in formats that vary from database records and e-mail messages to rich media content such as audio and video.
In addition to the primary operating system, Microsoft Windows Server™ 2003, and database management system, Microsoft SQL Server 2000, the solution relies heavily on Microsoft BizTalk Server 2004. With BizTalk Server, the solution connects several times daily to state and local agencies, importing documents in their original formats, and applying metadata to them for indexing purposes. The solution uses SQL Server 2000 Data Transformation Services to normalize the documents to a uniform format, apply the appropriate protections and permissions to them, and export them into a SQL Server 2000 database.
During the process, the solution authenticates each document and secures it with a digital “lock” that helps to prevent tampering and protect the integrity of the original document. For further protection, the solution uses Web services with Microsoft ASP.NET controls to help isolate the middle tier and database from the presentation tier and to support an e-commerce engine, enabling users to order certified paper copies of archived documents. “Using ASP.NET for this engine was essential because it provides an open standard based on XML, enabling us to verify credit card transactions,” says Lead Application Developer Atul Bahl. “Consequently, ASP.NET helped us to develop an e-commerce engine that is reliable and efficient.”
Benefits
In the first year of its deployment, the state of Washington digital archive solution has been successfully archiving census, naturalization, military, birth, death, marriage, voter registration, property, and other historic records from the executive and legislative branches as well as other rare state archives.
BizTalk Server Meets the Challenges
As envisioned, the solution is meeting the challenge of assembling documents in dozens of formats into a coherent and consistent whole. For this, State Archivist Jerry Handfield credits Microsoft BizTalk Server 2004. “It turns out that BizTalk is a wonderful product for integrating these many different records from all over the state, for doing it efficiently, and for doing it without altering or changing the form or content of the documents,” he says.
The solution is helping to preserve state history¾even very recent history—as Database Administrator Ben Abshire explains.
“Governor Locke, who served for eight years, was committed to the idea of using digital technology to streamline government and toward that end had his own Web site for much of his term,” Abshire reports. “But two days before he left office, we learned that nothing yet had been done to preserve the thousands of speeches, press releases, and other documents that largely existed only on the Web site. So we got on it, and within those two days, we had the entire site archived.”
Officials also expect the solution to help state government work more efficiently. As Bahl explains, having Web access to a robust, comprehensive, and searchable statewide archive of historical and legal documents relieves individual agencies of the need to maintain their own large data storage solutions.
“This goes back to the way the solution uses Web services to expose documents,” Bahl says. “When someone from an agency needs access to a document, they can use the Web services to access it and, assuming they have the necessary permissions, modify that document, if necessary.”
For the Long Run
Jansen says the scenario that Bahl describes speaks directly to the long-term advantages that the state is anticipating from having deployed the digital archive solution using the Microsoft .NET Framework and other Microsoft technologies.
“Regardless of technology changes, regardless of which direction databases go, we’ll have a document that is standalone, open standard,” Jansen says. “It’s a document we’ll always be able to pull up and view and learn what we need to know about that historical record.”
Jansen and his team aren’t the only ones looking to the future of how a digital archive can ultimately benefit government and citizens alike. Since deploying the solution, state of Washington officials have received visitors from across the United States and the world who are looking to implement similar archives for state or federal governments and private institutions.
“It’s become a point of pride for the state of Washington to have officials from some of the world’s largest governmental and private institutions come to us and say, ‘You’re on the right track, please share with us what you are doing,’” Jansen says. “We’re inspired and very proud of what we’ve accomplished with the digital archive solution.”
Microsoft Windows Server System
Microsoft Windows Server System integrated server infrastructure software is designed to support end-to-end solutions built on the Windows Server operating system. Windows Server System creates an infrastructure based on integrated innovation, Microsoft’s holistic approach to building products and solutions that are intrinsically designed to work together and interact seamlessly with other data and applications across your IT environment. This helps you reduce the costs of ongoing operations, deliver a more secure and reliable IT infrastructure, and drive valuable new capabilities for the future growth of your business.
For more information about Windows Server System, go to:
www.microsoft.com/windowsserversystem