FIM Disaster Recovery Planning

FIM Disaster Recovery Planning

<CUSTOMER NAME>

Thursday, 29 October 2015

Version 0.1Draft

Prepared by

Peter Geelen

Premier Field Engineer - Security & Identity

MICROSOFT MAKES NO WARRANTIES, EXPRESS OR IMPLIED, IN THIS DOCUMENT.

Complying with all applicable copyright laws is the responsibility of the user.Without limiting the rights under copyright, no part of this document may be reproduced, stored in or introduced into a retrieval system, or transmitted in any form or by any means (electronic, mechanical, photocopying, recording, or otherwise), or for any purpose, without the express written permission of Microsoft Corporation.

Microsoft may have patents, patent applications, trademarks, copyrights, or other intellectual property rights covering subject matter in this document.Except as expressly provided in any written license agreement from Microsoft, our provision of this document does not give you any license to these patents, trademarks, copyrights, or other intellectual property.

The descriptions of other companies’ products in this document, if any, are provided only as a convenience to you.Any such references should not be considered an endorsement or support by Microsoft.Microsoft cannot guarantee their accuracy, and the products may change over time. Also, the descriptions are intended as brief highlights to aid understanding, rather than as thorough coverage. For authoritative descriptions of these products, please consult their respective manufacturers.

© 2010 Microsoft Corporation. All rights reserved. Any use or distribution of these materials without express authorization of Microsoft Corp. is strictly prohibited.

Microsoft and Windows are either registered trademarks or trademarks of Microsoft Corporation in the United States and/or other countries.

The names of actual companies and products mentioned herein may be the trademarks of their respective owners.

Page 1

Revision and Signoff Sheet

Change Record

Reviewers

Table of Contents

1General

1.1DRP

1.2SQL

2Backup

2.1VM System Snapshot/Backup

2.1.1Advantage

2.1.2Disadvantage

2.1.3Known issues with Snapshots

1.Known issues with Snapshots

2.2Windows Backup

2.2.1Advantage

2.Entire machine backup

2.2.2Disadvantage

2.3Backup FIM Synchronization server

2.3.1FIM Sync Database

2.3.2FIM Sync encryption key

2.3.3FIM Sync registry

2.3.4Source Code

2.3.5Log files

2.3.6Configuration files

2.3.7Scripts

2.3.8Folder: MAData (Filebased MAs)

2.3.9Software setup files

2.3.10FIM Sync server configuration export

2.3.11Backup Task scheduler configuration

2.3.12Backup 3rd party clients & tools configuration

2.3.13PCNS

2.3.14Non-FIM components (OS, ...)

2.4Backup FIM Service

2.4.1FIM Service export

2.4.2FIM Service Database

2.4.3Backup FIM Service Config file

2.4.4Registry

2.4.5SQL Agent Jobs

2.4.6Tools

2.5Backup FIM Portal

2.6Tools

2.7Backup FIM Reporting

2.7.1Databases

2.8SCSM

2.9Backup BHOLD

2.10Backup FIM CM

2.11Backup Documentation

2.12Backup External Source & Target data

2.13Schedule Backup

2.13.1Configuration components

2.13.2Suggested backup schedule (Manual)

2.14Data components

2.14.1Suggested backup schedule (automated)

3Failover FIM Synchronization

3.1Failover - Warm-standby

3.2Failover - Cold standby

4Server recovery

4.1Full system recovery

4.2FIM Server reinstallation

4.3FIM Server Refresh (repair installation)

4.4Partial / component recovery

4.4.1FIM Sync Server

4.4.2External data source

4.5FIM Service Recovery

4.5.1Database restore

4.5.2Repair install with existing database

5Planning

5.1Review

6Crisis communication

7See also

8References

8.1FIM

8.2SQL

9Downloads

10Additional Resources

11Glossary, abbreviations & acronyms

Page 1

1General

1.1DRP

• A DRP plan must provide methods to recover different situations and scenarios
• its required to include multiple backup methods, as each method is to be used in different situations
• Do not rely on a single method, eg. 'system wide' backups only, as these have important disadvantages
• Whatever (combination of) method(s) you choose to backup your configuration, the value of your backup is only guaranteed if you are able to restore it.
• DRP (Disaster Recovery Planning) is only valid if you have tested it thoroughly, this includes
• testing backup
• restoring backups
• executing bare metal restoration
• In case of disaster, you don't want to spend time on considering which option is the best
• Make sure the DRP planning and execution is documented in detail
• Describe which methods to use in which case
• DRP should not depend on the skills of the FIM admins only
• Make sure you can execute the recovery blindly
• Make sure that server admins with limited FIM skills can execute the DR
• DRP planning must be reviewed on a regular basis (yearly)
• SQL

• Carefully verify the SQL backups, as the the backups in SQL have double functionality
• Backup data
• Optimize logs, truncate logs to minimize use of space
• Make sure to have the proper maintenance plans for your FIM databases
• Keep fragmentation under control (eg the FIM Sync database has the tendency to fragment...)

2Backup

This applies to any server on which FIM components are installed.

2.1VM System Snapshot/Backup

Lots of environments run their servers on a virtualized machines, most of these VM systems allow to snapshot the guest system.

2.1.1Advantage

• Entire machine backup
• Fast method to take a snapshot of a working machine
• Quick method to safe the system state just before or just after configuration changes
• Disadvantage

• Snapshots cannot be used for long term storage (as you will run into issues with AD thumbstone when you restore an old machine account)
• If the snapshot contains corruption of configuration issues, it's very hard to recover the system
• It's mostly an all-in-one recovery (no single component recovery)
• Known issues with Snapshots

• Windows Backup
• Advantage

1. Entire machine backup
2. Allows for bare metal restore
3. Quick method to safe the system state just before or just after configuration changes
4. Disadvantage

• This method usually takes some time to restore
• Snapshots cannot be used for long term storage (as you will run into issues with AD thumbstone when you restore an old machine account)
• If the snapshot contains corruption of configuration issues, it's very hard to recover the system
• It's mostly an all-in-one recovery (no single component recovery)
• Backup FIM Synchronization server

More details at: MIIS/ILM/FIM: How to Backup the Synchronization Engine

2.3.1FIM Sync Database

The FIMSynchronisationService database is hosted on SQL.

Make sure you have a regular backup of your database using the SQL Management tools.

It seems obvious, but also perform a restore once in a while. (Doing is the best practice!)

Advantage

• Contains all identity data and configuration items you configure in the GUI
• Contains most configuration data (eg files in Extensions directory), but not all files (like service config files, 3rd party client setup & config, source code ...)

Disadvantage

• Does NOT contain all configuration items
• Not able to restore partial configuration or single components
• If database is corrupt, this method is not valid
• Requires Encryption key to reinstall FIM on same DB, move DB, failover to standby...

Recommendation

• Do NOT rely on DB backup only, you must combine it with other backup methods (like configuration export)
• FIM Sync encryption key

When you install FIM, a tool MIISkmu.exe is installed with it, and available in the Programs > Microsoft Identity Integration Server menu.

Recommendation

• Take a backup (export) of the key regularly (eg before/after an hotfix, upgrade,...)
• FIM Sync registry

The following registry keys under the path

[HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\FIMSynchronizationService]:

• Logging
• ManagementAgents
• Parameters
• Performance
• Source Code

Safeguard the source files of the extensions you created and compiled.

This allows you to recompile the extensions if needed.

Before you start editing and recompiling the extensions, make a backup of the source.

2.3.5Log files

In some opinions maybe less critical, but certainly useful.

2.3.6Configuration files

Some extensions (like logging, GAL,…) use config files to read data.

Also the task scheduler or MASequencer configuration files, if applicable…

FYI: The FIM Sync Extensions directory is backed up into a Microsoft SQL Server table in the FIM Sync database, and the XML files are restored during the FIM Syncactivate process…

2.3.7Scripts

The scripts, commands and batch file you use to automate FIM Sync operations.

2.3.8Folder: MAData (Filebased MAs)

(Cfr. FIM Sync Design and Planning collection)

File-based management agent import and export files that are located in FIM Sync installation path\MAData

2.3.9Software setup files

And it’s always handy to make sure you have the software available you used to prepare your FIM Sync server.

Not only the FIM install files, but also the FIM Sync Resource Took Kit files, Visual Studio, SQL server and other tools you used for configuration, troubleshooting and other operations…

In general, make sure your backups are stored safely (and better at safe distance).

2.3.10FIM Sync server configuration export

Using the FIM Sync GUI (Identity Manager) you can export

• The entire FIM Sync server configuration (Menu File > Export Server Configuration)
• The MA configuration (export management agent, one by one)
• the MV configuration (Metaverse Designer > Export metaverse)

Important notice:

When exporting the FIM Sync configuration,the export does NOT export the content of the connector spaces, nor the metaverse. Also the exensions associated with the MA and MV are not exported.

2.3.11Backup Task scheduler configuration

Independent the tool used to schedule you FIM Sync run profiles, make sure you have a backup of

• the Run scheduler scripts, configuration files, ...
• the schedule used (via backup or via proper documentation)
• Backup 3rd party clients & tools configuration

Theconfiguration of the "non-FIM Sync"tools and clients

For some MA’s you need to configure a client or specific files, or specific network settings. You better make sure you’ve documented these settings or make a backup of this configuration part.

Let me explain: for example the configuration of the hosts file, the services file, system files that are customized to support clients like the Oracle or SAP client.

Other client specific configuration (names.orafile, ID files for Notes, …)

2.3.13PCNS

Although it might be difficult to "backup" the PCNS config. Documenting the setup and the procedures used is of important value.

2.3.14Non-FIM components (OS, ...)

Another part of non-FIM Sync backup is the OS, but I’ll suppose it’s sufficiently known how to backup Windows Server…

One way of backing up the system is a full system backup of your server.On server system level, virtualisation also offers a way of taking a snapshot of the server image.

But still this doesn’t cover the list if the FIM Sync data is distributed over more than one system (eg FIM Sync server with remote SQL).

Remark: be aware of the Microsoft support policy for systems in a virtualized environment (here and here)

2.4Backup FIM Service

Partially taken from: FIM Service Backup and Restore

2.4.1FIM Service export

• Configuration Migration Deployment Steps:
• Export-FIMConfig:

Sample script

2.4.2FIM Service Database

1. By default this database is called FIMService.
2. You do not have to stop the FIM Service when you create this backup.
3. If you added configuration data that was not included in the Setup process, you can also back up those items.
4. The FIM Service database is deployed in full recovery model by default. Using the full recovery model makes it possible to back up the log to provide incremental recovery points. See Backup Under the Full Recovery Model (for more information.

See also: How to back up and restore the relevant SQL databases. For more information, see Backup Overview (SQLServer) (

2.4.3Backup FIM Service Config file

The FIM Service Configuration File, which is located at %programfiles%\Forefront Identity Manager\2010\Service\Microsoft.ResourceManagement.Service.exe.config.

2.4.4Registry

The following registry keys under the path:

HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\FIMService:

• DatabaseServer
• DatabaseName
• CertificateThumbpoint
• DefaultKeySize
• DefaultTokenLifetimeInMinutes
• ServiceAccountSid
• PollExchangeEnabled
• SQL Agent Jobs

SQL Server Agent Jobs:

• FIM_DeleteExpiredSystemObjectsJob
• FIM_MaintainGroupsjob
• FIM_MaintainSetsJob
• FIM_TemporalEventsJob

For detailed guidance, check TechNet FIM Service Backup and Restore

2.4.6Tools

• FIM 2010 R2 Service and Portal Configuration Backup Tool
• Backup FIM Portal

See: FIM Portal Backup and Restore

2.6Tools

• FIM 2010 R2 Service and Portal Configuration Backup Tool
• Backup FIM Reporting
• Databases

See: FIM 2010 R2 Reporting Disaster Recovery:

2.8SCSM

See:

Best Practices for backing up the SCSM Management Server and Data Warehouse Server Databases

It is highly recommended that you follow the procedures outlined in the System Center Service Manager 2010 SP1 Disaster Recovery Guide. These procedures describe backing up the databases and the encryption key that is created when System Center is deployed. It also provides useful SQL scripts for capturing and preserving SQL Server logon and object level permissions information.

2.9Backup Documentation

Document your setup in detail and keep it up to date.

Documentation should include:

• Project documentation (business requirement, business analysis & design)
• Technical architecture/design
• Technical implementation
• DTAP migration procedures
• FIM hotfix & upgrade procedures
• Disaster recovery planning
• Operational management procedures of FIM (maintenance tasks, task scheduling, ...)
• ...
• Backup External Source & Target data

An important piece of data that FIM Sync is managing, is the data residing in the external data sources.

If for some reason FIM Sync exported wrong of faulty data, it can be quite hard to roll-back the export.

Better make sure the data source administrators got their data backed up.

Now you should have secured the most important pieces of the configuration.

Let’s hope we haven’t forgotten anything (otherwise send feedback).

Having a backup is one thing, restoring is another.

Depending on how bad your server got damaged, you have some options for recovery.

It depends on the type of disaster which options to choose or to combine.

2.11Schedule Backup

2.11.1Configuration components

Configuration components don't change frequently (or should not). Configuration items only need to backupped before and after the change to allow recovery.

2.11.2Suggested backup schedule (Manual)

It's a best practice for the administrators to take a backup of the configuraton before and after they change configuration.

2.12Data components

Component that change base on the FIM processes require frequent backup to allow recovery of the entire FIM content.

2.12.1Suggested backup schedule (automated)

It's higly suggested to take regular (automated) backup of the data components. But it's not required to take full backups on a high frequency

Suggestion

• regular low frequency (weekly, biweekly, monthly) full database backup
• regular high frequency (daily) incremental or differential database backups.

3Failover FIM Synchronization

3.1Failover - Warm-standby

A warm standby server is a 2nd FIM Sync up and running along your production system.

But an active server means you need a FIM Sync license for that system…

"FIM Sync Help : Activating a standby server Running ILM 2007 FP1" explains this procedure in detail (using FIM Syncactivate.exe to fail over).

When you run the FIM Syncactivate utility, you’ll need the current FIM Sync encryption key.

3.2Failover - Cold standby

This is the same procedure, but with the secondary server NOT up and running simultaneously.

For this reason there is no need for an additional FIM Sync license.

The procedure for recovery is the same, except for a bit more delay to boot up the standby server.

4Server recovery

4.1Full system recovery

Snapshot restore, bare metal restore, system restore… Get the existing system up and running in one step from scratch.

Easy! ;)

But usually a pretty lengthy job, that requires additional tasks...

4.2FIM Server reinstallation

The most time consuming way to recover is completely reinstall your FIM Sync server from scratch.

A well documented server setup you created before, will be your guide in that case…

Reinstall OS, restore FIM Sync DB, reinstall FIM Sync with existing encryption key, restore config and log files, restore MA data, … the full path.

4.3FIM Server Refresh (repair installation)

Recovering the FIM Sync application and service with a fresh FIM Sync install (database intact, encryption key OK, file backup OK…)

One of the interesting features of FIM Sync is the fact that you can rerun the FIM Sync setup on top of an existing installation without loosing the configuration.

In the case that FIM Sync gets corrupted as application, just rerun the setup and reconnect to the existing database when the wizard asks for the database location.

A fresh install of FIM/ ILM also serves other purposes:

• upgrading IIFP to ILM, ILM to FIM, FIM 2010 to FIM 2010 R2,
• FIM Sync update/upgrade
• eval version to licensed version
• relocating the FIM Sync database (documented here)
• …

Luckily, in some situations you don’t need a complete recovery or a complete reinstallation.

FIM Sync allows for exporting and importing the server configuration , the management agents and the metaverse, via the MIS GUI.

4.4Partial / component recovery

4.4.1FIM Sync Server

FIM Sync server configuration

This FIM Sync server configuration export only contains configuration data only, not the live data of objects in the CS or metaverse. The actual compiled extensions are not included in the FIM Sync config export.