Compacting an AD CS database
Given the healthy trend towards decreasing certificate validity periods, it is very likely that the size of an AD CS Certification Authority database will grow following that same trend.
Despite regular maintenance activities including deletion of records such as failed, denied etc. it could be that at some point the CA database reaches a problematic size.
Deleting records from an AD CS CA database will internally free up the space previously occupied by those records but will not decrease the size of the database file itself. I.e. new records will reuse the freed up space but the total space used by the database file will not decrease.
This helps keeping the database size under control but does not help if the size had already increased over acceptable limits; in this case the database must be compacted.
The CA is often a critical resource, so it is recommended to keep various backups of different nature, to try to make sure we can recover it in case something goes wrong.
Here a procedure to decrease the amount of storage required by the CA database.
1. Delete unnecesary records from the database
In order for the database compaction to help decrease the storage required, first we need to make sure there will be space to be claimed back, and that requires deleting unnecessary records.
Regular maintenance activities might already take care of the deletion of part of those unnecesary records, but if we have reached a situation where storage is running low it probably means additional cleanup is required.
A potential solution is to backup in text format a set of records before deleting them; this could be e.g. certificates issued a long time ago, certificates that have later be renewed, etc. which we might want to keep a record of but not necessarily in the CA database.
Once identified we can dump all those records to a text file (using certutil -view) and delete them afterwards.
Please note that deleting records from the CA database will generate additional transactions which means additional log data which means additional storage usage, so this is something that will likely require batching in order to avoid the storage issues we are trying to solve in the first place.
The approach could be to:
1.a Dump a batch of records
1.b Delete those records from the CA database
1.c Run a CA database backup to get rid of the extra log file generated
1.d Repeat the process until we have deleted a relevant amount of records to free up storage space
2. Make a backup of the CA database
Using the Certification Authority snap-in trigger a CA database backup. this requires the CA service to be running and the destination can be set to a network share, so should not eat up additional speace.
3. Stop the CA service
Next steps require the service to be stopped.
4. Backup the CA database files
Taking a network share as destination take a file based copy of the CA database, the complete CertLog directory (defaults to C:\Windows\System32\CertLog).
it would not be the first time that a CA database backup based restore does not work while a plain file system based backup of certlog does...
5. Run checksum and integrity checks on the CA database
Using the esentutl.exe command verify the checksum of the database:
esentutl /k "<CADatabaseFile.edb>"
and verify its integrity:
esentutl /g "<CADatabaseFile.edb>"
6. Defragment the CA database
This is the par of the procedure that reduces the size of the database file, but in order to do that it will generate a temporary database file as part of the process, which means that a relevant amount of storage is required for it (take the original database file as a reference), but again a network share can be used for that.
esentutl /d "<CADatabaseFile.edb>" /t"<FullNetworkPathToTheTemporaryDatabaseFile.edb>"
Please note thare is no space between the /t switch and the temporary database file parameter.
At the end of the defragmentation process the original database file will be replaced with the defragmented one (the temporary file). The new database file should be smaller than the original in size, actual decrease is dependent on the amount of records removed.
7. Start the CA service
Make sure the CA service can start normally.
8. Take a backup of the CA database
Using the Certification Authority snap-intake another backup of the CA database, with a network share as destination.
9. Cleanup
Once the CA is back in service without any issue and the available storage has increased any temporary files or backups not required can be removed, e.g. the previously taken file system based backup.
https://docs.microsoft.com/en-us/previous-versions/windows/it-pro/windows-server-2012-r2-and-2012/hh875546(v=ws.11)
https://techcommunity.microsoft.com/t5/ask-the-directory-services-team/the-case-of-the-enormous-ca-database/ba-p/398226
Despite regular maintenance activities including deletion of records such as failed, denied etc. it could be that at some point the CA database reaches a problematic size.
Deleting records from an AD CS CA database will internally free up the space previously occupied by those records but will not decrease the size of the database file itself. I.e. new records will reuse the freed up space but the total space used by the database file will not decrease.
This helps keeping the database size under control but does not help if the size had already increased over acceptable limits; in this case the database must be compacted.
The CA is often a critical resource, so it is recommended to keep various backups of different nature, to try to make sure we can recover it in case something goes wrong.
Here a procedure to decrease the amount of storage required by the CA database.
1. Delete unnecesary records from the database
In order for the database compaction to help decrease the storage required, first we need to make sure there will be space to be claimed back, and that requires deleting unnecessary records.
Regular maintenance activities might already take care of the deletion of part of those unnecesary records, but if we have reached a situation where storage is running low it probably means additional cleanup is required.
A potential solution is to backup in text format a set of records before deleting them; this could be e.g. certificates issued a long time ago, certificates that have later be renewed, etc. which we might want to keep a record of but not necessarily in the CA database.
Once identified we can dump all those records to a text file (using certutil -view) and delete them afterwards.
Please note that deleting records from the CA database will generate additional transactions which means additional log data which means additional storage usage, so this is something that will likely require batching in order to avoid the storage issues we are trying to solve in the first place.
The approach could be to:
1.a Dump a batch of records
1.b Delete those records from the CA database
1.c Run a CA database backup to get rid of the extra log file generated
1.d Repeat the process until we have deleted a relevant amount of records to free up storage space
2. Make a backup of the CA database
Using the Certification Authority snap-in trigger a CA database backup. this requires the CA service to be running and the destination can be set to a network share, so should not eat up additional speace.
3. Stop the CA service
Next steps require the service to be stopped.
4. Backup the CA database files
Taking a network share as destination take a file based copy of the CA database, the complete CertLog directory (defaults to C:\Windows\System32\CertLog).
it would not be the first time that a CA database backup based restore does not work while a plain file system based backup of certlog does...
5. Run checksum and integrity checks on the CA database
Using the esentutl.exe command verify the checksum of the database:
esentutl /k "<CADatabaseFile.edb>"
and verify its integrity:
esentutl /g "<CADatabaseFile.edb>"
6. Defragment the CA database
This is the par of the procedure that reduces the size of the database file, but in order to do that it will generate a temporary database file as part of the process, which means that a relevant amount of storage is required for it (take the original database file as a reference), but again a network share can be used for that.
esentutl /d "<CADatabaseFile.edb>" /t"<FullNetworkPathToTheTemporaryDatabaseFile.edb>"
Please note thare is no space between the /t switch and the temporary database file parameter.
At the end of the defragmentation process the original database file will be replaced with the defragmented one (the temporary file). The new database file should be smaller than the original in size, actual decrease is dependent on the amount of records removed.
7. Start the CA service
Make sure the CA service can start normally.
8. Take a backup of the CA database
Using the Certification Authority snap-intake another backup of the CA database, with a network share as destination.
9. Cleanup
Once the CA is back in service without any issue and the available storage has increased any temporary files or backups not required can be removed, e.g. the previously taken file system based backup.
https://docs.microsoft.com/en-us/previous-versions/windows/it-pro/windows-server-2012-r2-and-2012/hh875546(v=ws.11)
https://techcommunity.microsoft.com/t5/ask-the-directory-services-team/the-case-of-the-enormous-ca-database/ba-p/398226
Comments
Post a Comment