How to cleanup expired certificates from a Microsoft CA with PowerShell and Shrink the DB

This a shameless copy of the original post by André Gibel over at https://www.gibel.net/

The reason why I copied it is because there is very little info on this subject and even though the post is from 2014, it still applies today.

Regularly (depending on the number of issued certificates) you have to perform a clean-up of expired certificates from your CA (Certification Authority) DB and then shrink the DB to get rid of the “white space”.

You have to perform the following 3 steps in order:
1. Make a backup of your CA DB (protected with a password) to another Server / medium

- this backup also "removes" the maybe hundres of db log files (each of the has a size of 1 MB) – in my case 828

2.      Clean-up all expired certificates from all 4 categories  with my PowerShell Script

- in a first step it's the best to run the script in a "view only" modus to see which certificates would be deleted
- the script and all the details are explained below.

3.      Shrink your CA database to get rid of the “whitespace”

- for this you use the esentutl tool with the “/d” (= defragmentation) option

Before executing the esentutl command stop the AD Certificate service and disable it

- run the following command with the path to the .edb DB file

- at the end the DB - file is more than 100 MB smaller than before, depending on the size your database is.

- at this point you have to enable and start the CA Service again

Here I explain the PowerShell script in detail (the script is used in step 2)

The Microsoft Enterprise CA I’m responsible for is running on a Microsoft Windows Server 2008 Enterprise Server

- with PowerShell 2.0 installed
- no 3rd party PS modules are used
- the certutil.exe is used by the PowerShell (PS) script
- the PS script I created is "Cleanup_MSPKI_Cert_v1.1.ps1" and contains 3 functions

On this CA Server in the C:\ root drive I create a folder “_scripts “ (I don’t use PS remoting) and copy my PowerShell script “Cleanup_MSPKI_Cert_v1.1.ps1” into this folder

Per default the functions "Remove-ExpiredCertFromDB" writes the temporary files to a subfolder within C:\_scripts\PKICleanupLog.

You can change this default folder path with the parameter  “CleanedFolderLogPath”

The 3 functions I have implemented are:

A.) Get-PublishedCATemplate
B.) Get-IssuedCert
C.) Remove-ExpiredCertFromDB

A.) Get-PublishedCATemplate

When you run this function without a parameter, it displays all Templates from the "Certificate Templates" folder with it's OID. This OID is used by the other to functions to display or delete certificates issued with this certain template. In the following picture you see the corresponding templates from the PKI Snap In

function Get-PublishedCATemplate{             

    [CmdletBinding()]

    Param (

        [parameter()]

        [string]$filter   

    )      

    $FilterLen = ("msPKI-Cert-Template-OID =").length+3   

    $AllPublishedTemplates = Invoke-Expression "certutil.exe –catemplates –v | select-string msPKI-Cert-Template-OID"     

    $AllPublishedTemplates | foreach{       

        $tmp= ($_.line).Substring($FilterLen)       

        $splitarr = $tmp.split(" ",2)     

        $obj = New-Object PSObject                                     

        Add-Member -Input $obj -Name "name" -MemberType Noteproperty -Value $Splitarr[1].trim()

        Add-Member -Input $obj -Name "oid" -MemberType Noteproperty -Value $Splitarr[0].trim()              

        if ($PSBoundParameters["filter"]){  

            if ($Splitarr[1].trim() -match $filter){

                write-output $obj             

            }

        }

        else{

            write-output $obj             

        }

    }              

}

Below I run the script with the -filter parameter and so I only get templates with “SCCM” in their name

I assign the oid of ONE template (=> change filter that you get only one result)  to the variable WSTemplate

$WSTemplate = (Get-PublishedCATemplate -filter workstation).oid

B.) Get-IssuedCert

With this function  you can list the certificates  issued from all templates or a certain template (specified with it’s oid = $CertTemplate variable)  which are issued beginning at a certain date.

 function Get-IssuedCert{

  [CmdletBinding()]

  Param (

    [ValidatePattern('^([0-9\.\s])+$')]

    [string]$CertTemplate,

    [ValidatePattern('^\d\d[\./]{1}\d\d[\./]{1}\d\d\d\d$')]

    [string]$Date

  )

  if ($PSBoundParameters["CertTemplate"]){   

    Invoke-Expression "certutil.exe -view -restrict 'certificate template=$CertTemplate,disposition=20,notbefore>=$Date' -out 'Request.RequestID,Request.RequesterName,NotBefore,NotAfter,Request.Disposition'"       

  }

  else {

    # displays Certificates issued with any custom template   

    Invoke-Expression "certutil.exe -view -restrict 'disposition=20,notbefore>=$Date' -out 'Request.RequestID,Request.RequesterName,NotBefore,NotAfter,Request.Disposition'"             

  }

}

 

The following example lists all  29 certificates (from ALL templates) issued from 18 December 2014 and later …. (with this version it’s not possible to select a time range / only a “start-date”)

Get-IssuedCert  -Date 18.12.2014

The following example lists ONLY the 3 certificates which are issued with the Template $WSTemplate (OID of  “…- Workstation – Authentication Certificate”) beginning December 18. 2014

$WSTemplate = (Get-PublishedCATemplate -filter workstation).oid
Get-IssuedCert -CertTemplate $WSTemplate -Date 18.12.2014

C.) Remove-ExpiredCertFromDB

This is an advanced function and all available parameters are displayed with the get-help command

- the expired certificates to view (1st step) and then delete are in one of 4 folders

- I select this “folder” with the -state parameter

- the script creates a log file (also needed for further parsing) in a separate folder

These folders are created automatically if they don’t exist yet.
In a first step always run the cmdlet without the "-delete" parameter so nothing is really deleted.
I also recommend the ISE instead of the shell.
And I also always run this cmdlet with the “-verbose” parameter.

The following example displays all issued (and expired) certificates till 18.12.2014  --- they are not really deleted yet.

Remove-ExpiredCertFromDB -State issued -Date 18.12.2014  -Verbose 

Without the -delete switch parameter the log file has "-ViewOnly" in it’s name

Below is the output from the example above / 396 entries “would be” deleted from the “issued folder” (or category) if you run the cmdlet with “-delete”

The following example lists / deletes  certificates from a certain (workstation authentication) template expired up to 18 december 2014

$WSTemplate = (Get-PublishedCATemplate -filter workstation).oid
Remove-ExpiredCertFromDB -State issued -CertTemplate $WSTemplate -Date 18.12.2014 -Verbose

With the added-delete switch parameter you really delete the entries. 
This step can take some time if there are a lot of entries.

$WSTemplate = (Get-PublishedCATemplate -filter workstation).oid
Remove-ExpiredCertFromDB -State issued -CertTemplate $WSTemplate -Date 18.12.2014 -Verbose -delete

The output at the end (and the log file)

When  you run the same cmdlet again, you see that there aren’t any entries to delete from the DB

The latest (full) version of this script with the 3 functions you can download from the Microsoft Script Gallery: go to download

DirectDownload link

Since TechNet is retired and will be taken offline any time soon a backup can be downloaded here:

Cleanup_MSPKI_Cert_v1.2.ps1

Convert security group to mail enabled security group with Powershell

Find your AD group name:

Get-ADGroup -Identity "groupname"

Enable the group for Email:

Enable-DistributionGroup -Identity "groupname" -PrimarySMTPAddress groupname@domain.com

Download Files Through Authenticating Proxy with PowerShell

Download Files Through Authenticating Proxy with PowerShell

1st Method:

$source = "https://www.7-zip.org/a/7z1900-x64.exe";            
$dest = "C:\Temp\7z1900-x64.exe";            
[Net.ServicePointManager]::SecurityProtocol = [Net.SecurityProtocolType]::Tls12            
$WebClient = New-Object System.Net.WebClient;            
$WebProxy = New-Object System.Net.WebProxy("http://proxy:8080",$true);            
$Credentials = (New-Object Net.NetworkCredential("USERNAME","PASSWORD","domain.com")).GetCredential("http://proxy","80", "KERBEROS");            
#$Credentials = [System.Net.CredentialCache]::DefaultNetworkCredentials            
$WebProxy.Credentials = $Credentials;            
$WebClient.Proxy = $WebProxy;            
$WebClient.DownloadFile($source,$dest);

Alternative Methods:

New-Item -ItemType Directory -Force -Path C:\Temp # Create Temp folder if it doesn't already exist            
$proxy="http://proxy:8080";            
$exclusionList="localhost;*.domain.local;10.*"

# Set winhttp proxy for PowerShell

netsh winhttp set proxy $proxy $exclusionList

# Prepare PowerShell to Use Default Credentials

[system.net.webrequest]::defaultwebproxy = New-Object system.net.webproxy($proxy)            
[system.net.webrequest]::defaultwebproxy.credentials = [System.Net.CredentialCache]::DefaultNetworkCredentials            
[system.net.webrequest]::defaultwebproxy.BypassProxyOnLocal = $true            
[Net.ServicePointManager]::SecurityProtocol = [Net.SecurityProtocolType]::Tls12

# Download using the most efficient method

$url = "https://www.7-zip.org/a/7z1900-x64.exe"             
$temp = "C:\Temp\7z1900-x64.exe"            
Import-Module BitsTransfer            
Start-BitsTransfer -Source $url -Destination $temp

Or:

$source = "https://www.7-zip.org/a/7z1900-x64.msi";            
$dest = "C:\Temp\7z1900-x64.msi";            
$browser = New-Object System.Net.WebClient            
$browser.Proxy.Credentials =[System.Net.CredentialCache]::DefaultNetworkCredentials;            
[Net.ServicePointManager]::SecurityProtocol = [Net.SecurityProtocolType]::Tls12            
$browser.DownloadFile($source,$dest);

Or:

$source = "https://www.7-zip.org/a/7z1900-x64.msi";            
$dest = "C:\Temp\7z1900-x64.msi";            
[Net.ServicePointManager]::SecurityProtocol = [Net.SecurityProtocolType]::Tls12            
(new-object System.Net.WebClient).DownloadFile($source,$dest)

Or:

[Net.ServicePointManager]::SecurityProtocol = [Net.SecurityProtocolType]::Tls12            
[Net.WebRequest]::DefaultWebProxy.Credentials = [Net.CredentialCache]::DefaultCredentials; `
iex ((New-Object Net.WebClient).DownloadString('https://www.7-zip.org/a/7z1900-x64.msi'))

Or:

$page = (new-object net.webclient)            
$page.UseDefaultCredentials = $True            
$Page.DownloadString('https://www.7-zip.org/a/7z1900-x64.msi')

Azure Log Analytics - The client and server cannot communicate, because they do not possess a common algorithm

The client and server cannot communicate, because they do not possess a common algorithm

Well the error itself reveals a little bit of info about what it involves, algorithms.
Where are algorithms used? Well secure connections, such as TLS.

If you have TLS 1.2 enabled chances are your application defaults to TLS 1.0 which you probably have disabled just a did for TLS1.1.

If you have servers that have .Net 4.5.2 and below, and you have a .Net application that is trying to connect to something using TLS or Schannel, it will default to TLS 1.0 and fail if you have disabled tls 1.0 and 1.1.
But you can fix this by adding a registry key: HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\.NETFramework\v4.0.30319

and

HKEY_LOCAL_MACHINE\SOFTWARE\Wow6432Node\Microsoft\.NETFramework\v4.0.30319

called SchUseStrongCrypto and set that to 1.

After a reboot, the .Net applications will be forced to use TLS.1.2 and connect.

Full error from the monitoringAgent.log file:

Failed to connect, exception : System.Net.WebException: The underlying connection was closed: An unexpected error occurred on a receive. ---> System.ComponentModel.Win32Exception: The client and server cannot communicate, because they do not possess a common algorithm