Exchange 2016 - ContentIndexState stuck at HealthyAndUpgrading after CU 7

Question

We are running Microsoft Exchange 2016 on CU 5 and just installed CU7 KB4018115. After installing the update the index state on a few of my databases on one of my servers is at HealthyAndUpgrading. It has been about 24 hours at this point.

Comments

I have seen this same behavior Exchange 2013 CU 18 KB4022631

Answer 1

This is a new feature of exchange where it will upgrade the index before all of the members of the DAG are updated. Then replicate the updated index to the other members of the DAG The previous behavior was to only update the index schema after all members were upgraded.

As with most things Exchange take a look in your event log for more detailed information. You should see a large number of events 

"Microsoft Exchange Information Store service will perform an internal schema upgrade on (Folder) (xxxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxxx[x]-xxxxxxxxxx). The upgrade will bring the version from (0.143) to (0.144)."

Microsoft Exchange Information Store service has completed an internal schema upgrade on (Folder) (xxxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxxx[x]-xxxxxxxxxx). The upgrade has brought the version from (0.143) to (0.155).

The time it takes to complete depends on the size of your environment.

Answer 2

You can configure an alert on eventid 2009 to be notified when the rebuild is complete.

EventID 2009 - MSExchangeFastSearch
Schema update has completed for catalog of database xxxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxxx[x]-xxxxxxxxxx (dbname). Current version is 94489280534.0(Q=22.0 F=22.0).

You can create a powershell script to check the status of the HealthyandUpgrading databases. I am sure it can be cleaned up, but this is what I used.

$db1 = Get-Counter -ComputerName server1 -Counter "\MSExchange Search Indexes(db1)\Crawler: Mailboxes Remaining"
$db2 = Get-Counter -ComputerName server1 -Counter "\MSExchange Search Indexes(db2)\Crawler: Mailboxes Remaining"
$db3 = Get-Counter -ComputerName server2 -Counter "\MSExchange Search Indexes(db3)\Crawler: Mailboxes Remaining"
write-host "db1:"$db1.readings.split(':')[2] `r "db2:"$db2.readings.split(':')[2] `r "db3:"$db3.readings.split(':')[2]