Black white and grey screens of death in Citrix Virtual Apps and Desktops

Table of Contents

Case
Solution

Case #

Traditionally we have been referring to the so called "Blue Screens of Death (BSOD)" which are blue screens with diagnostic messages thrown at the user when there is a critical and blocking issue at the operating system level. These are most common in older Windows operating systems but continue to occur until today. They may be attributed to hardware, drivers or software applications.

In the case of End User Computing (EUC) and more specifically Citrix Virtual Apps and Desktops there are occurences of events where a black, white (blank) or grey screen appears during user logon. Most frequently the screen seems to be frozen at the black or grey screen and the user cannot logon, as shown in the screenshot below.

In the case of a white (blank) or grey screen, the user session is not established in Citrix and the user is not shown in the VDA server or in Citrix Director but a local profile folder is created for that user. In this case, an SMB connection is established to the file server where the user folder redirection data and user profile data are stored.

In the case of a black screen, the user Citrix session is established and the user is either eventually presented with the Citrix desktop after a long waiting time or is entered into a loop and never sees the Citrix desktop.

Technically speaking, the above issues are not equivalent to a BSOD but given that it is still a blocking issue for the end user, it could be called Black, White (blank) or Grey Screen of Death (BWGSOD).

Solution #

Overview #

There have been many cases especially of black screen issues in previous versions of Citrix Virtual Apps and Desktops, formely known as XenApp and XenDesktop. Most of these issues are resolved in the latest CR and LTSR versions of Citrix Virtual Apps and Desktops.

Important note: Always check the applicability of any solution or workaround to the specific version and edition of WIndows Server, Citrix Virtual Apps and Desktops, Citrix Gateway and Citrix Workspace App client.

There are various factors which can contribute to black, white (blank) or grey screens issue in Citrix Virtual Apps and Desktops. I am summarizing below all the possible root causes I have seen in production environments. Of course this list is not conclusive, so I will update it whenever a new root cause is discovered for the Black, White or Grey Screen of Death issue.

As with all cases, first check to ensure that your issue is not in the list of known issues of your Citrix Virtual Apps and Desktops, such as in the following example: https://docs.citrix.com/en-us/citrix-virtual-apps-desktops/1912-ltsr/known-issues.html.

White (blank) or grey screen of death #

Common root causes for a white (blank) or grey screen in Citrix Virtual Apps and Desktops are the following:

It could be due to the DPI policy in the local Citrix Workspace App client endpoing
It could be a DTLS issue with security filtering of some ISP providers. We need to check if UDP/DTLS or TCP/TLS is enabled by right clicking the local Citrix Workspace App client while a connection is established.

If the error involves black screen or grey screen, check the High DPI setting (see screenshot) on the user’s local endpoint Citrix Workspace App client, logoff and try again.

If black screen issue persists, then most likely it is due to security filtering of the ISP of the user, which is blocking DTLS protocol. The user will need to contact their ISP and request that the DTLS protocol traffic is allowed. This is a known case we have seen in the recent past.

There are a few more scenarios which can lead to white (blank) or grey screen of death issue.

KAPC Hooking disabled Issue #

If having a white or grey screen issue, you also need to check for the following events on the VDA machine after the time of boot of the machine when the issue happens.

Log Name: System
Source: CtxUvi
Date: 26-12-2019 11:08:50
Event ID: 1005
Task Category: None
Level: Error
Keywords: Classic
User: N/A
Computer: ************
Description:
The Citrix Universal DLL Injection Driver has encountered an unexpected error.

Log Name: System
Source: CtxUvi
Date: ***********
Event ID: 1003
Task Category: None
Level: Error
Keywords: Classic
User: N/A
Computer: *****
Description:
The Citrix Universal DLL Injection Driver has detected an integrity error during process creation. The Citrix Universal DLL Injection Driver has been disabled.

Then you will need to identify what is causing the ctxuvi hook to get disabled. In the Friendly View of XML Data of that event you can see which hook is getting disabled. Hook DLLs MfApHook configuration in the registry must not change at runtime HKLM\SOFTWARE\Citrix\CtxHook\AppInit_DLLs\<hook name>. If a change in configuration is detected the KAPC Hooking Driver disables itself (until next reboot).

In the past Citrix has seen such issues caused by third party software, GPO’s trying to modify permissions, timestamps, integrity flags of the Citrix Hook DLL’s or a third-party process temporarily locking access to Hook DLL's in C:\Program Files (x86)\Citrix\System32 or above registry location. In order to catch the issue on time, you need to enable certain logging in advance to catch the issue as it happens. You would need to run procmon and have a view of all activities running on the system when this is noticed.

EDT and MSS network configuration #

If you do not catch any events before the issue happens, then issues with the EDT protocol can also lead to Grey Screen Randomly which is mostly around bandwidth and MSS Size set on network devices.

Black screen of death #

Black screens of death in Citrix environments can be attributed to issues with profile management configuration and operations (Citrix CPM cases and FsLogix cases). The following sections provide common root causes for the black screen issue.

Check with your Profile Management solution provider to investigate known issues and fixes with black screens related to profile management operations. One such example with FsLogix black screens is the following: https://docs.microsoft.com/en-us/answers/questions/197447/fslogix-black-screen-at-login.html.
The black screen issue can also occur in Microsoft RDS sessions. See this KB article for more details on the RDS case.
One other common root cause is a known issue with the HKCU:\Software\Microsoft\Windows\CurrentVersion\UFH\SHC registry key. A workaround for this issue which contributes to a black screen at logon would be to delete the above registry key either at logon or logoff time for all affected users by running the below Powershell script as part of a logon/logoff script.

Remove-Item HKCU:\Software\Microsoft\Windows\CurrentVersion\UFH\SHC

Exclude the following registry entry from Citrix profile management. This means that this entry should not be synchronized with the user's Citrix profile.

Create in UPM a policy to exclude the key HKEY_CURRENT_USER\Software\Microsoft\Windows\CurrentVersion\UFH from the registry settings

Also a black screen of death may be due to a corrupt default user profile. In this case the user profile should be cleaned up or most likely replaced by copy pasting a healthy default profile from another healthy machine with same OS or Windows roaming profile version. Carry out the tasks outlined in the following articles for fixing this root cause:
- https://helgeklein.com/blog/2008/07/corrupt-user-profiles-do-they-even-exist/
- https://helgeklein.com/blog/2013/09/error-message-explained-user-profile-service-failed-logon/

The following registry optimizations should be applied to all affected Citrix VDA machines, either manually or via GPO to eliminate the black screen issue.
- Path HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\CurrentVersion\Policies\Explorer, value name UseDesktopIniCache, value type REG_DWORD and value data 0 (zero)
- Path HKEY_LOCAL_MACHINE\Key path SOFTWARE\Citrix\GroupPolicy, value name SaveRsopToFile, value type REG_DWORD and value data 1
- Path HKEY_LOCAL_MACHINE\Key path SOFTWARE\Citrix\GroupPolicy, value name SaveRsopToMemory, value type REG_DWORD and value data 0
- Path HKEY_LOCAL_MACHINE\Key path SOFTWARE\Citrix\GroupPolicy, value name SaveRsopToRegistry, value type REG_DWORD and value data 0
- Path HKEY_LOCAL_MACHINE\Key path SOFTWARE\Citrix\GroupPolicy, value name GpoCacheEnabled, value type REG_DWORD and value data 0
- Path HKEY_LOCAL_MACHINE\Key path SOFTWARE\Citrix\Reconnect, value name DisableGPCalculation, value type REG_DWORD and value data 1
- Path HKEY_LOCAL_MACHINE\Key path SOFTWARE\Citrix\Reconnect, value name DisableFullStreamWrite, value type REG_DWORD and value data 1
The issue could also be due to Citrix Gateway configuration for Citrix Receiver Policy vs Citrix Receiver for Web. In this case, the grey, white or black screen issue could be caused due to Citrix Gateway configuration and could work without issues when connecting either from native Citrix Workspace App or via a Web browser session. This is unique for each Citrix Gateway configuration/environment.

For more troubleshooting guidance on a series of known Citrix issues, you can consult my "Citrix Virtual Apps and Desktops Troubleshooting" e-book.