Chocolatey's signature is not valid

[AHuusom]

Description

Chocolatey fails when trying to be installed due to outdated signature.

Platforms affected

• Azure DevOps
• GitHub Actions - Standard Runners
• GitHub Actions - Larger Runners

Runner images affected

• Ubuntu 20.04
• Ubuntu 22.04
• Ubuntu 24.04
• macOS 12
• macOS 13
• macOS 13 Arm64
• macOS 14
• macOS 14 Arm64
• macOS 15
• macOS 15 Arm64
• Windows Server 2019
• Windows Server 2022

Image version and build link

Latest

Is it regression?

Don't know

Expected behavior

That it can download chocolatey

Actual behavior

2024-12-12T10:35:02+01:00: windows.azure-arm.windowsbuildserver: Install chocolatey
2024-12-12T10:35:02+01:00: windows.azure-arm.windowsbuildserver: Downloading package from https://chocolatey.org/install.ps1 to D:\temp\install.ps1...
2024-12-12T10:35:02+01:00: windows.azure-arm.windowsbuildserver: WARNING: Package download failed in 0.01 seconds
2024-12-12T10:35:02+01:00: windows.azure-arm.windowsbuildserver: WARNING: Exception calling "DownloadFile" with "2" argument(s): "An exception occurred during a WebClient request."
2024-12-12T10:35:02+01:00: windows.azure-arm.windowsbuildserver: WARNING: Waiting 30 seconds before retrying (retries left: 20)...
2024-12-12T09:19:16+01:00: ==> windows.azure-arm.windowsbuildserver: Get-AuthenticodeSignature : File D:\temp\install.ps1 was not found.
2024-12-12T09:19:16+01:00: ==> windows.azure-arm.windowsbuildserver: At C:\Program Files\WindowsPowerShell\Modules\ImageHelpers\InstallHelpers.ps1:980 char:18
2024-12-12T09:19:16+01:00: ==> windows.azure-arm.windowsbuildserver: + $signature = Get-AuthenticodeSignature $Path
2024-12-12T09:19:16+01:00: ==> windows.azure-arm.windowsbuildserver: + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
2024-12-12T09:19:16+01:00: ==> windows.azure-arm.windowsbuildserver: + CategoryInfo : ObjectNotFound: (:) [Get-AuthenticodeSignature], FileNotFoundException
2024-12-12T09:19:16+01:00: ==> windows.azure-arm.windowsbuildserver: + FullyQualifiedErrorId : SignatureCommandsBaseFileNotFound,Microsoft.PowerShell.Commands.GetAuthenticodeSignature
2024-12-12T09:19:16+01:00: ==> windows.azure-arm.windowsbuildserver: Command
2024-12-12T09:19:16+01:00: ==> windows.azure-arm.windowsbuildserver:
2024-12-12T09:19:16+01:00: ==> windows.azure-arm.windowsbuildserver: Signature status is not valid. Status:
2024-12-12T09:19:16+01:00: ==> windows.azure-arm.windowsbuildserver: At C:\Program Files\WindowsPowerShell\Modules\ImageHelpers\InstallHelpers.ps1:983 char:9
2024-12-12T09:19:16+01:00: ==> windows.azure-arm.windowsbuildserver: + throw "Signature status is not valid. Status: $($signature.St ...
2024-12-12T09:19:16+01:00: ==> windows.azure-arm.windowsbuildserver: + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
2024-12-12T09:19:16+01:00: ==> windows.azure-arm.windowsbuildserver: + CategoryInfo : OperationStopped: (Signature status is not valid. Status: :String) [], RuntimeException
2024-12-12T09:19:16+01:00: ==> windows.azure-arm.windowsbuildserver: + FullyQualifiedErrorId : Signature status is not valid. Status:

Repro steps

--

[baidarka]

It appears that https://chocolatey.org/install.ps1 cannot be downloaded (after 20 retries have been exhausted) for some reason.
It appears that the download retry loop has degraded.
The problem seems to be in the download, not the signature check.

EDIT 2024-12-12T12:35:00
It seems that if the line InstallHelpers.ps1#L187 fails, the exception occurs as you describe: 'Exception calling "DownloadFile" with "2" argument(s): '

EDIT 2024-12-12T12:43:00
Additional line of log, right before starting the download loop (these are the '2 arguments'):

    azure-arm.image: Downloading package from https://chocolatey.org/install.ps1 to D:\temp\install.ps1...

Question: can anyone still build the Windows images using the main branch?

Same issue in our Windows logs, see below:

    azure-arm.image: WARNING: Package download failed in 0 seconds
    azure-arm.image: WARNING: Exception calling "DownloadFile" with "2" argument(s): "An exception occurred during a WebClient request."
    azure-arm.image: WARNING: Waiting 30 seconds before retrying (retries left: 2)...
    azure-arm.image: WARNING: Package download failed in 0 seconds
    azure-arm.image: WARNING: Exception calling "DownloadFile" with "2" argument(s): "An exception occurred during a WebClient request."
    azure-arm.image: WARNING: Waiting 30 seconds before retrying (retries left: 1)...
==> azure-arm.image: Get-AuthenticodeSignature : File D:\temp\install.ps1 was not found.
==> azure-arm.image: At C:\Program Files\WindowsPowerShell\Modules\ImageHelpers\InstallHelpers.ps1:980 char:18
==> azure-arm.image: +     $signature = Get-AuthenticodeSignature $Path
==> azure-arm.image: +                  ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
==> azure-arm.image:     + CategoryInfo          : ObjectNotFound: (:) [Get-AuthenticodeSignature], FileNotFoundException
==> azure-arm.image:     + FullyQualifiedErrorId : SignatureCommandsBaseFileNotFound,Microsoft.PowerShell.Commands.GetAuthenticodeSignature
==> azure-arm.image:    Command
==> azure-arm.image:
==> azure-arm.image: Signature status is not valid. Status:
==> azure-arm.image: At C:\Program Files\WindowsPowerShell\Modules\ImageHelpers\InstallHelpers.ps1:983 char:9
==> azure-arm.image: +         throw "Signature status is not valid. Status: $($signature.St ...
==> azure-arm.image: +         ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
==> azure-arm.image:     + CategoryInfo          : OperationStopped: (Signature status is not valid. Status: :String) [], RuntimeException
==> azure-arm.image:     + FullyQualifiedErrorId : Signature status is not valid. Status:
==> azure-arm.image:
==> azure-arm.image: Provisioning step had errors: Running the cleanup provisioner, if present...
==> azure-arm.image: 
==> azure-arm.image: Deleting Virtual Machine deployment and its attatched resources...

[vidyasagarnimmagaddi]

Hi @baidarka , @AHuusom ,Thank you for bringing this issue to our attention. We are looking into this issue and will update you on this issue after investigating .

[baidarka]

Given the exception "Exception calling "DownloadFile" with "2" argument(s): "An exception occurred during a WebClient request.""
That exception will be thrown here InstallHelpers.ps1#L187 if the 'Path' parameter contains a non-existing drive or path.

In my log the value of the Path parameter appears to be 'D:\temp\install.ps1'.

    azure-arm.image: Downloading package from https://chocolatey.org/install.ps1 to D:\temp\install.ps1...

On my test machine the folder 'D:\temp' does not exist:

➜ $url = 'https://chocolatey.org/install.ps1'
# on my test machine the folder 'D:\temp' does not exist
➜ $path = "D:\temp\install.ps1"
#  images/windows/scripts/helpers/InstallHelpers.ps1#L187
➜ (New-Object System.Net.WebClient).DownloadFile($url, $path)
MethodInvocationException: Exception calling "DownloadFile" with "2" argument(s): "An exception occurred during a WebClient request."

So one scenario that mimics the exception on Win2019 and Win2022 is to call 'DownloadFile' with a target path containing a non-existent folder.

EDIT 20241212T14:42
Sure enough, the inner exception contains the giveaway:

WARNING: Exception calling "DownloadFile" with "2" argument(s): "An exception occurred during a WebClient request."
WARNING: System.Net.WebException: An exception occurred during a WebClient request.
 ---> System.IO.DirectoryNotFoundException: Could not find a part of the path 'D:\temp\install.ps1'.

Since 'WebClient' is obsolete, and used in InstallHelpers.ps1#L187, a replacement could be 'Invoke-WebRequest':

Invoke-WebRequest -Uri $url -Outfile $path

which yields: 'Could not find a part of the path 'D:\temp\install.ps1'.' straight of the bat.

All that remains is to make sure the target path exists.
I'd be willing to help out with fixing the code (which seems to have some rough edges), if you'd be willing to review my PR's ;)

[denyos]

We have been experiencing the same issue since November 25th, but haven't had the time to investigate it yet. I took some time today to investigate it further and found out that we currently use the VM type ' Standard_D8s_v4' image which doesn't come with a D drive.
This was causing the error. After failing to download the chocolatey setup script to D:\Temp, the script tries to validate the signature on a non-existing file.

By setting the temp_dir param in the HCL file to C:\Temp instead of D:\Temp, the image is getting build.
There is not an issue with the chocolatey signature.

[vidyasagarnimmagaddi]

Hi all, Please be informed.
The directory for temporary files has been moved to drive D: due to the new cleanup process.

The VM size (Standard_D8s_v4) does not have a local (temporary) disk. However Fsv2 sizes series do have such drive.

To workaround this issue, you can either update VM size variable "vm_size" or update the variable for temporary files variable "temp_dir". Thanks

[baidarka]

Thank you @denyos for investigating, and @vidyasagarnimmagaddi for your summary.
(And so, this brings us to a familiar place ... Please bear with me...)

1. Do not use obsolete System.Net.WebClient.DownloadFile, use Invoke-WebRequest -Uri -Outfile
   PR replace WebClient with Invoke-WebRequest #11180

2. In order to 'update VM size variable "vm_size" or update the variable for temporary files variable "temp_dir"'
   as suggested by @vidyasagarnimmagaddi, @denyos; these variables should be exposed in:

   • build-image.ps1#L46 should access the variables mentioned
   • image-generation.yml#L92

For example in the case of win2022 these vars should be exposed which are currently not exposed:
- temp_dir windows-2022.pkr.hcl#L75
- vm_size windows-2022.pkr.hcl#L151
- azure_tags windows-2022.pkr.hcl#L24

The above is equivalent to PR #10884 (which has been rejected).

[baidarka]

Would you be willing to consider a PR to fix the above, by exposing the variables listed in the 'build-image.ps1' and 'image-generation.yml' for windows and ubuntu? If yes, I will post such a PR.

[denyos]

It should be great if those params would become available in the build-image.ps1 file @baidarka .
Currently we are hacking our way around those limitation by parsing the raw pkr.hcl files and overwriting specific sections before calling the build-image.ps1. Having those params as inputs saves us this hassle and makes it more reliable.

Even a small thing like a key-value pair collection to overwrite those values will help without making the list of inputs to long/complex.

@vidyasagarnimmagaddi Can you log a feature request for this?

[vidyasagarnimmagaddi]

Hi @denyos , @baidarka
currently we dont have any plans to include "exposing the variables listed in the 'build-image.ps1' and 'image-generation.yml' for windows and ubuntu"
Thanks for the contribution in raising the PR : we are testing the PR in all stages and decision will be taken .
And for the PR #10884 , we are not including it. Thanks closing the issue.