-
Notifications
You must be signed in to change notification settings - Fork 478
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
extremely slow Network/Disk IO on Windows agent compared to Ubuntu/Mac #260
Comments
Perhaps consider not using dotnet install script or is the option to contribute a fix to dotnet install script? |
Hi @jetersen, we will try to resolve this problem. |
Hi Team - any news on this? |
Hello @PureKrome, So far no updates |
The problem is not reproduced anymore. Based on multiple runs, the action does not take more than 15 seconds. Most probably, the root cause of the problem was an infrastructure issue that has currently been resolved. In case the problem reoccurs, the solution is to avoid bulk copying to the OS drive, similar to the workaround applied for the same problem in the actions/setup-go: actions/setup-go#393 @jetersen did it help? |
@dsame I do not agree with the assessment that it is not reproducible 😓 So definitely an improvement but I feel like windows can perform better. https://github.com/jetersen/dotnet.restore.slow.github.action/actions/runs/6736238225 |
I don't think it is fair to say look it is fixed for the actions/setup-dotnet when we are talking about a simple if check to see if .NET 6 is already available on the actions runner image 😓 Testing with .NET 8 preview shows ubuntu with 7 seconds vs +30 seconds sometime a little less. For actions/setup-dotnet. While this issue remains open: #141 this will definitely not improve 😢 |
Thank you @jetersen for pointing out .NET 8 issue, it's confirmed |
Improved `commit` workflow job times from an average of 8m to: - 6m 30s uncached - 5m 30s cached Times were improved by: - Adding caching - Installing packages to the `D:\` drive, as described in <actions/setup-dotnet#260 (comment)>
@dalyIsaac interesting approach, does that really save that much 🤔 |
I'm fairly happy with the gains I've seen, but admittedly I didn't conduct a very rigorous study.
Footnotes
|
Hello @jetersen The quick fix is to set environment variable DOTNET_INSTALL_DIR to the value pointing to some path on the "D:" drive. This workaround is proven to solve the problem https://github.com/akv-demo/dotnet.restore.slow.github.action/actions/runs/6768243557/job/18392290993 and can be used until the action fix is available |
@dsame perhaps some of these fixes should be raised with @actions/runner-images? I assume we are hitting similar IO restrictions on the Windows images as this affects all windows based hosted runners 🫠 |
Hello @jetersen, generally it is a good idea but i doubt any of |
why is this? because these are 2 independent teams within GH and even though the actions team could make some changes based on this thread (which would benefit all users, by default) ... the intra-team still need to also do some changes but you're suggesting that this is a low priority so .. they go 'meh' ? |
Created actions/runner-images#8755 in hoping that we can find a generic solution. I was hoping they could simply change the disk setup on the windows packer scripts 🤔 |
hi |
Hi @jetersen 👋, However, a feasible workaround is to set the env:
DOTNET_INSTALL_DIR: D:\dotnet Please feel free to reach out if you have any concerns or need additional assistance. |
Thank you! I tested this. Note that this is only applicable to standard (4-core for public, 2-core for private repos) runners, not to larger ones, as those don't have a D drive. Results are here: It seems to make things slower. What do you think? |
Hi @Piedone, |
Sure, thank you! In my test, we're building the solution file in the root of our OSOCE project, as well as the solution in the NuGetTest folder. Runs:
You can disregard the failing builds; some parts of the workflow are flaky. The following steps of the executed jobs are directly using the .NET SDK:
|
@priyagupta108 I tested with my repro: There is a definite improvement. But still setup/dotnet install is still incredibly slow when comparing it to Linux or Mac. The .sh relies on curl or wget. Why couldn't Windows rely on curl? GitHub actions runner have curl installed and in my experience avoids any overheads that Powershell or Invoke-WebRequest has when it comes to downloading files. |
Hi @Piedone, Hi @jetersen, |
Thanks for the tip! This was very useful, since setting |
Description:
actions/runner-images#3577
DownloadFile and Extract-Dotnet-Package are awfully slow. Like 3x slower!
Task version:
v1.9.0
Platform:
Runner type:
Repro steps:
https://github.com/jetersen/dotnet.restore.slow.github.action
Expected behavior:
Faster downloads
Actual behavior:
SLOW downloads
The text was updated successfully, but these errors were encountered: