When TLS 1.2 breaks Invoke-Webrequest
Update[7/16/18]:
Marc R Kellerman kindly shared his approach on twitter with me today:
[Net.ServicePointManager]::SecurityProtocol = [System.Net.SecurityProtocolType].GetEnumValues()
It works as we can pass an array of values to the Net.ServicePointManager. I think this is nicely done! Thank you so much Marc! Head over to Marc's twitter and blog for more good information!
With more websites moving to TLS 1.2 only, I find using Invoke-WebRequest from Windows PowerShell becomes less reliable because Windows PowerShell's defaults to using Ssl3 and TLS1.0. In this post I am going to share how I worked around the challenge in Windows PowerShell. Very importantly, if you can use PowerShell Core 6 for Invoke-WebRequest, do it. The cmdlet is much improved in PowerShell Core 6 thanks to Mark Karus.
So over the past weekend, I was learning about English and was experimenting to parse the Princeton university's WordNet site with PowerShell. However, I got this bleeding red from Windows PowerShell. 😫
I asked myself what does the "The underlying connection was closed: An unexpected error occurred on a send." mean? Then I dived a bit deeper into the $Error variable using $Error[0] | Format-List * -Force
While HTTP protocol and the .Net class are still over my head, I see the message contains "authentication" and "TlsStream". Interesting! With a quick web search and for sure the good people on the internet has suggestions. (Thank you!) Looks like we get to control the communicating protocol Windows PowerShell (.Net in fact) uses with remote web server with following command:
After I used that command, it actually worked for this site. But I wasn't super happy about it as I either need to embed that line into my profile or know which sites needs the "trick". Forcing the TLS1.2 may also create other unexpected issues. There has to be a better way (yes, use PowerShell Core 6!) but here is my take on that challenge: Invoke-BetterWebRequest
The function is a wrapper around the original Invoke-WebRequest cmdlet. The function handles the exception (remember the bleeding red in the above screenshot?) and updates the [System.Net.ServicePointManager]::SecurityProtocol for us. When the first Invoke-WebRequest call in line 19 fails, we'll try changing the SecurityProtocol to TLS 1.2 and then Invoke-WebRequest again in line 25-26. Finally, we then revert the change. Here is the result. Looks better, eh?
What are your thoughts? What would you do to work around the problem? Let's chat on twitter!😁 In case if you are interested in the function:
Marc R Kellerman kindly shared his approach on twitter with me today:
[Net.ServicePointManager]::SecurityProtocol = [System.Net.SecurityProtocolType].GetEnumValues()
It works as we can pass an array of values to the Net.ServicePointManager. I think this is nicely done! Thank you so much Marc! Head over to Marc's twitter and blog for more good information!
With more websites moving to TLS 1.2 only, I find using Invoke-WebRequest from Windows PowerShell becomes less reliable because Windows PowerShell's defaults to using Ssl3 and TLS1.0. In this post I am going to share how I worked around the challenge in Windows PowerShell. Very importantly, if you can use PowerShell Core 6 for Invoke-WebRequest, do it. The cmdlet is much improved in PowerShell Core 6 thanks to Mark Karus.
So over the past weekend, I was learning about English and was experimenting to parse the Princeton university's WordNet site with PowerShell. However, I got this bleeding red from Windows PowerShell. 😫
I asked myself what does the "The underlying connection was closed: An unexpected error occurred on a send." mean? Then I dived a bit deeper into the $Error variable using $Error[0] | Format-List * -Force
While HTTP protocol and the .Net class are still over my head, I see the message contains "authentication" and "TlsStream". Interesting! With a quick web search and for sure the good people on the internet has suggestions. (Thank you!) Looks like we get to control the communicating protocol Windows PowerShell (.Net in fact) uses with remote web server with following command:
After I used that command, it actually worked for this site. But I wasn't super happy about it as I either need to embed that line into my profile or know which sites needs the "trick". Forcing the TLS1.2 may also create other unexpected issues. There has to be a better way (yes, use PowerShell Core 6!) but here is my take on that challenge: Invoke-BetterWebRequest
The function is a wrapper around the original Invoke-WebRequest cmdlet. The function handles the exception (remember the bleeding red in the above screenshot?) and updates the [System.Net.ServicePointManager]::SecurityProtocol for us. When the first Invoke-WebRequest call in line 19 fails, we'll try changing the SecurityProtocol to TLS 1.2 and then Invoke-WebRequest again in line 25-26. Finally, we then revert the change. Here is the result. Looks better, eh?
What are your thoughts? What would you do to work around the problem? Let's chat on twitter!😁 In case if you are interested in the function:
Comments
Post a Comment