When I deploy software, I’m lazy. Very lazy. This is why I lean heavily on Continous Deployment (CD) to automatically test and deploy software when it’s merged into my main branch. I don’t have time to deploy code by hand. So gauche!
Of course this requires a lot of trust in my automation and testing before merging code into main. But the overall time savings and agility gained through CD are tremendous!
It is possible to have a single production database shared by staging and production Azure App Service slots, and still having zero-downtime deployments. To do this, you need to ensure that all your migrations are backwards compatible, such that the current and new versions of the web app can run simultaneously with the same database. Deploying to Azure Static Websites For my website I use Azure Static Websites as a cheap host. It turns out that there's a GitHub Action already made for this, feeloor/azure-static-website-deploy which makes things super easy to deploy.
It’s possible to set up a continuous blue-green deployment process on Azure using deployment slots.
I did this recently, but ran into a subtle gotcha when I tried to set up a custom warm-up with my ASP.NET Core site. A custom warm-up is a URL that Azure will hit after code is deployed to the offline deployment slot or slots. This gives the app a chance to run custom warm-up code before the slot is swapped into production.
The documentation mentions two app settings you can use to set up a custom warm-up:
WEBSITE_SWAP_WARMUP_PING_PATH: The path to ping to warm up your site. Add this app setting by specifying a custom path that begins with a slash as the value. An example is/statuscheck. The default value is/.WEBSITE_SWAP_WARMUP_PING_STATUSES: Valid HTTP response codes for the warm-up operation. Add this app setting with a comma-separated list of HTTP codes. An example is200,202. If the returned status code isn’t in the list, the warmup and swap operations are stopped. By default, all response codes are valid.
Seems straightforward. So I set something up like this:
WEBSITE_SWAP_WARMUP_PING_PATH:/statuscheckWEBSITE_SWAP_WARMUP_PING_STATUSES:200
Afterwards, all my deployments failed. I used curl to confirm my /statuscheck page returned a 200 code.
I tried running the swap manually from the Azure Portal and it gave a very unhelpful error message. First of all, the error message contains the following string:
Cannot swap slots for site '{0}' because the application initialization in '{1}' slot either took too long or failed.
It looks like someone forgot to pass parameters to string.Format.
Second, it mentioned a Bad Request: 400 status code.
I then looked in my logs and didn’t see any 400 status code requests. Instead, I saw some 307 Temporary Redirects. I looked more closely and noticed requests for:
http://mysite.example.com/statuscheck
Notice the problem? The Azure warmup ping request is making HTTP requests, not HTTPS requests. And you can’t change this. The setting WEBSITE_SWAP_WARMUP_PING_PATH does not allow a fully qualified URL. It will always make an HTTP request.
The problem for me is that I have the following code in my app startup (Startup.cs).
You might recognize that UseHttpsRedirection method call from the default ASP.NET Core project templates. That adds middleware to redirect HTTP requests to HTTPS.
On top of that, I usually configure Azure Web Applications to be HTTPS only via the TLS/SSL settings. This is why the Azure swap warmup request fails. It expects a 200 but is getting a 307 and doesn’t follow the redirect.
In a useful troubleshooting blog post, the author mentions using rewrite rules to solve this. Not sure that’s going to work for my site since I don’t even have a web.config file and I’m not using rewrite rules to enforce HTTPS in the first place.
Fortunately the solution is straightforward. First, I went to the TLS/SSL settings for my site in the Azure Portal and turned HTTPS Only to Off.
Then I branched my request pipeline using the UseWhen extension method.
In my actual code, I wrote an extension method like so:
That way I could simplify the code in my startup class to:
UseWhen takes two parameters. The first is the condition. In this case, when the request is NOT a status check request. The second is the code to call when that condition is true, in this case it registers the HTTPS redirect middleware.
In effect, this code forks my app pipeline into two paths. For requests to /statuscheck HTTP requests are allowed. For all other requests, we continue to redirect HTTP to HTTPS.
Problem solved!
What’s particularly subtle about all this is if you set up WEBSITE_SWAP_WARMUP_PING_PATH but do not set up WEBSITE_SWAP_WARMUP_PING_STATUSES, the warm-up request by default accepts all status codes, including 307. So your deploys will pass, but it may not necessarily be done with the warm-up as it’s unclear if Azure follows the redirect in this case.
This was a frustrating experience, but I got it working in the end. My hope is that if you have a similar setup to me, this will save you time and headaches.
UPDATE: My deploys still fail. If I specify WEBSITE_SWAP_WARMUP_PING_STATUSES = 200 my deploys still fail even though I can use curl to see the status check URLs return 200. I’m at a loss here and I’ve opened up a support ticket with Microsoft. I’ll post back here with what I learn.
UPDATE: Resolved the issue. I tried to be clever and pass a query string parameter to my warmup URL. For example, by setting WEBSITE_SWAP_WARMUP_PING_PATH to /statuscheck?azure-warmup=true. This way I could quickly distinguish my own testing of this URL from the request made by Azure in my logs. Upon reflection, of course this doesn’t work because Azure URL Encodes WEBSITE_SWAP_WARMUP_PING_PATH before requesting it. So the request path is /statuscheck%3Fazure-warmup=true and not /statuscheck as I expected.
Conclusion
In the end, I figured out what I was doing wrong and it’s easy to chalk it up to PEBCAK (Problem Exists Between Chair And Keyboard). While that may be true, it’s also true that the feature could improve its usability to reduce PEBCAK occurrences.
Azure Deployment Slots Database
Sebastian Ros on Twitter suggested a couple of improvements:
- It should allow for https or accept a redirect to the https endpoint
- It should not encode the URL you passed
Azure Static Website Deployment Slots Software
I definitely agree with the first suggestion. Otherwise everyone deploying a standard ASP.NET Core app will run into this and my not even realize it!
My preference would be to follow the redirect so IT JUST WORKSTM. I understand this is not trivial and requires a bit of thought. For example, it can’t blindly follow the redirect. The initial request is made internal to Azure, bypassing (as I understand it) DNS and the load balances etc. So it would need to be smart about following the redirect. If the redirect is to the same endpoint, but only the scheme is changed, then re-make the original request with HTTPS.
Might be easier to add a radio button group to select between HTTP and HTTPS. Either way, the current behavior is a hidden gotcha.
As for the second suggestion, I also agree. It would require validation of the user input. For example, if I put something that isn’t allowed for the end of the URL, it should present an error message. Alternatively, have two fields, one for the path and one for the query string. Either way, the current behavior is also confusing and a hidden gotcha.