Calling Python from PHP in Azure Websites

I was recently asked for help by an ISV. They needed to generate a SAS (Shared Access Signature) to grant time-limited access to resources that they were storing in Azure Blob Storage. They had developed their solution using PHP and deployed it to Azure Websites. I had generated SAS before using C# and .NET but never using PHP. No problem I thought … we have an Azure SDK for PHP and it’s available on GitHub !

Azure SDK for PHP

But after pouring over the code I came to the realisation that there is no functionality to generate a SAS for Blob Storage in the PHP SDK. It seems as though this was first opened as an issue in 2012 but has not been resolved. So what to do?

I didn’t want to implement a solution that would place the burden of keeping up to date with Azure changes on the ISV. I had a quick look at the Python SDK and the weight lifted off my shoulders when I found sharedaccesssignature.py !!

Azure SDK for Python

I verified my approach with the Azure SDK team and got the thumbs up. I was good to go and I could sense the solution would be a matter of minutes away – little did I know …

Test Azure Website

I created a test Website in Azure called callingpythonfromphp and ensured that PHP was enabled (it is by default). I didn’t need to change the Python version setting since Python is installed by default (as are all the other languages). The switches you can see below enable http handlers for the respective languages.

Site Settings for callingpythonfromphp Website

You can check that Python is installed via the Debug Console in Kudu. Here you can see that Python 2.7 is available in D:\Python27.

Kudu - console

I added two files to the D:\home\site\wwwroot folder of my Website – generate-fakesas.py and test-fakesas.php.

Listing of wwwroot

The test-fakesas.php file simply called Python and passed it the generate-fakesas.py script to execute.

<?php echo ">> "; system('D:\Python27\python.exe generate-fakesas.py'); echo " <<"; ?>;

The generate-fakesas.py was so named since all I’m doing is returning the string “SAS” and not actually generating a SAS. This will be output in the PHP script. In the final version it can be assigned to a PHP variable and utilised.

print ("SAS")

This was a quick test to check whether or not I could call a Python script from PHP in an Azure Website. The >> and << in the PHP script would allow me to visually confirm that the output of the Python script had been inserted where I expected.

Hitting the test-fakesas.php script via the Debug Console in Kudu gave me the expected result. I could see the SAS being output. I had successfully called into Python from the PHP script.

Execute PHP script

I then hit the test-fakesas.php script from the browser. Hmm – this was not good. The SAS string was nowhere to be seen.

Calling php script from Website

Looking at the php_errors.log file in D:\home\LogFiles I found the following:

PHP Warning: system(): Unable to fork [D:\Python27\python.exe generate-fakesas.py] in D:\home\site\wwwroot\test-fakesas.php on line 1

Errors in php_errors.log

The Fix

So there was a difference in behaviour between the Debug Console and the manner in which php-cgi was being launched. After a few emails back and forth to the Azure Websites team, they discovered that the source of this issue was to do with the fastcgi.impersonate PHP setting. It was set to 1 by default and needed to be switched off (set to 0). This resulted in the following solution on the Kudu Xdt Transform Samples wiki page.

Custom php.ini

This solution basically allows you to deploy a custom php.ini file for your Azure Website and override settings. When you are doing this ensure that ALL instances of fastcgi.impersonate=1 are changed to fastcgi.impersonate=0 and are NOT commented out.

Note that the applicationhost.xdt and php.ini file should be deployed to your D:\home\site folder.

Solution deployed

And now the call works from both the Debug Console in Kudu and the browser.

Finally - it works !

Ok – so that’s all great. But I actually needed to generate a SAS.

Generating a genuine SAS

DISCLAIMER – I’m going to preface this entire section by saying that I am not a Python guru.

The recommended mechanism for installing Python packages is pip but when I ran pip via the Debug Console in Kudu to install the Azure SDK …

D:\Python27\Scripts\pip.exe install azure

I got the following error.

error: could not create 'D:\Python27\Lib\site-packages\azure': Access is denied

I found that the only way I could install the Python Azure SDK via pip into the Azure Website was by starting off with Python’s virtualenv. I found a great blog post that got me up and running quickly. I created a new development environment called myapp.

D:\Python27\Scripts\virtualenv --no-site-packages myapp

Create development environment

This creates an entire new and isolated Python environment and copies utilities and the Python executables into this environment.

Next activate the development environment.

myapp\Scripts\activate

Activate development environment

And then install the Azure SDK.

pip install azure

Install Azure SDK

I added the test-sas.php file to the D:\home\site\wwwroot\folder of my Website and the  generate-sas.py file to the D:\home\site\wwwroot\myapp folder .

Python and PHP scripts

The test-sas.php file simply called Python and passed it the generate-sas.py script to execute. You can see that it is using the Python executable in the myapps development environment. This means it will also have access to the Azure SDK I installed.

<?php echo ">> "; system('myapp\Scripts\python myapp\generate-sas.py'); echo " <<"; ?>;

The generate-sas.py was based on an example from StackOverflow. The script now will actually generate a SAS for read-only access to the images/flower.png blob within my imagesstoragepb storage. This SAS will be output in the PHP script.

from azure.storage import *
accss_plcy = AccessPolicy()
accss_plcy.start = '2015-01-08'
accss_plcy.expiry = '2015-01-10'
accss_plcy.permission = 'r'
sap = SharedAccessPolicy(accss_plcy)
sas = SharedAccessSignature('imagesstoragepb', 'DPnYqQIKTbNn6iC+nH03wjbvHcpm9htIYVZYkGQJEaWhUbEaGj6koypC0R8AW5Zc6L/g8Sj1tmTPMQlWY8m+NQ==')
qry_str = sas.generate_signed_query_string('images/flower.png','b', sap)
print (sas._convert_query_string(qry_str))

Running this in the browser led to the successful generation of the expected SAS !!

Generated a SAS from PHP via Python in Azure

So don’t be put off if a feature you need is not available in the PHP SDK. Check if you can leverage the Python SDK. And if you need to override the default PHP settings in Azure Websites you also now have a mechanism to do that.

Love at first site – scriptcs and WAML at Brisbane Azure User Group

I gave a talk at the Brisbane Azure User Group on the 13th November 2013 titled Love at first site – scriptcs and WAML. Here is my slide deck.

scriptcs is putting C# on a diet and decoupling your favourite language from Visual Studio. The Windows Azure Management Library (WAML) is a C# library that wraps the Windows Azure Management REST APIs. These two belong together !

In this talk I introduced scriptcs and the Windows Azure Management Library, before showing how to combine these two awesome resources to script the management of your Windows Azure assets with the full power of C#.

First Look at Built-in Autoscaling and Alerting at Brisbane Azure User Group

I gave a talk at the Brisbane Azure User Group on the 21st August 2013 titled First Look at Built-in Autoscaling and Alerting. Here is my slide deck.

image

Autoscaling has finally been built in to Windows Azure via Microsoft’s acquisition of MetricsHub. The Autoscaling funtionality from MetricsHub has been rolled directly into the Windows Azure platform. Other features that have also been rolled into the Windows Azure platform from MetricsHub include Availability Monitoring and Alerting.

Win an Aston Martin with your MSDN Subscription!

Most Microsoft developers have an MSDN subscription. Yet not many know that you get up to $150 Windows Azure credits per month with your MSDN subscription. Activate the Windows Azure benefits included with your MSDN subscription and you could win an Aston Martin V8 Vantage !

Simply activate your Windows Azure MSDN benefit before 30 September 2013 and deploy at least one Web Site or Virtual Machine. What are you waiting for ?

Activate your Windows Azure MSDN Benefit !

Still need convincing ?

You no longer need to provide credit card details when activating your Windows Azure MSDN benefit !

MSDN Professional Subscribers receive $50/month worth of Windows Azure monetary credits, MSDN Premium Subscribers $100/month, and MSDN Ultimate Subscribers $150/month. These credits can be applied towards any Windows Azure resource being used for Dev/Test purposes. It is up to you to decide how you would like to use them.

Here are some examples of how a MSDN Premium Subscriber could use their monthly monetary credits.

image

Get started now ! Activate your Windows Azure MSDN benefit.

Why you’ll love Windows Azure SDK 2.0 at the Brisbane Azure User Group

I gave a talk at the Brisbane Azure User Group on the 15th May 2013 titled Why you’ll love Windows Azure SDK 2.0. Here is my slide deck.

Why you'll love the Windows Azure SDK 2.0

This major refresh of the Windows Azure SDK was released on 30th April with some really great new features and enhancements.  In the talk I explored the new capabilities in version 2.0 using Scott Guthrie’s post and Damir Dobric’s excellent Service Bus 2.0 post and sample code as guidance.

We looked at the following:

Web Sites
Improved Visual Studio Tooling around Publishing
Management Support within the Visual Studio Server Explorer
Streaming Diagnostic Logs

Cloud Services
Support for new High Memory VM Instances
Faster Deployment Support with Simultaneous Update Option
Visual Studio Tooling for configuring, managing and viewing Diagnostics Data

Storage
Storage Client 2.0 included in New Projects
Visual Studio Table Explorer

Service Bus
Updated Client Library
Message Browse Support
New Message Pump Programming Model
Auto-delete for Idle Messaging Entities

PowerShell
PowerShell 3.0
PowerShell Remoting
New Automation Commands for Web Sites, Cloud Services, Virtual Machines, Service Bus, Windows Azure Store, Storage, Scaffolding cmdlets for Web/Worker Role

Understanding the benefits of Windows Azure geo-redundancy in Australia

The Windows Azure family has been extended with deployment regions in Australia. The deployment regions are paired for geo-redundancy and in Australia are located in the New South Wales and Victoria sub-regions.

Windows Azure data centres

Why is geo-redundancy important ?

Windows Azure Storage is an important service that underpins a number of other Windows Azure services – Blob Storage, Table Storage, and Virtual Machine (OS and Data Volumes).

Geo-replication is enabled by default in Windows Azure Storage and provides the highest level of storage durability. Data is asynchronously replicated from your primary location to a secondary location within the same region. These locations are guaranteed to be at least 400kms apart to ensure that data durability across catastrophic events.

The following picture shows the New South Wales and Victoria sub-regions within the Australia geographical region. Here the NSW deployment region is the primary location and is asynchronously replicating data to the Victoria deployment region which is the secondary location.

Geo-redundancy

In the event of a catastrophic event in the primary location where the primary location cannot be restored, all traffic will be failed over to the geo-replicated secondary location. This ensures business continuity.

What about redundancy within the deployment region ?

Within each deployment region there is an additional layer of redundancy. All data is synchronously replicated to 3 different storage nodes across 3 separate fault and upgrade domains. This allows each deployment region to recover from common issues such as disk, node or rack failure.

The following picture demonstrates the 3 copies across 3 separate racks and the creation of a new copy after the failure of a storage node.

image

When geo-replication is enabled, you will effectively have 6 copies of your data distributed across 2 geo-graphically dispersed deployment regions. The multiple layers and mechanisms ensuring highly durable data will provide business continuity across a number of scenarios.

How do these new deployment regions affect Australian businesses ?

The most obvious answers would be reduced latency and data storage within Australia.

Even though a Windows Azure CDN has been available out of Sydney for a while, it has only offered lower latency on Blob Storage for very specific read-only workloads. With an Australian region available now, lower latency is available to a wider range of Windows Azure services and this will benefit Australian businesses utilising the Windows Azure platform.

Content in Blob Storage that is produced and consumed within the local Australian market will benefit from the lower latency. A more compelling case can now also be made for cloud integrated storage solutions such as StorSimple. Those customers within Australia that have regulatory pressures preventing them from storing data outside of Australia will also be heartened and this announcement should remove this hurdle.

Australian businesses utilising Virtual Machines within the Windows Azure Infrastructure Services will also enjoy lower latency when connecting via Remote Desktop or SSH.

This is a truly exciting announcement that will hopefully see more Australian companies begin their journey into the cloud.

Blob Storage Deep Dive at Brisbane Azure User Group

I gave a talk at the Brisbane Azure User Group on the 16th January 2013 titled Windows Azure Blob Storage – A Deep Dive. Here is my slide deck.

Windows Azure Blob Storage - A Deep Dive

The deep dive covered the basics before diving into advanced, performance and security based topics. Fiddler was used extensively to show the actual REST API calls generated by the C# storage library.

I have pushed the demo code to GitHub and will be starting a series of posts where I will walk through each of the demos and the HTTP traffic observed via Fiddler.

image