The moment we’ve all been waiting for is finally there.  Windows Azure Infrastructure Services reached the General Availability milestone today.  This contains the following capabilities:

• Windows Azure Virtual Machines: you are able to have persistent virtual machines deployed in your Windows Azure subscription.  For this, you can bring your own (Linux or Microsoft) images or select them from the existing gallery.
• Windows Azure Virtual Networking: you are able to provision and manage virtual private networks (VPNs) in Windows Azure and securely connect them with your local IT infrastructure.

More information can be found on the blog of ScottGu: http://weblogs.asp.net/scottgu/archive/2013/04/16/windows-azure-general-availability-of-infrastructure-as-a-service-iaas.aspx

What is new with this release ?

There are some interesting new items available with the new release:

• High memory instances.  These are machines with much more memory assigned to it that can serve some specific workloads.  (these are not the high compute machines.  The names for these machines will be A6 and A7, instead of XXXL )
• Monthly SLAs.  There will from now on be SLA’s provided, that will measure on a monthly basis.  SLAs for Virtual Machines are 99.95% and require 2 instances.  For Virtual Networking, the monthly SLA will be 99.9%.
• More validated & supported Microsoft workloads.  There will be much more workloads and images added to the platform.  Also BizTalk Server 2013 will be available soon.
• The actual prices will start as from June 1, but are seriously decreased, also for PaaS images!

What does it mean for you ?

• From now on, you are able to have a real better together story that fits your preferences: cloud, on premises or hybrid.
• You can manage all these machines, using the System Center suite
• And you can share the same identity management through Active Directory and Federation Services
• There is no lock in.  This is a common concern for enterprises, but with these infrastructure services, you are now able to move virtual machines from the cloud and run them again in your own data center, because of the VM portability.
• You can now have both the software and the infrastructure layer provided by the same vendor: Microsoft.  This should help you in getting better quality and support through the wide network of Microsoft partners and local Microsoft teams.

What about BizTalk Server ?

As described in the MSDN article “Configuring BizTalk Server 2013 on a Windows Azure VM”, you can install BizTalk Server in Windows Azure infrastructure services.  Until now, these virtual machines were running in preview mode, meaning they were not officially supported or backed up by SLAs.  But from now on, you can have BizTalk Server officially installed and supported on Windows Azure.

Sam Vanhoutte

When trying to install SQL Server 2008 R2 on a Windows Server 2008 R2 Virtual Machine on Azure, you might come across the following error at the Setup Support Rules step:

This has to do with two manifest files that are corrupt.

If you have another working installation, you can copy the following two manifest files from the working server to the failing server:

• C:\Windows\winsxs\Manifests\amd64_microsoft.vc80.atl_1fc8b3b9a1e18e3b_8.0.50727.4053_none_8a1a02152edb659b.manifest
• C:\Windows\winsxs\Manifests\x86_microsoft.vc80.atl_1fc8b3b9a1e18e3b_8.0.50727.4053_none_d1c738ec43578ea1.manifest

If you don’t have another working installation, find and open the files mentioned above and paste the following xml in them:

• AMD64 Manifest:
  <?xml version="1.0" encoding="UTF-8" standalone="yes"?>
  <!-- Copyright © 1981-2001 Microsoft Corporation -->
  <assembly xmlns="urn:schemas-microsoft-com:asm.v1" manifestVersion="1.0">
  <noInheritable/>
  <assemblyIdentity type="win32" name="Microsoft.VC80.ATL" version="8.0.50727.4053" processorArchitecture="amd64" publicKeyToken="1fc8b3b9a1e18e3b"/>
  <file name="ATL80.dll" hash="99840dcc34e78af239d80841eba316c184e407cd" hashalg="SHA1"/>
  </assembly>
• X86 Manifest:
  <?xml version="1.0" encoding="UTF-8" standalone="yes"?>
  <!-- Copyright © 1981-2001 Microsoft Corporation –>
  <assembly xmlns="urn:schemas-microsoft-com:asm.v1" manifestVersion="1.0">
  <noInheritable/>
  <assemblyIdentity type="win32" name="Microsoft.VC80.ATL" version="8.0.50727.4053" processorArchitecture="x86" publicKeyToken="1fc8b3b9a1e18e3b"/>
  <file name="ATL80.dll" hash="6d7ce37b5753aa3f8b6c2c8170011b000bbed2e9" hashalg="SHA1"/>
  </assembly>

Note: You probably need to change the permissions for those files to be able to overwrite them.
To do so, go to the file properties, Security tab, Advanced, Owner tab, Edit, Select the Administrators group and click three times OK until all windows are closed.
Reopen the properties for the file, Security tab, Edit, Select the Administrators group and select Full Control.

Lately, a lot of CTP’s came available on the Windows Azure platform.  My goal is to try them all out.  I blogged on two of these technologies already:

• Using Windows Azure VM Role
• Azure connect - VPN as a service

This time, we’re getting on the data side, with SQL Azure Data Sync.  This data synchronization service is built on the Microsoft Sync Framework technologies. It provides bi-directional data synchronization and data management capabilities allowing data to be easily shared across SQL Azure databases within multiple data centers.

A very good and detailed overview on Data Sync can be found on the TechNet Wiki: http://social.technet.microsoft.com/wiki/contents/articles/sql-azure-data-sync-overview.aspx

Scenarios

This SQL Azure Data Sync service is a very promising and interesting feature that opens a lot of great scenarios:

• For development/testing purposes: provide data in your local database (on development) and synchronize with the Azure database to feed the cloud instances.
• Synchronizing configuration or master data across databases in a very easy way.
• Use SQL Azure Reporting Services on ‘local’ data.

Registration for CTP

Since this feature is still in CTP, you need to register first to get an invitation code.  To do all this, just browse to https://datasync.azure.com and sign in with your live id.  After this, you can enter your e-mail and other details and you should receive an invitation code some time later.  With that registration code, you can log on to the full features.

Concepts

The following concepts are important in setting up Data Sync:

• Sync group: a sync group is a group of databases that can be synchronized together
• Database: a database that gets registered for SQL Azure Data Sync
• Agent: an agent is a Windows Service that performs and orchestrates the actual synchronization

Configuring data synchronization

Step 1: download the Sync Agent

A Sync agent is only required when synchronizing data from an on-premise database.  If the synchronization is set up from Cloud to Cloud, then it is not required to download the Agent, since the synchronization logic will be run in the Cloud.

For this example, I will synchronize a local database with a SQL Azure database.

On the Data Sync portal, click the Agents tab page and download the Agent installer at the bottom of the page.  After running the installer (and providing credentials for the Data Sync windows service), you can open a local configuration tool, through the Start menu: SQL Azure Data Sync Agent CTP2.

This tool allows you to configure local databases for synchronization.  The following screenshot shows the default configuration window, after installation.  To join this agent to a Windows Azure subscription, it is important to configure the Agent Key first.  This can be done by clicking the Edit Agent Key button and providing the key that can be copied from the Data Sync Portal. 

Once this is configured, it should be possible to Ping the Azure Sync Service, by clicking the Ping Sync Service button.

Step 2: Add your local database

Now we have the local agent installed, we will add a local (on premise) database to the configuration tool, so that it can be synchronized later.

In this example, I am using a custom database with 3 tables: Customer, OrderLines, OrderEvents.

• I will add this database in the SQL Azure Data Sync Agent tool, by clicking the ‘Add Member’ button at the left.  This pops up a configuration window where the server, database and the authentication method need to be selected. (1) 
• The Data Sync services have some limitations on supported data types, etc.  It is possible to check the database for issues, by clicking the ‘Check Member Schema’ button. (2).
• I added a field with an unsupported data type (geography) to the customer table and the Schema validation provided a warning that indicated that this column would be skipped. (3) This is because geography is not supported in Windows Azure at this point in time.
• It is very important to start the Windows Service of the Azure Data Sync Agent, in order to register the client database with the Windows Azure subscriptions.

Step 3: Add the SQL Azure database

Adding a SQL Server database to the Data Sync subscription is much easier and can be done on the Data Sync management portal, by clicking the Databases tab and clicking the Add button.  In the pop up, you just need to provide the server, database and credentials, before saving. 

In my example, I am just adding a new empty database, here.

Step 4: Set up the synchronization, through a Sync Group

• In the management portal, click the New Sync Group button in the Sync Groups tab. And add all the databases you want to synch together to the database list(1) and click Next.
• In the next screen, you can add the tables, per database that you want to synchronize.  For this demo, I want to synchronize everything, except the events table. (2)  You can also enable a schedule for the synchronization (expressed in minutes).
• Once the group is created, you can synchronize the group easily.

Testing the data synchronization

Now I have the databases configured for synchronization, it’s time to play around with it a bit.  To execute the synchronization, you can either rely on the synchronization schedule , when configured on the Sync group, or you can manually trigger the synchronization.

1st synchronization: Creating tables in the cloud.

The first synchronization I did created the two synchronized tables on my empty Cloud database and added the data there.  One thing to notice is that the Location column (with the spatial data type) was not created on the SQL Azure database. 

2nd synchronization: Adding data locally, synchronizing to the cloud.

In this test, I added two customers and some order lines to the local database and synchronized with the Cloud database, to find out that everything was copied without problems.

3rd synchronization: Adding data in the cloud, synchronizing locally.

In this test, I added a customer to the cloud database and synchronized with the on premise database, to find out that everything was copied without problems.

4th synchronization: Adding data on premise and in the cloud, synchronizing bi-directionally

In this test, I added customers in both databases before synchronization to find out that the data was synchronized correctly. 

Data conflicts

Now, I wanted to simulate some conflicts to find out how the Data Sync would handle them.

Adding or updating data in both databases, with the same primary key.

I added two different customers, but with the same primary key in both databases.  But, surprisingly, the synchronization happened without any issue, but my local customer was overridden by the cloud customer, resulting in lost data. 

• Both records (local + cloud) were normally marked as new, so it should be possible to detect this.
• I was looking to find out if the ‘most recent’ record won, but in all scenarios, the data from the cloud database survived the synchronization.  (probably, because this database was added first)

Maybe something for a next CTP?

Deleting data that is being referenced by a non-synchronized table

Another interesting test was to delete an order line on my cloud database.  But this order line was being referenced by a record in my local database (OrderEvent).  Knowing that the OrderEvent table is not being synchronized, this should result in a conflict.

Here I did not receive an exception, but I also noticed that my record on my local database still existed, where my cloud record was deleted.  So here my data was out of synch. 

Maybe something for a next CTP?

Adding a third database

The last test I did, was adding a new empty (cloud) database to the sync group and after the synchronization, everything seemed to work automagically.  Great!

Underlying design

• When configuring a table for synchronization, triggers are being created for the three actions: insert, update, delete.  They have the following name structure: [TableName]_dss_[action]_trigger.
• These triggers add the data in new tables that are being created during configuration of the sync group.  For every data table, a sync table is being added with the following name: DataSync.[TableName]_dss_tracking
• Next to that, we can also see that a lot of new stored procedures are getting created

Conclusion

This CTP looks already very stable and the sychronization (between multiple databases) is very smooth.  I am just hoping that there will be a better view or configuration for synchronization exceptions (like explained in the conflicts section).

Sam Vanhoutte, Codit

Great news for the Azure community.  The Windows Azure SDK and Windows Azure Tools for Microsoft Visual Studio (November 2010) have been released!  This release contains a lot of new features and most of them were announced at PDC.

The release can be downloaded here.

The overview of new features:

• Virtual Machine (VM) Role (Beta):Allows you to create a custom VHD image using Windows Server 2008 R2 and host it in the cloud.
• Remote Desktop Access: Enables connecting to individual service instances using a Remote Desktop client.
• Full IIS Support in a Web role: Enables hosting Windows Azure web roles in a IIS hosting environment.
• Elevated Privileges: Enables performing tasks with elevated privileges within a service instance.
• Virtual Network (CTP): Enables support for Windows Azure Connect, which provides IP-level connectivity between on-premises and Windows Azure resources.
• Diagnostics: Enhancements to Windows Azure Diagnostics enable collection of diagnostics data in more error conditions.
• Networking Enhancements: Enables roles to restrict inter-role traffic, fixed ports on InputEndpoints.
• Performance Improvement: Significant performance improvement local machine deployment.

This blog shows a first highlight of the new features that are part of the new SDK.

Extra small instance size

One of the nice features that many people have been asking for is the introduction of a new, lightweight, VM size: the Extra small instance.  They come with a small amount of memory (786 MB) and a low CPU speed (1Ghz), but at a very low cost (5ct / compute hour).  Configuring this is very easy, through the Role Configuration tab.

Remote desktop features

It is now possible to connect to an Azure instance, by using a Remote Desktop Connection.  The publishing wizard has been updated to enable this easily.  When deploying an application, you can provide a user name and a password for a remote desktop user, together with a certificate that has to be uploaded to the Azure portal, after exporting it with the private key.  (Export certificate - Browse to the portal – Your project – Installed Certificates – Manage – Upload certificate)

In the new Azure management portal, it is now possible to connect to a specific instance of an Azure Role, by clicking on the Connect button.

Public ports in endpoints

In previous SDK’s, it was unknown until at runtime at what actual port the role endpoints were hosted.  This can now be fixed and configured in the endpoints tab page on an Azure role:

Using Azure Connect

The Azure Connect feature is a very interesting feature that allows to set up a virtual network between Azure roles and the local network.  The property page of the Azure role allows to configure the specific token that is need for the Azure Connect feature.  This feature will be opened for a beta program soon, so we will be able to blog about it as soon as we are allowed in the beta program.

Sam Vanhoutte, Codit

Early Thinking

This post describes some new Azure features and how they might help us in leveraging the power of the web/app.config files again.  The content is this article is written, before any CTP/Beta versions of these features are available, so the details might be incorrect and there might be some wrong assumptions.  I will verify all of these parts, once the features are made available.

Current situation

Until now, an important difference between deploying a web application (or web service) on a local or hosted server and deploying a web application (or web service) to Windows Azure was the usage of the web.config file.

In a standard web application (hosted on IIS), the web.config can easily be changed to update configuration settings.  When changing this file, the web application gets refreshed and the new settings are being applied.

On Azure, there was a total different story.  The web.config was compiled with the deployment package and to change it, a new version of the packaged needed to be uploaded, resulting in a lot of work/time before the new settings got applied.  Luckily, Windows Azure provided a nice alternative: the serviceconfiguration files (.cscfg).  This file allowed to specify configuration values that could be loaded at runtime through the following code:

RoleEnvironment.GetConfigurationSettingValue("myValue"); code.

Painpoints with the Azure serviceconfiguration files.

The idea is great, having service configuration files managed on Role level and not on instance level.  But there are some big disadvantages in the following scenarios:

WCF Configuration settings

WCF has tons of configuration settings and I am a big fan of having my service endpoints configured through the .config files.  This allows for easy change and full declarative configuration, without having the settings (binding/behaviors…) compiled with the program.  And since the introduction of the WCF Router service, even more configuration (the pub/sub logic) gets in the .config file.

Having all these settings in the Azure .cscfg file, would be very hard to maintain, or we’d need to switch to custom configuration files.

Custom configuration settings

In most of the applications that are developed at Codit, we use custom configuration sections to have a better structured configuration experience.  These configuration sections are complex types and no standard name/value configuration pairs.  Migrating these applications to Azure, would mean our configuration logic would need to be totally redesigned.

ASP.NET settings

ASP.NET also include a lot of settings that might need to be changed at runtime.

PDC announcements

At PDC a lot of stuff was announced that can help us here.  This article described the various announcements and their impact on the configuration issues I described above.

Remote desktop capabilities

Through the updated Azure management portal (didn’t that look a 100 times nicer than the current one?), we will soon be able to have Remote Desktop access to our web.  This will be done by adding the following Modules in the service definition file (.csdef)

<Imports>

<Import moduleName="RemoteAccess" />

<Import moduleName="RemoteForwarder" />

</Imports>

This allows us to connect to a running instance of an Azure role.  This way we can alter the web.config file and have everything updated as on a regular web application.

The biggest disadvantages of this approach are the following:

• When having multiple instances (which is required to have High Availability), we need to remote in on every running instance of our web role.
• Our changes will be undone when our instance restarts or gets redeployed, because the machine gets ‘prepped’ again, resulting in a fresh installation of our Azure package.

The new Azure VM Role

Through the announced Windows Azure VM Role, we will be able to customize our own Virtual Machine, by uploading a pre-built virtual hard disk (VHD) to a blob storage account.

The steps to do this are the following :

• Install Windows Server 2008 R2 Enterprise on the VHD
• Add any Additional software you might need on the VM Role instances
• Install Windows Azure Integration Components that include the following components
  • Agent
  • Runtime interface (topo, config, shutdown notification)
  • Remote Desktop configurator
  • Windows Azure Drives driver
• Generalize the VHD is recommended, by using sysprep  (machine name will be assigned by Azure.)

By using a difference VHD, it might be easy to upload a new version of that VHD with the updated configuration file on it and have the changes applied.  Besides that, the same approach can be taken as described in the previous section (remote desktop capabilities).

The biggest disadvantages of this approach are the following:

• To upload a new version of the VHD difference disk, the role might need to be reinitiated (to be tested, once the VM Role is available for CTP)
• Cumbersome way of changing the web.config.

Startup tasks

As shown in the CloudCover show of two weeks ago (watch episode), there is a possibility of executing startup tasks, when the role instance starts up.  This looks like the best suited feature to solve our config file problem.

I am making one crucial assumption here (that I’ll check once this is made available):  I believe the start up tasks are being executed prior to the Role Instance initiation – so before the web application is started.  This means we can still change the web.config without having the web application being recycled automatically.

Leveraging the startup tasks will allow us to take the web.config from a configurable blob container and write it to the application directory on the Azure instances.  Therefore, we just need to write a script that performs this logic.  (code will be in a future post).  Then we just need to configure that script in the Startup element in the csdef configuration file:

<Startup>
     <Task commandline="updateconfig.cmd" executionContext="elevated" 
               taskType="simple/foreground/background" />
</Startup>

This approach looks like the best approach to make our web.config and app.config files configurable again.

Conclusion

A lot of uncertainties, because there is no bit available yet to test this with.  So, please expect a reviewed post, once it has been made available. 
But one thing is for sure: the new Azure features will make Azure application development much more flexible again.

Sam Vanhoutte, Codit