System.Nerd.John.Blog

I was recently trying to ressurect an older project developed in Windows XP, .NET Framework 2.0, Visual Studio 2005, NHibernate, and SQL Server CE 3.1.  I’ve sinced moved to Windows 7 (64-bit) and Visual Studio 2008.

I ran into a surprising number of hurdles while trying to get the application up and running again on 64-bit Windows 7.  I figure I would document this here, just in case anyone else runs into the same issues.

Step 1) Try to build the solution.  Everything builds fine after installing SQL Server Compact Edition.

Step 2) Try to run the application.  Get an exception immediately:

“Could not create the driver from NHibernate.Driver.SqlServerCeDriver.”

InnerException:

“The IDbCommand and IDbConnection implementation in the assembly System.Data.SqlServerCe could not be found. Ensure that the assembly System.Data.SqlServerCe is located in the application directory or in the Global Assembly Cache. If the assembly is in the GAC, use <qualifyAssembly/> element in the application configuration file to specify the full name of the assembly.”

Turns out the issue here is that the System.Data.SqlServerCe dll has to be in the same folder as the application executable.  Pretty easy fix – set Copy Local to ‘True’ on the reference to System.Data.SqlServerCe.

Step 3) Run the application again – now I get a different exception:

“Unable to load DLL ’sqlceme35.dll’: The specified module could not be found. (Exception from HRESULT: 0×8007007E)”

Turns out the issue with this exception is that SQL Server Compact Edition is built for x86 and has to run in WoW mode on x64 systems.  My solution platform is set to ‘Any CPU’, which worked fine when I was developing on Windows XP.  To fix the issue, go through all of the Visual Studio projects – go to Properties > Build > Platform Target, and set Platform Target to ‘x86′ instead of ‘Any CPU’.

Step 4) Try to run the application again… and I get yet another exception:

“ADOException: cannot open connection” with InnerException of:

“The database file has been created by an earlier version of SQL Server Compact. Please upgrade using SqlCeEngine.Upgrade() method.”

This is kind of annoying – the Visual Studio 2008 Upgrade Wizard changed all my references from SQL Server CE 3.1 to SQL Server 3.5.  How thoughtful.  Unfortunately, I don’t know what the implications of ‘upgrading’ the database are.  Everything worked fine with 3.1 – why introduce any more change to the application?  So, I set the references back to SQL Server CE 3.1 instead of 3.5.

Step 5) Run the application… again.

No exceptions! Everything works with SQL Server 3.1! Upgrade complete.

One thing that any web developer worth their salt should know is the basics of search engine optimization (SEO).  Much of SEO comes down to basic code-level best practices, and it isn’t terribly difficult to simply bake SEO into your development process when working on public facing web applications.  However, keep in mind that SEO will always be an evolving, fuzzy science, changing on the whim of the indexing strategies of major search engines.  Immediate results are rare, and a long term process should be in place to truly understand the benefit (or detriment) incurred.

I break the concept of SEO down into a few categories that I’ll explain further below…

1. Content SEO (internal factors)
2. Strategic SEO (external factors)
3. Insight and Tracking

Content / Internal SEO

These ‘Content / Internal’ best practices are things that a developer or content creator can bake in during the site development process.  Only a few of these items will make a difference on their own, but as a whole can make an enormous impact.  These basic factors should lay the foundation for any SEO strategy. However, these internal factors absolutely cannot be the only part of your SEO strategy.  Here are a few of the most important ones…

• Page Titles.  Arguably one of the most important content level factor, this is one of the few that can make an enormous difference on their own.  Your page titles (what goes in the HTML <title> tag) should be relevant to what is on the page whereas I often come across page titles that only contain the name of the site.  Instead, you should have the ‘title’ of the page prefixed or appended to the name of your site.  Some believe that appending the name of your site to the page title is better than prefixing.
• Page URLs.  This goes hand in hand with your Page Titles, as page URLs carry almost equal important.  The URLs of your pages should mirror closely the titles of your pages, but don’t need to be exact.  Popular opinion is that the closer keywords in your URL are to the end of your domain name, the better.  Search engines have a very ‘human’ behavior in this case… tell me, which URL is more descriptive about this post? – “http://jsprunger.com/search-engine-optimization-101/” or “http://jsprunger.com/?p=88″.  Search engines think the same way.
• Freshly updated and unique content.  The more your web site content is updated, the more often it will be indexed by search engines.  Sites with freshly updated content seem to get a bonus from most search engines.  Bloggers in particular should ensure that their sites are configured to ‘ping’ a service like Ping-o-Matic whenever you create a new post, this will immediately notify Google and many other services of your new content.  Having unique content is perhaps one of the most important factors, simply rehashing or copying content will get you absolutely nothing from most major search engines – in fact, duplicate content can seriously hurt your rankings.
• Keyword usage in your content.  Also a highly important factor, whatever keywords that you want to rank for – make sure you’re using them in your content.  Think about what your customers or clients going to search for.  A few guidelines for keyword usage…
  • Don’t overuse your keywords, don’t be spammy.  Find the right balance between keyword usage and having readable, engaging content.
  • Make sure you have your keywords in your page title and URLs.
  • Use keywords within the first 100 words of the page or within HTML headers.
  • Get your keywords used in external links to your site.  More on this later…
• Image alt tags.  This is a pretty minor SEO factor, but very important if you have any interest in getting results from services like Google Image Search.  The productivity from image search results is usually pretty low for most businesses, but every little bit can help sometimes.  Some web sites (i.e. e-commerce, product catalogs) can benefit from image search much more than others.  Make sure you have descriptive ‘alt’ attributes on your <img> tags – this is a best practice for usability and accessibility in general though.
• Meta keywords and descriptions.  Long gone are the days of meta tags being useful for SEO.  However, the meta description tag can still play a huge role in your pages getting click-through from the search results. Google will use the meta description of your page as the ‘teaser’ for the search result, but if you’re missing this tag you’ll often just see garbage or irrelevant content for the teaser.  Users are much more likely to click through to your content in search results if the result description is accurate and compelling.
• Updated Sitemap and sitemap.xml file.  Keeping an up to date listing all of the content on your site in a sitemap will greatly enhance the ability of search engines to properly index 100% of the content on your web site.  You can use a tool like the Google Sitemap Generator to keep a continually updated sitemap file.
• Avoid so-called ‘black hat’ or any sort of sneaky SEO techniques.  These strategies usually revolve around hiding or cloaking text on your pages in an attempt to fool search engines.   It isn’t worth it – leading search engines can easily detect and adapt to these techniques, resulting in your search rankings taking a dive or even a complete blacklisting of your site.

Strategic / External SEO

Strategic SEO includes all of the factors external to your website that can affect your search engine rankings.  The number one external factor is getting ‘backlinks’ to your content, this is what made Google so ridiculously powerful and accurate – and their rankings are still very much based on the number, diversity, and quality of links to to your site.

Backlinking can be explained with this anecdote: Several years ago you could search for ‘Miserable Failure’ on Google and the number one result was the White House biography page for George Bush.  This was due to a simple viral campaign to get people to put links on their websites, comments, blog posts, etc. linking to the biography page with the anchor text ‘Miserable Failure’.  That’s how backlinks work.  The more external, inbound links to your site, the more ‘authoritative’ your site appears to be in the eyes of major search engines.

But how can you get these backlinks? A few examples…

• Mainstream media and press releases.  Old fashioned, but if this is relevant to your industry, press releases for important announcements make their way around the internet very quickly.  This obviously works best if the press releases link back to your web site.
• Getting linked and promoted in blog posts.  Do your friends, colleagues, or business partners have blogs or websites? Ask or barter with them to promote your content, requesting specific keywords be used in links to your web site.  This is a two way street – the more you’re willing to promote content from other sites, the more they’ll be willing to promote you back.  However, popular opinion is that one-way links are deemed to be of higher quality in the eyes of major search engines.
• Twitter (annoying or ridiculous as many believe it to be) can be a great way to spread the word about your content.  Maybe you’ll get lucky and someone with 30,000 followers will retweet your link if you’ve included the proper hashtags.  After this happens, you’ll start to see your links pop up all over the internet.
• Social Bookmarking.  Submitting your content to social bookmarking sites like Del.icio.us, Reddit, and Digg or more niche-specific sites can be a great way to spread the word about your content.  These services will also often directly link to your content with the exact text that you’ve specified – bonus!  Don’t be a spammer though, if you have high quality, unique content that people actually want to see – submit it.  If not, don’t bother.
• Make it easy for your readers to submit your content to social bookmarking sites, for example – drop an AddThis button on your website like the one at the top of this post.  This allows your users to easily link and promote your content if they find it valuable.
• Targeted submissions.  Do you have niche content? Find targeted venues for submitting your content and articles. For example, you’ve written an article relevant to the Healthcare industry. Track down some Healthcare industry groups on LinkedIn and submit your article to the news sections. Contact industry publications, they’re often happy to include high-quality articles.
• Alliances and partnerships.  Work with your business partners and allies to cross promote each other where applicable.  For example, you’re a partner for a specific vendor. If you work closely with that vendor, they’re often more than happy to promote their most capable partners by linking them on pages within their own websites.

Insight and Tracking

As mentioned previously, part of SEO includes a process testing out your SEO changes and tracking their effectiveness over time.  A variety of free and paid tools are available to assist you in analyzing your search rankings, search terms, and keyword effectiveness.  Below I’ve listed a few tools that can help.

• Google Analytics – by far the best free website traffic tracking software that I’ve ever used.  Formerly known as Urchin, Google Analytics allows you to slice, dice, drill down, and report into your tracking data any way you like.  Even better, Google Analytics allows you to configure “goals” for your web site which are basically actionable things that users of your site can perform that are of value to you, the business owner.  For example, submitting a contact form, downloading a white paper, completing a transaction, etc.  Dollar amounts, if applicable, can be tied to goals, allowing you to determine the exact revenue per visitor.  This effectively allows you to determine the most valuable incoming keywords and most effective traffic sources for your web site.  Beyond visitor value, Google Analytics can help you determine many more important statistics.  For example…
  • Most popular content on your website
  • Browser capabilities of your visitors
  • Location and language preferences of your visitors
  • Most popular search terms used to find your web site
  • Tracking of your CPC ad campaigns
  • Tracking visitor loyalty
  • Tracking the top exit pages for your web site (pages where visitors leave)
• Keyword ranking monitoring and reporting.  There are a variety of free and paid tools that will allow you to continually monitor and report on current and historical keyword rankings for your own website, as well as the keyword rankings for your competitor’s websites.  These tools will allow you to see if you’re making progress on increasing your search rankings.
  • Free Tools
    • Keyword Position Ranking Monitoring Service
    • DigitalPoint
  • Paid Tools
    • Caphyon Advanced Web Ranking
    • CheckRankings
• SEO analysis tools such as the Microsoft SEO Toolkit allow you to analyze the your website to check for content level flaws such as broken links and duplicate content that can affect your search engine rankings.  The Microsoft SEO Toolkit allows you to view detailed information about SEO problems on your website using built-in reports and dashboards – an extremely useful tool to use when analyzing the state of SEO on an existing web site.

There is much more to search engine optimization than can be written up in a single blog post (see also: thousands of blogs dedicated purely to the subject).  However, I hope this quick guide to the basics will give you the tools necessary to implement numerous high impact SEO quick wins for a client or personal web site. For web developers, the factors listed above should be kept in mind whenever developing customer-facing websites that could benefit from enhanced search results and search rankings.  Most of the ‘content / internal’ best practices can be easily baked into the development process of almost any e-commerce or content management system implementation project.

The impact of performance is much more readily apparent in .NET Compact Framework applications.  The mobile devices commonly have a CPU that is 10 times slower than your desktop CPU, and possibly up to 100 times less RAM than a desktop or server.  In Agile or XP development, the mantra is often to ignore performance considerations until necessary – I don’t think you can apply that to .NET CF development or it will really bite you in the end.  You don’t have to go nuts and optimize everything up front, but there are some very important things to keep in mind when developing a Windows Mobile application…

Standard .NET Framework Performance Considerations

Many of the standard .NET Framework performance best practices can become apparent very quickly including…

• Object Boxing and Unboxing.  Use generics wherever possible and avoid ArrayLists and type conversions.
• String and StringBuilder.  Need to perform lots of string concatenations? Use a StringBuilder instead of the ‘+’ operator.  When you use the ‘+’ operator, a new string object is created each time you concatenate, increasing memory usage.  The ‘+’ operator is much slower if you’re concatenating a large number of strings.
• Memory leaks.
  • When doing .NET CF development, if an object implements the Dispose() method – call it when you are finished with the object.
  • One of the most common causes of memory leaks is unhandling events when they’re no longer needed.  If you manually hook up an event with the ‘+=’ operator, ensure you’re unhandling it when finished with the ‘-=’ operator.
  • Pre-allocate collections if possible.  Standard .NET behavior is to automatically double the size of a collection when the upper limit is reached while adding items.  If you know the number of elements that are are going to be in a collection, pre-allocate the size of the collection when instantiating it.
• Don’t use Exceptions for flow control in an application.  Exceptions are an expensive operation, performance wise.  I’m not saying don’t use exceptions, but don’t use them in areas where you can perform simple checks to prevent them from being thrown.  For example, if you might divide by zero – perform a simple check before the operation occurs rather than handling a DivideByZeroException.  The check is much less expensive than the exception.

.NET Compact Framework-Specific Performance Considerations

However, the .NET Compact Framework is different than the full framework in many ways, leading to a slew of .NET CF specific performance considerations…

• Avoid making virtual function calls.  They are up to 40% slower than instance and static function calls.  I don’t completely understand the reason for this, but you can read more about it here if you’re interested.
• There are a few things in .NET CF that are slow because of virtual calls and object boxing/unboxing.  These include:
  • Reflection.  Very slow in .NET CF.
  • XML Deserialization and DataSets.  Extremely slow  because reflection is slow.
• Avoid creating many copies of Form objects.  Creating a Form is an expensive operation, and unused Form objects are a common cause of memory leak issues. You may want to create your Forms once and cache them in the background for reuse.
• You can increase the speed of binding data to controls by using the BeginUpdate and EndUpdate methods on a control before and after your data binding occurs.  This will cause the control to not repaint until the binding is finished.
• Cache expensive resources.  For example, don’t create many different copies of a web service client.  Create a single, cached instance of it that can be used throughout your application.
• Always test your application on a wide range of physical devices.  If the target device is known, at least test on that device.  Some things seem to perform much betterwhen running on the emulator or when executing unit tests on your desktop environment.
• This is a more general performance testing best practice, but always test with real data and real quantities of data. This can really bite you on deployment of your application.  I know this from experience – a great example is that deserializing a few hundred objects is MUCH much faster than deserializing 10,000 objects.  In my experience, deserializing 7,000 very simple DTO objects from an ASMX web service was taking up to 20 minutes in some cases.  To alleviate the issue, we ended up switching to a JSON web service, which was much faster to deserialize.

I’m starting up a short Windows Mobile project again, so I thought it would be a good time to collect some of my best practices for .NET Compact Framework development and post them.  I’m going to break them down into two sections -  usability, and performance best practices (in another post).

Windows Mobile Usability Best Practices

Microsoft has put together a very specific set of guidelines for Windows Mobile usability – the point of this is to get a consistent set of look and feel and application experiences on their platform.  Apple has the same sort of guidelines for iPhone development and it really pays off – most applications have the same consistent look and feel and excellent usability.  Of course, many of these usability guidelines are relevant across many development platforms, but there are some special considerations for mobile development.

Usability is a challenge in mobile development.  Some of the main concerns include…

• Limited screen real estate.  In Windows Mobile, the most common size is around 480 x 640 pixels.
• Limited input options.  Touch screen.  Potentially no hardware keyboard.  No mouse, and no scroll wheel.
• Lighting – Indoor / Outdoor usage.
• Gloves (i.e. warehouse users)
• Finger vs. Stylus

Here are some of the most important usability guidelines that Microsoft has set forth…

• Only display the most relevant information and options on the screen, i.e. don’t clutter up the screen with 100 different rarely used options.  If a feature is rarely used, place it in a menu or submenu.  If a feature or action is used very often, think about assigning it to one of the standard left or right soft keys.
• Use high contrast, sufficiently bright colors.   Lighting conditions are an important factor in mobile development.  For example, think about if your application could be used in low light or outdoor sunlight conditions.
• Avoid very small font sizes.  The screen on a mobile device is very small as-is, and actions on a mobile device are often performed at arms length away from the user (in a warehouse, for example).  If a user has to interrupt their workflow to bring the device in front of their face to read the text, then your font is too small.
• Make the user interface predictable and consistent in your application, keep ‘OK’ and ‘Cancel’ actions in the same location throughout your interface.  The same buttons should perform the same actions throughout your application.  To stay consistent with other Windows Mobile interfaces, one recommendation is to always assign the left soft key to ‘Back’ or ‘Cancel’ actions, and to assign the right soft key to ‘Next’ or ‘OK’ actions.  Another Microsoft recommendation is to avoid overriding the hardware buttons (i.e. the Home button).
• Ensure your UI elements are appropriately sized.  Buttons sized for a stylus should be at least 21 pixels squared, buttons sized for fingerse should be at least 38 pixels squared.
• Keep screen rotation in mind – developing to account for rotation is a pain, but very important for consumer applications.  Your options though are limited to either dynamically resizing the content, or to just design for a square screen.
• Scrolling is discouraged in Windows Mobile applications, because it is kind of a pain for the end user.  Try to keep your content on one screen length/width if possible.
• If your target devices may feature a keyboard, assign common actions to key shortcuts.  This can greatly increase efficiency for power users.
• For displaying information, make use of Summary, Detail, and Edit views.  A ‘Summary’ view displays only the most necessary and relevant information about an item.  To access less commonly used information about an item, the user can drill down to a more complete ‘Detail’ view.  If a user needs to edit the information, they can access an ‘Edit’ view.
• Ensure you’re setting focus on the appropriate text entry fields in bar code scanning scenarios, etc.  If a user is wearing gloves and has to take them off to set focus on a field before they scan a pallet, they’re going to hate your application.

Over time while using ASP.NET I’ve collected a pretty good handful of best practices that I try to employ on my projects – most of them are things that will simplify the ASP.NET development experience, solutions to common problems, or tips that will just make your life easier.  Most of the best practices are only applicable to WebForms, but some are applicable to ASP.NET MVC as well.

• Don’t write .NET code directly in your ASPX markup (unless it is for databinding, i.e. Eval statements). If you also have a code behind, this will put your code for a page in more than one place and makes the code less manageable. Put all .NET code in your code-behind.  Things can get complex and difficult to debug very quickly when you’re looking at code executing in two different places.
• SessionPageStatePersister can be used in conjunction with ViewState to make ViewState useful without increasing page sizes. Overriding the Page’s PageStatePersister with a new SessionPageStatePersister will store all ViewState data in memory, and will only store an encrypted key on the client side.  This will make your pages smaller and download faster if you have a lot of ViewState data for some reason, however it will increase your memory usage on the server – so tread carefully.  See example below for how to use SessionPageStatePersister.

• Create a BasePage that your pages can inherit from in order to reuse common code between pages.  Simple object oriented design principles – if you have common functions between pages, like security for example – put it in a base class that inherits from System.Web.Page, and have your pages inherit from that base page.
• Create a MasterPage for your pages for visual inheritance.  Don’t use ASP server-side includes.  Pages with vastly different visual styles should use a different MasterPage.  Don’t use a Master page for code inheritance.
• Make use of the ASP.NET Cache in order to cache frequently used information from your database.  Build (or reuse) a generic caching layer that will wrap the ASP.NET Cache.  If you’re loading the same list from the database into a drop down every time a page loads, you should be pulling that list from the cache based on how dynamic it needs to be.
• Wrap ViewState objects with Properties on your Pages to avoid development mistakes in spelling, etc. when referencing items from the ViewState collection.  For example, you should only have ViewState["key"] once in your page per property.  See example below.

• Avoid putting large objects and object graphs in ViewState, use it mainly for storing IDs or very simple DTO objects.  This is the reason people always complain about huge viewstate – they’re storing something like DataSets in ViewState (terrible idea).  If you stick to small objects with a limited number of properties or just integer IDs, your ViewState data will not be unmanageably large and ViewState is totally usable.
• Wrap the ASP.NET Session with a SessionManager class to avoid development mistakes in spelling, etc. when referencing items from Session.  Just another way to cut down simple development mistakes.
• Make extensive use of the applicationSettings key/value configuration values in the web.config – wrap the Configuration.ApplicationSettings with a class that can be used to easily retrieve strongly-typed configuration settings without having to remember the keys from the web.config.  If you have settings, behaviors, etc. that need to change between different deployments of your application, those should be control via settings in the web.config.  For example, we’ll often get requests like ‘We want feature X to go live at the end of the month” – so build, test, and deploy the update ahead of time.  But, add a web.config value that controls whether or not the feature appears i.e. FeatureXEnabled=”False”, on the day of go live just flip it to “True”.
• Avoid the easiness of setting display properties on your UI controls, instead use CSS styles and classes – this will make your styles more manageable.  Just a general web development best practice.
• Create UserControls in your application in order to reuse common UI functionality throughout your pages. For example, if a drop down list containing a collection of categories will be used in many places in the site – create a CategoryPicker control that will data bind itself when the page is loaded.  This is my #1 time-saving best practice, yet I’m always surprised how often I see the same drop down list with the same data getting used the same way on 20 different pages – yet the same type-unsafe databinding logic is duplicated 20 times!
• Use Properties on your UserControls to setup things like default values, different displays between pages, etc. Value type properties can be defined on your UserControls and then be set in your ASP.NET markup by using class level properties on UserControls.  This is a great way to get even more mileage out of reusing your UserControls – watch out for increased complexity of your UserControl logic though.
• Make use of the ASP.NET validation controls to perform simple validations, or use the CustomValidator to perform complex validations.
• Create an user-friendly error handling page that can be redirected to when an unhandled exception occurs within your website.  Log any exceptions that come to this page.  The redirection can occur via the Page_Error event in your Base Page, the Application_Error event in your Global.asax, or within the error handling section in the web.config.  Basically, whichever method you pick – make sure you’re not letting any exceptions go unhandled or unlogged!
• When working with pages that use a highly dynamic data driven display, use the 3rd party (free) DynamicControlsPlaceholder control created by Denis Bauer to simplify the code needed to save the state of dynamically added controls between postbacks.  This little control has saved me countless hours of pain in creating pages with highly dynamic UserControls.  One gotcha – if you use event handling delegates in a UserControl, you have to hook them up on every postback, little messy but not a big deal though usually.  Event handlers are the only “state” that isn’t saved between postbacks if you use this control.
• Turn ViewState off on controls and UserControls that don’t need it.

Ran into an interesting problem yesterday where a few months ago we helped a client redesign an ASP.NET web application to fit it into an iframe within their CMS rather than being a standalone site.  Easy enough task.  Testing is completed and site is rolled out.

Now, several months down the road after the application has been iframe’d and in production – one random feature of the application is unexpectedly breaking, but it doesn’t make any sense – the only way the behavior could possibly occur would be that an object retrieved from Session is coming back as null, which turned out to be the case.  The browser was somehow losing the ASP.NET Session cookie.  Furthermore, the feature was working fine in Firefox but not in Internet Explorer, very strange.

The problem was that Internet Explorer will not accept cookies from a page within an iframe where the domain name is different from the top level page.  So, the url of the iframe’d page was www.clientsite1.com and the url of the page hosting the iframe was www.clientsite2.com.

To get around this, you need to add a P3P Compact Policy to your HTTP responses.  P3P is a protocol that allows websites to pass information to the browser regarding their intent to use information collected from the user.  Internet Explorer is the only browser that implements the protocol, and only using it for cookie blocking at that.

To add a P3P in ASP.NET that will allow your cookies to be accepted by the browser from a different domain from within an iframe, add this block of code to your Global.asax.

Microsoft showed us Photosynth while we were touring the MTC in downtown Chicago a few months ago.  Photosynth is an impressive new Silverlight-based technology they’ve been working on that can be used to stitch hundreds of images together into a single explorable, zoomable, pannable, and web-viewable panoramic image.   I was looking for a situation where I could possibly make use of Photosynth since I first saw it.   The amazing panaromic views of Isle Royale National Park in Michigan proved to be the perfect opportunity.

The photosynths are easy to make – just stand in one place, turn in a circle and take about 50-100 overlapping pictures.  Be sure to zoom in a few times and take some detail shots.  After downloading the Photosynth client and selecting photos to stitch, some pre-processing takes place on your computer and the images are uploaded to the Photosynth website where the final combined result can be viewed.  The resulting photosynth can be embedded in a web page.

You can see an example from Isle Royale below.  Give it a few seconds to load for a better experience while zooming around.  Use the mouse wheel to zoom in and out, and drag the picture around to pan.

In reading various blog posts, forums, and Stack Overflow questions there is still quite a bit of confusion around the different data storage options available in Windows Azure.  This is probably mostly due to Microsoft changing their strategy, architecture, and naming conventions a few times.

For example, SQL Azure used to be called SQL Server Data Services (SSDS), was non-relational, and used an Entity Attribute Value (EAV) schema – giving it a major overlap in the functionality of Azure Table Storage.  This obviously provided a pretty poor migration path to the cloud for existing applications using relational SQL Server storage.  It was also confusing for developers, seeing that there were two non-relational storage options.  So, Microsoft dumped/revamped SSDS and turned it instead into the now fully-relational hosted SQL Server offering, SQL Azure.

Below I’ve outlined some key bullet points around the various Azure storage options, as well as when each option should be used.

Azure Blob Storage

• You can think of blob storage as the file system of Windows Azure.
• Blobs are stored in blob containers.  You can think of a blob container as a folder.
• You can give a blob a primary key and some key/value metadata when you upload the blob to storage.
• Use the StorageClient class included with the Windows Azure SDK Samples to interact with blob storage from .NET (rather than using the REST API directly).  This greatly simplifies things and adds some additional functionality like retries on failed calls.
• Use blob storage to store anything that you would normally store as a file or database blob.
• Querying is very limited – you can only pull a list of blobs by their container or by their primary key.
  • There are two steps to retrieving a blob – first you pull the metadata, and then you make a separate call to pull down the actual byte content.  For example, retrieving a list of blobs by their container retrieves a list of blob metadata.  This prevents you from pulling unnecessarily pulling down a ton of data.
• Azure Blob Storage can be accessed from within the cloud or from outside of the cloud (i.e. a desktop or intranet application).  Your blob storage API URL is publicly accessible, however it requires the use of a secret key to access unless you’re retrieving public blobs.

Azure Table Storage

• Use Azure Table storage to store simple structured data and objects.
• You can think of Azure Table Storage as a bunch of stand alone object tables with no relation to each other.  There are no real foreign keys except what you implement on your own.
• Azure table storage is implemented as an Entity-Attribute-Value (EAV) database in the backend, but this is mostly abstracted via the StorageClient class included with the Windows Azure SDK samples.
• You can store more than one type of object with varying properties in a table, but this is not recommended for the sake of keeping things simple.  This at least allows your objects to evolve over time as you add/remove fields.
• Every object that you store in Azure Table Storage must have a PartitionKey and a RowKey.
  • The combination of the Partition Key and Row Key is the Primary Key.
  • The Partition Key and Row Key are the only indexed values on the objects.
  • The Partition Key can be used to logically partition data within your tables if required – though you can hard code the Partition Key if you don’t need it.
  • The Row Key should be a unique key within the partition.  You could use a Guid, for example.
• You can query Azure Table Storage via LINQ, though not all operations (i.e. scalars) are supported.  Performance would definitely be a factor on queries from an EAV database.  Make use of the indexed partition and row keys if possible for faster querying.
• Azure Table Storage can also be accessed from within the cloud or from outside the cloud (i.e. a desktop or intranet application).  Your table storage API URL is publicly accessible, however it requires the use of a secret key to access.

Azure Queue Storage

• The main use of Azure Queue Storage is for communication between your Azure web and worker roles or between worker roles, i.e. for picking up and dropping off data to be processed.
• Can pass simple string messages in a queue.
• Similar to other message queueing frameworks.
• Also can use the StorageClient class in the Windows Azure SDK samples to abstract the HTTP REST API.

SQL Azure

• SQL Azure is very similar to an on-premise SQL Server databases.
• You can currently create a 1 GB database (currently $9.99/mo) or 10 GB database (currently $99/mo).  Much more expensive than Windows Azure Storage options (blob/table/queue).
• Use SQL Azure for storing complex relational data – i.e. any time you would normally store data in a database.
• For the most part, you can simply change your SQL Server connection string to point to your SQL Azure connection and your application will work fine in most cases.
• Your SQL Azure instance is publically accessible via a SQL TCP connection over the internet.  You can retrieve the connection string for your database from the SQL Azure dashboard.
  • Though the database is publicly acessible, you are given access to create basic inbound firewall rules for which hosts are allowed to access your SQL Azure database.  For example, you can set it up such that your database is only accessible from within your Windows Azure roles or you could set it up such that it would only be accessible from IPs originating from your organization.
• SQL Azure has much of the same capabilities as on premise SQL Server, i.e. full T-SQL querying capabilities, indexing, stored procedures, triggers, views, etc.
• You can connect to SQL Azure via all the standard methods, i.e. ODBC, ADO.NET, PHP Drivers, LLBL, NHibernate, Entity Framework, etc.
• Some features are not supported including SQL Profiler, backup, replication, filegroups, manipulation of physical file resources, etc.
• Many of the unsupported features are taken care of for you by Azure, i.e. backup, replication, high availability, resource governance, etc.

I recently ran into a situation where a client wanted to place their public facing ASP.NET website behind Oracle SSO to allow their customers to log in via their existing Oracle SSO accounts, yet also allow anonymous users to use the application without logging in.  We could have done a simple LDAP integration, but the client also wanted users to not have to log into the application if they’re already logged into their Oracle Portal account.

First step, we had to determine how to get Oracle SSO running on Oracle Application Server (OAS) to protect an application running on a separate IIS web server. In order for an ASP.NET application to be able to get user credentials from Oracle SSO, Oracle SSO has to run “in front of” your application – i.e. something has to intercept the browser request on the way to your web app, decrypt their SSO session cookie, and inject an HTTP header containing the the user’s username that your ASP.NET application can read. There are two ways to accomplish this – Apache Reverse Proxying and the Oracle SSO IIS Plug-in.

To preface, Apache Reverse Proxying will route all the calls to your application through Apache first. The other option, the Oracle SSO IIS plug-in is installed on your IIS server, requests are sent directly to the IIS web server, and the requests are intercepted and security is handled by the Oracle plugin.

The Apache reverse proxy will pass all calls sent to a URL in OAS to a site running on another web server.  For example, OAS will take all browser requests to http://oas.client.com/iissite/ and proxy them to http://iis.client.com/.  Responses from IIS will be sent back to OAS and OAS will send the response back to the originating web browser.  In testing this proved to be a little slow, as every single request to the application – images, javascript, css, ajax, postbacks, etc. would all  be sent through the reverse proxy server and require an additional hop for every time.

On top of the performance issues, the ASP.NET application had to run in the root of the IIS web server and use a host header to route requests to the proper site.  ASP.NET uses relative paths (i.e. <script src = ‘/ScriptResource.axd?etc…’/>) for included javascript used in ASP.NET AJAX and ASP.NET validation controls (ScriptResource.axd and WebResource.axd).  Normally this would work fine if the web requests were being sent directly to the IIS server.  However, when behind the reverse proxy server (remember, IIS has no idea it is serving pages behind a reverse proxy) – this causes the web browser to try to retrieve the javascript files from http://oas.client.com/ScriptResource.axd (directly from the OAS web server – where ScriptResource.axd obviously doesn’t exist and will send back a 404 error) instead of properly retrieving the files from http://oas.client.com/iissite/ScriptResource.axd.  This causes all of the ASP.NET AJAX and ASP.NET validation controls to break.

There isn’t any way to get ASP.NET to retrieve those resources from a different path or to somehow prepend a path to the ScriptResource.axd and WebResource.axd URLs.  The ScriptResource.axd issue can be fixed by manually including the individual ASP.NET AJAX javascript files by setting the ScriptPath on your ScriptManager.  This is a pain, but works fine with the Reverse Proxy and the browser will be able to properly retrieve the ASP.NET AJAX files.  Unfortunately, this still leaves the WebResource.axd pointing to the wrong path. WebResource.axd is used to retrieve the javascript used in the client-side validation for ASP.NET Validator controls and without it all client-side validation will be broken.  There isn’t any way that I could find to modify where ASP.NET will retrieve those files.  To get around the .axd issue, we had to get creative and create an IHttpModule that would rewrite the HTML responses and fix the paths on the fly.

Doing a simple find and replace on the .axd paths works fine for regular postback responses to fix the bad paths, but fails with ASP.NET AJAX partial-page updates.   You can find and replace in the partial-page updates, but then it will throw off the field lengths in the pipe-separated data that is sent back to the browser.  Thus, you need to actually find and rewrite the field lengths on the fly as well whenever you do a replace on the .axd paths.  You can see the implementation of this in the ReverseProxyPathFixModule.cs below – it is a little scary, and I’m sure it isn’t full proof because the partial page responses are chunked upon being sent back to the browser.  If there was an .axd path in between chunks, it wouldn’t be replaced – but I never saw this happen.

The most relevant portions of the code below are the Write() methods of PageFilter and PartialPageFilter – they do all the work. The rest of the code is just overridden Stream methods.

View ReverseProxyPathFixModule.cs

After implementing the custom HttpModule, the application was working almost perfectly behind the Reverse Proxy.

For the next hurdle, we couldn’t find any way to have Oracle SSO protect a resource in IIS (or even running in OAS for that matter) while allowing both anonymous and authenticated access.  There isn’t any built-in way to allow anonymous access to an application while it is protected by Oracle SSO.  After much research and reading this Extending Oracle SSO presentation and this Integration with Third-Party Access Management Systems help documentation from Oracle, we decided to create a custom Oracle SSO module that would “authenticate” a user and pass them to the application as the Oracle Portal “PUBLIC” account if they weren’t already logged in to SSO.  The implementation of this plugin is fairly simple – it’s a Java class that inherits from the default Oracle SSO module (SSOServerAuth) and implements the IPASAuthInterface interface.  The code simply checks the user’s cookies on the request – if the user has an Oracle Portal cookie, perform the authentication from the base class  by calling super.authenticate.  If the user doesn’t have a portal cookie, pass them on to the application and “authenticate” them as the PUBLIC user account.  This is definitely a hack, but it works pretty well.  See the implementation of the MixedAuthentication below.

View MixedAuthentication.java

Compilation of the code is a little tricky, you need to include ipastoolkit.jar, ossocls.jar, and servlet.jar in your classpath.  The ossocls.jar isn’t usually included or detailed in the documentation because most Oracle SSO plugins don’t inherit from SSOServerAuth (it isn’t required), but rather just implement IPASAuthInterface.  Deployment is also tricky, fortunately I found this blog post ‘Adding reCAPTCHA to Oracle SSO‘ that detailed how the plugin should be deployed to OC4J_SECURITY container, rather than the standard $ORACLE_HOME/sso/plugins location.

More hurdles! After successfully setting up our custom authentication plugin, we couldn’t figure out how to have our reverse proxy’d application use the custom plugin without it also affecting the client’s Oracle Portal installation.  After we would set the reverse proxy path to use the custom plugin, we would see strange behavior in the Oracle Portal even though portal would be set to use the standard MediumSecurity and our reverse proxy path would be set to use our custom ‘MixedSecurity’ setting.

This is how we tried to set up our Oracle SSO policy.properties file:

#add our custom security level.
MixedSecurity = 70 

#keep the default authentication level so as to not affect oracle portal security.
DefaultAuthLevel = MediumSecurity

#set our custom app behind reverse proxy to use our new custom security level.
oas.client.com/iissite\:80 = MixedSecurity
#not sure if you need the path on OAS or the reverse proxy site. also tried it this way.. didn't work.
iis.client.com\:80 = Mixed Security

#set the plugin class for our custom security level
MixedSecurity = com.client.authentication.MixedAuthentication

No matter what we tried with the SSO configuration we couldn’t get our application behind the reverse proxy to be protected by our custom plugin without also affecting the security of Oracle Portal.  If anyone knows how to actually do that, I’d be interested to hear where we went wrong in the comments.  Unfortunately, this meant that the work with the custom HttpModule, setting up the reverse proxy, etc. was all for naught.  We had to install the Oracle SSO IIS plugin.  This plugin is somewhat of a beast – the installation and configuration is one of the most complicated and least user-friendly  I’ve ever encountered and involves creating registry entries manually, providing many opportunities to make mistakes along the way.

Either way, after installing the IIS plugin everything worked fairly smoothly.  One thing to note – if you want to redirect the user from your ASP.NET application to log in to their actual Oracle SSO account rather than the PUBLIC account, you need to delete the user’s cookie that will look something like IAS_IDXXXXXX – this will “log out” the user from the PUBLIC account.  If the user isn’t logged out of the PUBLIC account before hitting the SSO logon page, they’ll be automatically redirected (to the url provided in the p_requested_url parameter when sending the user to the SSO logon page) when they hit the page because they’re actually already logged in to the PUBLIC account.

One remaining problem, the Oracle SSO IIS plug-in manages to randomly crash the worker process with an error like:

Faulting application w3wp.exe, version 6.0.3790.3959, stamp 45d6968e, faulting module oracle_osso.dll, version 0.0.0.0, stamp 41775fa1, debug? 0, fault address 0×00002454.

Checking the SSO plug-in log files yields nothing out of the ordinary either so this has been pretty difficult to track down, we still haven’t found any solution for this problem.   If anyone knowledgeable on the IIS or Oracle SSO side of things has some ideas or has seen this before, feel free to let me know in the comments.

Finally, after your ASP.NET application is safely behind Oracle SSO you can determine the logged in user’s username by checking the OSSO-USERNAME header like so:

After that, the user’s username from Oracle SSO will come over on the HTTP headers on every request to your application.