AppCenter Extensions for ASP.NET Core and Application Insights

In my previous post, I wrote about an open source project called AppCenterExtensions available at Github and nuget.org. I recently updated this project and added a few components for ASP.NET Core that enables including AppCenter diagnostic information in Application Insights.

The NuGet package is called AppCenterExtensions.AppInsights and contains extension methods and ITelemetryInitializer implementations to be used in a ASP.NET Core web app for including AppCenter diagnostic information when logging to Application Insights

Enabling this is easy. Assuming that the project is already configured to use Application Insights, just add the AppCenterExtensions.AppInsights NuGet package mentioned above to your ASP.NET Core and call services.AddAppCenterTelemetry() in the ConfigureServices method of the Startup class

Here’s an example:

public class Startup
{
    public Startup(IConfiguration configuration)
    {
        Configuration = configuration;
    }

    public IConfiguration Configuration { get; }

    public void ConfigureServices(IServiceCollection services)
    {
        // Configure and register services to the IoC

        services.AddApplicationInsightsTelemetry();
        services.AddAppCenterTelemetry();
    }

    public void Configure(IApplicationBuilder app, IWebHostEnvironment env)
    {
        // Configure app
    }
}

Once this is setup, AppCenter diagnostic information should now be searchable and visible in Application Insights.

Here’s a screenshot of search results for the x-supportkey header

and here’s a screenshot of the details of a single request containing AppCenter diagnostic information logged in Application Insights

With this flow you can now correlate Crash Reports and Analytics data from AppCenter with the HTTP requests for your backend systems in Application Insights. In the systems that I have been involved with building we include the AppCenter diagnostic information from our API Gateway to all calls to our internal Microservices

AppCenter Extensions for Xamarin.Forms

For the past 3 years or so I have been AppCenter for Crash Reporting and Analytics in Xamarin based apps. During this time, I have mostly built enterprise focused apps using Xamarin.Forms and as a developer I always think about code reuse which usually comes in the form of a library. Early this year, I decided to create and open source a set of convenience classes and extension methods to simplify Crash Reporting and Analytics using AppCenter and called it AppCenterExtensions.

The core features of the project are the following:

  • Simplified user interaction reporting using ICommand implementations
  • Automatic page tracking in Xamarin.Forms including time spent on screen
  • Extension methods for crash reporting
  • Anonymous user information configuration

This library is distributed as 2 NuGet packages

Getting Started

This library is configured almost the same way as the AppCenter SDK. You provide the AppCenter secrets, and specify whether to anonymize the user information. Both Crash Reporting and Analytics are always enabled when using AppCenterSetup.

AppCenterSetup.Instance.Start(
    "[iOS AppCenter secret]",
    "[Android AppCenter secret]",
    anonymizeAppCenterUser: true);

or

await AppCenterSetup.Instance.StartAsync(
    "[iOS AppCenter secret]",
    "[Android AppCenter secret]",
    anonymizeAppCenterUser: true);

The reason for the async API here is because anonymizeAppCenterUser internally relies on an async API. The synchronous API’s for starting AppCenter are non-blocking methods that do a fire-and-forget call to StartAsync(string,bool).

Anonymous User Information

The component AppCenterSetup exposes a method called UseAnonymousUserIdAsync() which sets the UserId in AppCenter to the first 8 characters a GUID that is unique per app installation. This can be used as a support key for uniquely identifying application users for instrumentation and troubleshooting. The support key can be attached to all HTTP calls by using the DiagnosticDelegatingHandler

AppCenter Crash Report

Error Reporting
The library exposes extension methods to the Exception class for conveniently reporting Exceptions to AppCenter

Example:

try
{
    // Something that blows up
    explosives.Detonate();
}
catch (Exception e)
{
    // Safely handle error then report
    e.Report();
}

HTTP Error Logging

The library provides a HttpMessageHandler implementation that logs non-successfuly HTTP results to AppCenter Analytics. This component will also attach HTTP headers describing the AppCenter SDK Version, Install ID, and a support key to all HTTP requests. The logged failed responses will contain the Endpoint URL (including the HTTP verb), Response status code, how the duration of the HTTP call. This will be logged under the event name HTTP Error
You will in most (if not all) cases would want to keep a singleton instance of the HttpClient. The DiagnosticDelegatingHandler is designed with unit testing in mind and accepts an IAnalytics and IAppCenterSetup interface, it also accepts an inner HttpMessageHandler if you wish to chain multiple delegating handlers.

Example:

var httpClient = new HttpClient(new DiagnosticDelegatingHandler());
await httpClient.GetAsync("https://entbpr4b9bdpo.x.pipedream.net/");

In the example above we made an HTTP GET call to the RequestBin endpoint https://entbpr4b9bdpo.x.pipedream.net. This will result in the following we inspected in RequestBin

AppCenter Crash Report

ITrackingCommand

This library provides 3 convenience implementations of ICommand that will report the action to AppCenter Analytics after successfully invoking the execute callback method

  • TrackingCommand – This implementation accepts an Action as the Execute callback and a Func<bool> as the CanExecute callback
  • TrackingCommand – This implementation accepts an Action<T> as the Execute callback and a Func<T, bool> as the CanExecute callback
  • AsyncTrackingCommand – This implementation accepts a Func<Task> as the execute callback and a Func<bool> as the CanExecute callback. This also exposes a CompletionTask property that the consumer can await if desired. The Execute(object parameter) method here is a non-blocking call

Example:

using System.Threading.Tasks;
using System.Windows.Input;
using ChristianHelle.DeveloperTools.AppCenterExtensions.Commands;
using ChristianHelle.DeveloperTools.AppCenterExtensions.Extensions;
using Microsoft.AppCenter.Crashes;
using Xamarin.Essentials;

namespace SampleApp.ViewModels
{
    public class AboutViewModel : BaseViewModel
    {
        public AboutViewModel()
        {
            AsyncButtonTappedCommand = new AsyncTrackingCommand(
                OnAsyncButtonTapped,
                nameof(AsyncButtonTappedCommand).ToTrackingEventName(),
                nameof(AboutViewModel).ToTrackingEventName());

            ButtonOneTappedCommand = new TrackingCommand(
                OnButtonOneTapped,
                nameof(ButtonOneTappedCommand).ToTrackingEventName(),
                nameof(AboutViewModel).ToTrackingEventName());

            ButtonTwoTappedCommand = new TrackingCommand<string>(
                OnButtonTapped,
                nameof(ButtonTwoTappedCommand).ToTrackingEventName(),
                nameof(AboutViewModel).ToTrackingEventName());
        }

        public ICommand AsyncButtonTappedCommand { get; }
        public ICommand ButtonOneTappedCommand { get; }
        public ICommand ButtonTwoTappedCommand { get; }

        private Task OnAsyncButtonTapped()
            => Browser.OpenAsync("https://xamarin.com");

        private void OnButtonOneTapped() { }

        private void OnButtonTwoTapped(string obj) { }
    }
}

Specifying the screenName argument in the constructor is optional and when this is not provided manually then it will use the declaring Type name from the method that instantiated the ITrackingCommand instance and convert it to a more analytics friendly event name using the ToTrackingEventName() extension method. In the example above, if the nameof(AboutViewModel).ToTrackingEventName() parameter is not provided then the owner declaring Type is AboutViewModel and the ScreenName will be set to "About"

Automatic Page Tracking

Automatic page tracking is enabled by replacing the base class of the ContentPage to classes to use TrackingContentPage class. By doing so the library will send page tracking information to AppCenter after leaving every page. Currently, the library will send the page Type, Title, and the duration spent on the screen. The library is rather opinionated on how to log information, and this will only change if I get a request to do so. Duration spent on screen is calculated using a Stopwatch that is started upon Page OnAppearing and is reported to Analytics upon OnDisappearing. The event name is based on the Type name of the Page and is split into multiple words based on pascal case rules and afterwards removes words like PageViewModelAsync. For example: UserSettingsPage or UserSettingsView becomes User Settings

XAML Example:

<?xml version="1.0" encoding="utf-8"?>
<ext:TrackingContentPage 
    xmlns="http://xamarin.com/schemas/2014/forms" 
    xmlns:x="http://schemas.microsoft.com/winfx/2009/xaml" 
    xmlns:d="http://xamarin.com/schemas/2014/forms/design" 
    xmlns:mc="http://schemas.openxmlformats.org/markup-compatibility/2006"
    xmlns:ext="clr-namespace:ChristianHelle.DeveloperTools.AppCenterExtensions.XamarinForms;assembly=AppCenterExtensions.XamarinForms"
    mc:Ignorable="d" 
    x:Class="SampleApp.Views.ItemDetailPage" 
    Title="{Binding Title}">

    <StackLayout Spacing="20" Padding="15">
        <Label Text="Text:" FontSize="Medium" />
        <Label Text="{Binding Item.Text}" d:Text="Item name" FontSize="Small" />
        <Label Text="Description:" FontSize="Medium" />
        <Label Text="{Binding Item.Description}" d:Text="Item description" FontSize="Small" />
    </StackLayout>

</ext:TrackingContentPage>

Custom Trace Listener

This library includes a trace listener implementation that reports to AppCenter. The reason for this is to cater to those who have implemented error handling or reporting using Trace Listeners, these types of users can just swap out (or add on) the AppCenterTraceListener

This implementation implements the following methods:

  • Write(object obj)
  • Write(object obj, string category)
  • WriteLine(object obj)
  • WriteLine(object obj, string category)

If the object provided is an Exception then this is reported to AppCenter Crash Reporting. If the object provided is an instance of AnalyticsEvent then this is sent to AppCenter Analytics

The AnalyticsEvent exposes 2 properties:

  • string EventName { get; } – self explanatory
  • IDictionary<string,string> Properties { get; } – Additional properties to attach to the Analytics event

To set it up you simply add an instance of AppCenterTraceListener to your existing Trace listeners:

Trace.Listeners.Add(new AppCenterTraceListener());

Here’s an example of how to use System.Diagnostics.Trace to report errors

try
{
    // Something that blows up
    explosives.Detonate();
}
catch (Exception e)
{
    // Safely handle error then report
    Trace.Write(e);

    // or
    Trace.Write(e, "Error");

    // or
    Trace.WriteLine(e);

    // or
    Trace.WriteLine(e, "Error");
}

and here’s an example of to use System.Diagnostics.Trace to send analytics data

public partial class App : Application
{
    private const string StateKey = "State";

    public App()
    {
        // Some initialization code ...

        Trace.Listeners.Add(new AppCenterTraceListener());
    }

    protected override void OnStart()
        => Trace.Write(
            new AnalyticsEvent(
                nameof(Application),
                new Dictionary<string, string>
                {
                    { StateKey, nameof(OnStart) }
                }));

    protected override void OnSleep()
        => Trace.Write(
            new AnalyticsEvent(
                nameof(Application),
                new Dictionary<string, string>
                {
                    { StateKey, nameof(OnSleep) }
                }));

    protected override void OnResume()
        => Trace.Write(
            new AnalyticsEvent(
                nameof(Application),
                new Dictionary<string, string>
                {
                    { StateKey, nameof(OnResume) }
                }));
}

Task Extensions

This library includes a few Task extension methods with AppCenter error reporting in mind. Possible exceptions that occur in the async operation are swallowed and reported to AppCenter. These extension methods will internally wrap the Task in a try/catch and await the Task using ConfigureAwait(false).

Here are usage some examples

Fire and Forget on a Task (Note: Forget() returns void)

var task = someClass.SomethingAsync()
task.Forget()

Awaitable Task (also available for Task<T>)

var task = someClass.SomethingAsync()
await task.WhenErrorReportAsync();

Generate Android Translations from Google Sheets

In previous articles Generating ResX translations from Google Sheets and Generate iOS InfoPlist.strings Translations from Google Sheets, I wrote about using Google Sheets as a translation tool by using the GOOGLETRANSLATE built in function to generate translation files for a Xamarin based solution. For this post, I will demonstrate something very similar, but instead of ResX files or InfoPlist.strings, I’ll generate strings.xml files for Android. For the sake of this article I created this sample Google Sheets

For a quick recap, we will use a tool called csvtrans written by my colleague and good friend, Ricky Kaare Engelharth. The tool is built with .NET Core and can be installed using this command

dotnet tool install -g csvtrans

Using the tool is also straight forward and it also comes with some quick start instructions

USAGE: csvtrans [--help] [--sheet <document id> <sheet name>]
                [--csv <url or path>] [--format <apple|android|resx>]
                [--outputdir <directory path>] [--name <string>]
                [--convert-placeholders <regex pattern>]

OPTIONS:

    --sheet, -s <document id> <sheet name>
                          specify a Google Sheet as input.
    --csv, -c <url or path>
                          specify a online or local cvs file as input.
    --format, -f <apple|android|resx>
                          specify the output format.
    --outputdir, -o <directory path>
                          specify the output directory.
    --name, -n <string>   specify an optional name for the output.
    --convert-placeholders, -p <regex pattern>
                          convert placeholders to match the output format.
    --help                display this list of options.

Here’s an example usage of tool

csvtrans --sheet 1mrMkhItrIDsPwEKMlR8JJ3Pgj1K6zUv0AhmBT4jWRqs Android --format android --outputdir .\Resources\

The first argument –-sheet is the Google Sheet document ID followed by the Sheet Name, the next argument –-format specifies the output file format, and the last argument –-outputdir specifies the output folder

You can get the Document ID from the URL of the Google Sheet

Here’s an example output

Now I can just bring these files into my project and use them directly. Well, almost! There’s one little problem, and that is that by default the Xamarin.Android csproj tooling explicitly adds each strings.xml file as an AndroidResource. Oddly enough, the csproj format allows to specify wild card folders, so if we want to enable dynamic generation of values/strings.xml translations then we need to manually edit the csproj.

This is actually very easy to do. We just need to replace the lines like

with

This opens up for dynamic translations at build time using your CI/CD build tools of choice

Generate iOS InfoPlist.strings Translations from Google Sheets

In my previous article Generating ResX translations from Google Sheets, I wrote about using Google Sheets as a translation tool by using the GOOGLETRANSLATE built in function to generate translation files for a Xamarin.Forms solution. For this post, I will demonstrate something very similar, but instead of ResX files I’ll generate InfoPlist.strings files in iOS for localizing the permission request prompts for accessing things like Camera, Location, Photo Gallery, etc. For the sake of this article I created this sample Google Sheets

For a quick recap, we will use a tool called csvtrans written by my colleague and good friend, Ricky Kaare Engelharth. The tool is built with .NET Core and can be installed using this command

dotnet tool install -g csvtrans

Using the tool is also straight forward and it also comes with some quick start instructions

USAGE: csvtrans [--help] [--sheet <document id> <sheet name>]
                [--csv <url or path>] [--format <apple|android|resx>]
                [--outputdir <directory path>] [--name <string>]
                [--convert-placeholders <regex pattern>]

OPTIONS:

    --sheet, -s <document id> <sheet name>
                          specify a Google Sheet as input.
    --csv, -c <url or path>
                          specify a online or local cvs file as input.
    --format, -f <apple|android|resx>
                          specify the output format.
    --outputdir, -o <directory path>
                          specify the output directory.
    --name, -n <string>   specify an optional name for the output.
    --convert-placeholders, -p <regex pattern>
                          convert placeholders to match the output format.
    --help                display this list of options.

Here’s an example usage of the tool

csvtrans --sheet 125id155PUq-6Odwg8Nf9fmkgBsKahTGbJYaYBD2rpSg iOS --format apple --outputdir .\Resources --name InfoPlist

The first argument –-sheet is the Google Sheet document ID followed by the Sheet Name, the next argument –-format specifies the output file format, the argument –-outputdir specifies the output folder, and the last argument --name specifies the output filename.

You can get the Document ID from the URL of the Google Sheet

Here’s an example output

Now I can just bring these files into my project and use them directly. Well, almost! There’s one little problem, and that is that by default the Xamarin.iOS csproj tooling explicitly adds each InfoPlist.strings file as a BundleResource. Oddly enough, the csproj format allows to specify wild card folders, so if we want to enable dynamic generation of InfoPlist.strings translations then we need to manually edit the csproj.

This is actually very easy to do. We just need to replace the lines like

with

This opens up for dynamic translations at build time using your CI/CD build tools of choice

Generate Resx Translations from Google Sheets

In my career, I have tried multiple translation tools for handling localization. This usually ends up in a spreadsheet sent back and forth that gets imported/exported with the actual translation tool. I have also tried giving my translators and customers direct access to the translation tool but that never really worked as they tend to blindly translate everything they see and usually miss out on the fact that some strings contain important placeholders that executable code expects. Anyway, at the end of the sending a spreadsheet back and forth seems to always work.

In a recent project, I built an Android and iOS app with Xamarin.Forms that used Resx files for handling cross platform translations, and InfoPlist.strings files in iOS for localizing OS requirement prompts for using things like Camera, Localization, Photos, etc. For this project we thought about playing around with Google Sheets as a translation tool. Google Sheets has built in Google Translate support so you can do something like =GOOGLETRANSLATE($B2,$B$1,C$1) where $B2 describes the text to translate, $B1 describes the source language, in this case English is the default, and $C1 describes the language to translate to. With this approach, I can very easily, blindly, add new translations to my app, like in this sample Google Sheets document, where I added Danish, German, Filipino, Simplified Chinese, Japanese, and Koreanusing the Google Translate tool. Of course, this needs to be proof-read by a translation professional who mastered the language, but this approach is very convenient for checking out how the app looks like in different languages.

Now here’s the awesome part. My colleague and good friend, Ricky Kaare Engelharth, created a translation tool called csvtrans that can produce Resx, iOS, and Android translation files from a publicly available Google Sheets document. The tool is written in .NET Core and is publicly available from nuget.org as a tool.

The tool can be installed using this command

dotnet tool install -g csvtrans

Using the tool is also straight forward and it also comes with some quick start instructions

USAGE: csvtrans [--help] [--sheet <document id> <sheet name>]
                [--csv <url or path>] [--format <apple|android|resx>]
                [--outputdir <directory path>] [--name <string>]
                [--convert-placeholders <regex pattern>]

OPTIONS:

    --sheet, -s <document id> <sheet name>
                          specify a Google Sheet as input.
    --csv, -c <url or path>
                          specify a online or local cvs file as input.
    --format, -f <apple|android|resx>
                          specify the output format.
    --outputdir, -o <directory path>
                          specify the output directory.
    --name, -n <string>   specify an optional name for the output.
    --convert-placeholders, -p <regex pattern>
                          convert placeholders to match the output format.
    --help                display this list of options.

Here’s an example usage of the tool

csvtrans --sheet 1icJ0a48MIIRkbHSIbPyLNXsbTZcPKI_U80QwdX5pWf8 Resx --format resx --outputdir .\Resources

The first argument –-sheet is the Google Sheet document ID followed by the Sheet Name, the next argument –-format specifies the output file format, and the last argument –-outputdir specifies the output folder.

You can get the Document ID from the URL of the Google Sheet

Here’s an example output

Now I can just bring these files into my project and use them directly. With the modern csproj format I don’t even need to do any changes to include these translation files, as long as the resx files are in the project folder they will be automagically included into the output. This opens up for dynamic translations at build time using your CI/CD build tools of choice

Scrollable UISegmentedControl for Xamarin.iOS

A few years ago, I had a full time job as a device developer in the Music Streaming industry. The applications we produced at the time targeted consumers and had a huge focus on UX and UI. One of the requirements our designers had was to have scrollable tabs. This was 5 years ago and before Xamarin.Forms existed so we built the iOS app and Android app separately. We used MvvmCross and the shared a lot of core code but the UI components were done per OS. When we started, the company did a recent switch to go full on with .NET on everything, so not only did we share code between apps, but we shared code across all systems within the entire organization.

Scrollable tabs come for free in Android using the built-in control TabLayout but on iOS we needed to re-create the UISegmentedControl and add scroll/pan/swipe functionality to it. On native code, you had a few options to choose from, so the first task was to find the best native implementation of it and port it to C#. A few Google searches later I found the HMSegmentedControl written by Hesham Abd-Elmegid. The component at the time was written in a single file and was a drop in replacement for the UISegmentedControl. It was functional, elegent, and directly portable to C#. It was perfect!

A few hours of focused coding later I managed to port the entire thing to C# and created a Github project for it. I originally called it HMSegmentedControl as a tribute to the author (I also sent him a thank you email at the time) but later changed it to ScrollableSegmentedControl as it was a better and more descriptive name that states exactly what it does. Recently, I re-visited this project to clean up, modernize the code, and structure of the repository. I added a README file with a useful description, screenshots, and code examples. I also published a NuGet package called ScrollableSegmentedControl to make it easier for others to use while keeping the responsibility of maintaining it.

So using the component is quite trivial. Here’s all you need to do:

Add the ScrollableSegmentedControl NuGet package

then you create an instance of ScrollableSegmentedControl and you add it to a View

using System;
using ChristianHelle.Controls.iOS;
using CoreGraphics;
using UIKit;

namespace ScrollableSegmentedControlSample
{
    public partial class ViewController : UIViewController
    {
        public ViewController(IntPtr handle) : base(handle)
        {
        }

        public override void ViewDidLoad()
        {
            base.ViewDidLoad();
            CreateScrollableSegmentedControl();
        }

        private void CreateScrollableSegmentedControl()
        {
            var sectionTitles = new[] { "One", "Two", "Three", "Four", "Five", "Six" };
            View.AddSubview(new ScrollableSegmentedControl(sectionTitles)
            {
                Font = UIFont.FromName("STHeitiSC-Light", 18.0f),
                Frame = new CGRect(0, 60, View.Frame.Width, 40),
                SegmentEdgeInset = new UIEdgeInsets(0, 10, 0, 10),
                SelectionStyle = ScrollableSegmentedControlSelectionStyle.FullWidthStripe,
                SelectionIndicatorLocation = ScrollableSegmentedControlIndicatorLocation.Down
            });
        }
    }
}

This would result in a segmented control that looks like this:

and can look like one of these examples depending on the SelectionStyle and SelectionIndicatorLocation

This post is probably 5 years too late but since I just only recently made it publicly available as a NuGet package I thought I should write a short article about it. I hope you find it useful!

Generating a REST API Client from Visual Studio 2017 and 2019

For the past year or so, I have been doing a lot of development that involves producing an OpenAPI specification document from a .NET Core based REST API and generating client code using things like AutoRestSwagger CodegenOpenAPI Codegen, and NSwag. My problem with these tools is that I often need to leave Visual Studio and quite often update the tool before I can re-generate my REST API client code. After doing this a couple of times I thought that I should just build a Visual Studio extension to make my life easier. At the end of last year I started work on a Visual Studio extension called the REST API Client Code Generator, A collection of Visual Studio custom tools for generating a strongly typed REST API Client from an Open API / Swagger specification file

With this tool I can easily switch from NSwag, AutoRest, Swagger Codegen, and OpenAPI Codegen, and re-generate my code by making changes directly to the OpenAPI specification document I have in my project.

I built Visual Studio Custom Tools for each code generator so every time I make changes to the OpenAPI specification document in my project, the client code gets automatically re-generated.

You can include an NSwag Studio file in the project and right click and re-generate my client code

And a feature that I just built today, adding a dialog for adding a new OpenAPI specification document file.

This project is open source and you get browse the repository on here and download the VSIX file from the Visual Studio Marketplace

New Challenges in the Cloud

I have been building mobile solutions for nearly 2 decades, I have seen things come and go, and I had loads of fun working in this space. A great part of my mobile development career involved consulting and for most of my work, I build the entire solution, including the web services and the data store and I am extremely proud of the things I have built. For the past 4 years I have been working in the Music Streaming industry. Using the tools from Xamarin, we built software for Android and iOS with a very large shared code base. The entire organization shares components through different platforms, it was great. Our products have a very high number of daily active users and has great ratings on the Apple AppStore and Google Play.

During these 4 years, a lot of things has happened. Microsoft acquired HockeyApp then later on Xamarin a few years back, and unfortunately, things went down hill. This is of course almost to be expected as Microsoft has bigger plans for these products and they will be all well integrated into a larger suite of products. I think things are starting to look better now, but for a very long while, it has been much worse. For instrumentation we had to switch from Xamarin Insights to RayGun. At that time when we did the switch, RayGun didn’t have the same feature set as Xamarin Insights so it felt like a down grade, with a higher price tag. RayGun eventually caught up and also improved the pricing plan.

Like anything else, if you work on the same suite of products for 4 years, no matter how fun it is, it becomes less and less exciting. Lately I have taken a keen interest in the recent developments in the Microsoft Azure space and early this year a very interesting opportunity presented itself. So after weeks of consideration, I decided to take a break from full time mobile development. For the past 2 months I have been a full time Cloud developer. Although I still do side projects and most of them will most likely still be mobile, I will be working on Cloud based solutions in my day job.

Working with Native Bitmap pixel buffers in Xamarin.Forms

I mentioned in my previous post that extracting pixel buffers from native Bitmap API’s can be quite tricky. In this post I would like to share the approach that I took for extracting native Bitmap pixel buffers into an collection of Xamarin.Forms.Colors objects so it can be used from a portable class library. I wrote and used a more complex version of the code mentioned in this post on my last project where I was working with image detection and color analysis for an app using Xamarin.Forms, in this project all my color analysis was done in a Portable Class Library using an abstraction over the native bitmap data.

BitmapData abstraction

In .NET you had access to an API called System.Drawing.Bitmap which encapsulates a low-level Windows API called Bitmap from GDI. The managed Bitmap class exposed a method called LockBits which in return gave you a BitmapData instance. BitmapData exposed information that allows you to manipulate the pixel buffer at a pointer level and is the fastest and recommended way to analyze and manipulate pixel information. I loved the BitmapData class but my portable class library implementation will not contain anything but a pixel buffer in ARGB and a Get/SetPixel(x,y, Color) method and a method for getting the average color of a certain area in the Bitmap to demonstrate what this can be used for

Here’s the code

public class BitmapData
{
public BitmapData(Size size, int[] pixelBuffer)
{
Size = size;
PixelBuffer = pixelBuffer;
}

public int[] PixelBuffer { get; }

public Size Size { get; }

public Color GetPixel(Point point) => GetPixel(point.X, point.Y);

public Color GetPixel(double x, double y) => Color.FromUint((uint)PixelBuffer[(int)x * (int)y]);

public void SetPixel(Point point, Color color) => SetPixel((int)point.X, (int)point.Y, color);

public void SetPixel(double x, double y, Color color) => PixelBuffer[(int)(x * y)] = (int)(color.A * byte.MaxValue) << 24 |
((int)color.R * byte.MaxValue) << 16 |
((int)color.G * byte.MaxValue) << 8 |
((int)color.B * byte.MaxValue) << 0;

public Color GetAverageColor(params Rectangle[] rectangles)
{
var colors = new List<Color>();
foreach (var rectangle in rectangles)
for (var y = rectangle.Y; y < rectangle.Y + rectangle.Height; y++)
for (var x = (int)rectangle.X; x < (int)rectangle.X + (int)rectangle.Width; x++)
colors.Add(GetPixel(x, y));

var red = (int)(colors.Average(c => c.R) * byte.MaxValue);
var blue = (int)(colors.Average(c => c.G) * byte.MaxValue);
var green = (int)(colors.Average(c => c.B) * byte.MaxValue);
var alpha = (int)(colors.Average(c => c.A) * byte.MaxValue);

return Color.FromRgba(red, blue, green, alpha);
}
}

UIImage to BitmapData (iOS)

To get the pixel buffer from a UIImage instance we need to draw it to a new drawing surface by calling DrawImage on an CGBitmapContext instance. When we construct the drawing surface we specify the pixel format and provide a pointer or an array of bytes in which the data will be written to. We need to specify that the pixels will contain a byte for each component, 4 bytes per pixel, and that the byte order is 32-bit Big Endian. We can also specify whether we specify the alpha component is in the most or least significant bits of each pixel, but for this example I will put it in the end since when I was researching about this, most of the examples I found used the least significant bit to store the alpha component.

Here’s the code

public BitmapData Convert(object nativeBitmap)
{
var image = (UIImage)nativeBitmap;
return new BitmapData(new Xamarin.Forms.Size(image.Size.Width, image.Size.Height), GetPixels(image));
}

private static int[] GetPixels(UIImage image)
{
const int bytesPerPixel = 4;
const int bitsPerComponent = 8;
const CGBitmapFlags flags = CGBitmapFlags.ByteOrder32Big | CGBitmapFlags.PremultipliedLast;

var width = (int)image.CGImage.Width;
var height = (int)image.CGImage.Height;
var bytesPerRow = bytesPerPixel * width;
var buffer = new byte[bytesPerRow * height];
var pixels = new int[width * height];

var handle = GCHandle.Alloc(buffer);
try
{
using (var colorSpace = CGColorSpace.CreateGenericRgb())
using (var context = new CGBitmapContext(buffer, width, height, bitsPerComponent, bytesPerRow, colorSpace, flags))
context.DrawImage(new RectangleF(0, 0, width, height), image.CGImage);

for (var y = 0; y < height; y++)
{
var offset = y * width;
for (var x = 0; x < width; x++)
{
var idx = bytesPerPixel * (offset + x);
var r = buffer[idx + 0];
var g = buffer[idx + 1];
var b = buffer[idx + 2];
var a = buffer[idx + 3];
pixels[x * y] = a << 24 | r << 16 | g << 8 | b << 0;
}
}
}
finally
{
handle.Free();
}

return pixels;
}

Bitmap to BitmapData (Android)

This is pretty easy to do in Android as the Bitmap class exposes the GetPixels method to get the pixel buffer and the pixel information is conveniently stored in ARGB

Here’s the code

public BitmapData Convert(object nativeBitmap)
{
var bitmap = (Bitmap)nativeBitmap;
var info = bitmap.GetBitmapInfo();
var pixels = new int[info.Width * info.Height];
bitmap.GetPixels(pixels, 0, (int)info.Width, 0, 0, (int)info.Width, (int)info.Height);
return new BitmapData(new Xamarin.Forms.Size(info.Width, info.Height), pixels);
}

WriteableBitmap to BitmapData (Universal Windows Platform)

To do this using the Universal Windows Platform is a bit similar to iOS but is less complex. The WriteableBitmap class exposes a PixelBuffer as an IBuffer in which you can call the extension method ToArray() on to get an array of integers. The interesting part about the WriteableBitmap PixelBuffer is that it doesn’t really say anywhere in the documentation (at least not directly) that the component order is BGRA, I only figured this out by reading the sample code provided in the WriteableBitmap documentation where it says in a code comment that WriteableBitmap uses BGRA format.

Here’s the code

public BitmapData Convert(object nativeBitmap)
{
var imageSource = (WriteableBitmap)nativeBitmap;
var pixelData = GetPixelDataFromImage(imageSource).ToArray();
return new BitmapData(new Size(imageSource.PixelWidth, imageSource.PixelHeight), pixelData);
}

private static IEnumerable<int> GetPixelDataFromImage(WriteableBitmap imageSource)
{
const int bytesPerPixel = 4;
var pixelHeight = imageSource.PixelHeight;
var pixelWidth = imageSource.PixelWidth;
var buffer = imageSource.PixelBuffer.ToArray();
var pixels = new int[buffer.Length];

for (var y = 0; y < pixelHeight; y++)
{
var offset = y * pixelWidth;
for (var x = 0; x < pixelWidth; x++)
{
var idx = bytesPerPixel * (offset + x);
var b = buffer[idx + 0];
var g = buffer[idx + 1];
var r = buffer[idx + 2];
var a = buffer[idx + 3];
pixels[x * y] = a << 24 | r << 16 | g << 8 | b << 0;
}
}

return pixels;
}

I remember struggling quite a bit when I was figuring out what I just shared and I hope that some one out there might be able to make some good use of it.