AdityaNag.com

Should all AI Systems be Explainable?

Wed, 20 Jan 2021 22:06:00 -0500

AI systems are increasingly making important decisions for ever-increasing numbers of people. The decisions these systems make can literally change people’s lives – and as AI systems become more entwined into real-world systems, the decisions they make need to be trustable.

The use of Black-box AI systems to make these decisions brings up basic questions of public trust. In this note, we’ll examine ways to mitigate these issues without losing the many benefits of using AI systems.

Types of Black-box Systems

There are two kinds of black-box AI systems.

The first is where the AI system is proprietary. The workings of the system are known but are held as a trade secret. The appropriate use of policy (corporate or political) can build trust in such systems, and we can consider this a solved problem that requires only the formulation of a good disclosure policy.

The second kind of black-box AI system is one that is too complicated for any human to comprehend, and the decisions made by these systems cannot be fully explained - even by the creators of the system.

Even if the decisions made by such an AI system are generally good, a society or business where important decisions are made by a mysterious entity that cannot truly be held to account runs contrary to democratic and corporate ideals of transparency and accountability.

AI Explainability has emerged as a major area of research that is attempting to answer challenging questions posed by the widespread deployment of the second type of black-box AI system.

Explainability Explained

An Explainable AI System is a system where the decision made by the AI can be explained to a wide range of users, and these users accept the explanation as reasonable, correct, and timely.

Before we go further, however, we should answer the fundamental question of why one should even care about explainability.

Let’s say you get a letter tomorrow that tells you your auto insurance rates are going up by 30%. If the letter does not provide an explanation, or provides an inadequate explanation, you are going to call your provider and you’re going to demand an explanation.

Depending on what they tell you, you can then do things; you move to another provider, you agree to pay the increased rates, or perhaps you negotiate a new rate.

Thus, explanations are solicited by humans for everyday or “local” events to help understand why something happened or is about to happen. An explanation empowers people to take actions that can lead to favorable outcomes for themselves.

Explainability is a basis for Trust

The lack of an explanation is funny when the AI gives you a head scratcher of a song recommendation, but similar AIs can be tasked with picking the right candidate for a job, or deciding who gets parole. These decisions demand an explanation, and governments across the world are starting to codify this demand into law.

The European Union came close to implementing a binding “Right to Explanation” into the General Data Protection Regulations (GDPR), but eventually published a non-binding version into law in 2016.

It is reasonable to assume that future laws will contain stronger versions of similar rights, and businesses need to be prepared to handle increasing demands for transparency and explainability from their customers as well as the government.

There are many benefits to having the explanation for an AI’s decision be generally accepted as reasonable and correct: User confidence and trust in the system will grow; there is a built-in safeguard against accusations of bias; the system can be shown to meet regulatory standards or policy requirements; and the developers of the system can continue to improve its performance.

Building Explainable AI Systems

There are various ways to build Explainable AI Systems. Three commonly used methods are listed below, but researchers are constantly working on creating new solutions.

Self-explainable models – the model itself is the provided explanation. This is the most interpretable and easily understood approach, and eliminates the issues associated with black-box models.
Global Explainable AI algorithms – this an approach that treats an AI algorithm as a black-box that can be queried to produce a model that explains the algorithm. This is not as good as a self-explainable model, but does allow for a higher degree of confidence in the decisions made by the system.
Per-Decision Explainable AI algorithms – these algorithms take both a black-box model that can be queried, and a single decision of this model and explains why the model made that particular decision.

Another way of verifying the algorithm comes through the use of counterfactuals – “what if” questions that can contrast multiple scenarios. For example, if an AI system takes an input of “Play a pop song from the 2000s” and responds with a certain decision, the response to the counterfactual “What if I ask for a rock song from the 2000s” can give us a better understanding of the inner workings of the system

Should all AI Systems be Explainable?

Given that it is apparently possible to build explainable AI systems, should all AI systems then be explainable?

On the face of it, this would seem like a reasonable thing to do. After all, can’t human beings explain all the decisions that they make?

Or can they?

As it turns out, human-produced explanations can be quite unreliable. Even more surprisingly, it turns out that for certain kinds of decisions, forcing a person to provide an explanation actually lowers the quality of the decision!

Gaining expertise in a subject makes the decision-making process increasingly unconscious and automatic, and forcing an explanation interferes with this automatic process.

Elite athletes are an obvious example – Roger Federer instinctively knows where to hit a tennis ball to win the point, but if he started to think about it as he hit it, he’d probably make a mess of it (or not, but then he’s Roger Federer).

This is an obvious analogue to certain black-box AI systems that appear to be delivering amazing results, and forcing all such systems to be explainable may result in losing the significant value of these systems.

The fundamental issue causing unease is the lack of accountability. It is not possible to hold an AI system accountable for a bad decision in the same way that we can hold a human being accountable. In addition, punitive structures that work with humans cannot be made to work with AI systems.

Human beings constantly self-regulate against a complex background of social and moral norms, business rules, and civil and criminal laws. An AI may be built to try to self-regulate in a similar manner, but if it fails there are no real consequences to that failure.

Trying to hold the creators of the system accountable is unjust on the face of it, since they themselves do not know why their system behaves the way it does after a certain point.

When is Explainability necessary?

Since we are primarily concerned with the lack of accountability, it would appear that any system where the decision is important enough to demand accountability needs to be explainable.

Systems that can make life-or-death decisions, or decisions where people’s lives can be affected in serious ways must make explainable decisions.

This can be regulated through policy (insurance companies in the US must be able to explain how they arrived at a rate decision) or will end up being regulated through social forces (the pushback against self-driving cars and social media engagement algorithms). There is little appetite for a black-box AI to make decisions of this nature.

However, there are many applications where AI systems may not need to be formally explainable. Many industries have decision making systems that don’t need perfect reproducibility or explainability.

For example, think about using AI to regulate the HVAC system for a large office building. As long as the temperature and humidity are maintained within certain norms, people are unlikely to ask for precise explanations why the system decided to set the thermostat to a certain temperature. The system can lead to large savings in energy costs.

Or take the case of a mapping application. As long as it gets you from point A to B in a safe and mostly reliable manner, you are not likely to ask why it took Daneel Street instead of Giskard Street. The benefits outweigh the consequences even if the system gets it occasionally wrong.

We do not necessarily have to make a trade-off (if one even exists) between utility and explainability.

As we enter the third decade of the 21st century, we can see the shape of the future to come. AI Systems are a real and important part of this future, and methods to increase the trust in these systems can only make them a more valued part of our society.

And perhaps this is a precursor to that distant dream of true Artificial General Intelligence – but that’s a subject for another time!

Ubuntu 18.04 LTS on a 2017 (5th Gen) Thinkpad X1 Carbon

Sun, 12 Aug 2018 20:06:00 -0400

I’ve recently decided to return to my open source roots. I’ve been using Windows 10 for the past couple of years, and OS X for a few years before that.. but back in the mid-2000’s, I ran only Linux on all my machines.

For one reason and another I’ve been growing frustrated with the state of proprietory operating systems, especially with all the emphasis on pushing all your data to the cloud and making everything subscription based. I can’t be the only one that just wants a simple machine that works without constantly wanting to nag me with “helpful suggestions”?

So this past weekend, I decided to try installing Linux on my laptop. I didn’t want to dual-boot, because the only way to really get to know an OS is to use it full time. I did leave myself an escape hatch though, just in case things go wrong, by using a completely new SSD. I took out the SSD with Windows on it and kept it aside, and put in a brand new 256 GB SSD. This blog post chronicles my first impresssions of Linux, in 2018, on a modern laptop.

The Hardware

I have a 2017 (5th Gen) Thinkpad X1 Carbon. It has 16 GB of RAM, and an Intel Core i7-7600U CPU @ 2.80GHz, so it’s no slouch, and the battery lasts anywhere between 5-9 hours in Windows. I chose the Full HD screen, so I don’t need to worry about High DPI support (more on this later). I checked online and apparently my machine is very well supported by all the major Linux distros, so I was feeling pretty good about trying out a few.

Fedora 28

I tried Fedora first. Installation went smoothly, and everything (with one exception) worked out of the box. Wifi, bluetooth, Thunderbolt, all the function keys for volume, brightness, etc.. everything except the fingerprint reader. I knew this going in so I wasn’t disappointed. However, I ran into a software issue that forced me to stop using Fedora.

I have a Synology NAS, and I use CloudSync (Like Dropbox, but it syncs to my own NAS) to keep my files in sync between my various machines. I was unable to find an RPM for CloudSync - Synology provides a .deb instead. I did about 20 min of Googling, but didn’t find an immediate solution. A couple of forum posts said I could use alien to convert the deb… but I really couldn’t be bothered.

This wasn’t too much of an issue since I was intending to try Ubuntu 18.04 LTS in any case. I tried Fedora just for the heck of it. I’m sure if I spent a couple of hours I would have found a solution, but I decided to move on to Ubuntu instead.

Ubuntu 18.04 LTS

Once again, installation was quick and easy. I like the new minimal install - instead of getting a lot of software, the minimal install just sets up a Firefox and the base Gnome system. There’s no Libre Office, or RhythmBox, or Thunderbird etc.

Once again, all the hardware (except the fingerprint scanner) worked out of the box. I didn’t have to tweak anything at all. This was a pleasant surprise - yes, I know my particular laptop is very Linux-friendly, but there’s still something nice about not having to tweak config files.

Tweaks and Software

Half the fun of using Linux is customizing your system to suit you. I spent a fun couple of hours getting re-acquainted with Gnome Extensions and Themes, and ended up with something that I really think works well for me. The feeling of not being beholden to a giant mega-corp is the cherry on top.

Here’s a quick list of software that I installed

TLP -> https://linrunner.de/en/tlp/tlp.html
KeePassXC -> https://keepassxc.org/
Remmina -> https://remmina.org/
Visual Studio Code -> https://code.visualstudio.com/ (this is free and Open Source)

And the Gnome Extensions that I used

Frippery Move Clock - moves the clock to the right. I have no idea why a clock in the middle of the screen makes any sense
Panel OSD - Notifications now show up on the right, not the middle of the screen
Transparent Gnome Panel - Does what it says on the tin

As you can see, I didn’t go too crazy with the extensions. I know from experience that it’s best to keep things simple.

Conclusion

It’s been a few days now, and things are really working well. I’ll post an update after a month, or possibly sooner.

Wiring up ASP.Net Identity with Custom Classes

Sun, 29 Jul 2018 00:28:00 -0400

I looked for a simple tutorial that shows how to hook up ASP.Net (Core & .Net) Identity with user generated classes, and I couldn’t really find one. So here are my notes on how to do it. These are rough notes, written quickly. I’ll come back later and update this tutorial so it’s more comprehensive.

The Goal

Create a simple CRUD app in ASP.Net Core 2.1 as well as ASP.Net MVC 5. This app should have user logins via the built in ASP.Net Identity service, and users should be able to save their data independently of other users. For example, a simple ToDo app. A user should be able to register, log in, and create, read, update, and delete their own todos - but other users should not be able to see their todos.

Sounds pretty simple, right? Well, it took me a few hours to figure out.

ASP.net Core 2.1 Example (Razor Pages)

Use Visual Studio templates to create a WebApp with Identities. Build and run to make sure we can log in. Nothing complex here, just tick the boxes in the wizard (or copy and paste into the terminal).

Next, create a custom class. Something like this

public class TodoListItem
{
    public int Id { get; set; }
    public string Title { get; set; }
    public bool isCompleted { get; set; }

    public virtual string ApplicationUserID { get; set; }
}

The critical line is the last one. By adding in the ApplicationUserID, we’re setting up a relationship between the auto-generated tables created by ASP.net Identity, and our custom class.

The next thing is to use scafollding to create the CRUD pages from this class. Again, just use Visual Studio’s built in features. Makes sure to choose the existing ApplicationDBContext so that the tables are properly linked.

Once the scaffolding is done, we need to edit the code to bring in the currently logged in user’s User ID. In ASP.Net Core, this is how you do it

ClaimsPrincipal currentUser = this.User;
  var currentUserID = currentUser.FindFirst(ClaimTypes.NameIdentifier).Value;

These two lines of code are used in the controller or Razor code behind to get the userID. Once we have the user Id, we can simply set the object’s property directly before we save it.

To retrieve the currently logged in user’s items, use this code

ClaimsPrincipal currentUser = this.User;
    var currentUserID = currentUser.FindFirst(ClaimTypes.NameIdentifier).Value;
    ToDo = await _context.TodoItem.Where(k => k.ApplicationUserID == currentUserID).ToListAsync();

Again, the important bit is getting the UserID, and then using LINQ to get all items that have the same UserID.

In ASP.NET MVC5, the user ID is returned in a different way.

var currentUserID = User.Identity.GetUserId();
    var currentUsersTodo = db.TodoListItems.Where(k => k.ApplicationUserID == currentUserID).ToList();

Adding the DBSet to the DBContext

Once figured out, it’s quite easy to create classes that are linked to the current user. Remember, we have to add the DBSet to the existing DBContext

This is different in .NET Core and .Net MVC.

In .Net Core

The file is called ApplicationDBContext.cs and it’s found inside the “Data” folder.

In .Net MVC 5

The DB Context is inside the IdentityModels.cs file, which is located in the “Models” folder

public DbSet<PriceTracker.Models.GroceryItem> GroceryItem { get; set; }

When a CMS is too complex

Sun, 03 Jun 2018 15:01:00 -0400

I spent the past few days working on Umbraco. I thought I might go back to using a CMS for this website, as I have some ideas for adding some features and functionality.

Umbraco is really powerful, but after spending hours on wiring it up by hand, I decided that it was too much work for a simple site. Writing in markdown and publishing a static site is a lot easier in my particular use case.

I still want to add some features to this site though, so I’m going to try and incorporate that under a sub-directory. That would mean moving this site from Netlify to my own server.

Let’s see how it goes. For now, I’m keeping it simple.

Using Linux through Visual Studio Code

Sun, 08 Apr 2018 01:37:00 -0400

Here’s how you can set the Windows Subsystem for Linux to be the default terminal inside VS Code. This allows for a very simple way to launch a bash prompt:

Ctrl + `

The first thing to do is install the Windows Subsystem for Linux, and then your distro of choice from the Windows Store. There are tons of guides for this, so I’m not going into the details - just search “How to install Windows Subsystem for Linux” and pick a guide that works for you.

Now that you have Linux installed and running, edit VS Code’s preferences. Use the keyboard shortcut:

Ctrl+,

or click on File > Preferences > Setting

Use this line to set the default terminal

"terminal.integrated.shell.windows": "C:\\Windows\\System32\\bash.exe",

Once this is done, save and restart VS Code. Now just use the Ctrl+` shortcut and VS Code will launch an intergated bash prompt with the current working directory set to the folder you have open.

If you want to change the CWD, use this line

"terminal.integrated.cwd": "C:\\Users\\$USERNAME\\AppData\\Local\\Packages\\CanonicalGroupLimited.UbuntuonWindows_79rhkp1fndgsc\\LocalState\\rootfs"

I set mine to the root folder in bash. Your username and the package name will be different, of course. You can set the path to anything you like.

This lets me write my blog posts, and then update my site from inside VS Code. Makes things nice and easy. It only takes me one quick command.

Automated Deploy Pipeline to Netlify

Thu, 29 Mar 2018 23:08:00 -0400

Through the magic of simple bash scripting, I now have a very easy deployment pipeline for my website. I wanted to do this all in Visual Studio Team Services, but as far as I can tell, I’ll need to register a custom Linux deployment agent (aka a Linux VM running somewhere).

Since I don’t want to do that, I used the Windows Subsystem for Linux, bash scripting, and the Netlify API to automate my deploys. This is how it works.

I write my blog post in my text editor. I save it, and push the commit to VSTS - this is mostly just for backup, since the deploy is all happening on my machine. After pushing to VSTS, I run this simple script in WSL:

!#/bin/bash

jekyll build -s $PATH_TO_SITE/AdityaNag.com > /dev/null 2>&1 
zip -r site.zip _site > /dev/null 2>&1
curl -H "Content-Type: application/zip" -H "Authorization: Bearer <OAUTH_ID>"--data-binary "@site.zip" https://api.netlify.com/api/v1/sites/adityanag.netlify.com/deploys > /dev/null 2>&1

This generates the site, zips it up, and deploys it to Netlify. The entire process takes between 25-40 seconds (my site isn’t very big). I could shrink that further by doing incremental builds or something, but I’m ok with this for now.

Finally, I use the email notification feature in Netlify to notify me when the build is sucessfully deployed.

I wish VSTS would provide a Linux agent that has Jekyll built in. I suppose I could always switch to Hugo, but that’s a bigger project - maybe a weekend project!

Moving to Visual Studio Team Services

Tue, 27 Mar 2018 13:52:00 -0400

Github pages are great, but they have one flaw - you have to have a public repo. This meant that I couldn’t really write drafts, or make future edits without the world knowing. I found that this was holding me back from updating my blog.

So today, I moved my site’s repo from Github to Visual Studio Team Services. It’s free, with private repos. Unfortunately, VSTS doesn’t really have a good way to deploy to Netlify (Free static hosting? Yes please), without setting up a convoluted build pipeline that I frankly don’t have the patience to do.

My interim solution is very simple. Write my posts on my local machines, and then use Jekyll via the Windows Subsystem for Linux to generate the static site. After that, simply zip it up and manually drag and drop into Netlify.

I know I can automate this, and I probably will when it starts getting tedious, but for now, it’s good enough. From finishing a new post to publishing takes me less than a minute.

Sometimes the easy solutions are the best.

My own personal cloud

Tue, 01 Aug 2017 12:47:00 -0400

Last Saturday, I colocated my server. This server has been living under the guest room bed for the past year, being used for everything from test labs to providing NAS backup for my devices. It worked really well, letting me spin up VMs at will without paying Amazon or Microsoft for the privilege. However, we just moved to a new apartment which is smaller and doesn’t have FIOS, and now I have no good place for it.

I thought about selling the server, but that doesn’t make sense as I do need something that acts as a testbed. And so I decided to colocate the server. I also looked at getting a dedicated server but those cost a LOT of money for the config I have (96 GB RAM, 8TB HDD, 24 cores) - I’d be paying hundreds of dollars a month for a similarly powerful server.

Never having coloed before, I didn’t quite know what to expect. I didn’t want to ship the server, so it would have to be a local colo provider, and I started searching and calling around. After getting some eye-watering quotes, I finally found a provider that understood my homelab-ish needs and gave me what I consider a quite reasonable deal.

I’m paying under $90 for a single 2U server, 2A of redundant power, 1 Gbps unmetered bandwidth, and a /29. There are a few caveats though - I told the provider that I’m running a homelab with minimal bandiwdth needs, so he agreed to give me a high speed connection with no metering. They’ll monitor the connection for three months and if I go past what they consider reasonable (more than 5 Tb, I think he said), they’ll let me know and we can talk about moving to a different plan. The same goes for power - if my server is routinely pulling down more than 3A, I’ll have to pay more.

This really worked out well for me. I know I’m not ever going to hit those bandwidth or power limits, so I can effectively run this however I want. Having public IPs is really nice too, giving me the ability to run some services on my own hardware rather than pay Digital Ocean - I’m saving $15 a month there, so my colo bill comes down to $75.

I have setup an IPSEC tunnel to the server so it feels just like it’s on the local LAN. Latency is around 30ms, and Remote Desktop and SSH work absolutely smoothly. I really should have coloed years ago. I’m saving $20 a month on power, $15 for my DO server which I no longer need, so for a sumtotal of $55 a month, I get a super high-speed connection to the net, my own cloud, and the ability to run it however I want.

Summer is finally here

Fri, 19 May 2017 15:41:00 -0400

It’s almost the end of May here in Massachusetts, and it’s finally starting to get warmer. And I’ve been learning Xamarin Forms, though I’m toying with the idea of just going straight to native. I have invested a couple of years into C# though, so it’s hard to move away from right now. I’m hoping that Xamarin Forms works out well for me.

I’ve created a basic CRUD app in Xamarin Forms, and it works well on my iPhone and my Android tablet. UWP works as well, of course, but I’m not focused on UWP till after the Fall Creators Update. I’m looking forward to learning some Fluent Design. I tried getting started with the SDK, but they hadn’t released all the bits when I looked. I’ll probably look again next month.

The Windows Subsystem for Linux Changes Everything

Thu, 13 Apr 2017 01:31:00 -0400

I used to run my site on Wordpress. Roughly two years ago, I moved it to Jekyll. I did this as I was tired of managing Wordpress, and didn’t want to deal with running a Database driven site anymore. Also, I wanted to host on Github, removing the hassle of running a server.

It’s worked well for the most part, but I’ve always had an issue with running Jekyll on my Laptop. I like to run Windows (though I have a Mac, and also run Linux), and it was a huge pain to get Jekyll running smoothly on Windows. So much so that I gave up. On Ubuntu, installing Jekyll is as simple as apt install jekyll, and I would use my Linux VM if I really wanted to test something locally. The Mac is easier than Windows, but I much prefer apt to homebrew, and I find it easier to get good documentation for Linux tools rather than the same tools on a Mac.

I started looking at the Windows Subsystem for Linux last year, and immediately realized that this is a game changer. Y’see, while I love Linux on the server, and all the lovely development tools and workflow, I’ve never really liked using it as a desktop. Back in the day, I ran purely on Linux for two years, but eventually gave up cause I got tired of constantly tinkering with my machine just to keep basic stuff running (editing X conf files for multi-monitor support..argh). So I ended up with a split workflow. Windows/OS X for daily activities and general computing, and Linux on the server.

The Windows Subsystem for Linux (WSL) is brilliant cause it allows me to use Windows as a desktop, while still covering my Linux needs. Take this blog, for example. Here’s what I did to get it running on my local machine

Install WSL
Git clone repo with one command git clone reponame
apt install jekyll
jekyll serve

That’s it. Now I’m writing this blog post on Windows with Visual Studio Code, while Jekyll runs in the background auto-generating the site everytime I hit save. Chrome is open, I hit F5, and everything updates. It’s magical!

Inotify works, so Jekyll (running under Linux via WSL) notices that I’ve edited a file (in VS Code running under Windows) and it regenerates the site. THIS is how it should be.

WSL in the new Creators Update covers 100% of my needs. I can run nginx, mysql, gdb, gcc, jekyll, node, .NET Core (which is fun, since I’m running .NET code on Linux on Windows). I can use Visual Studio to debug a Linux applicaiton running on my local machine. I can open a linux file in a windows app and a windows file in a linux app. I can use bash scripting and the power of sed, and awk, and grep and all the lovely bash tools to parse any file on my system. It’s truly the best of both worlds for me. Native SSH too.

Microsoft has ensured that my next laptop purchase will definitely be a Windows laptop, and not a Mac (even as I write this on a 2015 Retina MBP, running Windows via VMware Fusion, with Linux on Windows on a Mac… what a time to be alive!).

WSL is a game-changer and I can only imagine how much better it’s going to get.

2017 is the year of Linux on a Desktop - but the Desktop is running Windows. Think about that, and marvel.