Lwin Maung

View Menu
Blog Post 1

Securing information via face and moreLeveraging Facial Recognition in your systems

Security is an important aspect of our lives and we use various means of security every day. In order for us to secure items in our lives, we have to have a trust factor. This applies to our bank accounts, our passwords, and even vehicles that we utilize every day. Sometimes, security can be complex -- in the case of bank account being accessed over the internet, or a rolling security codes in your car's remote start module, or RFID and barcodes in your driver license or passport. At the end of the day, trust is what allows someone or a network system to allows for access. When you go to the bank, you have to provide a proof of who you are in a form of ID to even have a banker access your account. Now, let's think of something simple. Let's go back in time. Imagine that you are a newborn baby. The first few things you learn to trust is the sound of your mother. Your eyes are nowhere near 20-20 vision and the only thing you are you know is that you are potentially hungry. An infant mind has to associate the sound of your parent by a learned process. As you get older, you can now see a little better. As your senses grow, you learn to use them to authenticate your trust factors. Within months, you learn to recognize faces. You learn to associate familiar faces as good (trustworthy) people while unknowns as (potentially) bad non-trustworthy people in your short life thus far. Computers can be trained to think and act just like humans as well. Using technologies such as Microsoft's Cognitive Services, we use to duplicate the learning processes that we learned as infants. We don't have to use 256k secured keys to protect the contents of our digital lives. We can use our faces, voices, fingerprints and more to allow for physical computers to issue trust using biological traces that are unique to us humans. The best part is, just like infants, the computers can be taught to trust the good identifiable features from bad ones by reinforced training. Let's see how that is accomplished.

Let's explore face detection first. In order for us to demonstrate this, we will need an image capture device. For simple recognition, any webcams will do. If you want to make sure that a face is a true 3D object, we will need to use a Windows Hello enabled camera such as Logitech 4K webcam. Once you have a camera, the rest of the process is simple. We will capture an image (or a single frame image from a video stream), send the image to Azure Cognitive Services (Face API Service) and let the service decipher the face for us. The Face API service can perform the followings:

  • Face Detection
  • Face Recognition
  • Face verification

 

What does that mean?

 

Face

Face detection

The Face API can detect up to 64 human faces with high-precision face location in an image. The image can be specified by file (a byte stream) or with a valid URL. The face rectangle (left, top, width, and height) indicating the face location in the image is returned along with each detected face. Optionally, face detection extracts a series of face-related attributes such as pose, gender, age, head pose, facial hair, and glasses.

Face recognition

The ability to identify human faces is important in many scenarios including security, natural user interface, image content analysis and management, mobile apps, and robotics. The Face API service provides four face recognition functions: face verification, finding similar faces, face grouping, and person identification.

Face verification

Face verification performs an authentication against two detected faces or from one detected face to one person object. Using face verification, other types of information can be derived. Given a target detected face and a set of candidate faces to search with, the service finds a small set of faces that look most similar to the target face. Two working modes, matchFace and matchPerson are supported. matchPerson mode returns similar faces after applying a same-person threshold derived from Face - Verify. matchFace mode ignores the same-person threshold and returns top similar candidate faces. In the following example, candidate faces are listed.

 

Detect

What does the result look like?

Face detection would send successfully detected face as a JSON object. A sample is shown below:

[ { "faceId": "f7eda569-4603-44b4-8add-cd73c6dec644", "faceRectangle": { "top": 131, "left": 177, "width": 162, "height": 162 }, "faceAttributes": { "smile": 0.0, "headPose": { "pitch": 0.0, "roll": 0.1, "yaw": -32.9 }, "gender": "female", "age": 22.9, "facialHair": { "moustache": 0.0, "beard": 0.0, "sideburns": 0.0 }, "glasses": "NoGlasses", "emotion": { "anger": 0.0, "contempt": 0.0, "disgust": 0.0, "fear": 0.0, "happiness": 0.0, "neutral": 0.986, "sadness": 0.009, "surprise": 0.005 }, "blur": { "blurLevel": "low", "value": 0.06 }, "exposure": { "exposureLevel": "goodExposure", "value": 0.67 }, "noise": { "noiseLevel": "low", "value": 0.0 }, "makeup": { "eyeMakeup": true, "lipMakeup": true }, "accessories": [ ], "occlusion": { "foreheadOccluded": false, "eyeOccluded": false, "mouthOccluded": false }, "hair": { "bald": 0.0, "invisible": false, "hairColor": [ { "color": "brown", "confidence": 1.0 }, { "color": "black", "confidence": 0.87 }, { "color": "other", "confidence": 0.51 }, { "color": "blond", "confidence": 0.08 }, { "color": "red", "confidence": 0.08 }, { "color": "gray", "confidence": 0.02 } ] } } } ]

What is notable is that we can see the types of information being detected in an image (or a frame of a video). Azure Face API can detect a face and determine if a person is smiling, sex of a person, age, facial hair and head composition, emotions such as happiness, sadness, disgust, and even if a face is occluded by various means such as eyeglasses, hair, or makeup.

Train

In order for us to verify faces, we need to identify and teach the API to associate detected faces with a person by training a model that holds the face detection. Each and every time, we detect a face, we can use the PersonGroup api call and add and train the person or person to a list of recognized people for verification purposes.

 

TrainingStatus trainingStatus = null;
while(true)
{
  trainingStatus = await faceServiceClient.GetPersonGroupTrainingStatusAsync(personGroupId);
  if (trainingStatus.Status != Status.Running)
    {
      break;
    }
  await Task.Delay(1000);
}

You add a user by adding this to code block:
await faceServiceClient.AddPersonFaceAsync( personGroupId, personX.PersonId, File.OpenRead(imagePath));

Identify

Once the user is trained, we can then use https://[location].api.cognitive.microsoft.com/face/v1.0/identify call to identify the person on the verify list. If the face or faces are identified on an image, a result similar to the JSON object below will be returned to you so that you can verify the person.

[ { "faceId": "c5c24a82-6845-4031-9d5d-978df9175426", "candidates": [ { "personId": "25985303-c537-4467-b41d-bdb45cd95ca1", "confidence": 0.92 } ] }, { "faceId": "65d083d4-9447-47d1-af30-b626144bf0fb", "candidates": [ { "personId": "2ae4935b-9659-44c3-977f-61fac20d0538", "confidence": 0.89 } ] } ]

Next steps

We can consume the data at this point and keep training the model for additional accuracy. What we have to realize is that just like a child, we started in "Tabula Rasa" state and we will have to fill in that blank slate with data over time for better analysis over time. Next, we will discuss the Windows Hello and using 3D face matching with better accuracy and more.

 

Read More
Blog Post 1

Deploying LOB Store Apps (via Sneaker Net and beyond)

As a consultant, I run into weird one of a kind situations from time to time. Hopefully this post will help the next person from trying to figure things out from scratch.

Situation: You have written a Universal Windows App (C#/JS does not matter). This app needs to be deployed to devices. You can not use store deployment since the app will not be distributed via the stores. The app will be distributed to devices not in the same domain and not controlled by traditional admins.

So I scratched my head a number of times and spun anround the tracks for a while before I came up with the following working solution:


Create your UWP Package

Step 1. Write your UWP (Test etc).

Step 2. Once the app is ready, we have to sign and create the app for deployment. You need to get an official code signing cert from a trusted certification provider. Digicert comes to mind. Take your .pfx and import it to the machine. You can do so by double clicking on the .pfx file and following instructions.

Step 3. Now, go back into the visual studio project and select the newly imported certificate to sign your app. (package.appmanifest file)

Step 4. Save the project and build the project from command line using the following command:

 

MSBuild MyApp.csproj /p:Configuration=Release;AppxBundle=Always;AppxBundlePlatforms="x86|x64|ARM" /p:BuildAppxUploadPackageForUap=true

Step 5.  That will create a fully signed and ready to go package.

 

Deploy the package

In order for us to deploy the package, we have to do a few things. Make sure that the device that you are deploying is going to be in developer mode. Import the .cer file to the machine and finally installing the package.

Step 1. Developer Mode.
You can manually set the developer mode from settings app or set the registry keys from any traditional desktop app. Following code block sets the developer mode on a 64bit machine.

var basereg1 = RegistryKey.OpenBaseKey(RegistryHive.LocalMachine, RegistryView.Registry64);
var reg1 = basereg1.OpenSubKey(@"software\microsoft\windows\currentversion\appmodelunlock", true);
reg1.SetValue("AllowAllTrustedApps", 1, RegistryValueKind.DWord);
reg1.SetValue("AllowDevelopmentWithoutDevLicense", 1, RegistryValueKind.DWord);

Step 2. Import the cer certificate.
To import the certificate run the following code block. The runas verb will require admin access to the machine.

string cmdExe = "certutil.exe";
string path = "PathToYourUWPApp\\UPW APP.cer";
string cmdParm = "-addstore TrustedPeople " + path.ToString(System.Globalization.CultureInfo.InvariantCulture);
Process myProcess = new Process();
myProcess.StartInfo.FileName = cmdExe;
myProcess.StartInfo.Arguments = cmdParm;
myProcess.StartInfo.UseShellExecute = false;
myProcess.StartInfo.CreateNoWindow = true;
myProcess.StartInfo.Verb = "runas";
myProcess.Start();

You are esentially running this in command:

certutil.exe -addstore TrustedPeople PathToYourUWPApp.cer

Step 3. To install the app, you have to run the following command via powershell:

Add-AppxPackage -path PathToYourAppxPackage.appxpackage -ForceApplicationShutdown

That command set can be run via code as follows:

string cmdExe = "powershell.exe";
string path = "PathTOYourUWPApp.appxbundle";
string cmdParm = "Add-AppxPackage -path '" + path.ToString(System.Globalization.CultureInfo.InvariantCulture) + "' -ForceApplicationShutdown";
Process myProcess = new Process();
myProcess.StartInfo.FileName = cmdExe;
myProcess.StartInfo.Arguments = cmdParm;
myProcess.StartInfo.UseShellExecute = false;
myProcess.StartInfo.CreateNoWindow = true;
myProcess.Start();

Notice that add-package command does not require admin access so you can run it from any user as long as step 2 was done with admin user.

 

That should do the job. At this point, you can overwrite it with updated packages if needed by using lines in step 3 over and over again on the machine with updated appxpackage. If desired, you can even "phone home" and fetch an update and update or use a USB stick with a simple WPF/WinForm/CommandLine app and appxpackage for future distributions and updates.

 

Read More
Blog Post 1

Change Windows User Profile Image without code

There are many ways for a user to change the profile image on a Windows machine. Typical methods involves changing it manually by going to settings>acccount>update picture (on Windows 10) or similar methods. You can also do that via powershell and update your profile to that from AD (see link) or via UWP app using code (see link). What if you want to do it via command line?

In the case of command line, you will want to use this application: download userpic.exe

This is a command line app was pulled out of Microsoft RDX Demo experience.

The syntax is as follows:  

userpic.exe mynewphoto.accountpicture-ms

This will set the profile image to new photo. You can run it application in startup scripts, indiviually, batch scripts, or call it from apps.

 

 

Read More
Blog Post 1

Reading and writing (dangerously) to system registry

Read More
Blog Post 1

Microsoft Lumia 950 (3 months later)

Lorem ipsum dolor sit amet, consectetur adipiscing elit. Proin nec nisl nisl. Suspendisse gravida convallis ultricies. Sed mollis euismod ipsum, eu laoreet lacus feugiat quis. Donec nec felis lectus. Vivamus vitae sollicitudin turpis. Aenean elementum in nulla non varius. Duis interdum ornare metus id pellentesque. Mauris cursus ipsum sit amet fermentum pulvinar. Aliquam vel tempor risus. Sed et augue vel ante efficitur elementum quis sed est. Nam quis sem vestibulum justo sagittis venenatis in sollicitudin dui. Vestibulum rhoncus id diam eget accumsan. Nam cursus magna vitae risus dignissim, sed mattis ante vulputate.

Donec iaculis urna vitae facilisis ullamcorper. Curabitur quis finibus velit. Donec vitae nulla erat. Suspendisse maximus malesuada justo. Ut ut vehicula est, quis volutpat nisi. Proin faucibus, lorem vitae venenatis porttitor, risus elit cursus arcu, et aliquet dolor magna sed ipsum. Donec lobortis tortor nisi, sollicitudin suscipit risus vulputate non. Aliquam metus lorem, sodales et eleifend eu, placerat eu orci. Sed sagittis aliquam iaculis. Sed fringilla ligula sit amet velit pretium, id imperdiet ipsum imperdiet. Sed venenatis leo magna, non scelerisque ex suscipit id. Integer in risus eget justo sodales pretium. Aliquam urna leo, laoreet in accumsan et, feugiat vel mauris. Nam dignissim auctor lorem, vitae dignissim dui accumsan vel. Proin sit amet enim nunc. Integer iaculis vehicula nisl, sed aliquet mi facilisis id.

Donec gravida sapien consequat posuere varius. Pellentesque accumsan rutrum lacinia. Praesent tempus non justo vitae tincidunt. Nulla maximus lacinia accumsan. Curabitur porttitor augue eu metus mollis, ut lacinia augue rhoncus. Quisque aliquam urna ut tortor malesuada interdum. Praesent feugiat orci auctor tristique bibendum. Duis fermentum, nulla consequat egestas interdum, lorem mauris laoreet diam, ac sollicitudin mi turpis quis lacus. Sed eu consequat ligula, id cursus quam. Phasellus iaculis, nulla ac imperdiet pharetra, libero massa viverra arcu, auctor euismod augue est a sem. Sed congue ex vel ipsum luctus hendrerit. Sed sit amet massa luctus, iaculis lectus quis, iaculis nibh. Proin et convallis augue.

Nunc imperdiet pharetra velit sit amet egestas. Duis faucibus, nisl nec feugiat fringilla, diam eros auctor elit, in ultrices mi nulla eu lectus. Aliquam neque risus, auctor id porttitor ac, ornare eget arcu. Nulla sapien mi, lobortis sit amet ornare eget, laoreet a risus. Curabitur bibendum convallis metus id pulvinar. Mauris at ultrices lacus. Duis vulputate condimentum enim, vitae hendrerit lacus euismod in.

Aliquam odio nisl, tempor et neque rutrum, consequat porta sem. Aenean at egestas neque. Sed eget lorem in quam tristique consectetur. In tincidunt odio sollicitudin neque feugiat hendrerit. Donec nec egestas nisl, ut hendrerit ligula. Mauris sodales diam eget tortor varius, in gravida lacus viverra. Aenean quis mauris vehicula, iaculis augue ac, elementum est. Sed rhoncus ante vel eros aliquet commodo. Suspendisse potenti. Nunc est justo, gravida eu rutrum eu, vestibulum a risus. Vivamus quis massa quis nibh placerat aliquet et sagittis ipsum. Praesent gravida placerat diam id tincidunt. Morbi tincidunt neque vulputate tellus tincidunt, at finibus enim imperdiet.

Read More
Blog Post 1

3.14: Running Windows IoT on Raspberry Pi 2B

Read More
Blog Post 1

Surface Book: Developer Machine?

Lorem ipsum dolor sit amet, consectetur adipiscing elit. Morbi sed enim mattis lacus interdum pellentesque. Sed quis neque dui. Vivamus nulla magna, tempor in euismod quis, faucibus et dolor. Nam non placerat libero, sit amet posuere quam. Proin eu efficitur purus. Phasellus ac sodales leo, non eleifend velit. Aliquam fermentum metus vel nisi elementum, sed semper elit suscipit. Pellentesque ut rutrum erat, non ornare turpis. Sed sit amet magna et metus imperdiet maximus nec ut nisi.

Quisque sit amet porta libero. Aliquam quis ipsum pellentesque, semper nibh laoreet, ultricies tortor. Pellentesque aliquet sollicitudin lorem, vitae commodo nunc rutrum non. Sed laoreet egestas justo et posuere. Curabitur sit amet mattis leo. Curabitur porttitor turpis non purus congue, at vehicula risus eleifend. Integer egestas elementum urna, a malesuada dui hendrerit in. Nam auctor felis id scelerisque volutpat. Nulla in magna non purus fermentum gravida ac a massa. Nunc ante dui, pharetra in fringilla ac, suscipit sit amet augue.

Pellentesque ornare massa non massa luctus, id fringilla nisi venenatis. Mauris nec lacus malesuada, rutrum tortor tincidunt, pretium elit. Nullam eu ante lacinia, efficitur turpis vitae, fringilla augue. Fusce interdum quis lacus in commodo. Ut non metus pharetra, bibendum mi nec, condimentum nulla. Aliquam vel ornare lorem, et accumsan leo. Aenean pulvinar scelerisque quam at ultricies. Curabitur eros ligula, efficitur nec neque quis, facilisis rutrum sem. Phasellus a metus maximus, semper nunc non, cursus arcu. Nunc vehicula elit vel enim sagittis ultrices. Duis gravida in dolor vitae mollis. Pellentesque dictum eu lectus et egestas. Curabitur ullamcorper nibh eget ultrices mattis. Ut placerat, elit vel euismod aliquam, arcu risus iaculis dolor, vel placerat orci ante non magna. Mauris lacus lectus, pulvinar sit amet orci at, mattis imperdiet ante. Sed rutrum mauris et molestie hendrerit.

Sed imperdiet, neque vel vulputate dapibus, metus odio facilisis est, quis efficitur sem mauris at velit. Ut consequat finibus lectus, ut auctor leo. Sed nisi urna, scelerisque id auctor eget, semper eu tortor. Fusce blandit, metus id scelerisque ullamcorper, dolor nulla pellentesque eros, id interdum mi eros porta elit. Aliquam scelerisque diam metus, ac condimentum dui viverra vitae. Quisque in feugiat massa. Nunc sit amet pulvinar magna. Nunc volutpat risus augue, quis venenatis turpis ornare eget. Suspendisse malesuada egestas sapien sed dictum.

Read More
Blog Post 1

SRT++ Visual overlay for language packs

Read More
Blog Post 1

Nexus 5x: You're not the nexus I waited for

Read More