In this blog post I will give a short overview about my business card crm. It is a proof of concept for using almost serverless technology to solve an old problem.

What is the application good for?

This application can be used to make a photo of a business card. The image will be analyzed by the Google Vision API and afterwards the text will be feed into the Google Named Entity Recognition Engine. Based on some look up tables and regular expressions typically attributes like first name, last name, phone number and email address are extracted. The data will be saved in the local storage of the device. If multiple clients like a smartphone and a desktop PC want to share business card data they can join together through a WebRTC session and data can be send through this connection.

Video tour

Architecture

Source: Manuel Blechschmidt, created with Signavio Process Editor

Example image

The image above shows a typical business card like you can collect on a lot of different occasions. Typically these are entered manually into a CRM system. More advanced systems do employ ocr recognition technology to automate parts of this error prone process.

Google offers an API for analyzing such images and get the text. This API can directly be consumed with JavaScript after creating an access key.

The code looks in OpenUI5 like this:

 

var _gApiLoaded = new Promise(function(resolve) { jQuery.sap.includeScript("https://apis.google.com/js/api.js", "gapi", function() { gapi.load("client", function() { gapi.client.setApiKey("<Insert your key here>"); resolve(); }); }); }); // https://www.googleapis.com/auth/cloud-platform _gApiLoaded.then(function() { // https://developers.google.com/discovery/v1/reference/apis/list Just hit execute to get a list gapi.client.load("https://vision.googleapis.com/$discovery/rest", "v1", function() { var reader = new FileReader(); reader.readAsDataURL(oBlob); reader.onloadend = function() { var base64data = reader.result; gapi.client.vision.images.annotate({ "requests": [{ "features": [{ "type": "TEXT_DETECTION" }], "image": { "content": base64data.replace(/^data:image/jpeg;base64,/, "") } }] }).then(function(oResponse) { console.log({ "ImageBase64": base64data, "OCRResult": oResponse.result }); }); }; }); });

For the image above the result looks like this:

{ "responses": [ { "textAnnotations": [ { "locale": "en", "description": "Incentergy
GmbH
Manuel Blechschmidt
CEO
www.incentergy.de
Phone: +49 173 632 26 21
Mail: manuel.blechschmidt@incentergy.de
", "boundingPoly": { "vertices": [ { "x": 42, "y": 151 }, { "x": 901, "y": 151 }, { "x": 901, "y": 661 }, { "x": 42, "y": 661 } ] } }, { "description": "Incentergy", ... 

Afterwards the whole text is send to google NER engine.

{ "entities": [ { "name": "CEO", "type": "PERSON", "metadata": {}, "salience": 0.3584892, "mentions": [ { "text": { "content": "CEO", "beginOffset": 36 }, "type": "COMMON" } ] }, { "name": "Phone", "type": "CONSUMER_GOOD", "metadata": {}, "salience": 0.2520285, "mentions": [ { "text": { "content": "Phone", "beginOffset": 58 }, "type": "COMMON" } ] }, { "name": "Incentergy GmbH Manuel Blechschmidt", "type": "PERSON", "metadata": {}, "salience": 0.22339708, "mentions": [ { "text": { "content": "Incentergy
GmbH
Manuel Blechschmidt", "beginOffset": 0 }, "type": "PROPER" } ] }, ...

 

After letting Google do the hard lifting of OCR and NER the application find with some basic rules the correct strings.

Screenshots

SAP Web IDE

The application was completely develop in SAP Web IDE and published to github.

WebRTC

For synchronizing data between different devices the App can create a RTC session. For the details please consult the source code and the video shown above.

Summary

This applications shows what is possible with the newest HTML5 technology and machine learning including OpenUI5. It uses WebRTC, OpenUI5, LocalStorage, Google Vision API.