I have to implement a bi-directional sync between Business Central ERP and Shopify.
The sync has to be a scheduled one, not a real time one, and I have to optimise as much as I can to save API requests. The solution has to work on SaaS and I expect to have many requests to handle from both sides.
CREDITS 🙂
Just while I was getting depressed because I was finding no solutions, an online API course from Arend Jan Kauffmann showed as the icing on the cake and made me discover an unbelievable feature of API: the deltaLink and the deep inserts.
WHAT ARE ODATA DELTA LINKS AND WHY I USED THEM
The annoying thing about Business Central API is the current API usage limit.
When working in production environment the Http Error “429 Too Many Requests” was starting to be my nightmare. When dealing with synchronisation procedures this limit is a persistent problem.
Delta Links is a way to get deltas from a specific table. Basically when you get records you need to save a “delta link”, this delta link allows to retrieve only records that have been modified since previous requests.
First of all I created my custom API and I enable delta : ChangeTrackingAllowed = true;
This allows me to make a request with odata.track-changes http parameters.
All the magic is in the “@odata.deltaLink” URL.
Let’s supposed, after this initial request, I insert an item, update another one and delete the third. When calling again the provided odata.deltaLink I am going to get only the deltas as in picture:
Something really important is the delta links are “one time links”, it means that they allow one single requests and, after the request is served, they expire and you get a “410 Error Gone”. This means that sync mechanist must be implemented as “state of art” because if miss a request then the procedure has to reset from the initial request.
This is the way a deleted record is returned:
I just had to take to care about the way the insert action is managed in business central: it generates two events: an initial insert event and followed by an update event. This is by design.
A codeunit to upload files to different containers, implemented with interfaces and monitored with telemetry
SCENARIO
The requirement is to replace the reference of “File Management” codeunit with a new “Azure File Management” for the SaaS (Cloud) version of Business Central.
I want to implement a base “app” that manages file basic operation on different cloud platforms: upload, download, create folders, copy and move files and folders.
I want the app to be reusable by all other app depending on it.
I want to monitor the activity with telemetry and application insights.
THE WONDERFUL WORLD OF INTERFACES AND TELEMETRY
This project allows me to use in the real world two of the new powerful and wonderful features of Business Central: interfaces and telemetry. Interfaces are available with a minimum of runtime 5 (you have to kiss goodbye older versions of BC).
AZURE BLOBS VS AZURE FILES
One of the requirement was to work with Azure Files, not just with Azure Blobs. The web is full of samples of Azure Blobs integration but it lacks Azure Files implementation.
As you should already know, Azure Files and Azure Blobs are two different containers and you deal with them with a different set of APIs.
THE PROJECT
First of all I implemented the interface, in my GitHub sample I only give you the code of Download() 🙂
Then I implemented Download codeunits for Azure Files and Azure Blobs.
The “ALV AzBlob Service API” contains Azure Blob implementation, to deal with blobs is really easy and with a few lines of code the job is done. This is a dummy copy/paste from Demiliani sample integrated with telemetry monitoring.
The Azure File, implemented in “ALV AzFile Service API”, drove me mad because I didn’t find any ready-to-use solution on the internet and I had to implement all by myself.
The Azure Files management involves a tricky feature to calculate a Base64Hash SharedKey by using “Cryptographic Management” codeunit. The azure file API needs an authentication procedure which is not so easy to implement from scratch in AL language. By the way, here it is.
The “ALV Application Insights Mgt.” and “ALV Application Insights SDK” allow to deal with azure application insight telemetry. The Management codeunit implements the TraceInformation(), TraceError() and all the other function to write logs data.
I added just a function to manage telemetry where I manage the telemetry duration.
How to check if a lazy programmer has allowed malicious users (in the entire world) to install a spying Web Hook on your Business Central.
PART 1: HOW SOMEONE NASTY HIJACKS A WEB-KEY …..
PART 2: HOW A MALICIOUS USER USES IT IN A SNEAKY-SPYING WEB-HOOK
PART 1: HOW SOMEONE NASTY HIJACKS A WEBKEY …..
When you give access to Business Central to the external world (Azure Functions, Mobile Apps, Web Apps, Desktop apps) you usually give a WebKey to your external developer or to the Software House.
The WebKeys are used as credential and let third party applications to exchange data (such as customers, orders, invoices, warehouse stuffs) by using API, ODATAv4.
Let’s suppose the Business Central administrator uses a “normal” Business Central user and generates a web key on it. This usually happens because the customer refuses to pay for a license that will be used just to connect external apps.
The administrator gives this web-key to the developers. Most of programmers are very lazy (let’s say maybe almost all of them) and might just use this credentials this way:
Even worse, a very very lazy developer may insert these credentials inside application’s configuration files (values/Strings.xml or configuration JSONs in case of Windows apps).
In the nightmare scenario, the app end-user has to import a flat file or even configure by himself/herself the credentials that are given directly from the developer.
Now, when developing desktop or mobile applications, you end-up with deploying an EXE on Windows, an APK on Android, an IPA on IOS or an APP for Mac.
If the app is for the public it is then published in the markets. In case of enterprise app it is deployed to specific devices or computers usually with MDMs or direct deployment.
Let’s have a look what happens, for example, in the android app: the developer builds the app, deploys it in Google Play or directly gives the APK (maybe even compiled in debug).
It’s possibile and easy for everyone to get the original APK (as the EXE and the IPA).
Unless the developer has used obfuscation, disabled by default and commonly not enabled, with simple tools it’s possibile to easily browser the APK file for strings or configuration strings and extract all strings and resource strings, including the credentials.
apktool d YourApp.apk aapt d xmlstrings app.apk xmlfile strings yourfile.apk
Concluding, in a lot of different ways, these web-keys are in the hand of users.
You might think these web keys aren’t really dangerous .. BUT….
PART 2: HOW A MALICIOUS USER USES A WEB HEY IT TO SUBSCRIBE A WEB HOOK AND SILENTLY SPY DATA
Web Hooks are really powerful and they allow to implement amazing integrations.
Let’s have a look how easy is to implement a Spyware with web-hooks. 🙂
The credentials that have been stolen in part 1 cannot really be used with Business Central web client BUT, even worse, they can be used for ODATAv4 and API queries.
Since these credentials have the rights of the business central user, they might have unlimited and unrestricted access to all kind of tables.
Web Hooks might allow to configure an automated way to sneakily replicate customers, invoices, sales orders to an external repository.
To subscribe a web hook is quiete easy, by using Insomnia you just post a request to the Business Central by using the stolen credentials.
With this simple action every time an insert, update, delete operation happens the web hook automatically signal changes it to the subscriber. The subscriber has all the detail to get the entire record detail by using API.
HOW TO CHECK ACTIVE WEB HOOKS
The web-hook subscriptions are queryable using API exactly as all other Business Central tables.
So, just with a GET on subscriptions entity, you get the list of active subscriptions and check anytime if everything is as expected.
HOW TO AVOID THIS SCENARIO
The Web Key mustn’t be generated on a normal user but on a specific dedicated user;
The administrator has to configure access to specific data by configuring everything properly;
The web key has to be encrypted inside the third party applications;
You can create a web-hook on subscriptions entity that sends an email when something changes. You are alerted every time someone creates a new subscription 🙂
The actors of the comedy: Business Central Web hooks and APIs, Azure Functions, PayPal, Twilio and SendGrid SDKs.
SCENARIO
Consider this request: when back-office approves an order in Business Central the customer shall be automatically notified with an email and an SMS. The email should contain the PayPal link. Finally, when the customer pays, Business Central’s order have to be automatically updated.
Although there is an embedded Paypal service connector and there are dozen of PayPal extensions on AppSource you might want to develop your custom PayPal connector to handle your customised payment processes. It’s just so easy to integrate it by using Web Hooks 🙂
MY SOLUTION
When a Sales Order is being updated the subscribed Web Hook is invoked;
An Azure Function sends an email to the customer with a PayPal link to let the customer pay the order;
The Azure Function sends an SMS to alert the customer that the order is payable;
The customer proceed with payment using PayPal;
When the payment is completed or rejected the subscribed Web Hook is invoked and the result is notified to Business Central.;
THE RECIPE
The recipe is pretty easy and consists in a couple of ingredients:
A Business Central extension adding a few extra fields in the Sales Header table and page;
A Twilio account to send SMS;
A SendGrid account to send Email;
A PayPal Business Account;
An Azure Function that is subscribed as a web-hook both for BC and PayPal;
The Azure Function is in charge of the following tasks: asking Business Central API the order and customer detail, sending an SMS, creating on order on PayPal, sending an email with PayPal link to the customer.
SalesOrderPayPalWebHook.cs contains the azure function subscribed as Business Central WebHook. In my sample the WebHook is trigger every time a SalesOrder modification occurs.
Pay attention to these couple of lines of code, they are very important and this is the response the azure function must reply when you will subscribe your web hook in Business Central. Basically they response back the validation token to BC.
BusinessCentralConnector.cs contains class that manages calls to Business Central API. It connects to API and get the requested order and sales agent by applying filtering to API request.
PayPalConnector.cs contains class that manages calls to PayPal API. It posts to the Order API the json and gets back the order result.
MessageComposer.cs is the class that creates the response string used to send SMS and Email. The email contains the “PayPal Now” button.
MessageConnector.cs sends the email and the SMS to the customer. The SMS uses Twilio , the message uses SendGrid.
SalesOrders.cs and Customers.cs are the class representing Business Central SalesOrders and Customers tables. They are used for deserialising API data. I find useful https://app.quicktype.io/ to automatically convert Json returned from BC API to C# class. Just pay attention to DateTimeOffset field type that are not supported by JSON.
PayPalOrder.cs and PayPalResponseOrder.cs are the class representing PayPal Order and Order Response entities.
AzPayPal.cs produces the web page the user opens when clicking in the payment email. It’s just the standard paypal button that opens the paypal website.
PayPalPaymentWebHook.cs is the azure function that manages the PayPal webhooks. When the user completes the payment paypal calls this function passing the transaction data. This function updates business central order and send a confirmation email.
To have your solution working just download sources, configure your Business Central endpoint and deploy it to your azure subscription.
SUBSCRIBE A WEB HOOK IN BUSINESS CENTRAL
There is brief but well written article about web hooks here:
I am confident everybody already use PayPal for payments. First of all you need a PayPal Business Account, then you can create your credentials from PayPal developer site.
First of all you need to buy a Twilio phone number. Unfortunately some countries are not available when you buy a number, for example Italy is not. In my case I decided to buy a number from Belgium the subscription is 1€ per month and thanks to European Union roaming rules messages among EU countries are charged same price as national message.
Then you have to register all destination numbers in Twilio.
ABOUT SMS
An SMS nowadays might sound some kind of old-fashion manner to send a message. First thought you may have is: why don’t you use whatsapp, SMS is just such an old technology. It’s true but if you have have to deal with an international customer it’s not so obvious to use WhatsApp. In every region of the world you have different messaging platforms: WeChat in Asia, Line in Japan, KaKaoTalk in South Korea, Telegram is Middle East countries, Viber is former Russian countries. It’s a mess and I takes time to implement all of them. Instead SMS are working everywhere and they are just so easy to handle!
The Business Central extension consists mainly in the Control-Addin. This component allows to load an external website inside a page’s factbox.
There’re just a table extension extending the table Customer by adding latitude and logitude and a page extension extending the “Customer Card” and adding a factbox with the control add-in inside. When the factbox is shown the extension loads the control-add and every time the user changes the current record the extension updates the map with current customer location.
The control add-in manage the webview loaded inside the factbox and defines how business central and the webview interact with each other.
In StartupScript= is defined the javascript that is run when the control add in is loaded. In Script= is defined the javascript
The control addin basically defines the interface of the alv_mapscript.js javascript functions that are accessible from Business Central.
The javascript and business central may have a two way “chat”. You can define callbacks in the javascript function to invoke Business Central functions.
I noticed with some versions of Business Central that the ControlAddin javascripts aren’t load if the name of control add contains spaces. So I named it “ALVStreetViewerControlAddIn”, don’t even try to rename it to “ALV Street Viewer Control AddIn” or it will fail to load JavaScripts and you will completely run Business Central mad 🙂
To have the solution running don’t forget to open the “AVL Map Setup” table with its Page, the page contains the Azure Function url as published in your azure subscription.
In Business central we have a “Messages” table where there are messages to be dispatched to mobile devices;
In the Business Central we have a page where the user sends a push notification to the sales agents;
The sales agents receive the Push Notification containing the message in the Android App;
THE FLOW
RECIPE
A simple Xamarin Forms app;
An Azure Function;
An Azure Notification Hub account;
A Firebase Google Cloud Messaging or a Google Cloud Messaging account (*);
A Business Central Extension;
In this sample I am not using Azure Vault to store passwords and I just use settings.json, you should when in production environment 🙂
* Pay attention, GCM Google Cloud Messaging is not FCMFirebase Cloud Messaging. FCM and GCM are two different services to send push notifications available on the Google Messaging platform. The GCM is deprecated and the FCM is the new one. In this tutorial I cover both services since unfortunately you might have, as me, some apps already configured with GCM that cannot be migrated to FCM.
There’re two kind of notifications in FCM: Data messages and Notification messages.
If you are using the Data messages you will be in charge of handling the received message and present a notification to the user (if needed of course). But in this case you might miss notifications when your app is closed.
If you are using Notification Messages, FCM is handling the message for you and directly displays a notification if the app is in background/closed.
The solution I made for you in GitHub is already configured. If you start with your own project you have to do a lot of things like: add Xamarin.GooglePlayServices.Base, Xamarin.Firebase.Messaging Xamarin.Azure.NotificationHubs.Android nuget packages, add google-services.json to your project and set it as Build Action = “GoogleServiceJson”, change AndroidManifest.xml and properly add <uses-permissions> and <receiver> tags.
As in my solution, the packages list should appears as in picture, the google-service.json must be properly added and Android.Manifest.xml modified as shown below.
Te Xamarin Forms project is a simple single page app that just register and waits for a push notification.
The Xamarin Forms basically consists in three functions and all of then are in the AppDelegate.cs file.
Firstly the app checks the presence on device of Google Services and registers itself to the Firebase FCM Notification Service..
MyFirebaseMessagingService is the function that manage the push notification behaviour.
The Azure Hub Notification endpoints have to be configured in AzureNotificationConstants.cs
The Azure Function is pretty simple too. It wait for a GET request with the message as a query parameter. When it is called it dispatch the message by using NotificationHubClient.
BUSINESS CENTRAL EXTENSION
The Business Central extension sources are available from GitHub:
Insert your project Name and click “Create”. Click on alert button on top bar and select the project from the list. You are redirected in the project dashboard page.
In the project dashboard there is the “Project Number”. Note this number because you will need in next steps.
Find the API section inside dashboard page and click on “Go to APIs overview”.
You are now in API page, here click on “Enable APIs and Services”.
Search, find and select “Google Cloud Messaging” or “Cloud Messaging”.
Click “Enable”
Select “Create credentials” on the tool bar.
You are redirected in “Add credential to your project” page, here you click “API key” shortcut.
You are redirected in “API Key” page. Here select Create/Save. Here configure as in picture, by selecting “IP addresses” option with 0.0.0.0/0 address. For your security here you should insert the IP address your azure function is assigned to when created in Azure Portal.
A dialog appears with your own API Key. Save it somewhere because we will use it in Azure. This KEY is used on Azure Portal to authenticate on GCM services to send push notifications.
CONFIGURE GOOGLE FCM (THE NEW FIREBASE CLOUD MESSAGING)
When you have to implement an extension in a dozen of languages it is really painful to keep XLF files updated.
You probably noticed you waste a lot of time in editing these XLF file. That’s why I decided to implement an automated solution to save my time and keep me healthy.
THE RECIPE
The recipe to automate the translation of XLFF files in Business Central:
A Business Central repository with translations in various languages
A Visual Studio Code Extension for AL;
An Azure Function;
In this sample I am not using Azure Vault to store passwords and I just use settings.json, you should when in production environment 🙂
THE TRANSLATION REPOSITORY
I am using Business Central as a container itself for the translation table.
The business central extension source is available here:
The function accept two query parameters: the sentence to be translated and the the destination language and it returns back the translated sentence by querying Business Central API.
BusinessCentralConnector.cs is the class that manages calls to thr query API. It connects to API and get the requested translation by applying filtering to the API request.
Translations.cs is the class that represents Business Central Translations table for deserialising API data. I find useful https://app.quicktype.io/ to automatically convert Json returned from BC API to C# class. Just pay attention to DateTimeOffset field type that are not supported by JSON.
THE VISUAL STUDIO CODE EXTENSION
I used this Visual Studio Code Extension as a starting point for my customization:
This Vincent Nicolas extension is really awesome and automates the translation by using Azure Translator Text API. Unfortunately the translation for non latin languages is not that accurate and moreover some string have to be checked (eg: “Business Central” itself is translated in the local language and this causes funny results).
On Business Central, when a sales order is being updated the subscribed WebHook is invoked (WebHooks are a new feature in BC, they are really powerful and they deserve to be used);
The Web hook gets order details and send a SMS to the sales agent (eg: “The XYZ order has been released, do you want to send a confirmation email to your customer”?) and wait his/her reply;
The sales agent reply with a YES/NO and in case the Azure Function sends an SMS with order detail to the final customer;
THE RECIPE
The recipe is pretty easy and consists in three ingredients:
Buy a Twilio phone number and then configure twilio account.
Create a simple Azure function and deploy it.
Set-Up a BC WebHook on SalesOrder table (for newbies, later on I briefly describe the powerful technology of web hooks)
In this sample I am not using Azure Vault to store passwords and I just use settings.json, you should when in production environment 🙂
DOWNLOAD AND DEPLOYS AZURE FUNCTION
A simple Azure Function works as subscriber for Business Central WebHook.
The azure function consists basically in five .cs files.
SalesOrderWebHook.cs contains the azure function subscribed as Business Central WebHook. In my sample the WebHook is trigger every time a SalesOrder modification occurs. The function is in charge to manage the high level logic.
Pay attention to these couple of lines of code, they are very important and this is the response the azure function must reply when you will subscribe your web hook in Business Central. Basically they response back the validation token to BC.
BusinessCentralConnector.cs contains class that manages calls to API. It connects to API and get the requested order and sales agent by applying filtering to API request.
MessageComposer.cs is the class that create the response string used to reply to the request.
SalesOrders.cs and Employees.cs are the class representing Business Central SalesOrders and Empolyees tables. They are used for deserialising API data. I find useful https://app.quicktype.io/ to automatically convert Json returned from BC API to C# class. Just pay attention to DateTimeOffset field type that are not supported by JSON.
To have your solution working just download sources, configure your Business Central endpoint and deploy it to your azure subscription.
SUBSCRIBE A WEB HOOK IN BUSINESS CENTRAL
There is brief but well written article about web hooks here:
First of all you need to buy a Twilio phone number. Unfortunately some countries are not available when you buy a number, for example Italy is not. In my case I decided to buy a number from Belgium the subscription is 1€ per month and thanks to European Union roaming rules messages among EU countries are charged same price as national message.
Then you have to register all destination numbers in Twilio.
ABOUT SMS
An SMS nowadays might sound some kind of old-fashion manner to send a message. First thought you may have is: why don’t you use whatsapp, SMS is just such an old technology. It’s true but if you have have to deal with an international customer it’s not so obvious to use WhatsApp. In every region of the world you have different messaging platforms: WeChat in Asia, Line in Japan, KaKaoTalk in South Korea, Telegram is Middle East countries, Viber is former Russian countries. It’s a mess and I takes time to implement all of them. Instead SMS are working everywhere and they are just so easy to handle!
A sales man send a request with a Whatsapp message, eg: “What is XYZ order status?“.
The request is processed by an Azure function that queries Business Central API and gets the response.
The answer from Business Central is dispatched by the Azure function to the sales man via whatsapp chat, eg:
THE RECIPE
The recipe to integrate Whatsapp and Business Central is pretty easy and consists in two ingredients:
Configure a Twilio account to work with Whatsapp.
Create a simple Azure function.
In this sample I am not using Azure Vault to store passwords and I just use settings.json, you should when in production environment 🙂
CONFIGURE TWILIO
Configuring Twilio is the hardest part. I don’t give here detailed instructions about registration. After you get your account confirmed and sandbox enabled just double check your Azure Function endpoint in correctly configured in “When a message comes in”. https://www.twilio.com/console/sms/whatsapp/sandbox .
DOWNLOAD AND DEPLOYS AZURE APP
A simple Azure Function works as “middle layer” between Twilio API and Business Central API.
The azure function consists basically in five .cs files.
TwilioTrigger.cs contains the azure function invoked by Twilio every time it gets an incoming whatsapp message. The function parses the incoming whatsapp message, query Business Central API for the requested order number and returns the message to be replied to the sales agent.
The function should be in charge to manage security related stuffs too. In my example, of course, no security is managed at all 🙂
BusinessCentralConnector.cs contains class that manages calls to API. It connects to API and get the requested order by applying filtering to API request.
MessageComposer.cs is the class that create the response string used to reply to the request.
SalesOrders.cs is the class that represents Business Central SalesOrders table for deserialising API data. I find quiet useful https://app.quicktype.io/ to automatically convert Json returned from BC API to C# class. Just pay attention to DateTimeOffset field type that are not supported by JSON.
To have your solution working just download sources, configure your Business Central endpoint and deploy it to your azure subscription.
THE GRAND FINALE
When I write a whatsapp message with the specified template “What is XXX order status” I get the reply from BC.
WHAT YOU GET NEXT
In the future I plan to add some kind of artificial intelligence by using Azure Cognitive API to process requests.
And absolutely I am going to add a little bit of security by allowing the sales agent to have information just about his own orders.
The Xamarin Forms project is a simple single page app that just register and waits for a push notification.
The Xamarin Forms basically consists in three functions and all of then are in the AppDelegate.cs file.
Firstly the app registers itself to the Apple Notification Service managing the different ways to implement it in various IOS version (right now I use it with IOS 14.X, this part will change with next version of IOS so just have a look at github source for updates) .
Secondly the app registers itself to Microsoft Notification Hub.
ProcessNotification is the function that manage the push notification behaviour.
The Azure Hub Notification endpoints have to be configured in AzureNotificationConstants.cs
AZURE FUNCTION
The Xamarin Forms sources are available from GitHub:
The Azure Function is pretty simple too. It wait for a GET request with the message as a query parameter. When it is called it dispatch the message by using NotificationHubClient.
BUSINESS CENTRAL EXTENSION
The Business Central extension sources are available from GitHub:
The extension consists in very simple table with a page card and a page list that allow CRUD for records.
The codeunit “Cod70359948.ALV Send Push Notification.al” contains the SendPush function that calls the Azure Function to send the messages.
CONFIGURE APPLE PUSH CERTIFICATE
This tutorial does not cover the configuration and setup-up of Apple Push Certificates, it is a very long and exhausting procedure far away from being summarised in a short tutorial. So I give it for granted that you are familiar with this part.
Supposing you have already your app development ad production certificates and profiles, then this tutorial cover how to use them in Microsoft Notification Hub.
There add a new Key, digit the key name and enable the “Apple Push Notification service” checkbok.
Then download the P8 file, the file name will be {your_key_id}.p8 (in my case it is AuthKey_TAYAGJ3W48.p8). SAVE THE P8 file. You should never delete it.
Create the resource group, insert your namespace and hub name.
Wait Azure completes the creation of the resource, then edit the notification hub just created and go to “Settings” -> “Apple APNS” section. There just select the “Token” Authentication Mode, then you have to copy the key_id, the bundle_id and the key_id as they were noted in the Apple section.
Now open, with a text editor, the P8 file you have previously downloaded. Copy the key without the “—–BEGIN PRIVATE KEY—–” and “—–END PRIVATE KEY—–” placeholders and paste it in the “Token” field.
Save , then go to “Manage” -> “Access Policies” section and copy the Connection string with Listen Permission.
CONFIGURE XAMARIN FORM AND AZURE FUNCTION
In your Xamarin Forms solution, open AzureNotificationConstants.cs and copy the ListenConnectionString and the hub name.
Open Azure Function in Visual Studio Code, edit the “local.settings.json” file and copy the connection string and HubName here too.
AND WHAT’S NEXT?
This solution send broadcast messages, I will publish soon a complete solution where it is possibile to send a message to a specific user.
Yet another www.business-central.blog 