Integrate SharePoint Online with Dynamics AX Legacy Application

SharePoint Online is a wide spread solution for collaboration. Actual versions of Dynamics AX / 365 provide great integration capabilities and for all other purpose there exists the SharePoint Online Client API. However, legacy applications may not support the required libraries or .NET framework versions to use the SharePoint Online Client API. One way to overcome this issue is to utilize the REST API and communicate via HTTP. Here you can find a step by step guide how to provide an external application access to SharePoint Online via an App: http://www.ktskumar.com/2017/01/access-sharepoint-online-using-postman/

Register an App in SharePoint Online

The first thing you need to do is register an App in SharePoint Online. Open the registration site in a browser: https://YOUR_SITE.sharePoint.com/_layouts/15/appregnew.aspx . In my case I’m using a sub site, therefore my URL would look like this: https://YOUR_SITE.sharepoint.com/sites/development/_layouts/15/appregnew.aspx .Use the form to generate a client ID and a secret code. Provide any title you like. Set the domain to localhost and redirection URL to https://localhost

image

Set Permissions for the App

Open the following URL in a Browser: https://YOUR_SITE.sharepoint.com/_layouts/15/AppInv.aspx . Use the Client ID to lookup the created App. In the free text form add the required permissions e.g. access a list. You can find the syntax here: https://www.sumitagrawal.io/sharepoint-add-in-permission-xml-cheatsheet/ If you need access to more than one list, you have to repeat this step. In my case I only need access to one list. The XML Permission Code looks like this:

<AppPermissionRequests AllowAppOnlyPolicy="true">
<AppPermissionRequest
Scope="
http://sharepoint/content/sitecollection/web/list"
Right="Read" />
</AppPermissionRequests>

Click the Create button, this will forward you to a form where you can choose which list can be accessed by the app.

image

SharePoint Online Tenant ID

To find the Tenant ID open the following URL in your Browser https://YOUR_SITE.sharepoint.com/_layouts/15/appprincipals.aspx  Since I’m using a sub site my URL looks like this https://YOUR_SITE.sharepoint.com/sites/development/_layouts/15/appprincipals.aspx . There you can find the Tenant ID

image

Create a Token

To interact with SharePoint Online you need to generate a token.This is done by sending an HTTP POST requiest with client ID, secret, and desired resource to SharePoint Online. Here is the Code in X++ written in Ax 2009.

str clientID = „<YOUR_CLIENT_APP_ID>“;

str secret = „<YOUR_SECRET>“;

str tenant = „<YOUR_TENANT>“;

str ctx = „00000003-0000-0ff1-ce00-000000000000“; //i.e. SharePoint

str sp = „<YOUR_SITE>.sharepoint.com“;

str listName = „<NAME_OF_YOUR_LIST>“; // e.g. Customers

str listUrl = “https://YOUR_SITE.sharepoint.com/_api/web/lists/GetByTitle(‚“+listName+“‚)/items; “

str url = „https://accounts.accesscontrol.windows.net/“

+tenant+“/tokens/OAuth/2″;

System.Net.WebClient cl = new System.Net.WebClient();

System.Net.WebHeaderCollection headers = new System.Net.WebHeaderCollection();

System.Collections.Specialized.NameValueCollection body = new System.Collections.Specialized.NameValueCollection();

System.Byte[] response;

System.Text.Encoding enc = System.Text.Encoding::get_UTF8();

System.String responseText;

str xppResponse;

int indexOf;

int start;

int totalLength;

int length;

str token;

;

headers.Add(„Content-Type“,“application/x-www-form-urlencoded“);

body.Add(„grant_type“,“client_credentials“);

body.Add(„client_id“,clientID+“@“+tenant);

body.Add(„client_secret“,secret);

body.Add(„resource“,ctx+“/“+sp+“@“+tenant);

cl.set_Headers(headers);

response = cl.UploadValues(url,“POST“,body);

responseText = enc.GetString(response);

// cut token out of response text

indexOf = responseText.IndexOf(„access_token“);

start = indexOf + 15;

totalLength = responseText.get_Length();

length = totalLength – start – 2;

xppResponse = responseText;

token = responseText.Substring(start,length);

 

Query List Elements

The token has a valid timestamp measured in Unix Ticks. So it might be good idea to store the token in a parameter table and only request a new one if it is or will expire soon. However, in this example I go on and query the Customer List.

// .. Code from above

cl = new System.Net.WebClient();

headers = new System.Net.WebHeaderCollection();

headers.Add(„Accept“,“application/atom+xml“);

headers.Add(„Authorization“,“Bearer „+token);

cl.set_Headers(headers);

body.Clear();

url = listUrl;

responseText = cl.DownloadString(listUrl);

xppResponse = responseText;

info(xppResponse);

 

In this case the response type is an XML. You may parse the XML and only view e.g. the Title Tag or any other fields your are interested in. Here is the Infolog of the complete XML response from the SharePoint Online List.

image

Advertisements

Walkthrough: Configure XML/File Data Export via AIF in AX 2012

This is an example how to export the LedgerJournalTable + LedgerJournalTrans records as XML via AIF

Create a Query

  1. Create a new query and name it AxdLedgerJournal.
  2. Place the LedgerJournalTable as main Datasource
  3. Set the Fields Property Dynamic to Yes
  4. Add the LedgerJournalTrans as child data source
  5. Set the Fields property Dynamic of the LedgerJournalTrans data source to Yes
  6. Set the Relation property of the LedgerJournalTrans data source to Yes
  7. Set the Fetch Mode property of the LedgerJournalTrans data source to 1:n
  8. Set the Join Mode property of the LedgerJournalTrans data source to Outer Join

Document Query

 

Use the AIF Document Wizard to generate the service classes

  1. In the menu bar go to Tools > Application Integration Framework > Create document service
  2. Provide the AxdLedgerJournal query as source query for the document service
  3. Select  the options to generate the Read and Find methods
  4. Select the option to generate the AxBC classes
  5. Let the wizard generate the software artifacts for you

AIF Wizard

Fine tune the generated code

  1. Go to the private projects and open the newly generated project
  2. Compile the project
  3. If the cache() methods don’t compile delete them
  4. Review the open tasks and provide meaningful labels
    Fine tune AIF Code

Configure the Batch Jobs

  1. In AX go to System Administration > Inquires > Batch Jobs > Batch Jobs
  2. Create a new Batch Job and go to “View Task”
  3. Add four task and choose the following classes
  4. AifGatewayReceiveService
  5. AifGatewaySendService
  6. AifInboundProcessingService
    AifOutboundProcessingService
  7. Close the Task window and set the Recurrence of the Batch Job to run e.g. all 5 minutes

AIF Batch Jobs

 

Configure the File System Adapter

  1. In AX go to System Administration > Setup > Services and Application Integration Famework > Outbound Ports
  2. Create a new Outbound Port
  3. Select the FileSystemAdapter as Adapter
  4. Provide a target location where to place the files e.g. a share with write permissions for the AOS account
  5. Open the Service Operation Dialog and add the LedgerJournalService.read method
  6. Activate the Port

AIF Outbound Port with File System Adapter

Add a send method

On the LedgerJournalTable add the following method

server void sendElectronically(XMLDocPurpose _xmlDocPurpose,
                               AifSendMode _aifSendMode = AifSendMode::Async)
{
    AxdSendContext      axdSendContext = AxdSendContext::construct();
    AifEntityKey        aifEntityKey   = AifEntityKey::construct();
    Map                 keyData;
    AifConstraint       aifConstraint = new AifConstraint() ;
    AifConstraintList   aifConstraintList = new AifConstraintList();
    ;
    keyData = SysDictTable::getKeyData(this);

    aifEntityKey.parmTableId(this.TableId);
    aifEntityKey.parmRecId(this.RecId);
    aifEntityKey.parmKeyDataMap(keyData);

    axdSendContext.parmXMLDocPurpose(_xmlDocPurpose);
    axdSendContext.parmSecurity(false);

    aifConstraint.parmType(AifConstraintType::NoConstraint);
    aifConstraint.parmId(this.JournalNum);
    aifConstraintList.addConstraint(aifConstraint);

    AifSendService::submitDefault(
        classNum(LedgerJournalService),
        aifEntityKey,
        aifConstraintList,
        _aifSendMode,
        axdSendContext.pack());   
}

Test the export

Create a job and use the following code to test the export

static void JobTestAifExport(Args _args)
{
    LedgerJournalTable journal;
    ;
    journal = ledgerJournalTable::find("00001");
    journal.sendElectronically(XMLDocPurpose::Original);
}

In AX got to System Administration > Periodic > Services and Application Integration > Queue Manager. You will find a record for the LedgerJournal export. The Batch Job will first create an XML Document and in a second step perform the export to the file system.

AIF Queue Manager

Setup und Configure Office 365 Project Web App

Here you can find my video on how to setup Project Web App, link with Project Desktop client and how to setup a SharePoint subwebsite with a synchronized task list

pwa

Calculate and Visualize the Lorenz Curve in Power BI

The Lorenz curve (1) is tool in statistics to visualize the concentration of values within a dataset. A typical well known example is the concentration of wealth within the population. For example in 2017 the top 1% of the US population controls 36,8% of wealth (2). However, the Lorenz curve can also be utilized to visualize the product groups share on revenue.

Demo Data

There are 5 product groups (A,B,C,D,E,F) and multiple sales. The data is stored in the table “Sales” including three columns ProductGroup, Product and Qty. For demonstration purpose, each product group has the same share on total sold products: 20%

ProductGroup

Product

Qty

A A123 2
A A234 2
B B123 4
C C123 1
C C123 1
C C123 1
C C234 1
D D123 4
E E123 2
E E234 2

The invoices are stored in a separate table called Invoice, it contains three columns: The ProductGroup, SalesId and InvoiceAmount.

ProductGroup

SalesId

InvoiceAmount

A S-0001 5000
A S-0002 6000
E S-0003 1000
E S-0004 10000
D S-0005 11000
B S-0006 4000
B S-0007 4000
B S-0008 5000
C S-0009 5000
C S-0010 5000
C S-0011 4000

In order to visualize the lorenz curve, a data set is required that holds the cumulated frequency of Qty and InvoiceAmount, in ascending order based on the InvoiceAmount. . The first simple SQL Statements and results look like this

select ProductGroup, sum(Qty) as QtySum 
from Sales
group by ProductGroup

ProductGroup

QtySum

A 4
B 4
C 4
D 4
E 4

select ProductGroup, sum(InvoiceAmount) as InvoiceAmountSum
from Invoice
group by ProductGroup
order by InvoiceAmountSum asc

ProductGroup

InvoiceAmountSum

A 11000
D 11000
E 11000
B 13000
C 14000

Now comes the tricky part. A SQL statement is required which creates the cumulated sums. To do this the SQL function row_number is used to create a line number in the result set. Then the same result set is joined with all records smaller or equal the current row number and the InvoiceAmount is summed up. The SQL Statement looks like this.

select temp1.idx, temp1.ProductGroup, temp1.InvoiceAmountSum,
sum(temp2.InvoiceAmountSum) as CumulateSum
from
(

select ROW_NUMBER() over (order by InvoiceAmountSum asc) as idx, ProductGroup, InvoiceAmountSum
from (
    select ProductGroup, sum(InvoiceAmount) as InvoiceAmountSum
    from Invoice
    Group by ProductGroup)
temptable

)
as temp1
inner join
(
    select ROW_NUMBER() over (order by InvoiceAmountSum asc) as idx, ProductGroup, InvoiceAmountSUm
    from (
        select ProductGroup, sum(InvoiceAmount) as InvoiceAmountSum
        from Invoice
        Group by ProductGroup)
    temptable
)
as temp2 on temp1.idx >= temp2.idx
group by temp1.idx, temp1.ProductGroup, temp1.InvoiceAmountSum

The result looks like this

Idx

ProductGroup

InvoiceAmountSum

CumulatedSum

1 A 11000 11000
2 D 11000 22000
3 E 11000 33000
4 B 13000 46000
5 C 14000 60000

Finally  the SQL query needs to bee extended to include also the cumulated sales qty. Moreover, a line starting with 0 values is required to make the Lorenz cure look correct. This is done by adding a row using UNIOIN. Here is the complete SQL statement and the result set:

select temp1.idx, temp1.ProductGroup, temp1.InvoiceAmountSum,
sum(temp2.InvoiceAmountSum) as CumulatedSum, s.SalesQty,
sum(s.SalesQty) as CumulatedSales
from
(

select ROW_NUMBER() over (order by InvoiceAmountSum asc) as idx, ProductGroup, InvoiceAmountSum
from (
    select ProductGroup, sum(InvoiceAmount) as InvoiceAmountSum
    from Invoice
    Group by ProductGroup
    )
temptable

)
as temp1
inner join
(
    select ROW_NUMBER() over (order by InvoiceAmountSum asc) as idx,   
    ProductGroup, InvoiceAmountSUm
    from (
        select ProductGroup, sum(InvoiceAmount) as InvoiceAmountSum
        from Invoice
        group by ProductGroup
    )
    temptable
)
as temp2 on temp1.idx >= temp2.idx

inner join
(
    Select ProductGroup, sum(Qty) as SalesQty
    from Sales as s 
    group by ProductGroup
)
as s on s.ProductGroup = temp1.ProductGroup

group by temp1.idx, temp1.ProductGroup, temp1.InvoiceAmountSum, s.SalesQty

union

select 0 as idx,“ as ProductGroup, 0 as InvoiceAmountSum,0 as CumulatedSum,
0 as SalesQty,0 as CumulatedSales

Idx

ProuctGroup

InvoiceAmountSum

CumulatedSum

SalesQty

CumulatedQty

0   0 0 0 0
1 A 11000 11000 4 4
2 D 11000 22000 4 8
3 E 11000 33000 4 12
4 B 13000 46000 4 16
5 C 14000 60000 4 20

 

Power BI

The next step is load the data into Power BI and calculate the relative cumulated frequency  for Sales and Invoice. The is done by diving the cumulated frequency by the total numbers.

  1. Open a new Power BI report
  2. Select Get Data from SQL Server and provide the connection information
    In the details copy & paste the SQL Statement from above
  3. Rename the data set to SalesInvoice

After this there should be a dataset with the values from the table above in Power BI. Add two additional columns.to calculate the relative cumulated frequency. Format the columns as percentage.

RelativeFreqInvoice = SalesInvoice[CumulatedSum] / 
                      sum(SalesInvoice[InvoiceAmountSum])

and

RelativeFreqSales = SalesInvoice[CumulatedSales] / sum
                    (SalesInvoice[SalesQty])

The dataset in Power BI should look like this

Idx

Prouct
Group

InvoiceAmount
Sum

Cumulated
Sum

Sales
Qty

Cumulated Qty

Relative
FreqInvoice

Relative FreqSales

0   0 0 0 0 0% 0%
1 A 11000 11000 4 4 18% 20%
2 D 11000 22000 4 8 37% 40%
3 E 11000 33000 4 12 55% 60%
4 B 13000 46000 4 16 77% 80%
5 C 14000 60000 4 20 100% 100%

Go to the report designer and add a new line chart. Set as Axis the RelativeFreqSales. To get the diagonal line for the Lorenz curve add the RelativeFreqSales again as Value. Then also add the RelativeFreqInvoice as Value in the diagram properties. Now there are two lines, the 45° line and the lorenz curve underneath. In this example it should look like this:

Lorenz Curve in Power BI

1) https://en.wikipedia.org/wiki/Lorenz_curve 
2) http://money.cnn.com/2017/09/27/news/economy/inequality-record-top-1-percent-wealth/index.html

Last Day Of Month and Year as View Computed Column in AX 2012

View Computed Columns in Dynamics Ax 2012 support the use of TSQL functions to calculate additional view columns. Here is  an example how to calculate the Last Day of Month and Last Day of Year based on the TransDate at the CustTransView. First, Extend the SysComputedColumnDatePart  Base Enum and add two additional values Month and Year. Extend the getDateDiffFormatSelector() method at the SysComputedColumn class.

private static server str
getDateDiffFormatSelector(SysComputedColumnDatePart _dateDiffFormat)
{
    switch(_dateDiffFormat)
    {
        case SysComputedColumnDatePart::Day :
            return ‚d‘;
        case SysComputedColumnDatePart::Second :
            return ’s‘;
        case SysComputedColumnDatePart::Month:
            return ‚m‘;
        case SysComputedColumnDatePart::Year :
            return ‚yy‘;
    }

    return ‚d‘;
}

 

Last Day Of Month

Add a new static method to the view and call it lastDayOfMonth.

private static server str lastDayOfMonth()
{
    DictView    ctView = new DictView(tableNum(‚CustTransView‘));
    str sTransDate = ctView.computedColumnString(‚CustTrans‘,’TransDate‘);
    str sLastDay = "";

    sLastDay = SysComputedColumn::cast(
    SysComputedColumn::getDateAdd(‚-1‘,
    SysComputedColumn::getDateAdd(
    SysComputedColumn::getDateDiff(
    ‚0‘,sTransDate,SysComputedColumnDatePart::Month)+ ‚+1′,’0‘,
    SysComputedColumnDatePart::Month),
    SysComputedColumnDatePart::Day),
    ‚DateTime‘);

    return sLastDay;
}

Add a new DateTime column to the view and set the lastDayofMonth() method as ViewMethod

Last Day of Year

Add a new static method to the view and call it lastDayOfYear

private static server str lastDayOfYear()
{  
    DictView    ctView = new DictView(tableNum(‚CustTransView‘));
    str sTransDate = ctView.computedColumnString(‚CustTrans‘,’TransDate‘);
    str sLastDay = "";

    sLastDay = SysComputedColumn::cast(
    SysComputedColumn::getDateAdd(
    SysComputedColumn::getDateDiff
    (‚0‘,sTransDate,SysComputedColumnDatePart::Year) + ‚+1‘,
    ‚-1‘,
    SysComputedColumnDatePart::Year),
    ‚DateTime‘);

    return sLastDay;
}

Add a new DateTime column to the view and set the lastDayOfYear() method as ViewMethod

The View

Dynamics AX 2012 translates both fields in TSQL Code. The view looks like this

SELECT    
ACCOUNTNUM, VOUCHER, TRANSDATE, DATAAREAID, PARTITION, 1010 AS RECID,
CAST(CAST(DATEADD(yy, DATEDIFF(yy, 0, TRANSDATE) + 1, – 1) AS DateTime)
AS DATETIME) AS LASTDAYOFYEAR,
CAST(CAST(DATEADD(d, – 1, DATEADD(m, DATEDIFF(m, 0, TRANSDATE) + 1, 0)) AS DateTime) AS DATETIME) AS LASTDAYOFMONTH

FROM        
dbo.CUSTTRANS AS T1
GROUP BY ACCOUNTNUM, VOUCHER, TRANSDATE, DATAAREAID, PARTITION

The result in Dynamics AX looks like this

CustTransView with Computed Column

Visualize the firms contribution to total revenue in Power BI

We recently discussed at work how to display the contribution of multiple firms to the total revenue in Power BI, in a way that the firms can be filtered but the total revenue stays constant. We came up with a very simple no-code solution.

Data from Dynamics AX 2012 R3

All the required data can be fetched from the CustInvoiceJour table in Dynamics AX. There are three datasets in the Power BI data model. First, the invoices including the InvoiceAmountMST, InvoiceYear and DataAreaId. Second the sum of InvoiceAmountMST grouped by Year and DataAreaId. Finally, a table that contains the years. Here are the SQL statements to load the data.

select
InvoiceId, InvoiceAmountMST, InvoiceDate, DataAreaId,
year(InvoiceDate) as InvoiceYear
from CustInvoiceJour

Total Revenue:

select
sum(InvoiceAmountMST) as InvoiceAmountTotal,
year(InvoiceDate) as InvoiceYear, DATAAREAID
from CustInvoiceJour
group by year(InvoiceDate),DATAAREAID
order by DATAAREAID

In Power BI create a new dataset and enter the years manually, like 2010, 2011, 2012, etc.

Power BI Years Table

In Power BI go to the data model an connect the Invoices and Total Revenue with the dataset that contains the years

image

Line and stacked Column

Add a new Line and stacked Column chart to the Power BI Report. Set the field Year from the Year table as shared axis. From the CustInvoiceJour set the DataAreaId as Colum Series and the InvoiceAmountMST as Column  values. Set the InvoiceAmountTotal from the Totals table as line values. In the format tab disable the Y-Axis property Show Secondary.

Diagram settings 

Your diagram should look like this. Filtering the companies does not effect the total revenue line.

image

Inplace Upgrade of Windows Server 2012 HyperV to 2016

There is no upgrade path for Windows Server 2012 (non-R2) HyperV role to Windows Server 2016 HyperV. The easiest way would be to setup a new Windows Server 2016 and activate HyperV. However, there may be reasons to perform an inplace upgrade. This is an experience report doing so

Prepare for the Upgrade

First make sure there are no virtual machine related files left on C: drive. Move the configuration files and especially the virtual hard disk files to another drive. I recommend to make list where the VHDX files are place. This can easily be done via PowerShell

Get-VM –VMName * | Select-Object VMid | Get-VHD | select Path | ft

Next uninstall the HyperV role using the Server Manager. Simply select remove roles and features and select the HyperV role. This step requires a reboot and your server will come up a simple plain windows 2012 server.

image

After HyperV was uninstalled make sure to remove NIC teaming in case you are using this feature. I recommend to deactivate all but one network adapter. This can also be done using the Server Manager.

Perform Inplace Upgrade

Insert a disk or mount the Windows Server 2016 image. Start the upgrade process. When asked choose to keep all your data and apps. This will preserve your data and applications e.g. the RAID Manager software. The setup wizard warns you that an inplace upgrade is not the preferred way to setup server 2016. Accept and proceed. The upgrade will take a while and requires some reboots.

image

Setup HyperV on Server 2016

After the upgrade process has finished, its time to setup HyperV again. First configure the NIC teaming again. Activating the HyperV role in Server 2016 is almost the same as in Server 2012. This can be done by using the Server Manager installing new roles and features. Activating HyperV will require a reboot. After the reboot configure a virtual switch so your VMs can access the network again.

Import the Virtual Machines again

By default HyperV on Server 2012 has no clue about the VMs in former Server 2012. The virtual machines have to be imported manually. This can be done using the HyperV console. First, provide the folder where the virtual machine configuration is placed. Afterwards choose to directly register the VM.

image

Next provide the folder where the virtual hard disk files are placed. Don’t get confused because you don’t see the actual VHDX files in the selection dialog. The import wizard will check if the virtual hard disk named in the configuration file can be found in this folder. If the wizard can’t find the virtual hard disk file, take a look a the list of vhdx file paths generated by the PowerShell script.

image

When all the VMs are imported the upgrade is almost finished. The the network connection of the virtual machines and make sure they can access the network via the newly created virtual switch. Removing the virtual network adapter and adding the virtual network adapter again can help Zwinkerndes Smiley  Perform some tests, maybe reboot the server again and install the latest updates.