Semi additive measures in DAX and Closingbalancemonth

It’s been a while since we have visited Data Analysis Expressions (DAX) on this blog, but now we’re going to jump right in and discuss working with semi-additive measures. Semi-additive measures can’t be added across all dimensions, typically they can’t be added across the date/time dimension. Common examples of semi-additive measures are account balances and inventory levels.

  • Inventory levels can be added up across products, across different stores, but not across time. If you have 500 silver widgets at the end of day on Monday and you have 500 silver widgets at the end of day on Tuesday, how many widgets do you have? You only have 500 of course! We have to take this into consideration when building our model and measures.
  • The same is true of account balances as well. If I have $100 in my account on February 1st and I have $85 in my account on March 1st, what is my account balance? It’s only $85, it’s not the sum of both months.

The Scenario

In this scenario, I am looking at the stock price of Microsoft over time. We want to determine things like Closing and Opening price among others. For this example we are going to try and calculate the closing price for the month using the function CLOSINGBALANCEMONTH. This will present an interesting problem that we will discuss shortly. First, my data model has a simple measure which returns a SUM of the closing stock price as seen below:

image

Remember, this measure is valid for all the dimensions in my data model except for my date dimension. Similar to inventory levels and account balances we don’t want to add the closing stock price across time, this produces incorrect results. Back in 2012 the stock price of MSFT stock was around $30 a share, however, when I display our measure in a table with the year and month what we actually see are numbers that are much higher. This is because the measure is adding up the stock price for all days of the month and this is incorrect!

image

CLOSINGBALANCEMONTH

Definition: Evaluates the expression of the last date of the month in the current context.

Syntax: CLOSINGBALANCEMONTH( <expression>, <dates>)

CLOSINGBALANCEMONTH is one of the built in time intelligence functions and it works great, most of the time. Where it falls short is when you have blanks or gaps in your data. I am going to create a new measure using CLOSINGBALANCEMONTH using the following expression:

image

Now let’s take a look at the results:

image

Most months we are getting the correct value, but some months are blank, why? CLOSINGBALANCEMONTH returns the closing price of the stock for the last day of the month, unfortunately we are looking at stock market data and the stock market is not open every day of the month. So therefore, if the last day of the month has no closing stock price, then blank is returned. See screenshot below. This is typically not what we want when looking at semi-additive measures! We want to return the closing balance for the last day of the month that had a value.

LASTNONBLANK IN DAX to handle blanks!

We need to write a measure in DAX that is going to determine the last date of the month where the stock market was open. We are going to solve this problem by using the function LASTNONBLANK. This is an extremely useful and helpful function.

Definition: Returns the last value in the column, filtered by the current context, where the expression is not blank.

Syntax: LASTNONBLANK(<column>, <expression>)

I am going to build this out incrementally for demonstration and validation purposes. First, we are going to create a new measure just to see what this function returns:

image

Next, I will add this new measure to our table for validation. Everything looks perfect! The measure is not blindly returning the last day of the month, it’s returning the last day of the month that had a closing price for the stock, meaning the value returned for June, September, and March is exactly what we need.

image

The False Positive

With our knowledge of DAX we may now attempt modify our Close Price measure with the following:

image

This now returns the following results:

image

BOOM! Winner winner chicken dinner, I’m taking the rest of the day off and going to the beach! Wait a minute…. was the topic false positive?

Are the results above correct? Yes the are, just take my word for it or else this blog post is going to really, really long. However, it’s really easy to author formulas in DAX that work at one level but don’t work at other levels, and this is because of Filter context. As developers we have to always consider how the end users might slice the data. For example, if a user is looking at the data at the day level, will our measure still return he closing price for the month? Let’s check.

image

Immediately, we see two different items that tell us the measure is definitely not returning the end of month close price.

  1. First, the close price and the close price (eom) measure have identical values, this means our closing month measure is displaying the closing price for each individual day. That tricky filter context got us again!
  2. Secondly and most obvious, the close price (eom) values should be identical for every day of the month, they are not. Clearly this measure is not working. Back to the drawing board.

Go back and look at the definition for LASTNONBLANK, it works within the current filter context so when we filter our report down the day level it can only return that day.

PARALLELPERIOD IN DAX

Now it’s time to introduce you to one more function in DAX and that is the PARALLELPERIOD function.

Definition: Returns a table that contains a column of dates that represents a period parallel to the dates in the specified dates column, in the current context, with the dates shifted a number of intervals either forward in time or back in time.

(I just read that definition and now I’m confused….)

Syntax: PARALLELPERIOD(<dates>, <number_of_intervals>,<interval>)

The PARALLELPERIOD function will return all the dates in the interval that we specify within in the current context. What does that mean??? If you were looking at January 1st and you used PARALLELPERIOD to return all the dates at the month level then a table would be returned with all 31 days for January. This means that we can now return the closing price for the month even if the user is exploring the data at the day level!! I can feel your excitement as I write this.

Let’s jump right in and look at the final DAX expression. I am once again going to modify my existing calculated measure, this time I’m replacing ‘date’[date] with PARALLELPERIOD:

image

Here are the final results:
image

Enjoy!

How to add Data Labels to maps in Power BI

Recently, I was asked if the values for maps could be shown on the map and of course the first thing I thought was to just turn on data labels. Well, if you’re reading this then you already know there isn’t currently a way to add data labels. Now, I say currently because the Power BI team is always making changes and it could one day be there.

image

First, this was not my idea. When I realized there was not a data label option I knew someone in the community would have figured out a work around, and I was right!

Thanks to the awesome people who contribute at community.powerbi.com the answer was only a “google kungfu” search away. If you would like to see the original forum post where Sean “Super Contributor” helped out another Power BI user then check out the following URL:

https://community.powerbi.com/t5/Desktop/Data-Labels-on-Maps/td-p/79118

This blog is going to take a look at the following  three items:

  • Why this workaround works
  • How to set this up and configure it
  • Concerns and issues with this method

Why this workaround works

The map visual in Power BI will allow you to display the value of whatever is allocated to the Location field of the map. Generally you would simply put the country, state, city,  or some other valid location in this field. The limitation is that this field can only be populated with a calculated column. Therefore, Sean suggested combining the value you wanted to display into a calculated column field. Well, that allows you to now display the results, but it doesn’t map the data. The next step is you will need to have the latitude and longitude for each geographical location that needs to be mapped. The latitude and longitude coordinates will be added to the map visual to determine location and the location field will be used to display the data label.

Setup and configuration of Data Labels on Maps in Power BI!

Now, let’s dig in and build this out.

First, make sure your geography table has the required latitude and longitude columns. My geography table goes down to the zip code level of detail but I only brought in the Latitude and Longitude at the State level since I knew this is how I wanted to view the data in my map visual.

image

Next, it’s time to build the calculated column that will help you display the data.

  • Create a new calculated column on the Geography table.
  • The following DAX expression returns Total Sales by State combined with the state name.
  • I also added the FORMAT function to add some formatting to the values.
  • The part highlighted in the red box returns the Total Sales by State.

image

Now that the you have your latitude and longitudes and the calculated column has been built, it’s time to add this to a map! In the below screenshot I highlight where each of your fields should be assigned.

image

Finally, it’s time to display our Location on the map!

  • Under formatting options turn on Category Labels
  • Feel free to change the properties for color, text size, show background and transparency.

image

Concerns and Issues

First of all, you must store your values in a calculated column in order to display them.

  • This means that your values will be static, they won’t change as slicers change.
  • You need to know exactly what and how your end users will use the data, which is nearly impossible.
  • This column will take up resources in your data model.

Secondly, you must have the latitude and longitude for every location you want to put on your map.

  • It may be difficult to obtain a quality list of latitudes and longitudes for each address.
  • These additional columns in your data model will have a high level of uniqueness and will definitely consume valuable memory resources.

Enjoy!

PowerApps–Getting Started

WHOOAAA! PowerApps are awesome! Welcome to this new series I’m starting on PowerApps, this first blog will be a quick one to introduce you to PowerApps and help get you started.

Have you ever wanted to design your own app? Your options with PowerApps are literally endless… maybe you want to build a budget app, a shopping list, a game, or even a checklist. Buildings these apps are not only possible,  it’s also much easier than you can imagine. In no time you will be building your own apps from scratch using Microsoft PowerApps! Here is an example of a really nifty app I built in less than an hour to help score a board game we like to play called Ticket to Ride. Take a quick look at the screenshots below (Animated GIF provided at end of this blog):

SNAGHTML1432121

The “low-code” canvas app

Microsoft PowerApps is a “low-code” canvas based application that requires minimal coding. The coding that you do write is very basic and shares a lot of similarities with excel functions! That’s right, if you have written an expression in excel then you are half way to writing code in PowerApps.

What do you need to get started?

So what do you need to get started? First , you need a license to access Power Apps, fortunately if you have Office 365 you may already have a license! If not, you can sign up for a free trial of an E3 Microsoft Office Account.

  • Sign up for a new trial account here:
  • Enter your required information and then select Create my account.
  • Once you complete setting up your account you will receive a signin, this will be used for logging into Power Apps and Flow.

Next, you need to sign up for PowerApps

image

  • Use the signin from your E3 Office 365 account here.

Finally, you need to create a test environment

  • Navigate to the admin center. Click the settings cog and then select Admin Center:

SNAGHTML177acba

  • Next, click new environment
  • Give your environment a name, select a region, and environment type
  • Select the Join preview program box.
  • Create environment
  • Create database – > Choose currency and language options.
  • Select Include sample apps and data and then create the database by clicking Create Database.

Congratulations! You’re ready to start creating your own apps!

Thanks for checking out my blog, in future blog posts in this series I will show you how to quickly build apps so check back often!

Here is the animated GIF of the Ticket to Ride scoring app I created using PowerApps:

Ticket to Ride

Azure Data Factory–Copy Data Activity

In this blog, we are going to review the Copy Data activity. The Copy Data activity can be used to copy data among data stores located on-premises and in the cloud.

In part four of my Azure Data Factory series, I showed you how you could use the If Condition activity to compare the output parameters from two separate activities. We determined that if the last modified date of the file (located in blob storage) was greater than the last load time then the file should be processed. Now we are going to proceed with that processing using the Copy activity. If you are new to this series you can view the previous posts here:

If Condition Activity – If True activities

As mentioned in the intro, we are going to pick up where we left on in the last blog post. Remember that the If Condition activity evaluates an expression and returns either true or false. If the result is true then one set of activities are executed and if the result is false then another set of activities will be executed. In this blog post we are going to focus on the If True activities.

  • First, select the If Condition activity and then click on activities from the properties pane, highlighted in yellow in the image below.
  • Next, click on the box “Add if True Activity”.

image

This will launch a new designer window, and this is where we will be configuring our Copy Data activity.

image

Copy Data Activity, setup and configuration

Fortunately the Copy Data activity is rather simple to set up and configure, this is a good thing since it is likely you will be using this activity quite often. The first thing we need to do is add the activity to our pipeline. Select the Copy Data activity from the Data Transformation category and add it to the pipeline.

image

Now we need to set up the source and the sink datasets, and then map those datasets. You can think of the sink dataset as the destination for all intents and purposes.

Select the copy data activity and then click on the Source tab found in the properties window. For the dataset I am simply going to choose the same dataset we used in our first blog in this series.

image

Creating the Sink dataset in Azure Data Factory

Now it’s time to set up the Sink (destination) dataset. My destination dataset in this scenario is going to be an Azure SQL Database. I am going to click on Sink and then I will click on + New.

image

When you click on + New, this will launch a new window with all your data store options. I am going to choose Azure SQL Database.

image

This will launch a new tab in a Azure Data Factory where you can now configure your dataset. In the properties window select “Connection”, here you will select your linked service, this is your connection to your database. If you haven’t already created a linked service to your database then click on + New. Clicking new will open up a new window where you can input the properties for your linked service. Take a look at the screenshot below:

image

Once the linked service account has been created I will select the table I want to load. All available tables will show up in the drop down list. I will select my dbo.emp table from the drop down.

image

The final step for the dataset is to import the schema, this will be used when we perform the mapping between the source and sink datasets. Next, I am going to click on the Schema tab. On the Schema tab I will select import schema, this actually returns a column that I don’t need and so I will remove the column from the schema as well. Take a look at steps three and four in the following screenshot:

image

Copy Data activity – Mapping

Now that the source and sink datasets have been configured, it’s time to finish configuring the Copy Data activity. The final step is to map these two datasets. Now I will navigate back to the pipeline and click on the copy data activity. From the properties window I will select the Mapping tab. Then I will import the schemas, importing schemas will import the schemas from the source and the destination. This step will also attempt to automatically map the source and the destination as well and so it is important to verify that the mapping is correct. Unfortunately my source dataset was a file and it did not have any column headers. Azure Data Factory automatically created the column headers Prop_0 and Prop_1 for my first and last name columns. Take a look at the following screenshot:

image

This was a simple application of the Copy Data activity, in a future blog post I will show you how to parameterize the datasets to make this process dynamic. Thanks for reading my blog!

Quick Tips–Updating Parameters from the PBI Service

In this quick tip I want to share how you can update Parameters in Power BI from the service. Previously, this was not an option and Parameters could only be updated from Power BI Desktop.

Updating Parameters in Power BI

To update parameters, navigate to the Datasets section in the Power BI Service. Next, click on the schedule refresh icon. Updating parameters is done from the schedule refresh window.

SNAGHTML20f21a8d

Click on Parameters from the schedule refresh window. Once expanded you will see available parameters, simply update the parameters and you’re done!

image

Thanks for reading!

Quick Tips – Export data from Power BI using R

One of my favorite tricks in Power BI is to use R integration to quickly export data outside of Power BI. This is also a very popular question among Power BI enthusiasts. Once users realize the true capabilities and easy of the Power Query editor to transform and clean data they want to clean up their files using Power BI but the challenge is then how do I get the data out?

Prerequisite – R

If you want to test this method out you need to install R on your machine, for your convenience see the following link:

https://mran.microsoft.com/open/

Export data from Power BI using R

Like all of my quick tip blogs, this will be quick Smile.

You will export data from the Power Query Editor. To launch the Power Query Editor click Edit Queries from the Home ribbon in Power BI.

image

Inside the Power Query Editor, click the transform ribbon and then click on R.

SNAGHTML3d882ca

Once you click on the large letter R seen in the screenshot below a new window will open. Now you just have to type in some basic code. Here is the pseudo code:

write.csv(dataset, <destination>)

image

That’s it! Click ok and check your folder for the file. Do notice that I used two backslash characters, this is required. The other question I get is can I write the results to an excel file or to a SQL Database table and the answer is…. Yes! R has packages that make doing particular tasks easier and if you want to write to an Excel file or SQL Server table you would need to install those packages. My hope is to do some follow-up post around this topic.

image

Thanks for checking out my blog!