Month: August 2018

Semi additive measures in DAX and Closingbalancemonth

/ Mitchell Pearson / 3 Comments

It’s been a while since we have visited Data Analysis Expressions (DAX) on this blog, but now we’re going to jump right in and discuss working with semi-additive measures. Semi-additive measures can’t be added across all dimensions, typically they can’t be added across the date/time dimension. Common examples of semi-additive measures are account balances and inventory levels.

  • Inventory levels can be added up across products, across different stores, but not across time. If you have 500 silver widgets at the end of day on Monday and you have 500 silver widgets at the end of day on Tuesday, how many widgets do you have? You only have 500 of course! We have to take this into consideration when building our model and measures.
  • The same is true of account balances as well. If I have $100 in my account on February 1st and I have $85 in my account on March 1st, what is my account balance? It’s only $85, it’s not the sum of both months.

The Scenario

In this scenario, I am looking at the stock price of Microsoft over time. We want to determine things like Closing and Opening price among others. For this example we are going to try and calculate the closing price for the month using the function CLOSINGBALANCEMONTH. This will present an interesting problem that we will discuss shortly. First, my data model has a simple measure which returns a SUM of the closing stock price as seen below:

image

Remember, this measure is valid for all the dimensions in my data model except for my date dimension. Similar to inventory levels and account balances we don’t want to add the closing stock price across time, this produces incorrect results. Back in 2012 the stock price of MSFT stock was around $30 a share, however, when I display our measure in a table with the year and month what we actually see are numbers that are much higher. This is because the measure is adding up the stock price for all days of the month and this is incorrect!

image

CLOSINGBALANCEMONTH

Definition: Evaluates the expression of the last date of the month in the current context.

Syntax: CLOSINGBALANCEMONTH( <expression>, <dates>)

CLOSINGBALANCEMONTH is one of the built in time intelligence functions and it works great, most of the time. Where it falls short is when you have blanks or gaps in your data. I am going to create a new measure using CLOSINGBALANCEMONTH using the following expression:

image

Now let’s take a look at the results:

image

Most months we are getting the correct value, but some months are blank, why? CLOSINGBALANCEMONTH returns the closing price of the stock for the last day of the month, unfortunately we are looking at stock market data and the stock market is not open every day of the month. So therefore, if the last day of the month has no closing stock price, then blank is returned. See screenshot below. This is typically not what we want when looking at semi-additive measures! We want to return the closing balance for the last day of the month that had a value.

LASTNONBLANK IN DAX to handle blanks!

We need to write a measure in DAX that is going to determine the last date of the month where the stock market was open. We are going to solve this problem by using the function LASTNONBLANK. This is an extremely useful and helpful function.

Definition: Returns the last value in the column, filtered by the current context, where the expression is not blank.

Syntax: LASTNONBLANK(<column>, <expression>)

I am going to build this out incrementally for demonstration and validation purposes. First, we are going to create a new measure just to see what this function returns:

image

Next, I will add this new measure to our table for validation. Everything looks perfect! The measure is not blindly returning the last day of the month, it’s returning the last day of the month that had a closing price for the stock, meaning the value returned for June, September, and March is exactly what we need.

image

The False Positive

With our knowledge of DAX we may now attempt modify our Close Price measure with the following:

image

This now returns the following results:

image

BOOM! Winner winner chicken dinner, I’m taking the rest of the day off and going to the beach! Wait a minute…. was the topic false positive?

Are the results above correct? Yes the are, just take my word for it or else this blog post is going to really, really long. However, it’s really easy to author formulas in DAX that work at one level but don’t work at other levels, and this is because of Filter context. As developers we have to always consider how the end users might slice the data. For example, if a user is looking at the data at the day level, will our measure still return he closing price for the month? Let’s check.

image

Immediately, we see two different items that tell us the measure is definitely not returning the end of month close price.

  1. First, the close price and the close price (eom) measure have identical values, this means our closing month measure is displaying the closing price for each individual day. That tricky filter context got us again!
  2. Secondly and most obvious, the close price (eom) values should be identical for every day of the month, they are not. Clearly this measure is not working. Back to the drawing board.

Go back and look at the definition for LASTNONBLANK, it works within the current filter context so when we filter our report down the day level it can only return that day.

PARALLELPERIOD IN DAX

Now it’s time to introduce you to one more function in DAX and that is the PARALLELPERIOD function.

Definition: Returns a table that contains a column of dates that represents a period parallel to the dates in the specified dates column, in the current context, with the dates shifted a number of intervals either forward in time or back in time.

(I just read that definition and now I’m confused….)

Syntax: PARALLELPERIOD(<dates>, <number_of_intervals>,<interval>)

The PARALLELPERIOD function will return all the dates in the interval that we specify within in the current context. What does that mean??? If you were looking at January 1st and you used PARALLELPERIOD to return all the dates at the month level then a table would be returned with all 31 days for January. This means that we can now return the closing price for the month even if the user is exploring the data at the day level!! I can feel your excitement as I write this.

Let’s jump right in and look at the final DAX expression. I am once again going to modify my existing calculated measure, this time I’m replacing ‘date’[date] with PARALLELPERIOD:

image

Here are the final results:
image

Enjoy!

Advertisement

How to add Data Labels to maps in Power BI

/ Mitchell Pearson / 1 Comment

Recently, I was asked if the values for maps could be shown on the map and of course the first thing I thought was to just turn on data labels. Well, if you’re reading this then you already know there isn’t currently a way to add data labels. Now, I say currently because the Power BI team is always making changes and it could one day be there.

image

First, this was not my idea. When I realized there was not a data label option I knew someone in the community would have figured out a work around, and I was right!

Thanks to the awesome people who contribute at community.powerbi.com the answer was only a “google kungfu” search away. If you would like to see the original forum post where Sean “Super Contributor” helped out another Power BI user then check out the following URL:

https://community.powerbi.com/t5/Desktop/Data-Labels-on-Maps/td-p/79118

This blog is going to take a look at the following  three items:

  • Why this workaround works
  • How to set this up and configure it
  • Concerns and issues with this method

Why this workaround works

The map visual in Power BI will allow you to display the value of whatever is allocated to the Location field of the map. Generally you would simply put the country, state, city,  or some other valid location in this field. The limitation is that this field can only be populated with a calculated column. Therefore, Sean suggested combining the value you wanted to display into a calculated column field. Well, that allows you to now display the results, but it doesn’t map the data. The next step is you will need to have the latitude and longitude for each geographical location that needs to be mapped. The latitude and longitude coordinates will be added to the map visual to determine location and the location field will be used to display the data label.

Setup and configuration of Data Labels on Maps in Power BI!

Now, let’s dig in and build this out.

First, make sure your geography table has the required latitude and longitude columns. My geography table goes down to the zip code level of detail but I only brought in the Latitude and Longitude at the State level since I knew this is how I wanted to view the data in my map visual.

image

Next, it’s time to build the calculated column that will help you display the data.

  • Create a new calculated column on the Geography table.
  • The following DAX expression returns Total Sales by State combined with the state name.
  • I also added the FORMAT function to add some formatting to the values.
  • The part highlighted in the red box returns the Total Sales by State.

image

Now that the you have your latitude and longitudes and the calculated column has been built, it’s time to add this to a map! In the below screenshot I highlight where each of your fields should be assigned.

image

Finally, it’s time to display our Location on the map!

  • Under formatting options turn on Category Labels
  • Feel free to change the properties for color, text size, show background and transparency.

image

Concerns and Issues

First of all, you must store your values in a calculated column in order to display them.

  • This means that your values will be static, they won’t change as slicers change.
  • You need to know exactly what and how your end users will use the data, which is nearly impossible.
  • This column will take up resources in your data model.

Secondly, you must have the latitude and longitude for every location you want to put on your map.

  • It may be difficult to obtain a quality list of latitudes and longitudes for each address.
  • These additional columns in your data model will have a high level of uniqueness and will definitely consume valuable memory resources.

Enjoy!

PowerApps–Getting Started

/ Mitchell Pearson / Leave a comment

WHOOAAA! PowerApps are awesome! Welcome to this new series I’m starting on PowerApps, this first blog will be a quick one to introduce you to PowerApps and help get you started.

Have you ever wanted to design your own app? Your options with PowerApps are literally endless… maybe you want to build a budget app, a shopping list, a game, or even a checklist. Buildings these apps are not only possible,  it’s also much easier than you can imagine. In no time you will be building your own apps from scratch using Microsoft PowerApps! Here is an example of a really nifty app I built in less than an hour to help score a board game we like to play called Ticket to Ride. Take a quick look at the screenshots below (Animated GIF provided at end of this blog):

SNAGHTML1432121

The “low-code” canvas app

Microsoft PowerApps is a “low-code” canvas based application that requires minimal coding. The coding that you do write is very basic and shares a lot of similarities with excel functions! That’s right, if you have written an expression in excel then you are half way to writing code in PowerApps.

What do you need to get started?

So what do you need to get started? First , you need a license to access Power Apps, fortunately if you have Office 365 you may already have a license! If not, you can sign up for a free trial of an E3 Microsoft Office Account.

  • Sign up for a new trial account here:
  • Enter your required information and then select Create my account.
  • Once you complete setting up your account you will receive a signin, this will be used for logging into Power Apps and Flow.

Next, you need to sign up for PowerApps

image

  • Use the signin from your E3 Office 365 account here.

Finally, you need to create a test environment

  • Navigate to the admin center. Click the settings cog and then select Admin Center:

SNAGHTML177acba

  • Next, click new environment
  • Give your environment a name, select a region, and environment type
  • Select the Join preview program box.
  • Create environment
  • Create database – > Choose currency and language options.
  • Select Include sample apps and data and then create the database by clicking Create Database.

Congratulations! You’re ready to start creating your own apps!

Thanks for checking out my blog, in future blog posts in this series I will show you how to quickly build apps so check back often!

Here is the animated GIF of the Ticket to Ride scoring app I created using PowerApps:

Ticket to Ride

Azure Data Factory–Filter Activity

/ Mitchell Pearson / 9 Comments

image

The series continues! This is the sixth blog post in this series on Azure Data Factory, if you have missed any or all of the previous blog posts you can catch up using the provided links here:

What is the Filter activity in Azure Data Factory?

The Filter activity applies a filter expression to an input array. Understanding that definition will help simplify how and where to use this activity.

Let me set up the scenario for you. In this example, I want to use Azure Data Factory to loop over a list of files that are stored in Azure Blob Storage. I am going to use the Metadata activity to return a list of all the files from my Azure Blob Storage container. Unfortunately, I don’t want to process all the files in the directory location. Below I have posted the list of files currently in my storage account, notice the file name “inputEmp_tq.txt”. I want to remove this file from the list of files returned.

image

Get Metadata Activity – Get List of Files to process

I reviewed the metadata activity in the very first blog post in this Azure Data Factory series, therefore, I won’t bore you with those details again.

  1. The dataset is pointing to a folder location, not a specific file. This is important.
  2. I selected “Child Items” from the field list properties, this will return the name of all files in that directory location.

image

Filter Activity – Remove unwanted files from an input array

The first step is to add the filter activity to the pipeline and connect the activity to the successful output of the metadata activity:

image

Now it’s time to set up the Filter activity. The filter activity requires two items during configuration.

  1. Items – Input array on which filter should be applied.
  2. Condition – Condition to be used for filtering the input array.

The items will be the output of our metadata activity and the Condition I will build using the built in expression language. Only items that evaluate to true will be returned in the final array!

Filter Activity Configuration

First, I will configure the Items property. This is simply the output parameter from the metadata activity and so I will use the following code:

@activity(‘meta_GetListOfFiles’).output.childItems

image

Next, I will set up a condition that will remove any files that don’t match the naming pattern I want. In this scenario I am simply looking for file names that start with FactInternetSales_ and any files that don’t match this criteria will be removed from the final array. You can see the actual formula in the previous screenshot but I want to quickly show you how I found the function used for this example.

First, I am going to click my mouse cursor in the Condition box and then I will click on “Add dynamic content”.

image

If you have followed any of the blogs in this series then you are familiar with this new window that opens up. The function that I use in this example is a String Function called startswith. See the following screenshot on where I found this function:

image

Here is the final expression:

image

As always, thanks for reading my blog!