Monthly Archives: October 2019

Environment Variables preview

 

A number of folks have pointed out that solution environment variables are, now, in preview:

https://docs.microsoft.com/en-us/powerapps/maker/common-data-service/environmentvariables

I was looking at it yesterday, and, it seems, I’m having a bit mixed emotions.

Environment variables seem to represent a hybrid experience: we can work with them in the model-driven application, but we can also work with them in the solution designer.

For instance, I can find Environment Variables Definitions in the advanced find, run a query, get the list, and review the variables:

image

image

I can even update the values on the screen above:

image

Then, if I open https://make.preview.powerapps.com/ (notice “preview” in the url), I can, then, open the same environment variable definition in the admin experience. As you can see on the screenshot below, the change I’ve made in the regular interface is, now, reflected in the admin interface as well:

image

In other words, it’s a regular (well, not quite regular, but still) entity that’s also exposed through the admin interface. Come to think of it, that’s not that unusual since there are, also, entities for such “metadata” as views, for example. So, maybe, at some point environment variables will simply be hidden from the regular user interface.

That’s, really, just a little unusual twist, but what’s really interesting is the usage scenarios.

First of all, we can include environment variables definitions into our solutions. And, then, we can also include special json files which will overwrite those default values.

Basically, when using the solution packager to re-package our solution, we can choose to overwrite those values in the destination depending on the environment where the solution will be deployed.

However, if you look at it from the build/release pipeline standpoint… If a solution represents a build artefact, we would not be able to just get that zip file and push it through to the QA/UAT/PROD when every environment needs different values for the environment variables.

Of course we can re-package every time to add environment-specific values, but that breaks the idea of having solution files as build artefacts.

The other option, it seems, would be to set environment variables in the target environment after solution import using some kind of a script, but, then, is it that beneficial to have environment variables in the solution?

There are a few other caveats in the preview, you can read about them in the docs (see that link at the beginning of this post), but might also want to consider a few more:

  • Both default value and value attributes are configured to have 2000 max length. It seems environment variables are not meant to store large amounts of data (as in, it would not be “by design” to serialize all your reference data  into a json string, pass that string to the target environment through a solution, and de-serialize the json there through a plugin, for instance
  • Using environment variables in the classic workflows would be problematic yet (without some kind of custom workflow activity)

 

With all that said, I think it’s a good thing the product team is moving in that direction, since, in the end, having some kind of solution-aware configuration entity which everyone is using would be really great.

Unless you have other thoughts? Let me know…

Self-Service Purchase Capabilities for Power Platform

 

You may have heard of it already, but there is going to be a new way to purchase Power Platform services soon – Self-Service Purchase Capabilities are arriving in November:

image

There is a corresponding FAQ page that answers some of the questions you may have:

https://docs.microsoft.com/en-us/microsoft-365/commerce/subscriptions/self-service-purchase-faq

One thing to keep in mind, though, is that there will be no opt-out – every tenant in the commercial cloud will receive this update. In general, I’m guessing this is not a bad idea to give more flexibility to the individuals and to the groups within the organization, but there are a few related concerns, and, the way I see it, Microsoft leaves it to the organizations to resolve those concerns:

Organizations can then rely on their own internal policies, procedures and communications to ensure that those individuals making self-service purchases are complying with company policies.

Tenant administrators will be able to see who made the purchases, for instance, but, in general, this may create a bit of a shadow IT situation, and that might be more concerning for the organizations that have to comply with the additional regulatory, auditing, data security requirements, or even just budgeting requirements.

From that standpoint, the key thing to remember is that it’s still the organization that would be responsible if anything goes wrong (from the compliance standpoint) with the solutions purchased and implemented through the self-service.

That said, I believe self-service can be a good thing. Somebody willing to purchase a service won’t need to go through the long process of procurement anymore, and that might make the process faster/easier.

There is a possibility somebody would decide to exploit this new option and bypass the approvals/procurement completely. Are the risks that big? It would not be that different from a person in your company going rogue to start buying laptops for their department without first approving that purchase somehow, and that would not be a very likely scenario.

Still, if you do need those approvals to happen, you may want to add a few more notes to the self-service purchase policies your company has established so far.

Business logic for Canvas Apps: Plugins vs Flows vs WorkFlows

In the previous post I talked about using a “RetrieveMultiple” (+Update/Create) plugin to incorporate business logic into a Canvas App. And, of course, the very first question I got is “why using a plugin if you could use a Flow”?

It’s a fair question, and I figured I’d try to provide a fair answer. Have a feeling it might end up being not that clear-cut at all, though.

First of all, whatever business logic I was going to implement, it was always meant to work with CDS. I was not going to connect to other data sources there, so, from that standpoint, a Flow would not offer any benefits.

A plugin would have to be developed, of course, and that assumes there would have to be a “professional” developer. That would be another advantage of using a Flow where a “citizen” developer could actually do the job. Although, in reality this is a little more complicated, since, if you look at the classic workflows and compare them with plugins and Flows from the development effort perspective, I think the comparison would go like this:

image

*That’s if we leave custom workflow activities out of this, of course.

Of course plugins are way more on the development side than Flows, but Flows seem to require more effort from the “citizen” developers than classic workfows.

Either way, in my case there would be a “professional” developer to maintain the plugins, so I was not too concerned about that part.

Now, with the Flows being stripped off their two main advantages over the plugins (in this particular situation), let’s see why a plugin might be a better option:

  • When compared to the classic “code” development, no-code development will always be limited no matter how great the tools are. To some extent it’s fine, but, then, once the logic becomes really complicated, it’s actually much easier to maintain it in code, since it’s a lot more concise there
  • Plugins are offering better control of where and how they will start. Actually, neither the Canvas App, nor the model driven app have to worry about whether the plugin will start or not – it will happen automatically
  • Flows are asynchronous, and plugins can be both synchronous and asynchronous. That’s not always a big deal, but can be useful every now and then
  • Plugins can be added to the solutions and moved between the environments. The same is true for the Flows; although, I believe we still need to reset Flow connections in the target environment after importing the Flow there. Plugins will just start to work
  • When developing a plugin, we can rely on the usual source control merge tools since plugins are all about the source code

 

In other words, the reason I went with a plugin in case with the inspection canvas application is, basically, that I did not need those advantages Flows can offer, and, as such, it seemed a plugin would fit better.

Hope it makes sense.

PowerPlatform: when a plugin meets a Canvas App

One problem with my inspection canvas app was that I did need some “business logic” to be there, and, of course, with the limited coding we can do in the canvas applications, it just had to be implemented somewhere else.

A Microsoft Flow maybe? Well… I am talking about a CDS application, and this is where I would probably go with a plugin over a flow just about any time for anything more complex than a simple query.

Let’s see what a plugin can do, then.

Here is a diagram:

image

The idea is that there are a few places I would use the plugin:

  • In Pre-RetrieveMultiple to pre-process the request. This is where I would parse the query to get product ID from the query so I could use it in Post-Retrieve
  • In Post-RetrieveMultiple to post-process the request. This is where I would use the product id to check if an inspection is allowed, if all the conditions have been met, probably to create the inspection record, and, also, to add error messages to a dedicated field on the inspection entity. Also, the same plugin would add json-serialized data to a dedicated field on the inspection record so that the canvas app could extract results for every inspection item from that field
  • And, finally, when updating the inspection, the canvas app would prepare the json and pass it back to CDS. An OnUpdate plugin would kick in to prase that json and to update inspection item results as required

 

You will find exported solution, including the canvas app and plugin sources here:

https://github.com/ashlega/ItAintBoring.CanvasInspectionDemo

On the high level, here is what’s happening there:

1. Parsing JSON in the canvas app

In the OnVisible of the second screen (where the user is supposed to confirm the produc #), the Canvas App would use a Lookup to load the inspection record by product number:

image

This is where “RetrieveMultiple” plugin would kick in.

ConfirmationGroup would become visible once SelectedInspection variable has been initialized, but only if there was no error:

image

Otherwise, ErrorGroup would show up:

image

Once the user clicks “Confirm” button in the confirmation group, Canvas App would use a regex to parse the json (which would be coming from the plugin):

image

But, before that, if something goes completely wrong, here is what Canvas App user would see:

image

On the other hand, if the plugin decides there is a problem, it would pass it back to the Canvas App through a field on the “fake” inspection record:

image

And the error would look like this:

image

Otherwise, if there were no errors, the user would be able to continue with the inspection:image

2. Passing JSON back to the plugin to create inspection items

Once the inspection is finished, the user would click Finish button, and the canvas app would pack inspection results into a json string:

image

Here is how “OnSelect” for that “Finish” button looks like:

image

The plugin, in turn, would parse that json to create inspection items:

image

Of course I might add all those inspection items as “checkboxes” to the inspection entity itself, but, when it’s done using a separate entity, I have more flexibility when it comes to adding new inspection items later.

And there we are… There is a Canvas Application that delivers better user experience just where it’s required, but all the business logic is implemented in the plugin, and, even more, that plugin can sort of talk to the canvas app by returning errors through a field, and, also, by providing(and accepting) additional data through json.

 

PowerPlatform: when a CanvasApp saves a project

 

Sometimes, a Canvas App can save a project. This would be almost impossible for me to say even a year ago; however, the way my current project evolved in the last two weeks, I can think of no better tool to quickly deliver the functionality required by the client.

And, so, if you are more on the professional development side, and if you are still not quite buying those Canvas Apps, here is the story you may want to read. There will be a Canvas App, there will be a model-driven app, there will even be a plugin just so you don’t get bored… besides, there will be json parsing, regular expressions, tablets, web browsers. And it actually won’t take that long.

Either way, here is an additional requirement I got from the client having spent a few months developing a model-driven app: “we need to be able to do product inspections as quickly as possible, and we can’t afford all those mistakes the inspectors can make if they have to learn how to navigate in the model-driven app”. That was the essence of it, and there were some caveats about extra validations, color coding, etc. Basically, what I was hearing is that “a model-driven app will work for everyone else but not for the inspector”. Oh, nice…

First I went into the panic mode, then I thought of the options, and this is when I realized that a Canvas App can cover all of that, even though a developer in me is naturally biased against “excel-like” development.

For the rest of this post(s?), I‘ll be using a simplified version of what I ended up with, but, I think, that will still be a good enough evidence of why a Canvas App can, sometimes, save a project.

I’ll be using two entities for the demo solution:

  • “Inspection” entity
  • “Inspection item” entity

 

The idea is that a user can create an inspection, add inspection items, and pass/fail each of the items so that the whole inspection will pass or fail. In the model-driven app, here is how it would look like:

imageRemember this is a simplified version – on the actual project, there were inspection types, templates, dates, they would have to be booked in advance by different users, the products to inspect might not be available, etc. And, since the inspectors would be using this application only sporadically, you can probably imagine why they would not be comfortable doing it unless the process were streamlined. Not to mention they’d rather be doing it on their tablets.

Come to think of it, that particular part of the model-driven application can be implemented as a canvas app with only three screens:

image

That’s, really, all the inspectors need.

Again, in the actual app there was a bit more to it. There were about 30 inspection items per inspection, different tabs, different colors, inspection item templates, inspection setup logic, etc. But those screenshots above should already give you a good idea of why a Canvas app can work better in this scenario. And, yes, it can all run on tablets.

I don’t, really, mean to list each and every step we have to take to create a canvas app, so I’ll only focus on what I found challenging or not that straightforward. Namely:

  • Formula-based development
  • Interface and designer limitations
  • Run-time and design-time being mixed up in the PowerApp Studio
  • Workaround for the absence of reusable functions
  • Working with JSON and regular expressions
  • Implementing business logic in a CDS plugin

1. Formula-based development

Basically, you are working with some version of Excel. I would not go as far as to say that “if you know Excel you know PowerApps”, but there is some truth there.

Just about everything is either a static value or a formula. It’s not a function you can define and reuse, it’s a formula you can write to calculate a value for a property. For example, on the screenshot below, “Text” property of the label will take its value from the ProductNumber variable defined somewhere else, and, as soon as that variable gets updated, that “Text” property will immediately reflect updated value:

image

Can you set that “Text” property directly? Nope.

I am not quite sure why – we can set variables, but we can’t set properties. Go figure.

Can we use a more complicated formula? Not a problem, except that you may need to get used to the syntax:

image

In that sense, Excel it is.

For the whole list of out-of-the-box functions you can use in those formulas, have a look here:

https://docs.microsoft.com/en-us/powerapps/maker/canvas-apps/formula-reference

2. Interface and designer limitations

This is a weird one. There are out of the box controls we can add to the screens. There are certain properties we can configure. Anything beyond that? Not an option.

Well, unless you go for an experimental PCF control, which can be very interesting and promising, but it’s not quite feasible for production yet:

https://powerapps.microsoft.com/en-us/blog/announcing-experimental-release-of-the-powerapps-component-framework-for-canvas-apps/

Just to give you an example. I wanted to have a 3-state checkbox for those pass-fail-n/a options. Of course I would not be able to do it in the model-driver application either without some kind of custom development, but, at least, I’d have web resources and PCF on that side. In the Canvas App, I had to opt for a radio buttons group. Which works, but it’s not quite ideal if you consider how much space those radio button groups will be taking on the screen when there are 30 different “groups”.

In the actual app, I also had to implement a version of the tab control. Which would be a no-brainer in HTML/Javascript, but, with the Canvas Apps, those had to be buttons, and each button would have to change visibility of some controls on the screen.

Why would not I be using a separate screen per tab? Because “Select” method can only work with the controls on the same screen – I’ll talk about it in the “workaround for reusable functions” below. It helps that in Canvas Apps we can group controls:

image

We can select multiple controls (either using a mouse, or using by clicking on them one after the other while holding “shift”), and, then, group them.

One advantage is that we can, then, control visibility of all those controls using group properties:

image

And, then, if you have multiple groups on the same screen, you can hide all but one, update control settings in that group, then switch to the other, etc. That way, you can design each group separately.

Although, this is where you will quickly realize that there is no big difference between run-time and design-time in the PowerApps Studio, and that causes a bit of a problem, though that also presents an interesting workaround. Because, for example, if you use a variable to control group visibility(that’s what you would need to make it all work at run-time), you will not have control over the Visible property in the designer anymore – it’ll be greyed out:

image

And what if you wanted to switch to a different group? The trick is, you need to start your screen, ensure the condition for that “Visible” property are met for your group, and, then, return to the designer. The group will be visible, and you can keep working on the design of the grouped controls:

image

When trying to implement something like a “tab control”, this seems to help… a lot.

3. Run-time and design-time being mixed up in the PowerApp Studio

It’s very unusual when your design-time is affected by the results of the most recent start of your application. It’s also unusual when you can start any screen independently without going through all the screens which are supposed to be opened first.

At the same time, there is a difference between how the application behaves in the PowerApp Studio and how the end users work with the application. When we click start button, that opens up specific application screen:

image

The end users don’t have that option – in order to open that particular application screen, they would have to go through the previous screens.

Another way to think about it would be to say that we are not just designing the app, but we are actually debugging an app (and we are allowed to design it at the same time). We can get to a screen and switch to the “design mode”. From there, we can check current values of all the variables and/or collections:

image

Maybe we can’t update those values directly, but we can bring up another screen and do it. For that matter, we can even create a special screen which would not ever become visible for the regular users since there would be no navigation path to that screen, and, then, use it to set the variables the way we need them. Sounds weird? Well, leave your biases behind for now, and just give it a try. Chances are, a CanvasApp would still be the best fit for what you need to do under some circumstances.

4. Workaround for the absence of reusable functions

There are no reusable functions. I am not sure why Microsoft did it this way. Don’t get me wrong – I now have a couple of examples when a Canvas App saved a project for me, but there are things I just don’t understand, and this is one of them.

No matter what, it’s almost impossible to do any kind of development without having some sort of a function – it’s simply impossible to keep copy-pasting code from one place to another.

And there is a workaround, but, as any workaround, it’s not perfect.

See, we can use “Select” function.

We can create a button, put it on the screen, make it invisible, add some code to the “OnSelect” property, and, then, use “Select(buttonName)” to call that code from anywhere on the same screen.

Here is an example. Let’s say I wanted to store “fail/pass” results for each item in a variables (separately for each item). Of course I could do it in the “OnChange” of each radio button control, but I could also add a new button to the screen, hide it, and add the following code to the “OnSelect” of that button:

image

Which would allow me to call “Select(buttonSetVariables);” from any other place on the same screen:

image

How does it help?

What if I wanted to add common actions to each of those OnChange above? For instance, what if I wanted to update a record in the CDS database every time a radio button is updated? Now I could add a Patch call to just one place, pass all those variables to the Patch, and I would not have to remember to update every single OnChange instead.

Of course you might say that using Patch that way would cause an update to each and every attribute, and that’s not, necessarily, what I may want to do, but you can see how it simplifies code maintenance. Essentially, we are getting some kind of a function. If only Microsoft could introduce real functions…

It seems there is a related technique. You can use global variables, and you can add initialization code to the “OnVisible” property of the application screens. For example, in the inspection application above, I would initialize all global variables on the second screen (once the product # has been selected), and, then would keep updating them on the third screen through that hidden button. The idea is to keep those big chunks of code in one place, even if that means doing a bit more than you would, normally, do in a function.

At this point, there are two remaining topic which would better be discussed together, but which would qualify for a separate post: JSON/Regex and plugin-based business logic.

Hence, have fun for now, and stay tuned for a follow up post.

Readonly = impression, FieldSecurity = impression + access restrictions, Plugins = controlled changes

 

Why is it not enough to make a field readonly on the form if you want to secure your data from unintended changes?

Because there are at least 2 simple ways to unlock such fields:

1. Level up extension from Natraj Yegnaraman

https://chrome.google.com/webstore/detail/level-up-for-dynamics-crm/bjnkkhimoaclnddigpphpgkfgeggokam

Here is how read-only fields look before you apply “God Mode”:

image

Then I apply the “God Mode”:

image

And I can happily update my read-only field:

image

Which is absolutely awesome when I am a system administrator trying to quickly fix some data in the system, but it can become a nightmare from the data consistency standpoint if I am either not a system administrator or if, even as a system administrator, I am not really supposed to do those things.

2. When in the grid view, you can use “Open in Excel Online” option

image

“Read-only” is how you mark fields on the forms, not what really applies to the server side/excel/etc. So, once you’ve updated the records in excel, you can just save the changes:

image

Of course you can also export/import, but “online” is the quickest in that sense.

What about the field-level security, though?

It does help when the user in question is not a system admin:

https://docs.microsoft.com/en-us/power-platform/admin/field-level-security

“Record-level permissions are granted at the entity level, but you may have certain fields associated with an entity that contain data that is more sensitive than the other fields. For these situations, you use field level security to control access to specific fields.”

Is that good enough? This is certainly better, but it does not help with the opportunist system admins who just want to deal with the immediate data entry problem. They will still have access to all the fields.

Is there a way to actually control those changes?

Well, strictly speaking you can’t beat a system admin in this game. However, you might create a synchronous plugin, or, possibly, a real0time workflow to run on update of some entities and to raise errors whenever a user is not supposed to modify certain data.

Would it help all the time? To some extent – a system admin can just go into the environment and disable a plugin. Of course that’s a little more involved, and that’s the kind of intervention not every system admin would be willing to do, but still. However, for the most critical data you could, at least, use this method to notify the system administrator why updating such fields would not be desirable. Which is, often, part of the problem – if there is a read-only field and no explanations of why it is read-only, then… why not to unlock it, right? But, if a plugin throws an error after that with some extra details, even the system admin might decide not to proceed. Technically, you might also use a real-time workflow for this(just stop a real-time workflow with “cancelled” status), but it might be difficult/impossible to verify conditions properly in the workflow.

Anyway, those are the options, and, of course, in terms of complexity, making a field read-only would be the easiest but it would also be the least restrictive. Using field level security would be more involved but would restrict data access for anyone but for the system administrators. Plugins might give even more control, but that would certainly be development.

Trying out Microsoft bot technologies

 

I got a little different idea for the month of October, and it is not directly related to the PowerApps development, but it depends on what I can do with the bots. So, naturally, I’ve been trying Bot Framework, Virtual Agent, and may need to try Virtual Assistant samples.

If this interests you, carry on readingSmile

image

First of all, Bot Framework is in the core of any “agent/assistant/bot” technology from Microsoft. However, the way I see it:

  • Bot Framework is the underlying technology
  • Virtual Agent is a self-driving vehicle
  • Virtual Assistant is a Formula 1 car

And, of course, you can always develop your own bot from scratch directly on top of the Bot Framework.

Either way, let’s talk about the Virtual Agent in this post.

First of all, Virtual Agent is a Dynamics/PowerApps-connected technology. You can easily utilize Microsoft Flow directly from the Virtual Agent (but you can’t do much more in terms of “development”… which is how PowerPlatform often is – it’s a “low code”/”no code” platform in many cases)

Then, it’s in a preview. Don’t start posting questions to the community forum in the first 10 minutes after you’ve created a trial through the link below:

https://dynamics.microsoft.com/en-us/ai/virtual-agent-for-customer-service/

Wait till you see the confirmation:

image

Until then, you will see what can probably be described as “deployment work in progress” – some areas will be working, but others won’t. For example, all the buttons to create new topics or new bots will be greyed out and unavailable.

Either way, here is what I wanted to make my virtual agent do:

  • Wake up – easy to do, just need to define some trigger phrases. That said, the agent does not seem to recognize a trigger phrase if it’s even slightly different
  • Greet the visitor – not a problem, it’s just something the bot needs to say
  • Take the answer and get the name

 

This last ones seems simple, but it actually requires a Flow. The user might say “I am…”, “My name is…”, etc. However, all I can do in the agent designer is take that answer into a variable and use “is equal to” condition in the expressions:

image

Which is not quite what I need since I’d rather knew the visitor name. Hence, I need to call a Flow through what is called an “action” in the Virtual Agent to try to parse the answer. Actually, there is a demo of a similar Flow here:

https://www.youtube.com/watch?v=joXCzvi38Fo&feature=youtu.be

That’s overly simplified, though, since I need to parse the user response to remove “I am…”, “My name is…”, etc.

This is where I quickly found out that error reporting between Flows and Virtual Agent may still need some work because, once I had my Flow created and added as an action to the bot, I started to get this:

image

If I could only see what’s wrong somehow? Apparently, the Flow was running ok:

image

Turned out that was just a matter of forming HTTP response incorrectly in the Flow:

image

The agent was expecting “VisitorName” since that’s how the parameter was named in the schema, but the flow was returning “Name” instead. In the absence of detailed error reporting it’s always the simplest stuff that you verify last, so took me a while to figure it out – was a quick fix after that.

In the end, it was not too complicated, and, apparently, this relative simplicity is why we’d want to use a Virtual Agent:

 

image

From there, if I wanted this agent to run on my own web site, there is an iframe deployment option. Basically, as I mentioned before, this is all in line with the low-code/no-code approach.

And, because of the Flow integration(which, in turn, can connect to just about anything), the possibilities there are just mind-blowing. We can use a virtual agent to:

  • Confirm site visitor identity by comparing some info with what’s stored in Dynamics/PowerApps
  • Help site visitors open a support ticket
  • Provide an update on their support tickets
  • Surface knowledge base articles from Dynamics
  • Help them navigate through the phone directory
  • Search for the special offers
  • Etc etc

Besides, there is a Flow connector for LUIS, and I could probably add intent recognition and do other cool stuff using Flows:

https://flow.microsoft.com/en-US/connectors/shared_luis/luis/

 

I would definitely stick to trying it more, but I really wanted to integrate my bot with speech services, and, since this feature is not available yet (as per this thread: https://community.dynamics.com/365/virtual-agent-for-customer-service/f/dynamics-365-virtual-agent-for-customer-service-forum/358349/does-dynamics-365-virtual-agent-for-ce-support-voice-bot), will be moving on to the Bot Framework and Virtual Assistant for now.

Which means leaving the familiar PowerPlatform domain. Let’s see what’s out there in the following posts…

 

Lookups behavior in Wave 2 – recent items, wildcard search, magnifying glass button, etc

 

I am wondering how many of us have missed this post:

“Preview for usability enhancements to lookups in Unified Interface”

https://powerapps.microsoft.com/en-us/blog/preview-for-usability-enhancements-to-lookups-in-unified-interface/

I certainly did. So, to summarize, here is what’s happening now when you are clicking that magnifying glass button in the lookup field:

  • If “recent list” is enabled for the lookup, you will see recent items only
  • If “recent list” has been disabled through the configuration, you will see nothing (there will still be “new <record>” link though)
  • If you try typing “*” and clicking the magnifying glass after that, you will get nothing as well (there seem to be a bug when using “*” on its own)

 

If you wanted to bring up the list of records (not the “recent” records), there seem to be two options:

  • When in the lookup field, while it is still empty, push “enter” button
  • OR enter some text to search for and push “enter” or click the magnifying glass button(for example, “T*123” would work fine… as long as it’s not just “*”)