Summary: The Power Apps Test Studio is, in most cases, a lackluster tool for creating both unit tests and automated UI test cases. Yes, it mimics some of that functionality that other more mature platforms provide, and there is a case to be made for very simple micro apps possibly benefitting from this feature from a checkbox perspective, it’s wholly unuseful for large scale enterprise low code (or “pro code” as we are beginning to call it) applications. Here are my findings and some alternative paths.

Context

As I straddle two sides of the technical solutions delivery sphere. The first being delivery and build using all sorts of tech stacks, not just Power Apps, and the other as a technical pre-sales solution architect, I want to open this post with a brief note about scoping. If time permitted, I would dwell upon both UX and QA orchestrations; however, I have to limit this to QA as I have some other jobs to do, thankfully.

Our first scenario happens when you are engaging with either your stakeholders inside your organization at the front end of a project scoping conversation, ideally way before, but often times, right up to the point of budgetary funding being (or worst case having already been) allocated. The second? When a client comes to you with an idea for a project and you are asked to help create a solution path to do a build, provide the cost estimates, the human resources, and a viable delivery model toward success.

But This Usually Happens Instead

In both cases, let there be a red flag when you hear or use phrases like “developer testing.” That is a formula for pain and suffering. Especially if a testing framework has not been identified or recommended with time allocated to its integration. It’s better to, instead, bring the QA team into the fold, not roll these responsibilities to your other team members who are wearing different hats. QA folks are developers for the most part. They think as coders do. But they also go through the maneuvers of a user doing various test cases. Gherkin is a common one I learned to follow for one example. A QA process needs to be methodical, repeatable, careful of regression cases.

With this in mind, when you are building an app of medium to large complexity, it is always a good practice to use some sort of framework for unit testing. Ideally inclusive of UI testing. This is where Power Apps falls flat.

Advertised:

  1. Automated Testing:
    • Efficiency: Automates repetitive testing tasks, reducing the time and effort required for manual testing.
    • Consistency: Ensures consistent testing outcomes, reducing the risk of human error.
  2. Integration:
    • Seamless Integration: Integrates well with Power Apps, making it easier to create, manage, and run tests within the same environment where the app is developed.
  3. Test Management:
    • Organized Test Cases: Allows for organizing test cases and scenarios, making it easier to manage and maintain them.
    • Debugging Tools: Provides tools for debugging and troubleshooting test failures, aiding in quick resolution of issues.
  4. CI/CD Pipeline Support:
    • Continuous Integration: Can be integrated into CI/CD pipelines, enabling automated testing as part of the deployment process.
    • Quality Assurance: Helps in maintaining high-quality standards by catching issues early in the development cycle.
  5. User Interface Testing:
    • UI Focused: Specifically designed for testing the user interface of Power Apps, ensuring that the user experience remains intact after updates or changes.

Reality:

  1. Learning Curve:
    • Complexity: May have a steep learning curve for those new to automated testing or unfamiliar with Power Apps Test Studio and thereby not really fostering this as a practice for business and citizen developers.
  2. Limited Functionality:
    • Scope: While advertised as “powerful,” it has serious limitations in functionality compared to more mature automated testing frameworks.
    • Head Scratching Example: Within an app, with an in-app “Custom Component,” you can’t test those with the Test Studio, so if you followed good patterns of reusability, that disqualifies the Test Studio right out the gate.
    • Custom Scripts: Limited ability to create highly customized test scripts, which can be a constraint for complex testing scenarios.
  3. Resource Intensive:
    • Performance: Running multiple automated tests can be resource-intensive, potentially slowing down the development environment.
  4. Maintenance Overhead:
    • Test Maintenance: Now, this is true for all test scripts; it just bears mention that these are drag-and-drop style things here, working with them strictly in the UI, and not with code, is seriously burdensome as the app evolves.
  5. Dependency on Power Platform:
    • Platform-Specific: Tightly coupled with the Power Platform, which might be a drawback if you need to integrate with other testing tools or platforms outside the Microsoft ecosystem.

Alternatives

Thankfully, it is possible to use other web-based test frameworks instead of the native Power Apps Test Studio for testing Canvas applications. Here are some popular web-based test frameworks that can be used, along with considerations for integrating them with Power Apps Canvas applications.

  1. Selenium:
    • Pros: Widely used for web application testing, supports multiple programming languages, and can handle complex test scenarios.
    • Cons: Requires more setup and configuration compared to Power Apps Test Studio. Tests can be more brittle due to reliance on HTML element locators.
  2. Cypress:
    • Pros: Modern and developer-friendly, provides fast and reliable end-to-end testing, excellent for JavaScript-heavy applications.
    • Cons: Limited support for multiple browsers (primarily Chrome and Firefox), may require additional setup for complex testing scenarios.
  3. Playwright:
    • Pros: Supports multiple browsers (Chromium, Firefox, WebKit), designed for reliable cross-browser testing, and offers powerful testing capabilities.
    • Cons: Newer framework with a smaller community compared to Selenium, may require more effort to set up for Power Apps testing.
  4. Puppeteer:
    • Pros: Node.js library providing a high-level API to control Chrome/Chromium, great for headless browser testing, and highly performant.
    • Cons: Limited to Chrome/Chromium, may require custom scripts for complex scenarios.

Considerations for Integrating with Power Apps:

  1. Element Identification:
    • Ensure that Canvas app elements have stable and unique identifiers (like data-testid attributes - or whatever consistent naming convention you and your organization or client choose to adopt) to make them easier to locate and interact with in tests.
  2. Authentication:
    • Handle authentication workflows within your test scripts, as Power Apps typically requires authentication through Microsoft accounts.
  3. Environment Setup:
    • Set up a dedicated test environment for running automated tests to avoid interference with development or production environments.
  4. API Interaction:
    • If your Canvas app interacts with APIs, consider using mocking or stubbing to simulate API responses for more reliable and isolated testing.
  5. Continuous Integration:
    • Integrate your chosen test framework with a CI/CD pipeline (e.g., Azure DevOps, GitHub Actions) to automate test execution as part of your deployment process.

Example Integration with Selenium:

  1. Setup Selenium WebDriver:
    • Install Selenium WebDriver and a browser driver (e.g., ChromeDriver).
  2. Create Test Scripts:
    • Write test scripts using a programming language supported by Selenium (e.g., Python, Java, C#).
  3. Run Tests:
    • Execute your Selenium test scripts against your Power Apps Canvas application, ensuring that your test environment is properly configured.

Example Integration with Cypress:

  1. Setup Cypress:
    • Install Cypress via npm and configure your project.
  2. Create Test Scripts:
    • Write test scripts in JavaScript using Cypress commands to interact with your Canvas app elements.
  3. Run Tests:
    • Use Cypress CLI to run your tests and generate reports.

By using these frameworks, you can take advantage of their advanced features and broader ecosystem, while still ensuring the quality and reliability of your Power Apps Canvas applications. All while reducing the workload of testing from the developer’s workflow and providing the QA team with a much more stable product upon which to continue to do pre-UAT work to ensure the best quality product being delivered to your users.

Closing

With these words of caution. Power Apps Test Studio is wholly unfit for any type of unit and/or automated testing work needed for medium to large scale enterprise applications. There are alternatives to this, however. Lastly, please, for the love of the user, and the reason we do any of this work in the first place, give QA a seat at the table. Don’t be evil and think you will save a dime by not adding in hours for not just this, but time in your estimate for the selection and implementation of a write-your-test-first approach in the developer’s work stream. That is the path to success. All others? Well, that’s where project “war stories” come from.