The Good and the Bad of Ranorex GUI Test Automation Tool

The Graphical User Interface (GUI) — the look and feel of the application — is the first thing that catches a user’s eye. And, as a result, it’s what the app is judged by. Therefore, it’s vital to ensure proper GUI functionality and seamless interaction. This can be done by testing the GUI from a user’s perspective.

GUI testing is performed to verify the features visible to a user, like menus, buttons, icons, text boxes, lists, dialog boxes, etc. It also makes sure that the appearance elements, such as fonts and images, comply with the design specifications. GUI testing takes place at the system testing level. To find out more about the levels of testing, visit our software testing whitepaper.

Automating GUI testing can be very tricky and user behavior is complex. Consequently, it makes sense to use a ready-made GUI test automation tool. We have already elaborated on the pros and cons of the Selenium test automation tool. In this article, we will focus on another popular automation framework, Ranorex Studio.

What is Ranorex?

Introduced to the market in 2007 by Ranorex GmbH, an Austrian software development company, Ranorex Studio is a commercial Windows GUI test automation tool that provides testing for desktop, web, and mobile apps.

Ranorex doesn’t use any specific scripting language, it is built on Microsoft’s .NET platform. The framework supports standard programming languages C# and VB.NET to edit recordings or create custom tests.

Ranorex is based on the XPath query language enabling easier and more efficient search of components in a web-based application. The Ranorex record and playback interface allows for automating UI testing by recording UI actions. Making testing way simpler with its codeless approach, Ranorex is definitely a go-to framework for a beginner.

Ranorex Studio supports a wide range of environments, specifications, and settings. Its broad tech support includes:

• Windows OS 7 through 10 and Windows Server 2008 through 2016
• Windows desktop technologies, including .NET, ActiveX, Delphi, Java, Telerik, and Microsoft Office apps
• Mobile testing on Android and iOS
• Cross-browser testing on all major browsers

Ranorex is a good solution for implementing automated testing in continuous delivery and DevOps environments which demand continuous testing and fast feedback to testing results. Ranorex offers a broad range of testing tools to integrate with: most popular continuous integration (CI) tools, test management (TM) tools, and task scheduling tools.

Ranorex integration with the leading testing tools; Source: Ranorex Brochure

Through Selenium WebDriver integration Ranorex allows for built-in object mapping, automatic timeout handling, smart identification of dynamic web elements, testing on additional operating systems: Linux and macOS.

Main characteristics of Ranorex GUI test automation tool

To get the full picture of Ranorex automation tool basics, check the following resources:

• Download full-featured 30-day Ranorex Studio trial
• Ranorex Studio User Guide
• Ranorex Forum
• Ranorex automation packages on GitHub

General Use

Ranorex Studio is one of the most comprehensive automation tools on the market as it provides solutions for multiple environments, devices, and applications allowing for automated testing of any desktop, web, or mobile software. That said, Ranorex is considered more suitable for web-based applications.

Ranorex usage distribution among different platforms; Source: Ranorex Studio survey

Popularity

A data-driven research company, iDataLabs, whose data for Ranorex usage goes back almost three years, has identified 954 companies using Ranorex, which is about 1.2 percent of the market share in the Software Testing Tools category.

Ranorex Market Share and Competitors in Software Testing Tools; Image source — iDataLabs

The same research shows that most companies using Ranorex are in the Computer Software industry and located in the United States.

Forty-nine percent of Ranorex customers are in the United States and 10 percent are in the United Kingdom; Image source — iDataLabs

Compared to other popular test automation tools on G2 Crowd, a review platform, Ranorex outperforms in 4 out of 7 categories: ease of use, quality of support, ease of doing business with, and product direction.

Ranorex Studio, TestComplete, Selenium IDE, and Apache JMeter compared at G2 Crowd

To find out more about Ranorex in relation to other test automation frameworks, check our article comparing automated testing tools.

Pros of using Ranorex

+ Multiplatform Application

With Ranorex you can run automated tests for web and desktop applications as well as mobile apps, while competing products such as Selenium and Katalon Studio don’t support desktop testing, and Watir aims at web testing only.

+ Codeless test creation

Ranorex Recorder with its drag-and-drop interface allows for conducting script-free tests applying keyword-driven testing. Once keywords are set by technically experienced team members, no programming background is needed to create and maintain automated tests with these keywords. Consequently, tests are easy to read and fast to design, since keywords can be reused throughout test cases.

Ranorex provides two approaches to keyword-driven tests:

1. Keyword-driven framework with automation modules
2. Keyword-driven testing using the action table.

While the former option can be followed without writing a single line of code, the latter one is more advanced as it requires basic programming skills.

As we can see, Ranorex codeless testing allows non-programmers to apply test automation to their projects. Selenium, on the other hand, requires solid programming knowledge, which leaves non-technical testers behind.

+ Smooth learning curve

Given its user-friendly UI on top of the code layer, Ranorex is considered one of the most approachable test automation tools on the market. In addition, Ranorex Help Center offers a variety of resources to ease the first steps of working with the framework:

• Ranorex Demo Application leads you through the process of creating, recording, and analyzing your automated software tests. Accompanied by detailed instructions, the app helps you get familiar with the Ranorex Studio user interface.
• trainings are in three languages — English, French, and German
• guides have step-by-step tutorials
• free test automation webinars are held regularly

Ranorex Studio does a good job in facilitating the workflow with their tool, compared to other providers. In particular, Selenium courses and tutorials are supplied by third parties.

+ Built-in image comparison

Automated software testing mostly relies on UI-elements recognition. Although many UI-elements may be identified by text, oftentimes there are cases where image-based automation is the way to go. For example, when a tested item changes its screen position within the GUI, text-based automation may still be tracking its original position, which will lead to incorrect results at the end. Image-based automation gets around such problems.

Ranorex smart object identification technique is able to automatically detect any change in the UI. It includes the following GUI recognition elements:

• RanoreXPath — a subset of XPath that serves to describe, search, identify, and find even dynamic UI-elements within an application
• The path editor — a tool for specifying and editing tracked and identified UI-elements for RanoreXPath
• Ranorex Spy — a major functionality that makes Ranorex unique from other test automation tools. It explores and analyzes the application under test capturing necessary elements.

While Ranorex offers image-based automation, Selenium, on the other hand, doesn’t provide such feature which requires you to either use another library or do it manually.

+ High-quality customer support

According to user reviews, Ranorex customer service is professional, helpful, and quick to reply. The reviews show that 10 out of 48 users mention the timely support and comprehensive response provided by personnel through the Ranorex forum.

However, some users claim that the support provides only standard answers and the only way to resolve complex issues is to allow the Ranorex support team to connect directly to the user’s machine. This may be forbidden by corporate data protection policies.

+ Effective in-team collaboration

In many ways, automated test development needs to be treated as software development; therefore, the collaboration between team members should be coordinated accordingly. While most competitors rely on source control tools, Ranorex offers its own solutions for cross-functional teams.

Based on an open XML format, Unified Functional Testing (UFT) is another QA automation tool designed with team collaboration in mind. However, it doesn’t have the workflow segmentation that allows Ranorex teams to consistently introduce specific skillsets into a project.

The Ranorex test automation project comprises the following layers:

1. Developers and technical testers script flexible automation elements on the Ranorex core automation framework;
2. Meanwhile, non-technical testers are able to create codeless test cases or reuse existing core modules;
3. Then, using the comprehensive XML-based test report, project owners and managers can review test results and check the project progress.

This way, teams can effectively cooperate on test automation projects.

Team collaboration within Ranorex test automation project; Source: Ranorex Brochure

+ Automatically generated reports

Every test run in Ranorex Studio ends with an automatic report that provides details on test execution including visual screenshots for validation. This makes the workflow much easier than Selenium’s, where you have to build a separate reporting tool to plug into your test solutions. TestComplete also offers test reports, but they contain results only on current test runs. Ranorex reports are more comprehensive, as they provide Pie Charts with both current and previous test runs.

Cons of using Ranorex

– Paid license

Ranorex is a licensed tool, which can be a considerable drawback taking into account many open-source competitive alternatives such as Selenium, Katalon Studio, and Watir. Ranorex is expensive, but it can be cost-effective in the long run given the number of features it supports out of the box.

– Only a few supported languages

While Ranorex provides flexibility in terms of platforms and browsers, it supports only two scripting languages: C# and VB.NET, unlike Selenium that allows for coding with ten popular programming languages.

– No macOS support

Although Ranorex now supports web testing of Mac apps integrating with Selenium WebDriver, the framework still can’t be launched on macOS. Ranorex is built on .NET, which doesn’t run on macOS unless using the cross-platform Mono project. However, Mono isn’t supported by Ranorex. In addition, Ranorex needs access to certain parts of the OS that Mac does not allow. Locking down their systems, macOS hinders many Ranorex features that work on Windows systems.

– Small community

While free tools such as Selenium and Watir are backed by a massive community, the Ranorex user base is much smaller. iDataLabs research shows that Selenium and Watir are used by 24,725 and 1,376 companies respectively, while only 954 companies are applying Ranorex. As a result, it might be difficult to find solutions to your problems and pieces of advice you’re seeking on the web. Users believe that the Ranorex team should work on organizing the community to share more plugins, problem-solving, etc.

– Unstable releases

While being one of the most popular test automation providers, Ranorex frequently releases new versions. This, in turn, takes up time for updating the existing suites of automated tests. While new releases isn’t something to complain about, oftentimes it turns out that the new features are still unstable and contain bugs. That’s why it might be necessary to wait for the second update of a new version before making the change.

Is Ranorex a good choice for your project?

Organization size. Although Ranorex might not be the first choice for a small development team due to many open-source alternatives, companies with large budgets can safely invest in the framework. That said, Ranorex Customer Survey 2018 shows that the largest number of companies using Ranorex have more than one thousand workers employed.

Companies using Ranorex arranged according to the number of personnel; Source: Ranorex Customer Survey 2018

User groups. Given the Ranorex script-free test automation, a non-programmer can make use of this framework. At the same time, Ranorex’s full IDE empowers automation experts to enhance their test suites and recordings.

Ranorex user groups; Source: Ranorex Customer Survey 2018

Use cases. A lot of well-known brands are automating their test process with Ranorex Studio.

As a healthcare technology supplier, Siemens implemented automated UI tests in their project Siemens Healthcare Computed Tomography. Ranorex’s modularization approach enabled sharing test automation steps and code modules globally among testing teams located in Germany and China. The storing of Ranorex test cases and results as .xml files simplified the implementation process. As a result, Ranorex helped cut down test execution time by more than 40 percent.

Unycom, IP Management Software, chose the Ranorex automation tool for both its web and desktop support, as well as rapid test development. With Ranorex automated tests, the QA staff is now able to build multiple tests and complete them overnight.

TomTom, a company for traffic, navigation and mapping products, applied Ranorex for mobile test automation of their Portable Navigation Device feature. After switching from the Calabash tool to Ranorex, TomTom benefitted from its friendly learning curve, interaction between multiple devices, and integration into HP tools for test case management. The company claims to have saved over 90 percent in time with Ranorex automated tests.

Final word

Ranorex is an effective GUI test automation tool that helps counteract challenges you may encounter while shifting towards agile software development. This framework accommodates different skillsets and ensures productive team collaboration while being integrable and innovative.

That said, Ranorex has its drawbacks too, so it makes sense to define your priorities while choosing a test automation tool for your team. Whether your project has budget constraints or a lot of junior testers aboard, whether you develop primarily on macOS devices or look for strong image recognition features — all these and many other specifications will help you decide if Ranorex is the framework you should go for.

This post is a part of our “The Good and the Bad” series. For more information about the pros and cons of the most popular technologies, see the other articles from the series:

The Good and the Bad of Selenium Test Automation Tool

The Good and the Bad of .NET Framework Programming

The Good and the Bad of Android App Development

The Good and the Bad of Java Programming

Originally published at AltexSoft Tech Blog “The Good and the Bad of Ranorex GUI Test Automation Tool”