Testing custom applications

By writing testable code and automating your testing workflow, you can increase your efficiency when deploying custom applications. You can use testing to ensure that your existing applications remain compatible with new versions of Relativity services and robustly respond to this evolving platform.

This page contains information about how you can implement testing for new and existing custom applications that shorten your regression cycles and efficiently deploy applications into production. It also discusses the importance of writing testable code that uses automation in this process.

Types of tests for custom applications

It is time consuming, inefficient and non-sustainable to run tests manually for every change of Relativity services. Testable code and automation make regression cycles much shorter. If your software is available in the cloud, you don't have control over the release cycle. By testing your code quickly and on schedule, you can efficiently deploy it into production.

Unit tests

Unit tests are used to independently test a unit of code to evaluate proper operation. They are easy to maintain and can be run at any time. As a best practice, they should only take a few seconds to run. For more information, see Steps for writing unit tests.

Unit tests use mock objects that simulate the behavior of an external component. They isolate the code under test and help to identify whether a single code change has any effects on the code base. The following tools support unit testing with mock objects:

• Moq
• NSubstitute
• Rhinomocks

For example, you might use Moq as a mocking framework, and the NUnit 3.6.0 or above to write unit tests.

The disadvantage of unit tests is that they only test the functionality of a single unit. They may not catch system-wide errors, or integration errors that result from interactions with other parts of the system.

Integration tests

Integration tests check two or more units of code as a group. Unlike unit tests that check a unit of work in your code, integration tests analyze the integration of your code with other components within your application. They may fail due to a defect in your code even though the unit tests passed. For example, you may want to use the Relativity APIs to run NUnit integration tests against RelativityOne.

As a best practice, write integration tests to check the integration of your application with other modules, and to verify that your application interacts properly with Relativity. You can use integration tests to fully automate your functional tests, and eliminate any dependency on the UI.

An integration test has one or more of the following characteristics:

• Can’t be fully automated.
• Communicates directly with a database.
• Communicates across a network.
• Interacts with the file system.
• Requires special modifications to the environment to run it.
• Doesn't isolate dependencies by only testing the unit of code under test.
• Can’t run at the same time as any other unit tests.
• Must run in a specific order if it's part of other tests.
• Doesn't return the same result every time that you run it. For example, a unit test might run using random numbers and produce different results for each run.
• Tests more than a single logical concept.
• Doesn't run fast.

Continuous integration testing

To facilitate implementing and upgrading custom applications, consider writing unit and integration tests that you can easily push through a continuous integration (CI) pipeline. CI is an engineering practice that involves committing and merging code several times a day. This workflow ensures that you get rapid feedback on your code and catch defects quickly.

To use CI efficiently, implement a test for each code change that you push to your production environment. This practice helps you quickly identify the exact time of a failure and the change that caused it. As Relativity releases production code more frequently, using a CI workflow can simplify the process of deploying custom applications.

The Google Testing Blog illustrates the benefits of using CI to improve software quality with less risk. The Google code base undergoes over 20 code changes per minute, resulting in changes to 50% of its files every month. Google engineers develop and release each product from the head, relying on automated tests to verify the product behavior. The release frequency varies from multiple times per day to once every few weeks as determined by the product team. The shorter feedback cycles improve productivity by starting with changes being pushed to merge, and then adding them to staging without any manual intervention. They also help keep the focus on building new features rather than fixing defects.

Code coverage

Code coverage measures the test coverage and test quality that is used to check the functionality of specific features. Use these guidelines for code coverage:

• Provide 80% to 90% code coverage with unit tests for any new code.
• Aim for the maximum amount of coverage with the smallest number of quality unit tests. High code coverage doesn't necessarily mean that you have written quality tests.
• Consider writing golden flow integration tests for legacy code that doesn’t have many unit tests, because you may find it difficult to write unit tests for legacy code.

Testable code

To facilitate custom application development, write testable code that is iterative and simple to refactor. You can then easily create automated tests for it. Review the following guidelines for writing testable code:

• Move the business logic for your extensibility point code into a general class. For example, if you have an agent call called Manager, you might create a ManagerJob class for this purpose. This ManagerJob class has all of your agent business logic.
• Allow parameters to be mocked.
• Specify interfaces instead of object instances in method parameters.

  "New is glue" - In a strongly-typed language like C#, the new keyword is used to instantiate an object. For a flexible implementation of your code, introduce an interface to keep your code loosely-coupled, rather than bind your class to another one using the new keyword.

For example, the Relativity Manager-Worker Agent template demonstrates the manager and worker concepts used for Relativity agents. The manager agent creates a job record and the worker agent iterates through the job records in batches to make the application scalable and performant. For more information, see Relativity Manager-Worker Agent template.

Steps for writing unit tests

You can set up unit tests for your custom applications by using the steps outlined in this section. When writing an NUnit test, you can use attributes to identify different parts of it. For more information, see NUnit documentation on the NUnit website.

The sample code used here references the agent manager example in Testable code. Use the following steps to set up mocks for a test:

• Declare mocks - These mocks help create the data for the test:

  

• Instantiate the mocks - Instantiate the mocks on the interfaces. The System Under Test (SUT) object creates a new instance of ManagerJob class and takes the MockAgentHelper and MockSQLQueryHelper dependencies that are mocked:

  

• Pass data and call the Execute() method - Pass the start time, end time, current time, and expected result as different iterations of data. Stub out three methods in the SqlHelper to return the start time, end time, and current time, which are passed as parameters in the test. Call the Execute() method on the SUT, which is an object of type ManagerJob. The expected result passed in as a parameter equals result generated by calling the Execute() method on the SUT.

  

To simplify the process of creating test data, use the Relativity Test Helpers located in the Relativity repository on GitHub. This code sets up workspaces, users, groups, authentication providers, fields, and other test data, so that you can focus on the business logic of your code.

Testing Sandboxes for RelativityOne applications

If you develop custom applications for RelativityOne, you can obtain a sandbox that provides a real-time option to build and test customizations in a non-production setting. A RelativityOne subscription includes access to a sandbox, which is provided to the Relativity Admin Contact.

Use the sandbox only for functional testing, not for performance testing. Sandboxes do not support remote debugging. For more information about using Sandbox environments, see RelativityOne Sandbox.

Automated tests in RelativityOne Sandboxes

To write automated tests for RelativityOne and add custom application to it, we recommend that you enroll in the Regression Cycle for RelativityOne program by contacting the Developer Experience (DevEx) team.

This program offers the following assistance:

• A technical business analyst who works with you to to set up an automation strategy meeting within a month of enrollment.
• A test associate who works with you to create an automation testing plan and implement it. This plan describes how to write automated tests for your project. It explains how to create tests run against a developer test VM, and then how to run them in a RelativityOne sandbox.

Recommendations for application development

Review the following general recommendations for application development:

• Write code that is testable. Depending on your development goals, you can use the Relativity Agent Manager-Worker template to build your applications and write testable code. For more information, see the Relativity Manager-Worker Agent on Visual Studio Marketplace.

• Write unit tests and integration tests for all of your Relativity applications, especially those that require testing on a recurring regression cycle. Consider using these recommended tools for writing tests. For more information, see Testing custom applications.
  • Nunit framework for writing tests
  • Moq for writing unit tests
  • Relativity APIs for writing integration tests
• Engage with other Relativity developers by visiting the following sites:
  • Relativity Community - Review communications and other information posted and shared with customers and developers on this site.
  • Developer Group - Join and subscribe to receive Relativity platform news and updates communicated to the developer community through this site.
  • Relativity DevHelp Community - Visit the developer forum for Relativity. The following screen shot illustrates the DevHelp category used for posting questions to the community. For more information, Get started with the Relativity DevHelp Community.

    

Additional testing resources

Review the following list of additional testing resources to learn more about sample projects, testing, and Relativity templates.

Relativity repository on GitHub

The Relativity repository is an open source repository on GitHub, which contains samples projects that you can reference when building custom applications. It also includes helpers that you can use for creating an automation framework. The following list highlights sample projects and other information available in this repository:

• DbContext Helper - This helper accommodates the removal of the implementation of the IDBContext contract mentioned in this community post.
• Datagrid audit API samples - This project illustrates how you can convert an application so it consumes DataGrid, and how to use the APIs to test an application. If you are developing applications for RelativityOne, familiarize yourself with these APIs.
• Test data setup - The Relativity test helpers contain information about how to import data and documents into a workspace.

Additional NUnit testing information

You can find additional information about unit testing by reviewing the article The Constraint Assertion Model.

Visual Studio templates

To simplify your development process, use the Visual Studio templates available for Relativity. The Relativity Manager-Worker Agent template on Visual Studio Marketplace or on GitHub provides an example of the templates available for use. For more information, see Visual Studio templates.