Processing Filter Manager

Processing filters provide functionality to view and select processed documents before publishing them to Relativity. The Processing Filter Manager API exposes methods for programmatically creating, updating, and deleting filters. It supports applying these filters to data and retrieving the filtered data. Additionally, this service exposes helper methods for retrieving filters associated with a data source or available in a specific workspace. For more general information, see Filtering data.

As a sample use case, you may want to implement a simplified processing workflow by programmatically adding complex criteria to filters applied to processing jobs in Relativity. This approach eliminates the need to manually enter filter criteria through the UI, which may be time consuming and error prone. Similarly, you can also use the Processing Filter Manager service to modify the settings on these or other complex filters implemented in your environment.

The Processing Filter Manager service also supports the same functionality through REST. For more information, see Processing Filter Manager (REST).

See the topic Get started with the Processing API for general guidance on the Processing API, and links to documentation for the other interfaces and services in this API.

Fundamentals for the Processing Filter Manager API

Review the following information to learn about the methods, classes, and enumerations used by the Processing Filter Manager service.

Methods

The Relativity.Processing.Services namespace contains the IProcessingFilterManager interface that exposes the following methods:

Classes and enumerations

Relativity.Processing.Services.Interfaces.DTOs namespace contains the following classes and enumerations used by the Processing Filter Manager API. The following list highlights several of the classes and enumerations in this namespace. For a complete list, see Processing API in the Class library reference.

Guidelines for the Processing Filter Manager API

Review the following guidelines for working with the Processing Filter Manager API.

Access rights

Verify that you have access to the workspace where you want make calls to the Processing Filter Manager service.

Helper methods

Use the GetFilterByDataSourceAsync() or GetFiltersAsync() helper methods to get the name, identifier, and type for a filter. See Retrieve processing filters or Retrieve processing filters for a data source.

Filter expressions

To create or update a processing filter, use the expression types, operators, and properties supported by the Processing Filter Manager service for this purpose. For more information about expression classes and enumerations, see the Relativity.Processing.Services.Interfaces.DTOs namespace for the Processing API.

Expression types

You can find examples of expression types in Create a processing filter.

The following types of expressions are supported:

  • Composite expression - several shorter expressions that are combined by using the AND and OR operators. To create this expression type, instantiate a ProcessingFilterCompositeExpression object by setting the following properties:
    • Expressions - an array of expressions to be combined. See IProcessingFilterExpressionModel interface in the Processing API.
    • Operator - an operator used to combine expressions. See Operators.
  • Conditional expression - expressions that use Boolean conditions for evaluation. To create this expression type, instantiate a ProcessingFilterConditionalExpression object by setting the following properties:
    • Property - a property used for filtering. See Properties.
    • Constraint - an operator used to evaluate a property against a specified value. See Operators.
    • Value - a string compared to a property.
  • Empty expression - an empty expression object used to create expressions. See the ProcessingFilterEmptyExpression class in the Processing API.

Operators

The Relativity.Processing.Services.Interfaces.DTOs namespace includes the following enumerations for operators supported in expressions:

  • Constraint operators - The ProcessingFilterConstraint enumeration supports the following operators: Is, IsNot, IsIn, and IsNotIn.
  • Composite operators - The ProcessingFilterOperator enumeration supports the following operators: AND and OR.

Properties

The Property enumeration provides a list of properties supported for use in expressions.

Create a processing filter

To create or update a processing filter, you need to create an expression object that is referenced by the request object passed to the CreateExpressionFilterAsync() method.

The following code samples illustrate how to create a request object for each type of expression. For more information, see Filter expressions.

  • Method

    • CreateExpressionFilterAsync

  • Parameters
    • <int>workspaceID: The workspace artifact identifier.
    • CreateProcessingExpressionFilterRequest object
      • <string> FilterName: Name of the expression filter.
      • <List<int>>DataSourceIDs: The DataSourceIDs of the jobs where the filters needs to be created on.
      • <string>Expression: Specifies the expression use to create the expression filter.
      • <bool>IncludeFamily: Specifies if the result of the filter should contain the family.
  • Returns
    • ProcessingFilter object
      • <long>FilterID: The filter Id for the filter
      • <string> FilterName: The filter name for the filter.
      • <string>Type: The filter type for the filter

Apply a processing filter

Use the ApplyFilterAsync() method to apply a processing filter to a data set. The following code sample illustrates how to instantiate a ApplyProcessingFilterRequest object containing the Artifact ID of the workspace, filter ID, and a job priority. The request object is then passed to the ApplyFilterAsync() method, which returns a job ID.

  • Method
    • ApplyFilterAsync
  • Parameters
    • <int>workspaceID: The workspace artifact identifier.
    • <long> filterID: Filter ID of the filter to apply.

      Note: You can obtain the identifier for a filter by calling the GetFilterByDataSourceAsync() or GetFiltersAsync() methods. See Retrieve processing filters or Retrieve processing filters for a data source.

    • ApplyProcessingFilterRequest object
      • <int>Priority: priority of the job
  • Returns
    • <long>: the jobId to which the filter was applied

Update a processing filter

To update a processing filter, instantiate an UpdateProcessingExpressionFilterRequest object, and pass it to the UpdateExpressionFilterAsync() method, which returns a a ProcessingFilter object containing the name, identifier, and type of the filter. For more information, see Filter expressions.

  • Method
    • UpdateExpressionFilterAsync: Updates an expression filter
  • Parameters
    • <int>workspaceID: The workspace artifact identifier.
    • <long> filterID: Filter ID of the filter to update.

      • Note: You can obtain the identifier for a filter by calling the GetFilterByDataSourceAsync() or GetFiltersAsync() methods. See Retrieve processing filters or Retrieve processing filters for a data source.

    • UpdateProcessingExpressionFilterRequest object
      • <string> FilterName: Name of the expression filter.
      • <List<int>>DataSourceIDs: The DataSourceIDs of the jobs where the filters needs to be created on.
      • <string>Expression: Specifies the expression use to create the expression filter.
      • <bool>IncludeFamily: Specifies if the result of the filter should contain the family
  • Returns
    • ProcessingFilter object

Delete a processing filter

To remove a processing filter, instantiate a DeleteFilterRequest object containing the Artifact ID of the workspace, and the filter ID, and then pass this request to the DeleteFilterAsync() method.

Retrieve discovered documents

After you apply a processing filter, you can use the GetDiscoveredDocumentsAsync() method to retrieve discovered documents.

  • Method
    • GetDiscoveredDocumentsAsync: Get discovered documents
  • Parameters
    • <int>workspaceID: The workspace artifact identifier.
    • GetDiscoveredDocumentsRequest object
      • <int> StartingPointOfResult: Starting point to return rows from a result set.
      • <int>NumberOfResults: Count of rows after startingpoint to get in result set.
      • <string>Expression: Specifies the expression use to create the expression filter.
      • <int>AuditedSourceID: The view or saved filter ID for auditing.
      • <List<SortingOption>> SortingOptions: A list of sorting priority to which get applied upon result retrieval.
        • Property: enum of properties to sort on
        • Ordering: enum of ordering directions, Ascending or Descending
      • <bool>ExcludeTotalCount: Whether or not to calculate the total size of the discovered documents returned
  • Returns
    • ProcessingFilterData object

Retrieve discovered document IDs

After you apply a processing filter, you can use the GetDiscoveredDocumentIdsAsync() method to retrieve the Ids of discovered documents.

  • Method
    • GetDiscoveredDocumentIdsAsync: Get discovered document IDs
  • Parameters
    • <int>workspaceID: The workspace artifact identifier.
    • GetDiscoveredDocumentsRequest object
      • <int> StartingPointOfResult: Starting point to return rows from a result set.
      • <int>NumberOfResults: Count of rows after startingpoint to get in result set.
      • <string>Expression: Specifies the expression use to create the expression filter.
      • <int>AuditedSourceID: The view or saved filter ID for auditing.
      • <List<SortingOption>> SortingOptions: A list of sorting priority to which get applied upon result retrieval.
        • Property: enum of properties to sort on
        • Ordering: enum of ordering directions, Ascending or Descending
      • <bool>ExcludeTotalCount: Whether or not to calculate the total size of the discovered documents returned
  • Returns
    • ProcessingFileIdResults object
      • <long>FilterID
      • <int>TotalCount: Total result count for a filter.
      • <List<long>>Results: File ids (from the Invariant Matter table)

Retrieve Master Dedupe Filters and Filter Expressions

  • Method
    • GetMasterDedupeFiltersAsync: Gets Master Dedupe Filters and Filter Expressions
  • Parameters
    • <int>workspaceID: The workspace artifact identifier.
  • Returns
    • List of MasterDedupeSavedFilter objects
      • <string>Name
      • ProcessingFilterMasterDedupeExpression object
        • Type: “MasterDedupeExpression”,
        • FilterId: Identifier for the filter in the expression.
        • FilterConstraint: enum of type ProcessingDedupeFilterConstraint, options are Duplicate or MasterOrUnique

Retrieve the list of discovered files from invariant as a CSV

  • Method
    • PrepareDiscoveredDocumentListAsync: Creates a request to get the list of discovered files from invariant as a CSV
  • Parameters
    • <int>workspaceID: The workspace artifact identifier.
    • • GetDiscoveredDocumentListRequest object
      • <string>Expression
      • <long[]>FileIDs
      • <List<ExportColumn>> Columns: Column to be included in an exported CSV file
        • <enum>Property: The property value to include in this column of exported CSV data
        • <string>Header: The column header to include in the CSV for this property
      • <List<SortingOption>> SortingOptions: A list of sorting priority to which get applied upon result retrieval.
        • Property: enum of properties to sort on
        • Ordering: enum of ordering directions, Ascending or Descending
  • Returns
    • PrepareDiscoveredDocumentListResponse object
      • <Guid>Guid - The Guid used to retrieve the Discovered File List

Retrieve filtered data

After you apply a processing filter, you can use the GetFilterResultAsync() method to retrieve the filtered data.

  • Method
    • GetFilterResultAsync: Get Filter Result
  • Parameters
    • <int>workspaceID: The workspace artifact identifier.
    • <long>filterID: Filter ID from which the result is retrieved.

      • Note: You can obtain the identifier for a filter by calling the GetFilterByDataSourceAsync() or GetFiltersAsync() methods. See Retrieve processing filters or Retrieve processing filters for a data source.

    • <int>skip: Starting point to return rows from a result set.
    • <int>top: Count of rows after starting point to get in result set
  • Returns
    • ProcessingFilterData object
      • <long>FilterID.
      • <int>TotalCount: Total result count for a filter.
      • Results – list of ProcessingFilterResult objects with multiple properties including ContainerName, CustodianName, DiscoverGroupId, and more

Retrieve processing filters

You can retrieve all available processing filters for a specific workspace. The following code sample illustrates how to instantiate a GetProcessingFiltersRequest instance with the Artifact ID of a specific workspace. It shows how to call the GetFiltersAsync() method by passing the request object to it. This method returns a ProcessingFilter object, which has properties for the name, identifier, and type of the filter.

  • Method
    • GetFiltersAsync: Get Filters from invariant
  • Parameters
    • <int>workspaceID: The workspace artifact identifier
  • Returns

    • List of ProcessingFilter objects

Retrieve processing filters for a data source

You can retrieve processing filters for a specific data source. This method returns a list of ProcessingFilter object, which has properties for the name, identifier, and type of the filter.

  • Method
    • GetFiltersByDataSourceAsync: Get Filter Result
  • Parameters
    • <int>workspaceID : The workspace artifact identifier.
    • <int> dataSourceID: The Artifact ID of the processing data source to get filters from
  • Returns
    • List of ProcessingFilter objects

Pivot on discovered documents

After you apply a processing filter, you can use the PivotOnDiscoveredDocumentsAsync() method to retrieve filtered discovered documents on pivot.

  • Method
    • PivotOnDiscoveredDocumentsAsync: Get discovered documents with pivot on
  • Parameters
    • <int>workspaceID: The workspace artifact identifier.
    • GetDiscoveredDocumentsWithPivotOnRequest object
      • <string>Expression
      • PivotOnOption object: Group by options use to set group by conditions.
        • <Property enum> GroupByProperty: Name of the property to perform group by on.
        • <Ordering enum> GroupByOrdering: Sort order of group by condition.
        • <int>GroupByCount: Group by count used to select number of top or bottom rows of the condition. Default value 0
        • <Property enum> PivotOnProperty: Name of the property to perform pivot.
        • <Ordering enum> PivotOnOrdering: Sort order of pivot on ordering condition.
        • <int>PivotOnCount: Pivot On count used to select number of top or bottom rows of the condition. Default value 0
  • Returns
    • List of GetDiscoveredDocumentsWithPivotOnResponse objects
      • <string>GroupByIdentifier: Value of the property that is group by.
      • <string>PivotOnIdentifier: Value of the property that is pivot on identifier.
      • <int>ResultCount: Count of each property

Download a csv listing of discovered files

  • Method
    • DownloadDiscoveredDocumentListAsync: Downloads a csv listing of discovered files based on a request submitted earlier
  • Parameters
    • <int>workspaceID: Id of the workspace for which the request was defined.
    • <Guid>requestGuid: Guid of a request as defined by PrepareDiscoveredDocumentListAsync
  • Returns
    • IKeplerStream object for downloaded file(s)