Processing Filter Manager (REST) v2

Processing filters provide functionality to view and select processed documents before publishing them to Relativity. The Processing Filter Manager service exposes endpoints for programmatically creating, updating, and deleting filters. It supports applying these filters to data and retrieving the filtered data. Additionally, this service exposes helper endpoints for retrieving filters associated with a data source or available in a specific workspace. For more general information, see Filtering data.

As a sample use case, you may want to implement a simplified processing workflow by programmatically adding complex criteria to filters applied to processing jobs in Relativity. This approach eliminates the need to manually enter filter criteria through the UI, which may be time consuming and error prone. Similarly, you can also use the Processing Filter Manager service to modify the settings on these or other complex filters implemented in your environment.

The Processing Filter Manager service also supports the same functionality through .NET. For more information, see Processing Filter Manager (.NET) v2.

Client code sample

To use the Processing Filter Manager service, send an HTTP request to the target URL. Use PostAsync() for POST requests, GetAsync() for GET requests, and PutAsync() for PUT requests. You specify query conditions in the body of the request. See the base URL for this service:

Copy

1
<host>/Relativity.Rest/API/relativity-processing/v2/workspaces/{your workspace id}/

Guidelines for the Processing Filter Manager service

Review the following guidelines for working with this service.

Access rights

Verify that you have access to the workspace where you want make calls to the Processing Filter Manager service.

Filter expressions

To create or update a processing filter, use the expression types, operators, and properties supported by the Processing Filter Manager service for this purpose. For more information on objects used to build expressions, see Processing Filter Manager (.NET) v2.

Expression types

The following types of expressions are supported:

Composite expression - several shorter expressions that are combined by using the AND and OR operators. See Operators.
Conditional expression - expressions that use Boolean conditions for evaluation. See Operators.
Empty expression - an empty expression object used to create expressions.

Operators

The following operators are supported in expressions:

Constraint operators - Is, IsNot, IsIn, and IsNotIn.
Composite operators - AND and OR.

Properties

Click the following drop-down link to view a list of properties supported for use in expressions.

ContainerExtension
ContainerFileId
ContainerName
DocumentError
FileExtension
FileName
FileType
IsDeleted
OriginalPath
ParentDocumentId
LogicalID
ProcessingFolderPath
SourcePath
StorageError
StorageId
TextExtractionMethod
VirtualPath

Apply a processing filter

To apply a filter to the documents in a data set, send a POST request to the following URL. The value for {your filter Artifact ID} is the identifier assigned to a processing filter. You obtain the identifier for a filter by calling the GetFilterByDataSourceAsync or GetFiltersAsync endpoints. See Processing Filter Manager (REST) v2 or Processing Filter Manager (REST) v2.

Copy

1
<host>/Relativity.Rest/API/relativity-processing/v2/workspaces/{your workspace id}/filters/{your filter Artifact ID}/apply

The body of the request must contain the following fields unless specifically identified as optional:

request - represents a request to apply a specific filter to a data set:
- Priority - an integer value indicating the precedence of the filtering job. Jobs with the lowest values are processed first.

Copy

1
2
3
4
5
6
{
    "request":
    {
        "Priority" : 100
    }
}

When the request is successful, the response contains a value of type long, which is the ID for the job that was executed.

Copy

Retrieve discovered documents

After you apply a processing filter, you can use the documents endpoint to retrieve the filtered discovered documents. Send a POST request to the following URL:

Copy

1
<host>/Relativity.Rest/API/relativity-processing/v2/workspaces/{your workspace id}/filters/documents

The body of the request must contain the following fields unless specifically identified as optional:

request - represents a request to retrieve filtered discovered documents. It contains the following fields:

StartingPointOfResult - the index of the first row in the result set to return. The result set uses a zero-based index.
NumberOfResults - the total number of rows in the result set to return. The value in the StartingPointOfResult field indicates the first row of the result set to return.
Expression - the criteria used to select a subset of documents. The documents are filtered by this criteria. In this field, you set the type of expression, as indicated by \"Type\" : \"ConditionalExpression\". See Filter expressions.
Click the following drop-down link to view a table of constraint values supported based on data type.

SortingOptions - the priority of sorting applied after retrieval of discovered documents. The array contains the following fields:
- Property - represents a sortable property of discovered documents. Available property values are listed in the constraint values table.
- Order - the method of sorting applied to discovered documents. The value can be Ascending or Descending.
IncludeDuplicateHash - Include the calculated processing duplicate hash (PDH) in the response, typically used for deduplication. The value is true/false. If set to false, the PDH will not be included in the response data.
Including the PDH may impact performance. Set this value to false, unless you specifically need to retrieve the ProcessingDuplicateHash value.

Copy

1
2
3
4
5
6
7
8
9
10
{
    "request":
    {
        "StartingPointOfResult": 0,
        "NumberOfResults": 10, "Expression":"{\"Type\":\"CompositeExpression\",\"Expressions\":[{\"Type\":\"ConditionalExpression\",\"Property\":\"IsDeleted\",\"Constraint\":\"Is\",\"Value\":false},{\"Type\":\"ConditionalExpression\",\"Property\":\"LogicalID\",\"Constraint\":\"Is\",\"Value\":\"123\"}],\"Operator\":\"And\"}",
        "ExcludeTotalCount":false,
        "SortingOptions":[],
        "IncludeDuplicateHash":true
    }
}

View the JSON response containing the filtered data

Copy

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
{
    "FilterId": -1,
    "TotalCount": 11,
    "Results": [
        {
            "ContainerExtension": "",
            "ContainerID": 0,
            "ContainerName": "",
            "CustodianArtifactId": 1040723,
            "CustodianName": "Shah, Yash",
            "DataSourceArtifactId": 1040724,
            "DataSourceName": "Shah, Yash - 1040724",
            "DiscoverGroupId": 2,
            "ErrorMessage": "",
            "ExtractedTextLocation": "",
            "FileExtension": "BIN",
            "FileName": "1.txt",
            "FileSize": 0,
            "FileType": "Empty zero byte file",
            "FolderPath": "",
            "ImportSource": "\\\\P-DV-VM-EVE7CUT\\processingsource\\1.txt",
            "IngestionError": "",
            "IsContainer": false,
            "IsEmbedded": false,
            "IsPublished": true,
            "LogicalPath": "",
            "MD5Hash": "D41D8CD98F00B204E9800998ECF8427E",
            "OfficeEmbeddedItems": false,
            "OriginalPath": "",
            "ParentDocumentId": 0,
            "LogicalID": "97a02181f1ff22286d552ab844abb715f79b5361aa013a0c0f0b7caa42233d12",
            "SHA1Hash": "DA39A3EE5E6B4B0D3255BFEF95601890AFD80709",
            "SHA256Hash": "E3B0C44298FC1C149AFBF4C8996FB92427AE41E4649B934CA495991B7852B855",
            "TextExtractionMethod": "Native",
            "Unprocessable": false,
            "VirtualPath": "10TextFiles_LZMA2Compression.7z\\10 Text Files",
            "SenderDomain": "[Blank]",
            "SortDate": "2001-10-26T13:18:34",
            "ErrorStatus": "",
            "DedupeStatus": "Unique",
            "StorageId": "17",
            "ProcessingDuplicateHash": "ABC1234..."
        },
        {
            "ContainerExtension": "7Z",
            "ContainerID": 2,
            "ContainerName": "10TextFiles_LZMA2Compression.7z",
            "CustodianArtifactId": 1040727,
            "CustodianName": "Regan, Robert",
            "DataSourceArtifactId": 1040728,
            "DataSourceName": "Regan, Robert - 1040728",
            "DiscoverGroupId": 9,
            "ErrorMessage": "",
            "ExtractedTextLocation": "\\\\P-DV-VM-EVE7CUT\\fileshare\\EDDS1017804\\Processing\\1040728\\INV1017804\\INTERMEDIATE\\0\\3.TXT",
            "FileExtension": "TXT",
            "FileName": "TextFile1.txt",
            "FileSize": 6497,
            "FileType": "Text - Western European",
            "FolderPath": "10TextFiles_LZMA2Compression.7z\\10 Text Files",
            "ImportSource": "",
            "IngestionError": "",
            "IsContainer": false,
            "IsEmbedded": false,
            "IsPublished": true,
            "LogicalPath": "10TextFiles_LZMA2Compression.7z\\10 Text Files",
            "MD5Hash": "B98A72D799731A07334D2EFF845F0D93",
            "OfficeEmbeddedItems": false,
            "OriginalPath": "",
            "ParentDocumentId": 2,
            "LogicalID": "43a02181f6ff22286d552ab844abb715f79b5361aa013a0c0f0b7caa42231d16",
            "SHA1Hash": "ACC7C604E020001C6F726CC19F8577891AF88CE4",
            "SHA256Hash": "75265C2E912D0882FB4BB7ABF60EA5DE60BB13568A2C3A4404CDC7C5B888C0A6",
            "TextExtractionMethod": "Native",
            "Unprocessable": false,
            "VirtualPath": "10TextFiles_LZMA2Compression.7z\\10 Text Files",
            "SenderDomain": "[Blank]",
            "SortDate": "2001-10-26T13:18:34",
            "ErrorStatus": "",
            "DedupeStatus": "Unique",
            "StorageId": "17",
            "ProcessingDuplicateHash": "ABC1234..."
        }
    ]
}

Retrieve filtered data

After you apply a processing filter, you can send a GET request to the following URL to retrieve the filtered data. The value for {your filter Artifact ID} is the identifier assigned to a processing filter. You obtain the identifier for a filter by calling the GetFilterByDataSourceAsync or GetFiltersAsync endpoints. See Processing Filter Manager (REST) v2 or Processing Filter Manager (REST) v2.

Copy

1
<host>/Relativity.Rest/API/relativity-processing/v2/workspaces/{your workspace id}/filters/results/{your filter Artifact ID}?skip=0&top=10

View the JSON response containing the filtered data

Copy

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
{
    "FilterId": 2,
    "TotalCount": 10,
    "Results": [
        {
            "ContainerExtension": "7Z",
            "ContainerID": 2,
            "ContainerName": "10TextFiles_LZMA2Compression.7z",
            "CustodianArtifactId": 1040727,
            "CustodianName": "Regan, Robert",
            "DataSourceArtifactId": 1040728,
            "DataSourceName": "Regan, Robert - 1040728",
            "DiscoverGroupId": 9,
            "DocumentErrorMessage": "",
            "ExtractedTextLocation": "\\\\P-DV-VM-EVE7CUT\\fileshare\\EDDS1017804\\Processing\\1040728\\INV1017804\\INTERMEDIATE\\0\\4.TXT",
            "FileExtension": "TXT",
            "FileName": "TextFile10.txt",
            "FileSize": 9113,
            "FileType": "Text - Western European",
            "FolderPath": "10TextFiles_LZMA2Compression.7z\\10 Text Files",
            "ImportSource": "",
            "IngestionError": "",
            "IsContainer": false,
            "IsEmbedded": false,
            "LogicalPath": "10TextFiles_LZMA2Compression.7z\\10 Text Files",
            "MD5Hash": "C61FAF7FE97EC3BE0A5B6FC50570CE8F",
            "OfficeEmbeddedItems": false,
            "OriginalPath": "",
            "ParentDocumentId": 2,
            "LogicalID": "36a02181e2ff22286d552ab844abb715f79b5361aa013a0c0f0b7caa42235b25",
            "SHA1Hash": "1E4E46F2D0D1A114F1F1E9FADC298DB5B1103B23",
            "SHA256Hash": "3D8B8B6B053B7B13F04A82E46A86CD72C9E6CBDE2D1519059DCD3D6BCB01FDDC",
            "TextExtractionMethod": "Native",
            "Unprocessable": false,
            "VirtualPath": "10TextFiles_LZMA2Compression.7z\\10 Text Files"
        },
        {
            "ContainerExtension": "7Z",
            "ContainerID": 2,
            "ContainerName": "10TextFiles_LZMA2Compression.7z",
            "CustodianArtifactId": 1040727,
            "CustodianName": "Regan, Robert",
            "DataSourceArtifactId": 1040728,
            "DataSourceName": "Regan, Robert - 1040728",
            "DiscoverGroupId": 9,
            "DocumentErrorMessage": "",
            "ExtractedTextLocation": "\\\\P-DV-VM-EVE7CUT\\fileshare\\EDDS1017804\\Processing\\1040728\\INV1017804\\INTERMEDIATE\\0\\5.TXT",
            "FileExtension": "TXT",
            "FileName": "TextFile2.txt",
            "FileSize": 4331,
            "FileType": "Text - Western European",
            "FolderPath": "10TextFiles_LZMA2Compression.7z\\10 Text Files",
            "ImportSource": "",
            "IngestionError": "",
            "IsContainer": false,
            "IsEmbedded": false,
            "LogicalPath": "10TextFiles_LZMA2Compression.7z\\10 Text Files",
            "MD5Hash": "BACD97CA8B90595FAB6350DFB7E8165A",
            "OfficeEmbeddedItems": false,
            "OriginalPath": "",
            "ParentDocumentId": 2,
            "LogicalID": "51b02181f6ff22286d552ab844abb715f79b5361aa013a0c0f0b7caa42232d24",
            "SHA1Hash": "DC9B65C5AEC375AFCCB2EF41653E084B62AFB6DA",
            "SHA256Hash": "4DE4A5ED7979900CBA63F50868AE2228B3036C54D7BE61EAB7E34FC538FCC945",
            "TextExtractionMethod": "Native",
            "Unprocessable": false,
            "VirtualPath": "10TextFiles_LZMA2Compression.7z\\10 Text Files"
        }
    ]
}

Pivot on discovered documents

After you apply a processing filter, you can use the pivot-discovered-documents endpoint to retrieve filtered discovered documents on pivot. Send a POST request to the following URL:

Copy

1
2
<host>/Relativity.Rest/API/relativity-processing/v2/workspaces/{your workspace id}/filters/pivot-discovered-documents

The body of the request must contain the following fields unless specifically identified as optional:

request - represents a request to retrieve processing filters for a data source. It contains the following fields:
- Expression - the criteria used to select a subset of documents. The documents are filtered by this criteria. In this field, you set the type of expression, as indicated by \"Type\" : \"ConditionalExpression\". See Filter expressions. Click the following drop-down link to view a table of constraint values supported based on data type.

PivotOnOption - the total number of rows in the result set to return. The value in the StartingPointOfResult field indicates the first row of the result set to return.
- GroupByProperty - the name of the property to perform group by on.
- GroupByCount - the integer value used to select the number of top or bottom rows to group by. The default value is 0.
- PivotOnProperty - (optional) the string representation of the property to perform pivot on.
- PivotOnCount - (optional) the number of pivot results. The default value is 0.

Sample PivotOn request with conditional expression:

Copy

1
2
3
4
5
6
7
8
9
10
11
12
13
{
    "request":
    {
"Expression":"{\"Type\":\"CompositeExpression\",\"Expressions\":[{\"Type\":\"ConditionalExpression\",\"Property\":\"IsDeleted\",\"Constraint\":\"Is\",\"Value\":false},{\"Type\":\"ConditionalExpression\",\"Property\":\"LogicalID\",\"Constraint\":\"Is\",\"Value\":\"123\"}],\"Operator\":\"And\"}",
        "PivotOnOption":
        {
            "GroupByProperty": "ContainerID",
            "GroupByCount": "0",
            "PivotOnProperty": "CustodianArtifactID",
            "PivotOnCount": "0"
        }    
    }
}

When the request is successful, the response will return a List of GetDiscoveredDocumentsWithPivotOnResponse objects.

The response contains the following fields:

GroupByIdentifier - the value of the property that is being grouped by.
PivotOnIdentifier - the value of the property that is being pivoted on.
ResultCount - the total of each property.

View the JSON response for a request without PivotOn fields populated

Copy

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
[
    {
        "GroupByIdentifier": "Q1_Result",
        "PivotOnIdentifier": "",
        "ResultCount": 2
    },
    {
        "PropertyValue": "Q2_Result",
        "PivotOnIdentifier": "",
        "ResultCount": 4
    },
    {
        "PropertyValue": "Q3_Result",
        "PivotOnIdentifier": "",
        "ResultCount": 3
    }
]

View the JSON response for a request with PivotOn fields populated

Copy

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
[
    {
        "GroupByIdentifier": "Q1_Result",
        "PivotOnIdentifier": "1",
        "ResultCount": 2
    },
    {
        "PropertyValue": "Q2_Result",
        "PivotOnIdentifier": "2",
        "ResultCount": 4
    },
    {
        "PropertyValue": "Q3_Result",
        "PivotOnIdentifier": "3",
        "ResultCount": 3
    }
]

Get discovered documents IDs

You can retrieve discovered document IDs by sending a POST request to the following URL:

Copy

1
<host>/Relativity.Rest/API/relativity-processing/v2/workspaces/{your workspace id}/filters/document-ids

Request Body Sample

Copy

1
2
3
4
5
6
7
8
9
{
    "request":
    {
        "StartingPointOfResult": 0,
        "NumberOfResults": 10, "Expression":"{\"Type\":\"CompositeExpression\",\"Expressions\":[{\"Type\":\"ConditionalExpression\",\"Property\":\"IsDeleted\",\"Constraint\":\"Is\",\"Value\":false},{\"Type\":\"ConditionalExpression\",\"Property\":\"LogicalID\",\"Constraint\":\"Is\",\"Value\":\"123\"}],\"Operator\":\"And\"}",
        "ExcludeTotalCount":false,
        "SortingOptions":[]
    }
}

Download discovered documents list

Download discovered documents list by sending a POST request to the following URL:

Copy

1
<host>/Relativity.Rest/API/relativity-processing/v2/workspaces/{your workspace id}/filters/csvs

Request Body Sample

Copy

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
{
  "request": {
    "Expression": "{\"Type\":\"ConditionalExpression\",\"Property\":\"IsDeleted\",\"Constraint\":\"Is\",\"Value\":false}",
    "SortingOptions": [],
    "Columns": [
      {
        "Header": "File ID",
        "Property": 1
      },
      {
        "Header": "File Name",
        "Property": 9
      },
      {
        "Header": "File Type",
        "Property": 11
      }
    ],
    "LogicalIDs": []
  }
}

The LogicalIDs array takes precedence over Expression. If both are specified, then LogicalIDs array is used.

Download NISTed documents list for a processing set

Download a NISTed documents list for a specific processing set by sending a GET request to the following URL:

Copy

1
<host>Relativity.Rest/API/relativity-processing/v2/workspaces/{your workspace id}/filters/get-nisted-documents?processingSetId={processing set id}

When the request is successful, this downloads a CSV file containing data for each NISTed file in the specified processing set.

The CSV file contains the following for each NISTed file:

FileName—the file name of the file that was deNISTed.
NistFileName—the name of NIST file that the native was deNISTed against. This name will match the FileName on the NIST table in the Invariant database.
FileSize—the size of the native file that was deNISTed. This should match the filesize of the file in the NIST table of the Invariant database.
SHA1—the SHA1 hash of the native file that was deNISTed. This should match the SHA1 hash of the file in the NIST table of the Invariant database.

If the NISTed file has a parent, it also contains:

ParentFileName—the filename of the parent file that was deNISTed.
ParentLevel—the depth of the native file within nested containers at the time of deNISTing. A value of 1 represents a file directly contained within a single parent container, with higher values indicating deeper levels of nesting.
ParentVirtualPath—the virtual path showing the hierarchical location of the native file within its parent structure.
ParentContainerName—the name of the container file (such as a ZIP, PST, or EML) that directly contained the native file when it was deNISTed.
ParentSourcePath—the original source path of the parent container from which the native file was extracted. This reflects the container’s location at the time of processing.

Path information is not available for any NISTed files that do not have a parent.