Structured Analytics Manager (REST)

The Structured Analytics API provides an HTTP service, which you can use to automate operations on structured analytics sets through Representational State Transfer (REST). The REST endpoints expose functionality similar to that available through the .NET interface for the Structured Analytics API. You can use these endpoints on the Structured Analytics Manager service to run an analysis of a structured analytics set, check the status of the analysis, retrieve document and other errors, and perform additional tasks. For general information about REST, see the REST API.

You can implement custom workflows for programmatically running structured analytic sets using the REST endpoints. For example, you can automate an analysis of a structured analytics set after performing only minor setup tasks in the Relativity UI. You could also implement a custom page that uses these endpoints to display the status of an analysis or errors that may have occurred.

The Structured Analytics API supports the use of progress indicators and cancellation tokens, when you use it through .NET. The service doesn't support these features through REST. For more information, see Structured Analytics (.NET).

Note: You must install the Analytics application in your Relativity environment to automate workflows with the Structured Analytics Manager service. For more information, see Upgrading or installing your Analytics server on the Relativity Documentation site.

This page includes the following information:

See this related page:

Guidelines for the Structured Analytics Manager service

Use the following guidelines when working with the Structured Analytics Manager service:

  • Execute only valid operations for the state of a structured analytics set - Only certain operations are valid for the current state of your structured analytics set. See the following examples:
    • A call to the CancelAsync endpoint is only valid when an analysis is currently running.
    • A call to the RetryErrorsAsync endpoint is only valid when a structured analytics set is in an errored state.

    To get a list of valid operations, make a call with the GetValidTasksAsync() endpoint. The ValidTaskResult object returned from this call contains a valid list of operations. If you attempt to execute an invalid operation, you receive 400 status response.

  • Monitor the analysis progress - After making a successful call to the RunAsync endpoint, make calls to the GetStatusAsync endpoint to monitor the progress of your analysis.
  • Use Structured Analytics Set RDOs - The Structured Analytic Sets API doesn't expose standard CRUD operations for Structured Analytics Set RDO objects. Use the standard Relativity REST API to create Structured Analytics Set RDOs. For REST API information, see Relativity Dynamic Object (RDO), and for the Relativity Services API, see RDO.

Client code sample

To use the Structured Analytics Manager service, send an HTTP request that makes a POST method call. See the following base URL for this service:

Copy
<host>/Relativity.REST/api/Relativity.StructuredAnalytics.Services.Interfaces.StructuredAnalytics.IStructuredAnalyticsModule/Structured%20Analytics%20Manager/

You can use the following .NET code as a sample REST client for performing any of the operations available in the Structured Analytics Manager service. This code sample illustrates how to perform the following tasks:

  • Instantiate an HttpClient object for sending requests and responses using the URL for the Structured Analytics Manager service.
  • Set the required headers for the request.
  • Initialize the parameters for the workspace and structured analytics set.
  • Set the sasManagerRunUrl variable to the URL for the RunAsync operation.
  • Use the PostAsync() method to send a post request.
  • Return the results of the request and parse it.
Copy
public bool ExecuteRunAsync(int workspaceId, int sasArtifactId, bool analyzeAllDocs, bool repopulateAllDocuments)
{
    using (var httpClient = new System.Net.Http.HttpClient())
    {
 
        httpClient.BaseAddress = new Uri("http://localhost/Relativity.REST/API/");
        httpClient.DefaultRequestHeaders.Add("Accept", "application/json");
        httpClient.DefaultRequestHeaders.Add("X-CSFRF-Header", string.Empty);
        httpClient.DefaultRequestHeaders.Add("Authorization", "Basic bXkudXNlckBrY3VyYS5jb206Q250VGNoVGhzMTIzNCE=");
 
        //Initialize the parameters for the workspace and structured analytics set.
        var parameters = new
        {
            workspaceId = workspaceId,
            sasArtifactId = sasArtifactId,
            settings = new
                {
                    AnalyzeAll = analyzeAllDocs,
                    PopulateAll = repopulateAllDocuments
                }
        };
 
        System.Net.Http.StringContent parametersJson = new System.Net.Http.StringContent(Newtonsoft.Json.JsonConvert.SerializeObject(parameters), System.Text.Encoding.UTF8, "application/json");
 
        //Define the REST endpoint called RunAsync on the Structured Analytics Manager service.
        String sasManagerRunUrl = "Relativity.StructuredAnalytics.Services.Interfaces.StructuredAnalytics.IStructuredAnalyticsModule/Structured Analytics Manager/RunAsync";
 
        //Make the HTTP request against the RunAsync endpoint of the Structured Analytics Manager service.
        System.Net.Http.HttpResponseMessage response = httpClient.PostAsync(sasManagerRunUrl, bool).Result;
 
        bool success = response.StatusCode == System.Net.HttpStatusCode.OK;
        String result = response.Content.ReadAsStringAsync().Result;
 
        if (success)
        {
            Newtonsoft.Json.Linq.JObject results = Newtonsoft.Json.Linq.JObject.Parse(result);

            if(results.Value&lt;bool&gt;("IsSuccess") == true)
            {
                return true;
            }
            else
            {
                throw new Exception($"Call to run Analysis on SAS {sasArtifactId} on workspace {workspaceId} failed.  Error message: {results.Value&lt;string&gt;("Message")}");
            }
        } else {
            return false;
        }
    }
}

Generate result fields before running an analysis

You can optionally call the RunAnalysisPreparationAsync endpoint when you want to generate fields for storing results before you run an analysis on a new structured analytics set. For example, you might use the following workflow in this case:

  • Create a new structured analytics set.
  • Call the RunAnalysisPreparationAsync endpoint to generate fields used to store results from an analysis.
  • Use the fields generated by the RunAnalysisPreparationAsync endpoint to create views and saved searches. You can also use this process to add a structured analytics set to a template workspace.
  • Call the RunAsync endpoint to complete the analysis of the structured analytics set, and store results in the fields that you generated.

Send a request to this URL for the Structured Analytics Manager service:

Copy
<host>/relativity.rest/api/Relativity.StructuredAnalytics.Services.Interfaces.StructuredAnalytics.IStructuredAnalyticsModule/Structured Analytics Manager/RunAnalysisPreparationAsync

The request must contain the following fields:

  • workspaceId - the Artifact ID of the workspace that contains the structured analytics set.
  • sasArtifactId - the Artifact ID of the structured analytics set.
Copy
{
   "workspaceId":1234567,
   "sasArtifactId":7654321
}

The response contain the following fields:

  • IsSuccess - a Boolean value indicating whether the request succeeded or failed.
  • Message - a string that returns an informative message about the status of the request.
Copy
{
   "IsSuccess":true,
   "Message":"Analysis Preparation has been successfully started for Structured Analytics Set 7654321 in Workspace 1234567"
}

Retrieve valid operations for a structured analytics set

Use the GetValidTasksAsync endpoint to retrieve an array of valid operations for a structured analytics set, such as retrying errors, running an analysis, and others. Send a request to this URL for the Structured Analytics Manager service:

Copy
<host>/relativity.rest/api/Relativity.StructuredAnalytics.Services.Interfaces.StructuredAnalytics.IStructuredAnalyticsModule/Structured Analytics Manager/GetValidTasksAsync

The request must contain the following fields:

  • workspaceId - the Artifact ID of the workspace that contains the structured analytics set.
  • sasArtifactId - the Artifact ID of the structured analytics set.
Copy
{
   "workspaceId":1234567,
   "sasArtifactId":7654321
}

The response contain the following fields:

  • ValidTasks - an array of valid jobs that can be run on the specified analytics set. For a complete list of tasks, see the StructuredAnalyticsTask enumeration in the Structured Analytics API on the Relativity API reference page.
  • CopyToLegacyQualified - a Boolean value indicating whether the results of an analysis is copied to existing document fields. For more information, see Work with legacy document fields.
Copy
{
   "ValidTasks":[
      "RUN_ANALYSIS",
      "COPY_TO_LEGACY",
      "PREPARE_ANALYSIS"
   ],
   "CopyToLegacyQualified":true
}

Run an analysis of a structured analytics set

Use the RunAsync endpoint to analyze documents in a structured analytics set. Send a request to this URL for the Structured Analytics Manager service:

Copy
<host>/relativity.rest/api/Relativity.StructuredAnalytics.Services.Interfaces.StructuredAnalytics.IStructuredAnalyticsModule/Structured Analytics Manager/RunAsync

The request must contain the following fields:

  • workspaceId - the Artifact ID of the workspace that contains the structured analytics set.
  • sasArtifactId - the Artifact ID of the structured analytics set.
  • settings - represents a configuration used for an analysis run. This field represents an AnalysisSettings object, which contains the following fields:
    • AnalyzeAll - a Boolean value indicating whether all documents are updated with the analysis results. When this field is set to false, only new documents are updated with the results.
    • PopulateAll - a Boolean value indicating whether all documents are repopulated. When the PopulateAll field is set to false, only new documents are added to the existing population. Otherwise, all documents are repopulated.
Copy
{
   "workspaceId":1234567,
   "sasArtifactId":7654321,
   "settings":{
      "AnalyzeAll":true,
      "PopulateAll":true
   }
}

The response contain the following fields:

  • IsSuccess - a Boolean value indicating whether the request succeeded or failed.
  • Message - a string that returns an informative message about the status of the request.
Copy
{
   "IsSuccess":true,
   "Message":"Analysis has been successfully started for Structured Analytics Set 7654321 in Workspace 1234567"
}

Cancel an analysis

The Structured Analytics Manager service provides two endpoints that you can use to cancel an analysis of structured analytics set:

  • CancelAsync endpoint

    This endpoint makes a request to cancel the analysis of the structured analytics set that is currently running. It then returns control immediately to the caller. Use this endpoint if you want to cancel an analysis on a structured analytics set, but aren't interested in waiting for the cancel operation to complete. This approach is useful when you don't intend to perform another operation after the cancel operation finishes.

    Copy
    <host>/Relativity.REST/api/Relativity.StructuredAnalytics.Services.Interfaces.StructuredAnalytics.IStructuredAnalyticsModule/Structured%20Analytics%20Manager/CancelAsync
  • CancelAndWaitAsync endpoint

    This endpoint makes a request to cancel the analysis of a structured analytics set. It then waits until the operation finishes before returning control to the caller. Use this endpoint if you want to implement a workflow that cancels an analysis job, and on the return of the call, immediately uses the RunAsync endpoint to restart an analysis operation.

    Copy
    <host>/Relativity.REST/api/Relativity.StructuredAnalytics.Services.Interfaces.StructuredAnalytics.IStructuredAnalyticsModule/Structured%20Analytics%20Manager/CancelAndWaitAsync

The request must contain the following fields:

  • workspaceId - the Artifact ID of the workspace that contains the structured analytics set.
  • sasArtifactId - the Artifact ID of the structured analytics set.
Copy
{
   "workspaceId":123456,
   "sasArtifactId":134890
}

The response returns the following fields.

  • IsSuccess - a Boolean value indicating whether the request succeeded or failed.
  • Message - a string that returns an informative message about the status of the request.
Copy
{
   "IsSuccess":true,
   "Message":"The cancel analysis request has successfully started for structured analytics set 134890 in workspace 123456."
}

Check the status of an analysis

To retrieve the current status of an analysis for a structured analytics set, use the GetStatusAsync endpoint. Send a request to this URL for the Structured Analytics Manager service:

Copy
<host>/Relativity.REST/api/Relativity.StructuredAnalytics.Services.Interfaces.StructuredAnalytics.IStructuredAnalyticsModule/Structured%20Analytics%20Manager/GetStatusAsync

The request must contain the following fields:

  • workspaceId - the Artifact ID of the workspace that contains the structured analytics set.
  • sasArtifactId - the Artifact ID of the structured analytics set.
Copy
{
   "workspaceId":123456,
   "sasArtifactId":134890
}

The response returns the following JSON objects with their related fields:

  • JobName - contains a field that specifies the name of the structured analytics set.
  • JobState - contains fields related to the status of an analysis, including information about the percent completed, the duration of the job, and others. The following list provides additional information about selected fields on the JobState object:
    • Phase - the phases for analysis include setup, importing, running operations, and others. For a complete list, see the JobPhase enumeration in Structured Analytics API section on the Relativity reference page.
    • JobTimeElapsed - the number of seconds since the analysis has started.
    • OperationTimeElapsed - the number of seconds since the current operation has started.
  • SelectedOperations - lists the structured analytics operations performed on the documents added to the set. These operations include email threading, textual near duplicate identification, language identification, and repeated content identification. For more information, see Run Structured Analytics in Running structured analytics on the Relativity Documentation site.
  • ValidOperations - a list of valid operations for a structured analytics set, such as running a full analysis, canceling an analysis, getting document errors, and others. For a complete list, see the StructuredAnalyticsTask enumeration in Structured Analytics API section on the Relativity reference page.
  • JobResults - lists the number of documents analyzed in a job. It also includes a breakdown of email threading statistics as follows:
    • EmailsAnalyzed - lists the total number of emails analyzed by the email threading operation.
    • InclusiveEmails - lists the total count and percentage of all emails in the set that require review.
  • TextualNearDuplicatesStatistics - includes the information in the following fields:
    • TextualNearDuplicateGroupsCount - the total number of textual near duplicate groups.
    • AverageSimilarityPercentage - a percent that indicates how similar a document must be to a principal document in order to be grouped with it. For more information, see Textual near duplicate identification on the Relativity Documentation site.
  • NameNormalizationStatistics - includes the information in the following fields:
    • AliasesIdentified - a total count of all aliases in a group of email messages.
    • NewAliasesImported - the number of new aliases added to Relativity.

    The NameNormalizationStatistics field isn't listed in the sample JSON. For more information, see Structured Analytics (.NET).

Copy
{
   "JobName":"Sample Analytics",
   "JobState":{
      "IsRunning":false,
      "IsCompleted":true,
      "Status":"Completed analysis with errors",
      "Phase":"Completed",
      "PhaseProgressPercentage":0,
      "JobTimeElapsed":0,
      "OperationTimeElapsed":0
   },
   "SelectedOperations":[
      "EmailThreading",
      "TextualNearDuplicateId",
      "LanguageId",
      "RepeatedContentId"
   ],
   "ValidOperations":[
      "RUN_FULL_ANALYSIS",
      "RUN_INCREMENTAL_ANALYSIS",
      "GET_DOCUMENT_ERRORS"
   ],
   "JobResults":{
      "DocumentsAnalyzed":4798,
      "EmailThreadingStatistics":{
         "EmailsAnalyzed":4698,
         "InclusiveEmails":4645
      },
      "TextualNearDuplicatesStatistics":{
         "AverageSimilarityPercentage":98,
         "TextualNearDuplicateGroupsCount":12
      }
   }
}

Retry errors in an analysis

You can attempt to resolve errors that occurred during an analysis of a structured analytics set by rerunning the failed items with the RetryErrorsAsync endpoint. Send a request to this URL for the Structured Analytics Manager service:

Copy
<host>/Relativity.REST/api/Relativity.StructuredAnalytics.Services.Interfaces.StructuredAnalytics.IStructuredAnalyticsModule/Structured%20Analytics%20Manager/RetryErrorsAsync

The request must contain the following fields:

  • workspaceId - the Artifact ID of the workspace that contains the structured analytics set.
  • sasArtifactId - the Artifact ID of the structured analytics set.
Copy
{
   "workspaceId":123456,
   "sasArtifactId":134890
}

The response returns the following fields:

  • IsSuccess - a Boolean value indicating whether the request succeeded or failed.
  • Message - a string that returns an informative message about the status of the request.
Copy
{
   "IsSuccess":true,
   "Message":"The retry errors request has successfully started for structured analytics set 134890 in workspace 123456."
}

Retrieve analysis errors for a structure analytics set

The GetErrorsAsync endpoint retrieves set errors. These are errors aren't document specific and they cause an analysis to stop. For example, a set error may be a validation error for a structured analytics set, or it may occur when the Analytics server is disabled or goes down. To retrieve set errors, send a request to this URL for the Structured Analytics Manager service:

Copy
<host>/Relativity.REST/api/Relativity.StructuredAnalytics.Services.Interfaces.StructuredAnalytics.IStructuredAnalyticsModule/Structured%20Analytics%20Manager/GetErrorsAsync

The request must contain the following fields:

  • workspaceId - the Artifact ID of the workspace that contains the structured analytics set.
  • sasArtifactId - the Artifact ID of the structured analytics set.
  • start - indicates the index of the first error that you want to return.
  • length - the number of subsequent errors to return.
Copy
{
   "workspaceId":123456,
   "sasArtifactId":134890,
   "start":0,
   "length":1000
}

The response returns the following fields for each error:

  • Fulltext - a verbose description of the error that occurred.
  • Message - a short summary of the error that occurred.
  • Server - the location of the Relativity server that hosts the agent where the error occurred.
  • Source - detailed information about where the error originated, such as the agent or web server.

This JSON sample illustrates an array of errors retrieved by the GetErrorsAsync endpoint.

Copy
[
   {
      "Fulltext":"This is the full text of the error message",
      "Message":"This is a shortened version of the error message...",
      "Server":"SERVER_LOCATION",
      "Source":"ERROR_SOURCE"
   },
   {
      "Fulltext":"This is the full text of the error message",
      "Message":"This is a shortened version of the error message...",
      "Server":"SERVER_LOCATION",
      "Source":"ERROR_SOURCE"
   }
]

Retrieve document errors for an analysis

The GetDocumentErrorsAsync endpoint retrieves errors that are associated with the processing of a specific document. For example, this type of error occurs when a document is too large to be extracted or ingested in the Analytics engine, or when Analytics engine fails to process a document due to corruption. To retrieve document errors for a structured analytics set, send a request to this URL for the Structured Analytics Manager service:

Copy
<host>/Relativity.REST/api/Relativity.StructuredAnalytics.Services.Interfaces.StructuredAnalytics.IStructuredAnalyticsModule/Structured%20Analytics%20Manager/GetDocumentErrorsAsync

The request must contain the following fields:

  • workspaceId - the Artifact ID of the workspace that contains the structured analytics set.
  • sasArtifactId - the Artifact ID of the structured analytics set.
  • start - indicates the index of the first error that you want to return.
  • length - the number of subsequent errors to return.
Copy
{
   "workspaceId":123456,
   "sasArtifactId":134890,
   "start":0,
   "length":1000
}

The response returns the following fields for each document error:

  • DocumentArtifactId - the Artifact ID of the document associated with an error.
  • DocumentIdentifier - the control number of the document associated with an error occurred. Relativity uses the control number as a unique identifier for a document. For information, see Analytics profile on the Relativity Documentation site.
  • Message - a string that returns an informative message about the status of the request.
  • Status - the current status of the document. The following statuses may appear in this field:
    • IncludedInSet - the document remains in the analysis of the structured analytics set.
    • RemovedFromSet - the document is automatically removed from the analysis of the structured analytics set.
    • DataWarning - a problem with the document was identified during analysis and the document was removed from the results.
  • DateErrored - the date and time when the document error was logged.

This JSON sample illustrates an array of the document errors retrieved by the GetDocumentErrorsAsync endpoint.

Copy
[
   {
      "DocumentArtifactId":1383924,
      "DocumentIdentifier":"Document_1",
      "Message":"An error occurred during processing of email",
      "Status":"DataWarning",
      "DateErrored":"2017-01-06T11:33:00.357"
   },
   {
      "DocumentArtifactId":1383925,
      "DocumentIdentifier":"Document_2",
      "Message":"Processing failed for the given document",
      "Status":"RemovedFromSet",
      "DateErrored":"2017-01-06T11:33:00.357"
   },
   {
      "DocumentArtifactId":1383926,
      "DocumentIdentifier":"EmailThreadingTaskProblemDoc5",
      "Message":"A network error occurred.",
      "Status":"IncludedInSet",
      "DateErrored":"2017-01-06T11:33:00.357"
   }
]

Work with legacy document fields

As of Relativity 9.5.196, Structured Analytics stopped writing results for email threading and textual near duplicate identification to Document fields. Instead, it writes results to fields on the Structured Analytics Results object. However, you can copy these results to the legacy document result fields by calling the RunCopyToLegacyAsync endpoint. It copies the results from the newly created fields to the legacy document fields from pre 9.5.196 versions of Relativity. For more information, see Copy to Legacy Document Fields on the Relativity Documentation site.

Copy results to legacy document fields

Use the RunCopyToLegacyAsync endpoint to copy the results of an analysis to existing document fields. Send a request to this URL for the Structured Analytics Manager service:

Copy
<host>/relativity.rest/api/Relativity.StructuredAnalytics.Services.Interfaces.StructuredAnalytics.IStructuredAnalyticsModule/Structured Analytics Manager/RunCopyToLegacyAsync

The request must contain the following fields:

  • workspaceId - the Artifact ID of the workspace that contains the structured analytics set.
  • sasArtifactId - the Artifact ID of the structured analytics set.
Copy
{
   "workspaceId":1234567,
   "sasArtifactId":7654321
}

The response contain the following fields:

  • IsSuccess - a Boolean value indicating whether the request succeeded or failed.
  • Message - a string that returns an informative message about the status of the request.
Copy
{
   "IsSuccess":true,
   "Message":"Copy To Legacy Fields has been successfully started for Structured Analytics Set 7654321 in Workspace 1234567"
}

Cancel a job for copying content to legacy fields

Use the CancelCopyToLegacyAsync endpoint to cancel a job for copying the results of an analysis to existing document fields. Send a request to this URL for the Structured Analytics Manager service:

Copy
<host>/relativity.rest/api/Relativity.StructuredAnalytics.Services.Interfaces.StructuredAnalytics.IStructuredAnalyticsModule/Structured Analytics Manager/RunCopyToLegacyAsync

The request must contain the following fields:

  • workspaceId - the Artifact ID of the workspace that contains the structured analytics set.
  • sasArtifactId - the Artifact ID of the structured analytics set.
Copy
{
   "workspaceId":1234567,
   "sasArtifactId":7654321
}

The response contain the following fields:

  • IsSuccess - a Boolean value indicating whether the request succeeded or failed.
  • Message - a string that returns an informative message about the status of the request.
Copy
{
   "IsSuccess":true,
   "Message":"Cancel Copy to Legacy Fields has been successfully started for Structured Analytics Set 7654321 in Workspace 1234567"
}