Analytics Conceptual Index (.NET)

A conceptual index is a type of index used by Relativity Analytics. It uses Latent Semantic Indexing (LSI) to discover concepts between documents. This indexing process is based solely on term co-occurrence. The language, concepts, and relationships are defined entirely by the contents of your documents and learned by the index. For general information about Analytics indexes, see Analytics indexes and on the RelativityOne Documentation site.

The Analytics Conceptual Index API supports programmatically managing conceptual indexes in Relativity Analytics. It includes the following features:

  • Supports CRUD operations on Analytics indexes.
  • Provides helper methods that simplify working with index jobs. You can use these methods to submit and cancel jobs, and to check on their statuses. These jobs can populate new data, build new indexes, and activate indexes for use.

As a sample use case, you might use this API in an application to programmatically operate on a conceptual index.

You can also use the Conceptual Index API through REST. For more information, see Analytics Conceptual Index (REST).

The Relativity.Analytics.Conceptual.SDK contains this API. For compatibility and package installation instructions, see Download the SDKs and NuGet packages.

Fundamentals for the Conceptual Index API

Review the following information to learn about the methods, classes, and enumerations used by the Conceptual Index API.

Guidelines for using the Conceptual Index API

Review the following guidelines for working with the Conceptual Index API.

Common workflows

For Analytics indexes, common workflows include building new ones or incrementally updating existing ones.

Catch exceptions

When using any methods in the Conceptual Index API, catch the following exceptions:

  • PermissionException
  • ValidationException
  • Generic .NET Exception

Execute valid operations for the index state

Execute only valid operations for the state of an index. Only certain operations are valid for the current state of your index. See the following examples:

  • A call to cancel a running index job is only valid when a job is currently running for the index.
  • A call to build an index can't run on an index that hasn't been populated.
  • Note: To retrieve a list of valid operations, get the AnalyticsIndexStatus object. Examine the ConsoleButtons property on the returned object for a list of valid operations to call. You can also get additional information about the job from the StatusText and AdditionalStatusText properties. If you attempt to execute an invalid operation, you receive validation error. See Retrieve the status of a job.

Index states and job validation

When performing operations on an index, make sure the operation is applicable to its current state.

Monitor the analysis progress

After submitting an index job, monitor its progress by making calls to its status. See Retrieve the status of a job.

Create a conceptual index

Use the CreateAsync() method to add a new index to Relativity. Set the following properties on the ConceptualIndex object passed to the CreateAsync() method:

  • AnalyticsServer - an ArtifactRef object representing the Analytics server used to build the index. In RelativityOne environments, set the Artifact ID -1.
  • Name - the user-friendly name of the index.
  • Order - an integer representing the position of the index in drop-down menus. The default value is 0.
  • SearchableSet - an ArtifactRef object representing the saved search used to populate the searchable set.

When the operation succeeds, it returns the Artifact ID of the new index. Use the Artifact ID to submit the index for populating and building.

Copy
public async Task CreateAsyncSampleCode()
{
    ServiceFactorySettings settings = new ServiceFactorySettings(
      new Uri("net.pipe://localhost/relativity.services/"),
      new Uri("http://localhost/relativity.rest/api"),
      new IntegratedAuthCredentials());
    ServiceFactorySettings settings = new ServiceFactorySettings(new Uri(relativityServicesUri), new Uri(relativityRestUri), usernamePasswordCredentials);
    ServiceFactory serviceFactory = new ServiceFactory(settings);
 
    using (IConceptualIndexService indexService = serviceFactory.CreateProxy<IConceptualIndexService>())
    {
        int workspaceId = 1023289;
        int searchableSetSearchArtifactID = 1038051;
        int trainingSetSearchArtifactID = 1038051;
        int analyticsServerArtifactID = 1016891;
  
        ConceptualIndex indexToCreate = new ConceptualIndex
        {
            Name = "my new analytics index",
            Order = 1,
            SearchableSet = new ArtifactRef { ArtifactID = searchableSetSearchArtifactID },
            TrainingSet = new ArtifactRef { ArtifactID = trainingSetSearchArtifactID },
            AnalyticsServer = new ArtifactRef { ArtifactID = analyticsServerArtifactID },
            OptimizeTrainingSet = true,
            RemoveDocumentsThatErroredDuringPopulation = true,
            ContinueIndexStepsToCompletion = true,
            ConceptStopWords = "a\r\nable\r\nabout\r\nabove\r\naccording\r\n",
            Dimensions = 100,
            RemoveEnglishSignaturesAndFooters = true,
            EnableEmailHeaderFilter = true
        };
  
        try
        {
            int conceptualIndexID = await indexService.CreateAsync(workspaceId, indexToCreate);
            Debug.WriteLine("ID of the newly created index is: " + conceptualIndexID);
        }
        catch (PermissionException ex)
        {
            _logger.LogError(ex,
            "Failed to create index because of invalid permission. workspaceID: {workspaceId} - IndexToCreate: {@index}",
            workspaceId, indexToCreate);
            throw;
        }
        catch (ValidationException ex)
        {
            _logger.LogError(ex,
            "The index you were trying to create was invalid. workspaceID: {workspaceId} - IndexToCreate: {@index}",
            workspaceId, indexToCreate);
            throw;
        }
        catch (Exception ex)
        {
            _logger.LogError(ex,
            "Creating index failed. workspaceID: {workspaceId} - IndexToCreate: {@index}",
            workspaceId, indexToCreate);
            throw;
        }
    }
}

Retrieve a conceptual index

Use the ReadAsync() method to retrieve an index. Pass the Artifact IDs of the workspace and the index to this method. It returns a ConceptualIndex object. You can use this object to examine the index properties before submitting an index job, or updating or deleting the index.

Copy
public async Task ReadSingleAsyncSampleCode()
{
    string userName = "user@test.com";
    string password = "Test1234!";
  
    string relativityServicesUri = "http://localhost/relativity.services";
    string relativityRestUri = "http://localhost/relativity.rest/api";
  
    var usernamePasswordCredentials = new UsernamePasswordCredentials(userName, password);
    ServiceFactorySettings settings = new ServiceFactorySettings(new Uri(relativityServicesUri), new Uri(relativityRestUri), usernamePasswordCredentials);
    ServiceFactory serviceFactory = new ServiceFactory(settings);
  
    using (IConceptualIndexService indexService = serviceFactory.CreateProxy<IConceptualIndexService>())
    {
        int workspaceId = 1023289;
        int existingIndexArtifactID = 1049079;
  
        try
        {
            ConceptualIndex conceptualIndex = await indexService.ReadAsync(workspaceId, existingIndexArtifactID);
        }
        catch (PermissionException ex)
        {
            _logger.LogError(ex,
            "You don't have permission to retrieve the index. WorkspaceID {workspaceId} - IndexID {IndexId} - username: {username}",
            workspaceId, existingIndexArtifactID, userName);
            throw;
        }
        catch (NotFoundException ex)
        {
            _logger.LogError(ex,
            "Failed to retrieve the index because index could not be found. WorkspaceID: {workspaceID} - IndexID: {IndexId}",
            workspaceId, existingIndexArtifactID);
            throw;
        }
        catch (Exception ex)
        {
            _logger.LogError(ex,
            "Failed to retrieve the index. WorkspaceID {workspaceId} - IndexID {indexId}",
            workspaceId, existingIndexArtifactID);
            throw;
        }
    }
}

Update a conceptual index

Use the UpdateAsync() method to modify an index. You can update multiple properties on a ConceptualIndex object. For a list of properties, see the ConceptualIndex class in Classes and enumerations.

Copy
public async Task UpdateAsyncSampleCode()
{
    string userName = "user@test.com";
    string password = "Test1234!";
  
    string relativityServicesUri = "http://localhost/relativity.services";
    string relativityRestUri = "http://localhost/relativity.rest/api";
  
    var usernamePasswordCredentials = new UsernamePasswordCredentials(userName, password);
    ServiceFactorySettings settings = new ServiceFactorySettings(new Uri(relativityServicesUri), new Uri(relativityRestUri), usernamePasswordCredentials);
    ServiceFactory serviceFactory = new ServiceFactory(settings);
  
    using (IConceptualIndexService indexService = serviceFactory.CreateProxy<IConceptualIndexService>())
    {
        int workspaceId = 1023289;
        int existingIndexArtifactID = 1049079;
  
        ConceptualIndex conceptualIndex= null;
  
        try
        {
            try
            {
                conceptualIndex = await indexService.ReadAsync(workspaceId, existingIndexArtifactID);
            }
            catch (PermissionException ex)
            {
                _logger.LogError(ex,
                "You don't have permission to read the index. WorkspaceID {workspaceId} - IndexID {IndexId} - username: {username}",
                workspaceId, existingIndexArtifactID, userName);
                throw;
            }
            catch (NotFoundException ex)
            {
                _logger.LogError(ex,
                "Failed to read the index because index could not be found. WorkspaceID: {workspaceID} - IndexID: {IndexId}",
                workspaceId, existingIndexArtifactID);
                throw;
            }
            conceptualIndex.Name = "My Updated Analytics Index";
            await indexService.UpdateAsync(workspaceId, existingIndexArtifactID, conceptualIndex);
  
            conceptualIndex = await indexService.ReadAsync(workspaceId, existingIndexArtifactID);
  
            Debug.WriteLine("The updated name of the retrieved index is " + conceptualIndex.Name);
  
        }
        catch (PermissionException ex)
        {
            _logger.LogError(ex,
            "You don't have permission to update the index. WorkspaceID {workspaceId} - IndexID {IndexId} - username: {username}",
            workspaceId, existingIndexArtifactID, userName);
            throw;
        }
        catch (ValidationException ex)
        {
            _logger.LogError(ex,
            "The index you were trying to update has invalid data. workspaceID: {workspaceId} - IndexToUpdate: {@index}",
            workspaceId, conceptualIndex);
            throw;
        }
        catch (Exception ex)
        {
            _logger.LogError(ex,
            "Failed to update the index. WorkspaceID {workspaceId} - IndexID {indexId}",
            workspaceId, existingIndexArtifactID);
            throw;
        }
    }
}

Delete a conceptual index

Use the DeleteAsync() method to remove an index from Relativity. Pass the Artifact IDs of the workspace and the index to this method.

Copy
public async Task DeleteAsyncSampleCode()
{
    string userName = "user@test.com";
    string password = "Test1234!";
  
    string relativityServicesUri = "http://localhost/relativity.services";
    string relativityRestUri = "http://localhost/relativity.rest/api";
  
    var usernamePasswordCredentials = new UsernamePasswordCredentials(userName, password);
    ServiceFactorySettings settings = new ServiceFactorySettings(new Uri(relativityServicesUri), new Uri(relativityRestUri), usernamePasswordCredentials);
    ServiceFactory serviceFactory = new ServiceFactory(settings);
  
    using (IConceptualIndexService indexService = serviceFactory.CreateProxy<IConceptualIndexService>())
    {
        int workspaceId = 1023289;
        int existingIndexArtifactID = 1054085;
  
        try
        {
            await indexService.DeleteAsync(workspaceId, existingIndexArtifactID);
        }
        catch (PermissionException ex)
        {
            _logger.LogError(ex,
            "Deleting Index failed because of invalid permission. WorkspaceID {workspaceId} - IndexID {IndexId} - username: {username}",
            workspaceId, existingIndexArtifactID, userName);
            throw;
        }
        catch (NotFoundException ex)
        {
            _logger.LogError(ex,
            "Deleting Index failed because index could not be found. WorkspaceID {workspaceId} - IndexID {indexId}",
            workspaceId, existingIndexArtifactID);
            throw;
        }
        catch (DeletionDependencyException ex)
        {
            _logger.LogError(ex,
            "Deleting Index failed because there is a dependecy on it. WorkspaceID {workspaceId} - IndexID {indexId}",
            workspaceId, existingIndexArtifactID);
            throw;
        }
        catch (Exception ex)
        {
            _logger.LogError(ex,
            "Deleting index failed. WorkspaceID {workspaceId} - IndexID {indexId}",
            workspaceId, existingIndexArtifactID);
            throw;
        }
    }
}

Helper methods for conceptual index jobs

Use the following helper methods for managing jobs related to conceptual indexes.

Submit a conceptual index job

Use the SubmitJobAsync() method to submit a job after creating the index. Pass the Artifact IDs for the workspace and index, and a job type to this method. For a list of job types, see Classes and enumerations.

Copy
public async Task SubmitJobAsyncSampleCode()
{
    string userName = "user@test.com";
    string password = "Test1234!";
  
    string relativityServicesUri = "http://localhost/relativity.services";
    string relativityRestUri = "http://localhost/relativity.rest/api";
  
    var usernamePasswordCredentials = new UsernamePasswordCredentials(userName, password);
    ServiceFactorySettings settings = new ServiceFactorySettings(new Uri(relativityServicesUri), new Uri(relativityRestUri), usernamePasswordCredentials);
    ServiceFactory serviceFactory = new ServiceFactory(settings);
  
    using (IConceptualIndexService indexService = serviceFactory.CreateProxy<IConceptualIndexService>())
    {
        int workspaceId = 1023289;
        int existingIndexArtifactID = 1054087;
  
        try
        {
            await indexService.SubmitJobAsync(workspaceId, existingIndexArtifactID, AnalyticsIndexJobType.FullPopulation);
        }
        catch (PermissionException ex)
        {
            _logger.LogError(ex,
            "You don't have permission to submit the index. WorkspaceID {workspaceId} - IndexID {IndexId} - username: {username}",
            workspaceId, existingIndexArtifactID, userName);
            throw;
        }
        catch (Exception ex)
        {
            _logger.LogError(ex,
            "Failed to submit the index. WorkspaceID {workspaceId} - IndexID {indexId}",
            workspaceId, existingIndexArtifactID);
            throw;
        }
    }
}

Retrieve the status of a job

Use the GetStatusAsync() method to retrieve the status of an indexing job. Pass the Artifact IDs of the workspace and the index to this method. It returns an AnalyticsIndexStatus object.

Copy
public async Task GetStatusAsyncSampleCode()
{
    string userName = "user@test.com";
    string password = "Test1234!";
  
    string relativityServicesUri = "http://localhost/relativity.services";
    string relativityRestUri = "http://localhost/relativity.rest/api";
  
    var usernamePasswordCredentials = new UsernamePasswordCredentials(userName, password);
    ServiceFactorySettings settings = new ServiceFactorySettings(new Uri(relativityServicesUri), new Uri(relativityRestUri), usernamePasswordCredentials);
    ServiceFactory serviceFactory = new ServiceFactory(settings);
  
    using (IConceptualIndexService indexService = serviceFactory.CreateProxy<IConceptualIndexService>())
    {
        int workspaceId = 1023289;
        int existingIndexArtifactID = 1049072;
  
        try
        {
            AnalyticsIndexStatus status = await indexService.GetStatusAsync(workspaceId, existingIndexArtifactID);
            Debug.WriteLine("Status of the index is " + status.StatusText);
        }
        catch (PermissionException ex)
        {
            _logger.LogError(ex,
            "You don't have permission to get the index states. WorkspaceID {workspaceId} - IndexID {IndexId} - username: {username}",
            workspaceId, existingIndexArtifactID, userName);
            throw;
        }
        catch (NotFoundException ex)
        {
            _logger.LogError(ex,
            "Couldn't get the index status because index could not be found. WorkspaceID: {workspaceID} - IndexID: {IndexId}",
            workspaceId, existingIndexArtifactID);
            throw;
        }
        catch (Exception ex)
        {
            _logger.LogError(ex,
            "Failed to get the index status. WorkspaceID {workspaceId} - IndexID {indexId}",
            workspaceId, existingIndexArtifactID);
            throw;
        }
    }
}

Cancel a job

Use the CancelJobAsync() method to cancel an indexing job. Pass the Artifact IDs of the workspace and the index to this method.

Copy
public async Task CancelJobAsyncSampleCode()
{
    string userName = "user@test.com";
    string password = "Test1234!";
  
    string relativityServicesUri = "http://localhost/relativity.services";
    string relativityRestUri = "http://localhost/relativity.rest/api";
  
    var usernamePasswordCredentials = new UsernamePasswordCredentials(userName, password);
    ServiceFactorySettings settings = new ServiceFactorySettings(new Uri(relativityServicesUri), new Uri(relativityRestUri), usernamePasswordCredentials);
    ServiceFactory serviceFactory = new ServiceFactory(settings);
  
    using (IConceptualIndexService indexService = serviceFactory.CreateProxy<IConceptualIndexService>())
    {
        int workspaceId = 1023289;
        int existingIndexArtifactID = 1054087;
  
        try
        {
            await indexService.CancelJobAsync(workspaceId, existingIndexArtifactID);
        }
        catch (PermissionException ex)
        {
            _logger.LogError(ex,
            "Cancelling job failed because of invalid permission. WorkspaceID {workspaceID} - IndexID {indexID} - username: {username}",
            workspaceId, existingIndexArtifactID, userName);
            throw;
        }
        catch (NotFoundException ex)
        {
            _logger.LogError(ex,
            "Cancelling Job failed because index could not be found. WorkspaceID {workspaceID} - IndexID {indexId}",
            workspaceId, existingIndexArtifactID);
            throw;
        }
        catch (Exception ex)
        {
            _logger.LogError(ex,
            "Cancelling Job failed. WorkspaceID {workspaceId} - IndexID {indexId}",
            workspaceId, existingIndexArtifactID);
            throw;
        }
    }
}