NSA QSM Connect

This document is considered in draft format for review only as of 2/2/2020

Introduction

Welcome to the North Shore Automation QSM-Connect user and administration guide. This document outlines the functions, usage, and basic technical information of the application and its related systems. While we advise that all users read the entire manual, system operators may only need to view the first few chapters. A system administrator or installer will want to review the document in its entirety.

In a nutshell, the QSM-Connect application enables CatDV DAM systems to connect to Quantum Storage Manager archive systems. By using the QSM-Connect solution, CatDV can communicate with a Quantum Storage Manager system to archive and restore assets, then record archive-related metadata in the CatDV database.

This document will cover the following topics:

  • QSM-Connect user operation from the CatDV client application

  • QSM-Connect metadata fields

  • Archive system and QSM-Connect application overview 

  • Basic troubleshooting


Document Conventions and Assumptions


This document assumes knowledge of CatDV and Quantum Storage Manager software. If you need more detailed information on CatDV Pro or Quantum Storage Manager, please refer to their respective documentation.

In some cases (typically the overview sections), the term "archive action" may be used generically to refer to any archive related operation such as archive to tape, restore, or asset deletion. When a specific type of operation is referred to, such as a restore, it will be referred to by name.

These systems are highly customized in most cases, so all references to topology and system configuration are examples. Consider undertaking a system design with your CatDV and Quantum integrator to determine the correct configuration for your organization.


Chapter 1 Workflow & User Operations

Key Concepts


  • NSA QSM Connect manages communication between Quantum Storage Manager (Stornext) and a CatDV Server

  • NSA QSM Connect does not move any media files

  • CatDV Worker Node and QSM perform all media operations



The following is a simplified description of the workflow steps and applications performing them. For more detailed information see Chapter 3 for a full description of archive and restore operations. The application performing the operation for each step is in bold text.


  1. User flags asset/s for archive in CatDV UI / CatDV Server

  2. Worker Node performs media copy/move to Quantum archive policy

    1. Worker Node sets clip status in CatDV to “Archive Staged” when this operation is complete

  3. NSA QSM Connect reads “Archive Staged” status and triggers Stornext to archive the file

  4. NSA QSM Connect monitors the QSM archive job during the archive operation

  5. Once archive is complete, NSA QSM Connect updates CatDV Server with final archive status


For the purposes of this discussion we have defined the two main file locations as follows:


  • Active file location - The original file location on the Stornext volume

  • Archive file location - The file location when moved/copied to a Stornext Storage Manager archive policy

    • For more information on Archive Policies or Storage Manager functions please contact your Stornext administrator


Archive Workflows

There are 3 major archive workflows possible with NSA QSM Connect 2 the first are most common, the 3rd (archive in place) in included for completeness but is rare in production


Move To Archive - In this workflow the active file is moved to a relative path in the Archive file location in a StorNext policy. This removes the file from active storage, and allows archive and truncation increasing available space. See diagram below.


Note: If a move operation is performed from one volume to another volume (where the Archive Policy is located) this in effect is still a “copy” operation, though the active file may be deleted after the copy is complete.

Copy to archive - In this workflow the active file is copied to a relative path in the Archive file location in a StorNext policy. This does not remove the file from active storage and allows archive and truncation while keeping the original file in the Active file location where it is available for use. See diagram below, noting that the only visible difference between a move and copy operation is the removal of the original file.


Archive in place - In this workflow, the Active file location is actually the Archive file location. The policy is set on the original file folder. In this case QSM archives the file in its current location without a move or copy. Upon archive and truncation the representative “stub” file is created and is managed by StorNext and QSM for restore and other operations.

Move To Archive Workflow Diagram:




Move To Archive (Dual Volume) Workflow Diagram:




Archive In Place Workflow Diagram:





Troubleshooting

Storage Manager wsar_agent issues

The log below points to the Storage Manager "wsar_agent" service having crashed. 2021-05-17 08:57:18,315 ERROR com.quantum.rest.swx.FileResource - Unable to connect to the WSAR database com.quantum.rest.swx.WebServiceException: Connection refused (Connection refused) If the wsar_agent is down the API calls to Quantum go unanswered. It needs to be restarted via SSH on the StorNext server. Here are instructions below on how to do that.

Steps to restart wsar agent

Instructions for StorNext 6

1. Log into sudo rootsh 2. Type: ps -ef | grep wsar_agent (see if wsar agent is running. Results will looks like below.) a. root 7797 1 1 Aug21 ? 01:06:15 wsar_agent b. root 10923 41432 0 20:09 pts/1 00:00:00 grep wsar_agent 3. If wsar agent it down and there is not a "wsar_agent" process a. type: cd ~ 4. Type: wsar_agent a. This will start wsar agent 5. Type: ls -l /proc/ `ps --no-headers -o "%p" -C wsar_agent`/fd a. This will clean up sockets.

Instructions for StorNext 5

Go on the active MDC (in cvadmin, look for the * next to the file system names to indicate which one is the active server), and do a: # ps -ef | grep wsar_agent If it’s not running, I would suggest restarting it by running: # /usr/adic/bin/wsar_agent If it’s running, it may have run out of sockets. You can do: ls -l /proc/`ps --no-headers -o "%p" -C wsar_agent`/fd