CatDV-XenData User Manual

Introduction

This document is considered in draft format for review only as of 12/20/2020

Welcome to the North Shore Automation CatDV-XenData user and administration guide. This document outlines the functions, usage, and basic technical information of the application and its related systems. While we advise that all users read the entire manual, system operators may only need to view the first few chapters. A system administrator or installer will want to review the document in its entirety.

In a nutshell, the CatDV-XenData application enables CatDV DAM systems to connect to XenData archive systems. By using the CatDV-XenData solution, CatDV can communicate with an XenData system to archive and restore assets, then record archive-related metadata in the CatDV database.

This document will cover the following topics:

• • CatDV-XenData user operation from the CatDV client application

  • CatDV-XenData metadata fields

  • Archive system and CatDV-XenData application overview

  • Basic troubleshooting

Document Conventions and Assumptions

This document assumes knowledge of the CatDV and XenData software. If you need more detailed information on CatDV Pro or XenData, please refer to their respective documentation.

In some cases (typically the overview sections), the term "archive action" may be used generically to refer to any archive related operation such as archive to tape, restore, or asset deletion. When a specific type of operation is referred to, such as a restore, it will be referred to by name.

These systems are highly customized in most cases, so all references to topology and system configuration are examples. Consider undertaking a system design with your CatDV and XenData integrator to determine the correct configuration for your organization.

Chapter 1 - User Operations

This section will demonstrate how to initiate archive operations from the CatDV Pro interface.

All actions here are demonstrated on a single asset. However users may select as many assets as they choose and send all of them to a workflow at once. The system will queue the jobs and perform them sequentially. (Note that exceptionally large jobs may take many hours or days to complete, so be aware of this in large multi-user operations to ensure that multiple operators are not simultaneously submitting jobs and creating very large queues.)

If you require additional information about using CatDV Pro, please refer to CatDV's user documentation or the Squarebox website.

This chapter is included first in the manual for ease of access for users who want a refresher on how to perform basic archive actions and as a basic “quick start” guide. If you require deeper information or a more general overview of the system, you will find that data beginning with Chapter 2.

In most installations, the CatDV-XenData Archive App is accessed through the CatDV "XenData" details panel (as shown below). The specific fields used to manage and track archive jobs are defined and described in Chapter 2 "CatDV Metadata." Your system may be configured differently, but the field contents and functions will be comparable to what is documented in this chapter.

Note: XenData is abbreviated "XD" in all displays to keep label size down.

Archiving An Asset

Select an asset in the Main window of CatDV Pro.

Then, in the XenData Details Panel, select the "Send to Archive" value in the XD Archive Action pulldown.

Once you have selected the job type you want from the list, publish the selection to the CatDV Server ("Command-S" or "Control-S").

Note: You can include multiple records in an archive, restore, or delete job. Select multiple records in CatDV's "Main Window", then select the desired action in the "Details Panel".

Note: The XD Archive Action Field has four options you can choose, corresponding to the four XenData job types:

• • "Send to Archive" initiates an archive job for a specific record

  • "Update Metadata" initiates a metadata update job for a specific record

  • "Restore from Archive" initiates a restore job for a specific record

  • "Delete from Archive" initiates a delete job for a specific record

Viewing Archive Status

After submitting an asset/s for archive, if you refresh your CatDV server connection (in Server Menu or CMD+R on Mac or CTRL+R on Windows) you will see status updates on the selected record/s. Typically job submission times may vary depending on how your system is configured, but normally jobs are sent to XenData in 5 minutes or less.

While XenData processes an archive job, the CatDV Worker Node will update and reset the "Archive Action" field to blank.

Once a job is successfully submitted to XenData, the XD Job ID field is updated with a job ID.

Troubleshooting Tip: If  no Job ID is present after more than 5 minutes (and the user has confirmed a save and/or refresh in CatDV), this might indicate a problem with the archive operation.If the Archive Action field does not reset to blank (it still says "Send-to-Archive"), that may indicate that the Worker Node has failed to submit the job. If the Archive Action and the XD Job ID are both blank, it would indicate that Worker Node submitted the job, but XenData did not receive the data or there is a problem with the XenData system.

Indication Send to Archive Completed Successfully

Once XenData successfully completes an archive job, the app will update the CatDV record(s) with:

• • CRITICAL METADATA DATA

  • Single or multiple Tape IDs populated in the "XD Tape ID" field

  • ADDITIONAL DATA

  • A unique XenData reference number in the "XenData Job ID" field

  • "Archived" in the "XenData Status" field

  • The “Archive Status” field will be set to “ON DISK” until the asset has a valid location assigned to it by XenData. Subsequent "Metadata Update" requests which return a valid, XenData assigned Location will cause the status will be updated to “ARCHIVED”.

  • A detailed log for the job in the "XenData Log" field

Note: To see more information on the job, please refer to the XenData Administration app on the XenData server.

If there are errors in the operation, the XD Monitor Status will display "ERROR" and a brief section of the log will be written to the XD Log. Depending on the error displayed you may report the problem to your administrator or begin troubleshooting the appropriate system or workflow step.

Selecting a File Group for Send-to-XenData (optional)

As an optional step, you may want specific records to be archived to a specific XenData file group. Multiple files groups may be configured on a XenData system, each with different archive policies. For example one file group may be configured on the XenData system to store content to a non-replicated set of LTO cartridges and a second file group may be configured to write to a duplicated set of LTO cartridges. You can specify which file group to use before initiating an archive job by selecting a specific file group in the "XenData File Group" field.

Once you have selected the job type you want from the list, publish the selection to the CatDV Server ("Command-S" or "Control-S").

Note: This functionality requires your administrator to set up a list of XenData file groups in CatDV and a corresponding set of file group rules on the XenData system during the installation.

This image shows an archive job ready for submission to a File Group named "CatDV".

Restore From XenData

To initiate a restore, select Restore-From-XenData from the Archive Action menu and publish your changes. Please confirm was previously archived and that the asset is not online in the original path on storage. If the asset is online, and you still need to restore a copy, please move or temporarily rename the online copy or use an alternate restore path for this operation.

Once you have selected the job type you want from the list, publish the selection to the CatDV Server ("Command-S" or "Control-S").

Once again, using the Server Menu or f5 key will allow you to refresh your CatDV window and view restore status changes.

Note: CatDV records available for restore jobs will display the relevant XenData archive information in the XenData Job ID, XenData Job State, and XenData Log fields. If these fields are blank, it's likely that the asset has not been archived to the XenData system.

Because the CatDV2XenData application uses the XenData Workflow API as opposed to a drag and drop or copy workflow assets not archived using the CatDV2XenData application cannot be restored using CatDV Pro. To restore these assets, please use the XenData cache

Optional: Select alternate restore location

When initiating a restore you have two options

• 1. You may leave the location blank and the asset will be restored to its original location/file path.

  2. If your system administrator has set up specific alternate restore locations, you  may choose a predefined location from the ”XD Alt Restore Path” field, the asset will be restored to the designated location. You can specify which location to use before initiating a restore job in the "Restore Location" field, shown below. This will restore the file into the specified location preserving the original file path from root.

Note: If no location is specified in this field, the app directs XenData to restore to the location in the "Media Path" field for the record(s). This should be the original location of the asset on the storage. If that location does not exist (for example if that volume is not accessible to the XenData Server), the restore job will fail and an ERROR message will be displayed for the asset.

Restore in Progress

Indication that Restore has Completed

Upon completion of the restore operation, the XD Archive Action field will be reset to blank, the XD Log will display date and time of the restore, and the XD Monitor Status will be set to "Complete."

Update Metadata From XenData

In some cases you may want to confirm that the XenData-related metadata in CatDV is correct or up to date. Also, if you have completed a repack operation where older LTO tapes have been automatically migrated to new tapes inside the XenData system, the Tape ID’s will be out of date and therefore a metadata update of all assets on the re-packed tapes will be desired. Note: After a repack, you may choose to set up large batches of updates (perhaps on a tape by tape basis) or you could choose to do the updates ad hoc at the time of a desired restore. Please consider the workload on your system and Worker Node when planning a large number of metadata updates. This might be best done over a night or weekend to avoid stressing your Worker Node or archive system with a large number of jobs. To initiate a metadata update select Update Metadata from the XD Archive Action menu and publish your changes. (Ctrl-S) Once that Worker action runs (typically 1-2 minutes) you should see the new tape ID and current metadata in the XenData panels.

Delete From XenData

While assets cannot be truly “deleted” from archive media, due to their linear nature, without rewriting the entire tape or optical cartridge, XenData will support “hiding” assets, when a delete operation is requested. This is most commonly desired when an existing asset is modified or a new version of an asset is made to fix a problem with the media. In these cases the name of the file and its path will be the same as the old version and this will prevent it from being archived to the XenData system. By first deleting the file from the archive, you are able to archive the new version.

Note: This behavior is different than the traditional XenData behavior and follows the architecture of CatDV where an asset is identified by path, including file name and the CatDV Clip ID. For more information on the differences between these workflows, please contact XenData.

Before initiating a deletion job, the user will have to check the "Confirm Delete" box. While XenData processes a delete job, the app will update the CatDV record/s with "Restore-from-XenData" in the "Archive Action" field.

Note: CatDV records for delete jobs will display the relevant XenData archive information in the XenData Job ID, XenData History, and XenData Log fields.

Indication of Delete from XenData completed

Once XenData successfully completes an archive job, the app will update the CatDV record/s with:

• • A summary of the delete job in the "XD Log" field, along with previous archive jobs

  • "Deleted" in the "XenData Status" field

  • "Delete Job Sent" in the "XenData Log" field

Note: Any information about previous archive jobs in the "XenData Log" field will be overwritten by a delete job as this field only displays the log data for the last action. To see more information on all jobs, please refer to the XenData Administration app on the XenData server. However, the "XenData Job ID" field will show the job ID for the last completed archive prior to the deletion.

#IMAGE MISSING

Chapter 2 - CatDV Metadata

This section lists all the metadata fields in CatDV that are used and also gives a description of them.

The XenData metadata fields are created in the CatDV metadata during the system deployment and can be mapped to a details panel in CatDV. We recommend collecting these fields into a dedicated CatDV details pane for clarity.

In this example, the user has created a details panel called "XenData" in which the XenData and CatDV2XenData metadata are displayed.

Note: In the "Main View" of the CatDV window, you can set up a column display to view the XenData Status for every clip.

#IMAGE MISSING

This view is very useful in avoiding one of the most common sources of user errors and support tickets. Sending an asset to an archive more than once. If a previously archived asset is sent to archive a second time, it will be refused by the archive system. Additionally, this view will allow you to avoid trying to restore assets that have not been previously archived. These are very common sources of user errors and support tickets.  

The metadata fields for the CatDV2XenData workflow are named in a way that aids in tracking status and troubleshooting the archive workflow.

The data in fields labeled "XD" are typically coming from the XenData server via API. All other fields are either updated by user input or by a Worker Node action (e.g., Archive Action, XD File Group. Restore Location).

Note: There are two exceptions to the “XD” naming convention. “XD File Group” and “XD Delete Confirm” are user input fields that are set by the user, as needed.

CatDV2XenData Metadata Fields

File Name - Built-in CatDV field that lists the selected file's name

Media Path - Built-in CatDV field that lists the selected file's path. This will also be the asset’s original file path, stored in the archive, and used for restore operations.

Clip ID - Built-in CatDV field that lists the selected file's Clip ID (GUID)

XD Job ID - Returned XenData Job ID

XD Tape IDs - Comma separated list of tape IDs this asset exists on.

Note: The XD Tape ID for an asset may not be immediately available in the CatDV interface. For more information on this please refer to the introduction to Chapter 3. Retrieving the XD Tape ID is discussed in detail in this section.

XD Archive Action - CatDV User field watched by the Worker Node in order to start the job enqueue/archive process.  After the Worker Node action runs and the archive action has been submitted to XenData, this value will be set to blank (a null value).

Listed here are the possible values for the XD Archive Action field:

• • Send to Archive

  • Restore from Archive

  • Update Metadata

  • Delete from Archive

XD File Group - Archive destination in the XenData system. For more information on these groups or how they are configured, please refer to the XenData manual.

XD Confirm Delete - Required additional safety check to make sure ‘Delete from Archive’ actions only delete files you want them to delete.

XD Job State - The status of a job as reported directly by XenData

Listed here are the possible values for the XD Job State field:

• • In Progress

  • Waiting

  • Complete

  • Fault

  • etc

XD Monitor State - State as determined by monitoring code.

Listed here are the possible values for the XD Monitor State field:

• • Complete

  • Fault

  • In Progress

  • etc

XD Alt Restore Path - Allows restoring to an alternative path while preserving the original archive directory structure by adding the pre-defined prefix.

Example: If your administrator has designated an alternate restore path, such as:

\\unc-server\sharepoint\new\restore\location

And an asset has an original file path of:

/Volumes/SAN/old/file/location/file.mov

Then the asset will be restored to:

\\unc-server\sharepoint\new\restore\location\Volumes\SAN\old\file\location\file.mov

Note: A similar outcome can be achieved by setting a path mapping in the app config. This configuration and workflow requires additional professional services to configure and manage. If no alt restore path is specified, any path mapping specified in the app config will still be applied to the base restore path.

XD Bytes Processed - The number of bytes XenData has processed.

XD Bytes Total - The number of total bytes XenData will process.

XD Files Processed - The number of files XenData has processed.

XD Total Files - The total number of files XenData will process. Note: This will be 1 for normal clips and potentially multiple for metaclips.

XD Percent Processed - A report of percent complete of a XenData job.

XD Media Status - The status of the media file in the archive. "Online" status defines the media as being online to the XenData cache. "Nearline" status defines the media as being stored on the archive medium and no longer on the XenData cache.

XD Volume Status - The status of the archive volume (LTO or ODA media). "Online" defines the online status of the archive volume in the archive library. "Offline" status defines the archive volume is offline/removed from the archive library.

XD Monitor Flag - A flag to trigger the CatDV2XenData monitoring process to monitor this asset’s status because of a job in progress. Note: This field is hidden from users and is only for use by background processes.

XD Log - A log of actions on this asset by the archive, restore, and update operations of CatDV2XenData. Failures will be documented in more detail here.

CatDV2XenData Metadata Fields

Archive Action - showing different states of the files archive process.

XenData Queue Details Panel

The XenData Queue panel contains the XD Q fields which are populated once a clip is queued by the "catdv_to_xendata_enqueue" application that the Worker Node application initiates.

XD Q Mode - The type of archive action that will be performed by XenData. There are four types which are tied to the "XD Archive Action" values.

"Send to Archive" when queued sets the XD Q Mode to "Archive" mode

"Restore from Archive" when queued sets the XD Q Mode to "Restore" mode

"Update Metadata" when queued sets the XD Q Mode to "Update" mode

"Delete from Archive" when queued sets the XD Q Mode to "Delete" mode

XD Q File Group - The XenData File Group that the file will be sent for archive, restore, update or delete action based on the XD Q Mode value.

XD Q Server ID - The server name/IP address or letter drive of the file configured in the Northshore Automation"catdv_to_xendata.cfg" file and XenData "MediaServers.xml" file.

XD Q File Path - The file path or mapped relative file path after the "server name" of the file configured in the Northshore Automation"catdv_to_xendata.cfg" file.

XD Q Prefix - The Alt Restore Path prefix that is set when requesting a "Restore from Archive" with a value entered in the XD Alt Restore Path/Prefix field upon queuing.

XD Q Metadata - A text field that contains information about the queued clip/file. If the CatDV clip is a metaclip, the XD Q Metdata field will contains all of the metadata clip file paths for the metaclip contents.

XD Q Metadata will also show the "delayed" status of a clip/file if XenData is unable to perform the Archive, Restore, Update or Delete request. If delayed it will also show the next possible attempt to run the clip/file mode.

XD Q Try Count- The number of attempts/tries to perform the XD Q Mode. The number of tries is set in the "catdv_to_xendata.cfg". Below is a sample config showing up to "10" tries and what the delay times of five minutes would be.

# With initial_minute_delay = 5: # try 0; 0 minute delay # try 1; 5 minute delay # try 2; 10 minute delay # try 3; 20 minute delay # try 4; 40 minute delay # try 5; 80 minute delay # try n; minutes = <initial_minute_delay> * 2 ^ (n - 1) # 10 tries == 3.552 days of retrying # 0 means no retry limit max_submit_retry_count = 10

When queued this value will be set to "0". If the first attempt to perform the XD Q Mode fails for various reasons (file offline, no available tapes, file has already been archive, etc) then the XD Q Try Count will be set to "1".

This completes the description of the CatDV metadata fields used by the CatDV2XenData and XenData applications. If you have any questions, please contact your systems administrator or North Shore Automation.

Chapter 3 - CatDV2XenData App

This section provides a functional overview of CatDV2XenData and describes the application and basic system topology. The chapter includes a block diagram and step-by-step workflow overview describing not only the steps visible to the user but the back-end processes that are at work in archive operations.

The CatDV2XenData app is middleware, managing the information exchange between CatDV and XenData. The app delivers archive jobs, as metadata and commands, to XenData and delivers the responses from XenData back to the CatDV Server. CatDV2XenData does not directly move or manipulate media or files. All file archive actions are performed by XenData, and all metadata is stored by CatDV.

One important thing to note is that using XenData with CatDV2XenData presents a significant change in the archive workflow. Traditionally assets are archived to a XenData system by copying them to the XenData server’s cache. Restores are performed by accessing the stub files on the cache and then dragging them to their original location (or an alternate location) on the user’s storage.

When using the CatDV2XenData workflow to send to archive, the asset’s file path is delivered to the XenData Workflow API from CatDV and the XenData system then typically pulls the file into its archive cache and then moves it to a group of data tape or optical cartridges . Restores operate similarly, where the file is pushed back to its original file path (or an alternate path) by the XenData Workflow API

Another important, and frequently asked question, is how to access the Tape ID for an asset via CatDV. When archiving an asset the XenData system caches the file to disk and then notifies the CatDV2XenData app once this caching operation has completed. It then writes the cached data to tape or optical cartridges and stores the Tape ID in its database. However, this Tape ID is not reported back to CatDV2XenData by default. Currently an “Update Status” operation must be run from CatDV in order to collect the Tape ID from XenData.

Application Topology

This drawing shows a sample topology with an CatDV Pro Client, a CatDV Server, with a CatDV Worker Node system running both Worker Node and the CatDV2XenData software and a XenData system consisting of a server connected to an LTO Library.

Contact your integrator or system administrator or refer to your system deployment documentation for info on your specific configuration.

Workflow Overview

The following section describes the data interchange in the system. Only the archive action is described for brevity, but the other workflows (restore etc.) employ a similar methodology.

The order of operations in the workflow is as follows. (See the drawing below for a visual representation of these steps.)

Archive from CatDV to XenData

• 1. A user flags assets in CatDV to send to XenData archive by setting a metadata value in a field to “send-to-archive” (or similar)

  2. May also be triggered from Worker action based on metadata received from another workflow (e.g., an ingest operation may auto-set the archive metadata as part of the workflow)

  3. If XenData has multiple File Groups, the user can select a specific File Group from a pull-down menu

     1. If the File Group is not set by the user prior to submitting the Archive request, the default File Group specified in the system configuration will be used

     2. Each file can only be sent to a single File Group

  4. The CatDV Worker Node is triggered by a server query to send the asset to the CatDV2XenData app via a Worker Node command

  5. Assets are “queued” for archiving by the CatDV2XenData app

  6. Every five minutes queued assets are submitted to the XenData archive system as individual jobs by CatDV2XenData

  7. The XenData Server is queried for asset status

  8. If asset(s) are already in XenData, then:

• • 1. This info will be added to the XD Log Field in CatDV

    2. The XD Monitor State field will be updated to show “archived” (or similar)

    3. The asset(s) will not be sent to XenData

Note: In a XenData system using the Workflow API, if a job is submitted for an asset that exists in the system, XenData will fail the job and report an error. CatDV2XenData is designed to check and catch duplicate archive jobs to prevent this from happening in the case of a user submitting an asset to an archive more than one time.

• 1. If asset(s) are not in XenData, then the asset(s) are sent to XenData

  2. A unique Job ID is created for the asset by XenData

  3. The Job ID is written to the XD Job ID field in CatDV

     1. Example: 503BBCDE-56F1EB0B-00-00000012

Note: This ID may refer to a single media file or to the elements of a metaclip. Metaclips in CatDV have a single XD Job ID.

• 1. The archive status is updated in CatDV for each asset as its job progresses

  2. The XD Monitor Status field in CatDV lists the status of each submitted asset

     1. Example: “Complete”, “In-Progress” (or similar)

  3. If the archive action is successful, the asset’s XD Log field is updated with job information, and the operation is complete.

Log Example: 2016-03-22T17:28:39.803021 - Archive - Archive of asset 5177FE04 complete

2016-03-22T17:45:54.185589 - Restore - Asset 5177FE04 - Restore mode - Fault (Code:80): The file exists

2016-03-22T19:07:56.136539 - Restore - Restore of asset 5177FE04 complete

• 1. If an archive job fails, the XD monitor State field will display a “Fault” message. The CatDV XD Log Field will list more detailed error information.

  2. Operation complete

Chapter 4 - Worker Node

The CatDV Worker Node application is used to initiate archive actions. When the system is integrated, three Worker Node actions are set up:

• • Queue Archive to XenData

  • Queue Restore from XenData

  • Queue Delete from XenData

Worker Node Actions

See the Appendix for example screenshots of these actions.

Note that the Worker Node does not actually perform the archive operations, nor communicate directly with the XenData server. The Worker Node merely serves as a bridge from the CatDV Server to the CatDV2XenData application to queue an action. Once queued, all further actions are performed by CatDV2XenData or the XenData application. This leaves the Worker Node free to perform other automations while archive operations are running.

The CatDV2XenData application uses XenData's API and CatDV Server's REST API to pass information between the systems. The Worker Node CLI is not used in this workflow.

See the Worker Node Read Me doc for more information on managing and troubleshooting Worker Node.

This concludes the summary of the CatDV Worker Node and its roles in CatDV and XenData archiving operations.

For more detailed information on your CatDV system, please contact Square Box Systems or use the online support resources located at:

http://www.squarebox.com/support/

.

Chapter 5 - XenData

Xendata System Configuration Application

While a full discussion of XenData administration is beyond the scope of this document, we have included a quick overview of the main screens that a user might access while troubleshooting a system.

The XenData System Configuration application provides a way to configure the XenData system and view the system status. If you are seeing XenData errors reported in your asset records, access the XenData server either physically, or via remote access to check the system status, storage mounts, and error logs.

Note: Be careful to not open the Windows Explorer window for the XenData cache while the archiving operations are in progress. This may cause the archive operations to fail.

To access the XenData System Configuration application, launch it via the shortcut on the desktop or from the Start Menu as shown below.

Xendata System Configuration Interface

This is the default screen for the Xendata System Configuration Application. If you do not see the full list in the sidebar, click the folder icon to expand the list.

File Group Configuration

This screen shows the available XenData File Groups. This example shows a File Group named "CATDV" highlighted in the sidebar that is located on the E: drive. The example is set to use the CATDV Volume Set.

Volume Set Configuration

This screen shows the available XenData Volume Sets/Groups. The CATDV Volume Set is highlighted in the sidebar for viewing or editing.

LTO Tape Info

This screen shows the information for a specific LTO tape in the CatDV Volume Set.

XenData System Log

This screen shows the available XenData System Log. Note that it is accessed by clicking the Diagnostics>System icons in the sidebar.

Click the "System" icon to monitor the XenData processes in a log window. This will show the XenData system logs for the LTO hardware and data transfers.

XenData Drive Configuration

This screen shows the XenData Library configuration. This information is accessed by clicking the Diagnostics>Library icons in the sidebar.

Note: Your system configuration may vary.

Drive 0 Info

This screen shows the XenData Drive information. This  is accessed by clicking the Diagnostics>Library>Drive icons in the sidebar.

Note: Your system configuration may vary. This system has two LTO drives installed in a tape library.

Drive 1 Info

This screen shows the XenData Drive information. This  is accessed by clicking the Diagnostics>Library>Drive icons in the sidebar.

Note: Your system configuration may vary. This system has two LTO drives installed in a tape library.

This concludes our description of the Xendata System Configuration and the screens that may be most useful in troubleshooting CatDV and XenData archiving operations.

For more detailed information on your XenData system, please contact XenData directly or use the online support resources located at:

http://www.xendata.com/support/index.html

Chapter 6 - Troubleshooting

CatDV-XenData Worker Node Queuing Process

If clips in CatDV have a valid "XD Archive Action" and are published to the CatDV Server the next step in the process is for the Worker Node to queue them using the "catdv_to_xendata.exe" application setup in the bulk query "Queue XD Job" Worker Action.

If Worker Node has all "green" status lights and no new Worker tasks are populating in the Worker Node, try first restarting the Worker Node desktop application processes by clicking the "Restart" or quitting the program and reopening it to restart the processes.

If that does not find any Worker tasks for clips in CatDV set with a valid "XD Archive Action" value. Please double check the clips and metadata fields in CatDV to verify they were properly setup.

Windows Task Scheduler - "Process_XD_Queue" Task

The Windows Task Scheduler task "Process_XD_Queue" is used to execute the "catdv_to_xendata_monitor.exe" every 10 minutes which submits queued CatDV clips to XenData for the archive action in the XD Q Mode field.

The "Process_XD_Queue" will run automatically on schedule but can also be "Run" on command to troubleshoot the submission and monitoring process.

Once the task is running a command prompt window will run on the desktop which shows the program running through the queue and monitoring any current CatDV jobs in XenData. Do NOT close this window while running, this will cause the PID to become abandoned as noted above, and will require manually clearing the "catdv_to_xendata_monitor.pid" file.

Windows Task Scheduler - Remove_XD_PID

The Windows Task Scheduler task "Remove_XD_PID" is a troubleshooting on demand task to remove an abandoned "catdv_to_xendata_monitor.pid" if it's expected to be an issue. More information on the PID file can be found below and how to manually remove the PID file.

This should only be executed if the "catdv_to_xendata_monitor.exe" process is not running properly and queued clips in CatDV are not submitting to or being monitored from XenData in the allotted 10 minute period of the "Process_XD_Queue" task.

An easy way to determine if the "Remove_XD_PID" should be executed is by manually clicking "Run" on the task "Process_XD_Queue" in the Task Scheduler. If the command window for the "Process_XD_Queue" quickly opens and closes, then it's safe to assume it executed the program and the program found an existing PID file which would keep the program from fully executing.

This executes the "catdv_to_xendata_monitor.exe" with the "--remove-pid" variable to remove the PID from the defined location in the "catdv_to_xendata.cfg" file, reducing the need for a user to search for the PID file on the file system.

CatDV-XenData Queuing PID File Reset

If the Worker Node is completing CatDV XenData jobs but clips are not showing a "queued" status, it's possible the Worker Node application crashed or the system crashed during the CatDV-XenData queuing process. This would cause the "catdv_to_xendata.pid" to become "abandoned" on the file system. To confirm that the CatDV-XenData queuing app is not queuing due to an existing PID file, check for the "catdv_to_xendata_monitor.pid" file (see location below.)

The PID file (process identifier) is a file that keeps multiple CatDV-XenData processes from overrunning each other. It is a temporary file and will be recreated as needed, so you may safely delete it if the program is no longer running.

The PID file is most commonly stored in the "northshore" folder in the logged in Users folder: C:\Users\Administrator\northshore\catdv_to_xendata.pid

It could also be stored in the Program Files folder with the executable:

C:\Program Files\NSA\CatDVtoXendata\catdv_to_xendata.pid

If you are unsure as to the location of the PID file for your system, it is also noted in the config file of your CatDV-Xendata application:

pid_file_location = C:\Program Files\NSA\CatDVtoXendata\catdv_to_xendata.pid

CatDV-XenData Monitor PID File Reset

If clips in CatDV are showing a status of “queued” but clips are not submitting for archive, restore, metadata or deletion to XenData, (check the XenData appliance interface to confirm that jobs are running) there was likely a XenData Workflow API error or a XenData system restart during the archive process. To confirm that the CatDV-XenData Monitor app is not running due to an existing PID file, check for the "catdv_to_xendata_monitor.pid" file (see location below.)

The PID file (process identifier) is a file that keeps multiple CatDV-XenData processes from overrunning each other. It is a temporary file and will be recreated as needed, so you may safely delete it. Once the file is deleted, allow up to 5 minutes for the CatDV-Xendata process to start again and the system should begin to process jobs as normal. Monitor the “XenData System Log” for “Create”, “Archive”, and “Write” log entries for CatDV assets to see that the system is functional again. NOTE: After a restart, always check the status of all CatDV clips from stalled archive jobs. It may be necessary to re-send clips to the XenData archive if the process failed while processing those clips. 5 to 10 minutes after the process restarts, all clips should transition to status “Waiting” in the “XD Job State” field. The PID file is most commonly stored in the "northshore" folder in the logged in Users folder: C:\Users\Administrator\northshore\catdv_to_xendata_monitor.pid

It could also be stored in the Program Files folder with the executable:

C:\Program Files\NSA\CatDVtoXendata\catdv_to_xendata_monitor.pid If you are unsure as to the location of the PID file for your system, it is also noted in the config file of your CatDV-Xendata application: monitor_pid_file_location = C:\Program Files\NSA\CatDVtoXendata\catdv_to_xendata_monitor.pid

CatDV field "XD Log" for Errors

Please review the "XD Log" field in CatDV for error messages and codes for diagnosing the clip(s) issue for the archive action the system attempted to complete with XenData.

If submitting a ticket please include a screenshot of the "XD Log" field's latest log entry and the copied text from the field for faster first ticket response.

CatDV-XenData Log Code:87 Error

If you are receiving a Code:87 error, this most likely points to the XenData archive system's API not responding. Contact XenData support for instructions on how to restart the API. support@xendata.com

Appendix

Appendix A - CatDV Worker Node Configuration Examples

The following screenshots are included both for reference and educational purposes. Note that your system configuration may vary. If you have any questions about your system configuration contact your system administrator or North Shore Automation support.

They are in order, tab by tab, left to right (in the Worker Node interface) so that you may reference them as you confirm your Worker Node settings.

Trigger Tab

Server Query Tab

Processing Steps Tab

Processing Step: enqueue.exe

This should be set and correct, but you may hit the "Choose" button to view the path.

Processing Step: enqueue.exe

Post Processing Tab

End

Appendix B - System Requirements

• • CatDV Server

    • Confirm the latest version of CatDV Server 6.10.x or 7.1.x is installed on your CatDV Server

    • Confirm a CatDV Web Client license is available on the CatDV Server

    • Confirm and Supply CatDV Administrative login credentials.

    • Confirm and supply test media is available on the Media Storage and catalogued and clearly marked/labeled in CatDV Server

  • XenData Server

    • XenData Server Software version 6.20 build 2366 or later is installed

    • Confirm Full or Restricted CatDV Worker Node version 6.1.x or higher is installed on the XenData Server and licenses are applied or available.

    • Confirm that storage access to the test material that is catalogued in CatDV is available from the XenData Server

      • Confirm the volume is mounted on XenData Server

    • Confirm UNC access to the storage device/media catalogued in CatDV from the XenData Server

    • This must include UNC Login Credentials

  • Network Access

  • Confirm network access to the CatDV Server Web Client from the XenData Server for API communication is available