NSA CatDV FAQ

The following are Frequently Asked Questions about CatDV Pro, Server and Worker Node.

CatDV Server

Q: What server hardware spec should I use to host a CatDV VM?

North Shore offers pre-configured CatDV Server & Worker Node virtual machines built on CentOS Linux with MySQL and Apache installed (for Server) and configured for security and performance.

Operating System/Hypervisor:

OR

CatDV Server Hardware/Resources:

Worker Node Hardware/Resources:

VM Resource Recommendations

Here's a fast general rule set for systems containing under 1 million assets:

CatDV Server

Worker Node

Q: I have to reboot my CatDV Server machine. What apps and processes need to be manually restarted?

If you are running a system with MySQL (recommended for most use-cases) and CatDV Server only on the machine, then all processes should start automatically if your system is properly configured. You may launch the CatDV Control Panel and confirm that all indicators are green to check this.

If you have a Worker Node installed on the same system, that may need to be restarted manually and it needs to be running on a logged in system in order to operate. It's probably good to also check that all storage devices needed for your Worker Node workflows are mounted at this time.

Media Compatibility \ Playback

Q: What camera formats are compatible with the CatDV Pro MXF option and CatDV Pegasus?

This list is provided as a convenience and no guarantees are offered of specific version compatibility. Contact Square Box support for complete and up to date information. This is also not a complete list of all formats supported, only those confirmed by NSA at the time of the update.

Square Box CatDV Media Compatibility Info

NSA lists specific camera models as opposed to bulk designations such as "XDCAM" due to the many variations in camera manufacturer formats and specs. Note that this list should not replace strong pre-sale testing protocols as this data may not be up to date for all formats due to a lack of resources. As noted, this is provided as a courtesy only.

Updated 3/17/2017

*We find good compatibility for playback across the Sony XDCAM line with Calibrated Codecs, but each specific camera should be tested for playback and archive compatibility.

Q: What format should I use for my video proxy files?

CatDV Pro and Akomi are very flexible. Here's an article on some proxy selection guidelines.

Q: What format should I use for dates in CatDV Pro?

ISO 8601

Q: Why can't I see video in CatDV on our server machine? The playback window is black.

If you are trying to play back video on a Mac OS X server that doesn't have a monitor connected, the video window may show up as a blank, black window. Note that this doesn't affect the creation of proxy or any video transcodes, only playback.

To fix this issue, connect a monitor or dongle to "fake" a video connection. Here are some examples:

Newer Tech HDMI Dongle

Newer Tech Mini Displayport Dongle

Q: I can't playback certain videos in CatDV.

If you can't playback videos of a particular format, you may need to install a codec for that format on your machine. Check the Cu

You can find the Supported File Formats list here: https://www.quantum.com/en/service-support/downloads-and-firmware/catdv/

Here is the direct link to the March 2022 version: https://cdn.allbound.com/iq-ab/2022/05/CatDV-supported-file-formats-March-2022.pdf

Q: I need help understanding if/why I need any Calibrated plugins on a Worker Node and/or CatDV clients themselves to view certain formats.

OLD INFO - No longer needed as of 2021 - Please see Supported File Formats list above

First, if you have any question about Codecs vs Wrappers check out this youtube link when you can. It's Michael Kammes' web series on tech and the beginning of this episode gives a great explanation of the topic. 

5 Things on YouTube

CatDV has a somewhat confusingly named feature called "MXF option." This does not include any codecs. It allows CatDV to take all the parts of a camera card/clip, like a P2 clip, with separate audio and video, and gather them into a "metaclip" so you can manage and play them back... if you have the proper codecs installed.

So if you don't have say a codec for P2, then you can't play P2 files even if CatDV makes them into a metaclip. Most NLE's include the popular codecs with them (Apple FCP gives youP2 and XDCAM codecs, for instance) But if you don't have one of those apps installed, like on a Worker Node, then you don't have the codec to playback (and therefore DECode the video in order to COmpress it into a new proxy format. (CO-DEC see how they did that?) Some codecs are freely available like h.264, or ship with your OS but the high end pro stuff does not. Hence, we need to buy Calibrated Codecs (or some other brand.)

NOTE: Tin Man is a whole other subject and so all of this will change in coming months. (9-2018)

 

If you want to check if there are Calibrated codec on a machine, check this link:

https://www.calibratedsoftware.com/supportlibrary_view.php?FAQs_ID=5

Those are the uninstall instructions but if those folders don't exist, then you clearly don't have it installed. If you need help installing, all the info is there and the folks from Calibrated are super cool.

Q: How do I prevent getting duplicate clips in CatDV Pro?

There is a setting that might contribute to this (workflow issues can also be the cause.) We consider it the most important (and potentially the most problematic) setting in CatDV Pro.

In Preferences>Import there is a setting near the bottom of the list that says "Combine duplicates (same In & Out values.)

We recommend that you always leave this checked (unless for some reason that we could not understand, you need duplicates of an asset in a catalog, but hey, what do we know? It's your workflow!)

This makes the system ignore an asset that is already in the catalog. The problem that this prevents is if you are ingesting clips and then walk away or get distracted and forget if you've ingested a particular file or folder. If this is checked, CatDV will simply ignore the assets that are already there. If it is unchecked, CatDV will ADD THE ASSETS A SECOND TIME.

The thing that makes CatDV great is that you can do most anything you want with it. However, sometimes CatDV will allow a user to perform actions that can cause issues in certain workflows. For instance if you have a situation where an archive system is connected to CatDV and that system will fail a job that was previously submitted, you can see that duplicate clips can cause quite an issue.

We recommend you speak with your CatDV integrator about this setting (and all settings in CatDV) to ensure that your system is configured to properly import and process media.

Q: Can I search a field to see if proxy has been made for a clip?

If you're using the CatDV Pro client, with a non-automated proxy workflow, the only way to filter for the existence of a proxy file is to view the "Proxy File Name" field and sort by that in a view.

This field is not available in a Server Query or by the Worker Node as it is based on the CaDV Pro client's access to the proxy location.

Also note that the client can only display that field data if the proxy share/folder/location is mounted. This path is not stored anywhere in the db, (for good reason) as this field's status is conditional, depending on the mount status of the proxy location.

The reason for this is that if you spent a year making proxy in your office and then left the office without that proxy drive, just as an example, your proxy would be "offline" to your client app. If the database reported that the proxy was available, it would actually be wrong in this situation.

There's no way for CatDV Pro to know if that proxy is somewhere waiting to be mounted, or if, for instance, the drive was erased and you actually need to re-make all those proxy files. So the system can't really keep a status for proxy by default. It can only show you the location, if there is one. (The Proxy File Name field does this.)

If you have an automated system, in a stable environment, then you can create and flag a custom user field such as "proxy status" in the image above and know that the Worker Node will always set it when it makes a proxy. This can be done manually, but it can be hard to remember to do reliably. NSA always recommends you establish an automated proxy workflow so that status can be tracked by the automation.

Q: What version of Calibrated software do I need for the CatDV MXF option?

Calibrated import components are required for working with many MXF formats in CatDV Pro. There are many format choices and you should confirm all needed components with your reseller.

However, generally for clients working with Panasonic P2, Sony XDCAM HD, and XDCAM EX, we recommend The classic "Bundle #5" for OS X or Windows.

Those courtesy links will take you to the Calibrated store page, but know that your reseller can likely provide them as well as a part of your CatDV purchase. - North Shore Automation has no affiliation with Calibrated Software.

Q: How do I get rid of my old CatDV Pro Settings?

Some settings persist in CatDV Pro and are unaffected by loading Production Group settings, such as changes to the Built-in Field names. To delete all settings on OS X, simply quit CatDV Pro and go to ~/Library/Application Support/Square Box/ and make sure to delete the settings in there.

NOTE: When you restart CatDV Pro you will need to reenter your license information so be sure to have that handy before you start. Also, your settings files may be named differently. catdv-ppro.settings is the name as of 6-2018

Q: What Settings/Preference files does CatDV put on my system? Or how do I backup CatDV Pro, my CatDV Server db and the Worker Node settings?

A: The following files are the preferences for your CatDV Pro, CatDV Control Panel and CatDV Worker Node. They are located where your OS stores prefs.

CatDV Server Backup

In an environment running CatDV Server all your settings are stored in the CatDV database which can be backed up using the CatDV Control Panel. Click the backup button shown below and save the .sql file to a secure location.

CatDV Worker Backup

The Worker Node stores its settings and job queue in XM files. The location depends on which version of Worker Node you are running. Look in either your OS X home folder located at:

~/ or  ~/Library/Application Support  for later versions of Worker Node.

The files are named:

worker.xml

workset.xml

There may also be backup versions of each in the same directory but there's no need to grab those.

In OS X, the "<home>/Library" folder is hidden, so you will need to use the "Go to folder.." option located under the mac "View" menu. The keyboard shortcut is "shift-command-g."

You can just type in ~/Library/Application Support as shown here and it will take you to the correct location.

 

Q: How can I export Worker Actions to back them up or move them to another Worker?

A: Click the animated GIF below.

Q: I upgraded to CatDV 11. Why does the interface look different?

CatDV Pro upgraded their interface to make things simpler for new users or users who may have one specific role in the workflow. Ironically, this often confuses long time users who are used to the main window in CatDV looking one way.

The "traditional" CatDV Pro interface is easily accessed using the "Advanced" option in the "Workspace" drop down menu in the upper right-hand side of the CatDV Pro window, as shown below.

Q: What characters can I use in a file name to avoid problems?

The short answer for how to name a file is:

"Use standard alpha numeric characters (A-Z, 0-9) underscores and dashes."

View our article, "Invalid characters in file names", for more info.

Of particular note, CatDV does not validate XML on export. If you have invalid characters in any field in CatDV they will be output to XML creating an invalid XML file. This will cause problems if you try to use that XML with any other XML parser. It is vital that you maintain clean filenames and character sets in your metadata.

Q: We have a support and maintenance agreement for our software. Why would we also purchase hourly support time from North Shore?

CatDV and middleware software, by their nature exist in interconnected ecosystems. Maintenance and support contracts cover the software product but often an issue with a connecting system prevents the software from working. If you are the DIY type, it may be fine for North Shore to test our software and say, for example, "you have a problem with the incoming XML format, please fix this or find the problem. CatDV and the middleware is working." However, your staff may not be experts on XML formatting and may require assistance in remediating the problem or even communicating it effectively to the source of the XML, be it another company's product or an outside vendor.

Another common example is in archive workflows where the middleware may work perfectly but there is an issue with the DAM, archive software or hardware and several different skill sets may be required in order to find a solution. North Shore can field a team to address each part of a system with a single support contact.

North Shore likes to offer white glove service where we can solve an issue for our customers end to end, including managing the coordination of various vendors and resources until an issue is solved. This is why a block of support time might be desirable for your system in addition to the standard support and maintenance for your product.

CatDV Worker Node

Q: How do I export or backup my worker actions?

In the Worker UI:

Click "Edit Config"

Select "Watch Actions" or "Graphical" (either view works)

Select all actions in the window

Click the button in the upper right of the window with the arrow to the right ->

(see screenshot)

Save the file to a folder you can access and name it something like "PRM worker settings-<date>.catdv"

You can export a single action or select all actions at once.

You can drag and drop these files to import them to a new worker.

Q: How fast should my system be at encoding video? How can we speed up transcoding. Hardware, software, etc?

There's really no answer for all situations, but generally, for most codecs in modern times, we assume real time performance when designing a system. That said, we've seen up to 4X real time on certain codecs and far faster with others.

The answer to speed is "both", hardware and software. On hardware, while CPU is important, most CPUs now can far outstrip gigabit ethernet's capabilities to deliver data, so if the file isn't on a fast raid you probably can't saturate your CPI power. We'd start by experimenting with storage upgrades.

Additionally, companies like Adobe, Telestream, RED etc make use of GPU acceleration and also write their own encoding software for a reason. No one would buy a $30,000 transcoding server if it wasn't faster than ffmpeg of course.

It really comes down to what you're starting from and where you're going. If you're starting from a very compressed file, that requires a processing intensive DEC-ompression before it can be CO-mpressed again. That will cost you time. Many editorial teams start from a ProRes master that is far less compressed so those should generally move faster. If you are going from say h.264 to another MPEG format, your times will be longer.

Generally we start from this data when planning a system:

This data will get the process started. For example, one of our clients has a system that will encode a 60 minute ProRes file in 6 minutes to 1.5 Mb h.264 mp4 files using 3, 12 core Mac Pros and fairly inexpensive software. That same solution won't work for MPEG2 or more broadcast-centric formats.

Final consideration. If you are a "broadcaster", technically you have a legal requirement to have an MPEG license to create files for distribution. If you are creating MPEG formats (such as h.264) for use on the web you may want to confirm that the software you are using has you covered. Commercial software contains those licenses, in most cases.

We're happy to help, if you want to provide some more info, but if you just want to try to boost your current solution, you could start by speeding up your storage for the incoming and outgoing files.

Q: How do I set job priority on the fly in the Worker Node?

When setting up a Worker Action you can choose a priority for the action. However, often you need to prioritize a set of jobs on the fly to meet a deadline or fulfill a special request. ("The boss wants this now!") Here are the steps to do this:

First log into the desktop of the Worker Node (or use your Worker Manager, if you have this.)

Optional - You may want to stop the "Worker" with the 3rd checkbox in the upper left to make managing the queue easier. If you do this make sure you recheck this after you make your changes!!

If you stopped the Worker Process, make sure to restart it!

Q: How do I save CatDV Pro log files?

When requesting support for CatDV issues, a first step is saving the CatDV Pro error logs. Once you have the logs, Square Box and North Shore support team can analyze them and find the appropriate solution.

CatDV Desktop Client

To save the error logs, go to the CatDV Pro/Pegasus help menu and select "Get Error Log".

That will open the Error Log window. Click “Save Log File…” and save the log into a directory where you can find it to attach to an email.

CatDV Server

To save the error logs for your CatDV Server:

1. Open the CatDV Server app/panel on the system running the CatDV Server application.

2. Click the "Server Log" button in the 3rd panel from the top.

3. Click the "Save Log Files..." button in the top right of the window that opens.

CatDV Worker

To save the error logs for your CatDV Worker:

1. Open the CatDV Worker app/panel on the system running the CatDV Worker application.

2. Click the "View Log" button in the lower right hand corner of the main app window

3. Click the "Save Log Files..." button in the top right of the window that opens. (Similar to Server window shown above

Log Paths

A simple trick for finding logs in the CatDV Server, Worker and Desktop is to open the log window in any application and hover the mouse on any log file you want to find on the system and a pop up will display the path for the log.

Q: How do I make MySQL auto-start on OS X 10.10 Yosemite?

This came to us from John over at Squarebox. This is his article below.

There is an issue on newer OS X where MySQL does not auto-start on reboot. This is how to fix it (taken fromhttp://stackoverflow.com/questions/26461173/autostart-mysql-server-on-mac-os-x-yosemite )

 This is how MySQL autostarts again on Yosemite:

Create com.mysql.mysql.plist in /Library/LaunchDaemons containing the following:

 

-8"?>

<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">

<plist version="1.0">

  <dict>

    <key>KeepAlive</key>

    <true/>

    <key>Label</key>

    <string>com.mysql.mysqld</string>

    <key>ProgramArguments</key>

    <array>

    <string>/usr/local/mysql/bin/mysqld_safe</string>

    <string>--user=mysql</string>

    </array>

  </dict>

</plist>

 

Change the permissions:

sudo chown root /Library/LaunchDaemons/com.mysql.mysql.plist

sudo chgrp wheel /Library/LaunchDaemons/com.mysql.mysql.plist

sudo chmod 644 /Library/LaunchDaemons/com.mysql.mysql.plist

Finally run this command:

sudo launchctl load -w /Library/LaunchDaemons/com.mysql.mysql.plist

Q: Can my in-house developers create an XML so that we can import assets into CatDV with metadata? I'd love to import assets from another DAM or our Filemaker database for instance.

Here's a great article from Squarebox on how to create XML sidecar files for import into CatDV.

http://www.squarebox.com/xml-sidecar/

If you have a file path for the files, you can also create a batch file to import many assets with one XML file.

http://www.squarebox.com/workercatdv-xml-batch-files/

Note that North Shore also provides custom XML creation and coding for these sorts of operations if you don't have your own developers on staff.

Q: How do I create a Google Account from my existing email address? 

If you're starting a project with North Shore, you may be asked to provide a Google ID to facilitate document sharing. If you don't want to use a personal gmail or Google ID, you can create a Google account from your existing account by heading over to https://accounts.google.com/sigNup

Once there, you can fill in the information and click "I prefer to use my current email address" on the form, as highlighted in the screenshot below.  Just finish the form, and you're good to go.  You will receive a confirmation email once you complete the form and documents can be shared to your work email address.

Q: Will the HiDef Cowboy perform at my birthday party?

While he can be quite entertaining, the Cowboy is retired from trick roping. But enjoy the following as a consolation.

Click to play

CatDV Security Notices

LOG4J Vulnerability - 12/13/2021 

CatDV Software - In response to the recent security notice on log4j we are relieved to report that CatDV does not use log4j so no regular CatDV installations are affected.

From Quantum CatDV team:

CatDV does not use log4j in any of our standard products (CatDV desktop, CatDV Server or CatDV Worker)

It is possible that some of the third party libraries that are used for optional integrations with external systems may use log4j – so we will need to do an audit of those components. This would only affect customers using those third party integrations.

NSA Software - In addition NSA does not use Log4j in any of our software.

NSA VM deployments - A previous and unaffected version was installed as part of the base CentOS install on some older NSA VMs. It is an older version (1.2.x) and is not impacted by this vulnerability. This vulnerability was introduced in v2.x.

The old version can safely be removed from the VMs without impacting any of the software running on them with the following command:

sudo yum remove log4j

Thank you for joining us in making security a high priority.