Akomi Tech FAQ & Operations Guide

Akomi Planning

What sort of Akomi server do I need to support a given number of users?

This comes up often when planning a system and know that you should always work with your system integrator and NSA support to lock down a config for your use, but here are some general guidelines to get you started.

The selection of a server is mostly about bandwidth and storage needs. The CPU for most systems up to 100 users can be most anything modern. We have Mac Minis running as servers at many locations for instance. If you want to look for something heavier, a Linux or Windows server, (Something like an HP DL120 etc.) would be great to run as the VM host.

For storage, if you have the proxy rate and the estimated number of concurrent users you can select storage.  For example, let's say the proxy is an H.264 MP4 @ 1Mb per second and you wanted to be able to support 10 concurrent users. (Remember, not all users will be logged in at the same time.) Your drive would need to be able to serve 10Mb per second or roughly 1.25 MB per second. That should be easy for any SATA (or better) drive.

The biggest variable to consider is the site's internet/network connection. If the client has users accessing Akomi over their internal ethernet network, then the system will have no issues as internal LAN's are usually operate at at least 1Gb speed. For out of house users, focus on the upload speed of the connection.

To be safe, we at NSA are usually conservative and take the speed of their upload pipe (let's say 50Mb) and divide it in half (25 users). So if a site is hosting on their own internet connection we'd offer that they'd want a 10Mb upload speed at a minimum (5 concurrent users.) If the system doesn't have to be hosted in house, you are usually fine with any cloud service or data center.

One final note is that users are often overly optimistic about usage. We often find that due to the "check in and log off" nature of media viewing, even in systems with a few hundred users there are often no more than 10 concurrent users at any given time.

We hope this helps, as was mentioned before, you should always work with your system integrator and NSA support to design a server & overall infrastructure to fit your needs.

Why does NSA prefer direct-attached, dedicated storage for Akomi?

Akomi is a web application which is often used by users in multiple locations. It is very hard to schedule maintenance and notify users if a system is to be shut down. Add in several time zones and widely varying user work schedules and there is seldom a “good time” to shut a system off or reboot. If Akomi is running off of a NAS or SAN then if the storage needs maintenance or has an issue, the Akomi system goes offline. If dedicated storage (as well as a dedicated backup system) is configured, Akomi will be available to users even if the SAN/NAS is offline or being restarted.

Can Akomi use proxies stored on our SAN?

Akomi can be configured to store its proxy wherever you choose. However, we recommend a separate storage device for the Akomi system for security purposes. The only rule for Akomi storage is that it be accessible by the Worker Node so that CatDV can deliver files to the import folder. See this article for more information on connecting Akomi to shared storage.

(http://kb.northshoreautomation.com/akomi-portal/akomi-admin-guide-1/akomi-reference-guide#TOC-Shared-Storage-Server-if-applicable-)

Can we do an on-site POC?

We want to do a proof of concept for Akomi. Our IT dept. is concerned about us putting media on your server. Why do you do your POC engagements on a hosted server? Can we do an on-site POC?

The POC process was designed up to be hosted for the client for many reasons and over years of testing. Generally the proof of concept (POC) is performed with non-sensitive publicly available test media so that users can experience the usability of the product without security concerns. During this test period, North Shore engineers will work with the IT/Operations staff to ensure that the client's final installation will meet any and all security requirements.

An on-site POC is possible, at an additional charge, but this has serious downsides. For one, if the system is to be demonstrated outside of the building/LAN it can cause the system to be less secure as IT would have to quickly open ports etc. and this can lead to issues. If the system is not available outside the building then users in other offices/locations cannot access the system for testing.

Additionally, the resources needed from the IT/Operations staff for a short test period tend to be non-trivial and this expenditure of time/money would be necessary before the client would even know if the users liked using the system. For this reason, even if there is to be an on-site POC, we offer that it's best to begin with a simpler hosted POC, using publicly available test material.

How can we manage media availability in Akomi, including media that can only be available for a fixed time?

All media in Akomi can only be added via CatDV or manually through the upload form, and it can only be deleted via CatDV or manually through the “Delete” action menu. You can develop automations that will add or delete media based on a CatDV field setting.

For example, you could set a “Media Expiration” field in CatDV, and populate it with a “Date” value. A CatDV Worker Node script could then automatically check for any records with an a “Media Expiration” date older than the current date, and send Akomi a command to delete that record.

Does Akomi support HTTPS?

Yes, NSA recommends that all Akomi systems be deployed with an SSL Certificate. Contact your system integrator if you need assistance with this installation. (This service can be covered under NSA Advanced Deployment.)

Does Akomi connect to LDAP or Active Directory?

Not directly but this question needs more explanation/qualification in the context of a web application. Most (if not all) IT departments will not allow  directory service access outside of the firewall. Single Sign On (SSO) services offer a way to connect a web-app to an organization's internal directory system. Akomi can be deployed to connect to several existing SSO schemes and can be customized to work with most any industry standard SSO system. Please contact your reseller or distributor for more information on this type of integration.

What is the recommended proxy format for Akomi import?

Akomi itself is very flexible about proxy bitrate and size. However, we have found that certain files perform better across multiple platforms and internet connections. Here are the basic settings and follow the link below for more information on choosing a proxy format.

http://kb.northshoreautomation.com/catdv-portal/nsa-catdv-faq/proxy-format-recommendations#TOC-MPEG-4-Proxy-Settings

What is the recommended mezzanine or alternate format for Akomi?

As with proxy, Akomi is flexible about formats for mezzanine or alternate formats. Since they are only available for download there is no need to conform them to web standards. However we do impose certain limits and suggest the following as a start point.

Alternate formats should be in an mp4 wrapper and should be under 2GB in size for reliable download via http.

You can use the guidelines for proxy creation (linked above) to set up the basics.

Many clients use the following:

HD - 720p (1280X720) h.264 MP4 at 8Mb-10Mb per second

SD - 480i (720X486) h.264 MP4 at 3Mb-5Mb per second

Again, these are only guidelines and you should test to find the minimum bitrate for the quality you require. Also remember that you may make more than one alternate format available for download although most often users find the above formats sufficient.

Supported image and document formats

Depending upon the workflows and options on your Akomi system the following still image formats should display correctly:

Base Akomi:

Akomi with Galleries Module:

The browser/viewer behavior will depend on the options installed on your Akomi.

How does Akomi write metadata back to CatDV?

Akomi includes a feature called metadata caching that allows an Akomi system to send metadata changes to a CatDV Server via the CatDV REST API.

Metadata updates are written once per minute to reduce database calls and prevent session conflicts with the CatDV API.

In any of these configurations, should Akomi lose its connection to the CatDV Server (due to network issues or even just CatDV being taken down for maintenance, for example) the metadata cache will store the metadata changes in a local db until the CatDV Server is available again, thus preventing the systems from getting out of sync. (Up to 30 days of changes are stored.)

Do CatDV markers get sent to Akomi? Mine are not showing up as Akomi markers.

Due to the way Akomi and CatDV share data, we cannot currently import your existing CatDV markers into Akomi. However, markers created in Akomi write back to CatDV successfully. Since most Akomi users are publishing to Akomi after production is complete or for review and approval, this has not been a highly sought after feature.

We hope to change this behavior in the future. Please contact NSA for more information or if you would like to discuss accelerating the development of this feature.

Can a user request or access high resolution/master footage from Akomi?

As Akomi is a web-based application and since most "high resolution" footage has a far larger file size than can be downloaded successfully via a web (http) connection, we do not suggest publishing files larger than 2GB to Akomi as alternate file formats for download.

However, Akomi does offer our "Hi Res" feature (described below) which does offer access to high resolution video and audio files.

Akomi Operations

I had someone put clips into Akomi with the wrong Media Group. I deleted and reimported the clips but now I have Media Groups in the "show available" column that I don't want to keep. How do I get rid of them? IS there a config I need to edit?

Once the assets are removed, the Media Groups, as they are called, will disappear on their own. Just select the Media Groups from the pulldown menu and make sure there are no assets left in Akomi for those groups. This may take a minute or two to take effect after the assets are deleted.

When the Worker Node feeder actions were set up, two fields were created in the CatDV db, "Akomi Status" and "Add To Library." I don't see these in Akomi, where are they mapped?The Akomi Status field is used by CatDV Pro to trigger the Worker Node action that feeds Akomi. It is not required for Akomi operation. The "Add To Library" field determines whether an asset is published to the Inbox or Library in Akomi. It should be mapped as the "Import" field in the Advanced Settings panel of Akomi.

The default value in demo systems is CUSTOM19.

How do I instantly import or update assets in Akomi?

Akomi systems update and import assets from the import folder on a 1 minute interval. Sometimes, however, you just don't want to wait. Also, on our cloud demo/POC Akomi systems we don't run the automatic import and updates on a timed schedule like the regular Akomi deployments so we wanted to provide commands (fired by loading these URLs) to perform those actions.

To add new assets to Akomi after the Worker Node has published them to the import folder, use:

http://your-system-name-com/import/import

To update the metadata of assets in Akomi after the Worker Node has published the changes to the import folder, use:

http://your-system-name-com/import/update

 

How do I make a field searchable (or not) in Akomi?

Using an administrator account, navigate to System Administration>Field Mapping>Filters and then simply drag the fields you would like to make searchable from the Inactive Fields list to the Active Fields list. Reverse this process to make a field not searchable.

For more information on the search fields, please see the linked article in the Akomi Reference Guide.

How does a user access high resolution footage using the Hires Request or Send Hires workflows?

Akomi utilizes the power of CatDV to perform media workflows behind the scenes. There are two main ways in which this works for accessing master or full res video footage and audio files.

Send High Res - When this workflow is enabled and integrated by your system administrator, an option appears in your download menu to "Send High Res." When selected, Akomi sends a metadata update back to the selected record in CatDV Pro. A Worker Node action then copies the high resolution master file to an ftp (or other transfer method) folder, pre-configured for the user who will be granted access to files. Once the copy is completed, the Worker Node emails the user who requested the file notifying them of the file's availability. No staff intervention is required in this workflow.

Request High Res - When this workflow is enabled and integrated by your system administrator, an option appears in your download menu to "Request High Res." When selected, Akomi sends a metadata update back to the selected record in CatDV Pro. A Worker Node action then sends the administrator(s) an email containing the requesting user's information and whatever file info is configured (Clip ID, Name, Path, etc.) This information can then be used to arrange delivery of the file using your chosen method (ftp, physical drive delivery, Aspera, File Catalyst, etc.)

We are currently evaluating adding a File Catalyst and/or Aspera plugin to Akomi for clients who wish to enable direct, high res downloads from the Akomi interface. Please contact your CatDV dealer for more information on this feature.

Troubleshooting

Akomi Import Troubleshooting

(These instructions are video focused but may be adapted for image and audio workflows as needed.)

First, read the section on importing assets to Akomi in our k-base if you need a refresher on what files go where to do what.

http://kb.northshoreautomation.com/akomi-portal/akomi-admin-guide-1/akomi-tech-faq-articles#TOC-Importing-Assets-to-Akomi

Initial Steps

CatDV Worker Node

Akomi

The following steps are optional and more advanced, included here for more detail for advanced users. You may try them if you suspect you have a workflow issue but typically if you have performed the above steps and have not resolved the issue, you should contact your Akomi support team. 

Playlists

Exported playlist XML is blank

If you export and download an XML file of a playlist and it appears to have no clips (as shown below) please confirm that you have clicked the "Save" button on the playlist and then export it again. It will show your clip list, as expected.

Example Empty XML

<?xml version="1.0"?>

<CLIPS>

</CLIPS>

Articles

Akomi VirtualBox Basic Operations

Akomi VirtualBox Basic Operation Guide

Akomi Server Minimum System Requirements

OR

Hardware:

Akomi Server Minimum AWS Recommendation

AWS Cost Calculator

CatDV System Requirements:

Akomi Storage Requirements:

North Shore recommends direct-attached, dedicated storage for Akomi for security and maintenance reasons. However high-availability NAS or SAN systems are also appropriate if IT best practices are observed.

The Akomi storage must be accessible by a CatDV Worker Node so that CatDV can deliver files to the import folder. This delivery can be done via FTP or SCP if Akomi is hosted off-site

Minimum storage speed is dependent on number of concurrently connected users. The following are some general (conservative) guidelines. Know that as demands on the Akomi server increase, users will still be able to access the media but performance may begin to suffer. This linear ramp up allows a system to be purchased in a base config and upgraded as needed. NSA recommends that system designers choose a system sized for a mid-level of estimated need and then monitor the system as users are on-boarded.

Remember that concurrently connected user count may be far below the actual number of credentialed users for the system. NSA typically sees a range of 2% to 10% of concurrent users to credentialed users. (Ex. If there are 100 users in the system, we rarely see more than 10 users connected simultaneously unless the system is used for QC, viewing dailies or other high-usage workflow.) Work with your system designer to estimate your user load.

The following are some example storage platforms and their estimated user capacity:

NOTE: North Shore recommends that all media volumes are backed up at least daily to either direct connected storage (USB 3.0 or similar) or a NAS/SAN volume.

Akomi Backups

Akomi is setup to export a copy of the MySQL database every night at midnight. The northshore user has a crontab entry setup to trigger the job. By default, the database dump is written to akomi-data folder which is usually located on a volume that's external to the VM. The media is all stored inside the akomi-data folder as well.

For example, the internal path to the MySQL dump is:

/media/sf_akomi-data/mysqlbackup/<YYYY-MM-DD_HH.MM.SS>-akomi-mysql.sql.gz

The 10 most recent database dumps are retained.

Akomi Network Bandwidth Requirements

There are 2 basic types of connections to an Akomi server, in-house via the LAN, and remotely via the WAN (Internet.) NSA cannot accurately publish exact user count data due to the extreme variances in network infrastructure, data connection types, server and application overhead etc. However, the following data should allow you to correctly configure your system.

Importing Assets to Akomi

Akomi is most commonly fed assets automatically via a CatDV Worker Node action. To create a record in Akomi, at a minimum, a video proxy file and an XML file must be delivered into the import folder. Your system will be configured to feed Akomi by your integrator but the following is an overview of the data flow process behind the actions.

To import an asset in Akomi, the files delivered should be named in the following manner, with the proper 3 letter file extension:

filename.mp4

filename.xml

If desired, you can also send a .jpg into the import folder along with the asset at the time of initial import. This is done if you want to designate a specific thumbnail from CatDV, such as a user defined poster frame. Name the file using the same convention:

filename.jpg

Note: Only .jpg files may be delivered for thumbnails.

If you do not send a thumbnail, then Akomi will create one for you from the video file.

When the system is initially configured, a cron job is set to run the import operation every minute. When this operation runs, Akomi will import all assets for which a proxy and xml exist thus emptying the import folder.

Alternately, you may force the import job to run immediately by navigating to: http://machine.domain.com/import/import

Note: If you are running an Akomi server in demo mode or part of a proof of concept, you may need to manually import your data as these systems often do not have cron configured.

When the import operation runs, the Akomi system will create a database record from the XML metadata, then move the proxy files (and optional jpeg and/or alternate formats) to the “data” folder where they are stored in a replica of the original file path. Finally, the XML file is moved to the “imported” directory located inside the “import” directory.

This concludes the import action. The asset will now be available in the Akomi Inbox or Library, depending on the destination chosen in CatDV Pro.

XML format for Akomi import

As noted above, to create a record in Akomi, the minimum files required are a mp4 formatted proxy file and a CatDV XML record.

In cases where you need to perform an automation step using another system in between sending to Akomi or must transform a CatDV XML for other uses, the following is an example of the bare minimum data for an XML import record.

<CLIPS>

<CLIP INDEX="0">

<NAME>File_Name.mov</NAME>

<CLIPREF>2E004FA5</CLIPREF>

<USER19 name="Add To Library">Yes</USER19>

<USER31 name="Media-Group">Media_Group_Name </USER31>

<MEDIAPATH>/Volumes/Media/path/to/file/File_Name.mov</MEDIAPATH>

</CLIP>

</CLIPS>

The filename extension in <NAME> need not match the proxy file extension as CatDV Pro will use and resolve an mp4 proxy for a .mov file. Indeed, no extension is needed so long as the <NAME> in the XML matches the file name sans extension. (Ex. <NAME>File_Name</NAME> would match "File_Name.mov". Also note that the <MEDIAPATH> is the original file path of the high res file from CatDV and there is also no need for its file extension to match, however paths should be unique as duplicate assets are not allowed in Akomi.

NOTE: In WIP workflows, files may be temporary and not published into CatDV Server. In these types of workflows this path is irrelevant other than it will determine where Akomi stores its proxy file.)

The CLIPREF is the CatDV pseudo-GUID and any GUID may be substituted here. (This may also be passed back to CatDV for tracking. This is a little known feature/behavior of CatDV. The CLIPREF is not fixed and indeed some clients generate a true GUID for their assets and deliver them into the CatDV db via API or Worker Node CLI.)

It is important to use a unique CLIPREF/GUID for each asset in Akomi as the system does no allow duplicate records and checks each CLIPREF upon import.

The USER fields above map into the Akomi metadata schema and the values will change based on the settings in your Akomi Advanced Settings panel. While most systems have much more metadata, only these two are required for base functionality. Add USER fields as needed and as mapped in Akomi System Administration.

Optional XML Data:

If you would like the Akomi media player to display source or time of day timecode, you may include the following information:

<OUT>1:36:50;00</OUT>

<DURATION>0:00:30;00</DURATION>

<TIMECODE>29.97</TIMECODE>

<IN>1:36:20;00</IN>