Akomi Tech FAQ & Operations Guide

Akomi Planning

What sort of Akomi server do I need to support a given number of users?

This comes up often when planning a system and know that you should always work with your system integrator and NSA support to lock down a config for your use, but here are some general guidelines to get you started.

The selection of a server is mostly about bandwidth and storage needs. The CPU for most systems up to 100 users can be most anything modern. We have Mac Minis running as servers at many locations for instance. If you want to look for something heavier, a Linux or Windows server, (Something like an HP DL120 etc.) would be great to run as the VM host.

For storage, if you have the proxy rate and the estimated number of concurrent users you can select storage. For example, let's say the proxy is an H.264 MP4 @ 1Mb per second and you wanted to be able to support 10 concurrent users. (Remember, not all users will be logged in at the same time.) Your drive would need to be able to serve 10Mb per second or roughly 1.25 MB per second. That should be easy for any SATA (or better) drive.

The biggest variable to consider is the site's internet/network connection. If the client has users accessing Akomi over their internal ethernet network, then the system will have no issues as internal LAN's are usually operate at at least 1Gb speed. For out of house users, focus on the upload speed of the connection.

To be safe, we at NSA are usually conservative and take the speed of their upload pipe (let's say 50Mb) and divide it in half (25 users). So if a site is hosting on their own internet connection we'd offer that they'd want a 10Mb upload speed at a minimum (5 concurrent users.) If the system doesn't have to be hosted in house, you are usually fine with any cloud service or data center.

One final note is that users are often overly optimistic about usage. We often find that due to the "check in and log off" nature of media viewing, even in systems with a few hundred users there are often no more than 10 concurrent users at any given time.

We hope this helps, as was mentioned before, you should always work with your system integrator and NSA support to design a server & overall infrastructure to fit your needs.

Why does NSA prefer direct-attached, dedicated storage for Akomi?

Akomi is a web application which is often used by users in multiple locations. It is very hard to schedule maintenance and notify users if a system is to be shut down. Add in several time zones and widely varying user work schedules and there is seldom a “good time” to shut a system off or reboot. If Akomi is running off of a NAS or SAN then if the storage needs maintenance or has an issue, the Akomi system goes offline. If dedicated storage (as well as a dedicated backup system) is configured, Akomi will be available to users even if the SAN/NAS is offline or being restarted.

Can Akomi use proxies stored on our SAN?

Akomi can be configured to store its proxy wherever you choose. However, we recommend a separate storage device for the Akomi system for security purposes. The only rule for Akomi storage is that it be accessible by the Worker Node so that CatDV can deliver files to the import folder. See this article for more information on connecting Akomi to shared storage.

(http://kb.northshoreautomation.com/akomi-portal/akomi-admin-guide-1/akomi-reference-guide#TOC-Shared-Storage-Server-if-applicable-)

Can we do an on-site POC?

We want to do a proof of concept for Akomi. Our IT dept. is concerned about us putting media on your server. Why do you do your POC engagements on a hosted server? Can we do an on-site POC?

The POC process was designed up to be hosted for the client for many reasons and over years of testing. Generally the proof of concept (POC) is performed with non-sensitive publicly available test media so that users can experience the usability of the product without security concerns. During this test period, North Shore engineers will work with the IT/Operations staff to ensure that the client's final installation will meet any and all security requirements.

An on-site POC is possible, at an additional charge, but this has serious downsides. For one, if the system is to be demonstrated outside of the building/LAN it can cause the system to be less secure as IT would have to quickly open ports etc. and this can lead to issues. If the system is not available outside the building then users in other offices/locations cannot access the system for testing.

Additionally, the resources needed from the IT/Operations staff for a short test period tend to be non-trivial and this expenditure of time/money would be necessary before the client would even know if the users liked using the system. For this reason, even if there is to be an on-site POC, we offer that it's best to begin with a simpler hosted POC, using publicly available test material.

How can we manage media availability in Akomi, including media that can only be available for a fixed time?

All media in Akomi can only be added via CatDV or manually through the upload form, and it can only be deleted via CatDV or manually through the “Delete” action menu. You can develop automations that will add or delete media based on a CatDV field setting.

For example, you could set a “Media Expiration” field in CatDV, and populate it with a “Date” value. A CatDV Worker Node script could then automatically check for any records with an a “Media Expiration” date older than the current date, and send Akomi a command to delete that record.

Does Akomi support HTTPS?

Yes, NSA recommends that all Akomi systems be deployed with an SSL Certificate. Contact your system integrator if you need assistance with this installation. (This service can be covered under NSA Advanced Deployment.)

Does Akomi connect to LDAP or Active Directory?

Not directly but this question needs more explanation/qualification in the context of a web application. Most (if not all) IT departments will not allow directory service access outside of the firewall. Single Sign On (SSO) services offer a way to connect a web-app to an organization's internal directory system. Akomi can be deployed to connect to several existing SSO schemes and can be customized to work with most any industry standard SSO system. Please contact your reseller or distributor for more information on this type of integration.

What is the recommended proxy format for Akomi import?

Akomi itself is very flexible about proxy bitrate and size. However, we have found that certain files perform better across multiple platforms and internet connections. Here are the basic settings and follow the link below for more information on choosing a proxy format.

MP4 wrapper
Video Format: H.264
Data Rate: 700-1,000 kbits/sec
Image Size: w: 640 x h: 360
Frame Rate: Current or same as source

http://kb.northshoreautomation.com/catdv-portal/nsa-catdv-faq/proxy-format-recommendations#TOC-MPEG-4-Proxy-Settings

What is the recommended mezzanine or alternate format for Akomi?

As with proxy, Akomi is flexible about formats for mezzanine or alternate formats. Since they are only available for download there is no need to conform them to web standards. However we do impose certain limits and suggest the following as a start point.

Alternate formats should be in an mp4 wrapper and should be under 2GB in size for reliable download via http.

You can use the guidelines for proxy creation (linked above) to set up the basics.

Many clients use the following:

HD - 720p (1280X720) h.264 MP4 at 8Mb-10Mb per second

SD - 480i (720X486) h.264 MP4 at 3Mb-5Mb per second

Again, these are only guidelines and you should test to find the minimum bitrate for the quality you require. Also remember that you may make more than one alternate format available for download although most often users find the above formats sufficient.

Supported image and document formats

Depending upon the workflows and options on your Akomi system the following still image formats should display correctly:

Base Akomi:

Images: GIF, JPG/JPEG, PNG, TIF/TIFF
Documents: PDF, PSD (supported as mezzanine format)

Akomi with Galleries Module:

Images: CR2, GIF, JPG/JPEG, PNG, TIF/TIFF
- Documents: PDF, PSD (supported as mezzanine format)

The browser/viewer behavior will depend on the options installed on your Akomi.

How does Akomi write metadata back to CatDV?

Akomi includes a feature called metadata caching that allows an Akomi system to send metadata changes to a CatDV Server via the CatDV REST API.

Metadata updates are written once per minute to reduce database calls and prevent session conflicts with the CatDV API.

In any of these configurations, should Akomi lose its connection to the CatDV Server (due to network issues or even just CatDV being taken down for maintenance, for example) the metadata cache will store the metadata changes in a local db until the CatDV Server is available again, thus preventing the systems from getting out of sync. (Up to 30 days of changes are stored.)

Do CatDV markers get sent to Akomi? Mine are not showing up as Akomi markers.

Due to the way Akomi and CatDV share data, we cannot currently import your existing CatDV markers into Akomi. However, markers created in Akomi write back to CatDV successfully. Since most Akomi users are publishing to Akomi after production is complete or for review and approval, this has not been a highly sought after feature.

We hope to change this behavior in the future. Please contact NSA for more information or if you would like to discuss accelerating the development of this feature.

Can a user request or access high resolution/master footage from Akomi?

As Akomi is a web-based application and since most "high resolution" footage has a far larger file size than can be downloaded successfully via a web (http) connection, we do not suggest publishing files larger than 2GB to Akomi as alternate file formats for download.

However, Akomi does offer our "Hi Res" feature (described below) which does offer access to high resolution video and audio files.

Akomi Operations

I had someone put clips into Akomi with the wrong Media Group. I deleted and reimported the clips but now I have Media Groups in the "show available" column that I don't want to keep. How do I get rid of them? IS there a config I need to edit?

Once the assets are removed, the Media Groups, as they are called, will disappear on their own. Just select the Media Groups from the pulldown menu and make sure there are no assets left in Akomi for those groups. This may take a minute or two to take effect after the assets are deleted.

When the Worker Node feeder actions were set up, two fields were created in the CatDV db, "Akomi Status" and "Add To Library." I don't see these in Akomi, where are they mapped?The Akomi Status field is used by CatDV Pro to trigger the Worker Node action that feeds Akomi. It is not required for Akomi operation. The "Add To Library" field determines whether an asset is published to the Inbox or Library in Akomi. It should be mapped as the "Import" field in the Advanced Settings panel of Akomi.

The default value in demo systems is CUSTOM19.

How do I instantly import or update assets in Akomi?

Akomi systems update and import assets from the import folder on a 1 minute interval. Sometimes, however, you just don't want to wait. Also, on our cloud demo/POC Akomi systems we don't run the automatic import and updates on a timed schedule like the regular Akomi deployments so we wanted to provide commands (fired by loading these URLs) to perform those actions.

To add new assets to Akomi after the Worker Node has published them to the import folder, use:

http://your-system-name-com/import/import

To update the metadata of assets in Akomi after the Worker Node has published the changes to the import folder, use:

http://your-system-name-com/import/update

How do I make a field searchable (or not) in Akomi?

Using an administrator account, navigate to System Administration>Field Mapping>Filters and then simply drag the fields you would like to make searchable from the Inactive Fields list to the Active Fields list. Reverse this process to make a field not searchable.

For more information on the search fields, please see the linked article in the Akomi Reference Guide.

How does a user access high resolution footage using the Hires Request or Send Hires workflows?

Akomi utilizes the power of CatDV to perform media workflows behind the scenes. There are two main ways in which this works for accessing master or full res video footage and audio files.

Send High Res - When this workflow is enabled and integrated by your system administrator, an option appears in your download menu to "Send High Res." When selected, Akomi sends a metadata update back to the selected record in CatDV Pro. A Worker Node action then copies the high resolution master file to an ftp (or other transfer method) folder, pre-configured for the user who will be granted access to files. Once the copy is completed, the Worker Node emails the user who requested the file notifying them of the file's availability. No staff intervention is required in this workflow.

Request High Res - When this workflow is enabled and integrated by your system administrator, an option appears in your download menu to "Request High Res." When selected, Akomi sends a metadata update back to the selected record in CatDV Pro. A Worker Node action then sends the administrator(s) an email containing the requesting user's information and whatever file info is configured (Clip ID, Name, Path, etc.) This information can then be used to arrange delivery of the file using your chosen method (ftp, physical drive delivery, Aspera, File Catalyst, etc.)

We are currently evaluating adding a File Catalyst and/or Aspera plugin to Akomi for clients who wish to enable direct, high res downloads from the Akomi interface. Please contact your CatDV dealer for more information on this feature.

Troubleshooting

Akomi Import Troubleshooting

(These instructions are video focused but may be adapted for image and audio workflows as needed.)

First, read the section on importing assets to Akomi in our k-base if you need a refresher on what files go where to do what.

http://kb.northshoreautomation.com/akomi-portal/akomi-admin-guide-1/akomi-tech-faq-articles#TOC-Importing-Assets-to-Akomi

Initial Steps

Confirm that the Akomi system is functional
- Log into Akomi using web browser to confirm that the vm/server is operational
- Check the CatDV Worker Node Task Queue for last item submitted to Akomi
Confirm that the asset does not currently exist in Akomi
- Search the filename in both Inbox and Library search
  - NOTE: you must search in both sections of Akomi!

CatDV Worker Node

Check Worker Node for status and job failures
- Confirm 3 green lights for Worker Status in the “Processing” area
- Check for no red items/errors Worker Node in task queue
If there are issues, troubleshoot the Worker Actions with your CatDV support team and re-try the import job

Akomi

Check that Akomi Import folder is mounted and/or accessible to the vm.
- (For Akomi systems using networked storage)
Go to Akomi Import folder and check that last asset sent from Worker Node is present
- XML, MP4 and optional JPG would be in the import folder if Akomi failed to import the asset or if Akomi import was disabled.
If XML, mp4 and .jpg are present, try running Akomi manual import command
- http://<your-akomi-address>/import/import
- If this fails to import the asset, contact NSA support
If this succeeds, try sending other assets from CatDV, wait 5-10 minutes to ensure a full import cycle has run and then contact NSA support

The following steps are optional and more advanced, included here for more detail for advanced users. You may try them if you suspect you have a workflow issue but typically if you have performed the above steps and have not resolved the issue, you should contact your Akomi support team.

If MP4 and JPG are in the import folder, then the asset likely existed in Akomi already and the XML was imported as an update. The media update may have failed. Report to NSA support.
If XML file is alone (no media) in the import folder...
- The Worker Node failed to deliver the media
- The job was sent as an “Update” and either the record did not already exist in Akomi (leaving the XML orphaned in the import folder)
- The update operation has not run either due to schedule or failure of the update job in Akomi.
  - To test this, run the manual update command
  - http://<your-akomi-address>/import/update
If this fails, report to NSA support
If the asset actually exists in Akomi, you may delete the orphaned XML

Playlists

Exported playlist XML is blank

If you export and download an XML file of a playlist and it appears to have no clips (as shown below) please confirm that you have clicked the "Save" button on the playlist and then export it again. It will show your clip list, as expected.

Example Empty XML

<?xml version="1.0"?>

<CLIPS>

</CLIPS>

Articles

Akomi VirtualBox Basic Operations

Akomi VirtualBox Basic Operation Guide

Akomi Server Minimum System Requirements

- Operating System/Hypervisor:
- VMWare

Mac OS X/Linux/Windows running Oracle VirtualBox
- https://www.virtualbox.org
Virtualbox Supported Host Operating Systems
- https://www.virtualbox.org/manual/ch01.html#hostossupport
- Note: Virtualbox installation must support 64-bit guest OSes

Hardware:

2GHz i7 processor (or equivalent) 4 cores min.
- - Note: NSA recommends a Xeon processor (or equivalent) if the server will be hosting several VMs
  - The NSA VM ships with 4 cores enabled by default

- 16GB RAM
- SSD boot drive
- 100GB free space available for vm image
- Examples of examples of minimum VM host hardware:
  - Apple Mac Mini with direct attached Thunderbolt SATA drive for media, same size or larger, direct attached USB 3.0 drive for media backup
  - Windows Workstation or Server with internal SATA drive for media, same size or larger, USB 3.0 drive for media backup
  - Linux Server with internal SATA drive for media, with a USB 3.0 drive of the same size or larger, for media backup
  - Note that these are minimum configurations
    - You may scale hardware to match your needs and Akomi can run on enterprise server hardware or a variety of cloud platforms as well
    - Your reseller/system integrator can assist with matching a hardware solution to your needs and can offer you several levels of hardware solutions

Akomi Server Minimum AWS Recommendation

- EC2 t3.xlarge Instance (4 vCPU & 8GB RAM)
  - Note: Akomi VM ships with a 100GB system disk that will be deployed as an EBS volume. (Akomi VM Boot Volume)
- Minimum 100GB EBS General Purpose SSD for media storage
  - Note: This is a second EBS volume that should be sized to match your proxy and/or media format needs.
  - (See http://kb.northshoreautomation.com/akomi-portal/akomi-admin-guide-1/akomi-reference-guide#TOC-Proxy-Data-Storage-Needs)
  - Typically 1TB per 1,000 hours of footage is a good estimate
- The MySQL database server is running on the VM by default. As your needs scale that can be migrated to RDS to increase speed and durability
  - Recommended: RDS/MySQL multi-AZ t2.medium instance, 5GB
- Security Note: North Shore recommends setting up Security Groups to severely restrict access to the Akomi Server. Web traffic should be open to the world. SSH traffic should be limited to specified sources only. All other traffic should be blocked.

AWS Cost Calculator

https://calculator.s3.amazonaws.com/index.html
- Note: The recommended EC2 instance costs about $40 per month, base, as of 1/2017

CatDV System Requirements:

- CatDV Workgroup Server or better (Enterprise Server recommended)
  - v9.3.2 or higher
- One Fixed or Floating CatDV Web Client (for REST API access to CatDV Server)
- CatDV Worker Node
- Limited network access (LAN or WAN) between the Akomi Server and CatDV Server.
  - (See Akomi Deployment Example Drawings for more info.)

Akomi Storage Requirements:

- Akomi supports single drive, RAID, NAS, or SAN storage solutions
- Akomi requires approximately 1GB per hour of media, using a typical proxy resolution for video

North Shore recommends direct-attached, dedicated storage for Akomi for security and maintenance reasons. However high-availability NAS or SAN systems are also appropriate if IT best practices are observed.

The Akomi storage must be accessible by a CatDV Worker Node so that CatDV can deliver files to the import folder. This delivery can be done via FTP or SCP if Akomi is hosted off-site

Minimum storage speed is dependent on number of concurrently connected users. The following are some general (conservative) guidelines. Know that as demands on the Akomi server increase, users will still be able to access the media but performance may begin to suffer. This linear ramp up allows a system to be purchased in a base config and upgraded as needed. NSA recommends that system designers choose a system sized for a mid-level of estimated need and then monitor the system as users are on-boarded.

Remember that concurrently connected user count may be far below the actual number of credentialed users for the system. NSA typically sees a range of 2% to 10% of concurrent users to credentialed users. (Ex. If there are 100 users in the system, we rarely see more than 10 users connected simultaneously unless the system is used for QC, viewing dailies or other high-usage workflow.) Work with your system designer to estimate your user load.

The following are some example storage platforms and their estimated user capacity:

Internally mounted 3Gbps (bus speed) SATA or SAS drive - 50 concurrent users
SATA drive in external USB 3 enclosure - 50 concurrent users
Small RAID (6-8 drives in raid5) - 200 users
- Ex. Promise Pegasus or equivalent
Large RAID (12-16 drives in raid5) - More users than you likely need or could support with bandwidth. (See Network Bandwidth below.)
- Ex. Promise vTrak, Sonnet Fusion, etc.

NOTE: North Shore recommends that all media volumes are backed up at least daily to either direct connected storage (USB 3.0 or similar) or a NAS/SAN volume.

Akomi Backups

Akomi is setup to export a copy of the MySQL database every night at midnight. The northshore user has a crontab entry setup to trigger the job. By default, the database dump is written to akomi-data folder which is usually located on a volume that's external to the VM. The media is all stored inside the akomi-data folder as well.

For example, the internal path to the MySQL dump is:

/media/sf_akomi-data/mysqlbackup/<YYYY-MM-DD_HH.MM.SS>-akomi-mysql.sql.gz

The 10 most recent database dumps are retained.

Akomi Network Bandwidth Requirements

There are 2 basic types of connections to an Akomi server, in-house via the LAN, and remotely via the WAN (Internet.) NSA cannot accurately publish exact user count data due to the extreme variances in network infrastructure, data connection types, server and application overhead etc. However, the following data should allow you to correctly configure your system.

NSA estimates 200 - 300 users per ethernet port maximum for a LAN connection.
- This estimate is based on the average 1Gb connection being tested to provide 200-600 Mb in a real world environment.
- This is difficult to verify due to the challenges of large scale stress testing.
- The more advanced your server and network hardware the more throughput you can expect.
Contact NSA support for information on configuring high availability environments beyond 100 concurrent users.

Importing Assets to Akomi

Akomi is most commonly fed assets automatically via a CatDV Worker Node action. To create a record in Akomi, at a minimum, a video proxy file and an XML file must be delivered into the import folder. Your system will be configured to feed Akomi by your integrator but the following is an overview of the data flow process behind the actions.

To import an asset in Akomi, the files delivered should be named in the following manner, with the proper 3 letter file extension:

filename.mp4

filename.xml

If desired, you can also send a .jpg into the import folder along with the asset at the time of initial import. This is done if you want to designate a specific thumbnail from CatDV, such as a user defined poster frame. Name the file using the same convention:

filename.jpg

Note: Only .jpg files may be delivered for thumbnails.

If you do not send a thumbnail, then Akomi will create one for you from the video file.

When the system is initially configured, a cron job is set to run the import operation every minute. When this operation runs, Akomi will import all assets for which a proxy and xml exist thus emptying the import folder.

Alternately, you may force the import job to run immediately by navigating to: http://machine.domain.com/import/import

Note: If you are running an Akomi server in demo mode or part of a proof of concept, you may need to manually import your data as these systems often do not have cron configured.

When the import operation runs, the Akomi system will create a database record from the XML metadata, then move the proxy files (and optional jpeg and/or alternate formats) to the “data” folder where they are stored in a replica of the original file path. Finally, the XML file is moved to the “imported” directory located inside the “import” directory.

This concludes the import action. The asset will now be available in the Akomi Inbox or Library, depending on the destination chosen in CatDV Pro.

XML format for Akomi import

As noted above, to create a record in Akomi, the minimum files required are a mp4 formatted proxy file and a CatDV XML record.

In cases where you need to perform an automation step using another system in between sending to Akomi or must transform a CatDV XML for other uses, the following is an example of the bare minimum data for an XML import record.

<CLIPS>

<USER31 name="Media-Group">Media_Group_Name </USER31>

<MEDIAPATH>/Volumes/Media/path/to/file/File_Name.mov</MEDIAPATH>

</CLIP>

</CLIPS>

The filename extension in <NAME> need not match the proxy file extension as CatDV Pro will use and resolve an mp4 proxy for a .mov file. Indeed, no extension is needed so long as the <NAME> in the XML matches the file name sans extension. (Ex. <NAME>File_Name</NAME> would match "File_Name.mov". Also note that the <MEDIAPATH> is the original file path of the high res file from CatDV and there is also no need for its file extension to match, however paths should be unique as duplicate assets are not allowed in Akomi.

NOTE: In WIP workflows, files may be temporary and not published into CatDV Server. In these types of workflows this path is irrelevant other than it will determine where Akomi stores its proxy file.)

The CLIPREF is the CatDV pseudo-GUID and any GUID may be substituted here. (This may also be passed back to CatDV for tracking. This is a little known feature/behavior of CatDV. The CLIPREF is not fixed and indeed some clients generate a true GUID for their assets and deliver them into the CatDV db via API or Worker Node CLI.)

It is important to use a unique CLIPREF/GUID for each asset in Akomi as the system does no allow duplicate records and checks each CLIPREF upon import.

The USER fields above map into the Akomi metadata schema and the values will change based on the settings in your Akomi Advanced Settings panel. While most systems have much more metadata, only these two are required for base functionality. Add USER fields as needed and as mapped in Akomi System Administration.

Optional XML Data:

If you would like the Akomi media player to display source or time of day timecode, you may include the following information:

Page updated

Report abuse