M&E Journal: Balancing Machine Efficiency With the Human Touch in Quality Control

Automation has become an increasingly important aspect of media and entertainment workflows. It allows for more efficiency and quicker turnaround times while taking responsibility for routine tasks, enabling employees to focus on what humans do best.

This holds true when it comes to quality control, the last chance to catch potential distractions before content is released to consumers.

At a quality control facility, as in other areas of post-production, automated processes manage file ingest and egress, inter- and intra- network transfers, offline file testing, work scheduling, as well as other aspects of typical workflows.

These processes can be triggered using various methods — for example, via hot folders, by file types, or by a closer examination of file metadata.

Additionally, during the quality control phase of post-production, automated quality control tools play a significant role in determining a file’s viability for release.

These software QC systems are constantly being developed and have a robust set of tests available.

These tests can be divided roughly into two groups: tests that are suited for automated, offline systems and tests that need human oversight for final review of discovered issues.

There are two main tests that fall squarely in the “let the computer do it” category: photo-sensitive epilepsy (PSE) compliance, and overall audio loudness readings. PSE testing ensures that video content will not contain sequences that may trigger a seizure in sensitive individuals.

Although a human could point out sections of flashiness that may approach or cross acceptable thresholds, automated systems have been developed to accurately test video content against the latest set of PSE compliance standards, which can differ by region.

Similar to PSE compliance, there are various standards describing the acceptable overall average volume level of a feature or episode to ensure consistency in audio volume between different programs, commercials, etc.

Although this began as a broadcast standard, streaming platforms have now adopted these specifications, requiring that their content falls within acceptable limits.

Although overall loudness values can be generated during file playback, there are numerous tools that can derive these values offline, faster than real time. In sum, although humans can collect these values during linear playback, there is no need to assign them this responsibility; if a computer can reliably perform a task, it should.

There are many other additional impairments that can be reported by an automated quality control system, such as audio mutes, black frames, and some video and audio glitches.

However, these issues cannot simply be reported to the content owner or client — they must first be reviewed by an experienced quality control operator for accuracy. There are other tests that can be performed by an automated system that often generate many false positives, such as dead pixel detection.

While software quality control toolmakers are constantly refining their algorithms, if a human operator must spend a great deal of time reviewing reported issues and rejecting most of them, the balance of efficiency has likely tipped towards allowing the expert human operator to determine these issues in a real-time review of the file instead of assigning this to an automated system.

Expert quality control operators, who have honed their eyes and ears over years – or even decades-long careers, can detect a slew of impairments and distractions that are either not consistently detected by automated systems or cannot be detected at all.

As of now, a computer cannot reliably report the reflection of production equipment in a passing car window, for example.

When a popular program is released, the hive mind of the collective audience is quick to report any incongruities, and no content owner wants to be responsible for the coffee cup in a medieval drama.

One obvious benefit of using automated quality control systems is cost-effectiveness, which can be passed along to clients.

Balancing the level of quality control needed with budgetary considerations can determine the level of human engagement required, from a light-touch spot-check to numerous full passes on a variety of video monitors, speaker systems, and video and audio scopes and measurement tools.

As automated systems continue to evolve, they will be increasingly integrated into media and entertainment workflows.

In a file’s journey to the consumer, because of the importance of the quality control stage, it is essential to not overlook the contributions of a skilled human quality control operator in preventing distracting impairments from reaching the viewer.

Ideally, nothing should detract from the vision of the creative team behind the content.

A quality control strategy which balances automated tools with human expertise can help meet this goal.

* By Ramón Bretón, Chief Technology Officer, 3rd i Digital *


Click here to download the complete .PDF version of this article
Click here to download the entire Winter 2022 M&E Journal