arxiv.org web signal

Maswao auto-builds South African Sign Language corpus from news

By Published Updated

TL;DR

  • Maswao assembles a weakly aligned South African Sign Language translation corpus from public news broadcasts with minimal human labelling.
  • The pipeline detects sentence boundaries by using gradient descent to find valleys of non-motion in the cropped signer region.
  • Best Transformer scored BLEU-1 of 13.78 and BLEU-4 of 1.54 on Maswao, beating RNN baselines; ChrF 18.09, ROUGE-L 10.51.

South African Sign Language has a new weakly aligned translation corpus, Maswao ("Signs"), assembled from public news broadcasts with almost no human labelling. The pipeline, described in an arXiv preprint by Thapelo Sindane, Vukosi Marivate and collaborators, runs broadcast audio through Automatic Speech Recognition for text, then uses "gradient descent to locate valleys of non-motion" in the cropped signer region to cut long videos into sentence-level pairs.

The authors frame the work as an attempt to close the gap between a handful of well-resourced sign languages, including "the American, British, and German," and the "over 200 globally recognized sign languages" that "lag behind." Two researchers we track in AI Weekly's Who's Who had already circulated the preprint.

Benchmarking sign-to-text translation on the resulting corpus, the paper reports that "Transformer models proposed in the literature outperform recurrent baselines," with the best Transformer scoring BLEU-1 to BLEU-4 of 13.78, 5.21, 2.52, and 1.54, plus ChrF of 18.09 and ROUGE-L of 10.51. The authors call the numbers low but the task "promising" and warranting "further investigations."

Shared on Bluesky by 2 AI experts

Originally reported by arxiv.org

Read the original article →

Original headline: Maswao: An automatically collected, prepared, and benchmarked dataset for the South African Sign language