SALTA: Open-source tool and database for the segmentation of multimodal performances
- Hosting organisations
- University of Music and Performing Arts Vienna (mdw)
- Responsible persons
- Adrián Artacho
- Start
- End
The SALTA project develops an open-source tool and an associated data infrastructure for the quantitative segmentation of multimodal performances. This tool analyzes and segments performances using accelerometer data, cost-effective 2D video motion capture, and audio data. The project specifically targets researchers and artists who lack access to highly specialized motion capture (MOCAP) laboratory infrastructure, thereby creating new, comparable, and reproducible opportunities for the empirical study of performative practices. Furthermore, SALTA integrates aspects of open science and ethically responsible AI development within the field of Digital Humanities by openly licensing code and documentation and standardizing data access.
About the Project
The core objective of SALTA is to develop a web-based, user-friendly tool for the flexible, parameterized segmentation of performative actions. Users will be able to select parameters—such as body parts to analyze, modalities, or minimum segment durations that govern the segmentation process, enabling the comparison of different analytical perspectives. Technically, the planned implementation consists of three key components: the migration of the locally operated software to a web-based platform with an intuitive graphical user interface (GUI) and tutorials, the establishment of a GDPR-compliant open-access database for storing anonymized performance data and segmentation analyses, and community-building measures to network and further develop the tool (workshops, conferences, collaborations).
Methodologically, SALTA combines data-driven approaches with participatory formats. The database is intended to serve as a foundation for future machine learning models, which can be trained on diverse datasets to increasingly automate and refine semantic segmentations. On the infrastructure level, the project plans a container-based deployment architecture (Docker/Kubernetes) with a Python backend (Django), API access points, and CI/CD pipelines to ensure sustainable development and deployment of the application. Security and operational aspects are accounted for: database access will not be directly public but will occur via token-protected API endpoints of the Django backend; domain and SSL configurations, as well as monitoring and backup strategies, are part of the implementation plan. The implementation timeline is estimated at approximately 6–8 weeks for the first phase after approval, with initial infrastructure work already prepared as part of internal university services.