XReco

Description

While media organizations increasingly support non-linear experiences for the consumer, those are still limited to single channels and media domains. Although several media organizations have recently succeeded in breaking data silos, data sharing is mostly limited to the organization.

XReco will create a new data-driven ecosystem for the media industry, focusing on facilitating data sharing, search and discovery and supporting creation of news and entertainment content. The focus will be on creating and (re-)use of location-related 2D and 3D assets and creating XR experiences.

The ecosystem’s core, represented by a Neural Media Repository (NMR), will foster inter-organisation content sharing and provide increased access to content for media creators, considering novel data monetization and rights management policies. A set of AI-based media transformation services are built around the NMR to produce novel media-and XR experiences, including 3D neural reconstructions, neural based device localisation, image stitching, de-/re-lighting and holoportation.

Within the project, i2CAT leads the design, requirements, and specifications of the XReco technology, including the following objectives:

• Gather, document, validate the requirements for the technology developed and the demonstrators.
• Design of the architecture
• Develop the API specifications
• Guarantee the system integration

The developed technology will be validated in use case scenarios for the news media for XR-based broadcasting and automatic and customized multitarget news publishing, and for location-based information and entertainment content, with applications that also work in tourism and the automotive industry.

XReco will build an enabling technology and pave the way for lifting the use of XR media content from being occasionally involved in media production to being regularly integrated in the media industry.

Estimated impact

XReco aims at enabling efficient media data sharing, discovery, recommendations, transformations, and novel media content creation. Through spatially-present media and inter-organization data sharing based on intertwined emerging technologies (AI, blockchain, VR), XReco will be able to bring innovative solutions to news media technologies, tourism, and the mobility industry. Ingredients which all potentially have a long-term positive impact on European science, technology, economy, and environment.

XReco will provide a solution as an integrated ecosystem, with the aim of empowering the media sector in Europe as well as smaller members of the industry equally. The delivered platform will be composed of a set of horizontal and vertical technologies, with the former addressing data visibility, rights management, monetisation and previewing, and the latter providing a set of data transformation services based on recent, pioneer AI solutions that enable XR compatible content creation with minimal effort:

• XReco’s ecosystem will provide clear incentives for the media industry to be engaged in sharing media content. This will be achieved by delivering a platform that is heavily focused on rights management. To this end, XRECO’s ecosystem will utilise Smart Legal Contracts (SLCs) and blockchain technologies to formalise subject’s rights on digital assets and support their management, as well as game theory algorithms to manage monetisation of derivative digital assets (including data-driven models, and their products/derivatives as well) in a FRAND (Fair, Reasonable and Non-Discriminatory) way.
• XReco will advance current state-of-the-art in Information Retrieval (IR) by developing a solution for media content discovery based on neural network features, utilising Data Adapters for transforming the contributed content to a common cross-modal data model, described by Neural Descriptors.
• XReco will enable next generation news media production, driven by the generation of up-to-date news coverage in a close to real-time manner, using AI-based techniques and user-generated content.
• The solutions will be leveraged for live and offline news media XR productions. The news presenter will be teleported from the studio to a 3D reconstruction of a scene relevant for the news (i.e., a disaster area, concert, or sport event), and the production will be able to show this newly generated content from a potentially unlimited number of viewpoints, with improved detail and with the possibility to place the spectator in an XR immersive experience.
• The platform will facilitate the creation of XR experiences for tourism and in-car entertainment, with unprecedented experience enabled by high-accuracy localisation and the retrieval and adaptation of relevant media assets based on the user location.
• XReco will also facilitate reuse of XR content by developing proper archiving practices, formats, and tools for the heterogeneous set of assets and metadata for XR productions. Improving findability and interoperability of archived XR content also enables reusing of content by other organisations.

Long-term impacts are defined as follows:

• Increased inclusiveness, by supporting a human-centred approach to technology development that is aligned with European social and ethical values, as well as sustainability.
• Sustainable, high-quality jobs by targeting skills by mismatches, the need to empower workers, and ethical considerations relating to technological progress.

Funded by the European Union. Views and opinions expressed are however those of the author(s) only and do not necessarily reflect those of the European Union. Neither the European Union nor the granting authority can be held responsible for them.