EUMSSI aims at developing technologies for identifying and aggregating data presented as unstructured information in sources of different nature (text, video, audio and social media), including media from online news providers and broadcasters as well as from social sites (e.g. Twitter, YouTube). This is accomplished thanks to the integration in a cross-modal semantic representation framework of state-of-the-art technologies from a diversity of fields, among which text and audio-visual analysis, semantic enrichment, recommendation and social intelligence. The aggregation of information is carried out in an interactive manner, so that the metadata resulting from one media helps reinforce the analysis results from other media and the resulting system allows for an interactive exploration of the enriched data, while helping discover new, implicit information.