Oliver Schreer & Jean-François Macq 
Media Production, Delivery and Interaction for Platform Independent Systems 
Format-Agnostic Media

Presents current trends and potential future developments by
leading researchers in immersive media production, delivery,
rendering and interaction

The underlying audio and video processing technology that is
discussed in the book relates to areas such as 3D object
extraction, audio event detection; 3D sound rendering and face
detection, gesture analysis and tracking using video and depth
information. The book will give an insight into current trends and
developments of future media production, delivery and reproduction.
Consideration of the complete production, processing and
distribution chain will allow for a full picture to be presented to
the reader. Production developments covered will include integrated
workflows developed by researchers and industry practitioners as
well as capture of ultra-high resolution panoramic video and 3D
object based audio across a range of programme genres. Distribution
developments will include script based format agnostic network
delivery to a full range of devices from large scale public
panoramic displays with wavefield synthesis and ambisonic audio
reproduction to 'small screen’ mobile devices. Key
developments at the consumer end of the chain apply to both passive
and interactive viewing modes and will incorporate user interfaces
such as gesture recognition and 'second screen’ devices
to allow manipulation of the audio visual content.

* Presents current trends and potential future developments by
leading researchers in immersive media production, delivery,
rendering and interaction.

* Considers the complete production, processing and distribution
chain illustrating the dependencies and the relationship between
different components.

* Proposes that a format-agnostic approach to the production and
delivery of broadcast programmes will overcome the problems faced
with the steadily growing number of production and delivery
formats.

* Explains the fundamentals of media production in addition to
the complete production chain, beyond current-state-of-the-art
through to presenting novel approaches and technologies for future
media production.

* Focuses on the technologies that will allow for the realization
of an E2E media platform that supports flexible content
representations and interactivity for users.

An essential read for Researchers and developers of audio-visual
technology in industry and academia, such as engineers in broadcast
technology companies and students working toward a career in the
rapidly changing area of broadcast both from a production and an
engineering perspective.

Spis treści

List of Editors and Contributors xiii

List of Abbreviations xvii

Notations xxiii

1 Introduction 1
Oliver Schreer, Jean-Francois Macq, Omar Aziz Niamut, Javier Ruiz-Hidalgo, Ben Shirley, Georg Thallinger and Graham Thomas

2 State-of-the-Art and Challenges in Media Production, Broadcast and Delivery 5
Graham Thomas, Arvid Engstrom, Jean-Francois Macq, Omar Aziz Niamut, Ben Shirley and Richard Salmon

2.1 Introduction 5

2.2 Video Fundamentals and Acquisition Technology 7

2.3 Audio Fundamentals and Acquisition Technology 21

2.4 Live Programme Production 34

2.5 Coding and Delivery 43

2.6 Display Technology 50

2.7 Audio Reproduction Technology 56

2.8 Use of Archive Material 62

2.9.1 Limitations of Current Production and Delivery Approaches 65

2.10 Conclusion 68

3 Video Acquisition 74
Oliver Schreer, Ingo Feldmann, Richard Salmon, Johannes Steurer and Graham Thomas

3.1 Introduction 74

3.2 Ultra-High Definition Panoramic Video Acquisition 75

3.3 Use of Conventional Video Content to Enhance Panoramic Video 94

3.4 High Frame Rate Video 102

3.5 High Dynamic Range Video 112

3.6 Conclusion 125

4 Platform Independent Audio 130
Ben Shirley, Rob Oldfield, Frank Melchior and Johann-Markus Batke

4.1 Introduction 130

4.2 Terms and Definitions 132

4.3 Definition of the Problem Space 135

4.4 Scene Representation 144

4.5 Scene Acquisition 149

4.6 Scene Reproduction 153

4.7 Existing Systems 156

4.8 Conclusion 161

5 Semi-Automatic Content Annotation 166
Werner Bailer, Marco Masetti, Goranka Zoric, Marcus Thaler and Georg Thallinger

5.1 Introduction 166

5.2 Metadata Models and Analysis Architectures 170

5.3 Domain-independent Saliency 177

5.4 Person Detection and Tracking 180

5.5 Online Detection of Concepts and Actions 189

5.6 Supporting Annotation for Automated Production 195

5.7 Conclusion 204

6 Virtual Director 209
Rene Kaiser and Wolfgang Weiss

6.1 Introduction 209

6.2 Implementation Approaches 219

6.3 Example Architecture and Workflow 225

6.4 Virtual Director Subprocesses 230

6.5 Behaviour Engineering: Production Grammar 237

6.6 Virtual Director: Example Prototype 243

6.7 Conclusion 251

7 Scalable Delivery of Navigable and Ultra-High Resolution Video 260
Jean-Francois Macq, Patrice Rondao Alface, Ray van Brandenburg, Omar Aziz Niamut, Martin Prins and Nico Verzijp

7.1 Introduction 260

7.2 Delivery of Format-Agnostic Content: Key Concepts and State-of-the-Art 262

7.3 Spatial Random Access in Video Coding 267

7.4 Models for Adaptive Tile-based Representation and Delivery 276

7.5 Segment-based Adaptive Transport 281

7.6 Conclusion 294

8 Interactive Rendering 298
Javier Ruiz-Hidalgo, Malte Borsum, Axel Kochale and Goranka Zoric

8.1 Introduction 298

8.2 Format-Agnostic Rendering 299

8.3 Device-less Interaction for Rendering Control 311

8.4 Conclusions 331

9 Application Scenarios and Deployment Domains 337
Omar Aziz Niamut, Arvid Engstrom, Axel Kochale, Jean-Francois Macq, Graham Thomas and Goranka Zoric

9.1 Introduction 337

9.2 Application Scenarios 338

9.3 Deployment in the Production Domain 340

9.4 Deployment in the Network Domain 347

9.5 Deployment in the Device Domain 351

9.5.1 Device Capabilities 351

9.6 Deployment in the User Domain 356

9.7 Conclusion 357

References 357

Index 359

O autorze

Oliver Schreer, Fraunhofer Heinrich-Hertz-Institut, Berlin,
Germany

Dr. Schreer is Scientific Project Manager of the 'Immersive
Media & 3D Video’ Group in the Image Processing Department at
HHI, Berlin. He is responsible for the lead of European Research
projects (e.g. Fascinat E project) and coordinating research on 3D
video processing and immersive media.

Graham Thomas, BBC Research & Development, London,
UK

Dr. Thomas is Principal Research Engineer and Section Lead of
the Production Magic department at BBC R&D. He leads a team of
engineers developing 3D image processing and graphics techniques
for TV production. His work has led to many commercial products,
e.g. the Piero sports graphics system, which won a Queen’s
Award in 2011. He holds 20 patents and is Visiting Professor at
University of Surrey.

Ben Shirley, University of Salford, UK

Ben Shirley is Senior Lecturer at the University of Salford and
Academic Lead for the Digital Innovation Cluster in the
University’s College of Science & Technology at
Media City UK. He has been principal investigator for a range of
broadcast related research projects including leading
Salford’s work in the EU Fascinat E project. He teaches audio
technology and broadcast engineering at undergraduate and
postgraduate level.

Georg Thallinger, Joanneum Research, Graz, Austria

Dr. Thallinger is Head of the research group 'Audiovisual
Media’ at JOANNEUM RESEARCH, Austria. He oversees a group active in
the field of digital media with a focus on video and image
processing as well as metadata for applications in media production
and archiving, film restoration, media monitoring and security. Dr.
Thallinger is involved in the EU Fascinat E project.

Jean-Francois Macq, Alcatel-Lucent, Antwerp, Belgium

Dr. Macq is a Senior Research Engineer at Bell Labs, the
research organization of Alcatel-Lucent. He has been responsible
for several national and international research projects in the
field of video coding and delivery, currently including the EU
Fascinat E project.

Javier Ruiz-Hidalgo, Universitat Politècnica de
Catalunya (UPC), Barcelona, Spain

Dr. Ruiz-Hidalgo is an Associate Professor at the Universitat
Politècnica de Catalunya. He has been involved in various
European Projects as a principal researcher from the Image
Processing Group at UPC, including the EU Fascinat E project.

Omar Niamut, Netherlands Organization for Applied Scientific
Research (TNO), Delft, The Netherlands

Dr. Niamut is a Senior Research Scientist in the 'Media
Networks and Services’ Group at TNO. He has advised the European
Parliament on the harmonization of mobile TV and was editor of the
technology report on mobile TV in the European Commission-assigned
European Mobile Broadcast Council. He participated in ETSI TISPAN
standardization with over 200 contributions on interactive services
for next generation IPTV and advised the Singapore government on
the use of IPTV standards. He has been involved in a number of
European projects, including the Fascinate project. Dr. Niamut
holds 15 patents.