#GCT633: Audio and Multimedia Programming KAIST GSCT Fall 2013 Instructor: Dr. Graham Wakefield ([grrrwaaa@gmail.com](http://www.grahamwakefield.net)) Hours: Tue/Thu 14:30 - 16:00 Location: N25 3239 Language: English Credit: 3 Web: http://grrrwaaa.github.com/gct633/ ## Overview This course explores the joy of writing software for audio and media processing, including some of the well-established (and some of the not so well-established) techniques, how to overcome the many challenges it poses, and what potentials for novel research can be encountered. At the same time, the course asks: what does programming for perceptual media imply for creativity, art, design and culture? Historic disciplines of art and music (and theatre, film, sculpture, architecture, engineering, philosophy) converge and diverge into less distinguishable categories, movements, tendencies, moments... some or many of these may become the seeds for great cultural contributions of the future. We aim to nurture these seeds through the application of computation. But like any interaction, passes influences in two directions, so we may also ask how understanding media may influence software and technology development in the future. We can consider elements of time, sound, color, space and form, and their respective syntheses. Each have their own intensive tendencies, peculiar to their format, their history, and our cognitive-perceptive systems. The course will first focus on audio. Although low-dimensional, audio programming can be surprisingly demanding. New data must be generated at reliable, steady rates of forty thousand samples per second or better, and latencies must remain low for musical sensitivity. It occupies a fascinating boundary between the apparently continuous (analog) but actually discrete (digital), and brings software architecture and user-experience issues to the fore in the mapping of perceptual/musical concepts to actual implementations. The course will then broaden to bring these concepts to bear on other media domains, including 2D image processing, 3D rendering and interactive sensor stream processing (such as from the Microsoft Kinect). To approach such a broad subject more concretely we may focus on a particular application areas (to be confirmed). Areas of interest identified by the students include: * Connecting image & sound, music visualization, visual music * Aesthetics of generative art & the computational sublime * Generative programming (metaprogramming) * Live coding * Immersive environments * Responding to audience movement * Speech analysis / spectromorphology ### Format Theoretical issues, technical models, and exemplary works are presented in lecture sessions, and implemented during lab sessions. Toward the end of the course lab sessions will be oriented towards helping students with their final projects. Lecture and course materials, including the software engine, will be presented and disseminated through the class web-site. ### Objectives By the end of the course, students will: * Understand the many themes of audio and multimedia programming, how they have been applied in interactive arts, and how it may be extrapolated into future culture technologies. * Understand the many themes of audio and multimedia programming, how they have been applied in generative and interactive arts, and how it may be extrapolated into future culture technologies. * Be able to apply these techniques for tasks of problem-solving as well as interactive and creative works, as demonstrated in a project portfolio. ### Prerequisites The course is designed to be suitable to students with different levels of technical expertise, but prior programming experience is assumed. ### Assignments & final project Assignments must be completed individually. Collaboration is permitted for final projects, however each student must be able to clearly identify which parts of the project they were responsible for. The final project may extend existing work from one (or more) of the assignments, but will be evaluated only according to the new contributions. ### Grading * The followings evaluation criteria may change: - A. Attendance: 10% - B. Midterm exam: 0% - C. Final exam: 0% - D. Quiz: 0% - E. Report: 0% - F. Assignments: 40% - G. Project: 40% - H. Presentation: 10% * Limitations on course retaking, if any: Some prior programming experience.