Composers and musicians began using cell phones as musical instruments years ago, such as Golan Levin in Dialtones: A Telesymphony (2001). Lars Oberhaus and Marcus Zihn carried out sound experiments in school projects using the same musical material, ring tones and keyboard noises. (Note 1) The resulting pieces of music were mainly conceptual in character. (Note 2) Research institutions, above all the CCRMA (Center for Computer Research in Music and Acoustics) at Stanford University, also turned their attention to mobile devices at an early stage. In California, one of the first ensembles, the MoPhO (The Stanford Mobile Phone Orchestra, 2007-2010), was founded, in which scientists and students played music on smartphones.

Thanks to mobile processors and large-format displays, smartphones are becoming pocket-sized universal devices similar to computers. Integrated sensors enable operation via a multi-touch screen, GPS or microphone. One in three Germans already owns a smartphone. In two years at the latest, cell phones without screen control should have largely disappeared from the market. Smartphones and tablets are ideal as communication, gaming or reading devices and, last but not least, as musical instruments.

The specific dynamic between those involved is already palpable as they work together to find new ways of creatively dealing with music. To date, music apps have mainly been used in the hobby sector. A number of YouTube videos document the experiments of amateur musicians with simple piano, guitar or drum applications that only rudimentarily reproduce their instrumental role models. However, innovative concepts and the constant further development of instrument applications are also increasingly attracting the interest of professional musicians such as Jordan Rudess. Music apps played on smartphones and tablets can sometimes also be heard on stage.

In addition to the established software companies and manufacturers of musical instruments (Yamaha or Korg), it is primarily hobby programmers who develop music apps. Sales are handled via app stores from Apple (iOS), Google (Android) or Microsoft (Windows 8) via the Internet. Interested parties have a wide range of tools at their disposal, currently over 12,000 for Apple devices and far fewer, around 400, for other platforms. An important technical basis for making music with apps like an instrument is the delay-free sound that only iOS can offer so far. Corresponding requirements have been announced for Android and Windows 8, meaning that an expanded range can also be expected for these platforms.

What makes apps so interesting?
Although artistic practice with music apps is still young, interest in innovative applications is high, as the popularity of relevant videos shows. But what fascinates people about music apps? The widely commented music videos and blog posts, such as those on Palm Sounds, shed light on this. (Note 3) In the following, I refer to the comments on the two music apps TableDrum and impactor.

With the app TableDrum (Note4) any sound, whether drumming on the tabletop or tapping against a metal object, can be synchronized with the digital device and linked to freely selectable drum sounds. In this way, a virtual drum set can be played without having to tap on the device surface itself. The drums played in this way can be heard through headphones. This principle of acoustic control of digital samples is implemented in the app Impactor expanded to include synthesizer elements. The ability to record and sample sounds and noises promises additional creative potential.

In the blog comments, users mainly express their desire to experiment and their joy in trying out something new. Even if some doubt the actual playing accuracy or the actual usefulness of these sound-controlled apps, they emphasize that they support innovative ideas and are interested in their further development. Experts also discuss the strengths and weaknesses of these music apps, analyze pieces of music, make comparisons with previous applications and discuss possible technical alternatives.

The element of exploration and do-it-yourself runs like a guiding principle through all the discussions. The music is created in interaction with the medium. Users want to be musically creative, be confronted with manageable challenges and feel a sense of achievement. The main challenge for app developers is therefore to provide them with easy-to-use instruments. The sound result should be of good quality and entertaining at the same time. Successful apps also offer the option of integrating your own sound material and exporting music productions in order to send them to friends or publish them on the internet.

Innovation instead of digital imitation
The most successful music app to date is GarageBand for iPad, was introduced by Apple for the release of the iPad 2 and has been used specifically for promotional purposes ever since. Equipped with musical instruments such as a guitar, drums or keyboard as well as a sampler, sequencer and effects unit, this app is characterized above all by its extensive functions. However, the conceptual orientation of the instruments offered to the real models quickly reveals limitations, for example in the limited playing surfaces or in the visualization of mechanical processes such as the vibration of strings, which cannot be experienced haptically. The unfortunate conclusion: the app resembles a real instrument, except that it can do much less. Instead of developing innovative concepts that are geared towards the characteristics of the digital device, the programmers strove to emulate the play and sound of the original instruments, a claim that is bound to fail. Making music with music apps should therefore by no means be based on experience with the app. GarageBand for iPad be reduced.

In my opinion, mobile technologies such as smartphones and tablets represent a radical developmental step for musical practice. Difficulties in the realization of precise sound ideas as well as obstacles in the implementation of familiar instruments or playing styles should be understood as a call to find other ways of sound control and new principles of musical structuring. When the special features and strengths of a new medium are used consistently, art forms are created that could not have been realized with conventional means. One trend is that the use of music apps involves forms of musical practice that are individually tailored to the personal needs and skills of the user, turning smartphones into "instruments of use". By flexibly combining different instrumental concepts, they open up a variety of playful applications and creative expression possibilities for the user.

Musical expression thanks to sensors
Ultimately, only a small number of the many music apps available allow music to be used creatively. (Note 5) But which apps turn smartphones and tablets into digital musical instruments?

To answer this question, I would like to take a closer look at the aspect of physical experience when playing an instrument. Body movement can play at least as big a role in making music with mobile digital devices as it does in traditional music-making. Digital sensors are needed for this. These are technical components that record certain physical properties of the environment and convert them into digital data. Depending on the programming, music apps can interpret this data in different ways and provide the user with feedback in the form of an acoustic event or sound modulation.
Compared to laptops or computers, smartphones and tablets have a large number of built-in sensors. In addition to screen control via touch, the microphone, an acceleration sensor in all three axes, a digital compass, the digital camera and a gyroscope, which detects the position of the device in space, are suitable for making music. This makes it possible to incorporate auditory and visual experiences as well as tactile or gestural activities. To illustrate the functionality of individual sensors, a selection of specialized music apps (for iOS) is presented here:

• The multi-touch display has a sophisticated application in the app Pitch Painter. You can create graphic scores with your finger and then play them.
• The integrated camera makes it possible to control virtual musical instruments, which is possible in the app AirGuitar is utilized. You can play air guitar and play various chords.
• The vibration sensor is often used in drum apps. In Samplodica selected samples can be controlled with shaking movements.
• The gyroscope measures the position of the smartphone. The app GyroSynth transforms the smartphone into a kind of "sound glove" by using the change in position to modulate the sound. This allows musical parameters such as volume, pitch or filter settings to be controlled by turning and tilting movements.
• In some music apps, the microphone acts as a wind sensor. The sound is controlled via the volume of the breath on the microphone. The app Wivi Band has 15 modulated wind instruments such as trumpet, saxophone and clarinet.
• The digital compass is used in the app Sound wall is used to control the pitch via the spatial orientation.

Some music apps like ThumbJam combine different sensors with each other. The more sensors are used to consciously control sounds, the more musical the application ultimately becomes. The intuitive control of music interfaces using sensors enhances the musical experience. In addition, music-making is also made available to new target groups, especially people without a musical background or with physical limitations.