Audio

Reference information

For reference information related to the Pepper audio API, see the following documentation:

• pp::AudioConfig class
• pp::Audio class
• audio_config.h
• audio.h
• PP_AudioSampleRate

About the Pepper audio API

The Pepper audio API lets Native Client modules play audio streams in a browser. To play an audio stream, a module generates audio samples and writes them into a buffer. The browser reads the audio samples from the buffer and plays them using an audio device on the client computer.

This mechanism is simple but low-level. If you want to play plain sound files in a web application, you may want to consider higher-level alternatives such as using the HTML <audio> tag, JavaScript, or the new Web Audio API.

The Pepper audio API is a good option for playing audio data if you want to do audio processing in your web application. You might use the audio API, for example, if you want to apply audio effects to sounds, synthesize your own sounds, or do any other type of CPU-intensive processing of audio samples. Another likely use case is gaming applications: you might use a gaming library to process audio data, and then simply use the audio API to output the processed data.

The Pepper audio API is straightforward to use:

1. Your module creates an audio configuration resource and an audio resource.
2. Your module implements a callback function that fills an audio buffer with data.
3. Your module invokes the StartPlayback and StopPlayback methods of the audio resource (e.g., when certain events occur).
4. The browser invokes your callback function whenever it needs audio data to play. Your callback function can generate the audio data in a number of ways—e.g., it can generate new data, or it can copy pre-mixed data into the audio buffer.

This basic interaction is illustrated below, and described in detail in the sections that follow.

Digital audio concepts

Before you use the Pepper audio API, it’s helpful to understand a few concepts that are fundamental to how digital audio is recorded and played back:

sample rate

the number of times an input sound source is sampled per second; correspondingly, the number of samples that are played back per second

bit depth

the number of bits used to represent a sample

channels

the number of input sources recorded in each sampling interval; correspondingly, the number of outputs that are played back simultaneously (typically using different speakers)

The higher the sample rate and bit depth used to record a sound wave, the more accurately the sound wave can be reproduced, since it will have been sampled more frequently and stored using a higher level of quantization. Common sampling rates include 44,100 Hz (44,100 samples/second, the sample rate used on CDs), and 48,000 Hz (the sample rate used on DVDs and Digital Audio Tapes). A common bit depth is 16 bits per sample, and a common number of channels is 2 (left and right channels for stereo sound).

The Pepper audio API currently lets Native Client modules play audio streams with the following configurations:

• sample rate: 44,100 Hz or 48,000 Hz
• bit depth: 16
• channels: 2 (stereo)

Setting up the module

The code examples below describe a simple Native Client module that generates audio samples using a sine wave with a frequency of 440 Hz. The module starts playing the audio samples as soon as it is loaded into the browser.

The Native Client module is set up by implementing subclasses of the pp::Module and pp::Instance classes, as normal.

class SineSynthInstance : public pp::Instance {
 public:
  explicit SineSynthInstance(PP_Instance instance);
  virtual ~SineSynthInstance() {}

  // Called by the browser once the NaCl module is loaded and ready to
  // initialize.  Creates a Pepper audio context and initializes it. Returns
  // true on success.  Returning false causes the NaCl module to be deleted
  // and no other functions to be called.
  virtual bool Init(uint32_t argc, const char* argn[], const char* argv[]);

 private:
  // Function called by the browser when it needs more audio samples.
  static void SineWaveCallback(void* samples,
                               uint32_t buffer_size,
                               void* data);

  // Audio resource.
  pp::Audio audio_;

  ...

};

class SineSynthModule : public pp::Module {
 public:
  SineSynthModule() : pp::Module() {}
  ~SineSynthModule() {}

  // Create and return a SineSynthInstance object.
  virtual pp::Instance* CreateInstance(PP_Instance instance) {
    return new SineSynthInstance(instance);
  }
};

Creating an audio configuration resource

bool SineSynthInstance::Init(uint32_t argc,
                             const char* argn[],
                             const char* argv[]) {

  // Ask the browser/device for an appropriate sample frame count size.
  sample_frame_count_ =
      pp::AudioConfig::RecommendSampleFrameCount(PP_AUDIOSAMPLERATE_44100,
                                                 kSampleFrameCount);

  // Create an audio configuration resource.
  pp::AudioConfig audio_config = pp::AudioConfig(this,
                                                 PP_AUDIOSAMPLERATE_44100,
                                                 sample_frame_count_);

  // Create an audio resource.
  audio_ = pp::Audio(this,
                     audio_config,
                     SineWaveCallback,
                     this);

  // Start playback when the module instance is initialized.
  return audio_.StartPlayback();
}

Creating an audio resource

Once the module has created an audio configuration resource, it can create an audio resource. To do so, it instantiates a pp::Audio object, passing in a pointer to the module instance, the audio configuration resource, a callback function, and a pointer to user data (data that is used in the callback function). See the example above.

Implementing a callback function

The browser calls the callback function associated with an audio resource every time it needs more samples to play. The callback function can generate new samples (e.g., by applying sound effects), or copy pre-mixed samples into the audio buffer. The example below generates new samples by computing values of a sine wave.

The last parameter passed to the callback function is generic user data that the function can use in processing samples. In the example below, the user data is a pointer to the module instance, which includes member variables sample_frame_count_ (the sample frame count obtained from the browser) and theta_ (the last angle that was used to compute a sine value in the previous callback; this lets the function generate a smooth sine wave by starting at that angle plus a small delta).

class SineSynthInstance : public pp::Instance {
 public:
  ...

 private:
  static void SineWaveCallback(void* samples,
                               uint32_t buffer_size,
                               void* data) {

    // The user data in this example is a pointer to the module instance.
    SineSynthInstance* sine_synth_instance =
        reinterpret_cast<SineSynthInstance*>(data);

    // Delta by which to increase theta_ for each sample.
    const double delta = kTwoPi * kFrequency / PP_AUDIOSAMPLERATE_44100;
    // Amount by which to scale up the computed sine value.
    const int16_t max_int16 = std::numeric_limits<int16_t>::max();

    int16_t* buff = reinterpret_cast<int16_t*>(samples);

    // Make sure we can't write outside the buffer.
    assert(buffer_size >= (sizeof(*buff) * kChannels *
                           sine_synth_instance->sample_frame_count_));

    for (size_t sample_i = 0;
         sample_i < sine_synth_instance->sample_frame_count_;
         ++sample_i, sine_synth_instance->theta_ += delta) {

      // Keep theta_ from going beyond 2*Pi.
      if (sine_synth_instance->theta_ > kTwoPi) {
        sine_synth_instance->theta_ -= kTwoPi;
      }

      // Compute the sine value for the current theta_, scale it up,
      // and write it into the buffer once for each channel.
      double sin_value(std::sin(sine_synth_instance->theta_));
      int16_t scaled_value = static_cast<int16_t>(sin_value * max_int16);
      for (size_t channel = 0; channel < kChannels; ++channel) {
        *buff++ = scaled_value;
      }
    }
  }

  ...
};

Starting and stopping playback

To start and stop audio playback, the module simply reacts to JavaScript messages.

const char* const kPlaySoundId = "playSound";
const char* const kStopSoundId = "stopSound";

void SineSynthInstance::HandleMessage(const pp::Var& var_message) {
  if (!var_message.is_string()) {
    return;
  }
  std::string message = var_message.AsString();
  if (message == kPlaySoundId) {
    audio_.StartPlayback();
  } else if (message == kStopSoundId) {
    audio_.StopPlayback();
  } else if (...) {
    ...
  }
}