aircore logo symbol
aircore logo

Flex SDK for Android: Quickstart

The Aircore Flex SDK for Android allows you to add high-quality, real-time voice chat to your Android app quickly and easily, without the need to understand the complexities of building a real-time system on your own.

Apps that use the Aircore Flex SDK automatically take advantage of Aircore's advanced media optimization and adaptation framework and leverage Aircore's world-wide media distribution system. Taken together, these technologies allow you to provide your users with high-quality, low-latency multi-party voice chat features for groups of up to fifty participants.

Key Concepts

This section introduces some of the key concepts you'll need to understand to use the Aircore Flex SDK for Android. These include the different types of API keys you use to interact with the SDK, as well as Channels, Remote Streams, and Local Streams.

API Keys

You can use two different types of API keys to enable the Flex SDK within your app: Secret API Keys and Publishable API Keys.

Secret API Keys provide the highest degree of flexibility and security, but require additional implementation complexity: to use them, your backend must communicate with the Aircore provisioning service to create Session Authorization Tokens. The Session Authorization Token must then be securely communicated to your client software for use with the Flex SDK.

Publishable API Keys allow you to get your app up and running quickly at the cost of some flexibility and security. These keys can be embedded directly in your app source code and distributed to users. They can then be used directly to initialize the Flex SDK.

More information about API keys and Session Authorization Tokens can be found here.

To get an API key, create an app in the Developer Console.

Copying a publishable API key from the console.

Engine

The singleton provided by Engine.getInstance acts as a central location for configuring the Flex SDK's behavior and for creating Channels.

Channels

Channels represent a virtual space in which your users can interact with each other using the Aircore Flex SDK. Participants in a channel can choose to publish Local Streams with audio from their microphones into the channel, and all participants in a channel can hear the streams contributed by other users.

Channels have names or identifiers that your app defines, and you can control which users are allowed to publish audio to the channel and which users are only allowed to listen.

Remote Streams

Remote Streams are representations of audio streams published by other users within a channel. Your app can mute and un-mute individual remote streams and receive voice activity notifications when the remote participant is speaking.

The Flex SDK receives the audio from each remote stream and automatically plays it out through the active output device.

Local Streams

Local Streams are representations of audio streams published by your app into a channel. Your app can mute and un-mute local streams it publishes, and the mute state will be automatically communicated to subscribers.

The Flex SDK takes care of configuring the user's microphone, capturing audio data, and distributing it to the channel.

Using the SDK

Installation

Adding Maven Central to Your Repositories List

The Aircore Flex SDK versions are distributed using Maven Central. Make sure that you have mavenCentral() in the repositories section of your application's build.gradle or build.gradle.kts file.

// build.gradle

repositories {
    mavenCentral()
}

Adding a Dependency to the Aircore Flex SDK

To make use of the Aircore Flex SDK you will need to specify an implementation dependency in your build.gradle or build.gradle.kts file.

// build.gradle

dependencies {
    implementation "io.aircore.media:android-media:1.0.0"
}

Application Configuration

To publish audio into a Channel, your application must request permission to use the microphone.

See https://developer.android.com/reference/android/app/Activity.html#requestPermissions for general information on how to request permissions.

The specific permission to request is Manifest.permission.RECORD_AUDIO.

Configuring the Engine

The singleton returned by Engine.getInstance will automatically configure itself.

Creating and Joining Channels

The io.aircore.media package provides a singleton via Engine.getInstance that can be used to produce Channel objects.

Channels represent a connection to a real-time session with other users of the app. Joining a channel allows you to hear other users who are live in the same channel and to publish your own audio to the group.

Channels are identified by channel IDs, which are strings that uniquely identify individual channels within your app.

After you create a channel using the Engine's createChannel function, you must retain the resulting object to stay connected. The channel will not receive remote streams until you call the join method.

Only one Channel object should be connected at a time. Call leave on a connected Channel object before join on another.

import io.aircore.media.Channel;
import io.aircore.media.Engine;

// The Session Authorization Token to join the channel with.
String token = "SAT from your backend"

// Set up variables for Notification.
String appName = "";
// Message that will be displayed on the notification when the application is running in background.
String message = "";
@DrawableRes int icon = null;

// Create the channel.
Channel channel = Engine.getInstance(context, appName, message, icon)
    .createChannel(token);
// Check if the channel was successfully created
if (channel == null) {
    // Handle error case.
}

// Join the channel
channel.join();

// Store the channel somewhere safe
channelStorage.channel = channel;

Leaving a Channel

A Channel is terminated when all references to the Channel go out of scope, if an unrecoverable error occurs, or if the leave method is called on the Channel. leave can be called on the Channel even before the Channel is joined. Leaving a channel ends all local and remote streams and releases any system resources used by the channel.

A channel that has been terminated cannot be re-joined. To resume interacting with other users in the channel, you must create a new Channel object and invoke the join method.

Channel Join States

Channels are backed by state machines that control their "join state". A join state represents whether or not a Channel is connected and ready to discover new RemoteStreams.

There are actions that can only be taken in specific join states. For example, one can only create a local stream if the Channel is in either the JOINED or REJOINING states.

You can use the join state of a channel to indicate to your users the progress or condition of their link to the encompassing space they are a part of or attempting to connect to. To be updated on the Channel's join state, you subscribe to the corresponding channel notification:

import io.aircore.media.Channel;

class ChannelDelegate implements Channel.Delegate {
    @override
    void joinStateDidChange(Channel.JoinState newState, Channel.JoinState oldState) {
        switch (newState) {
           case Channel.JoinState.NOT_JOINED:
                // The initial state. The channel has not yet connected.
                // No interaction with other users is possible.
                // NOTE: A notification for NOT_JOINED is not sent to the delegate.
                break;
            case Channel.JoinState.JOINING:
                // The channel is connecting for the first time.
                break;
            case Channel.JoinState.JOINED:
                // The channel is connected and automatically plays remote streams.
                break;
            case Channel.JoinState.REJOINING:
                // The channel connected and then disconnected.
                // It is now reconnecting.
                break;
            case Channel.JoinState.TERMINATED:
                // The channel has permanently disconnected and its resources
                // are being cleaned up. You can check the termination cause
                // here.
                // NOTE: You can't reuse the channel, but you can detect if the
                // channel terminated unexpectedly and either create a new
                // channel or show an error to the user.
                Channel.TerminationCause terminationCause = channel.getTerminationCause();
                break;
            default:
        }
    }
}
ChannelDelegate channelDelegate = new ChannelDelegate();

// Register a delegate with the channel before joining to
// make sure that you receive all notifications.
channel.addDelegate(channelDelegate);

// Start the channel
channel.join();

You can also query the join state directly at any time with Channel.getJoinState.

You can use the join state to build your user experience and UI:

  • In the NOT_JOINED state, you can let the user join the channel. Or, you can have your app automatically join the channel.

  • In the JOINED and REJOINING states, you can let the user publish a local stream. Or, you can have your app automatically publish a local stream.

  • In the JOINING, JOINED, and REJOINING states, you can let the user leave the channel.

  • In the TERMINATED state, you can show the user that the channel is disconnected. You can also let the user create a new channel.

Updating Session Authorization Tokens

When the Session Authorization Token used to join a channel is due to expire, the Channel.Delegate.sessionAuthTokenNearingExpiry method will be called on all registered Channel delegates for that Channel instance. You must replace the expiring token with a new one before the token expires to continue using the channel. Your backend can acquire a new authorization token from the Aircore provisioning service, and your application can provide it to the active channel by calling Channel.setSessionAuthToken.

This example illustrates the process of registering for and handling token expiry notifications:

import io.aircore.media.Channel;

class ChannelDelegate implements Channel.Delegate {
    @override
    void sessionAuthTokenNearingExpiry() {
       String newToken = getTokenFromMyBackend();
       channel.setSessionAuthToken(newToken);
    }
}
ChannelDelegate channelDelegate = new ChannelDelegate();

// Register a delegate with the channel before joining to
// make sure that you receive all notifications.
channel.addDelegate(channelDelegate);

You can also provide an updated authorization token to change a user's permissions within a channel. For example, you can provide a new token to grant a user permission to begin publishing their own audio into a channel, even if they only had permission to listen when they first joined.

Note: The application ID, channel ID, and user ID parameters of the updated authorization token must match the values specified when the channel was first created. If one or both of those parameters does not match, the channel will be terminated.

Working With Remote Streams

After you connect to a channel, your application will begin receiving and playing audio published by other users in the same channel. Each incoming audio stream is represented by a Remote Stream object that allows you to control the playback of audio associated with the stream and to receive notifications about the state of the stream.

The Channel.getRemoteStreams method returns a list of references to each of the remote streams active within the channel. You can use this to show active streams to the user. Your app can receive notifications when streams are added to or removed from this set:

import io.aircore.media.Channel;
import io.aircore.media.RemoteStream;

class RemoteStreamDelegate implements RemoteStream.Delegate {
}
RemoteStreamDelegate remoteStreamDelegate = new RemoteStreamDelegate();

class ChannelDelegate implements Channel.Delegate {
    @override
    void remoteStreamWasAdded(RemoteStream remoteStream) {
        // Register to receive notifications about this stream's state.
        remoteStream.addDelegate(remoteStream);

        // Remember this RemoteStream
        channelStorage.remoteStreams.add(remoteStream);
    }

    @override
    void remoteStreamWasRemoved(RemoteStream remoteStream, RemoteStream.TerminationCause cause) {
        switch (cause) {
            case RemoteStream.TerminationCause.STOPPED:
                // The stream has stopped normally.
                break;
            case RemoteStream.TerminationCause.NOT_FOUND:
                // The stream does not exist when attempting to connect.
                break;
            case RemoteStream.TerminationCause.NOT_TERMINATED:
                // The stream is still active. This will not
                // reported in this notification.
                break;
            case RemoteStream.TerminationCause.INTERNAL_ERROR:
                // The stream encountered a fatal error condition.
                break;
            default:
        }
        // No longer need this RemoteStream
        channelStorage.remoteStreams.remove(remoteStream);
    }
}
ChannelDelegate channelDelegate = new ChannelDelegate();

// Register a delegate with the channel before joining to
// make sure that you receive all notifications.
channel.addDelegate(channelDelegate);

Remote Stream Volume

Volume can be set for an entire channel through Channel.setOutputVolume. This setting affects the mixed audio from all the remote streams for a channel and can be set before or after actually joining a channel.

Remote Stream Notifications and Properties

Remote Streams are represented by RemoteStream objects which provide notifications and properties that allow you to interrogate and manage the stream.

Connection State

The current connection state of a remote stream is accessible via RemoteStream.getConnectionState. This property is observable through notifications:

import io.aircore.media.Channel;
import io.aircore.media.RemoteStream;

class RemoteStreamDelegate implements RemoteStream.Delegate {
    @override
    void connectionStateDidChange(
            RemoteStream.ConnectionState newState,
            RemoteStream.ConnectionState oldState) {
        switch (newState) {
           case RemoteStream.ConnectionState.CONNECTING:
               // The remote stream is connecting.
               break;
           case RemoteStream.ConnectionState.CONNECTED:
               // The remote stream is connected.
               break;
           case RemoteStream.ConnectionState.TERMINATED:
               // The remote stream is terminated and its
               // resources are being cleaned up.
               break;
           default:
        }
    }
}
RemoteStreamDelegate remoteStreamDelegate = new RemoteStreamDelegate();

class ChannelDelegate implements Channel.Delegate {
    @override
    void remoteStreamWasAdded(RemoteStream remoteStream) {
        // Register to receive notifications about this stream's state.
        remoteStream.addDelegate(remoteStream);

        // Remember this RemoteStream
        channelStorage.remoteStreams.add(remoteStream);
    }
}
ChannelDelegate channelDelegate = new ChannelDelegate();

// Register a delegate with the channel before joining to
// make sure that you receive all notifications.
channel.addDelegate(channelDelegate);

Termination Cause

When a remote stream has been terminated, RemoteStream.getTerminationCause provides the reason that the stream ended.

Terminated streams are removed from the channel's remoteStreams set and will be destroyed when your application no longer holds a reference to them.

Mute States

Remote streams can be muted in two ways: by the publisher of the stream or by the local application. These states are tracked by separate properties of the RemoteStream object: remoteAudioMuted and localAudioMuted.

RemoteStream.getRemoteAudioMuted indicates whether the publisher of the remote stream has chosen to mute their audio. When this value is changed, RemoteStream.Delegate.remoteAudioMuteStateDidChange is called for all registered delegates.

RemoteStream.getLocalAudioMuted indicates whether the app has chosen to mute the stream locally. When this property is set, audio sent by the publisher is still received, but it is not played. Apps can mute an individual remote stream using the RemoteStream.muteAudio method. When this value is changed, RemoteStream.Delegate.localAudioMuteStateDidChange is called for all registered delegates.

Voice Activity

RemoteStream.hasVoiceActivity indicates whether or not audible speech has been detected for a remote stream. When voice activity changes, RemoteStream.Delegate.voiceActivityStateDidChange is called for all registered delegates.

Publishing Local Streams

To publish audio to other participants in a channel, your app creates a LocalStream instance using the Channel interface.

You can only create a LocalStream when the channel is in the JOINED or REJOINING state. You can only start a LocalStream when the channel is in the JOINED or REJOINING state and the channel does not already have an active local stream attached.

Local Stream Notifications and Properties

A default LocalStream starts unmuted. A muted LocalStream can be created by configuring LocalStreamParams. After a local stream has been started, it can be muted and un-muted as desired. Muting a stream prevents any audio from being captured and delivered to the channel.

Note: Muting and un-muting a local stream is an asynchronous operation. In particular, un-muting a stream may fail if the authorization token used to join the channel does not grant the user permission to publish audio.

Notifications for a LocalStream's mute state, voice activity, and connection state can all be observed. These notifications are provided to all registered LocalStream.Delegate delegates.

import io.aircore.media.Channel;
import io.aircore.media.LocalStream;
import io.aircore.media.LocalStreamParams;

class LocalStreamDelegate implements LocalStream.Delegate {
    @override
    void audioMuteStateDidChange(boolean muted) {
        if (muted) {
           // The local stream is now muted.
        } else {
           // The local stream is now unmuted.
        }
    }

    @override
    void connectionStateDidChange(
             LocalStream.ConnectionState newState,
             LocalStream.ConnectionState oldState) {
        switch (newState) {
           case LocalStream.ConnectionState.NOT_STARTED:
               // The initial state, prior to starting the stream.
               // A notification for NOT_STARTED is not sent to the delegate.
               break;
           case LocalStream.ConnectionState.CONNECTING:
               // The local stream is connecting.
               break;
           case LocalStream.ConnectionState.CONNECTED:
               // The local stream is connected.
               break;
           case LocalStream.ConnectionState.TERMINATED:
               // The local stream is terminated and its
               // resources are being cleaned up.
               break;
           default:
        }
    }

    @override
    void voiceActivityStateDidChange(boolean isActive) {
        if (isActive) {
            // The currently captured audio contains human speech.
        } else {
            // The currently captured audio does not contain human speech.
        }
    }
}

LocalStreamDelegate localStreamDelegate = new LocalStreamDelegate();

LocalStreamParams params = new LocalStreamParams.Builder().build();

// Make sure that the channel is connected.
Channel.JoinState joinState = channel.getJoinState();
if (joinState != Channel.JoinState.JOINED || joinState != Channel.JoinState.REJOINING) {
    return;
}

// Create a local stream
LocalStream localStream = channel.createLocalStream(params);
if (localStream == null) {
  // Can't create a stream.
}

// Register a delegate with the local stream before starting to
// make sure that you receive all notifications.
localStream.addDelegate(localStreamDelegate);

// Make sure that you have microphone permissions before calling start().
if (ContextCompat.checkSelfPermission(getActivity(),
        Manifest.permission.RECORD_AUDIO) != PackageManager.PERMISSION_GRANTED) {
    return;
}

// Start the local stream.
localStream.start();

// Mute the local stream.
localStream.muteAudio(true);

// Unmute the local stream.
localStream.muteAudio(false);

// Stop the local stream and clean up its resources.
localStream.stop();

Stream Metadata

Stream metadata is any optional information you as the application would like to include with a stream. The metadata can be initialized from an object of key-value pairs where the keys are strings and the values are simple types (string, number, boolean, or null).

To attach metadata to a stream, pass a map to setStreamMetadata on LocalStreamParams.Builder. This metadata is permanently associated with the local stream, and when this stream is received by members of the channel as a RemoteStream object. The metadata can be read using RemoteStream.getStreamMetadata.

The following example shows how to add StreamMetaData to a LocalStream.

import io.aircore.media.Channel;
import io.aircore.media.LocalStream;
import io.aircore.media.LocalStreamParams;
import io.aircore.media.StreamMetaData;

// Define key-value pairs to send to RemoteStream objects.
Map<String, Object> metaData = new HashMap<String, Object>();
metaData.put("stringValue", "hello");
metaData.put("integerValue", Integer(33));
metaData.put("floatingPointValue", Double(33.6));
metaData.put("booleanValue", true);
metaData.put("nullValue", null);

// Create a LocalStream with specific configuration.
LocalStreamParams localStreamParams = new LocalStreamParams.Builder()
    // Set StreamMetadata map to be serialized for LocalStream
    .setStreamMetadata(metaData)
    .build();

if (localStreamParams.getStreamMetaData() == null) {
    // Could not contruct a stream metadata object.
    // For example, the encoded metadata may exceed the 2 kilobyte maximum.
}

channel.createLocalStream(localStreamParams);

The following example shows how to retrieve StreamMetaData from a RemoteStream.

import io.aircore.media.RemoteStream;
import io.aircore.media.StreamMetaData;

StreamMetaData streamMetaData = remoteStream.getStreamMetaData();
if (streamMetaData == null) {
    // No MetaData associated with this stream.
} else {
    Map<String, Object> metaData = streamMetaData.getData();
    if (metaData == null) {
       // MetaData contains no key-value pairs.
    } else {
       // Strings can be directly extracted.
       String s = (String)metaData.getOrDefault("stringValue", String("default"));

       // Be careful to treat numeric MetaData as Number since the underlying type is not guaranteed.
       int i = ((Number)metaData.getOrDefault("integerValue", Integer(0))).intValue();
       double d = ((Number)metaData.getOrDefault("floatingPointValue", Double(0.0))).doubleValue();

       // boolean values may be treated as Boolean.
       boolean b = ((Boolean)metaData.getOrDefault("booleanValue", Boolean(false))).booleanValue();

       // You can just check for the key for key-value pairs with null values.
       boolean hasProperty = metaData.containsKey("nullValue");
    }
}