BytePlus

The BytePlus Effects extension encapsulates the core APIs of the BytePlus Effects SDK. This guide is provided by Voicemod. Agora is planning a documentation upgrade program for all extensions on the marketplace. Please stay tuned.

The BytePlus Effects extension encapsulates the core APIs of the BytePlus Effects SDK. By calling the setExtensionProperty or setExtensionPropertyWithVendor method of the Agora Video SDK v4.x and passing in the corresponding key and value, you can quickly integrate BytePlus's capabilities.

Taking setExtensionProperty as an example, the key is named after the BytePlus API, and the value wraps either certain or all of the parameters of that API in JSON. When you call setExtensionProperty and pass in the pair of key and value, it is equivalent to calling the corresponding BytePlus API. The case is the same for setExtensionPropertyWithVendor.

Currently, the extension encapsulates part of the APIs of the BytePlus Effects SDK. For details, see the key-value overview.

The development environment has to meet the following requirements:

• Android Studio 4.1 or later.
• A physical device (not an emulator) running Android 5.0 or later.

The BytePlus Effects extension works together with the Video SDK v4.x. Refer to the following doc to integrate the SDK and start a basic video call:

• SDK quickstart

1. In Agora Console, download the Android package of BytePlus Effects.

2. Unzip the package, and save all .aar files to the /app/libs path of your project folder.

3. Contact Agora to get the resource package of the BytePlus extension. Save the files you need to the /app/src/main/assets path of the project folder. For details, see Resource package structure.

4. In the app/build.gradle file, add the following line in dependencies:

   
   

   _1

   implementation fileTree(dir: "libs", include: ["*.jar", "*.aar"])

   
   

5. Import the required classes:

   
   

   _7

   import io.agora.rtc2.Constants;

   _7

   import io.agora.rtc2.IMediaExtensionObserver;

   _7

   import io.agora.rtc2.IRtcEngineEventHandler;

   _7

   import io.agora.rtc2.RtcEngine;

   _7

   import io.agora.rtc2.RtcEngineConfig;

   _7

   import io.agora.rtc2.video.VideoCanvas;

   _7

   import io.agora.rte.extension.bytedance.ExtensionManager;

   
   

This section describes the call sequence of using the extension. For a detailed parameter description, see the API reference.

When initializing RtcEngine, call enableExtension to enable the extension:

_5

private void enableExtension(boolean enabled) {

_5

// Initialize ExtensionManager before calling enableExtension

_5

ExtensionManager.getInstance(mRtcEngine).initialize(this);

_5

mRtcEngine.enableExtension("ByteDance", "Effect", enabled);

_5

}

To initialize the extension, call setExtensionProperty, and pass in the corresponding keys and values:

1. Check the license Set the key as bef_effect_ai_check_license and the value as the path of the license file LicenseBag.bundle.
2. Initialize the effects Set the key as bef_effect_ai_init and the value as the path of the model file ModelResource.bundle and the device name.

Take Android as an example:

_48

private void initExtension() {

_48

String[] resources = new String[] {"byte_dance/LicenseBag.bundle", "byte_dance/ModelResource.bundle"};

_48

loadBundle(resources, () -> {

_48

File destFile = getExternalFilesDir(null);

_48

_48

// Gets the path of the license file

_48

File licensePath = new File(

_48

destFile,

_48

"byte_dance/LicenseBag.bundle/" + io.agora.rte.extension.bytedance.example.Constants.mLicenseName);

_48

_48

// Check the license

_48

try {

_48

JSONObject jsonObject = new JSONObject();

_48

jsonObject.put("licensePath", licensePath.getPath());

_48

setExtensionProperty("bef_effect_ai_check_license", jsonObject.toString());

_48

} catch (JSONException e) {

_48

Log.e(TAG, e.toString());

_48

}

_48

_48

// Gets the path of the model file

_48

File strModelDir = new File(destFile, "byte_dance/ModelResource.bundle");

_48

_48

// Initializes the effects

_48

try {

_48

JSONObject jsonObject = new JSONObject();

_48

jsonObject.put("strModelDir", strModelDir.getPath());

_48

jsonObject.put("deviceName", "");

_48

setExtensionProperty("bef_effect_ai_init", jsonObject.toString());

_48

} catch (JSONException e) {

_48

Log.e(TAG, e.toString());

_48

}

_48

_48

// Set whether the overlay effects can coexist with stickers

_48

try {

_48

JSONObject jsonObject = new JSONObject();

_48

jsonObject.put("mode", 1);

_48

jsonObject.put("orderType", 0);

_48

setExtensionProperty("bef_effect_ai_composer_set_mode", jsonObject.toString());

_48

} catch (JSONException e) {

_48

Log.e(TAG, e.toString());

_48

}

_48

});

_48

}

_48

_48

// Only updates the key and value when calling setExtensionProperty

_48

private void setExtensionProperty(String key, String property) {

_48

mRtcEngine.setExtensionProperty("ByteDance", "Effect", key, property);

_48

}

Call setExtensionProperty, and pass in the corresponding keys and values. You can implement the following functions:

• Set the orientation of the mobile phone
• Set superimposed effects (face beautification, shape beautification, and beauty makeup) and their intensity
• Set stickers
• Set filters and their intensity

You can call the method as needed. For a full list of keys and values, see the API reference.

The complete sample code and project is provided on GitHub:

Platform	Language	Sample project
Android	Java	ByteDance/android

​

This section lists the APIs related to using extensions with the Agora SDK.

• addExtension in the RtcEngineConfig class
• enableExtension in the RtcEngine class
• setExtensionProperty in the RtcEngine class
• onChannelMediaRelayEvent in the IRtcEngineEventHandler class

​

This section might not provide the latest information. You can find a more detailed explanation from Effects Interface - C, where you use the name of the key to search the corresponding API.

To implement the BytePlus Effects extension in your app, pass in the corresponding key-value pair when calling setExtensionProperty or setExtensionPropertyWithVendor:

• key: bef_effect_ai_check_license
• value: licensePath. String. The path of the license file.

See Special effects handle license.

• key: bef_effect_ai_init
• value: The value includes:
  • strModelDir: String. The path of the resource file.
  • deviceName: String. The device name. Generally you can enter an empty string ("").

See Initialize special effects handle.

• key: bef_effect_ai_composer_set_mode
• value: The value includes:
  • mode: Int.
    • 0: Do not allow such coexistence.
    • 1: Allow such coexistence.
  • orderType: Int. The rendering order of effects and stickers. For now you need to set it as 0.

See Set whether the overlay effects may coexist with stickers.

• key: bef_effect_ai_set_orientation
• value: orientation. The parameter type is bef_ai_rotate_type. You can set it to 0 (no rotation), 1（rotate 90 degrees clockwise), 2（rotate 180 degrees clockwise), or 3（rotate 270 degrees clockwise.

See Set the phone angle.

• key: bef_effect_ai_composer_set_nodes
• value: nodePaths. A string array containing the paths of each effect resource.

See Set superimposed effects.

• key: bef_effect_ai_composer_update_node
• value: The value includes:
  • nodePath: String. The path of the effect resource.
  • nodeTag: String. The key of the effect. See Functions of Resource Keys.
  • value: Float. The intensity of a single node for the combined effects.

See Set the intensity of superimposed effects.

• key: bef_effect_ai_set_effect
• value: strPath. String. The path of the effect resource.

See Set stickers.

• key: bef_effect_ai_set_color_filter_v2
• value: strPath. String. The path of the effect resource.

See Set filters.

• key: bef_effect_ai_set_intensity
• value: fIntensity. Float. The value ranges between [0.0, 1.0], where 0 means disabling the filter.

See Set filter intensity.