Tutorial for NetMesh

Ⅰ、Overviews

The MSDK 2.5 version and the Autel UAV firmware 1.8 version support the netmesh function. It is allowed to add multiple remote controllers and multiple UAVs to the same network. After successful netmesh, there is only one master remote controller and one relay aircraft among these devices, and the other remote controllers are slave remote controllers, and the UAVs are ordinary node aircraft.

Currently, only the stream of one aircraft is supported for viewing. For single-control devices, this device needs to be set as a watch device at the same time.

1.NetMesh Model

2.Noun Explanation

Concept	Explanation	Attention
Netmesh	Multi device add to the same network
DeviceId	Device ID
NodeId	Node ID In this Netmesh
Relay aircraft	Central node aircraft
Master remote controller	The remote controller that initiates netmesh
Slave remote controller	The remote controller joining the net
Single control	Select one aircraft and control it	must set as a watch device
All control	Select allaircrafts and control it
watch	View the video transmission stream of the aircraft

3.Netmesh roles and permissions

Role	Control permission	Role switch	Multi video stream
Master remote controller	Full permissions	Cannot switch	View the video stream of controlled
Slave remote controller	The joystick is unavailable	Cannot switch	View the video streamsof controlled
Relay aircraft	-	master remote controller can switch	Video stream can be pushed to all remote controllers
Leaf aircraft	-	can be switched to the controlledaircraft	Can not push video streams
Controlled aircraft	-	can be switched to the leaf aircraft	Video stream can be pushed to all remote controllers

Ⅱ、Netmesh Interface

1.DeviceManager

DeviceManager is responsible for the creation and management of various devices. The interface related to netmesh operations can be obtained through DeviceManager.getMultiDeviceOperator().

interface IMultiDeviceOperator

    /**
     * get netmesh interface
     */
    fun getNetMeshManager(): INetMeshManager

    /**
     * is single control
     */
    fun isSingleControl(): Boolean

    /**
     * is netmesh version
     */
    fun isNetMesh(): Boolean

    /**
     * listener for control mode change
     */
    fun addControlChangeListener(listener: IControlDroneListener)
    fun removeControlChangeListener(listener: IControlDroneListener)

    /**
     * listener for watch drone change
     */
    fun addWatchChangeListener(listener: IWatchDroneListener)
    fun removeWatchChangeListener(listener: IWatchDroneListener)

    /**
     * obtain all drones
     */
    fun getDroneDevices(): List<IAutelDroneDevice>

    /**
     * obtain all remote controller device info
     */
    fun getRemoteInfoList(): List<DeviceInfoBean>

    /**
     * current control mode
     */
    fun getControlMode(): ControlMode

    /**
     * obtain drone by device id
     */
    fun getDroneDeviceById(deviceId: Int): IAutelDroneDevice?

    /**
     * obtain drone by node id
     */
    fun getDroneDeviceByNodeId(nodeId: Int): IAutelDroneDevice?

    /**
     * get local remote contorller device
     */
    fun getLocalRemoteDevice(): RemoteDevice

    /**
     * obtain all ids of connected device
     */
    fun getConnectedDeviceIds(): List<Int>

    /**
     * obtain keyManagerList by ids
     */
    fun generateKeyManagerList(deviceIdList: List<Int>): MutableList<IKeyManager>

    /**
     * execute keymananger performAction for drone list 
     */
    fun <Param, Result> performActionList(
        deviceIdList: List<Int>, key: AutelKey.ActionKey<Param, Result>, param: Param,
        callback: DeviceManager.CompletionCallbackWithParam<DeviceManager.DeviceActionResult<Result>>?
    )
    /**
     * multi drones monitor for key event
     */
    fun <Result> addDroneDevicesListener(key: AutelKey<Result>, callbacks: DeviceManager.KeyManagerListenerCallBack)
    fun <Result> removeDroneDevicesListener(key: AutelKey<Result>, callbacks: DeviceManager.KeyManagerListenerCallBack)

    /**
     * obtain central drone device
     */
    fun getCenterDroneDevice(): IAutelDroneDevice?

    /**
     * obtain all controlled drone devices
     */
    fun getControlledDroneList(): MutableList<IAutelDroneDevice>

    /**
     * obtain all watched drone devices
     */
    fun getWatchedDroneList(): MutableList<IAutelDroneDevice>

    /**
     * is master remote controller
     */
    fun isMainRC():Boolean

    /**
     * listener of device change for netmesh
     */
    fun addNetMeshChangeListener(listener: IMeshDeviceChangedListener)
    fun removeNetMeshChangeListener(listener: IMeshDeviceChangedListener)
} 

2. Interfaces related to netmesh

Mainly provide interfaces related to netmesh: functions such as initiating netmesh / joining the network, adding, deleting, renaming devices, setting relay devices, setting Watch devices, switching control modes, and dissolving the netmesh, etc.

interface INetMeshManager {

    /**
     * obtain device info
     */
    fun getAllMeshDeviceList():List<DeviceInfoBean>

    /**
     * local remote controller name
     */
    fun getLocalRCName():String?

    /**
     * master remote controller name
     */
    fun getMainRCName():String?

    /**
     * the watched drone of master remote controller
     */
    fun getMainRcWatchDrone():IAutelDroneDevice?

    /**
     * weather in netmeshing
     */
    fun isNetMeshing():Boolean

    /**
     * weather netmesh disband
     */
    fun isMeshDisband():Boolean

    /**
     * start netmesh
     * @param bean :CreateDeviceNetworkReq
     * @param meshMode: configure the max number of device,but final depend on device firmware.
     * @param callback
     */
    fun startNetMeshMatching(
        bean: CreateDeviceNetworkReq,
        meshMode: MeshModeEnum? = MeshModeEnum.STANDARD,
        callback: CommonCallbacks.CompletionCallbackWithParam<CreateDeviceNetworkResp>
    )

    /**
     * finish netmesh
     */
    fun completeNetMeshMatching(
        callback: CommonCallbacks.CompletionCallbackWithParam<Int>
    )

    /**
     * delete device from netmesh
     */
    fun delNetMeshDevice(
        deviceId: Int,
        callback: CommonCallbacks.CompletionCallbackWithParam<Int>
    )

    /**
     * set device as central drone
     */
    fun setCenterNode(
        deviceId: Int,
        callback: CommonCallbacks.CompletionCallbackWithParam<Int>
    )

    /**
     * join netmesh
     */
    fun joinDeviceNetMesh(
        joinReq: JoinDeviceNetworkReq?,
        callback: CommonCallbacks.CompletionCallbackWithParam<Int>
    )

    /**
     * disband netmesh
     */
    fun disbandNetMesh(
        groupId: Long?,
        callback: CommonCallbacks.CompletionCallbackWithParam<Int>
    )

    /**
     * exit netmesh
     */
    fun quitNetMeshMatching(
        groupId: Long?,
        callback: CommonCallbacks.CompletionCallbackWithParam<Int>
    )

    /**
     * set device name
      */
    fun nameDeviceNetMeshMatching(
        editDeviceNameReq: EditDeviceNameReq,
        callback: CommonCallbacks.CompletionCallbackWithParam<Int>
    )

    /**
     * set watch device
     * return : 1：success 2：failure 0：unknown
     * */
    fun setWatchDevice(
        selectDevice: List<Int>?,
        callback: CommonCallbacks.CompletionCallbackWithParam<Int>
    )

    /** 
    * NetMesh Stream Control
    */
    fun setNetMeshStreamControl(streamList: List<CameraStreamInfo>, callback: CommonCallbacks.CompletionCallbackWithParam<Int>)

     /** 
    * switch control mode 
    */
    fun switchControlMode(mode: ControlMode,id: Int,callback: CommonCallbacks.CompletionCallbackWithParam<Void>)
}

3. Aircraft Evenvt monitoring

It provides device event monitoring for multi-aircraft netmesh, facilitating the monitoring of events in multi-aircraft scenarios.

DeviceManager.getDeviceManager().addDroneDevicesListener(reportKey, devicesListener)
DeviceManager.getDeviceManager().removeDroneDevicesListener(reportKey, devicesListener)

private val devicesListener = object : DeviceManager.KeyManagerListenerCallBack {
    override fun onListenerValueChanged(value: DeviceManager.DeviceListenerResult<*>) {
        (value.result as? MissionWaypointStatusReportNtfyBean)?.let { bean ->
             missionStatusMap[value.drone.getDeviceNumber()] = bean
        }
    }
}

Ⅲ、Autel Netmesh sample

1. Master remote controller

• Start netmesh pairing: if set 30 seconds timeout. After timeout, you can continue to initiate netmesh.

  • DeviceManager.getMultiDeviceOperator().getNetMeshManager().startNetMeshMatching(bean,callback)

• Delete device ：can't delete relay device(or call disband()

  • DeviceManager.getMultiDeviceOperator().getNetMeshManager().delNetMeshDevice(id,callback)

• set Relay drone: if you want to finish netmesh, you must set relay drone first

  • DeviceManager.getMultiDeviceOperator().getNetMeshManager().setCenterNode(id,callback)

• Finish netmesh：all devices will be reconnected after finish netmesh

  • DeviceManager.getMultiDeviceOperator().getNetMeshManager().completeNetMeshMatching(callback)

• Disband netmesh： delete all devices

  • DeviceManager.getMultiDeviceOperator().getNetMeshManager().disbandNetMesh(groupdId,callback)

•  Rename device name

  • DeviceManager.getMultiDeviceOperator().getNetMeshManager().nameDeviceNetMeshMatching(req,callback)

• set Control Mode（id GROUP mode : groupId; SINGLE mode ：NodeId; ALL mode：This parameter is ignored）

  • DeviceManager.getMultiDeviceOperator().getNetMeshManager().switchControlMode(mode, id, callback)

    enum class ControlMode(val value: Int) {
        /**
         * unknown
         */
        UNKNOWN(0),
        /**
         *Group control
         */
        GROUP(1),
        /**
         * single control
         */
        SINGLE(2),
        /**
         * control all
         */
    ALL(3);
    }

2. Slave remote controller

• Join in Netmesh

  • DeviceManager.getMultiDeviceOperator().getNetMeshManager().joinDeviceNetMesh(req,callback)
• exit form netmesh

  • DeviceManager.getMultiDeviceOperator().getNetMeshManager().quitNetMeshMatching（groupdId,callback）

3. Netmesh monitoring

• Device status monitoring (device life cycle)

  • They are in sequence: device creation -> device connection change -> device main service availability change -> device capability set change -> device destruction

  • interface IAutelDroneListener {
    /**
         * drone create
         * @param drone drone device
         */
        fun onDroneCreate(drone: IAutelDroneDevice){}
    
         /**
         * connection monitoring
         *
         * @param connected  - is connected
         * @param drone - drone device
         */
        fun onDroneChangedListener(connected: Boolean, drone: IAutelDroneDevice){}
    
        /**
         * main service is available
         * @param valid true available，false unavailable用
         * @param drone drone device
         */
        fun onMainServiceValid(valid: Boolean, drone: IAutelDroneDevice) {}
    
        /**
         * camera ability ready
         * @param localFetched true： loacl cache read
         * @param remoteFetched true： downlaod from drone 
         */
        fun onCameraAbilityFetchListener(localFetched: Boolean, remoteFetched: Boolean, drone: IAutelDroneDevice){}
    
        /**
         * SDK error listener
         * @param errorInfo
         */
        fun onSDKErrorListener(errorInfo: SDKErrorUtil) {}
    
        /**
         * drone destroy
         * @param drone drone device
         */
        fun onDroneDestroy(drone: IAutelDroneDevice){}
    
    }

• Device control change (triggered during the netmesh process and triggered after the user issues a control switch)

DeviceManager.getMultiDeviceOperator()
.addControlChangeListener(object : IControlDroneListener {
    override fun onControlChange(mode: ControlMode, droneList: List<IAutelDroneDevice>){
    }
})

• Device watch change monitoring

  It is necessary to monitor the changes of the watch device. The relay aircraft has an independent video channel ID configuration, and all leaf aircraft share a set of channel ID configurations. Therefore, when switching the code stream, it is necessary to determine whether it is a relay aircraft. For the channel configuration, please refer to ISDKConstants.

DeviceManager.getMultiDeviceOperator().addWatchChangeListener(object:object : IWatchDroneListener {
    override fun onWatchChange(droneList: List<IAutelDroneDevice>) {
        if(droneList.contans(currentDrone)){
        val playerId = DeviceUtils.getPlayerId(droneDevice, lensTypeEnum)
        if (playerId != null) {
            val autelPlayer = AutelPlayerManager.getInstance().getAutelPlayer(playerId)

            if (autelPlayer != null) {
                autelPlayer?.removeVideoView()
            }
            autelPlayer?.addVideoView(renderView)

            AutelLog.i("SwitchDrone", "startPlay = " + autelPlayer + " port:" + playerId + " view :" + renderView.toString())
            getCurrentAutelPlayerKeyFrame(playerId)
        }
    }
})