MediaPipe 系列 51：Android 部署——JNI 集成完整指南

前言：Android 部署的重要性

51.1 Android 平台优势

为什么选择 Android 部署 IMS？

优势 说明
开放生态 便于集成和定制
设备广泛 手机、车机、平板
硬件强大 GPU/NPU 加速
API 丰富 CameraX、NDK、Vulkan

IMS 应用场景：

  • 便携式 DMS 设备
  • 车载后视镜、HUD
  • 后排娱乐系统
  • 测试与演示工具

五十二、部署架构

52.1 整体架构

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
┌─────────────────────────────────────────────────────────────────────────┐
│                    Android MediaPipe 部署架构                           │
├─────────────────────────────────────────────────────────────────────────┤
│                                                                         │
│   ┌─────────────────────────────────────────────────────────┐           │
│   │           Application Layer (Java/Kotlin)               │           │
│   │                                                         │           │
│   │   • UI (Jetpack Compose/View)                          │           │
│   │   • Business Logic                                      │           │
│   │   • Lifecycle Management                                │           │
│   │                                                         │           │
│   └─────────────────────────────────────────────────────────┘           │
│                              │                                          │
│                              ▼                                          │
│   ┌─────────────────────────────────────────────────────────┐           │
│   │              JNI Bridge Layer                           │           │
│   │                                                         │           │
│   │   • Java Native Interface (JNI)                        │           │
│   │   • Type Conversion (Java ↔ C++)                       │           │
│   │   • Memory Management                                   │           │
│   │                                                         │           │
│   └─────────────────────────────────────────────────────────┘           │
│                              │                                          │
│                              ▼                                          │
│   ┌─────────────────────────────────────────────────────────┐           │
│   │           Native Layer (C++)                           │           │
│   │                                                         │           │
│   │   • MediaPipe Graph                                     │           │
│   │   • Calculator Framework                                │           │
│   │   • OpenGL ES Rendering                                 │           │
│   │   • Hardware Acceleration                               │           │
│   │                                                         │           │
│   └─────────────────────────────────────────────────────────┘           │
│                              │                                          │
│                              ▼                                          │
│   ┌─────────────────────────────────────────────────────────┐           │
│   │           Hardware Layer                                │           │
│   │                                                         │           │
│   │   • Camera2 API                                         │           │
│   │   • OpenGL ES / Vulkan                                  │           │
│   │   • GPU (Adreno/Mali)                                   │           │
│   │   • NPU (Hexagon)                                       │           │
│   │                                                         │           │
│   └─────────────────────────────────────────────────────────┘           │
│                                                                         │
└─────────────────────────────────────────────────────────────────────────┘

52.2 目录结构

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
app/
├── src/main/
│   ├── java/com/ims/dms/
│   │   ├── IMSDMS.java              # Java API
│   │   ├── DMSResult.java           # 结果数据类
│   │   └── MainActivity.java        # 主界面
│   ├── cpp/
│   │   ├── mediapipe_jni.cc         # JNI 实现
│   │   └── CMakeLists.txt           # Native 编译
│   ├── jniLibs/
│   │   └── arm64-v8a/
│   │       └── libmediapipe_jni.so  # Native 库
│   ├── assets/
│   │   ├── dms_graph.pbtxt          # Graph 配置
│   │   └── *.tflite                 # 模型文件
│   └── AndroidManifest.xml
└── WORKSPACE                         # Bazel 配置

五十三、Bazel 编译配置

53.1 WORKSPACE

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
# WORKSPACE

# Android SDK 配置
android_sdk_repository(
    name = "androidsdk",
    path = "/path/to/Android/Sdk",
    api_level = 30,
    build_tools_version = "30.0.3",
)

# Android NDK 配置
android_ndk_repository(
    name = "androidndk",
    path = "/path/to/Android/Sdk/ndk-bundle",
    api_level = 21,
)

# MediaPipe 本地仓库
local_repository(
    name = "mediapipe",
    path = "path/to/mediapipe",
)

workspace(name = "com_ims_dms")

53.2 BUILD 文件

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
# BUILD

# JNI Native 库
cc_library(
    name = "mediapipe_jni_lib",
    srcs = ["mediapipe_jni.cc"],
    deps = [
        "//mediapipe/framework:calculator_framework",
        "//mediapipe/framework/formats:image_frame",
        "//mediapipe/java/com/google/mediapipe/framework/jni:mediapipe_jni",
        "//mediapipe/graphs/ims:dms_fatigue_graph",
    ],
    copts = ["-std=c++17", "-fPIC", "-O3"],
    linkopts = ["-llog", "-landroid"],
)

# Android APK
android_binary(
    name = "ims_dms_apk",
    srcs = glob(["src/main/java/com/ims/dms/**/*.java"]),
    manifest = "src/main/AndroidManifest.xml",
    deps = [":mediapipe_jni_lib"],
)

五十四、JNI 集成

54.1 Java 端 API

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
// IMSDMS.java
package com.ims.dms;

import android.graphics.Bitmap;
import android.os.Handler;
import android.os.HandlerThread;

public class IMSDMS {
    static {
        System.loadLibrary("mediapipe_jni");
    }
    
    private long nativeHandle;
    private HandlerThread handlerThread;
    private Handler handler;
    
    public interface ResultListener {
        void onResult(DMSResult result);
        void onError(String error);
    }
    
    public IMSDMS(String graphPath) {
        nativeHandle = nativeInit(graphPath);
        handlerThread = new HandlerThread("MediaPipe-Thread");
        handlerThread.start();
        handler = new Handler(handlerThread.getLooper());
    }
    
    public void process(Bitmap image, long timestamp, ResultListener listener) {
        handler.post(() -> {
            try {
                DMSResult result = nativeProcess(nativeHandle, image, timestamp);
                listener.onResult(result);
            } catch (Exception e) {
                listener.onError(e.getMessage());
            }
        });
    }
    
    public void release() {
        if (nativeHandle != 0) {
            nativeRelease(nativeHandle);
            nativeHandle = 0;
        }
        handlerThread.quitSafely();
    }
    
    // Native 方法
    private native long nativeInit(String graphPath);
    private native DMSResult nativeProcess(long handle, Bitmap image, long timestamp);
    private native void nativeRelease(long handle);
}

54.2 C++ 端实现

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
// mediapipe_jni.cc
#include <jni.h>
#include <android/log.h>
#include <android/bitmap.h>
#include "mediapipe/framework/calculator_graph.h"
#include "mediapipe/framework/formats/image_frame.h"

#define LOG_TAG "MediaPipeJNI"
#define LOGI(...) __android_log_print(ANDROID_LOG_INFO, LOG_TAG, __VA_ARGS__)
#define LOGE(...) __android_log_print(ANDROID_LOG_ERROR, LOG_TAG, __VA_ARGS__)

namespace {
    std::unordered_map<jlong, std::unique_ptr<mediapipe::CalculatorGraph>> g_graphs;
    jlong g_next_handle = 1;
    
    std::unique_ptr<mediapipe::ImageFrame> ConvertBitmapToImageFrame(
        JNIEnv* env, jobject bitmap) {
        AndroidBitmapInfo info;
        AndroidBitmap_getInfo(env, bitmap, &info);
        
        void* pixels;
        AndroidBitmap_lockPixels(env, bitmap, &pixels);
        
        auto frame = std::make_unique<mediapipe::ImageFrame>(
            mediapipe::ImageFormat::SRGBA, info.width, info.height);
        
        std::memcpy(frame->MutablePixelData(), pixels, 
                    info.width * info.height * 4);
        
        AndroidBitmap_unlockPixels(env, bitmap);
        return frame;
    }
}

extern "C" {

JNIEXPORT jlong JNICALL
Java_com_ims_dms_IMSDMS_nativeInit(JNIEnv* env, jobject, jstring graph_path) {
    const char* path = env->GetStringUTFChars(graph_path, nullptr);
    LOGI("Initializing MediaPipe: %s", path);
    
    auto graph = std::make_unique<mediapipe::CalculatorGraph>();
    
    std::ifstream file(path);
    std::stringstream buffer;
    buffer << file.rdbuf();
    
    mediapipe::CalculatorGraphConfig config;
    config.ParseFromString(buffer.str());
    
    if (graph->Initialize(config).ok() && graph->StartRun({}).ok()) {
        jlong handle = g_next_handle++;
        g_graphs[handle] = std::move(graph);
        env->ReleaseStringUTFChars(graph_path, path);
        return handle;
    }
    
    env->ReleaseStringUTFChars(graph_path, path);
    return 0;
}

JNIEXPORT jobject JNICALL
Java_com_ims_dms_IMSDMS_nativeProcess(JNIEnv* env, jobject, 
                                       jlong handle, jobject bitmap, jlong timestamp) {
    auto it = g_graphs.find(handle);
    if (it == g_graphs.end()) return nullptr;
    
    auto frame = ConvertBitmapToImageFrame(env, bitmap);
    if (!frame) return nullptr;
    
    auto packet = mediapipe::Adopt(frame.release())
        .At(mediapipe::Timestamp(timestamp));
    
    it->second->AddPacketToInputStream("input", packet);
    
    jclass result_class = env->FindClass("com/ims/dms/DMSResult");
    jmethodID constructor = env->GetMethodID(result_class, "<init>", "(FFI)V");
    return env->NewObject(result_class, constructor, 0.8f, 0.5f, 2);
}

JNIEXPORT void JNICALL
Java_com_ims_dms_IMSDMS_nativeRelease(JNIEnv*, jobject, jlong handle) {
    g_graphs.erase(handle);
}

}

五十五、CameraX 相机输入

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
// CameraManager.java
package com.ims.dms.camera;

import androidx.camera.core.*;
import androidx.camera.lifecycle.ProcessCameraProvider;

public class CameraManager {
    private ProcessCameraProvider cameraProvider;
    private ExecutorService executor = Executors.newSingleThreadExecutor();
    
    public void start(LifecycleOwner owner, int cameraId, FrameProcessor processor) {
        ProcessCameraProvider.getInstance().addListener(() -> {
            cameraProvider = ProcessCameraProvider.getInstance();
            
            ImageAnalysis analysis = new ImageAnalysis.Builder()
                .setBackpressureStrategy(ImageAnalysis.STRATEGY_KEEP_ONLY_LATEST)
                .setTargetResolution(new Size(640, 480))
                .build();
            
            analysis.setAnalyzer(executor, image -> {
                Bitmap bitmap = imageToBitmap(image);
                processor.process(bitmap, System.currentTimeMillis());
                image.close();
            });
            
            CameraSelector selector = new CameraSelector.Builder()
                .requireLensFacing(cameraId == 0 ? CameraSelector.LENS_FACING_FRONT 
                                                 : CameraSelector.LENS_FACING_BACK)
                .build();
            
            cameraProvider.bindToLifecycle(owner, selector, analysis);
        }, ContextCompat.getMainExecutor(context));
    }
    
    public void stop() {
        if (cameraProvider != null) cameraProvider.unbindAll();
        executor.shutdown();
    }
}

五十六、性能优化

56.1 GPU 加速

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
#include <GLES3/gl3.h>

class GpuProcessor {
    GLuint texture_id_;
public:
    void Initialize() {
        glGenTextures(1, &texture_id_);
        glBindTexture(GL_TEXTURE_2D, texture_id_);
    }
    
    void ProcessFrame(const ImageFrame& frame) {
        glTexImage2D(GL_TEXTURE_2D, 0, GL_RGBA,
                     frame.Width(), frame.Height(), 0,
                     GL_RGBA, GL_UNSIGNED_BYTE, frame.PixelData());
    }
};

56.2 内存池化

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
public class BitmapPool {
    private static final int MAX_SIZE = 5;
    private static LinkedList<Bitmap> pool = new LinkedList<>();
    
    public static Bitmap obtain(int w, int h) {
        for (Bitmap b : pool) {
            if (b.getWidth() == w && b.getHeight() == h) {
                pool.remove(b);
                return b;
            }
        }
        return Bitmap.createBitmap(w, h, Bitmap.Config.ARGB_8888);
    }
    
    public static void recycle(Bitmap b) {
        if (pool.size() < MAX_SIZE) pool.add(b);
        else b.recycle();
    }
}

五十七、部署脚本

1
2
3
4
5
6
7
8
9
10
11
12
13
#!/bin/bash
# deploy_android.sh

# 编译
bazel build -c opt --config=android_arm64 \
    //mediapipe/examples/android/src/java/com/google/mediapipe/apps/imsdms:imsdms

# 安装
adb install bazel-bin/mediapipe/examples/android/.../imsdms.apk

# 推送模型
adb push models/*.tflite /sdcard/ims/
adb push graphs/*.pbtxt /sdcard/ims/

五十八、总结

要点 说明
架构 Java → JNI → C++ → Hardware
编译 Bazel 构建原生库
JNI Java ↔ C++ 数据转换
相机 CameraX API 输入
优化 GPU 加速、Bitmap 池化

系列进度： 51/55
更新时间： 2026-03-12

MediaPipe 系列 > 跨平台部署
#MediaPipe #Bazel #Android #JNI #CameraX #GPU
MediaPipe 系列 51：Android 部署——JNI 集成完整指南
https://dapalm.com/2026/03/13/MediaPipe系列51-Android部署：JNI集成/
作者
Mars
发布于
2026年3月13日
许可协议