Frontiers 2026系统综述：EEG+眼动追踪融合监测认知与生理状态

Frontiers 2026系统综述：EEG+眼动追踪融合监测认知与生理状态

论文来源： Frontiers in Neuroergonomics
期刊： Frontiers in Neuroergonomics, January 29, 2026
核心发现： EEG+眼动融合优于单模态，准确率提升15-20%

论文信息

项目 内容
标题 Combining EEG and eye-tracking for cognitive and physiological states monitoring: a systematic review
期刊 Frontiers in Neuroergonomics
类型 系统综述（Systematic Review）
研究对象 54篇EEG+眼动融合研究

核心问题：单模态局限

EEG单独局限

局限 描述
空间分辨率低 无法精确定位认知活动区域
干扰敏感 运动、肌肉信号干扰
佩戴不适 电极帽不适合长时间驾驶
实时性差 信号处理复杂

眼动单独局限

局限 描述
无法检测内部状态 只看眼睛，不知道大脑在想什么
认知分心难检测 眼睛看前方，但大脑走神
光照敏感 黑暗环境眼动检测困难
遮挡问题 眼镜、墨镜遮挡

融合优势

性能对比

方法 认知分心准确率 疲劳检测准确率 思维走神准确率
EEG单独 72% 85% 68%
眼动单独 65% 78% 60%
EEG+眼动融合 89% 94% 85%
提升幅度 +17% +9% +17%

EEG特征提取

关键脑区与频段

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
import numpy as np

class EEGFeatureExtractor:
    """
    EEG特征提取器
    
    Frontiers 2026综述核心：
    
    关键脑区：
    - 额叶（Fz, F3, F4）：注意力、工作记忆
    - 中央区（Cz, C3, C4）：运动规划
    - 顶叶（Pz, P3, P4）：感觉整合
    
    关键频段：
    - Delta (0.5-4 Hz)：深度睡眠
    - Theta (4-8 Hz)：困倦、冥想
    - Alpha (8-12 Hz)：放松、闭眼
    - Beta (12-30 Hz)：活跃思考
    - Gamma (30-100 Hz)：高级认知
    """
    
    def __init__(self, 
                 sampling_rate: float = 256.0,
                 channels: list = None):
        self.fs = sampling_rate
        self.channels = channels or ["Fz", "F3", "F4", "Cz", "Pz"]
        
        # 频段定义
        self.frequency_bands = {
            "delta": (0.5, 4),
            "theta": (4, 8),
            "alpha": (8, 12),
            "beta": (12, 30),
            "gamma": (30, 100)
        }
    
    def extract_band_power(self, 
                           eeg_signal: np.ndarray,
                           band: str) -> np.ndarray:
        """
        提取频段功率
        
        Args:
            eeg_signal: EEG信号, shape=(N, num_channels)
            band: 频段名称
        
        Returns:
            band_power: 频段功率, shape=(num_channels,)
        """
        from scipy.fft import fft
        
        low, high = self.frequency_bands[band]
        
        N = len(eeg_signal)
        
        # FFT
        spectrum = fft(eeg_signal)
        
        # 频率轴
        freq = np.fft.fftfreq(N, 1/self.fs)
        
        # 提取目标频段
        band_mask = (freq >= low) & (freq <= high)
        
        # 计算功率
        band_power = np.mean(np.abs(spectrum[:, band_mask]) ** 2, axis=1)
        
        return band_power
    
    def compute_theta_alpha_ratio(self, 
                                   eeg_signal: np.ndarray) -> float:
        """
        计算Theta/Alpha比值
        
        疲劳指标：比值越高，疲劳程度越高
        
        Args:
            eeg_signal: EEG信号
        
        Returns:
            ratio: Theta/Alpha比值
        """
        theta_power = self.extract_band_power(eeg_signal, "theta")
        alpha_power = self.extract_band_power(eeg_signal, "alpha")
        
        # 平均所有通道
        theta_mean = np.mean(theta_power)
        alpha_mean = np.mean(alpha_power)
        
        ratio = theta_mean / (alpha_mean + 1e-6)
        
        return ratio
    
    def detect_drowsiness(self, 
                          eeg_signal: np.ndarray) -> dict:
        """
        检测困倦状态
        
        指标：
        - Theta增加（困倦）
        - Alpha增加（放松）
        - Beta减少（活跃降低）
        
        Args:
            eeg_signal: EEG信号
        
        Returns:
            result: {"state": str, "confidence": float}
        """
        theta_alpha_ratio = self.compute_theta_alpha_ratio(eeg_signal)
        
        beta_power = self.extract_band_power(eeg_signal, "beta")
        beta_mean = np.mean(beta_power)
        
        # 困倦判定
        if theta_alpha_ratio > 1.5 and beta_mean < 10:
            return {"state": "drowsy", "confidence": 0.85}
        
        elif theta_alpha_ratio > 1.0 and beta_mean < 20:
            return {"state": "mild_fatigue", "confidence": 0.75}
        
        else:
            return {"state": "alert", "confidence": 0.9}
    
    def detect_cognitive_load(self, 
                               eeg_signal: np.ndarray) -> dict:
        """
        检测认知负荷
        
        指标：
        - 额叶Theta增加（高负荷）
        - 额叶Alpha减少（专注）
        - 顶叶Alpha增加（资源分配）
        
        Args:
            eeg_signal: EEG信号
        
        Returns:
            result: {"load_level": str, "confidence": float}
        """
        # 额叶Theta
        frontal_theta = self.extract_band_power(eeg_signal[:, :3], "theta")
        frontal_theta_mean = np.mean(frontal_theta)
        
        # 额叶Alpha
        frontal_alpha = self.extract_band_power(eeg_signal[:, :3], "alpha")
        frontal_alpha_mean = np.mean(frontal_alpha)
        
        # 顶叶Alpha
        parietal_alpha = self.extract_band_power(eeg_signal[:, 4:], "alpha")
        parietal_alpha_mean = np.mean(parietal_alpha)
        
        # 认知负荷判定
        if frontal_theta_mean > 20 and frontal_alpha_mean < 5:
            return {"load_level": "high", "confidence": 0.85}
        
        elif frontal_theta_mean > 10 and parietal_alpha_mean > 8:
            return {"load_level": "medium", "confidence": 0.75}
        
        else:
            return {"load_level": "low", "confidence": 0.9}


# 测试示例
if __name__ == "__main__":
    extractor = EEGFeatureExtractor(sampling_rate=256.0)
    
    # 模拟EEG信号（5通道，10秒）
    N = 2560
    num_channels = 5
    
    # 正常状态：Alpha高，Theta低
    normal_eeg = np.random.randn(N, num_channels) * 10
    
    # 困倦状态：Theta高，Alpha高
    drowsy_eeg = normal_eeg.copy()
    drowsy_eeg[:, :3] += 5 * np.sin(2 * np.pi * 6 * np.arange(N) / 256.0)  # 添加Theta
    
    print("=== 正常状态检测 ===")
    result_normal = extractor.detect_drowsiness(normal_eeg)
    print(f"困倦检测: {result_normal}")
    
    print("\n=== 困倦状态检测 ===")
    result_drowsy = extractor.detect_drowsiness(drowsy_eeg)
    print(f"困倦检测: {result_drowsy}")
    
    theta_alpha = extractor.compute_theta_alpha_ratio(drowsy_eeg)
    print(f"Theta/Alpha比值: {theta_alpha:.2f}")

眼动特征提取

关键指标

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
import numpy as np

class EyeTrackingFeatureExtractor:
    """
    眼动追踪特征提取器
    
    Frontiers 2026综述核心：
    
    关键指标：
    - 瞳孔直径：认知负荷、情绪
    - 眨眼频率：疲劳
    - 眨眼时长：疲劳程度
    - 眼动速度：注意力
    - 注视时长：专注度
    - 注视熵：注意力分散程度
    """
    
    def __init__(self):
        self.blink_threshold = 0.2  # 眨眼阈值
        self.gaze_entropy_bins = 10  # 注视熵bins
    
    def compute_pupil_features(self, 
                                pupil_diameter: np.ndarray) -> dict:
        """
        计算瞳孔特征
        
        Args:
            pupil_diameter: 瞳孔直径序列, shape=(N,)
        
        Returns:
            features: {"mean", "std", "change_rate"}
        """
        mean_pupil = np.mean(pupil_diameter)
        std_pupil = np.std(pupil_diameter)
        
        # 瞳孔变化率
        pupil_change = np.diff(pupil_diameter)
        change_rate = np.mean(np.abs(pupil_change))
        
        return {
            "mean": mean_pupil,
            "std": std_pupil,
            "change_rate": change_rate
        }
    
    def compute_blink_features(self, 
                                eye_openness: np.ndarray) -> dict:
        """
        计算眨眼特征
        
        Args:
            eye_openness: 眼睑开度序列, shape=(N,)
        
        Returns:
            features: {"blink_rate", "blink_duration", "perclos"}
        """
        # 眨眼检测
        blink_mask = eye_openness < self.blink_threshold
        
        # 眨眼频率（次/分钟）
        blink_count = np.sum(blink_mask)
        blink_rate = blink_count / (len(eye_openness) / 30.0 / 60.0)  # 假设30fps
        
        # 眨眼时长
        blink_durations = self._compute_blink_durations(blink_mask)
        blink_duration = np.mean(blink_durations) if len(blink_durations) > 0 else 0
        
        # PERCLOS
        perclos = np.sum(blink_mask) / len(eye_openness) * 100
        
        return {
            "blink_rate": blink_rate,
            "blink_duration": blink_duration,
            "perclos": perclos
        }
    
    def compute_gaze_entropy(self, 
                              gaze_positions: np.ndarray) -> float:
        """
        计算注视熵
        
        注视熵越高，注意力越分散
        
        Args:
            gaze_positions: 注视位置, shape=(N, 2)
        
        Returns:
            entropy: 注视熵
        """
        # 空间bins
        x_bins = np.linspace(0, 1, self.gaze_entropy_bins)
        y_bins = np.linspace(0, 1, self.gaze_entropy_bins)
        
        # 计算每个bin的频率
        hist, _, _ = np.histogram2d(
            gaze_positions[:, 0],
            gaze_positions[:, 1],
            bins=[x_bins, y_bins]
        )
        
        # 归一化
        hist_norm = hist / np.sum(hist)
        
        # Shannon熵
        entropy = -np.sum(hist_norm * np.log2(hist_norm + 1e-6))
        
        return entropy
    
    def detect_mind_wandering(self, 
                               eye_features: dict) -> dict:
        """
        检测思维走神
        
        指标：
        - 注视熵高（注意力分散）
        - 瞳孔变化率低（认知活动低）
        - 眨眼频率异常
        
        Args:
            eye_features: 眼动特征字典
        
        Returns:
            result: {"state": str, "confidence": float}
        """
        gaze_entropy = eye_features.get("gaze_entropy", 0)
        pupil_change_rate = eye_features.get("pupil_change_rate", 0)
        blink_rate = eye_features.get("blink_rate", 0)
        
        # 思维走神判定
        if gaze_entropy > 2.5 and pupil_change_rate < 0.05:
            return {"state": "mind_wandering", "confidence": 0.80}
        
        elif gaze_entropy > 2.0 and blink_rate > 20:
            return {"state": "possible_distraction", "confidence": 0.70}
        
        else:
            return {"state": "focused", "confidence": 0.85}
    
    def _compute_blink_durations(self, 
                                  blink_mask: np.ndarray) -> list:
        """计算每次眨眼时长"""
        durations = []
        
        in_blink = False
        blink_start = 0
        
        for i, is_blink in enumerate(blink_mask):
            if is_blink and not in_blink:
                # 眨眼开始
                in_blink = True
                blink_start = i
            elif not is_blink and in_blink:
                # 眨眼结束
                in_blink = False
                duration = i - blink_start
                durations.append(duration)
        
        return durations


# 测试示例
if __name__ == "__main__":
    extractor = EyeTrackingFeatureExtractor()
    
    # 模拟眼动数据
    N = 1800  # 60秒 @ 30fps
    
    # 正常状态：瞳孔稳定，注视集中
    normal_pupil = np.random.normal(4.0, 0.2, N)
    normal_gaze = np.random.multivariate_normal([0.5, 0.5], [[0.01, 0], [0, 0.01]], N)
    normal_eye_openness = np.random.normal(0.8, 0.1, N)
    
    # 思维走神状态：注视分散
    wandering_gaze = np.random.rand(N, 2)  # 随机分布
    
    print("=== 正常状态眼动特征 ===")
    pupil_features = extractor.compute_pupil_features(normal_pupil)
    print(f"瞳孔特征: {pupil_features}")
    
    gaze_entropy_normal = extractor.compute_gaze_entropy(normal_gaze)
    print(f"注视熵: {gaze_entropy_normal:.2f}")
    
    blink_features = extractor.compute_blink_features(normal_eye_openness)
    print(f"眨眼特征: {blink_features}")
    
    print("\n=== 思维走神状态检测 ===")
    wandering_entropy = extractor.compute_gaze_entropy(wandering_gaze)
    print(f"思维走神注视熵: {wandering_entropy:.2f}")
    
    wandering_features = {
        "gaze_entropy": wandering_entropy,
        "pupil_change_rate": 0.03,
        "blink_rate": 25
    }
    result = extractor.detect_mind_wandering(wandering_features)
    print(f"思维走神检测: {result}")

融合方法

特征级融合

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
import numpy as np

class EEGEyeFusion:
    """
    EEG + 眼动融合
    
    Frontiers 2026综述方法：
    
    融合层级：
    1. 特征级融合：特征拼接 → 分类器
    2. 决策级融合：独立分类 → 投票/加权
    3. 深度学习融合：端到端神经网络
    """
    
    def __init__(self):
        self.eeg_extractor = EEGFeatureExtractor()
        self.eye_extractor = EyeTrackingFeatureExtractor()
    
    def feature_level_fusion(self, 
                              eeg_signal: np.ndarray,
                              pupil_diameter: np.ndarray,
                              gaze_positions: np.ndarray,
                              eye_openness: np.ndarray) -> np.ndarray:
        """
        特征级融合
        
        Args:
            eeg_signal: EEG信号
            pupil_diameter: 瞳孔直径
            gaze_positions: 注视位置
            eye_openness: 眼睑开度
        
        Returns:
            fused_features: 融合特征向量
        """
        # EEG特征
        theta_alpha_ratio = self.eeg_extractor.compute_theta_alpha_ratio(eeg_signal)
        
        eeg_features = np.array([
            theta_alpha_ratio,
            np.mean(self.eeg_extractor.extract_band_power(eeg_signal, "theta")),
            np.mean(self.eeg_extractor.extract_band_power(eeg_signal, "alpha")),
            np.mean(self.eeg_extractor.extract_band_power(eeg_signal, "beta"))
        ])
        
        # 眼动特征
        pupil_features = self.eye_extractor.compute_pupil_features(pupil_diameter)
        blink_features = self.eye_extractor.compute_blink_features(eye_openness)
        gaze_entropy = self.eye_extractor.compute_gaze_entropy(gaze_positions)
        
        eye_features = np.array([
            pupil_features["mean"],
            pupil_features["change_rate"],
            blink_features["blink_rate"],
            blink_features["perclos"],
            gaze_entropy
        ])
        
        # 特征拼接
        fused_features = np.concatenate([eeg_features, eye_features])
        
        return fused_features
    
    def decision_level_fusion(self, 
                               eeg_result: dict,
                               eye_result: dict,
                               method: str = "weighted") -> dict:
        """
        决策级融合
        
        Args:
            eeg_result: EEG检测结果
            eye_result: 眼动检测结果
            method: 融合方法 "weighted" or "voting"
        
        Returns:
            fused_result: 融合决策结果
        """
        if method == "weighted":
            # 加权融合（EEG权重更高）
            eeg_weight = 0.6
            eye_weight = 0.4
            
            eeg_state = eeg_result.get("state", "alert")
            eye_state = eye_result.get("state", "focused")
            
            # 如果两者一致，置信度更高
            if eeg_state == eye_state:
                confidence = 0.95
                state = eeg_state
            else:
                # 不一致时，使用EEG结果（权重更高）
                confidence = eeg_weight * eeg_result.get("confidence", 0.5) + \
                            eye_weight * eye_result.get("confidence", 0.5)
                state = eeg_state
            
            return {"state": state, "confidence": confidence}
        
        elif method == "voting":
            # 投票融合
            states = [eeg_result.get("state"), eye_result.get("state")]
            
            # 如果两者一致
            if states[0] == states[1]:
                return {"state": states[0], "confidence": 0.90}
            else:
                # 不一致，返回"不确定"
                return {"state": "uncertain", "confidence": 0.50}
        
        return {"state": "unknown", "confidence": 0.0}


# 测试示例
if __name__ == "__main__":
    fusion = EEGEyeFusion()
    
    # 模拟数据
    N = 2560
    num_channels = 5
    
    eeg_signal = np.random.randn(N, num_channels) * 10
    pupil_diameter = np.random.normal(4.0, 0.2, N)
    gaze_positions = np.random.multivariate_normal([0.5, 0.5], [[0.01, 0], [0, 0.01]], N)
    eye_openness = np.random.normal(0.8, 0.1, N)
    
    print("=== 特征级融合 ===")
    fused_features = fusion.feature_level_fusion(
        eeg_signal, pupil_diameter, gaze_positions, eye_openness
    )
    print(f"融合特征维度: {fused_features.shape}")
    print(f"融合特征: {fused_features}")
    
    print("\n=== 决策级融合 ===")
    eeg_result = {"state": "drowsy", "confidence": 0.85}
    eye_result = {"state": "drowsy", "confidence": 0.78}
    
    weighted_result = fusion.decision_level_fusion(eeg_result, eye_result, "weighted")
    print(f"加权融合: {weighted_result}")
    
    voting_result = fusion.decision_level_fusion(eeg_result, eye_result, "voting")
    print(f"投票融合: {voting_result}")

IMS应用启示

车载EEG方案可行性

方案 可行性 成本 备注
脑电头带 ⚠️ 中 $200-500 需驾驶员主动佩戴
座椅嵌入式电极 🔴 低 $100-300 干电极，信号质量差
耳内EEG 🟡 高 $50-100 耳机集成，舒适性好

眼动+EEG融合优先级

功能 单眼动准确率 眼动+EEG准确率 优先级
疲劳检测 78% 94% 🔴 高
认知分心 60% 85% 🔴 高
思维走神 50% 75% 🟡 中

参考资料

  1. Frontiers in Neuroergonomics 2026
  2. Nature 2024 - EEG Mind Wandering
  3. Euro NCAP 2026 Cognitive Distraction Requirements

总结

EEG+眼动融合核心发现：

  1. 准确率提升15-20%：融合优于单模态
  2. 认知分心突破：85%准确率（单眼动60%）
  3. 疲劳检测增强：94%准确率（单眼动78%）
  4. 思维走神检测：75%准确率（首次量产可行）

IMS开发优先级：

  • 🔴 高：疲劳检测融合（立即可行）
  • 🟡 中：认知分心融合（需更多研究）
  • 🟢 低：耳内EEG硬件集成（未来方向）

下一步行动：

  • 研究耳内EEG硬件方案
  • 实现EEG+眼动特征融合算法
  • 对齐Euro NCAP认知分心评估场景
#认知分心 #眼动追踪 #EEG #Frontiers #系统综述
Frontiers 2026系统综述：EEG+眼动追踪融合监测认知与生理状态
https://dapalm.com/2026/07/09/2026-07-09-frontiers-2026-eeg-eye-tracking-fusion-systematic-review/
作者
Mars
发布于
2026年7月9日
许可协议