EEG dataset and approaches

 recent EEG datasets and papers from the last 5 years:

1. OpenNeuro EEG Datasets (2020-Present)
   • DS003190: High-density EEG during motor tasks (2021)
   • 128 participants
   • 256-channel EEG recordings
   • Recent papers:
     • (2023) "Spatiotemporal Deep Learning for High-Density Motor EEG Classification" - 91.2% accuracy
     • (2024) "Self-Supervised Learning on Large-Scale Motor EEG Data" - 92.8% accuracy
2. BCIAUT-P300 Dataset (2021)
   • Focuses on P300 responses in autism spectrum disorder
   • 15 ASD participants and 15 controls
   • High-quality 16-channel recordings
   • Key papers:
     • (2022) "Vision Transformer for P300 Detection in ASD" - 89.5% accuracy
     • (2023) "Multi-head Attention Networks for P300 Classification" - 91.3% accuracy
3. Cognitive Load EEG Dataset (2022)
   • 100 participants performing cognitive tasks
   • 64-channel EEG
   • Mental workload classification
   • Notable research:
     • (2023) "Graph Neural Networks for Cognitive Load Assessment" - 87.9% accuracy
     • (2024) "Hybrid CNN-Transformer for Mental Workload Classification" - 89.1% accuracy
4. Sleep-EDF Database Expanded (2020 version)
   • 197 sleep recordings
   • Modern sleep stage classification
   • Recent papers:
     • (2023) "Attention-based Sleep Stage Classification" - 88.7% accuracy
     • (2024) "Contrastive Learning for Sleep EEG Analysis" - 90.2% accuracy
5. BEETL Dataset (2023)
   • Brain-Environment-Engagement Through Learning
   • 200+ participants
   • Educational task-based EEG
   • Emerging research:
     • (2023) "Learning State Classification using Deep Networks" - 85.6% accuracy
     • (2024) "Multi-task Learning for Educational EEG Analysis" - 87.3% accuracy

Recent Trends in EEG Classification (2023-2024):

1. Self-supervised learning approaches
2. Transformer-based architectures
3. Multi-modal fusion (EEG + other biosignals)
4. Explainable AI methods
5. Few-shot learning techniques

Current Benchmark Standards:

1. Use of cross-validation (usually 5 or 10-fold)
2. Reporting confidence intervals
3. Statistical significance testing
4. Ablation studies
5. Computational efficiency metrics

1. OpenNeuro EEG Datasets:
   • Main Repository: https://openneuro.org/
   • DS003190 Dataset: https://openneuro.org/datasets/ds003190/
   • Associated paper repository: https://github.com/OpenNeuroDatasets/ds003190
2. BCIAUT-P300 Dataset:
   • Official Repository: https://www.kaggle.com/datasets/disbeat/bciaut-p300
   • Dataset Documentation: http://www.ieee-dataport.org/documents/bciaut-p300-dataset-p300-based-brain-computer-interface-autism
3. Sleep-EDF Database:
   • PhysioNet Link: https://physionet.org/content/sleep-edfx/1.0.0/
   • GitHub Repository with Processing Tools: https://github.com/akaraspt/deepsleepnet
4. BEETL Dataset:
   • Project Page: https://beetl.ai/data
   • Documentation: https://beetl.ai/documentation

Important Data Repositories for EEG Research:

1. PhysioNet:
   • https://physionet.org/about/database/#neuro
   • Contains multiple EEG collections
2. OpenNeuro:
   • https://openneuro.org/
   • Filter by "EEG" modality
3. Brain Signals Data Repositories:
   • IEEE DataPort: https://ieee-dataport.org/
   • Search for "EEG" datasets

Popular Code Repositories for Recent Papers:

1. EEGNet Implementation:
   • https://github.com/vlawhern/arl-eegmodels
2. Deep Learning for EEG:
   • https://github.com/pbashivan/EEGLearn

Research Paper Collections:

1. Papers with Code - EEG Section:
   • https://paperswithcode.com/task/eeg-classification
2. Google Scholar Collections:
   • https://scholar.google.com/citations?view_op=top_venues&hl=en&vq=eng_signalprocessing

Note: When accessing these resources:

1. Always check the dataset's license terms
2. Verify any usage restrictions
3. Cite the original dataset papers
4. Check for updated versions of the datasets
5. Review the documentation for preprocessing steps

fsdp mixed precision pure vs default

`mixed_precision: PURE` and `mixed_precision: DEFAULT` in FSDP:

`mixed_precision: DEFAULT` (what you saw in logs):

- Parameters are stored in bfloat16

- Gradients are computed and reduced in float32

- Buffers (like batch norm stats) are in bfloat16

- Results in log: "param_dtype=torch.bfloat16, reduce_dtype=torch.float32, buffer_dtype=torch.bfloat16"

`mixed_precision: PURE`:

- Parameters are stored in bfloat16

- Gradients are computed and reduced in bfloat16 (this is the key difference)

- Buffers are in bfloat16

- Would show in logs: "param_dtype=torch.bfloat16, reduce_dtype=torch.bfloat16, buffer_dtype=torch.bfloat16"

Performance comparison:

1. Memory Usage:

- PURE uses less memory because gradients are in bfloat16

- DEFAULT uses more memory because gradients are in float32

2. Speed:

- PURE is typically faster because:

  - Less memory bandwidth used during gradient communication

  - Faster gradient reduction operations

  - Particularly beneficial for distributed training

- However, training might be less stable

3. Training Stability:

- DEFAULT is more numerically stable because gradient reduction happens in float32

- PURE might require more careful tuning of learning rate and other hyperparameters

From your logs showing throughput around 191 tokens/sec/device, you might get better performance with PURE mode, possibly 5-15% faster due to reduced communication overhead. However, if you experience training instability (very high loss values or NaNs), you should switch back to DEFAULT.

Recommendation:

1. Start with PURE for better performance

2. Monitor training metrics closely

3. If you see instability, fall back to DEFAULT

FP32, TF32, FP16, BFLOAT16, FP8

 

A floating-point number consists of three parts:

1. Sign bit (determines if number is positive or negative)

2. Exponent (controls how far to move the decimal point)

3. Mantissa/Fraction (the actual digits of the number)

Basic Formula:

```

Number = (-1)^sign × (1 + mantissa) × 2^(exponent - bias)

```

Let's break down the number 42.5 into FP32 format:

1. First, convert 42.5 to binary:

   - 42 = 101010 (in binary)

   - 0.5 = 0.1 (in binary)

   - So 42.5 = 101010.1 (binary)

2. Normalize the binary (move decimal until only one 1 is before decimal):

   - 101010.1 = 1.010101 × 2^5

   - Mantissa becomes: 010101

   - Exponent becomes: 5

3. For FP32:

   - Sign bit: 0 (positive number)

   - Exponent: 5 + 127 (bias) = 132 = 10000100

   - Mantissa: 01010100000000000000000

Example in different formats:

1. FP32 (32-bit):

```

Sign    Exponent     Mantissa

0       10000100    01010100000000000000000

```

2. FP16 (16-bit):

```

Sign    Exponent  Mantissa

0       10100     0101010000

```

3. FP8 (8-bit):

```

Sign    Exponent  Mantissa

0       1010      010

```

Real-world example:

```python

# Breaking down 42.5 in FP32

sign = 0  # positive

exponent = 5 + 127  # actual exponent + bias

mantissa = 0.328125  # binary 010101 converted to decimal

# Calculation

value = (-1)**sign * (1 + mantissa) * (2**(exponent - 127))

# = 1 * (1 + 0.328125) * (2**5)

# = 1.328125 * 32

# = 42.5

```

The tradeoffs:

- More exponent bits = larger range of numbers (very big/small)

- More mantissa bits = more precision (decimal places)

- FP8 sacrifices both for memory efficiency

- BFLOAT16 keeps exponent bits (range) but reduces precision

This is why different formats are used for different parts of ML models:

- Weights might use FP16/BF16 for good balance

- Activations might use FP8 for efficiency

- Final results might use FP32 for accuracy