PSYC4411

Beat the Baseline

Week 10 Challenge Lab

Neural Network Training & Diagnostics on EEG Brain Signals

Today's Challenge

  • Phase 1: Baseline (familiar)
    • Logistic regression on EEG motor imagery features
    • Report accuracy and F1 — this is your target to beat
  • Phase 2: Neural Network (new!)
    • Build a simple MLP in PyTorch — your first neural network
    • Train it, track the loss curves, diagnose issues
  • Phase 3: Honest Comparison
    • Did the neural network beat the baseline?
    • Was the added complexity worth it?
  • Present your results (1 slide, ~3 min)
    • Baseline vs MLP, loss curves, debugging story, was it worth it?

The Dataset: EEG Motor Imagery

  • 109 participants imagined moving their left or right hand
  • 64-channel EEG recorded brain activity at 160 Hz
  • 4,918 trials — ~45 per person, 3 runs each
  • 320 features per trial (64 channels × 5 frequency bands)
  • Balanced: 50.4% left / 49.6% right

This is real BCI data! The same type of signal used to help people with paralysis control prosthetic limbs and communicate through thought.

Expect ~60% accuracy. Chance is 50%. In BCI research, 60% on a binary task is a meaningful result. Don’t expect 95%!

New LLM Skill: Complex Debugging

Week 2: Prompting · Week 4: Debugging · Week 6: Refactoring · Week 8: Documentation · Week 10: Complex Debugging

Weak

“My neural network doesn’t work.”

Strong

“Training loss stays flat at 0.693 (ln(2)) after 50 epochs. Accuracy hovers at 50%. I’m using CrossEntropyLoss + Adam(lr=0.001), 320 input features, 64 hidden units, ReLU, dropout(0.3). Here’s my full training loop: [paste]. What could be wrong?”

No traceback = no easy fix. When a neural network silently fails, you must describe the symptoms — loss curve shape, expected vs actual accuracy, data characteristics.

Getting Started

Steps

  1. Open starter.ipynb or starter.py
  2. Run cells 1–5 (data loaded, explored, split & scaled)
  3. Ask your AI to plan first
  4. Phase 1: LogReg baseline (sklearn)
  5. Phase 2: Build & train MLP (PyTorch)
  6. Phase 3: Compare, diagnose, reflect, present

Target Numbers

Chance: 50.0%

LogReg baseline: ~60% accuracy

Simple MLP: ~58–62% (may not beat LogReg!)

Key diagnostic: loss curve should decrease below 0.693