spotlightcam/docs/TODO.md

# TODO - spotlight.cam

**Active tasks and roadmap**

---

## Current Status

**Phase:** MVP Complete - Production Ready
**Tests:** 350/350 passing - 100% ✅ (73% coverage)
**Recent Work:**
- 2025-11-30: Matching runs audit, ratings & stats system, comprehensive test suite
- 2025-11-30: Rate limiting & spam protection, socket notifications for suggestions

**Full implementation history:** See `docs/archive/COMPLETED.md`

---

## Activity Log System (In Progress)

**Status:** Phase 5/8 Complete (Backend Complete ✅)
**Started:** 2025-12-02
**Commits:** `f9cdf2a` (Ph1), `c9beee9` (Ph2), `d83e416` (Ph3), `4dd6603` (Ph4), `d641e3f` (Ph5)
**Admin User:** spotlight@radziel.com (password: Dance123!)

### Purpose
Comprehensive activity logging system for admin monitoring with real-time streaming dashboard.
- Track all user actions (auth, events, matches, chat, admin)
- Real-time WebSocket streaming (like `tail -f`)
- Filter by time range, action type, username
- Admin-only access with `requireAdmin` middleware

### Completed ✅

**Phase 1: Database Schema**
- ✅ ActivityLog model with indexes (43 lines)
- ✅ User.isAdmin flag for access control
- ✅ Admin user created: spotlight@radziel.com
- **Files:** `backend/prisma/schema.prisma`

**Phase 2: Backend Services**
- ✅ ActivityLog service (300+ lines) - centralized logging, fire-and-forget pattern
  - 18 action constants (AUTH_LOGIN, MATCH_CREATE, etc.)
  - Query interface with filtering
  - Socket.IO emission for real-time
  - Statistics and action types endpoints
- ✅ Request utility - IP extraction (X-Forwarded-For support)
- ✅ Admin middleware - `requireAdmin()` protects admin routes
- **Files:**
  - `backend/src/services/activityLog.js`
  - `backend/src/utils/request.js`
  - `backend/src/middleware/admin.js`

**Phase 3: Logging Integration (14 actions)**
- ✅ Auth controller: register, login, verify email (token & code), password reset (4 actions)
- ✅ Events routes: checkin, leave event (2 actions)
- ✅ Socket handlers: join event chat, leave event chat, join match room (3 actions)
- ✅ Matches routes: create, accept, reject match (3 actions)
- ✅ Admin routes: matching run + secured all routes with requireAdmin (1 action)
- **Files:**
  - `backend/src/controllers/auth.js`
  - `backend/src/routes/events.js`
  - `backend/src/socket/index.js`
  - `backend/src/routes/matches.js`
  - `backend/src/routes/admin.js`

**Phase 4: Admin API Endpoints**
- ✅ `GET /api/admin/activity-logs` - Query logs with filters (date range, action, category, username, success, pagination)
- ✅ `GET /api/admin/activity-logs/actions` - Get unique action types
- ✅ `GET /api/admin/activity-logs/stats` - Statistics dashboard (total, failures, by category, 24h activity)
- ✅ ADMIN_VIEW_LOGS action logging
- **File:** `backend/src/routes/admin.js`

**Phase 5: Socket.IO Real-Time Streaming**
- ✅ `join_admin_activity_logs` handler with admin verification
- ✅ `leave_admin_activity_logs` handler
- ✅ Emits `activity_log_entry` to admin room (already in Phase 2 service)
- ✅ Fresh DB check for admin status on join
- **File:** `backend/src/socket/index.js`

### Remaining Tasks (Frontend)

**Phase 6-7: Frontend Admin Page**
- [ ] Create `frontend/src/pages/admin/ActivityLogsPage.jsx`
- [ ] Stats dashboard (total logs, failures, by category)
- [ ] Filter UI (date range, action, username)
- [ ] Log table with pagination
- [ ] Real-time streaming toggle with auto-scroll
- [ ] Add navigation link for admins

**Phase 8: Testing & Polish**
- [ ] Test all 14 action logging points
- [ ] Test admin-only access enforcement
- [ ] Test real-time streaming with multiple admins
- [ ] Mobile responsive design
- [ ] Documentation

### Implementation Notes
- **Fire-and-forget**: Logging never blocks requests or crashes app
- **Denormalized**: Username stored to avoid JOINs
- **Scalability**: Partitioning after 10M+ rows
- **Security**: Admin-only with fresh DB checks

---

## Matching System - Comprehensive Test Scenarios

**Last Updated:** 2025-11-30
**Status:** Documented for team discussion

### Implementation Status

#### ✅ Implemented Scenarios (See docs/archive/COMPLETED.md for details)
- **S1-S3:** Basic flow, collision detection, limits - 19 integration tests
- **S7.1-7.2:** Manual match blocks auto suggestions - Implemented & tested
- **S10:** Ratings & Stats System - 9 E2E tests (atomic updates, race prevention)
- **S11.3-11.4:** Matching Run Details API - Admin endpoints with filtering
- **S12:** Multi-heat collision detection - Covered in matching algorithm tests
- **S14.1:** Only recorder can accept/reject - Implemented in MVP
- **S15.1-15.2:** Rate Limiting & Spam Protection - 8 comprehensive tests
  - Max 20 pending outgoing match requests
  - Rate limit: 10 match requests per minute
- **S16.1:** Socket Notifications - Real-time notification when new suggestion created
- **Matching Runs Audit:** 6 comprehensive tests (origin_run_id tracking)

#### 🔴 Critical Gaps (P0 - Before Production)

1. **S14.2: Admin Middleware** - **SECURITY**
   - Admin endpoints not protected: `/admin/events/:slug/run-now`, `/admin/matching-runs`
   - Need: `requireAdmin` middleware

2. **S14.3: Event Participant Validation** - **SECURITY**
   - Inconsistent checks across endpoints
   - Need: Audit all suggestion/match endpoints for participant validation

#### ⚠️ High Priority (P1 - First Month)

3. **E9/S13.2: Manual match created AFTER auto suggestion**
   - Current: Manual blocks only NEW auto suggestions, old pending remain
   - Need: Cleanup conflicting pending auto suggestions when manual match created

#### 📋 Medium Priority (P2 - Q1 2025)

6. **S15.3: Zombie Matches Cleanup**
   - Auto-cancel pending matches older than 30 days

7. **S16.3: Email Reminders**
   - Reminder before event for accepted recording assignments

### Test Scenarios by Category

<details>
<summary><b>S1: BASIC FLOW</b> ✅ Implemented</summary>

#### S1.1: Happy path - basic assignment
- **Given:** Event with deadline, user A has heat H1, user B no conflict
- **When:** Matching runs after deadline
- **Then:** Creates suggestion B records A in H1 (pending)

#### S1.2: Recorder accepts suggestion
- **Given:** Pending suggestion: B records A
- **When:** B clicks Accept
- **Then:** Status→accepted, creates Match (type: auto), both see on /matches

#### S1.3: Recorder rejects suggestion
- **Given:** Pending suggestion: B records A
- **When:** B clicks Reject
- **Then:** Status→rejected, A sees read-only, next run may assign different recorder

</details>

<details>
<summary><b>S2: TIME COLLISION DETECTION</b> ✅ Implemented</summary>

#### S2.1: Can't record while dancing
- **Given:** A has heat H1, B has heat H1 (same slot)
- **Then:** B NOT candidate to record A in H1

#### S2.2: Buffer before - preparation time
- **Given:** A has heat H5, B has heat H6 (next)
- **When:** HEAT_BUFFER_BEFORE = 1
- **Then:** B NOT candidate (needs H5 for prep)

#### S2.3: Buffer after - rest time
- **Given:** A has heat H5, B has heat H4 (previous)
- **When:** HEAT_BUFFER_AFTER = 1
- **Then:** B NOT candidate (needs H5 for rest)

#### S2.4: Schedule config - divisions in same slot
- **Given:** Schedule config: slot 1 = [Open, Newcomer]; A has Open H3, B has Newcomer H3
- **Then:** B can't record A (same slot despite different divisions)

</details>

<details>
<summary><b>S3: LIMITS & CONSTRAINTS</b> ✅ Implemented</summary>

#### S3.1: Max 3 recordings per person
- **Given:** B already assigned to record 3 others
- **Then:** B NOT candidate for dancer A

#### S3.2: Opt-out from recording
- **Given:** B checked "I don't want to record others"
- **Then:** B completely excluded as potential recorder

#### S3.3: No self-recording
- **Given:** A has heat H1
- **Then:** A can't be recorder for themselves

</details>

<details>
<summary><b>S4: FAIRNESS & TIER SYSTEM</b> ✅ Implemented (but stats not updating!)</summary>

#### S4.1: BASIC user - normal frequency
- **Given:** C (BASIC): done=5, received=0; A,B (BASIC): done=0, received=0
- **Then:** C has priority (fairness debt = -5)

#### S4.2: SUPPORTER user - reduced frequency (-10)
- **Given:** A (SUPPORTER) vs B (BASIC), both 0/0
- **Then:** B has priority (A has -10 penalty)

#### S4.3: COMFORT user - significantly reduced (-50)
- **Given:** A (COMFORT), B (SUPPORTER), C (BASIC), all 0/0
- **Then:** Priority: C > B > A

#### S4.4: Location preference
- **Given:** Dancer A in LA,US; B (LA,US), C (NYC,US), D (London,UK)
- **Then:** Priority: B > C > D

</details>

<details>
<summary><b>S5: INCREMENTAL MATCHING</b> ✅ Implemented</summary>

#### S5.1: Accepted suggestions preserved
- **Given:** Run 1 created suggestion B→A (pending), B accepted
- **When:** Run 2
- **Then:** A-B suggestion NOT deleted or overwritten

#### S5.2: Pending suggestions regenerated
- **Given:** Run 1 created B→A (pending), B didn't accept
- **When:** Run 2 (new users joined)
- **Then:** Pending A-B deleted, new suggestion may assign C→A

#### S5.3: Accepted suggestions block slots
- **Given:** B accepted recording A in heat H5
- **When:** Run 2 for dancer D (also has H5)
- **Then:** B NOT candidate (busy in H5)

</details>

<details>
<summary><b>S6: SCHEDULER</b> ✅ Implemented</summary>

#### S6.1: Waits until deadline
- **Given:** Event has registrationDeadline in 2 hours
- **Then:** Scheduler does NOT run matching (waits)

#### S6.2: Runs after deadline
- **Given:** registrationDeadline passed 1 minute ago
- **Then:** Runs matching (run 1/5)

#### S6.3: 5 runs at 5-minute intervals
- **Then:** Run 1 (immediately), Run 2 (+5min), Run 3 (+10min), Run 4 (+15min), Run 5 (+20min), STOP

#### S6.4: Audit trail
- **Given:** Matching ran
- **Then:** Admin sees table of runs with timestamps and stats (total/pending/accepted/rejected/not_found)

</details>

<details>
<summary><b>S7: MANUAL vs AUTO MATCHES</b> ✅ S7.1-7.4 Implemented</summary>

#### S7.1: Manual match blocks auto suggestion ✅
- **Given:** A sent manual match request to B (pending)
- **When:** Matching algorithm runs
- **Then:** Does NOT create auto suggestion A↔B or B↔A

#### S7.2: All manual statuses block ✅
- **Given:** Manual match A↔B in status: pending/accepted/completed
- **Then:** All cases block auto suggestion

#### S7.3: /matches shows both types with badges ✅
- **Given:** A has manual match with B (pending), auto suggestion with C (pending - C records A)
- **Then:** A sees 2 items: B (badge "Manual"), C (badge "Auto")

#### S7.4: Auto pending - redirect to Records ✅
- **Given:** A has auto suggestion with C (pending, C records A)
- **Then:** Button "Go to Records" → /events/X/chat?tab=records (no Accept/Reject for dancer)

</details>

<details>
<summary><b>S10: RATINGS & STATS</b> ✅ IMPLEMENTED (2025-11-30)</summary>

#### S10.1: Auto match completed → stats updated exactly once ✅
- **Given:** Auto Match A↔B (source='auto', statsApplied=false), both rated
- **When:** Second rating submitted
- **Then:**
  - `recordingsDone++` for recorder (user2)
  - `recordingsReceived++` for dancer (user1)
  - `match.status = 'completed'`
  - `match.statsApplied = true`
- **Implementation:** `backend/src/routes/matches.js:961-995` (atomic check-and-set)
- **Tests:** `backend/src/__tests__/ratings-stats-flow.test.js` (STEP 6b)

#### S10.2: Only one rated → no stats ✅
- **Given:** Auto Match A↔B, only A rated
- **Then:** `statsApplied` stays false, stats don't change
- **Tests:** `backend/src/__tests__/ratings-stats-flow.test.js` (STEP 6a)

#### S10.3: Manual match completion → no stats update ✅
- **Given:** Match A↔B (source='manual'), both rated
- **Then:** Stats don't change (manual matches don't affect fairness)
- **Implementation:** `backend/src/services/matching.js:682` (early return if source !== 'auto')
- **Tests:** `backend/src/__tests__/ratings-stats-flow.test.js` (STEP 8)

#### S10.4: Rating edit → no double counting ✅
- **Given:** User tries to rate same match twice
- **Then:** 400 error "already rated", stats unchanged
- **Implementation:** Unique constraint: `(matchId, raterId, ratedId)`
- **Tests:** `backend/src/__tests__/ratings-stats-flow.test.js` (STEP 7)

**Implementation:**
```javascript
// Match model (Prisma schema)
source: String // 'auto' | 'manual'
statsApplied: Boolean @default(false)
suggestionId: Int? // null for manual matches

// Stats application (backend/src/services/matching.js:679-701)
async function applyRecordingStatsForMatch(match) {
  if (match.source !== 'auto') return; // Manual matches ignored

  await prisma.$transaction([
    prisma.user.update({
      where: { id: match.user2Id }, // recorder
      data: { recordingsDone: { increment: 1 } }
    }),
    prisma.user.update({
      where: { id: match.user1Id }, // dancer
      data: { recordingsReceived: { increment: 1 } }
    })
  ]);
}
```

</details>

<details>
<summary><b>S11: AUDIT & origin_run_id</b> ✅ Partially Implemented</summary>

#### S11.1: New suggestions get correct origin_run_id ✅
- **Given:** Run #5 runs matching
- **Then:** Each new recording_suggestion has origin_run_id = 5

#### S11.2: Existing origin_run_id preserved ✅
- **Given:** Suggestion S1 from run #3 (accepted/completed)
- **When:** Run #4
- **Then:** S1 keeps origin_run_id=3 (not deleted)

#### S11.3: Get suggestions by run 🔴 NOT IMPLEMENTED
- **Need:** `GET /matching-runs/:id/suggestions`
- **Then:** Returns only suggestions with origin_run_id=:id

#### S11.4: Filters work correctly 🔴 NOT IMPLEMENTED
- **Need:** Query params `onlyAssigned`, `includeNotFound`
- **Then:** Filters recorder presence and NOT_FOUND status

</details>

<details>
<summary><b>S12: MULTIPLE HEATS & COMPLEX COLLISIONS</b> ✅ Implemented</summary>

#### S12.1: Same dancer in multiple heats
- **Given:** Dancer A in H10 and H12 (Novice J&J), B no conflict
- **Then:** B can record A in both heats (if under limit 3)

#### S12.2: Recorder records multiple dancers in different slots
- **Given:** Recorder B has zero own heats, A has H5 (slot 2), C has H8 (slot 3)
- **Then:** B can be assigned to both A and C (different slots)

#### S12.3: Same recorder can't record two people in same slot
- **Given:** B accepted recording A in Novice H10 (slot 2), D has Intermediate H10 (also slot 2)
- **When:** Next matching run for D
- **Then:** B excluded (busySlot recording A)

</details>

<details>
<summary><b>S13: MANUAL vs AUTO - FURTHER INTERACTIONS</b> ⚠️ Edge Case</summary>

#### S13.1: Manual before matching ✅ DONE
- See S7.1

#### S13.2: Manual created AFTER auto suggestion ⚠️ EDGE CASE
- **Given:** Run #1 created auto suggestion A↔B (pending), then user A creates manual match with B
- **When:** Run #2
- **Then:**
  - **Current behavior:** Manual blocks NEW auto suggestions, old pending remains
  - **Recommended:** Cleanup conflicting pending auto suggestions when manual created

**Recommended implementation:**
```javascript
// routes/matches.js - POST /
// After creating manual match:
await prisma.recordingSuggestion.deleteMany({
  where: {
    eventId,
    OR: [
      { heatId: { in: user1HeatIds }, recorderId: user2Id },
      { heatId: { in: user2HeatIds }, recorderId: user1Id },
    ],
    status: 'pending', // only pending, don't touch accepted
  }
});
```

</details>

<details>
<summary><b>S14: PERMISSIONS & VISIBILITY</b> ⚠️ Partially Implemented</summary>

#### S14.1: Only recorder can update status ✅
- **Given:** Suggestion B records A (pending), A tries PUT /suggestions/:id/status
- **Then:** 403 Forbidden

#### S14.2: Admin-only endpoints 🔴 NOT SECURE
- **Given:** Regular user tries POST /admin/events/:slug/run-now or GET /admin/matching-runs
- **Then:** Should get 403, but **middleware missing!**

**Need to implement:**
```javascript
// middleware/auth.js
const requireAdmin = (req, res, next) => {
  if (req.user.role !== 'admin') {
    return res.status(403).json({ error: 'Admin access required' });
  }
  next();
};
```

#### S14.3: Event participant visibility ⚠️ INCONSISTENT
- **Need to audit:** Do all suggestion/match endpoints verify user is event participant?
- **Locations to check:**
  - GET /api/matches?eventSlug=X
  - GET /api/events/:slug/matching/suggestions
  - Frontend: /events/:slug/chat?tab=records

</details>

<details>
<summary><b>S15: SECURITY & ABUSE PREVENTION</b> 🔴 NOT IMPLEMENTED</summary>

#### S15.1: Rate limiting on manual match requests
- **Given:** User A sends 100 manual requests in 1 minute
- **Then:** 429 Too Many Requests after 10 requests/minute

#### S15.2: Max pending requests limit
- **Given:** User A has 20 pending outgoing requests
- **Then:** Can't send more (limit 20)

#### S15.3: Zombie matches cleanup
- **Given:** Pending match older than 30 days
- **Then:** Auto-cancelled by cron job

</details>

<details>
<summary><b>S16: NOTIFICATIONS & REAL-TIME</b> 🔴 NOT IMPLEMENTED</summary>

#### S16.1: Socket event for new suggestion
- **Given:** Matching created suggestion B records A, B is online
- **Then:** B receives socket event → toast notification

#### S16.2: Push notification for offline users
- **Given:** B offline when suggestion created
- **Then:** Email/push: "You have a new recording assignment"

#### S16.3: Reminder before event
- **Given:** B accepted recording A, event in 2 days
- **Then:** Email: "Don't forget to record @userA on Saturday"

</details>

### Edge Cases for Discussion

#### E1: User has 10 heats, no one can record
- **Question:** Show all as NOT_FOUND? Prioritize which heats are most important?

#### E2: Recorder wants to un-accept
- **Current:** No "un-accept" option
- **Question:** Add cancel feature for accepted suggestions?

#### E3: Manual match rejected, then matching runs
- **Current:** Rejected manual still blocks auto suggestions
- **Question:** Should rejected manual stop blocking?

#### E4: User joins AFTER all 5 scheduler runs
- **Current:** Won't get auto suggestions (scheduler stopped)
- **Question:** Allow admin to manually trigger additional run?

#### E5: Timezone for deadline
- **Current:** Server timezone
- **Question:** Use event/organizer timezone?

#### E6: Load balancing - one recorder has best scores
- **Current:** Can get max 3 assignments
- **Question:** Distribute more evenly even if scores worse?

#### E7: Dancer rejects auto suggestion
- **Current:** Dancer CAN'T reject (only recorder in MVP)
- **Question:** Allow dancers to reject in future?

#### E8: Event without schedule config
- **Current:** Fallback to division-based slots
- **Question:** Show admin warning that matching may be less efficient?

#### E9: Manual match created AFTER auto suggestion exists
- See S13.2 above

#### E10: User joins BETWEEN scheduler runs
- **Current:** ✅ Works - each run regenerates pending suggestions
- **Status:** No issue

#### E11: Recorder accepted but didn't actually record
- **Question:** Timeout (7 days after event → expired)? "Report issue" button? Admin override?

### Priority Ranking

#### P0 - CRITICAL (Before Production)
1. ✅ Manual blocks auto suggestions (DONE 2025-11-30)
2. 🔴 **Implement ratings and stats system** (S10 - fairness broken without this!)
3. 🔴 **Admin middleware** (S14.2 - security)
4. 🔴 **Event participant validation** (S14.3 - security)

#### P1 - HIGH (First Month of Production)
5. ⚠️ Manual match cleanup on conflict (S13.2 / E9)
6. ⚠️ Rate limiting and spam protection (S15.1, S15.2)
7. ⚠️ Socket notifications for new suggestions (S16.1)

#### P2 - MEDIUM (Q1 2025)
8. 📋 Endpoint /matching-runs/:id/suggestions with filters (S11.3, S11.4)
9. 📋 Zombie matches cleanup (S15.3)
10. 📋 Email reminders (S16.3)

#### P3 - LOW (Nice to Have)
11. 📋 Report issue for dancers (E11)
12. 📋 Admin manual override for suggestions
13. 📋 Analytics dashboard (success rate, acceptance rate)

### Questions for Team

1. **Is ratings system in roadmap?** If yes - THIS IS THE MOST IMPORTANT (fairness is broken without it)
2. **Do we have existing admin middleware?** If not - must add ASAP for security
3. **Which edge case (E1-E11) is most likely in production?**
4. **Should we implement S13.2 cleanup before launch?** (manual match after auto suggestion)

---

## Security Audit Findings

### Critical Issues (Must Fix Before Production)

| Issue | Severity | File | Action |
|-------|----------|------|--------|
| AWS Credentials in Git | 10/10 | `.env.production` | Rotate & remove from history |
| Weak JWT Secret | 9/10 | `.env.production` | Generate 64+ byte secret |
| Default DB Password | 8/10 | `docker-compose.yml` | Remove default fallback |

### High Priority Issues

| Issue | Severity | Action |
|-------|----------|--------|
| Missing HTTPS/TLS | 8/10 | Configure SSL certificate |
| Missing nginx security headers | 6/10 | Add X-Frame-Options, CSP, etc. |
| Dependency vulnerabilities | HIGH | Run `npm audit fix` |
| Excessive body size (500M) | 4/10 | Reduce to 10M |

### Positive Security Findings
- Strong authentication (JWT, bcrypt, email verification)
- Input validation (express-validator)
- Security headers (Helmet.js)
- Rate limiting implemented
- CORS configured
- SQL injection prevention (Prisma)
- Account lockout implemented
- WebRTC P2P (videos don't touch server)

---

## Pre-Deployment Checklist

- [ ] Rotate AWS credentials, remove from Git history
- [ ] Generate strong JWT_SECRET (64+ bytes)
- [ ] Set strong PostgreSQL password
- [ ] Configure HTTPS/TLS
- [ ] Add nginx security headers
- [ ] Run npm audit fix
- [ ] Reduce nginx body size limit

---

## Future Improvements (Optional)

### UX Enhancements
- Activity Feed (timeline of user activities)
- Smart sort order (unread first, pending ratings)
- User statistics (total matches, average rating)
- Sidebar filters (by nationality, division)

### Security Hardening
- Increase bcrypt rounds (10 → 12)
- Implement refresh token pattern
- Add Socket.IO rate limiting
- Sanitize chat messages

### Infrastructure
- CI/CD pipeline (GitHub Actions)
- E2E tests (Playwright)
- Security logging

---

## Progress Summary

| Phase | Status | Time |
|-------|--------|------|
| Phase 0: Frontend Mockup | ✅ Done | ~8h |
| Phase 1: Backend Foundation | ✅ Done | ~14h |
| Phase 1.5: Email & WSDC & Profiles | ✅ Done | ~12h |
| Phase 1.6: Competition Heats | ✅ Done | ~8h |
| Phase 2: Matches & Ratings | ✅ Done | ~10h |
| Phase 2.5: WebRTC P2P | ✅ Done | ~10h |
| Phase 3: MVP Finalization | ✅ Done | ~20h |
| **Total MVP** | **✅ Complete** | **~82h** |

---

## Quick Commands

```bash
# Development
docker compose up --build

# Tests
docker compose exec backend npm test

# Access
http://localhost:8080
```

---

**Last Updated:** 2025-11-29
**Full Details:** See `docs/archive/COMPLETED.md` for implementation details