learning_ai_common_plat/docs/workstreams/MOBILE_WORKSTREAM_REMAINING.md

Mobile Workstream — Remaining Items

Last updated: 2026-02-15 Sources:

• learning_voice_ai_agent/mobile_app/common/docs/MOBILE_APP_PLAN.md
• learning_multimodal_memory_agents/docs/IMPLEMENTATION_PLAN_v2.md

iOS Progress Log (This Machine)

• 2026-02-15: MindLyst iOS voice capture (AVAudioEngine 16kHz mono) + live transcription wired into Capture screen.
  • https://github.com/saravanakumardb1/learning_multimodal_memory_agents/commit/3b9c44a
• 2026-02-15: MindLyst iOS deep links + WidgetKit quick-capture widget source (requires adding Widget Extension target in Xcode).
  • https://github.com/saravanakumardb1/learning_multimodal_memory_agents/commit/e31f8f4
• 2026-02-15: MindLyst iOS Azure Speech SDK streaming STT scaffold (compile-safe placeholder; still needs SDK + wiring).
  • https://github.com/saravanakumardb1/learning_multimodal_memory_agents/commit/8834322
• 2026-02-15: MindLyst iOS Share Extension scaffold (URLs/text/images; needs Xcode Share Extension target + App Group follow-up).
  • https://github.com/saravanakumardb1/learning_multimodal_memory_agents/commit/298236c
• 2026-02-15: MindLyst iOS now builds as a real Xcode project (XcodeGen), including Widget + Share Extension targets; iOS 16 compatibility fixes.
  • https://github.com/saravanakumardb1/learning_multimodal_memory_agents/commit/40af991
• 2026-02-15: LysnrAI iOS sound + haptic feedback for record start/stop (uses existing Settings toggle sound_effects).
  • https://github.com/saravanakumardb1/learning_voice_ai_agent/commit/2e79ed1
• 2026-02-15: MindLyst iOS MemoryStore + CaptureModeKind now power the Home feed, Capture screen, and triage confirmation flow with persistent data.
  • https://github.com/saravanakumardb1/learning_multimodal_memory_agents/commit/84f72d0
• 2026-02-15: LysnrAI iOS TestFlight Build 7 + Build 8 uploaded to App Store Connect (entitlements workaround for provisioning profile).
  • https://github.com/saravanakumardb1/learning_voice_ai_agent/commit/8f852f3
• 2026-02-15: LysnrAI iOS + backend + web: delete individual entries and clear-all history support.
  • https://github.com/saravanakumardb1/learning_voice_ai_agent/commit/bdb9f3f
• 2026-02-15: LysnrAI: dictation sessions design doc — push-to-talk, dual-prompt (transcription hint + composition prompt), multi-format export (PDF/Markdown/text).
  • https://github.com/saravanakumardb1/learning_voice_ai_agent/commit/5ac6adf
• 2026-02-15: Production readiness checks passed: 568 desktop + 92 backend tests, all 3 dashboards typecheck clean.

Home Laptop Priority (No Proxy Issues)

• LysnrAI: smoke-test the audio PoC on at least 1 real iPhone + 1 real Android device
• MindLyst: verify KMP build (iOS simulator): cd learning_multimodal_memory_agents/mindlyst-native && GRADLE_USER_HOME=/tmp/gradle ./gradlew :shared:compileKotlinIosSimulatorArm64 (verified 2026-02-15)
• Visual consistency spot-check: design tokens across MindLyst iOS/Android/web previews

LysnrAI Mobile (iOS + Android)

Phase 1 — MVP (Finish Week 4-5)

• Text cleanup client (Android + iOS)
• License activation flow (auth/licensing)
• Cloud sync (settings + history)
• Settings persistence (local + synced)
• Copy/share output — swipe-to-share in HistoryView + delete individual/all history
• Sound feedback (iOS done) — https://github.com/saravanakumardb1/learning_voice_ai_agent/commit/2e79ed1
• Stats screen (wire UI to real data)
• Polish + bug fixes
• Device testing (Pixel 7/S24, Pixel 5, iPhone 15, iPhone 13)
• TestFlight build (internal) + metadata sanity — Build 7 + Build 8 uploaded (2026-02-15)
• Play Store internal testing track build
• Close "design mockups" item: create mocks or explicitly skip (screens already fleshed out)

Phase 2 — Enhanced Input

• iOS keyboard extension
• Android input method
• Home screen widgets (both platforms)
• Android Quick Settings tile
• Haptic feedback
• Offline STT exploration

Phase 3 — Advanced

• Apple Watch / Wear OS companion
• Siri Shortcuts / Google Assistant integration
• Live transcription mode (continuous, no cleanup)
• Multi-language switching
• iPad-optimized layout

MindLyst Native (KMP + iOS + Android)

Phase 0 — Setup/Infra (carryovers)

• P1 — GitHub Actions: Android debug build on PR (blocked: no Android SDK in CI yet)
• P3 — Fastlane setup for iOS TestFlight + Android Play Store beta
• P1 — Set up Stripe account for MindLyst subscriptions (separate from LysnrAI Stripe)
• Cosmos DB accessible with test read/write
• CI runs on every PR

Phase 1 — Capture + Triage Foundation (Week 1-2)

Voice capture pipeline

• P0 — Implement microphone recording on iOS (AVAudioEngine, 16kHz mono) — https://github.com/saravanakumardb1/learning_multimodal_memory_agents/commit/3b9c44a
• P0 — Implement microphone recording on Android (AudioRecord, 16kHz mono)
• P0 — Integrate Azure Speech SDK for real-time STT on iOS (scaffolded) — https://github.com/saravanakumardb1/learning_multimodal_memory_agents/commit/8834322
• P0 — Integrate Azure Speech SDK for real-time STT on Android
• P1 — Upload raw audio to Azure Blob Storage after capture
• P1 — Backend: user-scoped SAS URL generation for Blob uploads (platform-service POST /api/blob/sas)

Text capture

• P0 — Backend: persist raw text to memory_items Cosmos container (platform-service POST /api/memory-items)
• P0 — Mobile: call platform-service POST /api/memory-items during capture/triage flow
• iOS: MindLyst Capture confirm now POSTs memory-items (dev token in Settings)
  • https://github.com/saravanakumardb1/learning_multimodal_memory_agents/commit/3511124
• Android: dev settings + Confirm now POSTs memory-items (mock triage cards)
  • https://github.com/saravanakumardb1/learning_multimodal_memory_agents/commit/d589294

Image/screenshot capture

• P0 — Camera capture button on iOS + Android
• P0 — Upload image to Blob Storage, store reference in memory_items

Triage persistence

• P0 — Backend: store structured MemoryItem in Cosmos DB with triage fields (platform-service memory_items)
• P0 — Mobile: wire triage confirmation to send triageResult + brainIds to backend (POST /api/memory-items)
• iOS: send triageResult on create
  • https://github.com/saravanakumardb1/learning_multimodal_memory_agents/commit/94b1a19
• Android: send triageResult on create (mock flow)
  • https://github.com/saravanakumardb1/learning_multimodal_memory_agents/commit/098e8c2

Blob persistence

• P1 — Backend: accept blob media refs in memory-items create payload (audio/image)
  • https://github.com/saravanakumardb1/learning_ai_common_plat/commit/e64bba2

Widgets

• P0 — iOS WidgetKit: home screen widget with Voice/Text/Photo buttons (source scaffolded) — https://github.com/saravanakumardb1/learning_multimodal_memory_agents/commit/e31f8f4
• P0 — Android AppWidget: home screen widget with Voice/Text/Photo buttons
• P1 — Widget voice capture -> STT -> triage (without opening app)
• P1 — Haptic confirmation on successful capture
• P2 — Lock screen widget variant (iOS 16+ Live Activities)

Phase 1 exit criteria

• User can capture voice, text, image, and link on iOS + Android
• Widget captures without opening app
• All data persisted to Cosmos DB + Blob Storage

Phase 2 — Brain Interaction + Daily Loop (Week 3-4)

• P0 — Voice input for brain questions (speech-to-text)
• P1 — Spoken response (TTS) for brain answers
• P0 — Store generated brief in DailyBrief Cosmos container (currently in-memory)
• P1 — Audio brief: 30-60 second spoken summary (TTS)
• P0 — iOS Share Extension: receive URLs, text, images from any app (scaffolded) — https://github.com/saravanakumardb1/learning_multimodal_memory_agents/commit/298236c
• P0 — Android Share target: receive intents for URLs, text, images
• Share sheet captures content from any app (native only)

Phase 3 — Weekly Reflection + Share (Week 5-6)

• P0 — Store in ReflectionReport Cosmos container (currently in-memory)
• P1 — Audio digest: 2-minute spoken reflection (TTS)
• P0 — Share via native share sheet (iOS/Android)

Phase 4 — Insights (Week 7-8)

• P1 — Store insights in BrainInsight Cosmos container (currently in-memory)

Phase 5 — Reliability / Performance / Privacy (Week 9-10)

• P1 — Widget works fully offline (queues for sync)
• P0 — Widget capture to haptic confirmation < 1s
• P1 — App cold start < 2s on modern devices
• P0 — Encryption in transit (HTTPS/TLS) and at rest (Cosmos + Blob)
• P1 — Crash reporting (Firebase Crashlytics or equivalent)

Phase 6 — Beta / Store Submission (Week 11-12)

• P0 — App Store submission (TestFlight) with all metadata
• P0 — Google Play internal testing track
• Beta build submitted to TestFlight + Play Store
• P0 — App Store review submission (allow 1 week for review)
• P1 — App Store screenshots: 5 screens per device size (iPhone, iPad, Android)

Phase 8+ — Post-MVP Add-ons

• P1 — watchOS app: raise wrist -> speak -> capture
• P1 — Morning brief glance on watch face
• P1 — Haptic tap for forgotten item nudges
• P2 — Complication showing streak count
• P1 — Read-only calendar access (EventKit iOS, CalendarProvider Android)
• P1 — capture@mindlyst.app email address per user