Current Tasks

• [x] display interim text for google stt response on top of text input (EST: 1h)
• [ ] saving the audio for the last 10 seconds at all times to avoid record delay after start keyword detected (Jacob: probably < 10 sec is ok, 5 sec?. Also, I’m ok with alternate approaches to achive this same goal) (EST: 3h)
  • [ ] Jacob Re: saving the audio for the last 10 seconds at all times to avoid record delay after start keyword detected I think this is a pretty good approach. I’m also open to other approaches if you think of them. For instance, no pressure to do this, but if you see fit you could run Google streaming speech to text all the time and parse it. We could even make a version of the app that can still work even if a call to whisper fails (it would fall back on Google). It’s unnecessary to do that If it would take a bunch of extra time for now.
• [ ] It seems that if I ask a question that it gives a long answer to. Every subsequent question I ask it returns the error. “call to whisper, failed” cause by “teach me about Spinoza” (EST: 1h-2h, need time to replicate the issue)
• [ ] Improve “silence detection”: the problem with current silence detection is that if there's any background noise, Even not speech, it doesn’t detect silence. In additional to the current method, can we additionally consider silence as detected for the sake of the end word detection if google STT doesn’t return anything for x number of seconds? @Jacob Cole I do not understand this phrase, please elaborate.
• [ ] Test: Try having a 5 minute conversation with the app, asking a series of longer-reply questions, like “teach me about spinoza”, and make sure it doesn’t crash (EST: 30m)

—

React Native

• [ ] make the react native app work when you are in other applications or on the lock screen (require more time, this is better to handover to external)

—

• [x] transfer credentials for frontend useWhisper to something secure and backend? (EST: 1h)

Links

https://talktogpt-git-google-stt-ra.ideaflow.app/chat • https://talktogpt.ideaflow.app/ • http://talktogpt.io/ • https://github.com/IdeaFlowCo/talktogpt

https://www.figma.com/file/J8F2KJ8dixos0Uxrs71CpF/TalkToGPT-%23Design?type=design&node-id=0%3A1&mode=design&t=lYjFD0jdbS4qMazS-1 • v1.0 Final ‣