Private LLM - Local AI Chatbot

First, I have an iPhone 14 Pro, and the app has layout issues. There's no option to save temperature for next launch. Opening llama 3 will crash the app. Shortcuts functionality is poor, as it doesn't allow custom system prompts only queries. Also, executing a shortcut function opens the app instead of quietly running in the background. Fixing these issues would instantly make this review a 5-star one

The app is shipped with RedPajamas 3B model, and it offers to download an unnamed 7B model, but if you refuse or close the app until the download is completed, it won’t offer it ever again until you completely reinstall the app. Model selection and download seems to be supported only on MacOS version, regardless of what written in patch notes and on the website. Sad, because i was really hoping to try Phi-3 on iOS.

Добавьте выбор любой модели - и будет вообще супер. Огромное спасибо за вашу работу. Отлично работает на iphone и mac. Единственное, что представленные модели очень плохо работают с русским языком. Еще раз спасибо за то, что вы делаете)

The app seems more stable than the others I’ve tried. It would be good to see the ability to browse and add models from huggingface and to have RAG available document querying.

When I first tried this app, it was a letdown - just one slow option and I wasn't impressed. But man, was I wrong! The developer has put in some serious hard work, pulling off a 'No Man’s Sky' style transformation. Now, it's packed with well performing models for whichever device you have presently, and honestly it’s become my go-to. Also, I will always appreciate and recommend this app based on its privacy oriented USP alone - because finding apps that care about privacy (especially in a field so ripe for exploit) feels like panning for gold in a septic tank. I’ve gone from regretting my purchase to relying on it daily. It’s versatile, reliable, and an absolute game-changer for anyone looking to work with LLMs privately. I've got to give a huge shoutout to the developers for their commitment to turning things around. Your hard work hasn’t gone unnoticed, and I'm sincerely grateful for it. Wishing you all the success in the future - you’ve got a fan in me! Had to update my review to give the devs a huge thumbs up for such an impressive turnaround!

After I saw the only review here on the App Store that’s nothing but gibberish, I had to write my own. I love that this app works offline and everything you put into it is private and kept on your device. Plus, there’s no money hungry subscription service. From what I’ve seen on reddit, the developer is working hard on improving the app and I’m looking forward to seeing what’s implemented in the coming updates. So far it’s been great for every day questions and information.

I'm astounded that I can run such capable models on my laptop. The developer is very active and constantly improving the app. It makes running local private models extremely easy and as of the latest update has a great range of different models to chose from. I have replaced a decent portion of my Open AI usage with this App

asked it english word meanings and it failed

Here was my prompt: Request a LinkedIn recommendation from a friend who offered to do one for you. No more than 4 sentences. Tone should be very friendly and familiar and ask them to focus on these 2 things (make them placeholders), if possible. It wrote a recommendation that I would be giving for a friend, basically is not able to deduce or understand the meaning of the prompt. This happened repeatedly with prompts input. Put same prompt into ChatGPT - near perfect response. Tried a few different models including dolphin, LLama - same issues. Not sure if it’s in how to app approaches the LLM or some other factor, but it’s not as good as using ChatGPT or even Ollama on Mac.

My phone isn’t outdated as developer responded…. Default LLM is the only one that half way works but it isn’t worth it. It always states this at the end of everything. “The second paragraph is in English. 번역결과 넷, 제가 언어를 구한 2009년 3월 16일 LLM-데이터 수량을 5밀라 7밀라 8밀라 9밀라 10밀라의 요청은 "앞으로 어떻게 원하는 대지? 4단위 4단위 6단위 8단위 9단위 10 단위 11단위 12단위 13단위 14단위 15단위 17단위 18단위 19단위 20단위 22단위 24단위 27단위 28단위 29단위 30단위 등 참고해 주세요."