Private LLM - Local AI Chat

This is an amazing app. Love it, I would really love a way to upload documents, offline so I can ask questions when I’m out and about. I would love to chart out my record collection to remember what I have and to cross reference serial numbers. I would like something where I can upload health documents and not worried about being profiled on google. Having a chattable document or database or however its handle is %100 what users need (imo) and just not being given by any of the 100+ chat apps. They all talk, they all generate pictures (or aspire too) no backdoor I’ll please. Just something easy.

See title

I really hope that this app can support the network search function, so that I can have a real intelligent assistant.

Have had the Private LLM app a few weeks and have enjoyed putting my new Mac mini M4 Pro to the test with a lot of the LLMs. So far so good. Everything is on local ssd but may try and experiment with it on my TB 4 DAS or even my older NAS. Will try and keep Private LLM itself on my local and move just the data to my NAS or DAS. Mac OS infrastructure keeps what I need in a well known path and folder so I’m going to experiment. Just need to figure out where to change to new Data/Folder path. Dev(s) Thank you for such a great app.

Fantastic app, I like the selection of models and they run great on the M4 chip of the iPhone 16 Max and iPad Pro. System prompt is also great for some tweaks to responses. The most simple and important feature (aside from model memory) that Private LLM needs is to be able to edit the responses of the AI. I enjoy the chatbot's responses but hate that it'll pull context from previous responses and loop entire phrases regardless if I regenerate its response or edit mine. Sometimes I don't like how the model formatted its response and want to change it without clearing the convo and retrying multiple times. The second feature that would be a quality of life is to be able to save conversations within the app. At the very least, export them as a whole without needing to screenshot, grab text, paste, correct, and format it elsewhere. I learned early on that copying text blocks at a time can cause the app to reload its model and even clear the conversation. A bot refresh would also be nice to clear older message context during a longer conversation to keep the model fast and responsive, but that can coincide with model memory to more easily pull specific facts you want it to remember in the first place without having to create a queue of explanations to pick up where you left off at. Useful if you switch models for different purposes like organizing, summarizing, q&a vs. generation, editing, brainstorming, vs. chatting.

Private LLM delivers exactly what it promises, and for $10, it’s a great deal. The app works seamlessly, and it’s worth every penny if you’re looking for a reliable, private AI assistant on your mobile device. However, as a writer, I’ve found its capabilities somewhat limited. While it excels in many areas, the nuances required to truly make language pop—metaphors, tone shifts, and narrative flair—are a challenge for any mobile AI app. The constraints of running on-device mean it’s capped at 8 billion parameters, which, while impressive for mobile, can’t compete with the richer, more nuanced capabilities of models like ChatGPT with its trillion parameters. That said, if you’re not relying on it for deep creative work, this app is a standout performer. For privacy, portability, and general utility, it’s one of the best out there. Just keep in mind its inherent limitations for high-level writing tasks. This version keeps your key points but polishes the tone and structure for better flow and impact.

I was asking the bots questions about the local processing of my conversations and it said that it actually sent it all to servers. I downloaded the app because it said it doesn’t leave the phone but then it tells me that it does… not very cool

I think there should be a history for previous chats. Maybe auto delete at one week.

For an app with such a high price point, I expected it to have a great user experience on the iPhone. Unfortunately these are the annoyances I ran into: * There is only one conversation slot available (I would have liked a history of conversations, even if they need to be read-only for technical reasons) * It takes 4 taps to start typing into a new conversation, since you have to clear it first. It should take 1 or 2 taps. It really slows down the experience when you have to tap so many times whenever you open the app before you start typing. * The scrolling is really janky, so you cannot easily read the response while it is being generated. This is a big problem with 7B+ models, as they are understandably slow, so you want to be reading it as it generates to determine whether to stop generation early or let it rip. Other AI apps anchor your scrolling to the top so that generation doesn’t cause text to shift constantly. * The model settings aren’t all optimized, and there are no granular settings to fix them yourself. For example Llama 3.1 8b instruct generates word salad sometimes, which I know is not a problem with the model itself. * The tap zone on the text input box is too small, so you’ll find yourself poking your screen more than once to try to type to the LLM. * When I tried creating a Shortcut, it gave me a strange error code even though it appeared that I was using the action in the exact same way as in the example shortcuts. There’s no tutorials or documentation for how to make a Shortcut with this app, and I didn’t feel like emailing the developer to ask about the error. They should add KB pages. * The progress bar for downloading models appears to get stuck sometimes in an indeterminate state where you’re not sure whether it’s successfully downloaded. Just overall lacking some polish. If this app were cheap or free, this would all be forgivable. But for a premium price, I expected it to be a premium app experience with careful user design to make it a joy to use. It just needs a lot more polish before it’s ready for this price point, so I returned it. (If it were fully polished, an even higher price point could be justified.) What I will say in this app’s favor is that the model selection is great, and the performance is great too. They must have some special sauce that keeps my phone from thermal throttling on large models.

I get that they are trying to be inclusive for older models and having LLMs that don’t require more than 6gb of ram. But I was really hoping to take advantage of my iPhone 16 pro’s 8gb. It could easily handle llama 3.2 11b parameter model. Addd it addd it.