Print

Please login or register.
1 Hour 1 Day 1 Week 1 Month Forever
Login with username, password and session length

[sambojin]

Again, not a game, but....
In my continuing quest to actually use the processing power of my crappy little phone, I've chucked a low-key local implementation of Stable Diffusion on it. SDAI to be precise.

Does it work well? No, and it only does 512x512 (otherwise it would take forever). Does it work fast? No (takes about 15-20mins for one image). But does it work? Yes! And you can run it locally, so it's free, as well as the more usable "just use someone else's processing power for this stuff", as is the fashion at this time. So it does both, but I like a non-internet'y thing, without messing around with Termux or Winlator.

And you can download a fair few different models/ LORAs to try out, depending on your artistic needs. Landscapes? Yep. Photorealistic? Umm, kinda (we're talking like dalle2'ish levels of stuff). Anime? Yeah, sure, I guess.

Grab it here:
https://play.google.com/store/apps/details?id=com.shifthackz.aisdv1.app

It's not "fully featured", but you've got a positive prompt (space out terms with commas), a negative prompt, a size thingy, amount of steps for generation quality, and other stuff that I don't know what it does. And a fair few free generative models to download, assuming you've got the space on your phone. 1-1.3gigs each. And considering it works fine as a pointy-clicky-does-a-thing solution, without any other knowledge of virtually anything else necessary, it's good.

Can it run in the background, while you're posting on forums, or watching YouTube videos? Kinda. It runs really slowly, but it's running now, as I post this. It's one of those RAM limitation things. Make a small image, and it'll probably chug along fine. A 512x512 one? Yeah, that'll crash 8gigs of RAM on Android, mostly. Still, it can do it. My next phone will have 12-16gigs of RAM, so I'll see what task swapping can do. But crank your YouTube down to 360p, and your image generation size down, and it'll slowly do it. Just check on it every minute or two, for processor priority purposes.

(Just remember to put "nsfw" on all your prompts, because it makes it work better)

[sambojin]

Another local LLM front end for Android, Layla is way more fully featured than MLC chat. The paid version gives heaps of utilities and character creation options, but the free version works fine

Comes with a range of downloadable models, and I get pretty acceptable speed out of the small and medium models on my Oppo A96 (old octacore/ 8gb ram). There's even a tiny model you can try if your phone is a complete potato, or is lacking in RAM (though the responses will be a bit dumber). I'm getting about 3 tokens/second out of the small model, with better responses due to character creation (want code? create a computer programmer character. Want short stories written? Create a writer, though it already comes with one), whereas I was only getting ~0.7-1token/s in MLC for a similar 3B model. The difference in speed is amazing. My phone isn't that much better than a potato, but 3tokens/second makes it pretty bareable.

The tiny model seems to use about 1.8gig of ram while running (so will probably work on damn near anything) and generates about 4.4tokens a second on my phone. Probably faster without analytics on, and I'm pretty sure my phone has heaps of random background apps/tasks running. Quality appears to be about the level of RedPyjama3B (MLC Chat's small model), but may be worse. At least it's quick. Oh, and the Writer character that comes with Layla will write nearly anything, whereas some characters won't.

So, it's just better in general than MLC, with way more model tweaking options, changeable context size, prettier interface, seemingly far better speed, and all kinds of other stuff. Apparently also gets frequent updates, that don't break everything (MLC broke for me after an update). Initial load times are a bit slow, but after that they're good.

I'll probably buy the paid version, just because I like supporting this sort of project. But as mentioned, free works fine.

Anyway, give it a go. You can get it here (an actual Play store app, no side loading required):
https://play.google.com/store/apps/details?id=com.laylalite


It also allows loading of custom GGUF files for different language models, so I'll have a go at getting phi-3 going on it (a new model from Microsoft that is apparently fairly performant).
Yeah, phi-3 works fine. It's censored, and is slower than phi-2 (which is what Layla uses for its small model, uncensored) at about 2-3tokens/sec, but apparently it's a lot smarter. Was only using about 3.4gigs of ram, so should squeeze into plenty of phone's hardware specs. I'll probably stick to phi-2/ small for most stuff, because the extra speed is awesome, and I'm often not requiring genius level understanding. I'd probably just move up to 7B parameter models if I really needed context or a lack of hallucinations. You can grab it here if you want to try it out:
https://huggingface.co/microsoft/Phi-3-mini-4k-instruct-gguf/blob/main/Phi-3-mini-4k-instruct-q4.gguf