This update continues on the path outlined in the last dev diary: Dream - https://www.layla-network.ai/post/dream-layla-s-dev-diary-march-17th-2024
In particular there are two features we wish to highlight:
Task Manager
This new app gives you more visibility on what Long-term memory is doing behind the scenes. With this app, you are able to see what tasks are queued up by the LTM ingestion process, what status they are in, and which ones have finished.
Additionally, the ingestion process has been re-written to take advantage of long-running background workers in your phone OS, resulting in more stable ingestion process during the night.
The ingestion framework has also been abstracted to support more tasks coming in the next updates related to PDF processing and Dream (previously Proactive Chat).
MLC
MLC (https://github.com/mlc-ai/mlc-llm) is an experimental framework for running LLMs on your phone. It utilises your phones GPU better to provide better performance on the latest flagship phones.
However, as observed by our wonderful alpha testers, while the response speed has increased significantly, it is still very unstable and will freeze up your phone often. This app is put in the "experimental" section in Layla for all to try.
Changelog:
New features:
Task Manager app - manage your long term memory ingestion tasks
MLC Inference - highly experimental app that replaces Layla's inference engine with MLC
Improvements:
improved ingestion time for long-term memory
long-term memory ingestion will now run continuously during the night when your phone is charging
improved model download to show better progress so it doesn't look like it's hanging
adjusted small grid view to not crop the images too much
added filter by tags in character selection screen
added setting to configure "enter key sends message instead of newline"
added ability to configure "n batch" in advanced settings
Bug fixes:
fixed bug where the download screen would appear briefly before entering the main screen
fixed bug where character selection header does not return to normal when exiting from the create/edit character screen
fixed bug where roleplay templates cannot be edited
fixed bug where some text were getting cutoff on the right side of apps
fixed bug where proactive chat notifications were not opening the chat screen properly
fixed bug where MobileVLM crashes with non-jpg or non-png files
fixed bug where Wallpaper app was conflicting with Character Selection Layout settings
fixed bug where "n threads" in advanced setting was not being applied correctly
fixed bug where context length setting was limited for custom models
Comments