How I Am Using the Muse AI Assistant
Edward J. SchwartzComputer Security Researcher4 min. read

Recently I have been enjoying using Muse, Meta's new personal AI agent, which runs on its own dedicated virtual machine. I have found a number of ways it has been helpful to me, both as a researcher and in my personal life. Here are some of those stories.

The useful stuff, it turns out, is mostly things that repeat. A weekly job takes some setup, but once it is set up, it mostly takes care of itself. Much of this post was drafted while driving, as one story below shows.

Research work

Paper triage

Since the advent of AI, I receive more notifications about new papers "I might be interested in" than ever! When new papers arrive, Muse reads the abstracts first, so that it can summarize each paper and recommend what to do with it. Then we go through them together, and I decide what action to take on each one, if any: file it in Mendeley, add a reading todo, or just ignore it.

Library organization

My paper library lives in Mendeley, and is split between personal libraries and libraries shared with research colleagues. It's easy for things to get misfiled and duplicated. Muse helps keep my library organized. It suggests locations for new papers, and a periodic audit proposes ways to restructure the library, which I rarely have time to think about myself.

I have also been thinking about switching from Mendeley to Zotero, and one of the things stopping me has been the difficulty of transferring my data. Muse mirrors my changes across both, which solves that problem completely.

Email triage

I receive a lot of messages that need some kind of decision or action from me. When I'm busy, these pile up in my inbox. I have a weekly job that looks at emails older than a few days and attempts to triage them:

  • Many emails are de facto todo items that I keep until the task is done. For these, it suggests archiving the email and adding a todo item.
  • Many are notifications about new papers and the like. For these, it suggests filing them, or disregarding them if they don't seem relevant to me.

Because it knows what is going on in my life, it often makes reasonable suggestions. It can also write draft responses, unsubscribe from spammy mailing lists, and more.

Waiting and monitoring

Muse is really good at waiting and monitoring, which is especially useful when delegating. When I hand something to another person, Muse waits for evidence that the task was completed. For example, I recently asked someone at work to schedule a trip for me. Sometimes balls get dropped, so I asked Muse to keep track of it. It checks for emails about the trip, and if none arrive in a couple of days, it lets me know. If one does arrive, I don't need to hear or think about it, which is the whole point.

Another example: I recently received a notification about a paper, VMSpec. I was very interested in the implementation, which the paper said would be released. So I asked Muse to watch for the release and notify me when it comes out. (On the personal side, it can also watch for good prices on items I want to buy.)

Driving

Much of this post was drafted while I was driving to Pittsburgh. I talked, Muse wrote, and the post you are reading is the result. Driving time is normally dead time, and voice turns it into productive time.

I had positive experiences talking to an LLM while driving even before Muse. One example is reformulating OOAnalyzer as a search/optimization problem. I spent most of a four-hour drive to Pittsburgh conversing with GPT about various frameworks for modeling problems. We eventually arrived at Answer Set Programming, which is conceptually the perfect fit. A long drive can be a good place to think and brainstorm, because there aren't many interruptions. (Caveat: I am mostly driving on the very straight PA Turnpike, which admittedly does not require my full attention.)

In addition to drafting this blog post, I also had Muse create podcasts about several papers that seemed interesting, and listened to them during the drive. A podcast is not a replacement for reading the paper, but it's good enough to get an understanding of the work, and to decide whether it's worth reading in full.

Personal routines

The same repeated-work pattern shows up outside of research, too.

Back pain logging

I have chronic back pain that seems to crop up without rhyme or reason. I've been collecting data on possible factors so that I can do some statistical modeling. Twice a day, Muse prompts me to rate my back pain on a scale of 0 to 4. No big deal so far. But it also fills in information about possible causes: weather, plus steps, sleep, and exercise data from Garmin and Health Connect. All of the data is logged into a Google Sheet.

We then worked together to develop a Bayesian statistical model to fit the data with an interative demo page.

Weekends

On Saturdays and Sundays, Muse checks my calendar to see whether my day is light, and if it is, it suggests two or three undated tasks from my backlog. It also nags me to do one small thing each weekend to clean up my office. These little nudges can add up to a lot over the long term.

Connectors

A frequent and frustrating problem is trying to use AI for a task, only to find that it can't integrate with the service it needs. Muse has two features for dealing with this. First, it can run a real browser, which lets it interact with most web-based services. Second, and even cooler, Muse can build its own connectors. For example, Muse did not have connectors for Mendeley or Zotero, so it built its own using those services' developer APIs. This is a godsend, and in hindsight an obvious killer feature for agents that can write code.

Of course, handing over your credentials to an AI agent is a concerning proposition. We all remember the horror stories from the early days of OpenClaw, such as the agent that deleted hundreds of a user's emails despite being told to ask first. Muse has an elegant design for avoiding this problem. Credentials and tokens are stored separately and are never provided to the LLM itself. Instead, the LLM is given temporary opaque tokens, which are swapped for the real ones as requests leave Muse's VM. That swap point is also where Muse enforces policy-based controls. Since the LLM never sees your real credentials, it can't bypass the gate. I like this design a lot.

Conclusion

I've found a lot of value in Muse, even though the model itself is not quite as powerful as frontier LLMs. Hopefully this post gives you some ideas for how Muse or another assistant could help in your life too!

Powered with by Gatsby 5.0