Does anyone run their own AI service?

If so,

Which one?

What hardware?

Do you have a way to access it from outside or from a phone?

And, if you’re open to sharing, what do you do with it?

I know AI-enabling hardware is insanely expensive right now, but some of the smaller (<32GB) more efficient models (think deepseek- or qwen-based) don’t absolutely need specialized hardware to run and perform well enough for many purposes.

I am so aware of the absolutely abysmal privacy practices of basically all AI companies that I’m not comfortable using them to help me with web searches and the such, because there’s no way that all this information won’t be going into a profile. This makes me want to have my own “helper” service.

  • FrankLaskey@lemmy.mlEnglish
    41·
    9 hours ago

    Would be interested to know your experience with fcli if you’d share more. Just hearing about it.

    • kensand@sopuli.xyzEnglish
      31·
      8 hours ago

      I mean, I haven’t really started discussing it outside IRL friends, so not surprised you haven’t heard of it 😅

      In essence, the problem is twofold:

      1. LLMs tend to use CLIs more effectively than MCP, at least in the domains I use. However, installing all the CLI tools and configuring them can be cumbersome and not easily repeated. CLIs also cannot be modified, and are often quite verbose.
      2. LLMs will repeat very similar commands over and over with just a few tweaks. This often happens when there is a gap in a tool or poor documentation.

      So fcli is intended to be an easily extended, configured, and authenticated CLI. The “one CLI” to do everything your LLM needs. Ideally, you set up a personal repo to store ‘ucks’ (f packages) in, and you set up credentials for that repo so your LLM can push any new ucks to it. You can then share that repo with other users if they could also benefit from any of that functionality.

      As you go along in a session, when you notice the LLM repeatedly running similar verbose commands, you can have it create an uck for it instead and save those tokens.

      Likewise, if, say, an MCP server doesn’t have full functionality, you can have your LLM import the SDK from npm into an uck and write a basic fcli wrapper around it to start using.

      The other nice thing is that it makes your tool configuration repeatable between LLMs and harnesses. You don’t need to set up your MCP servers in both Claude and Codex and Pi, just in f and tell them to use f.

      There’s also a creds uck to make your secrets more portable and (a bit) safer.

      In general, I’m trying to model it a bit like pi.dev, but as part of the tooling instead of the harness itself. I think they work quite well together :)

      Interesting paper comparing MCP and CLI