I saw Generative AI for Beginners from Microsoft on GitHub. I’ve looked at https://fmhy.pages.dev/ai but I’m not sure what I’m really looking for.

I write fiction, and I want a chatbot that will function like chat gpt3.5, but not shut down if things get bloody or sexy, as they so often do.

You know ready, aim, fire? I’m in the AIM stage.

  • CetaceanNeeded@lemmy.world
    31·
    10 months ago

    I’ve been using GPT4All on my laptop and using mostly 7B models due to my RAM limitations and I am amazed how good some of them are.

    It’s been really easy to use. There are models you can download from within the UI or you can get adventurous and download them from elsewhere, they just need to be in the .gguf format. I get most from TheBloke on hugging face.

    So far my favourite has been solar-10.7B-instruct-v1.0-uncensored, it has been astonishingly good.

    • neidu2@feddit.nl
      5·
      10 months ago

      Oooh, do tell me more, please. I’ve been toying with the idea of setting up gpt4all myself, but I haven’t really had the time to look into it very much yet. I have a couple of questions, though:

      • I guess it’s safe to assume that it runs on linux?
      • Is it possible, with some scripting, to provide additional training data, such as connecting it with a wikipedia crawler?
      • By combining it with some script-foo, can I have it also look up stuff for me on the fly, for example "extract THIS kind of information from THAT site ?
      • CetaceanNeeded@lemmy.world
        3·
        10 months ago

        Yes it runs on Linux, my laptop is running Manjaro and I installed it from the AUR. I’m not sure if the scripting is possible, there is an openAI compliant web API you can turn on so maybe possible through that, you would probably have to feed in the content of the site with the prompt though, I’m not sure there is a better way but I guess that sort of behaviour is a bit out of scope for GPT4All.

        There is a local documents feature that allows it to access text files on your machine that you give it specific access to but I think it’s fairly limited in its ability.

      • zeluko@kbin.social
        2·
        10 months ago

        The GPT services out there use something called ‘tools’.
        They get presented to the model and the model can ‘call’ a tool with arguments, which can then extract some data and input it into the context for the model to continue.

        I found out, the models which can run on a normal PC (or even a Laptop) are okay, but not super great. (around or a bit worse than ChatGpt3)
        The good stuff (e.g. Nous-Capybara 31B or the Mistral/Mixtral ones) needs some more memory and compute.

  • blargbluuk@sh.itjust.works
    10·
    10 months ago

    LM Studio is one of the most user friendly ways to play around with LLMs imo. You can run some of the smaller models without too much memory (it will be slow without a decent GPU though).

  • Wyre@lemmy.world
    6·
    10 months ago

    I’ve been playing a bit with llama2 in Ollama it does not have any restrictions perhaps using Ollama to run models locally is something that would solve some problems for you?

  • Gamma@beehaw.orgEnglish
    4·
    10 months ago

    From what I’ve heard Mistal is what you’d want to generate explicit content. Not sure what you’d want to run locally, but be warned that is slooooww unless you’ve got a beefy laptop

  • wathek@discuss.online
    41·
    10 months ago

    I would look into NovelAI for writing, it’s quite specifically for that. It’s a paid servicd similar to chatgpt, but it’s uncensored and private.

    You can run your own lightweight LLM on a laptop but the output will be useless. Good output requires big boy compute.

    If you do want to run it on your own hardware, look into Ollama. There’s also options to run your own LLM in the cloud with a not too difficult process for non-techies.

    Frankly, id find the right LLM for your needs and just pay for it per month, maybe novelai, maybe something else, but chatgpt is not great for creative fiction.

    • Melatonin@lemmy.dbzer0.comOP
      3·
      10 months ago

      I got a little TOO MUCH involvement from NovelAI. I guess I want suggestion help, idea spitball help, but it’s specialized what I’m looking for.

      I want my ai to stay on the shelf with my thesaurus until I’m ready to use it.

      • lazylion_ca@lemmy.ca
        3·
        10 months ago

        What does too much involvement mean? As someone looking to get into this myself I’m curious about your experience.

        • Melatonin@lemmy.dbzer0.comOP
          3·
          10 months ago

          The two things that stand out to me are

          1. Creating backstories, characterizations, etc etc etc. All that stuff they tell you to do in novel writing class that sometimes you might do, or if you’re me you don’t.

          2. The AI likes to just start writing narratively, taking on the job of crafting words. That’s not what I’M looking for. I want a sounding board to bounce ideas off, suggest alternative directions, possible motives, help me brainstorm.

          Point two has always been a problem for me, because they tell me I’m autistic. All I know is I have some coping mechanisms I’ve had to use over the years. It wasn’t really a thing when I was a kid. But I don’t see alternatives very well. So having an outside entity suggest things that hadn’t occurred to me gives me more choice in plot lines and decision points. I write ok alone. But I’m excited to see what ai can help me do.

      • wathek@discuss.online
        2·
        10 months ago

        Interesting, im vaguely interested in this too. i have half of a world written that i want to turn into a game maybe (probably not but, amhaving fun) I have the hardware to turn what i have into an embedding for an open model, and the hardware to run it. So that’s the way i would go about it, though i can’t advocate for how helpful it would be (yet)

    • Melatonin@lemmy.dbzer0.comOP
      5·
      10 months ago

      I’ve been doing that for 40+ years. I just want to try using AI as a partner to bounce ideas back and forth. As I explained in another post, I have trouble generating alternatives once I’ve created an Idea. It’s nice to have a means to get outside myself.

      I intend to do the writing, it’s the planning and plotting I’m using AI for.

  • Mahlzeit@feddit.de
    2·
    10 months ago

    You should probably hook up with the SillyTavern crowd. It’s a frontend to chat with LLMs that will do what you want. Its main purpose is chat role-play. You can assign a persona to the LLM and ST will handle the prompt to make it work. It also handles jailbreaks if you want to use one of the big ones (no idea if it works well). You can also connect to other services that run open models, including aihorde.

    https://github.com/SillyTavern/SillyTavern

    https://www.reddit.com/r/SillyTavernAI/

    If you want to host your own model you can find more help here:

    https://www.reddit.com/r/LocalLLaMA/

    !localllama@sh.itjust.works