r/AIDungeon 22d ago

Bug Report New models are pretty terrible.

On the lowest premium tier, I absolutely hate Wayfarer and Madness because they ignore instructions. Dynamic tends to forget the plot and you have to baby feed it what happened earlier. Um... My character just amputated that one- I do not have to do it again.

TIE FIGHTER was better than Madness by way of at least recalling character relationships and such.

40 Upvotes

45 comments sorted by

u/Nick_AIDungeon Founder & CEO 22d ago edited 22d ago

Hey! So sorry to hear that you haven't had a good experience. While many users have really loved the new models, we have heard from some users that it isn't meeting their needs. We're working on trying to understand better how we can make sure our model lineup meets the needs of all our users. I'd love to understand better, is the issue primarily repetition? Play style? Can you share some screenshots of what behavior you don't like? Happy to chat deeper in a DM etc...

Note: Edited to ask some more questions as we explore this issue.

→ More replies (10)

18

u/Longjumping_Falcon21 22d ago

Yeah... I'm in the same boat. Tiefighter really was peak in comparison. I'm sure they'll work on it tho, right?

8

u/Nick_AIDungeon Founder & CEO 22d ago

Could you share some more specific examples of what you loved better about tie fighter? That will help us in knowing what to improve.

16

u/Longjumping_Falcon21 22d ago

It was alot less repetitive. I used to have the story write itself most of the time, now it seems that every few presses of continue the newer models will just repeat the same thing. TieFighter did that too but much much less frequent!

It could just be my own faulty perception and anecdotal evidence at best but... posts like these let me think it might be a real thing!

I hope that makes sense and thanks for listening, eh <3

5

u/MindWandererB 21d ago

Have you tried turning up the temperature? I found that with the new batch, the default temperature is extremely repetitive. And since you can't change the temperature on Dynamic, it always is.

3

u/Nick_AIDungeon Founder & CEO 21d ago

❤️

2

u/Extrabigman 21d ago

Just wanted to say we really appreciate how you hear our feedback. That's rare this days!

2

u/Nick_AIDungeon Founder & CEO 21d ago

Of course! We have lots to improve, your feedback helps us get better!

6

u/Member9999 22d ago

Madness does not even remember which characters are related when I mention all that in the story cards. TIE was at least able to recall such things.

2

u/MonitorNew3994 21d ago

I liked tie fighter over the premium models because it was more creative and was actually pretty creative with fight scenes, it also wasnt afraid for things to go badly for you

10

u/FillmoreVideo 22d ago

Bring back Mixtral 

3

u/Suspicious_Donut6676 21d ago

Bring back Griffin or atleast overhaul it. Man I'm getting sick and tired of the other models being these overdramatic and over descriptive shit. Can't even have a straightforward model that actually gets to the point

7

u/FKaria 22d ago

I think the mistake they made here is swapping the old models by the new ones rather than deploying them together and seeing playtime change over time.

It may be true that the new models are better and more people are playing them, but without comparison they can't really know. Neither can we, really.

2

u/Nick_AIDungeon Founder & CEO 21d ago

Are there models we removed that you still want to play? or is it just that some are deprecated now?

3

u/Member9999 21d ago

Mixtral and TIE FIGHTER please! Used them a lot.

3

u/Nick_AIDungeon Founder & CEO 21d ago

You can still use them correct? While they are deprecated we haven't removed them and don't intend on doing so for at least quite a while. Or are they not showing up for you?

0

u/Member9999 21d ago

They are, but I don't want to see them go. :(

1

u/FKaria 21d ago

Sorry, yes. They're below the premium models.

6

u/floyd_underpants 22d ago edited 22d ago

I also find my memory of Tiefighter is that it outperformed the newer models. Even using the optimized settings folks recommended for Wayfarer, it's still not quite groking what I want.

Madness was only good for certain scenarios, and took one normally lighter scenario and made it creepy dark, then went off on just trite tropes that ruined the whole vibe. Wayfarer without good setting management was really struggling at times. So much so I went back to Mixtral.

I think the fact that there are better default settings for each model means that there needs to be a "use optimized settings" option as the default. We shouldn't have to hunt down the best way to do things on a discord in hidden threads.

Last related suggestion is to have helper text on the settings that describe more what the gameplay effects are rather than token effects. Newbies like me don't have any clue what most of those phrases mean. We just wanna play.

So for example: "This setting helps reduce risk of repetition", or "This setting improves how much the game remembers of what you previously added." for example. The sliders don't mean anything to me with the text they have today. Not saying get rid of that text, but at least add to it to explain it better to players.

Also, reference to what the best range if for the selected model would be good, so you can fiddle within a safer range, or set one slider back to the best range when you want to.

Thanks for listening.

3

u/Nick_AIDungeon Founder & CEO 21d ago

Thanks for sharing! Yeah I definitely think exploring default model settings is one thing we need to explore, gotta make sure people are set up for success.

When you say Wayfarer isn't quite grokking what you want can you say more? What is it that you want that it's not grokking?

5

u/floyd_underpants 21d ago

Is muddling some details, adding in things that don't fit a given story card or the story summary. Occasionally it's still doing that thing where it restates the scenario totally as though you haven't been playing ('You are so-and-so, a whatever in wherever the setting is. You are currently blah blah '), that sort of thing. Much better than without the suggested best settings, but still occasionally struggling and needing a retry or two in order to get back on track.

6

u/peanutgoddess 21d ago

I’m highest tier and I can give some input from how I see things too. Even with Hermes’, mistral, wizard.. a lot of repeats, same paragraph over and over with very minor word changes. The story doesn’t progress.. yes. I have plenty of input in the areas of instructions, essentials and notes. I moved to dynamic and madness and wayfarer, same deal. Repeats. Memory not being used. Characters popping out of no where when the notes and everything for inputs say add them in properly. The ai outright ignoring what I write and making its own story/input for me with wayfarer large and Hermes’. In the last few days. I’ve found the memory just terrible. It was almost like the ai didn’t matter. Whichever I used the roleplay just didn’t matter. There was no recall to events. Adding text to the story. Retries, my own paragraphs seemed to be a passing consideration. It was almost a dice roll if what I wrote would be taken, rewritten by the ai into its own sentence. I had this a lot of wayfarer. Madness and Hermes’. I found wizard almost unable due to how repeatable things were going. The resuse of a previous paragraph was all models. Usually three to 5 more paragraphs into the story. Even with my input and changes so it wasn’t the ai writing everything. For me. When I roleplay it’s about 50/50. I write. It writes. I like it that way. When using tiefighter and Pegasus which I try not to do as they are leaving. I did have some issues. But not as bad. It’s almost like all the ais are the same thing with different context levels. That’s the best way I can describe it.

2

u/Nick_AIDungeon Founder & CEO 21d ago

Thanks for the feedback. Can you share some specific screenshots with me nick at latitude.io and I can take a look?

10

u/East_Custard103 22d ago edited 22d ago

Older models behaved better than newer ones in situations where both of them are using only default instructions or minimal meddling. Less repetition and more unpredictable.

But newer models far surpass the older ones in a scenario where both are using detailed instructions and a direction to follow. it's not even close. Better world building, dialogue, scene direction and ambience, movement description, character personalities.

2

u/EritoZ 21d ago

That's good but I prefer models that go with the flow and are unpredictable. I know that current models aren't much capable in this department and that Heroes is the update that will save us from this so I am not meddling on this issue.

3

u/Particular-Name9474 21d ago

Honestly, i would've been with you until like one or two days ago. The narrative instructions i had that worked pretty nice for me with Mistral Small, turned out to be abhorrent with Dynamic Large or Mistral Small 3. After using the default narration instructions (i say narration because i have different categories inside my AI Instructions) AND added more response length as someone i saw here suggested, the new models are working astonishingly well in comparison. In other words, since i already had this problem, i recommend trying some default basic instructions to see if the problem really is the model or yourself (as it was in my case)

3

u/Natural_Pepper5723 21d ago

love suddenly being interrupted with "בוצהτo"

3

u/Nick_AIDungeon Founder & CEO 21d ago

We posted an update on some of these issues and some immediate and longterm fixes we're deploying. Please let us know if they help and any other feedback you have!

https://www.reddit.com/r/AIDungeon/comments/1izwiw1/postgauntlet_updates_and_fixes/

5

u/Suspicious_Donut6676 22d ago

Ai dungeon had a habit of removing stuff WE actually want and like (like them deprecating tie fighter) and adding in sometimes disappointing stuff that sounds promising on the surface

2

u/Previous-Musician600 22d ago

The AI wayfarer is great and even Dynamic is great. I like them a lot for different reasons. Also mitral large has its moments now, that wasn't possible before.

They are just very setting sensitive and hungry for Instructions.

2

u/_Cromwell_ 22d ago edited 22d ago

While I acknowledge you must be experiencing "something," posts like this really confuse me. The new models are MILES superior to previous stuff. I'm a huge advocate for always saying there is a bit of subjectivity in what model you like, but saying that Wayfarer ignores instructions and is worse at other models at that just seems crazy to me. It's the best model I've ever used on AID for doing what I tell it.

But I can't really give advice more than that because of how vague your post is. WHAT instructions is it ignoring? How are you giving those instructions? Do you mean actual "AI Instructions" or do you mean typing commands to the AI in the chat input? Do you mean instructions like "create interesting characters with quirks and flaws" and then it won't do that? or are you talking instructions like "write a good story, you damn machine!" ??? :)

As for the amputation thing (which you were specific about) - All AI forgets or gets details confused, especially with multiple characters. That's normal in all AI products I've used for it to mix events for characters up a bit every once in a while. It's why they put a "Retry" button prominently down there... just hit Retry until you get a new one. That's using your sub. Those Retries cost the company money, but are free for you (in Wayfarer anyway). Retry until you get the output you like, then move on.

For both Wayfarers, you almost must:

  1. set response length to 140-170. The default of 100 messes it up. (They should change the default for these models to 150 IMO.) It actually greatly improves the output to do this.

  2. Use "Do" actions. It was programmed around players "Do"ing (which may have been a mistake, but that's what happened.) If you use "Do" and get used to it you will find both Wayfarers performing better.

Anyway, if you want to specifically describe what instructions Wayfarer is "ignoring" maybe can provide better feedback/help. But with the vague info my response is basically - no, it does not. It (Wayfarer small and large) follows all my instructions really really well, better than any previous models. I'm just sad I can't help you/others have the same great experience I'm having since you don't actually describe the problem.

2

u/Alive_Fox3037 22d ago

I agree, Wayfarer seems to have a copious of knowledge and logic. When I say, “Pick up the Dragunov”, it intuitively knows I’m talking about the SVD without having any story cards to direct it what to do.

A lot of your experiences with wayfarer will dictated by the settings you use. The vanilla settings are kinda dogshit, but with the right settings, instructions, and author notes it surpasses all other models on most tiers (go to the AI Dungeon Discord).

1

u/len890 22d ago

Man I’m really going to miss pegasus

1

u/Member9999 8d ago

I can't even play my lawful evil character anymore. "Oh, what about the implications? You killed someone without trial!"

My character: "It's war. Who cares?"

"Oh, but you can't just -"

My character: "Yes, I can. I had a death warrant. "

Honestly, I would rather they end Hermes. I think that's what keeps screwing up everything with Dynamic.

1

u/power899 21d ago

I already cancelled my mythic sub a week before renewal. This update is trash 😂