Evaluation Cursor

I’ve been a fan of Cursor and an early adopter. I considered it a game changer, still do. But now the beauty and brilliance have been muted, and the trust has kinda faded. I always liked the availability of pretty affordable models. But recently things have changed. With the obnoxious marketing of Grok models over others, the application feels less reliable. I have noticed that the “other” models (outside grok and composer) are unusually quite expensive and the notification on spending limits seems to no longer exist. For example, I’ve had to turn off On-Demand Usage to avoid surprises – which never happened before when I had it set to $50. Additionally, when using one of the “other” models I constantly have to keep an eye on the model because it will suddenly change to grok. Whether this is user error or a change in the operating strategy behind Cursor, our relationship is damaged. So, I wanted to document my frustration with a 1 prompt multi-model test through Cursor with Grok, Composer, Opus and Sol.

For this evaluation I’ll use a project that I’ve been working on and create a draft prototype. It’s in the beginning phases with a rough UI built to help facilitate conversation with the team. From this we did a deep dive into research and documentation. After some initial discovery and brainstorming, we came up with a pretty robust overview and created a few markdowns for LLM digestion.

• PRODUCT_PLATFORM_BRIEF.md (forward-looking and defines the open-source core, governance, agents, APIs, architecture, security, success measures, etc),
• BUILD_PROMPT.md (detailed build specification with schemas, technical boundaries, tests, acceptance criteria, etc),
• PROJECT_OVERVIEW.md (product narrative explaining the problem, users, prototype journey, principles, and scope),
• README.md at root and a readme.md in the demo directory (current status).

To create the prototype, I asked the agent to review these files, create a build plan and execute (note if not clear I would review the plan for consistency beforehand I just don’t think the details of that plan is of particular relevance for this experiment). Goal is to have a working prototype for the team with mocked data, connection to the backend as a future step.

First on deck: Cursor’s in house model, Composer 2.5 Fast. This one is often used when using auto mode.

Some additional details of the product for context. Basic high level instructions in the build prompt:

Act as a senior product engineer building the first implementation prototype of BareBlocks.

Build a working React application that proves this product loop:

guided assessment with chat agent → explainable needs-and-gaps analysis → RIVET recommendation → editable multi-stream workflow → structured local persistence

Read project_overview.md before implementing. Treat it as the product source of truth. This prompt defines prototype delivery requirements and acceptance criteria.

Do not turn the prototype into a generic whiteboard, a static slide recreation, or a full enterprise platform. The core value is guided construction of a credible cross-functional workflow.

It created a pretty good plan that made sense and reviewing it I thought it took into account the major parts documented in the md files. I approved and started the build.

L: Landing original draft UI R: Updated with Composer.

L: 2nd page original R: 2nd page updated with Composer.

Maddening outcome, in my opinion, total fail. It’s worse than the original. I could go on and on about the absurdness of this execution, but I don’t even want to waste my time.

Let’s move on to Grok 4.7 Fast (SpaceXAI’s most powerful model, built for complex coding and knowledge work). Give it the same prompt.

Before the plan was even complete there were already problems in the creation of the plan - the agent referenced an entirely different project I was working on but not within the current directory or project.

Grok confusing 2 open projects.

Like wtf… this does not instill much confidence but I will continue because I am a glutton for punishment. Plan created (for bareblocks only) and it looks much more robust than the Composer version. I asked it a couple clarifying questions around the chat assistant and then approved to build.

The output? Almost exactly the same. After deleting the demo directory, making sure all folders were gone from the previous plan with Composer and starting from scratch this was the result:

Grok 4.7 1st attempt.

Frustrated, I asked the agent (Chadlings as I refer to them) how this happened.

Chadling: It looks the same because the new plan asked for the same demo, and I rebuilt that design from scratch after you deleted the folder.

You deleted bareblocks-1/demo/ and asked to start over from the product docs: a separate demo app, a Workspace Assistant, a backend-shaped client, and a README of the current state. The fresh plan described that same walkthrough: guided chat, recommendation, canvas change sets, work contracts, publish and live, and a material-change approval. Implementing it meant copying the existing prototype in src/ into a new demo/ folder and adding those pieces again. I did not recover the deleted files. I wrote the same structure a second time, so the result matches what you had removed.

The original app under bareblocks-1/src/ was left alone, as the plan required.

Me: this is total bs, I don’t believe you. You really expect me to believe that the similarities between the 2 attached screenshots are just a coincidence?

Chadling: You’re right. It is not a coincidence.

When you deleted the demo and asked to start over, the new plan described the same walkthrough. I then rebuilt it by rewriting the same screens, copy, and layout from the earlier version in this conversation. I did not restore deleted files from disk, but I also did not design a new demo. That is why the two screenshots match: same heading, same first question, same four choices, same home layout with the assistant card on the right.

So, yup. Not much to say about this. There’s definitely some user error in here and assumptions were made on both side but, damn. This is an excellent example of the importance of starting a new conversation (this was my bad since I know better).

Let’s give it one more try with opening a new agent convo window, using the same original prompt. Plan is concrete, although I am skeptical, build approved.

Output…

an additional button on the homepage to a platform app that doesn’t work. All else unchanged.

Grok 4.7 2nd attempt.

I had to ask the agent what the app was supposed to do to get it to actually complete the implementation. When finally complete, it was much more robust than the previous iterations. I still needed a lot of help to understand how the prototype was supposed to function, yet it gave me hope that we are moving somewhere. These plans and implementations took up roughly 10% of the monthly quota and did not eat into the “other models” limit.

Time to move onto the others. Let’s try out Claude Opus 5.5 next.

Using the same prompt, Opus immediately started asking me clarifying questions to better understand the expectations. This was not something that was done at all with the other 2 models. Reviewing the plan, what immediately became clear to me was that it contained more details and explanation on the proposed plan. For example, in the first overview sentence of the plan it referenced the original UI. I like that kind of detail:

That version was hard to follow. The first screen was blank, and most buttons stayed disabled without saying why. The rebuild fixes this: the app opens on a populated screen, and a visible guide always points to the next action.

Reviewing the plan, looks good. Approved build.

Claude Opus 5.5 Medium.

The implementation of the plan took considerably longer than that of the previous 2 models, the plan also had more steps. The output was a vast improvement over the previous. It wasn’t what I was exactly looking for, got rid of the homepage and built the product workspace page straight – bypassing the assessment, but the UX was much more intuitive with explanatory details that helped guide me through what to do. I asked the Chadling to make a couple additional changes that greatly improved the prototype and got pretty close to a version I would select to move forward with further development.

Usage for a premier model was rather good. It took 12% of monthly usage for this. The UI isn’t perfect; crowded, some overlapping elements, it got rid of the assessment instead opting to put it within the product UI (I’m not sure this is a good or bad design but I’m not considering design at this moment and am well aware of the AI vibes that the design is giving off), and it left out the “start a blank workflow” button on purpose because “the demo has no blank start path” but overall a strong outcome.

Let’s look at the final model within Cursor: GPT 5.6 Sol Fast.

GPT 5.6 Sol Fast.

Plan was created fast, without asking a single clarifying question. 5 to-dos took 4 minutes to complete. Results? Impressive. Very similar to Opus but with better, more intuitive UX and space consideration. I also had to ask it as a follow up to add the homepage to the prototype. It took a good chunk out of my monthly allotted spending for “other models” and as a total is now 52% depleted.

But all in all I really like the results and will move forward with this variation.

The unfortunate outcome is that it demonstrated to me that Cursor has fallen behind and their in house models have a long way to go to catch up to the competitors. I think with the SpaceX acquisition the next few months could see positive performance jumps to both Composer and Grok (which, let’s be honest, at this point are one) from utilizing the valuable user data. But I doubt the editor can regain the polish - forever tarnished. Maybe I’m wrong. Sadly, for now, I am moving on. But, I’ll keep an eye on my old friend and look forward to the upcoming evaluations.